From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794912626; b=aDfnZVWp7pcB7yOxca8PMUaO9NvasrqF+RlzURX8hRIv5xwFghxKU3qoP7iVMs5cek9N W4blKWbfw+G/KPxkye4GUVDWg0ZXDsV5y0ORLJx3gFiTzwkTfBkVAENPfbSb9LX60ywtq NqBTiY0n8YNeYpvns/cdLwSqQ6xU8Z0oe7ddjkHMLerS+MX0egbcGs6/XDRUcP2HKCzid gH8DlE8PTr67KCzPaB4r9j4i+oD5rVZlbuxBY1R5QrpYL9PpGp/hRiJQhmeKjffvbcRrU CBcvmFb+wSPUyyTju42LvZvOQSOeb75b9PW//WYsCBBmaGEk1gW4xysdvgwxt0fN9Cw== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794912626; h=Received:DKIM-Signature:From:To:Cc:Subject:Date:Message-ID:X-Mailer: In-Reply-To:References:MIME-Version:Content-Transfer-Encoding; bh=yRXwLzaDT06b8RPgqReFuPbVsVDXZUm9S792xRviclA=; b=vJbYYKmsEvI3Jpg9WPz3kvDZhstZqdPNJxLyB/FgfYYngaSjFcsju45UcrMHCK4pFAcL Js2aBVISe5a0DB6aOBNHyo0kXD9of+mTub+r1/jWsYXeiODZkeNDMKcVjOPGqQXy4mRMH 9/CUlCx8pvDNTt24YyBkWfkG4MkSCCP+HSrILgPrly5aSPtDsEuYJHdoit0JKI2zqETSP ShYkXRsAGEJn45I9xyYUKfjXX0ZfTSL1fy2lPgiFkzSsKqYnBDAn1HaBNJo6/ClNQvb7G L8n9Ds7yIYiIoREBDwamZ3SmhRP4KF0sNbPUCErHPrezqObDUsc0KNFJPsJSV0jq3Vw== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=linuxfoundation.org header.i=@linuxfoundation.org header.a=rsa-sha256 header.s=korg header.b="NeHmT/Si"; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4516606A323; Tue, 17 Nov 2026 10:50:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.a=rsa-sha256 header.s=korg header.b=NeHmT/Si Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0337E1F000C5; Tue, 17 Nov 2026 10:47:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=korg; t=1794912626; bh=yRXwLzaDT06b8RPgqReFuPbVsVDXZUm9S792xRviclA=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=NeHmT/SivtiLZKl+0pArrE3eH8Pmuig2f1H4LRwyvY0tmlptztmGGu0IMvnyrNFnr JLEmrFauXGXst1SMKjapw9XM/8LWVjWNajW03TLrwOKY5IvylnK+bM6RwflJW1fUk1 hTvtzrtYVqkGmpwzTtQmvq7lfsG+qD5EZGRxUFSY= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org, akpm@linux-foundation.org, torvalds@linux-foundation.org, stable@vger.kernel.org Cc: lwn@lwn.net, jslaby@suse.cz, Greg Kroah-Hartman Subject: Re: Linux 6.12.118 Date: Tue, 17 Nov 2026 11:45:24 +0100 Message-ID: <2026111724-snowbound-swear-ef83@gregkh> X-Mailer: git-send-email 2.56.0 In-Reply-To: <2026111724-motivate-lifting-94f1@gregkh> References: <2026111724-motivate-lifting-94f1@gregkh> X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Transfer-Encoding: 8bit diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 886095f45290..c0748a3c5aa6 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -4214,6 +4214,11 @@ This can be set from sysctl after boot. See Documentation/admin-guide/sysctl/vm.rst for details. + nvme_tcp.tls_handshake_timeout= + [NVME] NVMe/TCP host TLS handshake timeout in seconds. + Valid values range from 0 to the smaller of + UINT_MAX / 1000 and MAX_JIFFY_OFFSET / HZ. Default: 10 + ohci1394_dma=early [HW,EARLY] enable debugging via the ohci1394 driver. See Documentation/core-api/debugging-via-ohci1394.rst for more info. @@ -7260,6 +7265,11 @@ q = USB_QUIRK_FORCE_ONE_CONFIG (Device claims zero configurations, forcing to 1); + r = USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE (Device + fails during initialization when asked for + 9-bytes configuration descriptor request. + Ask for 255-bytes request instead to mirror + Windows' behavior); Example: quirks=0781:5580:bk,0a5c:5834:gij usbhid.mousepoll= diff --git a/Documentation/filesystems/fsverity.rst b/Documentation/filesystems/fsverity.rst index 0e2fac7a16da..f879e052d4ec 100644 --- a/Documentation/filesystems/fsverity.rst +++ b/Documentation/filesystems/fsverity.rst @@ -170,6 +170,7 @@ fatal signal), no changes are made to the file. FS_IOC_ENABLE_VERITY can fail with the following errors: - ``EACCES``: the process does not have write access to the file +- ``EBADF``: the file descriptor is not open for reading - ``EBADMSG``: the builtin signature is malformed - ``EBUSY``: this ioctl is already running on the file - ``EEXIST``: the file already has verity enabled @@ -233,6 +234,7 @@ regardless of the size of the file. FS_IOC_MEASURE_VERITY can fail with the following errors: +- ``EBADF``: the file descriptor is not open for reading - ``EFAULT``: the caller provided inaccessible memory - ``ENODATA``: the file is not a verity file - ``ENOTTY``: this type of filesystem does not implement fs-verity @@ -305,6 +307,7 @@ blocks from disk without actually verifying the path to the root node. FS_IOC_READ_VERITY_METADATA can fail with the following errors: +- ``EBADF``: the file descriptor is not open for reading - ``EFAULT``: the caller provided inaccessible memory - ``EINTR``: the ioctl was interrupted before any data was read - ``EINVAL``: reserved fields were set, or ``offset + length`` diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst index c51fb6f2651f..0e01bfc75fce 100644 --- a/Documentation/filesystems/overlayfs.rst +++ b/Documentation/filesystems/overlayfs.rst @@ -678,7 +678,8 @@ When decoding an overlay file handle, the following steps are followed: 1. Find underlying layer by UUID and path type information. 2. Decode the underlying filesystem file handle to underlying dentry. - 3. For a lower file handle, lookup the handle in index directory by name. + 3. For a lower file handle, encode the decoded underlying inode again and + lookup that handle in index directory by name. 4. If a whiteout is found in index, return ESTALE. This represents an overlay object that was deleted after its file handle was encoded. 5. For a non-directory, instantiate a disconnected overlay dentry from the diff --git a/Documentation/hwmon/coretemp.rst b/Documentation/hwmon/coretemp.rst index c609329e3bc4..6089bee1159f 100644 --- a/Documentation/hwmon/coretemp.rst +++ b/Documentation/hwmon/coretemp.rst @@ -47,6 +47,9 @@ All Sysfs entries are named with their core_id (represented here by 'X'). ================= ======================================================== tempX_input Core temperature (in millidegrees Celsius). + Reading fails with -ENODATA while the CPU reports the + temperature as invalid. Some CPUs do this when the + temperature is too low to be measured. tempX_max All cooling devices should be turned on (on Core2). tempX_crit Maximum junction temperature (in millidegrees Celsius). tempX_crit_alarm Set when Out-of-spec bit is set, never clears. diff --git a/Makefile b/Makefile index 1464c3c0e2a5..c0d66e826498 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 VERSION = 6 PATCHLEVEL = 12 -SUBLEVEL = 117 +SUBLEVEL = 118 EXTRAVERSION = NAME = Baby Opossum Posse diff --git a/arch/arm64/boot/dts/qcom/ipq9574.dtsi b/arch/arm64/boot/dts/qcom/ipq9574.dtsi index 81ccd0600c5a..31f97309c533 100644 --- a/arch/arm64/boot/dts/qcom/ipq9574.dtsi +++ b/arch/arm64/boot/dts/qcom/ipq9574.dtsi @@ -315,7 +315,7 @@ gcc: clock-controller@1800000 { <0>, <0>, <0>, - <0>; + <&usb_0_qmpphy>; #clock-cells = <1>; #reset-cells = <1>; #interconnect-cells = <1>; diff --git a/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts b/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts index cb5e8364c973..fc79641d7bb5 100644 --- a/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts +++ b/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts @@ -7,12 +7,20 @@ /dts-v1/; +#include "k3-am625.dtsi" #include "k3-am62x-sk-common.dtsi" / { compatible = "ti,am62-lp-sk", "ti,am625"; model = "Texas Instruments AM62x LP SK"; + memory@80000000 { + /* 2G RAM */ + reg = <0x00000000 0x80000000 0x00000000 0x80000000>; + device_type = "memory"; + bootph-pre-ram; + }; + vmain_pd: regulator-0 { /* TPS65988 PD CONTROLLER OUTPUT */ compatible = "regulator-fixed"; diff --git a/arch/arm64/boot/dts/ti/k3-am625-sk-common.dtsi b/arch/arm64/boot/dts/ti/k3-am625-sk-common.dtsi new file mode 100644 index 000000000000..db85d354133f --- /dev/null +++ b/arch/arm64/boot/dts/ti/k3-am625-sk-common.dtsi @@ -0,0 +1,304 @@ +// SPDX-License-Identifier: GPL-2.0-only OR MIT +/* + * Common dtsi for AM625 SK and derivatives + * + * Copyright (C) 2021-2024 Texas Instruments Incorporated - https://www.ti.com/ + */ + +#include "k3-am62x-sk-common.dtsi" + +/ { + opp-table { + /* Add 1.4GHz OPP for am625-sk board. Requires VDD_CORE to be at 0.85V */ + opp-1400000000 { + opp-hz = /bits/ 64 <1400000000>; + opp-supported-hw = <0x01 0x0004>; + clock-latency-ns = <6000000>; + }; + }; + + vmain_pd: regulator-0 { + /* TPS65988 PD CONTROLLER OUTPUT */ + bootph-all; + compatible = "regulator-fixed"; + regulator-name = "vmain_pd"; + regulator-min-microvolt = <5000000>; + regulator-max-microvolt = <5000000>; + regulator-always-on; + regulator-boot-on; + }; + + vcc_5v0: regulator-1 { + /* Output of LM34936 */ + bootph-all; + compatible = "regulator-fixed"; + regulator-name = "vcc_5v0"; + regulator-min-microvolt = <5000000>; + regulator-max-microvolt = <5000000>; + vin-supply = <&vmain_pd>; + regulator-always-on; + regulator-boot-on; + }; + + vcc_3v3_sys: regulator-2 { + /* output of LM61460-Q1 */ + bootph-all; + compatible = "regulator-fixed"; + regulator-name = "vcc_3v3_sys"; + regulator-min-microvolt = <3300000>; + regulator-max-microvolt = <3300000>; + vin-supply = <&vmain_pd>; + regulator-always-on; + regulator-boot-on; + }; + + vdd_mmc1: regulator-3 { + /* TPS22918DBVR */ + bootph-all; + compatible = "regulator-fixed"; + regulator-name = "vdd_mmc1"; + regulator-min-microvolt = <3300000>; + regulator-max-microvolt = <3300000>; + regulator-boot-on; + enable-active-high; + vin-supply = <&vcc_3v3_sys>; + gpio = <&exp1 3 GPIO_ACTIVE_HIGH>; + }; + + vdd_sd_dv: regulator-4 { + /* Output of TLV71033 */ + bootph-all; + compatible = "regulator-gpio"; + regulator-name = "tlv71033"; + pinctrl-names = "default"; + pinctrl-0 = <&vdd_sd_dv_pins_default>; + regulator-min-microvolt = <1800000>; + regulator-max-microvolt = <3300000>; + regulator-boot-on; + vin-supply = <&vcc_5v0>; + gpios = <&main_gpio0 31 GPIO_ACTIVE_HIGH>; + states = <1800000 0x0>, + <3300000 0x1>; + }; + + vcc_1v8: regulator-5 { + /* output of TPS6282518DMQ */ + compatible = "regulator-fixed"; + regulator-name = "vcc_1v8"; + regulator-min-microvolt = <1800000>; + regulator-max-microvolt = <1800000>; + vin-supply = <&vcc_3v3_sys>; + regulator-always-on; + regulator-boot-on; + }; +}; + +&main_pmx0 { + main_mmc0_pins_default: main-mmc0-default-pins { + bootph-all; + pinctrl-single,pins = < + AM62X_IOPAD(0x220, PIN_INPUT, 0) /* (Y3) MMC0_CMD */ + AM62X_IOPAD(0x218, PIN_OUTPUT, 0) /* (AB1) MMC0_CLK */ + AM62X_IOPAD(0x214, PIN_INPUT, 0) /* (AA2) MMC0_DAT0 */ + AM62X_IOPAD(0x210, PIN_INPUT_PULLUP, 0) /* (AA1) MMC0_DAT1 */ + AM62X_IOPAD(0x20c, PIN_INPUT_PULLUP, 0) /* (AA3) MMC0_DAT2 */ + AM62X_IOPAD(0x208, PIN_INPUT_PULLUP, 0) /* (Y4) MMC0_DAT3 */ + AM62X_IOPAD(0x204, PIN_INPUT_PULLUP, 0) /* (AB2) MMC0_DAT4 */ + AM62X_IOPAD(0x200, PIN_INPUT_PULLUP, 0) /* (AC1) MMC0_DAT5 */ + AM62X_IOPAD(0x1fc, PIN_INPUT_PULLUP, 0) /* (AD2) MMC0_DAT6 */ + AM62X_IOPAD(0x1f8, PIN_INPUT_PULLUP, 0) /* (AC2) MMC0_DAT7 */ + >; + }; + + main_rgmii2_pins_default: main-rgmii2-default-pins { + bootph-all; + pinctrl-single,pins = < + AM62X_IOPAD(0x184, PIN_INPUT, 0) /* (AE23) RGMII2_RD0 */ + AM62X_IOPAD(0x188, PIN_INPUT, 0) /* (AB20) RGMII2_RD1 */ + AM62X_IOPAD(0x18c, PIN_INPUT, 0) /* (AC21) RGMII2_RD2 */ + AM62X_IOPAD(0x190, PIN_INPUT, 0) /* (AE22) RGMII2_RD3 */ + AM62X_IOPAD(0x180, PIN_INPUT, 0) /* (AD23) RGMII2_RXC */ + AM62X_IOPAD(0x17c, PIN_INPUT, 0) /* (AD22) RGMII2_RX_CTL */ + AM62X_IOPAD(0x16c, PIN_OUTPUT, 0) /* (Y18) RGMII2_TD0 */ + AM62X_IOPAD(0x170, PIN_OUTPUT, 0) /* (AA18) RGMII2_TD1 */ + AM62X_IOPAD(0x174, PIN_OUTPUT, 0) /* (AD21) RGMII2_TD2 */ + AM62X_IOPAD(0x178, PIN_OUTPUT, 0) /* (AC20) RGMII2_TD3 */ + AM62X_IOPAD(0x168, PIN_OUTPUT, 0) /* (AE21) RGMII2_TXC */ + AM62X_IOPAD(0x164, PIN_OUTPUT, 0) /* (AA19) RGMII2_TX_CTL */ + >; + }; + + ospi0_pins_default: ospi0-default-pins { + bootph-all; + pinctrl-single,pins = < + AM62X_IOPAD(0x000, PIN_OUTPUT, 0) /* (H24) OSPI0_CLK */ + AM62X_IOPAD(0x02c, PIN_OUTPUT, 0) /* (F23) OSPI0_CSn0 */ + AM62X_IOPAD(0x00c, PIN_INPUT, 0) /* (E25) OSPI0_D0 */ + AM62X_IOPAD(0x010, PIN_INPUT, 0) /* (G24) OSPI0_D1 */ + AM62X_IOPAD(0x014, PIN_INPUT, 0) /* (F25) OSPI0_D2 */ + AM62X_IOPAD(0x018, PIN_INPUT, 0) /* (F24) OSPI0_D3 */ + AM62X_IOPAD(0x01c, PIN_INPUT, 0) /* (J23) OSPI0_D4 */ + AM62X_IOPAD(0x020, PIN_INPUT, 0) /* (J25) OSPI0_D5 */ + AM62X_IOPAD(0x024, PIN_INPUT, 0) /* (H25) OSPI0_D6 */ + AM62X_IOPAD(0x028, PIN_INPUT, 0) /* (J22) OSPI0_D7 */ + AM62X_IOPAD(0x008, PIN_INPUT, 0) /* (J24) OSPI0_DQS */ + >; + }; + + vdd_sd_dv_pins_default: vdd-sd-dv-default-pins { + bootph-all; + pinctrl-single,pins = < + AM62X_IOPAD(0x07c, PIN_OUTPUT, 7) /* (P25) GPMC0_CLK.GPIO0_31 */ + >; + }; + + main_gpio1_ioexp_intr_pins_default: main-gpio1-ioexp-intr-default-pins { + bootph-all; + pinctrl-single,pins = < + AM62X_IOPAD(0x01d4, PIN_INPUT, 7) /* (B15) UART0_RTSn.GPIO1_23 */ + >; + }; +}; + +&main_gpio0 { + bootph-all; +}; + +&main_gpio1 { + bootph-all; +}; + +&main_i2c1 { + bootph-all; + exp1: gpio@22 { + bootph-all; + compatible = "ti,tca6424"; + reg = <0x22>; + gpio-controller; + #gpio-cells = <2>; + gpio-line-names = "GPIO_CPSW2_RST", "GPIO_CPSW1_RST", + "PRU_DETECT", "MMC1_SD_EN", + "VPP_LDO_EN", "EXP_PS_3V3_En", + "EXP_PS_5V0_En", "EXP_HAT_DETECT", + "GPIO_AUD_RSTn", "GPIO_eMMC_RSTn", + "UART1_FET_BUF_EN", "WL_LT_EN", + "GPIO_HDMI_RSTn", "CSI_GPIO1", + "CSI_GPIO2", "PRU_3V3_EN", + "HDMI_INTn", "PD_I2C_IRQ", + "MCASP1_FET_EN", "MCASP1_BUF_BT_EN", + "MCASP1_FET_SEL", "UART1_FET_SEL", + "TSINT#", "IO_EXP_TEST_LED"; + + interrupt-parent = <&main_gpio1>; + interrupts = <23 IRQ_TYPE_EDGE_FALLING>; + interrupt-controller; + #interrupt-cells = <2>; + + pinctrl-names = "default"; + pinctrl-0 = <&main_gpio1_ioexp_intr_pins_default>; + }; +}; + +&sdhci0 { + bootph-all; + non-removable; + pinctrl-names = "default"; + pinctrl-0 = <&main_mmc0_pins_default>; + status = "okay"; +}; + +&sdhci1 { + vmmc-supply = <&vdd_mmc1>; + vqmmc-supply = <&vdd_sd_dv>; +}; + +&cpsw3g { + pinctrl-names = "default"; + pinctrl-0 = <&main_rgmii1_pins_default>, <&main_rgmii2_pins_default>; +}; + +&cpsw_port2 { + phy-mode = "rgmii-rxid"; + phy-handle = <&cpsw3g_phy1>; +}; + +&cpsw3g_mdio { + cpsw3g_phy1: ethernet-phy@1 { + reg = <1>; + ti,rx-internal-delay = ; + ti,fifo-depth = ; + ti,min-output-impedance; + }; +}; + +&fss { + bootph-all; +}; + +&ospi0 { + bootph-all; + status = "okay"; + pinctrl-names = "default"; + pinctrl-0 = <&ospi0_pins_default>; + + flash@0 { + bootph-all; + compatible = "jedec,spi-nor"; + reg = <0x0>; + spi-tx-bus-width = <8>; + spi-rx-bus-width = <8>; + spi-max-frequency = <25000000>; + cdns,tshsl-ns = <60>; + cdns,tsd2d-ns = <60>; + cdns,tchsh-ns = <60>; + cdns,tslch-ns = <60>; + cdns,read-delay = <4>; + + partitions { + bootph-all; + compatible = "fixed-partitions"; + #address-cells = <1>; + #size-cells = <1>; + + partition@0 { + label = "ospi.tiboot3"; + reg = <0x0 0x80000>; + }; + + partition@80000 { + label = "ospi.tispl"; + reg = <0x80000 0x200000>; + }; + + partition@280000 { + label = "ospi.u-boot"; + reg = <0x280000 0x400000>; + }; + + partition@680000 { + label = "ospi.env"; + reg = <0x680000 0x40000>; + }; + + partition@6c0000 { + label = "ospi.env.backup"; + reg = <0x6c0000 0x40000>; + }; + + partition@800000 { + label = "ospi.rootfs"; + reg = <0x800000 0x37c0000>; + }; + + partition@3fc0000 { + bootph-pre-ram; + label = "ospi.phypattern"; + reg = <0x3fc0000 0x40000>; + }; + }; + }; +}; + +&tlv320aic3106 { + DVDD-supply = <&vcc_1v8>; +}; diff --git a/arch/arm64/boot/dts/ti/k3-am625-sk.dts b/arch/arm64/boot/dts/ti/k3-am625-sk.dts index 0fa11d3aa71e..52954c77df80 100644 --- a/arch/arm64/boot/dts/ti/k3-am625-sk.dts +++ b/arch/arm64/boot/dts/ti/k3-am625-sk.dts @@ -7,317 +7,17 @@ /dts-v1/; -#include "k3-am62x-sk-common.dtsi" +#include "k3-am625.dtsi" +#include "k3-am625-sk-common.dtsi" / { compatible = "ti,am625-sk", "ti,am625"; model = "Texas Instruments AM625 SK"; - opp-table { - /* Add 1.4GHz OPP for am625-sk board. Requires VDD_CORE to be at 0.85V */ - opp-1400000000 { - opp-hz = /bits/ 64 <1400000000>; - opp-supported-hw = <0x01 0x0004>; - clock-latency-ns = <6000000>; - }; - }; - memory@80000000 { - device_type = "memory"; /* 2G RAM */ reg = <0x00000000 0x80000000 0x00000000 0x80000000>; - - }; - - vmain_pd: regulator-0 { - /* TPS65988 PD CONTROLLER OUTPUT */ - bootph-all; - compatible = "regulator-fixed"; - regulator-name = "vmain_pd"; - regulator-min-microvolt = <5000000>; - regulator-max-microvolt = <5000000>; - regulator-always-on; - regulator-boot-on; - }; - - vcc_5v0: regulator-1 { - /* Output of LM34936 */ - bootph-all; - compatible = "regulator-fixed"; - regulator-name = "vcc_5v0"; - regulator-min-microvolt = <5000000>; - regulator-max-microvolt = <5000000>; - vin-supply = <&vmain_pd>; - regulator-always-on; - regulator-boot-on; - }; - - vcc_3v3_sys: regulator-2 { - /* output of LM61460-Q1 */ - bootph-all; - compatible = "regulator-fixed"; - regulator-name = "vcc_3v3_sys"; - regulator-min-microvolt = <3300000>; - regulator-max-microvolt = <3300000>; - vin-supply = <&vmain_pd>; - regulator-always-on; - regulator-boot-on; - }; - - vdd_mmc1: regulator-3 { - /* TPS22918DBVR */ - bootph-all; - compatible = "regulator-fixed"; - regulator-name = "vdd_mmc1"; - regulator-min-microvolt = <3300000>; - regulator-max-microvolt = <3300000>; - regulator-boot-on; - enable-active-high; - vin-supply = <&vcc_3v3_sys>; - gpio = <&exp1 3 GPIO_ACTIVE_HIGH>; - }; - - vdd_sd_dv: regulator-4 { - /* Output of TLV71033 */ - bootph-all; - compatible = "regulator-gpio"; - regulator-name = "tlv71033"; - pinctrl-names = "default"; - pinctrl-0 = <&vdd_sd_dv_pins_default>; - regulator-min-microvolt = <1800000>; - regulator-max-microvolt = <3300000>; - regulator-boot-on; - vin-supply = <&vcc_5v0>; - gpios = <&main_gpio0 31 GPIO_ACTIVE_HIGH>; - states = <1800000 0x0>, - <3300000 0x1>; - }; - - vcc_1v8: regulator-5 { - /* output of TPS6282518DMQ */ - compatible = "regulator-fixed"; - regulator-name = "vcc_1v8"; - regulator-min-microvolt = <1800000>; - regulator-max-microvolt = <1800000>; - vin-supply = <&vcc_3v3_sys>; - regulator-always-on; - regulator-boot-on; - }; -}; - -&main_pmx0 { - main_mmc0_pins_default: main-mmc0-default-pins { - bootph-all; - pinctrl-single,pins = < - AM62X_IOPAD(0x220, PIN_INPUT, 0) /* (Y3) MMC0_CMD */ - AM62X_IOPAD(0x218, PIN_INPUT, 0) /* (AB1) MMC0_CLK */ - AM62X_IOPAD(0x214, PIN_INPUT, 0) /* (AA2) MMC0_DAT0 */ - AM62X_IOPAD(0x210, PIN_INPUT_PULLUP, 0) /* (AA1) MMC0_DAT1 */ - AM62X_IOPAD(0x20c, PIN_INPUT_PULLUP, 0) /* (AA3) MMC0_DAT2 */ - AM62X_IOPAD(0x208, PIN_INPUT_PULLUP, 0) /* (Y4) MMC0_DAT3 */ - AM62X_IOPAD(0x204, PIN_INPUT_PULLUP, 0) /* (AB2) MMC0_DAT4 */ - AM62X_IOPAD(0x200, PIN_INPUT_PULLUP, 0) /* (AC1) MMC0_DAT5 */ - AM62X_IOPAD(0x1fc, PIN_INPUT_PULLUP, 0) /* (AD2) MMC0_DAT6 */ - AM62X_IOPAD(0x1f8, PIN_INPUT_PULLUP, 0) /* (AC2) MMC0_DAT7 */ - >; - }; - - main_rgmii2_pins_default: main-rgmii2-default-pins { - bootph-all; - pinctrl-single,pins = < - AM62X_IOPAD(0x184, PIN_INPUT, 0) /* (AE23) RGMII2_RD0 */ - AM62X_IOPAD(0x188, PIN_INPUT, 0) /* (AB20) RGMII2_RD1 */ - AM62X_IOPAD(0x18c, PIN_INPUT, 0) /* (AC21) RGMII2_RD2 */ - AM62X_IOPAD(0x190, PIN_INPUT, 0) /* (AE22) RGMII2_RD3 */ - AM62X_IOPAD(0x180, PIN_INPUT, 0) /* (AD23) RGMII2_RXC */ - AM62X_IOPAD(0x17c, PIN_INPUT, 0) /* (AD22) RGMII2_RX_CTL */ - AM62X_IOPAD(0x16c, PIN_OUTPUT, 0) /* (Y18) RGMII2_TD0 */ - AM62X_IOPAD(0x170, PIN_OUTPUT, 0) /* (AA18) RGMII2_TD1 */ - AM62X_IOPAD(0x174, PIN_OUTPUT, 0) /* (AD21) RGMII2_TD2 */ - AM62X_IOPAD(0x178, PIN_OUTPUT, 0) /* (AC20) RGMII2_TD3 */ - AM62X_IOPAD(0x168, PIN_OUTPUT, 0) /* (AE21) RGMII2_TXC */ - AM62X_IOPAD(0x164, PIN_OUTPUT, 0) /* (AA19) RGMII2_TX_CTL */ - >; - }; - - ospi0_pins_default: ospi0-default-pins { - bootph-all; - pinctrl-single,pins = < - AM62X_IOPAD(0x000, PIN_OUTPUT, 0) /* (H24) OSPI0_CLK */ - AM62X_IOPAD(0x02c, PIN_OUTPUT, 0) /* (F23) OSPI0_CSn0 */ - AM62X_IOPAD(0x00c, PIN_INPUT, 0) /* (E25) OSPI0_D0 */ - AM62X_IOPAD(0x010, PIN_INPUT, 0) /* (G24) OSPI0_D1 */ - AM62X_IOPAD(0x014, PIN_INPUT, 0) /* (F25) OSPI0_D2 */ - AM62X_IOPAD(0x018, PIN_INPUT, 0) /* (F24) OSPI0_D3 */ - AM62X_IOPAD(0x01c, PIN_INPUT, 0) /* (J23) OSPI0_D4 */ - AM62X_IOPAD(0x020, PIN_INPUT, 0) /* (J25) OSPI0_D5 */ - AM62X_IOPAD(0x024, PIN_INPUT, 0) /* (H25) OSPI0_D6 */ - AM62X_IOPAD(0x028, PIN_INPUT, 0) /* (J22) OSPI0_D7 */ - AM62X_IOPAD(0x008, PIN_INPUT, 0) /* (J24) OSPI0_DQS */ - >; - }; - - vdd_sd_dv_pins_default: vdd-sd-dv-default-pins { - bootph-all; - pinctrl-single,pins = < - AM62X_IOPAD(0x07c, PIN_OUTPUT, 7) /* (P25) GPMC0_CLK.GPIO0_31 */ - >; - }; - - main_gpio1_ioexp_intr_pins_default: main-gpio1-ioexp-intr-default-pins { - bootph-all; - pinctrl-single,pins = < - AM62X_IOPAD(0x01d4, PIN_INPUT, 7) /* (B15) UART0_RTSn.GPIO1_23 */ - >; - }; -}; - -&main_gpio0 { - bootph-all; -}; - -&main_gpio1 { - bootph-all; -}; - -&main_i2c1 { - bootph-all; - exp1: gpio@22 { - bootph-all; - compatible = "ti,tca6424"; - reg = <0x22>; - gpio-controller; - #gpio-cells = <2>; - gpio-line-names = "GPIO_CPSW2_RST", "GPIO_CPSW1_RST", - "PRU_DETECT", "MMC1_SD_EN", - "VPP_LDO_EN", "EXP_PS_3V3_En", - "EXP_PS_5V0_En", "EXP_HAT_DETECT", - "GPIO_AUD_RSTn", "GPIO_eMMC_RSTn", - "UART1_FET_BUF_EN", "WL_LT_EN", - "GPIO_HDMI_RSTn", "CSI_GPIO1", - "CSI_GPIO2", "PRU_3V3_EN", - "HDMI_INTn", "PD_I2C_IRQ", - "MCASP1_FET_EN", "MCASP1_BUF_BT_EN", - "MCASP1_FET_SEL", "UART1_FET_SEL", - "TSINT#", "IO_EXP_TEST_LED"; - - interrupt-parent = <&main_gpio1>; - interrupts = <23 IRQ_TYPE_EDGE_FALLING>; - interrupt-controller; - #interrupt-cells = <2>; - - pinctrl-names = "default"; - pinctrl-0 = <&main_gpio1_ioexp_intr_pins_default>; - }; -}; - -&sdhci0 { - bootph-all; - non-removable; - pinctrl-names = "default"; - pinctrl-0 = <&main_mmc0_pins_default>; - status = "okay"; -}; - -&sdhci1 { - vmmc-supply = <&vdd_mmc1>; - vqmmc-supply = <&vdd_sd_dv>; -}; - -&cpsw3g { - pinctrl-names = "default"; - pinctrl-0 = <&main_rgmii1_pins_default>, <&main_rgmii2_pins_default>; -}; - -&cpsw_port2 { - phy-mode = "rgmii-rxid"; - phy-handle = <&cpsw3g_phy1>; -}; - -&cpsw3g_mdio { - cpsw3g_phy1: ethernet-phy@1 { - reg = <1>; - ti,rx-internal-delay = ; - ti,fifo-depth = ; - ti,min-output-impedance; - }; -}; - -&mailbox0_cluster0 { - mbox_m4_0: mbox-m4-0 { - ti,mbox-rx = <0 0 0>; - ti,mbox-tx = <1 0 0>; - }; -}; - -&fss { - bootph-all; -}; - -&ospi0 { - bootph-all; - status = "okay"; - pinctrl-names = "default"; - pinctrl-0 = <&ospi0_pins_default>; - - flash@0 { - bootph-all; - compatible = "jedec,spi-nor"; - reg = <0x0>; - spi-tx-bus-width = <8>; - spi-rx-bus-width = <8>; - spi-max-frequency = <25000000>; - cdns,tshsl-ns = <60>; - cdns,tsd2d-ns = <60>; - cdns,tchsh-ns = <60>; - cdns,tslch-ns = <60>; - cdns,read-delay = <4>; - - partitions { - bootph-all; - compatible = "fixed-partitions"; - #address-cells = <1>; - #size-cells = <1>; - - partition@0 { - label = "ospi.tiboot3"; - reg = <0x0 0x80000>; - }; - - partition@80000 { - label = "ospi.tispl"; - reg = <0x80000 0x200000>; - }; - - partition@280000 { - label = "ospi.u-boot"; - reg = <0x280000 0x400000>; - }; - - partition@680000 { - label = "ospi.env"; - reg = <0x680000 0x40000>; - }; - - partition@6c0000 { - label = "ospi.env.backup"; - reg = <0x6c0000 0x40000>; - }; - - partition@800000 { - label = "ospi.rootfs"; - reg = <0x800000 0x37c0000>; - }; - - partition@3fc0000 { - bootph-pre-ram; - label = "ospi.phypattern"; - reg = <0x3fc0000 0x40000>; - }; - }; + device_type = "memory"; + bootph-pre-ram; }; }; - -&tlv320aic3106 { - DVDD-supply = <&vcc_1v8>; -}; diff --git a/arch/arm64/boot/dts/ti/k3-am62x-sk-common.dtsi b/arch/arm64/boot/dts/ti/k3-am62x-sk-common.dtsi index 9d45d8dad381..be5cc73cf2e4 100644 --- a/arch/arm64/boot/dts/ti/k3-am62x-sk-common.dtsi +++ b/arch/arm64/boot/dts/ti/k3-am62x-sk-common.dtsi @@ -8,7 +8,6 @@ #include #include #include -#include "k3-am625.dtsi" / { aliases { @@ -27,13 +26,6 @@ chosen { stdout-path = "serial2:115200n8"; }; - memory@80000000 { - bootph-pre-ram; - device_type = "memory"; - /* 2G RAM */ - reg = <0x00000000 0x80000000 0x00000000 0x80000000>; - }; - reserved-memory { #address-cells = <2>; #size-cells = <2>; diff --git a/arch/arm64/include/asm/sdei.h b/arch/arm64/include/asm/sdei.h index 484cb6972e99..3b689185f371 100644 --- a/arch/arm64/include/asm/sdei.h +++ b/arch/arm64/include/asm/sdei.h @@ -44,7 +44,8 @@ asmlinkage unsigned long __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg); unsigned long do_sdei_event(struct pt_regs *regs, - struct sdei_registered_event *arg); + struct sdei_registered_event *arg, + unsigned long vbar); unsigned long sdei_arch_get_entry_point(int conduit); #define sdei_arch_get_entry_point(x) sdei_arch_get_entry_point(x) diff --git a/arch/arm64/include/asm/suspend.h b/arch/arm64/include/asm/suspend.h index eb60c9735553..2cc5fb405769 100644 --- a/arch/arm64/include/asm/suspend.h +++ b/arch/arm64/include/asm/suspend.h @@ -2,7 +2,7 @@ #ifndef __ASM_SUSPEND_H #define __ASM_SUSPEND_H -#define NR_CTX_REGS 14 +#define NR_CTX_REGS 15 #define NR_CALLEE_SAVED_REGS 12 /* diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index ea3876d99c2e..4033ee7005a4 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -27,6 +27,7 @@ #include #include #include +#include /* * Handle IRQ/context state management when entering from kernel mode. @@ -1087,6 +1088,7 @@ asmlinkage void noinstr __noreturn handle_bad_stack(struct pt_regs *regs) asmlinkage noinstr unsigned long __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg) { + unsigned long elr, spsr, esr, far, vbar; unsigned long ret; /* @@ -1111,10 +1113,46 @@ __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg) else if (cpu_has_pan()) set_pstate_pan(0); + /* + * Nothing saved the interrupted context's exception registers on the + * way in, and it may be in the middle of exception entry or return + * (SDEI ignores DAIF). Save them before anything can take an exception + * and overwrite them. + */ + elr = read_sysreg(elr_el1); + spsr = read_sysreg(spsr_el1); + esr = read_sysreg(esr_el1); + far = read_sysreg(far_el1); + + /* + * With KPTI, VBAR_EL1 holds the trampoline vectors from the return to + * EL0 until the next exception from EL0 maps the kernel. They only + * handle exceptions from EL0: an exception taken here would run their + * empty current-EL half and never come back. Switch to the kernel + * vectors. The IRQ-like resume keeps using the interrupted VBAR, so it + * still goes through the trampoline. + */ + vbar = read_sysreg(vbar_el1); + if (arm64_kernel_unmapped_at_el0() && + vbar == (unsigned long)raw_cpu_read(this_cpu_vector)) { + write_sysreg((unsigned long)vectors, vbar_el1); + isb(); + } + arm64_enter_nmi(regs); - ret = do_sdei_event(regs, arg); + ret = do_sdei_event(regs, arg, vbar); arm64_exit_nmi(regs); + /* + * Put them back unconditionally. An unchanged ELR does not rule out a + * nested exception: one may have returned to the same address. + */ + write_sysreg(elr, elr_el1); + write_sysreg(spsr, spsr_el1); + write_sysreg(esr, esr_el1); + write_sysreg(far, far_el1); + write_sysreg(vbar, vbar_el1); + return ret; } #endif /* CONFIG_ARM_SDE_INTERFACE */ diff --git a/arch/arm64/kernel/machine_kexec_file.c b/arch/arm64/kernel/machine_kexec_file.c index af1ca875c52c..dc6948967ceb 100644 --- a/arch/arm64/kernel/machine_kexec_file.c +++ b/arch/arm64/kernel/machine_kexec_file.c @@ -128,6 +128,8 @@ int load_other_segments(struct kimage *image, vfree(headers); goto out_err; } + + vfree(image->elf_headers); image->elf_headers = headers; image->elf_load_addr = kbuf.mem; image->elf_headers_sz = headers_sz; diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index 255d12f881c2..a6d78ceb1364 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -203,21 +203,49 @@ unsigned long sdei_arch_get_entry_point(int conduit) return 0; } +/* + * Exceptions an SDEI handler may take without harm once __sdei_handler() has + * restored the interrupted context's exception registers. Their handlers are + * safe in NMI context: + * - EL1 data aborts on translation, permission and access-flag faults: + * faulthandler_disabled() is true in NMI context, so do_page_fault() never + * takes the mmap lock. This covers the _nofault() accessors used by the + * generic dump_stack()/show_regs() code (e.g. print_worker_info()). + * - BRK: kprobes, WARN/BUG, kCFI and UBSAN traps, all NMI-capable. + * Anything else taken at EL1 is fatal on its own, or unexpected here. + */ +static bool sdei_nested_exception_ok(unsigned long esr) +{ + switch (ESR_ELx_EC(esr)) { + case ESR_ELx_EC_DABT_CUR: + return esr_fsc_is_translation_fault(esr) || + esr_fsc_is_permission_fault(esr) || + esr_fsc_is_access_flag_fault(esr); + case ESR_ELx_EC_BRK64: + return true; + default: + return false; + } +} + /* * do_sdei_event() returns one of: * SDEI_EV_HANDLED - success, return to the interrupted context. * SDEI_EV_FAILED - failure, return this error code to firmare. * virtual-address - success, return to this address. + * + * vbar is the interrupted context's VBAR_EL1, for the IRQ-like resume. */ unsigned long __kprobes do_sdei_event(struct pt_regs *regs, - struct sdei_registered_event *arg) + struct sdei_registered_event *arg, + unsigned long vbar) { u32 mode; int i, err = 0; int clobbered_registers = 4; u64 elr = read_sysreg(elr_el1); + u64 nested_elr, nested_esr, nested_far; u32 kernel_mode = read_sysreg(CurrentEL) | 1; /* +SPSel */ - unsigned long vbar = read_sysreg(vbar_el1); if (arm64_kernel_unmapped_at_el0()) clobbered_registers++; @@ -229,18 +257,27 @@ unsigned long __kprobes do_sdei_event(struct pt_regs *regs, } err = sdei_event_handler(regs, arg); - if (err) - return SDEI_EV_FAILED; - if (elr != read_sysreg(elr_el1)) { - /* - * We took a synchronous exception from the SDEI handler. - * This could deadlock, and if you interrupt KVM it will - * hyp-panic instead. - */ - pr_warn("unsafe: exception during handler\n"); + /* + * A synchronous exception taken by the handler has overwritten the + * exception registers; __sdei_handler() puts them back. Only the last + * one's syndrome survives; report it if it is not one known to be + * harmless. + * + * If the event interrupted a KVM world switch, VBAR points at the KVM + * hyp vectors and any such exception is a hyp panic instead. + */ + nested_elr = read_sysreg(elr_el1); + nested_esr = read_sysreg(esr_el1); + nested_far = read_sysreg(far_el1); + if (nested_elr != elr && !sdei_nested_exception_ok(nested_esr)) { + pr_warn_once("unexpected exception during handler: ESR %#llx FAR %#llx near %pS\n", + nested_esr, nested_far, (void *)nested_elr); } + if (err) + return SDEI_EV_FAILED; + mode = regs->pstate & (PSR_MODE32_BIT | PSR_MODE_MASK); /* diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index 052e9b9945fe..326cf60d8d58 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -289,9 +289,7 @@ int counters_read_on_cpu(int cpu, smp_call_func_t func, u64 *val) if (WARN_ON_ONCE(irqs_disabled())) return -EPERM; - smp_call_function_single(cpu, func, val, 1); - - return 0; + return smp_call_function_single(cpu, func, val, 1); } /* @@ -318,10 +316,19 @@ bool cpc_ffh_supported(void) return true; } +static bool cpc_ffh_reg_valid(const struct cpc_reg *reg) +{ + return reg->bit_width && reg->bit_width <= 64 && + reg->bit_offset <= 64 - reg->bit_width; +} + int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val) { int ret = -EOPNOTSUPP; + if (!cpc_ffh_reg_valid(reg)) + return -EINVAL; + switch ((u64)reg->address) { case 0x0: ret = counters_read_on_cpu(cpu, cpu_read_corecnt, val); diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c index 754914d9ec68..89ad3aee4715 100644 --- a/arch/arm64/kvm/arch_timer.c +++ b/arch/arm64/kvm/arch_timer.c @@ -506,8 +506,6 @@ static void timer_save_state(struct arch_timer_context *ctx) goto out; switch (index) { - u64 cval; - case TIMER_VTIMER: case TIMER_HVTIMER: timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTV_CTL)); @@ -537,11 +535,12 @@ static void timer_save_state(struct arch_timer_context *ctx) case TIMER_PTIMER: case TIMER_HPTIMER: timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTP_CTL)); - cval = read_sysreg_el0(SYS_CNTP_CVAL); - - cval -= timer_get_offset(ctx); - - timer_set_cval(ctx, cval); + /* + * With an offset, memory already holds the guest's CVAL (the + * trap handler or __deactivate_traps() wrote it). + */ + if (!timer_get_offset(ctx)) + timer_set_cval(ctx, read_sysreg_el0(SYS_CNTP_CVAL)); /* Disable the timer */ write_sysreg_el0(0, SYS_CNTP_CTL); @@ -626,7 +625,8 @@ static void timer_restore_state(struct arch_timer_context *ctx) cval = timer_get_cval(ctx); offset = timer_get_offset(ctx); set_cntpoff(offset); - cval += offset; + if (offset) + cval = timer_apply_offset(cval, offset, kvm_phys_timer_read()); write_sysreg_el0(cval, SYS_CNTP_CVAL); isb(); write_sysreg_el0(timer_get_ctl(ctx), SYS_CNTP_CTL); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 97d993f52b57..ec59faa70f97 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1189,30 +1189,33 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) local_irq_disable(); - kvm_vgic_flush_hwstate(vcpu); - - kvm_pmu_update_vcpu_events(vcpu); - /* * Ensure we set mode to IN_GUEST_MODE after we disable * interrupts and before the final VCPU requests check. * See the comment in kvm_vcpu_exiting_guest_mode() and * Documentation/virt/kvm/vcpu-requests.rst + * + * Requests to self must be posted before this point, or + * won't be taken into account until the next exit. No state + * loading other than what happens at vcpu_load() time can + * be done before we have checked for pending requests. */ smp_store_mb(vcpu->mode, IN_GUEST_MODE); if (ret <= 0 || kvm_vcpu_exit_request(vcpu, &ret)) { - isb(); /* Ensure work in x_flush_hwstate is committed */ - kvm_pmu_sync_hwstate(vcpu); if (unlikely(!irqchip_in_kernel(vcpu->kvm))) kvm_timer_sync_user(vcpu); - kvm_vgic_sync_hwstate(vcpu); smp_store_release(&vcpu->mode, OUTSIDE_GUEST_MODE); local_irq_enable(); preempt_enable(); continue; } + /* From here onwards, we are commited to entering the guest */ + kvm_vgic_flush_hwstate(vcpu); + + kvm_pmu_update_vcpu_events(vcpu); + kvm_arm_setup_debug(vcpu); kvm_arch_vcpu_ctxflush_fp(vcpu); diff --git a/arch/arm64/kvm/hyp/exception.c b/arch/arm64/kvm/hyp/exception.c index 424a5107cddb..e20979138305 100644 --- a/arch/arm64/kvm/hyp/exception.c +++ b/arch/arm64/kvm/hyp/exception.c @@ -92,6 +92,7 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode, enum exception_type type) { unsigned long sctlr, vbar, old, new, mode; + struct kvm *kvm; u64 exc_offset; mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT); @@ -131,8 +132,10 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode, new |= (old & PSR_C_BIT); new |= (old & PSR_V_BIT); - if (kvm_has_mte(kern_hyp_va(vcpu->kvm))) + kvm = vcpu_get_kvm(vcpu); + if (kvm && kvm_has_mte(kvm)) new |= PSR_TCO_BIT; + vcpu_put_kvm(vcpu, kvm); new |= (old & PSR_DIT_BIT); diff --git a/arch/arm64/kvm/hyp/include/hyp/adjust_pc.h b/arch/arm64/kvm/hyp/include/hyp/adjust_pc.h index 4fdfeabefeb4..f55950ee2a7e 100644 --- a/arch/arm64/kvm/hyp/include/hyp/adjust_pc.h +++ b/arch/arm64/kvm/hyp/include/hyp/adjust_pc.h @@ -13,6 +13,34 @@ #include #include +#ifdef __KVM_NVHE_HYPERVISOR__ +#include + +/* Under pKVM a host vCPU's ->kvm is host-writable. */ +static inline struct kvm *vcpu_get_kvm(struct kvm_vcpu *vcpu) +{ + if (is_protected_kvm_enabled()) + return pkvm_vcpu_get_kvm(vcpu); + + return kern_hyp_va(vcpu->kvm); +} + +static inline void vcpu_put_kvm(struct kvm_vcpu *vcpu, struct kvm *kvm) +{ + if (is_protected_kvm_enabled()) + pkvm_vcpu_put_kvm(vcpu, kvm); +} +#else +static inline struct kvm *vcpu_get_kvm(struct kvm_vcpu *vcpu) +{ + return vcpu->kvm; +} + +static inline void vcpu_put_kvm(struct kvm_vcpu *vcpu, struct kvm *kvm) +{ +} +#endif + static inline void kvm_skip_instr(struct kvm_vcpu *vcpu) { if (vcpu_mode_is_32bit(vcpu)) { diff --git a/arch/arm64/kvm/hyp/include/nvhe/fixed_config.h b/arch/arm64/kvm/hyp/include/nvhe/fixed_config.h index f957890c7e38..3a9909f565f2 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/fixed_config.h +++ b/arch/arm64/kvm/hyp/include/nvhe/fixed_config.h @@ -92,7 +92,8 @@ */ #define PVM_ID_AA64MMFR0_RESTRICT_UNSIGNED (\ FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64MMFR0_EL1_PARANGE), ID_AA64MMFR0_EL1_PARANGE_40) | \ - FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64MMFR0_EL1_ASIDBITS), ID_AA64MMFR0_EL1_ASIDBITS_16) \ + FIELD_PREP(ARM64_FEATURE_MASK(ID_AA64MMFR0_EL1_ASIDBITS), ID_AA64MMFR0_EL1_ASIDBITS_16) | \ + SYS_FIELD_PREP_ENUM(ID_AA64MMFR0_EL1, TGRAN16, IMP) \ ) /* diff --git a/arch/arm64/kvm/hyp/include/nvhe/pkvm.h b/arch/arm64/kvm/hyp/include/nvhe/pkvm.h index 24a9a8330d19..1a9dc0c3b7db 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/pkvm.h +++ b/arch/arm64/kvm/hyp/include/nvhe/pkvm.h @@ -69,5 +69,7 @@ int __pkvm_teardown_vm(pkvm_handle_t handle); struct pkvm_hyp_vcpu *pkvm_load_hyp_vcpu(pkvm_handle_t handle, unsigned int vcpu_idx); void pkvm_put_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu); +struct kvm *pkvm_vcpu_get_kvm(struct kvm_vcpu *vcpu); +void pkvm_vcpu_put_kvm(struct kvm_vcpu *vcpu, struct kvm *kvm); #endif /* __ARM64_KVM_NVHE_PKVM_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 7c464340bcd0..ba7d6280efa4 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -262,6 +262,48 @@ void pkvm_put_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) hyp_spin_unlock(&vm_table_lock); } +static struct pkvm_hyp_vm *pkvm_get_loaded_hyp_vm(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu *running_vcpu = host_data_ptr(host_ctxt)->__hyp_running_vcpu; + struct pkvm_hyp_vcpu *hyp_vcpu; + + if (!running_vcpu) + return NULL; + + hyp_vcpu = container_of(running_vcpu, struct pkvm_hyp_vcpu, vcpu); + + if (vcpu == &hyp_vcpu->vcpu || vcpu == hyp_vcpu->host_vcpu) + return pkvm_hyp_vcpu_to_hyp_vm(hyp_vcpu); + + return NULL; +} + +/* + * A loaded vCPU's VM is the hyp VM. An unloaded host vCPU's is mapped at + * EL2 only while pinned, so it's read once and pinned; the pin fails for + * memory the host isn't sharing. + */ +struct kvm *pkvm_vcpu_get_kvm(struct kvm_vcpu *vcpu) +{ + struct pkvm_hyp_vm *hyp_vm = pkvm_get_loaded_hyp_vm(vcpu); + struct kvm *kvm; + + if (hyp_vm) + return &hyp_vm->kvm; + + kvm = kern_hyp_va(READ_ONCE(vcpu->kvm)); + if (hyp_pin_shared_mem(kvm, kvm + 1)) + return NULL; + + return kvm; +} + +void pkvm_vcpu_put_kvm(struct kvm_vcpu *vcpu, struct kvm *kvm) +{ + if (kvm && !pkvm_get_loaded_hyp_vm(vcpu)) + hyp_unpin_shared_mem(kvm, kvm + 1); +} + static void unpin_host_vcpu(struct kvm_vcpu *host_vcpu) { if (host_vcpu) diff --git a/arch/arm64/kvm/hyp/nvhe/sys_regs.c b/arch/arm64/kvm/hyp/nvhe/sys_regs.c index 2860548d4250..039badc77bf1 100644 --- a/arch/arm64/kvm/hyp/nvhe/sys_regs.c +++ b/arch/arm64/kvm/hyp/nvhe/sys_regs.c @@ -191,6 +191,15 @@ static u64 get_pvm_id_aa64mmfr0(const struct kvm_vcpu *vcpu) set_mask = get_restricted_features_unsigned(id_aa64mmfr0_el1_sys_val, PVM_ID_AA64MMFR0_RESTRICT_UNSIGNED); + set_mask |= SYS_FIELD_PREP(ID_AA64MMFR0_EL1, TGRAN4, + min_t(int, cpuid_feature_extract_signed_field(id_aa64mmfr0_el1_sys_val, + ID_AA64MMFR0_EL1_TGRAN4_SHIFT), + ID_AA64MMFR0_EL1_TGRAN4_IMP)); + set_mask |= SYS_FIELD_PREP(ID_AA64MMFR0_EL1, TGRAN64, + min_t(int, cpuid_feature_extract_signed_field(id_aa64mmfr0_el1_sys_val, + ID_AA64MMFR0_EL1_TGRAN64_SHIFT), + ID_AA64MMFR0_EL1_TGRAN64_IMP)); + return (id_aa64mmfr0_el1_sys_val & PVM_ID_AA64MMFR0_ALLOW) | set_mask; } diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 496abfd3646b..b364d9319cea 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -195,7 +195,8 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu) offset = read_sysreg_s(SYS_CNTPOFF_EL2); if (map.direct_ptimer && offset) { - write_sysreg_el0(val + offset, SYS_CNTP_CVAL); + val = timer_apply_offset(val, offset, arch_timer_read_cntpct_el0()); + write_sysreg_el0(val, SYS_CNTP_CVAL); isb(); } } diff --git a/arch/arm64/kvm/ptdump.c b/arch/arm64/kvm/ptdump.c index e4a342e903e2..ce9a64715bcd 100644 --- a/arch/arm64/kvm/ptdump.c +++ b/arch/arm64/kvm/ptdump.c @@ -97,22 +97,35 @@ static struct kvm_ptdump_guest_state *kvm_ptdump_parser_create(struct kvm *kvm) { struct kvm_ptdump_guest_state *st; struct kvm_s2_mmu *mmu = &kvm->arch.mmu; - struct kvm_pgtable *pgtable = mmu->pgt; + s8 start_level; + u32 ia_bits; int ret; + /* + * We only need the pgt start level and IPA size to initialize the ptdump, + * get them while holding the mmu_lock. It's fine if the pgt gets freed + * afterwards, we'll check again when doing the actual dump. + */ + scoped_guard(read_lock, &kvm->mmu_lock) { + if (!mmu->pgt) + return ERR_PTR(-ENOENT); + start_level = mmu->pgt->start_level; + ia_bits = mmu->pgt->ia_bits; + } + st = kzalloc(sizeof(struct kvm_ptdump_guest_state), GFP_KERNEL_ACCOUNT); if (!st) return ERR_PTR(-ENOMEM); - ret = kvm_ptdump_build_levels(&st->level[0], pgtable->start_level); + ret = kvm_ptdump_build_levels(&st->level[0], start_level); if (ret) { kfree(st); return ERR_PTR(ret); } st->ipa_marker[0].name = "Guest IPA"; - st->ipa_marker[1].start_address = BIT(pgtable->ia_bits); - st->range[0].end = BIT(pgtable->ia_bits); + st->ipa_marker[1].start_address = BIT(ia_bits); + st->range[0].end = BIT(ia_bits); st->kvm = kvm; st->parser_state = (struct ptdump_pg_state) { @@ -141,9 +154,11 @@ static int kvm_ptdump_guest_show(struct seq_file *m, void *unused) parser_state->seq = m; - write_lock(&kvm->mmu_lock); + guard(write_lock)(&kvm->mmu_lock); + if (!mmu->pgt) + return 0; + ret = kvm_pgtable_walk(mmu->pgt, 0, BIT(mmu->pgt->ia_bits), &walker); - write_unlock(&kvm->mmu_lock); return ret; } @@ -193,17 +208,27 @@ static const struct file_operations kvm_ptdump_guest_fops = { static int kvm_pgtable_range_show(struct seq_file *m, void *unused) { - struct kvm_pgtable *pgtable = m->private; + struct kvm_s2_mmu *mmu = m->private; + struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu); + + guard(read_lock)(&kvm->mmu_lock); + + if (mmu->pgt) + seq_printf(m, "%2u\n", mmu->pgt->ia_bits); - seq_printf(m, "%2u\n", pgtable->ia_bits); return 0; } static int kvm_pgtable_levels_show(struct seq_file *m, void *unused) { - struct kvm_pgtable *pgtable = m->private; + struct kvm_s2_mmu *mmu = m->private; + struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu); + + guard(read_lock)(&kvm->mmu_lock); + + if (mmu->pgt) + seq_printf(m, "%1d\n", KVM_PGTABLE_MAX_LEVELS - mmu->pgt->start_level); - seq_printf(m, "%1d\n", KVM_PGTABLE_MAX_LEVELS - pgtable->start_level); return 0; } @@ -211,15 +236,12 @@ static int kvm_pgtable_debugfs_open(struct inode *m, struct file *file, int (*show)(struct seq_file *, void *)) { struct kvm *kvm = m->i_private; - struct kvm_pgtable *pgtable; int ret; if (!kvm_get_kvm_safe(kvm)) return -ENOENT; - pgtable = kvm->arch.mmu.pgt; - - ret = single_open(file, show, pgtable); + ret = single_open(file, show, &kvm->arch.mmu); if (ret < 0) kvm_put_kvm(kvm); return ret; diff --git a/arch/arm64/kvm/vgic/vgic-mmio-v3.c b/arch/arm64/kvm/vgic/vgic-mmio-v3.c index 6e157b0b21ad..c62666adc2f8 100644 --- a/arch/arm64/kvm/vgic/vgic-mmio-v3.c +++ b/arch/arm64/kvm/vgic/vgic-mmio-v3.c @@ -276,9 +276,19 @@ static void vgic_mmio_write_v3r_ctlr(struct kvm_vcpu *vcpu, if (ctlr != GICR_CTLR_ENABLE_LPIS) return; + /* + * Yes, disabling LPIs is painful, since it can be done from + * a *remote* vcpu! So let's not take any chance, and make + * sure that the target vCPU has written their LRs back to the + * irq structures, and release any reference they would have. + * + * If it hurts, don't do it. + */ + kvm_arm_halt_guest(vcpu->kvm); vgic_flush_pending_lpis(vcpu); vgic_its_invalidate_all_caches(vcpu->kvm); atomic_set_release(&vgic_cpu->ctlr, 0); + kvm_arm_resume_guest(vcpu->kvm); } else { ctlr = atomic_cmpxchg_acquire(&vgic_cpu->ctlr, 0, GICR_CTLR_ENABLE_LPIS); diff --git a/arch/arm64/mm/mmap.c b/arch/arm64/mm/mmap.c index 7e3ad97e27d8..61da56ed0b5a 100644 --- a/arch/arm64/mm/mmap.c +++ b/arch/arm64/mm/mmap.c @@ -59,7 +59,9 @@ int valid_phys_addr_range(phys_addr_t addr, size_t size) */ int valid_mmap_phys_addr_range(unsigned long pfn, size_t size) { - return !(((pfn << PAGE_SHIFT) + size) & ~PHYS_MASK); + const phys_addr_t limit = PHYS_MASK + 1; + + return size <= limit && pfn <= (limit - size) >> PAGE_SHIFT; } static int __init adjust_protection_map(void) diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 7afbece885a3..c7b7fa6491c3 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -103,6 +103,10 @@ SYM_FUNC_START(cpu_do_suspend) alternative_if ARM64_HAS_TCR2 mrs x2, REG_TCR2_EL1 str x2, [x0, #104] +alternative_else_nop_endif +alternative_if ARM64_SME + mrs_s x2, SYS_TPIDR2_EL0 + str x2, [x0, #112] alternative_else_nop_endif ret SYM_FUNC_END(cpu_do_suspend) @@ -142,6 +146,10 @@ alternative_if ARM64_HAS_TCR2 ldr x2, [x0, #104] msr REG_TCR2_EL1, x2 alternative_else_nop_endif +alternative_if ARM64_SME + ldr x2, [x0, #112] + msr_s SYS_TPIDR2_EL0, x2 +alternative_else_nop_endif msr sctlr_el1, x12 set_this_cpu_offset x13 diff --git a/arch/loongarch/include/asm/hw_breakpoint.h b/arch/loongarch/include/asm/hw_breakpoint.h index d202052df8a1..f4478936787d 100644 --- a/arch/loongarch/include/asm/hw_breakpoint.h +++ b/arch/loongarch/include/asm/hw_breakpoint.h @@ -121,12 +121,12 @@ bool watchpoint_handler(struct pt_regs *regs); #ifdef CONFIG_HAVE_HW_BREAKPOINT extern void ptrace_hw_copy_thread(struct task_struct *task); -extern void hw_breakpoint_thread_switch(struct task_struct *next); +extern void hw_breakpoint_thread_switch(struct task_struct *prev, struct task_struct *next); #else static inline void ptrace_hw_copy_thread(struct task_struct *task) { } -static inline void hw_breakpoint_thread_switch(struct task_struct *next) +static inline void hw_breakpoint_thread_switch(struct task_struct *prev, struct task_struct *next) { } #endif diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h index 8e6366a21fdd..f42080a20861 100644 --- a/arch/loongarch/include/asm/loongarch.h +++ b/arch/loongarch/include/asm/loongarch.h @@ -1069,6 +1069,9 @@ #define CSR_FWPS_SKIP_SHIFT 16 #define CSR_FWPS_SKIP (_ULCAST_(1) << CSR_FWPS_SKIP_SHIFT) +#define CSR_MWPS_SKIP_SHIFT 16 +#define CSR_MWPS_SKIP (_ULCAST_(1) << CSR_MWPS_SKIP_SHIFT) + /* * CSR_ECFG IM */ diff --git a/arch/loongarch/include/asm/processor.h b/arch/loongarch/include/asm/processor.h index c3bc44b5f5b3..81af48cbee7b 100644 --- a/arch/loongarch/include/asm/processor.h +++ b/arch/loongarch/include/asm/processor.h @@ -138,6 +138,7 @@ struct thread_struct { struct loongarch_lbt lbt; /* Also conditionally copied */ /* Hardware breakpoints pinned to this task. */ + bool hbp_break_skip, hbp_watch_skip; struct perf_event *hbp_break[LOONGARCH_MAX_BRP]; struct perf_event *hbp_watch[LOONGARCH_MAX_WRP]; }; diff --git a/arch/loongarch/include/asm/switch_to.h b/arch/loongarch/include/asm/switch_to.h index 5b225aff3ba2..3b662b18e19a 100644 --- a/arch/loongarch/include/asm/switch_to.h +++ b/arch/loongarch/include/asm/switch_to.h @@ -36,7 +36,7 @@ extern asmlinkage struct task_struct *__switch_to(struct task_struct *prev, do { \ lose_fpu_inatomic(1, prev); \ lose_lbt_inatomic(1, prev); \ - hw_breakpoint_thread_switch(next); \ + hw_breakpoint_thread_switch(prev, next); \ (last) = __switch_to(prev, next, task_thread_info(next), \ __builtin_return_address(0), __builtin_frame_address(0)); \ } while (0) diff --git a/arch/loongarch/kernel/hw_breakpoint.c b/arch/loongarch/kernel/hw_breakpoint.c index 1a05350159f7..df149ff06dd5 100644 --- a/arch/loongarch/kernel/hw_breakpoint.c +++ b/arch/loongarch/kernel/hw_breakpoint.c @@ -156,6 +156,9 @@ static int hw_breakpoint_slot_setup(struct perf_event **slots, int max_slots, void ptrace_hw_copy_thread(struct task_struct *tsk) { + tsk->thread.hbp_break_skip = 0; + tsk->thread.hbp_watch_skip = 0; + memset(tsk->thread.hbp_break, 0, sizeof(tsk->thread.hbp_break)); memset(tsk->thread.hbp_watch, 0, sizeof(tsk->thread.hbp_watch)); } @@ -485,6 +488,7 @@ NOKPROBE_SYMBOL(update_bp_registers); bool breakpoint_handler(struct pt_regs *regs) { int i; + unsigned int clear_mask = 0; bool need_sigtrap = false; struct perf_event *bp, **slots; @@ -500,11 +504,13 @@ bool breakpoint_handler(struct pt_regs *regs) if (bp->attr.sigtrap) need_sigtrap = true; - csr_write32(0x1 << i, LOONGARCH_CSR_FWPS); - update_bp_registers(regs, 0, 0); + clear_mask |= (0x1 << i); } } + if (clear_mask) + csr_write32(clear_mask | CSR_FWPS_SKIP, LOONGARCH_CSR_FWPS); + return need_sigtrap; } NOKPROBE_SYMBOL(breakpoint_handler); @@ -512,6 +518,7 @@ NOKPROBE_SYMBOL(breakpoint_handler); bool watchpoint_handler(struct pt_regs *regs) { int i; + unsigned int clear_mask = 0; bool need_sigtrap = false; struct perf_event *wp, **slots; @@ -527,11 +534,13 @@ bool watchpoint_handler(struct pt_regs *regs) if (wp->attr.sigtrap) need_sigtrap = true; - csr_write32(0x1 << i, LOONGARCH_CSR_MWPS); - update_bp_registers(regs, 0, 1); + clear_mask |= (0x1 << i); } } + if (clear_mask) + csr_write32(clear_mask | CSR_MWPS_SKIP, LOONGARCH_CSR_MWPS); + return need_sigtrap; } NOKPROBE_SYMBOL(watchpoint_handler); @@ -555,7 +564,7 @@ static int __init arch_hw_breakpoint_init(void) } arch_initcall(arch_hw_breakpoint_init); -void hw_breakpoint_thread_switch(struct task_struct *next) +void hw_breakpoint_thread_switch(struct task_struct *prev, struct task_struct *next) { u64 addr, mask; struct pt_regs *regs = task_pt_regs(next); @@ -567,6 +576,26 @@ void hw_breakpoint_thread_switch(struct task_struct *next) csr_write32(CSR_FWPS_SKIP, LOONGARCH_CSR_FWPS); regs->csr_prmd |= CSR_PRMD_PWE; } else { + unsigned int fwps = csr_read32(LOONGARCH_CSR_FWPS); + unsigned int mwps = csr_read32(LOONGARCH_CSR_MWPS); + + prev->thread.hbp_break_skip = !!(fwps & CSR_FWPS_SKIP); + prev->thread.hbp_watch_skip = !!(mwps & CSR_MWPS_SKIP); + + if (!next->thread.hbp_break_skip) + csr_write32(0, LOONGARCH_CSR_FWPS); + else { + csr_write32(CSR_FWPS_SKIP, LOONGARCH_CSR_FWPS); + next->thread.hbp_break_skip = 0; + } + + if (!next->thread.hbp_watch_skip) + csr_write32(0, LOONGARCH_CSR_MWPS); + else { + csr_write32(CSR_MWPS_SKIP, LOONGARCH_CSR_MWPS); + next->thread.hbp_watch_skip = 0; + } + /* Update breakpoints */ update_bp_registers(regs, 1, 0); /* Update watchpoints */ diff --git a/arch/loongarch/kernel/numa.c b/arch/loongarch/kernel/numa.c index 84fe7f854820..fc62ec78a840 100644 --- a/arch/loongarch/kernel/numa.c +++ b/arch/loongarch/kernel/numa.c @@ -318,6 +318,8 @@ static int __init fake_numa_init(void) phys_addr_t start = memblock_start_of_DRAM(); phys_addr_t end = memblock_end_of_DRAM() - 1; + set_cpuid_to_node(0, 0); + early_numa_add_cpu(0, 0); node_set(0, numa_nodes_parsed); pr_info("Faking a node at [mem %pap-%pap]\n", &start, &end); @@ -353,8 +355,10 @@ int __init init_numa_memory(void) return -EINVAL; for_each_node_mask(node, node_possible_map) { - node_mem_init(node); - node_set_online(node); + if (!cpumask_empty(&cpus_on_node[node])) { + node_mem_init(node); + node_set_online(node); + } } max_low_pfn = PHYS_PFN(memblock_end_of_DRAM()); diff --git a/arch/m68k/68000/entry.S b/arch/m68k/68000/entry.S index 72e95663b62f..0d3cbededc43 100644 --- a/arch/m68k/68000/entry.S +++ b/arch/m68k/68000/entry.S @@ -18,6 +18,13 @@ .text +/* get thread_info pointer into a2 */ + .macro getthreadinfo + movel %sp,%d1 + andl #-THREAD_SIZE,%d1 + movel %d1,%a2 + .endm + .globl system_call .globl resume .globl ret_from_exception @@ -70,10 +77,10 @@ ENTRY(system_call) movel %sp@(PT_OFF_ORIG_D0),%d0 - movel %sp,%d1 /* get thread_info pointer */ - andl #-THREAD_SIZE,%d1 - movel %d1,%a2 - btst #(TIF_SYSCALL_TRACE%8),%a2@(TINFO_FLAGS+(31-TIF_SYSCALL_TRACE)/8) + /* Doing a trace ? */ + getthreadinfo + moveb %a2@(TINFO_FLAGS+2),%d1 + andl #((_TIF_SYSCALL_TRACE + _TIF_SECCOMP) >> 8),%d1 jne do_trace cmpl #NR_syscalls,%d0 jcc badsys @@ -96,16 +103,15 @@ Luser_return: /* heavy interrupt load*/ andw #ALLOWINT,%sr - movel %sp,%d1 /* get thread_info pointer */ - andl #-THREAD_SIZE,%d1 - movel %d1,%a2 + getthreadinfo 1: - move %a2@(TINFO_FLAGS),%d1 /* thread_info->flags */ + /* check if any of the flags are set */ + moveb %a2@(TINFO_FLAGS + 3),%d1 /* thread_info->flags (low 8 bits) */ jne Lwork_to_do RESTORE_ALL Lwork_to_do: - movel %a2@(TINFO_FLAGS),%d1 /* thread_info->flags */ + /* check if reschedule needs to be called */ btst #TIF_NEED_RESCHED,%d1 jne reschedule diff --git a/arch/m68k/coldfire/entry.S b/arch/m68k/coldfire/entry.S index 4ea08336e2fb..d658abdc5207 100644 --- a/arch/m68k/coldfire/entry.S +++ b/arch/m68k/coldfire/entry.S @@ -72,7 +72,8 @@ ENTRY(system_call) movel %d2,%a0 movel %a0@,%a1 /* save top of frame */ movel %sp,%a1@(TASK_THREAD+THREAD_ESP0) - btst #(TIF_SYSCALL_TRACE%8),%a0@(TINFO_FLAGS+(31-TIF_SYSCALL_TRACE)/8) + moveb %a0@(TINFO_FLAGS+2),%d2 + andl #((_TIF_SYSCALL_TRACE + _TIF_SECCOMP) >> 8),%d2 bnes 1f movel %d3,%a0 diff --git a/arch/microblaze/include/asm/entry.h b/arch/microblaze/include/asm/entry.h index 6c42bed41166..74c5a99a37f8 100644 --- a/arch/microblaze/include/asm/entry.h +++ b/arch/microblaze/include/asm/entry.h @@ -21,6 +21,19 @@ #define PER_CPU(var) var +/* + * The MicroBlaze ABI has the caller reserve an argument home area: + * REG_PARM_STACK_SPACE is 24 and OUTGOING_REG_PARM_STACK_SPACE is 1, so a + * callee may write to [caller_sp + 4, caller_sp + 28). The kernel calls C + * with r1 at the frame base, so reserve that area below pt_regs and reach + * the saved registers through PTO. This restores what 6e83557c38b4 + * ("microblaze: Remove r0_ram pointer and PTO alignment") removed; the old + * value of 24 was one word short and left the last argument slot + * overlapping pt_regs' r0. + */ +#define PTO 28 +#define STATE_SAVE_SIZE (PT_SIZE + PTO) + # ifndef __ASSEMBLY__ DECLARE_PER_CPU(unsigned int, KSP); /* Saved kernel stack pointer */ DECLARE_PER_CPU(unsigned int, KM); /* Kernel/user mode */ diff --git a/arch/microblaze/include/asm/processor.h b/arch/microblaze/include/asm/processor.h index 4e193c7550df..fe82fdb49f27 100644 --- a/arch/microblaze/include/asm/processor.h +++ b/arch/microblaze/include/asm/processor.h @@ -73,7 +73,7 @@ unsigned long __get_wchan(struct task_struct *p); # define task_regs(task) ((struct pt_regs *)task_tos(task) - 1) # define task_pt_regs_plus_args(tsk) \ - ((void *)task_pt_regs(tsk)) + (((void *)task_pt_regs(tsk)) - PTO) # define task_sp(task) (task_regs(task)->r1) # define task_pc(task) (task_regs(task)->pc) diff --git a/arch/microblaze/kernel/entry.S b/arch/microblaze/kernel/entry.S index efa4ab056fb5..ebc8bcb98c52 100644 --- a/arch/microblaze/kernel/entry.S +++ b/arch/microblaze/kernel/entry.S @@ -177,78 +177,78 @@ syscall_debug_table: 1: #define SAVE_REGS \ - swi r2, r1, PT_R2; /* Save SDA */ \ - swi r3, r1, PT_R3; \ - swi r4, r1, PT_R4; \ - swi r5, r1, PT_R5; \ - swi r6, r1, PT_R6; \ - swi r7, r1, PT_R7; \ - swi r8, r1, PT_R8; \ - swi r9, r1, PT_R9; \ - swi r10, r1, PT_R10; \ - swi r11, r1, PT_R11; /* save clobbered regs after rval */\ - swi r12, r1, PT_R12; \ - swi r13, r1, PT_R13; /* Save SDA2 */ \ - swi r14, r1, PT_PC; /* PC, before IRQ/trap */ \ - swi r15, r1, PT_R15; /* Save LP */ \ - swi r16, r1, PT_R16; \ - swi r17, r1, PT_R17; \ - swi r18, r1, PT_R18; /* Save asm scratch reg */ \ - swi r19, r1, PT_R19; \ - swi r20, r1, PT_R20; \ - swi r21, r1, PT_R21; \ - swi r22, r1, PT_R22; \ - swi r23, r1, PT_R23; \ - swi r24, r1, PT_R24; \ - swi r25, r1, PT_R25; \ - swi r26, r1, PT_R26; \ - swi r27, r1, PT_R27; \ - swi r28, r1, PT_R28; \ - swi r29, r1, PT_R29; \ - swi r30, r1, PT_R30; \ - swi r31, r1, PT_R31; /* Save current task reg */ \ + swi r2, r1, PTO+PT_R2; /* Save SDA */ \ + swi r3, r1, PTO+PT_R3; \ + swi r4, r1, PTO+PT_R4; \ + swi r5, r1, PTO+PT_R5; \ + swi r6, r1, PTO+PT_R6; \ + swi r7, r1, PTO+PT_R7; \ + swi r8, r1, PTO+PT_R8; \ + swi r9, r1, PTO+PT_R9; \ + swi r10, r1, PTO+PT_R10; \ + swi r11, r1, PTO+PT_R11; /* save clobbered regs after rval */\ + swi r12, r1, PTO+PT_R12; \ + swi r13, r1, PTO+PT_R13; /* Save SDA2 */ \ + swi r14, r1, PTO+PT_PC; /* PC, before IRQ/trap */ \ + swi r15, r1, PTO+PT_R15; /* Save LP */ \ + swi r16, r1, PTO+PT_R16; \ + swi r17, r1, PTO+PT_R17; \ + swi r18, r1, PTO+PT_R18; /* Save asm scratch reg */ \ + swi r19, r1, PTO+PT_R19; \ + swi r20, r1, PTO+PT_R20; \ + swi r21, r1, PTO+PT_R21; \ + swi r22, r1, PTO+PT_R22; \ + swi r23, r1, PTO+PT_R23; \ + swi r24, r1, PTO+PT_R24; \ + swi r25, r1, PTO+PT_R25; \ + swi r26, r1, PTO+PT_R26; \ + swi r27, r1, PTO+PT_R27; \ + swi r28, r1, PTO+PT_R28; \ + swi r29, r1, PTO+PT_R29; \ + swi r30, r1, PTO+PT_R30; \ + swi r31, r1, PTO+PT_R31; /* Save current task reg */ \ mfs r11, rmsr; /* save MSR */ \ - swi r11, r1, PT_MSR; + swi r11, r1, PTO+PT_MSR; #define RESTORE_REGS_GP \ - lwi r2, r1, PT_R2; /* restore SDA */ \ - lwi r3, r1, PT_R3; \ - lwi r4, r1, PT_R4; \ - lwi r5, r1, PT_R5; \ - lwi r6, r1, PT_R6; \ - lwi r7, r1, PT_R7; \ - lwi r8, r1, PT_R8; \ - lwi r9, r1, PT_R9; \ - lwi r10, r1, PT_R10; \ - lwi r11, r1, PT_R11; /* restore clobbered regs after rval */\ - lwi r12, r1, PT_R12; \ - lwi r13, r1, PT_R13; /* restore SDA2 */ \ - lwi r14, r1, PT_PC; /* RESTORE_LINK PC, before IRQ/trap */\ - lwi r15, r1, PT_R15; /* restore LP */ \ - lwi r16, r1, PT_R16; \ - lwi r17, r1, PT_R17; \ - lwi r18, r1, PT_R18; /* restore asm scratch reg */ \ - lwi r19, r1, PT_R19; \ - lwi r20, r1, PT_R20; \ - lwi r21, r1, PT_R21; \ - lwi r22, r1, PT_R22; \ - lwi r23, r1, PT_R23; \ - lwi r24, r1, PT_R24; \ - lwi r25, r1, PT_R25; \ - lwi r26, r1, PT_R26; \ - lwi r27, r1, PT_R27; \ - lwi r28, r1, PT_R28; \ - lwi r29, r1, PT_R29; \ - lwi r30, r1, PT_R30; \ - lwi r31, r1, PT_R31; /* Restore cur task reg */ + lwi r2, r1, PTO+PT_R2; /* restore SDA */ \ + lwi r3, r1, PTO+PT_R3; \ + lwi r4, r1, PTO+PT_R4; \ + lwi r5, r1, PTO+PT_R5; \ + lwi r6, r1, PTO+PT_R6; \ + lwi r7, r1, PTO+PT_R7; \ + lwi r8, r1, PTO+PT_R8; \ + lwi r9, r1, PTO+PT_R9; \ + lwi r10, r1, PTO+PT_R10; \ + lwi r11, r1, PTO+PT_R11; /* restore clobbered regs after rval */\ + lwi r12, r1, PTO+PT_R12; \ + lwi r13, r1, PTO+PT_R13; /* restore SDA2 */ \ + lwi r14, r1, PTO+PT_PC; /* RESTORE_LINK PC, before IRQ/trap */\ + lwi r15, r1, PTO+PT_R15; /* restore LP */ \ + lwi r16, r1, PTO+PT_R16; \ + lwi r17, r1, PTO+PT_R17; \ + lwi r18, r1, PTO+PT_R18; /* restore asm scratch reg */ \ + lwi r19, r1, PTO+PT_R19; \ + lwi r20, r1, PTO+PT_R20; \ + lwi r21, r1, PTO+PT_R21; \ + lwi r22, r1, PTO+PT_R22; \ + lwi r23, r1, PTO+PT_R23; \ + lwi r24, r1, PTO+PT_R24; \ + lwi r25, r1, PTO+PT_R25; \ + lwi r26, r1, PTO+PT_R26; \ + lwi r27, r1, PTO+PT_R27; \ + lwi r28, r1, PTO+PT_R28; \ + lwi r29, r1, PTO+PT_R29; \ + lwi r30, r1, PTO+PT_R30; \ + lwi r31, r1, PTO+PT_R31; /* Restore cur task reg */ #define RESTORE_REGS \ - lwi r11, r1, PT_MSR; \ + lwi r11, r1, PTO+PT_MSR; \ mts rmsr , r11; \ RESTORE_REGS_GP #define RESTORE_REGS_RTBD \ - lwi r11, r1, PT_MSR; \ + lwi r11, r1, PTO+PT_MSR; \ andni r11, r11, MSR_EIP; /* clear EIP */ \ ori r11, r11, MSR_EE | MSR_BIP; /* set EE and BIP */ \ mts rmsr , r11; \ @@ -265,11 +265,11 @@ syscall_debug_table: lwi r1, r0, TOPHYS(PER_CPU(ENTRY_SP)); \ /* FIXME: I can add these two lines to one */ \ /* tophys(r1,r1); */ \ - /* addik r1, r1, -PT_SIZE; */ \ - addik r1, r1, CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - PT_SIZE; \ + /* addik r1, r1, -STATE_SAVE_SIZE; */ \ + addik r1, r1, CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - STATE_SAVE_SIZE; \ SAVE_REGS \ brid 2f; \ - swi r1, r1, PT_MODE; \ + swi r1, r1, PTO+PT_MODE; \ 1: /* User-mode state save. */ \ lwi r1, r0, TOPHYS(PER_CPU(CURRENT_SAVE)); /* get saved current */\ tophys(r1,r1); \ @@ -277,12 +277,13 @@ syscall_debug_table: /* MS these three instructions can be added to one */ \ /* addik r1, r1, THREAD_SIZE; */ \ /* tophys(r1,r1); */ \ - /* addik r1, r1, -PT_SIZE; */ \ - addik r1, r1, THREAD_SIZE + CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - PT_SIZE; \ + /* addik r1, r1, -STATE_SAVE_SIZE; */ \ + addik r1, r1, THREAD_SIZE + CONFIG_KERNEL_BASE_ADDR \ + - CONFIG_KERNEL_START - STATE_SAVE_SIZE; \ SAVE_REGS \ lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP)); \ - swi r11, r1, PT_R1; /* Store user SP. */ \ - swi r0, r1, PT_MODE; /* Was in user-mode. */ \ + swi r11, r1, PTO+PT_R1; /* Store user SP. */ \ + swi r0, r1, PTO+PT_MODE; /* Was in user-mode. */ \ /* MS: I am clearing UMS even in case when I come from kernel space */ \ clear_ums; \ 2: lwi CURRENT_TASK, r0, TOPHYS(PER_CPU(CURRENT_SAVE)); @@ -292,7 +293,7 @@ syscall_debug_table: .extern cpuinfo C_ENTRY(mb_flush_dcache): - addik r1, r1, -PT_SIZE + addik r1, r1, -STATE_SAVE_SIZE SAVE_REGS addik r3, r0, cpuinfo @@ -305,12 +306,12 @@ C_ENTRY(mb_flush_dcache): addk r9, r9, r8 RESTORE_REGS - addik r1, r1, PT_SIZE + addik r1, r1, STATE_SAVE_SIZE rtsd r15, 8 nop C_ENTRY(mb_invalidate_icache): - addik r1, r1, -PT_SIZE + addik r1, r1, -STATE_SAVE_SIZE SAVE_REGS addik r3, r0, cpuinfo @@ -323,7 +324,7 @@ C_ENTRY(mb_invalidate_icache): addk r9, r9, r8 RESTORE_REGS - addik r1, r1, PT_SIZE + addik r1, r1, STATE_SAVE_SIZE rtsd r15, 8 nop @@ -350,18 +351,18 @@ C_ENTRY(_user_exception): addik r1, r1, THREAD_SIZE; tophys(r1,r1); - addik r1, r1, -PT_SIZE; /* Make room on the stack. */ + addik r1, r1, -STATE_SAVE_SIZE; /* Make room on the stack. */ SAVE_REGS - swi r0, r1, PT_R3 - swi r0, r1, PT_R4 + swi r0, r1, PTO+PT_R3 + swi r0, r1, PTO+PT_R4 - swi r0, r1, PT_MODE; /* Was in user-mode. */ + swi r0, r1, PTO+PT_MODE; /* Was in user-mode. */ lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP)); - swi r11, r1, PT_R1; /* Store user SP. */ + swi r11, r1, PTO+PT_R1; /* Store user SP. */ clear_ums; 2: lwi CURRENT_TASK, r0, TOPHYS(PER_CPU(CURRENT_SAVE)); /* Save away the syscall number. */ - swi r12, r1, PT_R0; + swi r12, r1, PTO+PT_R0; tovirt(r1,r1) /* where the trap should return need -8 to adjust for rtsd r15, 8*/ @@ -380,18 +381,18 @@ C_ENTRY(_user_exception): beqi r11, 4f addik r3, r0, -ENOSYS - swi r3, r1, PT_R3 + swi r3, r1, PTO+PT_R3 brlid r15, do_syscall_trace_enter - addik r5, r1, PT_R0 + addik r5, r1, PTO+PT_R0 # do_syscall_trace_enter returns the new syscall nr. addk r12, r0, r3 - lwi r5, r1, PT_R5; - lwi r6, r1, PT_R6; - lwi r7, r1, PT_R7; - lwi r8, r1, PT_R8; - lwi r9, r1, PT_R9; - lwi r10, r1, PT_R10; + lwi r5, r1, PTO+PT_R5; + lwi r6, r1, PTO+PT_R6; + lwi r7, r1, PTO+PT_R7; + lwi r8, r1, PTO+PT_R8; + lwi r9, r1, PTO+PT_R9; + lwi r10, r1, PTO+PT_R10; 4: /* Jump to the appropriate function for the system call number in r12 * (r12 is not preserved), or return an error if r12 is not valid. @@ -432,8 +433,8 @@ C_ENTRY(_user_exception): /* Entry point used to return from a syscall/trap */ /* We re-enable BIP bit before state restore */ C_ENTRY(ret_from_trap): - swi r3, r1, PT_R3 - swi r4, r1, PT_R4 + swi r3, r1, PTO+PT_R3 + swi r4, r1, PTO+PT_R4 /* * Entry point for returns that must not store r3/r4 back into pt_regs, * i.e. rt_sigreturn, which has already restored them from the signal @@ -441,7 +442,7 @@ C_ENTRY(ret_from_trap): * so it stays correct if the number of stores above ever changes. */ ret_from_trap_no_rval: - lwi r11, r1, PT_MODE; + lwi r11, r1, PTO+PT_MODE; /* See if returning to kernel mode, if so, skip resched &c. */ bnei r11, 2f; /* We're returning to user mode, so check for various conditions that @@ -453,7 +454,7 @@ ret_from_trap_no_rval: beqi r11, 1f brlid r15, do_syscall_trace_leave - addik r5, r1, PT_R0 + addik r5, r1, PTO+PT_R0 1: /* We're returning to user mode, so check for various conditions that * trigger rescheduling. */ @@ -472,7 +473,7 @@ ret_from_trap_no_rval: andi r11, r19, _TIF_SIGPENDING | _TIF_NOTIFY_RESUME; beqi r11, 4f; /* Signals to handle, handle them */ - addik r5, r1, 0; /* Arg 1: struct pt_regs *regs */ + addik r5, r1, PTO; /* Arg 1: struct pt_regs *regs */ bralid r15, do_notify_resume; /* Handle any signals */ add r6, r30, r0; /* Arg 2: int in_syscall */ add r30, r0, r0 /* no more restarts */ @@ -484,7 +485,11 @@ ret_from_trap_no_rval: VM_OFF; tophys(r1,r1); RESTORE_REGS_RTBD; - addik r1, r1, PT_SIZE /* Clean up stack space. */ + addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space. */ + /* + * Deliberately PT_SIZE, not STATE_SAVE_SIZE: r1 is back at the stack + * top, so PTO cancels and the saved SP sits at PT_R1 - PT_SIZE. + */ lwi r1, r1, PT_R1 - PT_SIZE;/* Restore user stack pointer. */ bri 6f; @@ -493,7 +498,7 @@ ret_from_trap_no_rval: VM_OFF; tophys(r1,r1); RESTORE_REGS_RTBD; - addik r1, r1, PT_SIZE /* Clean up stack space. */ + addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space. */ tovirt(r1,r1); 6: TRAP_return: /* Make global symbol for debugging */ @@ -534,7 +539,7 @@ C_ENTRY(sys_rt_sigreturn_wrapper): */ addik r15, r0, ret_from_trap_no_rval - 8 brid sys_rt_sigreturn /* Do real work */ - addik r5, r1, 0; /* add user context as 1st arg */ + addik r5, r1, PTO; /* add user context as 1st arg */ /* * HW EXCEPTION rutine start @@ -545,7 +550,7 @@ C_ENTRY(full_exception_trap): addik r17, r17, -4 SAVE_STATE /* Save registers */ /* PC, before IRQ/trap - this is one instruction above */ - swi r17, r1, PT_PC; + swi r17, r1, PTO+PT_PC; tovirt(r1,r1) /* FIXME this can be store directly in PT_ESR reg. * I tested it but there is a fault */ @@ -555,7 +560,7 @@ C_ENTRY(full_exception_trap): mfs r7, rfsr; /* save FSR */ mts rfsr, r0; /* Clear sticky fsr */ rted r0, full_exception - addik r5, r1, 0 /* parameter struct pt_regs * regs */ + addik r5, r1, PTO /* parameter struct pt_regs * regs */ /* * Unaligned data trap. @@ -581,14 +586,14 @@ C_ENTRY(unaligned_data_trap): lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP)); SAVE_STATE /* Save registers.*/ /* PC, before IRQ/trap - this is one instruction above */ - swi r17, r1, PT_PC; + swi r17, r1, PTO+PT_PC; tovirt(r1,r1) /* where the trap should return need -8 to adjust for rtsd r15, 8 */ addik r15, r0, ret_from_exc-8 mfs r3, resr /* ESR */ mfs r4, rear /* EAR */ rtbd r0, _unaligned_data_exception - addik r7, r1, 0 /* parameter struct pt_regs * regs */ + addik r7, r1, PTO /* parameter struct pt_regs * regs */ /* * Page fault traps. @@ -611,30 +616,30 @@ C_ENTRY(unaligned_data_trap): C_ENTRY(page_fault_data_trap): SAVE_STATE /* Save registers.*/ /* PC, before IRQ/trap - this is one instruction above */ - swi r17, r1, PT_PC; + swi r17, r1, PTO+PT_PC; tovirt(r1,r1) /* where the trap should return need -8 to adjust for rtsd r15, 8 */ addik r15, r0, ret_from_exc-8 mfs r6, rear /* parameter unsigned long address */ mfs r7, resr /* parameter unsigned long error_code */ rted r0, do_page_fault - addik r5, r1, 0 /* parameter struct pt_regs * regs */ + addik r5, r1, PTO /* parameter struct pt_regs * regs */ C_ENTRY(page_fault_instr_trap): SAVE_STATE /* Save registers.*/ /* PC, before IRQ/trap - this is one instruction above */ - swi r17, r1, PT_PC; + swi r17, r1, PTO+PT_PC; tovirt(r1,r1) /* where the trap should return need -8 to adjust for rtsd r15, 8 */ addik r15, r0, ret_from_exc-8 mfs r6, rear /* parameter unsigned long address */ ori r7, r0, 0 /* parameter unsigned long error_code */ rted r0, do_page_fault - addik r5, r1, 0 /* parameter struct pt_regs * regs */ + addik r5, r1, PTO /* parameter struct pt_regs * regs */ /* Entry point used to return from an exception. */ C_ENTRY(ret_from_exc): - lwi r11, r1, PT_MODE; + lwi r11, r1, PTO+PT_MODE; bnei r11, 2f; /* See if returning to kernel mode, */ /* ... if so, skip resched &c. */ @@ -666,7 +671,7 @@ C_ENTRY(ret_from_exc): * complete register state. Here we save anything not saved by * the normal entry sequence, so that it may be safely restored * (in a possibly modified form) after do_notify_resume returns. */ - addik r5, r1, 0; /* Arg 1: struct pt_regs *regs */ + addik r5, r1, PTO; /* Arg 1: struct pt_regs *regs */ bralid r15, do_notify_resume; /* Handle any signals */ addi r6, r0, 0; /* Arg 2: int in_syscall */ bri 1b @@ -678,7 +683,7 @@ C_ENTRY(ret_from_exc): tophys(r1,r1); RESTORE_REGS_RTBD; - addik r1, r1, PT_SIZE /* Clean up stack space. */ + addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space. */ lwi r1, r1, PT_R1 - PT_SIZE; /* Restore user stack pointer. */ bri 6f; @@ -687,7 +692,7 @@ C_ENTRY(ret_from_exc): VM_OFF; tophys(r1,r1); RESTORE_REGS_RTBD; - addik r1, r1, PT_SIZE /* Clean up stack space. */ + addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space. */ tovirt(r1,r1); 6: @@ -720,10 +725,10 @@ C_ENTRY(_interrupt): tophys(r1,r1); /* MS: I have in r1 physical address where stack is */ /* save registers */ /* MS: Make room on the stack -> activation record */ - addik r1, r1, -PT_SIZE; + addik r1, r1, -STATE_SAVE_SIZE; SAVE_REGS brid 2f; - swi r1, r1, PT_MODE; /* 0 - user mode, 1 - kernel mode */ + swi r1, r1, PTO+PT_MODE; /* 0 - user mode, 1 - kernel mode */ 1: /* User-mode state save. */ /* MS: get the saved current */ @@ -733,23 +738,23 @@ C_ENTRY(_interrupt): addik r1, r1, THREAD_SIZE; tophys(r1,r1); /* save registers */ - addik r1, r1, -PT_SIZE; + addik r1, r1, -STATE_SAVE_SIZE; SAVE_REGS /* calculate mode */ - swi r0, r1, PT_MODE; + swi r0, r1, PTO+PT_MODE; lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP)); - swi r11, r1, PT_R1; + swi r11, r1, PTO+PT_R1; clear_ums; 2: lwi CURRENT_TASK, r0, TOPHYS(PER_CPU(CURRENT_SAVE)); tovirt(r1,r1) addik r15, r0, irq_call; irq_call:rtbd r0, do_IRQ; - addik r5, r1, 0; + addik r5, r1, PTO; /* MS: we are in virtual mode */ ret_from_irq: - lwi r11, r1, PT_MODE; + lwi r11, r1, PTO+PT_MODE; bnei r11, 2f; 1: @@ -765,7 +770,7 @@ ret_from_irq: 5: andi r11, r19, _TIF_SIGPENDING | _TIF_NOTIFY_RESUME; beqid r11, no_intr_resched /* Handle a signal return; Pending signals should be in r18. */ - addik r5, r1, 0; /* Arg 1: struct pt_regs *regs */ + addik r5, r1, PTO; /* Arg 1: struct pt_regs *regs */ bralid r15, do_notify_resume; /* Handle any signals */ addi r6, r0, 0; /* Arg 2: int in_syscall */ bri 1b @@ -778,7 +783,7 @@ no_intr_resched: VM_OFF; tophys(r1,r1); RESTORE_REGS - addik r1, r1, PT_SIZE /* MS: Clean up stack space. */ + addik r1, r1, STATE_SAVE_SIZE /* MS: Clean up stack space. */ lwi r1, r1, PT_R1 - PT_SIZE; bri 6f; /* MS: Return to kernel state. */ @@ -801,7 +806,7 @@ restore: VM_OFF /* MS: turn off MMU */ tophys(r1,r1) RESTORE_REGS - addik r1, r1, PT_SIZE /* MS: Clean up stack space. */ + addik r1, r1, STATE_SAVE_SIZE /* MS: Clean up stack space. */ tovirt(r1,r1); 6: IRQ_return: /* MS: Make global symbol for debugging */ @@ -820,29 +825,29 @@ C_ENTRY(_xtmr_manager_reset): lwi r1, r0, xmb_manager_stackpointer /* Restore MSR */ - lwi r2, r1, PT_MSR + lwi r2, r1, PTO+PT_MSR mts rmsr, r2 bri 4 /* restore Special purpose registers */ - lwi r2, r1, PT_PID + lwi r2, r1, PTO+PT_PID mts rpid, r2 - lwi r2, r1, PT_TLBI + lwi r2, r1, PTO+PT_TLBI mts rtlbx, r2 - lwi r2, r1, PT_ZPR + lwi r2, r1, PTO+PT_ZPR mts rzpr, r2 #if CONFIG_XILINX_MICROBLAZE0_USE_FPU - lwi r2, r1, PT_FSR + lwi r2, r1, PTO+PT_FSR mts rfsr, r2 #endif /* restore all the tlb's */ addik r3, r0, TOPHYS(tlb_skip) - addik r6, r0, PT_TLBL0 - addik r7, r0, PT_TLBH0 + addik r6, r0, PTO+PT_TLBL0 + addik r7, r0, PTO+PT_TLBH0 restore_tlb: add r6, r6, r1 add r7, r7, r1 @@ -868,9 +873,9 @@ ret_from_reset: VM_OFF /* MS: Restore all regs */ RESTORE_REGS - lwi r14, r1, PT_R14 - lwi r16, r1, PT_PC - addik r1, r1, PT_SIZE + 36 + lwi r14, r1, PTO+PT_R14 + lwi r16, r1, PTO+PT_PC + addik r1, r1, STATE_SAVE_SIZE + 36 rtbd r16, 0 nop @@ -885,11 +890,11 @@ C_ENTRY(_xmb_manager_break): * Reserve memory in the stack for context store/restore * (which includes memory for storing tlbs (max two tlbs)) */ - addik r1, r1, -PT_SIZE - 36 + addik r1, r1, -STATE_SAVE_SIZE - 36 swi r1, r0, xmb_manager_stackpointer SAVE_REGS - swi r14, r1, PT_R14 /* rewrite saved R14 value */ - swi r16, r1, PT_PC; /* PC and r16 are the same */ + swi r14, r1, PTO+PT_R14 /* rewrite saved R14 value */ + swi r16, r1, PTO+PT_PC; /* PC and r16 are the same */ lwi r6, r0, TOPHYS(xmb_manager_baseaddr) lwi r7, r0, TOPHYS(xmb_manager_crval) @@ -903,25 +908,25 @@ C_ENTRY(_xmb_manager_break): /* Save the special purpose registers */ mfs r2, rpid - swi r2, r1, PT_PID + swi r2, r1, PTO+PT_PID mfs r2, rtlbx - swi r2, r1, PT_TLBI + swi r2, r1, PTO+PT_TLBI mfs r2, rzpr - swi r2, r1, PT_ZPR + swi r2, r1, PTO+PT_ZPR #if CONFIG_XILINX_MICROBLAZE0_USE_FPU mfs r2, rfsr - swi r2, r1, PT_FSR + swi r2, r1, PTO+PT_FSR #endif mfs r2, rmsr - swi r2, r1, PT_MSR + swi r2, r1, PTO+PT_MSR /* Save all the tlb's */ addik r3, r0, TOPHYS(tlb_skip) - addik r6, r0, PT_TLBL0 - addik r7, r0, PT_TLBH0 + addik r6, r0, PTO+PT_TLBL0 + addik r7, r0, PTO+PT_TLBH0 save_tlb: add r6, r6, r1 add r7, r7, r1 @@ -978,28 +983,28 @@ C_ENTRY(_debug_exception): lwi r1, r0, TOPHYS(PER_CPU(ENTRY_SP)); /* Reload kernel stack-ptr*/ /* BIP bit is set on entry, no interrupts can occur */ - addik r1, r1, CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - PT_SIZE; + addik r1, r1, CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - STATE_SAVE_SIZE; SAVE_REGS; /* save all regs to pt_reg structure */ - swi r0, r1, PT_R0; /* R0 must be saved too */ - swi r14, r1, PT_R14 /* rewrite saved R14 value */ - swi r16, r1, PT_PC; /* PC and r16 are the same */ + swi r0, r1, PTO+PT_R0; /* R0 must be saved too */ + swi r14, r1, PTO+PT_R14 /* rewrite saved R14 value */ + swi r16, r1, PTO+PT_PC; /* PC and r16 are the same */ /* save special purpose registers to pt_regs */ mfs r11, rear; - swi r11, r1, PT_EAR; + swi r11, r1, PTO+PT_EAR; mfs r11, resr; - swi r11, r1, PT_ESR; + swi r11, r1, PTO+PT_ESR; mfs r11, rfsr; - swi r11, r1, PT_FSR; + swi r11, r1, PTO+PT_FSR; /* stack pointer is in physical address at it is decrease - * by PT_SIZE but we need to get correct R1 value */ - addik r11, r1, CONFIG_KERNEL_START - CONFIG_KERNEL_BASE_ADDR + PT_SIZE; - swi r11, r1, PT_R1 + * by STATE_SAVE_SIZE but we need to get correct R1 value */ + addik r11, r1, CONFIG_KERNEL_START - CONFIG_KERNEL_BASE_ADDR + STATE_SAVE_SIZE; + swi r11, r1, PTO+PT_R1 /* MS: r31 - current pointer isn't changed */ tovirt(r1,r1) #ifdef CONFIG_KGDB - addi r5, r1, 0 /* pass pt_reg address as the first arg */ + addi r5, r1, PTO /* pass pt_reg address as the first arg */ addik r15, r0, dbtrap_call; /* return address */ rtbd r0, microblaze_kgdb_break nop; @@ -1015,16 +1020,16 @@ C_ENTRY(_debug_exception): addik r1, r1, THREAD_SIZE; /* calculate kernel stack pointer */ tophys(r1,r1); - addik r1, r1, -PT_SIZE; /* Make room on the stack. */ + addik r1, r1, -STATE_SAVE_SIZE; /* Make room on the stack. */ SAVE_REGS; - swi r16, r1, PT_PC; /* Save LP */ - swi r0, r1, PT_MODE; /* Was in user-mode. */ + swi r16, r1, PTO+PT_PC; /* Save LP */ + swi r0, r1, PTO+PT_MODE; /* Was in user-mode. */ lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP)); - swi r11, r1, PT_R1; /* Store user SP. */ + swi r11, r1, PTO+PT_R1; /* Store user SP. */ lwi CURRENT_TASK, r0, TOPHYS(PER_CPU(CURRENT_SAVE)); tovirt(r1,r1) set_vms; - addik r5, r1, 0; + addik r5, r1, PTO; addik r15, r0, dbtrap_call; dbtrap_call: /* Return point for kernel/user entry + 8 because of rtsd r15, 8 */ rtbd r0, sw_exception @@ -1032,7 +1037,7 @@ dbtrap_call: /* Return point for kernel/user entry + 8 because of rtsd r15, 8 */ /* MS: The first instruction for the second part of the gdb/kgdb */ set_bip; /* Ints masked for state restore */ - lwi r11, r1, PT_MODE; + lwi r11, r1, PTO+PT_MODE; bnei r11, 2f; /* MS: Return to user space - gdb */ 1: @@ -1051,7 +1056,7 @@ dbtrap_call: /* Return point for kernel/user entry + 8 because of rtsd r15, 8 */ 5: andi r11, r19, _TIF_SIGPENDING | _TIF_NOTIFY_RESUME; beqi r11, 4f; /* Signals to handle, handle them */ - addik r5, r1, 0; /* Arg 1: struct pt_regs *regs */ + addik r5, r1, PTO; /* Arg 1: struct pt_regs *regs */ bralid r15, do_notify_resume; /* Handle any signals */ addi r6, r0, 0; /* Arg 2: int in_syscall */ bri 1b @@ -1062,7 +1067,7 @@ dbtrap_call: /* Return point for kernel/user entry + 8 because of rtsd r15, 8 */ tophys(r1,r1); /* MS: Restore all regs */ RESTORE_REGS_RTBD - addik r1, r1, PT_SIZE /* Clean up stack space */ + addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space */ lwi r1, r1, PT_R1 - PT_SIZE; /* Restore user stack pointer */ DBTRAP_return_user: /* MS: Make global symbol for debugging */ rtbd r16, 0; /* MS: Instructions to return from a debug trap */ @@ -1073,9 +1078,9 @@ DBTRAP_return_user: /* MS: Make global symbol for debugging */ tophys(r1,r1); /* MS: Restore all regs */ RESTORE_REGS_RTBD - lwi r14, r1, PT_R14; - lwi r16, r1, PT_PC; - addik r1, r1, PT_SIZE; /* MS: Clean up stack space */ + lwi r14, r1, PTO+PT_R14; + lwi r16, r1, PTO+PT_PC; + addik r1, r1, STATE_SAVE_SIZE; /* MS: Clean up stack space */ tovirt(r1,r1); DBTRAP_return_kernel: /* MS: Make global symbol for debugging */ rtbd r16, 0; /* MS: Instructions to return from a debug trap */ @@ -1173,7 +1178,7 @@ ENTRY(_switch_to) .ent xmb_inject_err .type xmb_inject_err, @function xmb_inject_err: - addik r1, r1, -PT_SIZE + addik r1, r1, -STATE_SAVE_SIZE SAVE_REGS /* Switch to real mode */ @@ -1197,7 +1202,7 @@ xmb_inject_err: nop; 1: RESTORE_REGS - addik r1, r1, PT_SIZE + addik r1, r1, STATE_SAVE_SIZE rtsd r15, 8; nop; .end xmb_inject_err diff --git a/arch/microblaze/kernel/hw_exception_handler.S b/arch/microblaze/kernel/hw_exception_handler.S index 07ea23965f81..4693916be526 100644 --- a/arch/microblaze/kernel/hw_exception_handler.S +++ b/arch/microblaze/kernel/hw_exception_handler.S @@ -313,6 +313,11 @@ _MB_HW_ExceptionVectorTable: .align 4 .ent _hw_exception_handler _hw_exception_handler: + /* + * This handler saves into the static pt_pool_space buffer, not into a + * stack frame, so the plain PT_* offsets are right here -- do not add + * the PTO of entry.S's frames. + */ swi r1, r0, TOPHYS(pt_pool_space + PT_R1); /* GET_SP */ /* Save date to kernel memory. Here is the problem * when you came from user space */ diff --git a/arch/microblaze/kernel/process.c b/arch/microblaze/kernel/process.c index 56342e11442d..504e73854bc2 100644 --- a/arch/microblaze/kernel/process.c +++ b/arch/microblaze/kernel/process.c @@ -19,6 +19,7 @@ #include #include #include +#include void show_regs(struct pt_regs *regs) { @@ -65,7 +66,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) * the registers. That's OK for a brand new thread.*/ memset(childregs, 0, sizeof(struct pt_regs)); memset(&ti->cpu_context, 0, sizeof(struct cpu_context)); - ti->cpu_context.r1 = (unsigned long)childregs; + ti->cpu_context.r1 = (unsigned long)childregs - PTO; ti->cpu_context.r20 = (unsigned long)args->fn; ti->cpu_context.r19 = (unsigned long)args->fn_arg; childregs->pt_mode = 1; @@ -79,7 +80,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) childregs->r1 = usp; memset(&ti->cpu_context, 0, sizeof(struct cpu_context)); - ti->cpu_context.r1 = (unsigned long)childregs; + ti->cpu_context.r1 = (unsigned long)childregs - PTO; childregs->msr |= MSR_UMS; /* we should consider the fact that childregs is a copy of the parent diff --git a/arch/mips/crypto/Kconfig b/arch/mips/crypto/Kconfig index ee9604fd2037..f951a75dfef3 100644 --- a/arch/mips/crypto/Kconfig +++ b/arch/mips/crypto/Kconfig @@ -34,7 +34,7 @@ config CRYPTO_MD5_OCTEON config CRYPTO_SHA1_OCTEON tristate "Hash functions: SHA-1 (OCTEON)" - depends on CPU_CAVIUM_OCTEON + depends on CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN select CRYPTO_SHA1 select CRYPTO_HASH help @@ -44,7 +44,7 @@ config CRYPTO_SHA1_OCTEON config CRYPTO_SHA256_OCTEON tristate "Hash functions: SHA-224 and SHA-256 (OCTEON)" - depends on CPU_CAVIUM_OCTEON + depends on CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN select CRYPTO_SHA256 select CRYPTO_HASH help @@ -54,7 +54,7 @@ config CRYPTO_SHA256_OCTEON config CRYPTO_SHA512_OCTEON tristate "Hash functions: SHA-384 and SHA-512 (OCTEON)" - depends on CPU_CAVIUM_OCTEON + depends on CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN select CRYPTO_SHA512 select CRYPTO_HASH help diff --git a/arch/mips/mm/cex-gen.S b/arch/mips/mm/cex-gen.S index e528583d1331..6989fe274aab 100644 --- a/arch/mips/mm/cex-gen.S +++ b/arch/mips/mm/cex-gen.S @@ -8,6 +8,7 @@ * * Cache error handler */ +#include #include #include #include @@ -37,6 +38,21 @@ nop nop +#if defined(CONFIG_64BIT) && VMLINUX_LOAD_ADDRESS >= XKPHYS && \ + VMLINUX_LOAD_ADDRESS < XKSEG + /* + * An XKPHYS kernel may lie outside the 256MB region a j from the + * vector reaches, so jump to the handler's uncached XKPHYS address. + */ + PTR_LA k0, cache_parity_error + PTR_LI k1, TO_PHYS_MASK + and k0, k0, k1 + PTR_LI k1, UNCAC_BASE + or k0, k0, k1 + jr k0 + nop +#else j cache_parity_error nop +#endif END(except_vec2_generic) diff --git a/arch/mips/pci/ops-mace.c b/arch/mips/pci/ops-mace.c index 951d8070fb48..8827e5f2ea46 100644 --- a/arch/mips/pci/ops-mace.c +++ b/arch/mips/pci/ops-mace.c @@ -35,15 +35,27 @@ static inline int mkaddr(struct pci_bus *bus, unsigned int devfn, (reg & 0xfc); } +/* + * On CRIME 1.1, a MACE PCI register read followed by two register writes + * can hang the machine under DMA load, so every register write in this + * file is followed by a read. Config cycles to empty slots end in a + * master abort; ack it with interrupts still off so the bridge error + * interrupt does not report it. + */ +static void mace_pci_ack_abort(void) +{ + u32 error = mace->pci.error; + + if (error & MACEPCI_ERROR_MASTER_ABORT) { + mace->pci.error = error & ~MACEPCI_ERROR_MASTER_ABORT; + mace->pci.error; + } +} static int mace_pci_read_config(struct pci_bus *bus, unsigned int devfn, int reg, int size, u32 *val) { - u32 control = mace->pci.control; - - /* disable master aborts interrupts during config read */ - mace->pci.control = control & ~MACEPCI_CONTROL_MAR_INT; mace->pci.config_addr = mkaddr(bus, devfn, reg); switch (size) { case 1: @@ -56,9 +68,7 @@ mace_pci_read_config(struct pci_bus *bus, unsigned int devfn, *val = mace->pci.config_data.l; break; } - /* ack possible master abort */ - mace->pci.error &= ~MACEPCI_ERROR_MASTER_ABORT; - mace->pci.control = control; + mace_pci_ack_abort(); /* * someone forgot to set the ultra bit for the onboard * scsi chips; we fake it here @@ -77,6 +87,7 @@ mace_pci_write_config(struct pci_bus *bus, unsigned int devfn, int reg, int size, u32 val) { mace->pci.config_addr = mkaddr(bus, devfn, reg); + mace->pci.error; switch (size) { case 1: mace->pci.config_data.b[(reg & 3) ^ 3] = val; @@ -88,6 +99,7 @@ mace_pci_write_config(struct pci_bus *bus, unsigned int devfn, mace->pci.config_data.l = val; break; } + mace_pci_ack_abort(); DPRINTK("write%d: reg=%08x,val=%02x\n", size * 8, reg, val); diff --git a/arch/mips/pci/pci-ip32.c b/arch/mips/pci/pci-ip32.c index 7ae89d0c7099..41ec8453c865 100644 --- a/arch/mips/pci/pci-ip32.c +++ b/arch/mips/pci/pci-ip32.c @@ -77,6 +77,8 @@ static irqreturn_t macepci_error(int irq, void *dev) } mace->pci.error = flags; + /* follow the write with a read, see mace_pci_ack_abort() */ + mace->pci.error; return IRQ_HANDLED; } @@ -125,10 +127,13 @@ static int __init mace_init(void) { PCIBIOS_MIN_IO = 0x1000; - /* Clear any outstanding errors and enable interrupts */ + /* + * Clear any outstanding errors and enable interrupts. PCI interrupts + * are masked in CRIME, so the per-slot enables stay on. + */ mace->pci.error_addr = 0; mace->pci.error = 0; - mace->pci.control = 0xff008500; + mace->pci.control = 0xff0085ff; printk("MACE PCI rev %d\n", mace->pci.rev); diff --git a/arch/mips/sgi-ip32/ip32-irq.c b/arch/mips/sgi-ip32/ip32-irq.c index 29d04468a06b..b9c72af481ed 100644 --- a/arch/mips/sgi-ip32/ip32-irq.c +++ b/arch/mips/sgi-ip32/ip32-irq.c @@ -160,38 +160,6 @@ static struct irq_chip crime_edge_interrupt = { .irq_unmask = crime_enable_irq, }; -/* - * This is for MACE PCI interrupts. We can decrease bus traffic by masking - * as close to the source as possible. This also means we can take the - * next chunk of the CRIME register in one piece. - */ - -static unsigned long macepci_mask; - -static void enable_macepci_irq(struct irq_data *d) -{ - macepci_mask |= MACEPCI_CONTROL_INT(d->irq - MACEPCI_SCSI0_IRQ); - mace->pci.control = macepci_mask; - crime_mask |= 1 << (d->irq - CRIME_IRQ_BASE); - crime->imask = crime_mask; -} - -static void disable_macepci_irq(struct irq_data *d) -{ - crime_mask &= ~(1 << (d->irq - CRIME_IRQ_BASE)); - crime->imask = crime_mask; - flush_crime_bus(); - macepci_mask &= ~MACEPCI_CONTROL_INT(d->irq - MACEPCI_SCSI0_IRQ); - mace->pci.control = macepci_mask; - flush_mace_bus(); -} - -static struct irq_chip ip32_macepci_interrupt = { - .name = "IP32 MACE PCI", - .irq_mask = disable_macepci_irq, - .irq_unmask = enable_macepci_irq, -}; - /* This is used for MACE ISA interrupts. That means bits 4-6 in the * CRIME register. */ @@ -299,8 +267,8 @@ static struct irq_chip ip32_maceisa_edge_interrupt = { .irq_unmask = enable_maceisa_irq, }; -/* This is used for regular non-ISA, non-PCI MACE interrupts. That means - * bits 0-3 and 7 in the CRIME register. +/* This is used for regular non-ISA MACE interrupts. That means bits 0-3 + * and 7-15 in the CRIME register. */ static void enable_mace_irq(struct irq_data *d) @@ -436,20 +404,13 @@ void __init arch_init_irq(void) mips_cpu_irq_init(); for (irq = CRIME_IRQ_BASE; irq <= IP32_IRQ_MAX; irq++) { switch (irq) { - case MACE_VID_IN1_IRQ ... MACE_PCI_BRIDGE_IRQ: + case MACE_VID_IN1_IRQ ... MACEPCI_SHARED2_IRQ: irq_set_chip_and_handler_name(irq, &ip32_mace_interrupt, handle_level_irq, "level"); break; - case MACEPCI_SCSI0_IRQ ... MACEPCI_SHARED2_IRQ: - irq_set_chip_and_handler_name(irq, - &ip32_macepci_interrupt, - handle_level_irq, - "level"); - break; - case CRIME_CPUERR_IRQ: case CRIME_MEMERR_IRQ: irq_set_chip_and_handler_name(irq, diff --git a/arch/openrisc/kernel/or32_ksyms.c b/arch/openrisc/kernel/or32_ksyms.c index 212e5f85004c..c63d7f25bc57 100644 --- a/arch/openrisc/kernel/or32_ksyms.c +++ b/arch/openrisc/kernel/or32_ksyms.c @@ -35,6 +35,7 @@ DECLARE_EXPORT(__divsi3); DECLARE_EXPORT(__umodsi3); DECLARE_EXPORT(__modsi3); DECLARE_EXPORT(__muldi3); +DECLARE_EXPORT(__mulsi3); DECLARE_EXPORT(__ashrdi3); DECLARE_EXPORT(__ashldi3); DECLARE_EXPORT(__lshrdi3); diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h index cdea5dccaefe..02a1ea9334d1 100644 --- a/arch/powerpc/net/bpf_jit.h +++ b/arch/powerpc/net/bpf_jit.h @@ -13,6 +13,13 @@ #include #include +/* + * We need at least 2 passes for proper code generation, and may need + * additional passes if code size changes between passes. + */ +#define CODEGEN_MIN_PASSES 2 +#define CODEGEN_MAX_PASSES 3 + #ifdef CONFIG_PPC64_ELF_ABI_V1 #define FUNCTION_DESCR_SIZE 24 #else diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c index 180175e072ad..2d8cecc36b73 100644 --- a/arch/powerpc/net/bpf_jit_comp.c +++ b/arch/powerpc/net/bpf_jit_comp.c @@ -29,11 +29,10 @@ static void bpf_jit_fill_ill_insns(void *area, unsigned int size) int bpf_jit_emit_exit_insn(u32 *image, struct codegen_context *ctx, int tmp_reg, long exit_addr) { - if (!exit_addr || is_offset_in_branch_range(exit_addr - (ctx->idx * 4))) { + if (exit_addr && is_offset_in_branch_range(exit_addr - (long)(ctx->idx * 4))) { PPC_JMP(exit_addr); - } else if (ctx->alt_exit_addr) { - if (WARN_ON(!is_offset_in_branch_range((long)ctx->alt_exit_addr - (ctx->idx * 4)))) - return -1; + } else if (ctx->alt_exit_addr && is_offset_in_branch_range( + (long)(ctx->alt_exit_addr) - (long)(ctx->idx * 4))) { PPC_JMP(ctx->alt_exit_addr); } else { ctx->alt_exit_addr = ctx->idx * 4; @@ -151,6 +150,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp) */ if (cgctx.seen & SEEN_TAILCALL || !is_offset_in_branch_range((long)cgctx.idx * 4)) { cgctx.idx = 0; + cgctx.alt_exit_addr = 0; if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) { fp = org_fp; goto out_addrs; @@ -187,10 +187,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp) code_base = (u32 *)(image + FUNCTION_DESCR_SIZE); fcode_base = (u32 *)(fimage + FUNCTION_DESCR_SIZE); - /* Code generation passes 1-2 */ - for (pass = 1; pass < 3; pass++) { + /* Code generation passes 1-2+, loop until program size converges. */ + for (pass = 1; pass <= CODEGEN_MAX_PASSES; pass++) { + u32 prev_proglen = proglen; + /* Now build the prologue, body code & epilogue for real. */ cgctx.idx = 0; + cgctx.exentry_idx = 0; cgctx.alt_exit_addr = 0; bpf_jit_build_prologue(code_base, &cgctx); if (bpf_jit_build_body(fp, code_base, fcode_base, &cgctx, addrs, pass, @@ -200,11 +203,27 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp) fp = org_fp; goto out_addrs; } + addrs[fp->len] = cgctx.idx * 4; bpf_jit_build_epilogue(code_base, &cgctx); + proglen = cgctx.idx * 4; + if (bpf_jit_enable > 1) pr_info("Pass %d: shrink = %d, seen = 0x%x\n", pass, - proglen - (cgctx.idx * 4), cgctx.seen); + prev_proglen - proglen, cgctx.seen); + + /* Check if program size has converged, but ensure minimum passes */ + if (pass >= CODEGEN_MIN_PASSES && proglen == prev_proglen) + break; + + if (pass == CODEGEN_MAX_PASSES && proglen != prev_proglen) { + pr_err("BPF JIT: Program did not converge after %d passes\n", + CODEGEN_MAX_PASSES); + bpf_arch_text_copy(&fhdr->size, &hdr->size, sizeof(hdr->size)); + bpf_jit_binary_pack_free(fhdr, hdr); + fp = org_fp; + goto out_addrs; + } } if (bpf_jit_enable > 1) @@ -264,7 +283,7 @@ int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, u32 *fimage, int pass u32 *fixup; /* Populate extable entries only in the last pass */ - if (pass != 2) + if (pass < CODEGEN_MIN_PASSES) return 0; if (!fp->aux->extable || diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c index 4bcb6f222065..aad5200b7bc1 100644 --- a/arch/powerpc/platforms/pseries/iommu.c +++ b/arch/powerpc/platforms/pseries/iommu.c @@ -69,6 +69,8 @@ static struct iommu_table *iommu_pseries_alloc_table(int node) return tbl; } +static phys_addr_t pseries_ddw_max_ram __ro_after_init; + #ifdef CONFIG_IOMMU_API static struct iommu_table_group_ops spapr_tce_table_group_ops; #endif @@ -1284,15 +1286,19 @@ struct failed_ddw_pdn { static LIST_HEAD(failed_ddw_pdn_list); -static phys_addr_t ddw_memory_hotplug_max(void) +static phys_addr_t __init ddw_memory_hotplug_max(void) { - resource_size_t max_addr; + resource_size_t max_addr = memory_hotplug_max(); + struct device_node *memory; -#if defined(CONFIG_NUMA) && defined(CONFIG_MEMORY_HOTPLUG) - max_addr = hot_add_drconf_memory_max(); -#else - max_addr = memblock_end_of_DRAM(); -#endif + for_each_node_by_type(memory, "memory") { + struct resource res; + + if (of_address_to_resource(memory, 0, &res)) + continue; + + max_addr = max_t(resource_size_t, max_addr, res.end + 1); + } return max_addr; } @@ -1447,7 +1453,7 @@ static struct property *ddw_property_create(const char *propname, u32 liobn, u64 static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mask) { int len = 0, ret; - int max_ram_len = order_base_2(ddw_memory_hotplug_max()); + int max_ram_len = order_base_2(pseries_ddw_max_ram); struct ddw_query_response query; struct ddw_create_response create; int page_shift; @@ -1617,12 +1623,12 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mas if (default_win_removed && pmem_present && !direct_mapping) { /* DDW is big enough to be split */ if ((1ULL << ddw_sz) >= - MIN_DDW_VPMEM_DMA_WINDOW + (1ULL << max_ram_len)) { + MIN_DDW_VPMEM_DMA_WINDOW + pseries_ddw_max_ram) { direct_mapping = true; /* offset of the Dynamic part of DDW */ - dynamic_offset = 1ULL << max_ram_len; + dynamic_offset = pseries_ddw_max_ram; } /* DDW will at least have dynamic allocation */ @@ -1669,7 +1675,7 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mas if (direct_mapping) { /* DDW maps the whole partition, so enable direct DMA mapping */ - ret = walk_system_ram_range(0, ddw_memory_hotplug_max() >> PAGE_SHIFT, + ret = walk_system_ram_range(0, pseries_ddw_max_ram >> PAGE_SHIFT, win64->value, tce_setrange_multi_pSeriesLP_walk); if (ret) { dev_info(&dev->dev, "failed to map DMA window for %pOF: %d\n", @@ -1774,8 +1780,7 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mas * to directly map both RAM and vPMEM, then we need to set DMA limit. */ if (pmem_present && direct_mapping && len != MAX_PHYSMEM_BITS) - dev->dev.bus_dma_limit = dev->dev.archdata.dma_offset + - (1ULL << max_ram_len); + dev->dev.bus_dma_limit = dev->dev.archdata.dma_offset + pseries_ddw_max_ram; dev_info(&dev->dev, "lsa_required: %x, lsa_enabled: %x, direct mapping: %x\n", limited_addr_req, limited_addr_enabled, direct_mapping); @@ -2422,23 +2427,35 @@ static int iommu_mem_notifier(struct notifier_block *nb, unsigned long action, { struct dma_win *window; struct memory_notify *arg = data; + unsigned long limit = arg->nr_pages; + unsigned long max_ram_pages = pseries_ddw_max_ram >> PAGE_SHIFT; int ret = 0; /* This notifier can get called when onlining persistent memory as well. * TCEs are not pre-mapped for persistent memory. Persistent memory will - * always be above ddw_memory_hotplug_max() + * always be above pseries_ddw_max_ram */ + if (arg->start_pfn >= max_ram_pages) + return NOTIFY_OK; + + /* RAM is being DLPAR'ed. The range should never exceed max ram. + * Just in case, clamp the range and throw a warning. + */ + if (arg->start_pfn + limit > max_ram_pages) { + limit = max_ram_pages - arg->start_pfn; + WARN_ONCE(1, "Limiting Page Range %lx - %lx to Max Mem Pages: %lx\n", + arg->start_pfn, arg->start_pfn + arg->nr_pages, + max_ram_pages); + } switch (action) { case MEM_GOING_ONLINE: spin_lock(&dma_win_list_lock); list_for_each_entry(window, &dma_win_list, list) { - if (window->direct && (arg->start_pfn << PAGE_SHIFT) < - ddw_memory_hotplug_max()) { + if (window->direct) { ret |= tce_setrange_multi_pSeriesLP(arg->start_pfn, - arg->nr_pages, window->prop); + limit, window->prop); } - /* XXX log error */ } spin_unlock(&dma_win_list_lock); break; @@ -2446,12 +2463,10 @@ static int iommu_mem_notifier(struct notifier_block *nb, unsigned long action, case MEM_OFFLINE: spin_lock(&dma_win_list_lock); list_for_each_entry(window, &dma_win_list, list) { - if (window->direct && (arg->start_pfn << PAGE_SHIFT) < - ddw_memory_hotplug_max()) { + if (window->direct) { ret |= tce_clearrange_multi_pSeriesLP(arg->start_pfn, - arg->nr_pages, window->prop); + limit, window->prop); } - /* XXX log error */ } spin_unlock(&dma_win_list_lock); break; @@ -2535,6 +2550,14 @@ void __init iommu_init_early_pSeries(void) register_memory_notifier(&iommu_mem_nb); set_pci_dma_ops(&dma_iommu_ops); + + /* During init determine the max memory an LPAR can have and set it. This + * will be used for pre-mapping RAM in DDW. memblock_end_of_DRAM() can + * change during the running of LPAR - daxctl can add pmemory as + * "system-ram". This memory range should not be pre-mapped in DDW since + * the address of pmemory can be much higher than the DDW size. + */ + pseries_ddw_max_ram = ddw_memory_hotplug_max(); } static int __init disable_multitce(char *str) diff --git a/arch/s390/pci/pci_irq.c b/arch/s390/pci/pci_irq.c index e73be96ce5fe..836db17f4e8f 100644 --- a/arch/s390/pci/pci_irq.c +++ b/arch/s390/pci/pci_irq.c @@ -450,15 +450,10 @@ static int __init zpci_directed_irq_init(void) if (!zpci_sbv) return -ENOMEM; - iib.diib.isc = PCI_ISC; - iib.diib.nr_cpus = num_possible_cpus(); - iib.diib.disb_addr = virt_to_phys(zpci_sbv->vector); - zpci_set_irq_ctrl(SIC_IRQ_MODE_DIRECT, 0, &iib); - zpci_ibv = kcalloc(num_possible_cpus(), sizeof(*zpci_ibv), GFP_KERNEL); if (!zpci_ibv) - return -ENOMEM; + goto out_free_sbv; for_each_possible_cpu(cpu) { /* @@ -470,13 +465,30 @@ static int __init zpci_directed_irq_init(void) AIRQ_IV_CACHELINE | (!cpu ? AIRQ_IV_ALLOC : 0), NULL); if (!zpci_ibv[cpu]) - return -ENOMEM; + goto out_free_dibv; } + + iib.diib.isc = PCI_ISC; + iib.diib.nr_cpus = num_possible_cpus(); + iib.diib.disb_addr = virt_to_phys(zpci_sbv->vector); + zpci_set_irq_ctrl(SIC_IRQ_MODE_DIRECT, 0, &iib); on_each_cpu(cpu_enable_directed_irq, NULL, 1); zpci_irq_chip.irq_set_affinity = zpci_set_irq_affinity; return 0; + +out_free_dibv: + for_each_possible_cpu(cpu) { + if (zpci_ibv[cpu]) + airq_iv_release(zpci_ibv[cpu]); + } + kfree(zpci_ibv); + zpci_ibv = NULL; +out_free_sbv: + airq_iv_release(zpci_sbv); + zpci_sbv = NULL; + return -ENOMEM; } static int __init zpci_floating_irq_init(void) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 383d2bb218de..d73d6cc24922 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -483,6 +483,11 @@ #define X86_FEATURE_CLEAR_BHB_LOOP_ON_VMEXIT (21*32+ 4) /* Clear branch history at vmexit using SW loop */ #define X86_FEATURE_FAST_CPPC (21*32 + 5) /* AMD Fast CPPC */ #define X86_FEATURE_INDIRECT_THUNK_ITS (21*32 + 6) /* Use thunk for indirect branches in lower half of cacheline */ +/* + * Upstream defines X86_FEATURE_ZEN6 as (3*32+6). That bit is still + * X86_FEATURE_P3 in this kernel, so use a free bit in this word. + */ +#define X86_FEATURE_ZEN6 (21*32+7) /* CPU based on Zen6 microarchitecture */ #define X86_FEATURE_TSA_SQ_NO (21*32+11) /* AMD CPU not vulnerable to TSA-SQ */ #define X86_FEATURE_TSA_L1_NO (21*32+12) /* AMD CPU not vulnerable to TSA-L1 */ diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index 861d080ed4c6..63bdbd15efcd 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -78,6 +78,7 @@ KVM_X86_OP(interrupt_allowed) KVM_X86_OP(nmi_allowed) KVM_X86_OP(get_nmi_mask) KVM_X86_OP(set_nmi_mask) +KVM_X86_OP_OPTIONAL(clear_hlt) KVM_X86_OP(enable_nmi_window) KVM_X86_OP(enable_irq_window) KVM_X86_OP_OPTIONAL(update_cr8_intercept) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index cec497aed149..93d3f8010a1f 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1735,6 +1735,13 @@ struct kvm_x86_ops { int (*nmi_allowed)(struct kvm_vcpu *vcpu, bool for_injection); bool (*get_nmi_mask)(struct kvm_vcpu *vcpu); void (*set_nmi_mask)(struct kvm_vcpu *vcpu, bool masked); + /* + * Force the vCPU out of any hardware-tracked halted/inactive state so + * that it will fetch and execute instructions on the next VM-Enter. + * Only needed by VMX, where VMCS.GUEST_ACTIVITY_STATE persists across + * VM-Exit/VM-Enter; SVM has no equivalent VMCB field. + */ + void (*clear_hlt)(struct kvm_vcpu *vcpu); /* Whether or not a virtual NMI is pending in hardware. */ bool (*is_vnmi_pending)(struct kvm_vcpu *vcpu); /* diff --git a/arch/x86/include/asm/vmware.h b/arch/x86/include/asm/vmware.h index 4220dae14a2d..fb011cef01e7 100644 --- a/arch/x86/include/asm/vmware.h +++ b/arch/x86/include/asm/vmware.h @@ -276,20 +276,22 @@ unsigned long vmware_hypercall_hb_out(unsigned long cmd, unsigned long in2, asm_inline volatile ( UNWIND_HINT_SAVE + "mov %[in6], %%" _ASM_AX "\n\t" "push %%" _ASM_BP "\n\t" UNWIND_HINT_UNDEFINED - "mov %[in6], %%" _ASM_BP "\n\t" + "mov %%" _ASM_AX ", %%" _ASM_BP "\n\t" + "mov %[magic], %%eax\n\t" "rep outsb\n\t" "pop %%" _ASM_BP "\n\t" UNWIND_HINT_RESTORE - : "=a" (out0), "=b" (*out1) - : "a" (VMWARE_HYPERVISOR_MAGIC), - "b" (cmd), + : "=&a" (out0), "=b" (*out1) + : "b" (cmd), "c" (in2), "d" (in3 | VMWARE_HYPERVISOR_PORT_HB), "S" (in4), "D" (in5), - [in6] VMW_BP_CONSTRAINT (in6) + [in6] VMW_BP_CONSTRAINT(in6), + [magic] "i" (VMWARE_HYPERVISOR_MAGIC) : "cc", "memory"); return out0; } @@ -304,20 +306,22 @@ unsigned long vmware_hypercall_hb_in(unsigned long cmd, unsigned long in2, asm_inline volatile ( UNWIND_HINT_SAVE + "mov %[in6], %%" _ASM_AX "\n\t" "push %%" _ASM_BP "\n\t" UNWIND_HINT_UNDEFINED - "mov %[in6], %%" _ASM_BP "\n\t" + "mov %%" _ASM_AX ", %%" _ASM_BP "\n\t" + "mov %[magic], %%eax\n\t" "rep insb\n\t" "pop %%" _ASM_BP "\n\t" UNWIND_HINT_RESTORE - : "=a" (out0), "=b" (*out1) - : "a" (VMWARE_HYPERVISOR_MAGIC), - "b" (cmd), + : "=&a" (out0), "=b" (*out1) + : "b" (cmd), "c" (in2), "d" (in3 | VMWARE_HYPERVISOR_PORT_HB), "S" (in4), "D" (in5), - [in6] VMW_BP_CONSTRAINT (in6) + [in6] VMW_BP_CONSTRAINT(in6), + [magic] "i" (VMWARE_HYPERVISOR_MAGIC) : "cc", "memory"); return out0; } diff --git a/arch/x86/include/asm/xor_avx.h b/arch/x86/include/asm/xor_avx.h index 7f81dd5897f4..c6450acd055d 100644 --- a/arch/x86/include/asm/xor_avx.h +++ b/arch/x86/include/asm/xor_avx.h @@ -50,6 +50,7 @@ do { \ p1 = (unsigned long *)((uintptr_t)p1 + 512); } + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -81,6 +82,7 @@ do { \ p2 = (unsigned long *)((uintptr_t)p2 + 512); } + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -116,6 +118,7 @@ do { \ p3 = (unsigned long *)((uintptr_t)p3 + 512); } + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -155,6 +158,7 @@ do { \ p4 = (unsigned long *)((uintptr_t)p4 + 512); } + asm volatile("vzeroupper"); kernel_fpu_end(); } diff --git a/arch/x86/kernel/acpi/cppc.c b/arch/x86/kernel/acpi/cppc.c index 12fc5f553405..d9cfed143b79 100644 --- a/arch/x86/kernel/acpi/cppc.c +++ b/arch/x86/kernel/acpi/cppc.c @@ -4,6 +4,8 @@ * Copyright (c) 2016, Intel Corporation. */ +#include + #include #include #include @@ -43,10 +45,20 @@ bool cpc_ffh_supported(void) return true; } +static bool cpc_ffh_reg_valid(const struct cpc_reg *reg) +{ + return reg->address <= U32_MAX && reg->bit_width && + reg->bit_width <= 64 && + reg->bit_offset <= 64 - reg->bit_width; +} + int cpc_read_ffh(int cpunum, struct cpc_reg *reg, u64 *val) { int err; + if (!cpc_ffh_reg_valid(reg)) + return -EINVAL; + err = rdmsrl_safe_on_cpu(cpunum, reg->address, val); if (!err) { u64 mask = GENMASK_ULL(reg->bit_offset + reg->bit_width - 1, @@ -63,6 +75,9 @@ int cpc_write_ffh(int cpunum, struct cpc_reg *reg, u64 val) u64 rd_val; int err; + if (!cpc_ffh_reg_valid(reg)) + return -EINVAL; + err = rdmsrl_safe_on_cpu(cpunum, reg->address, &rd_val); if (!err) { u64 mask = GENMASK_ULL(reg->bit_offset + reg->bit_width - 1, diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c index f06966220602..686e0d1df92f 100644 --- a/arch/x86/kernel/cpu/amd.c +++ b/arch/x86/kernel/cpu/amd.c @@ -531,6 +531,12 @@ static void bsp_init_amd(struct cpuinfo_x86 *c) case 0xd0 ... 0xd7: setup_force_cpu_cap(X86_FEATURE_ZEN5); break; + case 0x50 ... 0x5f: + case 0x80 ... 0xaf: + case 0xc0 ... 0xcf: + case 0xd8 ... 0xef: + setup_force_cpu_cap(X86_FEATURE_ZEN6); + break; default: goto warn; } diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index 9a95d00f1423..49b3cc215a6a 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -120,6 +120,14 @@ static void nmi_check_duration(struct nmiaction *action, u64 duration) { int remainder_ns, decimal_msecs; + /* sched_clock() went backwards? */ + if ((s64)duration < 0) { + pr_warn_once("NMI handler (%ps): sched_clock() went backwards by %lld ns on CPU%d\n", + action->handler, -(s64)duration, + smp_processor_id()); + return; + } + if (duration < nmi_longest_ns || duration < action->max_duration) return; diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c index 0a2bbd674a6d..c1e20abc7fe5 100644 --- a/arch/x86/kernel/x86_init.c +++ b/arch/x86/kernel/x86_init.c @@ -36,7 +36,7 @@ static void iommu_shutdown_noop(void) { } bool __init bool_x86_init_noop(void) { return false; } void x86_op_int_noop(int cpu) { } int set_rtc_noop(const struct timespec64 *now) { return -EINVAL; } -void get_rtc_noop(struct timespec64 *now) { } +void get_rtc_noop(struct timespec64 *now) { *now = (struct timespec64){0}; } static __initconst const struct of_device_id of_cmos_match[] = { { .compatible = "motorola,mc146818" }, diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 3d9e8fcc5589..c4f33dc6dabc 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2156,6 +2156,16 @@ static struct kvm_mmu_page *kvm_mmu_find_shadow_page(struct kvm *kvm, continue; } + /* + * Note! Unlike fully direct MMUs, direct SPs in indirect MMUs + * can have an arbitrary number of parents, e.g if the guest is + * mapping a huge PTE at many points in its page tables. + */ + if (pte_list_count(&sp->parent_ptes) >= RMAP_RECYCLE_THRESHOLD) { + kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list); + break; + } + /* unsync and write-flooding only apply to indirect SPs. */ if (sp->role.direct) goto out; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 10e9e87f1420..aa6abaa60117 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -321,6 +321,13 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, gpa_t real_gpa; gfn_t gfn; + /* + * KVM might sleep/resched when retrying the walk if the updating A/D + * fails because the gPTE changed between reading the gPTE and trying + * the CMPXHG to set A/D bits. + */ + might_sleep(); + trace_kvm_mmu_pagetable_walk(addr, access); retry_walk: walker->level = mmu->cpu_role.base.level; @@ -471,8 +478,10 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, addr, write_fault); if (unlikely(ret < 0)) goto error; - else if (ret) + else if (ret) { + cond_resched(); goto retry_walk; + } } return 1; diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index e2e9ab085885..3c9c20d7855f 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -1763,6 +1763,7 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu, if (!(save->cr0 & X86_CR0_PG) || !(save->cr0 & X86_CR0_PE) || (save->rflags & X86_EFLAGS_VM) || + ((save->efer & EFER_LMA) && !(save->efer & EFER_LME)) || !__nested_vmcb_check_save(vcpu, &save_cached)) goto out_free; diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 3f83e36a657b..8a922ce0d19c 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -90,6 +90,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .nmi_allowed = vmx_nmi_allowed, .get_nmi_mask = vmx_get_nmi_mask, .set_nmi_mask = vmx_set_nmi_mask, + .clear_hlt = vmx_clear_hlt, .enable_nmi_window = vmx_enable_nmi_window, .enable_irq_window = vmx_enable_irq_window, .update_cr8_intercept = vmx_update_cr8_intercept, diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index c2aae9db372d..88654327a707 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -4968,6 +4968,7 @@ static void nested_vmx_restore_host_state(struct kvm_vcpu *vcpu) __func__, j, h.index, h.value); goto vmabort; } + break; } } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index a36d9f504ba5..8e23e901a759 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1814,7 +1814,7 @@ int vmx_skip_emulated_instruction(struct kvm_vcpu *vcpu) return skip_emulated_instruction(vcpu); } -static void vmx_clear_hlt(struct kvm_vcpu *vcpu) +void vmx_clear_hlt(struct kvm_vcpu *vcpu) { /* * Ensure that we clear the HLT state in the VMCS. We don't need to diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 5e4ce13ab305..4348674de9f3 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -96,6 +96,7 @@ int vmx_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection); int vmx_nmi_allowed(struct kvm_vcpu *vcpu, bool for_injection); bool vmx_get_nmi_mask(struct kvm_vcpu *vcpu); void vmx_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked); +void vmx_clear_hlt(struct kvm_vcpu *vcpu); void vmx_enable_nmi_window(struct kvm_vcpu *vcpu); void vmx_enable_irq_window(struct kvm_vcpu *vcpu); void vmx_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 68599667907c..ee3b528ec1e2 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8835,10 +8835,12 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, } else if (vcpu->arch.complete_userspace_io) { writeback = false; r = 0; - } else if (r == EMULATION_RESTART) + } else if (r == EMULATION_RESTART) { + cond_resched(); goto restart; - else + } else { r = 1; + } writeback: if (writeback) { @@ -11281,6 +11283,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, set_bit(KVM_APIC_SIPI, &vcpu->arch.apic->pending_events); } else vcpu->arch.mp_state = mp_state->mp_state; + + /* + * Force the vCPU out of any hardware-tracked halted state, e.g. VMX's + * GUEST_ACTIVITY_STATE=HLT, when userspace puts the vCPU into a state + * other than HALTED. The hardware state is sticky across VM-Exit and + * VM-Enter and is not touched by any other ioctl, so a vCPU that halted + * with HLT-exiting disabled stays wedged even after userspace rewrites + * its registers, e.g. when a VMM emulates a machine reset. + */ + if (kvm_hlt_in_guest(vcpu->kvm) && + mp_state->mp_state != KVM_MP_STATE_HALTED) + kvm_x86_call(clear_hlt)(vcpu); + kvm_make_request(KVM_REQ_EVENT, vcpu); ret = 0; @@ -11781,18 +11796,23 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) kvm_x86_call(set_efer)(vcpu, 0); kvm_x86_call(update_exception_bitmap)(vcpu); + if (old_cr0 & X86_CR0_PG) + kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); + /* - * On the standard CR0/CR4/EFER modification paths, there are several - * complex conditions determining whether the MMU has to be reset and/or - * which PCIDs have to be flushed. However, CR0.WP and the paging-related - * bits in CR4 and EFER are irrelevant if CR0.PG was '0'; and a reset+flush - * is needed anyway if CR0.PG was '1' (which can only happen for INIT, as - * CR0 will be '0' prior to RESET). So we only need to check CR0.PG here. + * Unconditionally reset the MMU, even though super strictly speaking, + * a MMU reset is only necessary if and only one of the page role bits + * is changing. All of the CR0/CR4/EFER bits effectively get covered + * by checking CR0.PG=1, as CR0.WP and the paging-related bits in CR4 + * and EFER are irrelevant if CR0.PG was '0'. Subtly, if the vCPU was + * in guest mode (L2) or SMM (although emulating INIT in SMM _should_ + * be impossible), an MMU reset is also necessary as "guest mode" and + * "smm" are both encoded in the shadow page role. As a result, the + * only scenarios where an MMU reset is *not* needed is if the vCPU is + * in Real Mode or Unpaged Protected Mode, and micro-optimizing those + * paths are not worth the risk of running with a stale MMU context. */ - if (old_cr0 & X86_CR0_PG) { - kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu); - kvm_mmu_reset_context(vcpu); - } + kvm_mmu_reset_context(vcpu); /* * Intel's SDM states that all TLB entries are flushed on INIT. AMD's diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c index b922b9fea6b6..1bbc117f91ab 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/mm/mem_encrypt_identity.c @@ -101,7 +101,7 @@ static void __head sme_clear_pgd(struct sme_populate_pgd_data *ppd) pgd_t *pgd_p; pgd_start = ppd->vaddr & PGDIR_MASK; - pgd_end = ppd->vaddr_end & PGDIR_MASK; + pgd_end = (ppd->vaddr_end - 1) & PGDIR_MASK; pgd_size = (((pgd_end - pgd_start) / PGDIR_SIZE) + 1) * sizeof(pgd_t); diff --git a/block/blk-iocost.c b/block/blk-iocost.c index 4707b9349305..b33df932ea1b 100644 --- a/block/blk-iocost.c +++ b/block/blk-iocost.c @@ -2819,6 +2819,9 @@ static void ioc_rqos_done(struct rq_qos *rqos, struct request *rq) if (!ioc->enabled || !rq->alloc_time_ns || !rq->start_time_ns) return; + if (unlikely(rq->rq_flags & RQF_FLUSH_SEQ)) + return; + switch (req_op(rq)) { case REQ_OP_READ: pidx = QOS_RLAT; diff --git a/block/blk-mq.c b/block/blk-mq.c index e01437e64c74..c5be2e8578a7 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -3122,17 +3122,15 @@ void blk_mq_submit_bio(struct bio *bio) blk_mq_bio_to_request(rq, bio, nr_segs); + if (bio_zone_write_plugging(bio)) + blk_zone_write_plug_init_request(rq); + ret = blk_crypto_rq_get_keyslot(rq); if (ret != BLK_STS_OK) { - bio->bi_status = ret; - bio_endio(bio); - blk_mq_free_request(rq); + blk_mq_end_request(rq, ret); return; } - if (bio_zone_write_plugging(bio)) - blk_zone_write_plug_init_request(rq); - if (op_is_flush(bio->bi_opf) && blk_insert_flush(rq)) return; diff --git a/block/bsg.c b/block/bsg.c index 72157a59b788..7231b1de663f 100644 --- a/block/bsg.c +++ b/block/bsg.c @@ -130,7 +130,10 @@ static long bsg_ioctl(struct file *file, unsigned int cmd, unsigned long arg) case SG_SET_TIMEOUT: if (get_user(val, intp)) return -EFAULT; - bd->timeout = clock_t_to_jiffies(val); + if (val < 0) + return -EIO; + bd->timeout = min(clock_t_to_jiffies(val), + min(UINT_MAX, MAX_JIFFY_OFFSET)); return 0; case SG_GET_TIMEOUT: return jiffies_to_clock_t(bd->timeout); diff --git a/block/ioctl.c b/block/ioctl.c index 919066b4bb49..f801bddf6fd1 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -736,8 +736,8 @@ long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg) return compat_put_ulong(argp, bdev_nr_sectors(bdev)); /* The data is compatible, but the command number is different */ - case BLKBSZGET_32: /* get the logical block size (cf. BLKSSZGET) */ - return put_int(argp, bdev_logical_block_size(bdev)); + case BLKBSZGET_32: /* get the soft block size (cf. BLKSSZGET) */ + return put_int(argp, block_size(bdev)); case BLKBSZSET_32: return blkdev_bszset(file, mode, argp); case BLKGETSIZE64_32: diff --git a/crypto/ecc.c b/crypto/ecc.c index f97ff234086f..a289d47d5651 100644 --- a/crypto/ecc.c +++ b/crypto/ecc.c @@ -1410,9 +1410,22 @@ static void ecc_point_add(const struct ecc_point *result, vli_set(result->x, q->x, ndigits); vli_set(result->y, q->y, ndigits); vli_mod_sub(z, result->x, p->x, curve->p, ndigits); + if (vli_is_zero(z, ndigits)) { + if (vli_cmp(p->y, q->y, ndigits)) { + /* P + (-P) is the point at infinity. */ + vli_clear(result->x, ndigits); + vli_clear(result->y, ndigits); + return; + } + /* The co-Z addition formula does not handle P == Q. */ + z[0] = 1; + ecc_point_double_jacobian(result->x, result->y, z, curve); + goto out; + } vli_set(px, p->x, ndigits); vli_set(py, p->y, ndigits); xycz_add(px, py, result->x, result->y, curve); +out: vli_mod_inv(z, z, curve->p, ndigits); apply_z(result->x, result->y, z, curve); } @@ -1452,22 +1465,38 @@ void ecc_point_mult_shamir(const struct ecc_point *result, vli_set(rx, point->x, ndigits); vli_set(ry, point->y, ndigits); vli_clear(z + 1, ndigits - 1); - z[0] = 1; + z[0] = !ecc_point_is_zero(point); for (--i; i >= 0; i--) { ecc_point_double_jacobian(rx, ry, z, curve); idx = !!vli_test_bit(u1, i); idx |= (!!vli_test_bit(u2, i)) << 1; point = points[idx]; - if (point) { + if (point && !ecc_point_is_zero(point)) { u64 tx[ECC_MAX_DIGITS]; u64 ty[ECC_MAX_DIGITS]; u64 tz[ECC_MAX_DIGITS]; + if (vli_is_zero(z, ndigits)) { + /* Adding to infinity starts a new accumulator. */ + vli_set(rx, point->x, ndigits); + vli_set(ry, point->y, ndigits); + z[0] = 1; + continue; + } vli_set(tx, point->x, ndigits); vli_set(ty, point->y, ndigits); apply_z(tx, ty, z, curve); vli_mod_sub(tz, rx, tx, curve->p, ndigits); + if (vli_is_zero(tz, ndigits)) { + if (!vli_cmp(ry, ty, ndigits)) + ecc_point_double_jacobian(rx, ry, z, + curve); + else + /* Adding opposite points yields infinity. */ + vli_clear(z, ndigits); + continue; + } xycz_add(tx, ty, rx, ry, curve); vli_mod_mult_fast(z, z, tz, curve); } diff --git a/crypto/testmgr.c b/crypto/testmgr.c index 2f5f6b52b2d4..e7848ea55ca1 100644 --- a/crypto/testmgr.c +++ b/crypto/testmgr.c @@ -4209,8 +4209,10 @@ static int test_akcipher_one(struct crypto_akcipher *tfm, crypto_akcipher_verify(req) : /* Run asymmetric encrypt */ crypto_akcipher_encrypt(req), &wait); - if (err) { - pr_err("alg: akcipher: %s test failed. err %d\n", op, err); + if (err != vecs->verify_error) { + pr_err("alg: akcipher: %s test failed: expected %d, got %d\n", + op, vecs->verify_error, err); + err = err ?: -EINVAL; goto free_all; } if (!vecs->siggen_sigver_test && c) { @@ -4232,9 +4234,9 @@ static int test_akcipher_one(struct crypto_akcipher *tfm, /* * Don't invoke (decrypt or sign) test which require a private key - * for vectors with only a public key. + * for vectors with only a public key or an invalid signature. */ - if (vecs->public_key_vec) { + if (vecs->public_key_vec || vecs->verify_error) { err = 0; goto free_all; } diff --git a/crypto/testmgr.h b/crypto/testmgr.h index 9b38501a17b2..2f0aa6f4454f 100644 --- a/crypto/testmgr.h +++ b/crypto/testmgr.h @@ -157,6 +157,7 @@ struct akcipher_testvec { unsigned int param_len; unsigned int m_size; unsigned int c_size; + int verify_error; bool public_key_vec; bool siggen_sigver_test; enum OID algo; @@ -778,6 +779,146 @@ static const struct akcipher_testvec ecdsa_nist_p192_tv_template[] = { }; static const struct akcipher_testvec ecdsa_nist_p256_tv_template[] = { + /* Invalid signature: Q = G, r = s, e = 3r mod n. */ + { + .key = + "\x04\x6b\x17\xd1\xf2\xe1\x2c\x42\x47\xf8\xbc\xe6\xe5\x63\xa4\x40" + "\xf2\x77\x03\x7d\x81\x2d\xeb\x33\xa0\xf4\xa1\x39\x45\xd8\x98\xc2" + "\x96\x4f\xe3\x42\xe2\xfe\x1a\x7f\x9b\x8e\xe7\xeb\x4a\x7c\x0f\x9e" + "\x16\x2b\xce\x33\x57\x6b\x31\x5e\xce\xcb\xb6\x40\x68\x37\xbf\x51" + "\xf5", + .key_len = 65, + .m = + "\xb2\xc3\x09\x72\x03\x17\x9d\xaa\x5c\x4a\xd8\xff\x02\xfe\xcb\x15" + "\x21\x53\x81\x77\x45\xec\x45\x24\x3d\x68\x10\xb9\xf2\xa5\x88\x3a", + .m_size = 32, + .c = + "\x30\x46\x02\x21\x00\x90\xeb\xad\xd0\x56\x5d\x34\x8e\x74\x18\xf2" + "\xff\xab\xaa\x43\xb1\x9f\x68\xd4\x0c\x4f\x01\x4b\xe3\x10\x60\x9e" + "\x7e\xfa\x58\x39\xd9\x02\x21\x00\x90\xeb\xad\xd0\x56\x5d\x34\x8e" + "\x74\x18\xf2\xff\xab\xaa\x43\xb1\x9f\x68\xd4\x0c\x4f\x01\x4b\xe3" + "\x10\x60\x9e\x7e\xfa\x58\x39\xd9", + .c_size = 72, + .verify_error = -EKEYREJECTED, + .public_key_vec = true, + .siggen_sigver_test = true, + }, + /* Invalid signature: Q = -G, r = s, e = 3r mod n. */ + { + .key = + "\x04\x6b\x17\xd1\xf2\xe1\x2c\x42\x47\xf8\xbc\xe6\xe5\x63\xa4\x40" + "\xf2\x77\x03\x7d\x81\x2d\xeb\x33\xa0\xf4\xa1\x39\x45\xd8\x98\xc2" + "\x96\xb0\x1c\xbd\x1c\x01\xe5\x80\x65\x71\x18\x14\xb5\x83\xf0\x61" + "\xe9\xd4\x31\xcc\xa9\x94\xce\xa1\x31\x34\x49\xbf\x97\xc8\x40\xae" + "\x0a", + .key_len = 65, + .m = + "\xb2\xc3\x09\x72\x03\x17\x9d\xaa\x5c\x4a\xd8\xff\x02\xfe\xcb\x15" + "\x21\x53\x81\x77\x45\xec\x45\x24\x3d\x68\x10\xb9\xf2\xa5\x88\x3a", + .m_size = 32, + .c = + "\x30\x46\x02\x21\x00\x90\xeb\xad\xd0\x56\x5d\x34\x8e\x74\x18\xf2" + "\xff\xab\xaa\x43\xb1\x9f\x68\xd4\x0c\x4f\x01\x4b\xe3\x10\x60\x9e" + "\x7e\xfa\x58\x39\xd9\x02\x21\x00\x90\xeb\xad\xd0\x56\x5d\x34\x8e" + "\x74\x18\xf2\xff\xab\xaa\x43\xb1\x9f\x68\xd4\x0c\x4f\x01\x4b\xe3" + "\x10\x60\x9e\x7e\xfa\x58\x39\xd9", + .c_size = 72, + .verify_error = -EKEYREJECTED, + .public_key_vec = true, + .siggen_sigver_test = true, + }, + /* Wycheproof secp256r1/SHA-256 tcId 60. */ + { + .key = + "\x04\x29\x27\xb1\x05\x12\xba\xe3\xed\xdc\xfe\x46\x78\x28\x12\x8b" + "\xad\x29\x03\x26\x99\x19\xf7\x08\x60\x69\xc8\xc4\xdf\x6c\x73\x28" + "\x38\xc7\x78\x79\x64\xea\xac\x00\xe5\x92\x1f\xb1\x49\x8a\x60\xf4" + "\x60\x67\x66\xb3\xd9\x68\x50\x01\x55\x8d\x1a\x97\x4e\x73\x41\x51" + "\x3e", + .key_len = 65, + .m = + "\x70\x23\x9d\xd8\x77\xf7\xc9\x44\xc4\x22\xf4\x4d\xea\x4e\xd1\xa5" + "\x2f\x26\x27\x41\x6f\xaf\x2f\x07\x2f\xa5\x0c\x77\x2e\xd6\xf8\x07", + .m_size = 32, + .c = + "\x30\x44\x02\x20\x64\xa1\xaa\xb5\x00\x0d\x0e\x80\x4f\x3e\x2f\xc0" + "\x2b\xde\xe9\xbe\x8f\xf3\x12\x33\x4e\x2b\xa1\x6d\x11\x54\x7c\x97" + "\x71\x1c\x89\x8e\x02\x20\x6a\xf0\x15\x97\x1c\xc3\x0b\xe6\xd1\xa2" + "\x06\xd4\xe0\x13\xe0\x99\x77\x72\xa2\xf9\x1d\x73\x28\x6f\xfd\x68" + "\x3b\x9b\xb2\xcf\x4f\x1b", + .c_size = 70, + .public_key_vec = true, + .siggen_sigver_test = true, + }, + /* Wycheproof secp256r1/SHA-256 tcId 210. */ + { + .key = + "\x04\xc6\xa7\x71\x52\x70\x24\x22\x77\x92\x17\x0a\x6f\x8e\xee\x73" + "\x5b\xf3\x2b\x7f\x98\xaf\x66\x9e\xad\x29\x98\x02\xe3\x2d\x7c\x31" + "\x07\xbc\x3b\x4b\x5e\x65\xab\x88\x7b\xbd\x34\x35\x72\xb3\xe5\x61" + "\x92\x61\xfe\x3a\x07\x3e\x2f\xfd\x78\x41\x2f\x72\x68\x67\xdb\x58" + "\x9e", + .key_len = 65, + .m = + "\xbb\x5a\x52\xf4\x2f\x9c\x92\x61\xed\x43\x61\xf5\x94\x22\xa1\xe3" + "\x00\x36\xe7\xc3\x2b\x27\x0c\x88\x07\xa4\x19\xfe\xca\x60\x50\x23", + .m_size = 32, + .c = + "\x30\x45\x02\x20\x7c\xf2\x7b\x18\x8d\x03\x4f\x7e\x8a\x52\x38\x03" + "\x04\xb5\x1a\xc3\xc0\x89\x69\xe2\x77\xf2\x1b\x35\xa6\x0b\x48\xfc" + "\x47\x66\x99\x78\x02\x21\x00\xb6\xdb\x6d\xb6\x24\x92\x49\x25\x49" + "\x24\x92\x49\x24\x92\x49\x24\x62\x5b\xd7\xa0\x9b\xec\x4c\xa8\x1b" + "\xcd\xd9\xf8\xfd\x6b\x63\xcc", + .c_size = 71, + .public_key_vec = true, + .siggen_sigver_test = true, + }, + /* Q = G, d = 1, k = 2. */ + { + .key = + "\x04\x6b\x17\xd1\xf2\xe1\x2c\x42\x47\xf8\xbc\xe6\xe5\x63\xa4\x40" + "\xf2\x77\x03\x7d\x81\x2d\xeb\x33\xa0\xf4\xa1\x39\x45\xd8\x98\xc2" + "\x96\x4f\xe3\x42\xe2\xfe\x1a\x7f\x9b\x8e\xe7\xeb\x4a\x7c\x0f\x9e" + "\x16\x2b\xce\x33\x57\x6b\x31\x5e\xce\xcb\xb6\x40\x68\x37\xbf\x51" + "\xf5", + .key_len = 65, + .m = + "\x35\x9d\x17\x29\xb9\x1a\x52\xe9\x8b\xb5\x95\xc3\x86\x28\x3c\x10" + "\x30\x44\x95\x1a\xa1\x08\x20\xf0\xa0\x22\xc4\x18\xc2\x71\xa0\xcc", + .m_size = 32, + .c = + "\x30\x44\x02\x20\x7c\xf2\x7b\x18\x8d\x03\x4f\x7e\x8a\x52\x38\x03" + "\x04\xb5\x1a\xc3\xc0\x89\x69\xe2\x77\xf2\x1b\x35\xa6\x0b\x48\xfc" + "\x47\x66\x99\x78\x02\x20\x59\x47\xc9\x21\x23\x0e\xd1\x34\x0b\x03" + "\xe6\xe3\x45\x6e\xab\x69\xf8\x66\xff\x7e\x8c\x7d\x1e\x13\x23\x17" + "\x06\x8a\x84\xec\x1d\x22", + .c_size = 70, + .public_key_vec = true, + .siggen_sigver_test = true, + }, + /* Q = -G, d = n - 1, k = 2. */ + { + .key = + "\x04\x6b\x17\xd1\xf2\xe1\x2c\x42\x47\xf8\xbc\xe6\xe5\x63\xa4\x40" + "\xf2\x77\x03\x7d\x81\x2d\xeb\x33\xa0\xf4\xa1\x39\x45\xd8\x98\xc2" + "\x96\xb0\x1c\xbd\x1c\x01\xe5\x80\x65\x71\x18\x14\xb5\x83\xf0\x61" + "\xe9\xd4\x31\xcc\xa9\x94\xce\xa1\x31\x34\x49\xbf\x97\xc8\x40\xae" + "\x0a", + .key_len = 65, + .m = + "\x35\x9d\x17\x29\xb9\x1a\x52\xe9\x8b\xb5\x95\xc3\x86\x28\x3c\x10" + "\x30\x44\x95\x1a\xa1\x08\x20\xf0\xa0\x22\xc4\x18\xc2\x71\xa0\xcc", + .m_size = 32, + .c = + "\x30\x45\x02\x20\x7c\xf2\x7b\x18\x8d\x03\x4f\x7e\x8a\x52\x38\x03" + "\x04\xb5\x1a\xc3\xc0\x89\x69\xe2\x77\xf2\x1b\x35\xa6\x0b\x48\xfc" + "\x47\x66\x99\x78\x02\x21\x00\xdc\x55\x4e\x07\x96\x0b\x81\xb6\x80" + "\xb1\xae\xe0\x40\xb9\x90\xa5\xf4\xc4\x90\x49\xbb\xa2\xa1\x62\x70" + "\xc5\x88\x51\x39\xe8\xa8\xfb", + .c_size = 71, + .public_key_vec = true, + .siggen_sigver_test = true, + }, { .key = "\x04\xb9\x7b\xbb\xd7\x17\x64\xd2\x7e\xfc\x81\x5d\x87\x06\x83\x41" @@ -1218,7 +1359,8 @@ static const struct akcipher_testvec ecdsa_nist_p521_tv_template[] = { }; /* - * EC-RDSA test vectors are generated by gost-engine. + * EC-RDSA test vectors generated by gost-engine, followed by constructed + * exceptional-point regression vectors. */ static const struct akcipher_testvec ecrdsa_tv_template[] = { { @@ -1369,6 +1511,201 @@ static const struct akcipher_testvec ecrdsa_tv_template[] = { .public_key_vec = true, .siggen_sigver_test = true, }, + /* + * Constructed EC-RDSA exceptional-point vectors. + * With Q = dG, choose k = z1 + d*z2 (mod n), r = x(kG) mod n, + * e = -r/z2 (mod n), and s = r*d + k*e (mod n). + * The input digest is e, encoded little-endian. + */ + /* cp256a: Q=G, z1 = 3, z2 = 1. */ + { + .key = + "\x04\x40\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x14\x1e\x9f\x9e\x9c\xc9\xac\x22\xb1\xe3\x23\xdf\x2d\x4f" + "\x29\x35\x76\x2b\x3f\x45\x5a\x50\xdf\x27\xda\x9c\x98\xe0\x71\xe4" + "\x91\x8d", + .key_len = 66, + .params = /* OID_gostCPSignA */ + "\x30\x13\x06\x07\x2a\x85\x03\x02\x02\x23\x01\x06\x08\x2a\x85\x03" + "\x07\x01\x01\x02\x02", + .param_len = 21, + .c = + "\x1a\xed\x44\xae\xd4\x4a\xed\x44\xae\xd4\x4a\xed\x44\xae\xd4\x49" + "\x32\x67\xe0\x26\x16\xfd\xb7\xaf\xa4\xd7\x3e\x61\xd4\xf9\x7b\x94" + "\xf7\x06\x3e\x70\x63\xe7\x06\x3e\x70\x63\xe7\x06\x3e\x70\x63\xe7" + "\x06\x3e\x70\x63\xe7\x06\x3e\x70\x63\xe7\x06\x3e\x70\x63\xe4\xb7", + .c_size = 64, + .m = + "\xdc\xd3\xfd\x46\xcb\x14\x9d\xe1\x8f\x92\x54\xb2\x0c\xa0\x22\x66" + "\x18\x9c\x8f\xc1\xf9\x18\x9c\x8f\xc1\xf9\x18\x9c\x8f\xc1\xf9\x08", + .m_size = 32, + .algo = OID_gost2012PKey256, + .public_key_vec = true, + .siggen_sigver_test = true, + }, + /* cp256a: Q=-G, z1 = 7, z2 = 6. */ + { + .key = + "\x04\x40\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x83\xdf\x60\x61\x63\x36\x53\xdd\x4e\x1c\xdc\x20\xd2\xb0" + "\xd6\xca\x89\xd4\xc0\xba\xa5\xaf\x20\xd8\x25\x63\x67\x1f\x8e\x1b" + "\x6e\x72", + .key_len = 66, + .params = /* OID_gostCPSignA */ + "\x30\x13\x06\x07\x2a\x85\x03\x02\x02\x23\x01\x06\x08\x2a\x85\x03" + "\x07\x01\x01\x02\x02", + .param_len = 21, + .c = + "\x2a\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa" + "\x92\x10\x2d\x68\x19\x8f\x22\xd5\x60\xeb\x59\xd6\xf3\xe5\x9e\xc2" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01", + .c_size = 64, + .m = + "\xc3\x9e\xe5\xf3\xd6\x59\xeb\x60\xd5\x22\x8f\x19\x68\x2d\x10\x92" + "\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\x2a", + .m_size = 32, + .algo = OID_gost2012PKey256, + .public_key_vec = true, + .siggen_sigver_test = true, + }, + /* cp256a: Q=-2G, z1 = 5, z2 = 2. */ + { + .key = + "\x04\x40\x95\xfd\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\x14\x1e\x9f\x9e\x9c\xc9\xac\x22\xb1\xe3\x23\xdf\x2d\x4f" + "\x29\x35\x76\x2b\x3f\x45\x5a\x50\xdf\x27\xda\x9c\x98\xe0\x71\xe4" + "\x91\x8d", + .key_len = 66, + .params = /* OID_gostCPSignA */ + "\x30\x13\x06\x07\x2a\x85\x03\x02\x02\x23\x01\x06\x08\x2a\x85\x03" + "\x07\x01\x01\x02\x02", + .param_len = 21, + .c = + "\x7f\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff" + "\xb6\x30\x88\x38\x4c\xad\x68\x80\x22\xc2\x0d\x84\xdb\xb0\xdc\x47" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01", + .c_size = 64, + .m = + "\x49\xdc\xb0\xdb\x84\x0d\xc2\x22\x80\x68\xad\x4c\x38\x88\x30\xb6" + "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x7f", + .m_size = 32, + .algo = OID_gost2012PKey256, + .public_key_vec = true, + .siggen_sigver_test = true, + }, + /* tc512a: Q=G, z1 = 3, z2 = 1. */ + { + .key = + "\x04\x81\x80\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\xa4\xf2\x15\x52\xcb\x89\xa5\x89\xb8\xf5\x35\xc2\x5f" + "\xfe\x28\x80\xe9\x41\x3a\x0e\xa5\xe6\x75\x3d\xe9\x36\xd0\x4f\xbe" + "\x26\x16\xdf\x21\xa9\xef\xcb\xfd\x64\x80\x77\xc1\xab\xf1\xac\x93" + "\x1c\x5e\xce\xe6\x50\x54\xe2\x16\x88\x1b\xa6\xe3\x6a\x83\x7a\xe8" + "\xcf\x03\x75", + .key_len = 131, + .params = /* OID_gostTC26Sign512A */ + "\x30\x0b\x06\x09\x2a\x85\x03\x07\x01\x02\x01\x02\x01", + .param_len = 13, + .c = + "\xd8\xc0\xd7\xe3\xfb\xa6\xca\xff\x6a\xd1\xb4\xb4\x82\x1d\x15\x6f" + "\xa7\xc7\xbd\x63\x95\xb8\x29\x0d\xd4\xfa\xb1\x1c\x36\xbd\x32\x8d" + "\x1a\xba\xf1\x2e\xbc\x7b\xf2\x02\x96\xf8\xef\x2a\x5e\x72\x64\x86" + "\x67\x62\xfb\x62\x8e\x83\x53\x63\xb7\xe6\x78\x3c\x42\xd7\x0d\x1d" + "\xb7\xbf\xb8\x09\x56\xc8\x67\x00\x31\xba\x19\x19\x29\xf6\x4e\x30" + "\x1d\x68\x16\x34\x23\x6d\x47\xa6\x0e\x57\x1a\x4b\xed\xc0\xef\x25" + "\x74\x52\xef\x78\xb5\xb9\x8d\xbb\x3d\x9f\x31\x29\xd9\x34\x94\x33" + "\xce\x2a\x3a\x35\xcb\x51\x9c\x91\xe2\xd6\x33\xd7\xb3\x73\xae\x16", + .c_size = 128, + .m = + "\x5f\x04\x9d\x6b\x69\x7d\xf7\xe7\xcb\x1b\x81\x2f\x76\xfe\x20\xcd" + "\x2c\xcc\xd0\x74\x63\xfa\x52\x32\x56\xfb\xd3\x3e\xba\xa5\x93\xb3" + "\xd9\x10\x3f\x12\xb4\xe5\xa8\xf1\x59\xb8\x92\xdc\xcb\xe9\x97\xe2" + "\xcf\xb1\x09\xd6\xe6\xe6\x45\xce\xff\x98\x37\xa9\xf6\x47\x40\x48", + .m_size = 64, + .algo = OID_gost2012PKey512, + .public_key_vec = true, + .siggen_sigver_test = true, + }, + /* tc512a: Q=-G, z1 = 7, z2 = 6. */ + { + .key = + "\x04\x81\x80\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x23\x0b\xea\xad\x34\x76\x5a\x76\x47\x0a\xca\x3d\xa0" + "\x01\xd7\x7f\x16\xbe\xc5\xf1\x5a\x19\x8a\xc2\x16\xc9\x2f\xb0\x41" + "\xd9\xe9\x20\xde\x56\x10\x34\x02\x9b\x7f\x88\x3e\x54\x0e\x53\x6c" + "\xe3\xa1\x31\x19\xaf\xab\x1d\xe9\x77\xe4\x59\x1c\x95\x7c\x85\x17" + "\x30\xfc\x8a", + .key_len = 131, + .params = /* OID_gostTC26Sign512A */ + "\x30\x0b\x06\x09\x2a\x85\x03\x07\x01\x02\x01\x02\x01", + .param_len = 13, + .c = + "\x7f\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff" + "\x93\xf3\x4a\x99\x7a\x46\xc4\x88\xb7\xf9\x15\xc6\xa7\x02\xb0\x30" + "\x4d\xa5\x9c\x55\xfd\x69\x5c\x2e\xe5\x66\xd8\xa0\x8f\x88\x59\x37" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03", + .c_size = 128, + .m = + "\x3a\x59\x88\x8f\xa0\xd8\x66\xe5\x2e\x5c\x69\xfd\x55\x9c\xa5\x4d" + "\x30\xb0\x02\xa7\xc6\x15\xf9\xb7\x88\xc4\x46\x7a\x99\x4a\xf3\x93" + "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x7f", + .m_size = 64, + .algo = OID_gost2012PKey512, + .public_key_vec = true, + .siggen_sigver_test = true, + }, + /* tc512a: Q=-2G, z1 = 5, z2 = 2. */ + { + .key = + "\x04\x81\x80\x7c\x13\xea\xd2\xd9\xaf\x82\xb9\x1f\xb2\xfd\xe1\x8d" + "\xf4\x5b\x37\xe4\xf2\x04\x6a\x2a\x1d\x15\x16\x95\x2a\x5d\x40\xcd" + "\xbb\x34\x44\x2f\x2b\x13\x0a\x47\xb0\xf6\xe5\xa5\x58\x3a\xf4\x54" + "\x09\xb0\x1d\xf5\x5c\xf1\xbf\x9d\x86\xa5\x97\xab\x96\x29\x62\xfc" + "\xdc\x89\x3b\x76\x3d\x50\x6e\x8a\xcf\x02\x4c\x74\x0e\x6e\xe0\x1f" + "\x28\xab\x27\x9b\x20\x02\xc8\xc4\x12\xcb\x8a\xe3\xa4\x27\xce\x39" + "\x62\xdf\x35\x6d\xc1\x65\x7f\x7b\x48\x90\x11\x8f\x19\x7c\x1e\x67" + "\x91\x97\xeb\x90\x85\x25\xfc\x86\xb3\x88\x6e\x5c\x57\x1f\x1d\x1d" + "\x3b\xec\x37", + .key_len = 131, + .params = /* OID_gostTC26Sign512A */ + "\x30\x0b\x06\x09\x2a\x85\x03\x07\x01\x02\x01\x02\x01", + .param_len = 13, + .c = + "\x7f\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff" + "\x93\xf3\x4a\x99\x7a\x46\xc4\x88\xb7\xf9\x15\xc6\xa7\x02\xb0\x30" + "\x4d\xa5\x9c\x55\xfd\x69\x5c\x2e\xe5\x66\xd8\xa0\x8f\x88\x59\x33" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03", + .c_size = 128, + .m = + "\x39\x59\x88\x8f\xa0\xd8\x66\xe5\x2e\x5c\x69\xfd\x55\x9c\xa5\x4d" + "\x30\xb0\x02\xa7\xc6\x15\xf9\xb7\x88\xc4\x46\x7a\x99\x4a\xf3\x93" + "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff" + "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x7f", + .m_size = 64, + .algo = OID_gost2012PKey512, + .public_key_vec = true, + .siggen_sigver_test = true, + }, }; /* diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c index 03cf3d63e609..045acf13c131 100644 --- a/drivers/acpi/apei/ghes.c +++ b/drivers/acpi/apei/ghes.c @@ -583,7 +583,7 @@ static bool ghes_handle_arm_hw_error(struct acpi_hest_generic_data *gdata, * and don't filter out 'corrected' error here. */ if (is_cache && has_pa) { - queued = ghes_do_memory_failure(err_info->physical_fault_addr, flags); + queued |= ghes_do_memory_failure(err_info->physical_fault_addr, flags); p += err_info->length; continue; } @@ -809,11 +809,11 @@ static bool ghes_do_proc(struct ghes *ghes, atomic_notifier_call_chain(&ghes_report_chain, sev, mem_err); arch_apei_report_mem_error(sev, mem_err); - queued = ghes_handle_memory_failure(gdata, sev, sync); + queued |= ghes_handle_memory_failure(gdata, sev, sync); } else if (guid_equal(sec_type, &CPER_SEC_PCIE)) { ghes_handle_aer(gdata); } else if (guid_equal(sec_type, &CPER_SEC_PROC_ARM)) { - queued = ghes_handle_arm_hw_error(gdata, sev, sync); + queued |= ghes_handle_arm_hw_error(gdata, sev, sync); } else if (guid_equal(sec_type, &CPER_SEC_CXL_GEN_MEDIA_GUID)) { struct cxl_cper_event_rec *rec = acpi_hest_get_payload(gdata); diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c index 8ad6d12b7772..95ab801679e8 100644 --- a/drivers/acpi/cppc_acpi.c +++ b/drivers/acpi/cppc_acpi.c @@ -36,6 +36,7 @@ #include #include #include +#include #include #include #include @@ -94,6 +95,16 @@ static DEFINE_PER_CPU(int, cpu_pcc_subspace_idx); */ static DEFINE_PER_CPU(struct cpc_desc *, cpc_desc_ptr); +/* Protect immutable capability queries against descriptor removal. */ +static DEFINE_MUTEX(cpc_desc_lock); + +static void cpc_set_desc(unsigned int cpu, struct cpc_desc *desc) +{ + mutex_lock(&cpc_desc_lock); + per_cpu(cpc_desc_ptr, cpu) = desc; + mutex_unlock(&cpc_desc_lock); +} + /* pcc mapped address + header size + offset within PCC subspace */ #define GET_PCC_VADDR(offs, pcc_ss_id) (pcc_data[pcc_ss_id]->pcc_comm_addr + \ 0x8 + (offs)) @@ -910,12 +921,12 @@ int acpi_cppc_processor_probe(struct acpi_processor *pr) } /* Plug PSD data into this CPU's CPC descriptor. */ - per_cpu(cpc_desc_ptr, pr->id) = cpc_ptr; + cpc_set_desc(pr->id, cpc_ptr); ret = kobject_init_and_add(&cpc_ptr->kobj, &cppc_ktype, &cpu_dev->kobj, "acpi_cppc"); if (ret) { - per_cpu(cpc_desc_ptr, pr->id) = NULL; + cpc_set_desc(pr->id, NULL); kobject_put(&cpc_ptr->kobj); goto out_free; } @@ -967,6 +978,8 @@ void acpi_cppc_processor_exit(struct acpi_processor *pr) if (!cpc_ptr) return; + cpc_set_desc(pr->id, NULL); + /* Free all the mapped sys mem areas for this CPU */ for (i = 2; i < cpc_ptr->num_entries; i++) { addr = cpc_ptr->cpc_regs[i-2].sys_mem_vaddr; @@ -1028,6 +1041,9 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val) } *val = 0; + if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) + return cpc_read_ffh(cpu, reg, val); + size = GET_BIT_WIDTH(reg); if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_IO) { @@ -1054,8 +1070,6 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val) } else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY) vaddr = reg_res->sys_mem_vaddr; - else if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) - return cpc_read_ffh(cpu, reg, val); else return acpi_os_read_memory((acpi_physical_address)reg->address, val, size); @@ -1101,6 +1115,9 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val) struct cpc_desc *cpc_desc; unsigned long flags; + if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) + return cpc_write_ffh(cpu, reg, val); + size = GET_BIT_WIDTH(reg); if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_IO) { @@ -1125,8 +1142,6 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val) } else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY) vaddr = reg_res->sys_mem_vaddr; - else if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE) - return cpc_write_ffh(cpu, reg, val); else return acpi_os_write_memory((acpi_physical_address)reg->address, val, size); @@ -1276,6 +1291,11 @@ static int cppc_set_reg_val(int cpu, enum cppc_regs reg_idx, u64 val) reg = &cpc_desc->cpc_regs[reg_idx]; + /* Integer 1 describes autonomous selection that is always enabled. */ + if (reg_idx == AUTO_SEL_ENABLE && reg->type == ACPI_TYPE_INTEGER && + reg->cpc_entry.int_value == 1) + return val == 1 ? 0 : -EOPNOTSUPP; + /* if a register is writeable, it must be a buffer and not null */ if ((reg->type != ACPI_TYPE_BUFFER) || IS_NULL_REG(®->cpc_entry.reg)) { pr_debug("CPC register is not supported\n"); @@ -1632,6 +1652,32 @@ int cppc_get_auto_sel(int cpu, bool *enable) } EXPORT_SYMBOL_GPL(cppc_get_auto_sel); +/** + * cppc_auto_sel_is_immutable - Check for always-enabled autonomous selection. + * @cpu: CPU whose _CPC descriptor to check. + * + * Context: Process context. + * Return: true for Integer 1, false for a register or an absent descriptor. + */ +bool cppc_auto_sel_is_immutable(int cpu) +{ + struct cpc_desc *cpc_desc; + struct cpc_register_resource *reg; + bool immutable = false; + + mutex_lock(&cpc_desc_lock); + cpc_desc = per_cpu(cpc_desc_ptr, cpu); + if (cpc_desc) { + reg = &cpc_desc->cpc_regs[AUTO_SEL_ENABLE]; + immutable = reg->type == ACPI_TYPE_INTEGER && + reg->cpc_entry.int_value == 1; + } + mutex_unlock(&cpc_desc_lock); + + return immutable; +} +EXPORT_SYMBOL_GPL(cppc_auto_sel_is_immutable); + /** * cppc_set_auto_sel - Write autonomous selection register. * @cpu : CPU to which to write register. diff --git a/drivers/acpi/riscv/cppc.c b/drivers/acpi/riscv/cppc.c index 10a2b554b3cc..ef3b052f9dad 100644 --- a/drivers/acpi/riscv/cppc.c +++ b/drivers/acpi/riscv/cppc.c @@ -99,6 +99,7 @@ bool cpc_ffh_supported(void) int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val) { struct sbi_cppc_data data; + int ret; if (WARN_ON_ONCE(irqs_disabled())) return -EPERM; @@ -109,19 +110,27 @@ int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val) data.reg = FFH_CPPC_SBI_REG(reg->address); - smp_call_function_single(cpu, sbi_cppc_read, &data, 1); + ret = smp_call_function_single(cpu, sbi_cppc_read, &data, 1); + if (ret) + return ret; + if (data.ret.error) + return sbi_err_map_linux_errno(data.ret.error); *val = data.ret.value; - return (data.ret.error) ? sbi_err_map_linux_errno(data.ret.error) : 0; + return 0; } else if (FFH_CPPC_TYPE(reg->address) == FFH_CPPC_CSR) { data.reg = FFH_CPPC_CSR_NUM(reg->address); - smp_call_function_single(cpu, cppc_ffh_csr_read, &data, 1); + ret = smp_call_function_single(cpu, cppc_ffh_csr_read, &data, 1); + if (ret) + return ret; + if (data.ret.error) + return data.ret.error; *val = data.ret.value; - return data.ret.error; + return 0; } return -EINVAL; @@ -130,6 +139,7 @@ int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val) int cpc_write_ffh(int cpu, struct cpc_reg *reg, u64 val) { struct sbi_cppc_data data; + int ret; if (WARN_ON_ONCE(irqs_disabled())) return -EPERM; @@ -141,14 +151,18 @@ int cpc_write_ffh(int cpu, struct cpc_reg *reg, u64 val) data.reg = FFH_CPPC_SBI_REG(reg->address); data.val = val; - smp_call_function_single(cpu, sbi_cppc_write, &data, 1); + ret = smp_call_function_single(cpu, sbi_cppc_write, &data, 1); + if (ret) + return ret; return (data.ret.error) ? sbi_err_map_linux_errno(data.ret.error) : 0; } else if (FFH_CPPC_TYPE(reg->address) == FFH_CPPC_CSR) { data.reg = FFH_CPPC_CSR_NUM(reg->address); data.val = val; - smp_call_function_single(cpu, cppc_ffh_csr_write, &data, 1); + ret = smp_call_function_single(cpu, cppc_ffh_csr_write, &data, 1); + if (ret) + return ret; return data.ret.error; } diff --git a/drivers/ata/Kconfig b/drivers/ata/Kconfig index 120a2b7067fc..ff81f707d0b2 100644 --- a/drivers/ata/Kconfig +++ b/drivers/ata/Kconfig @@ -329,13 +329,6 @@ config SATA_AHCI_SEATTLE If unsure, say N -config SATA_INIC162X - tristate "Initio 162x SATA support (Very Experimental)" - depends on PCI - select SATA_HOST - help - This option enables support for Initio 162x Serial ATA. - config SATA_ACARD_AHCI tristate "ACard AHCI variant (ATP 8620)" depends on PCI diff --git a/drivers/ata/Makefile b/drivers/ata/Makefile index 20e6645ab737..81e04e96b807 100644 --- a/drivers/ata/Makefile +++ b/drivers/ata/Makefile @@ -9,7 +9,6 @@ obj-$(CONFIG_SATA_AHCI_SEATTLE) += ahci_seattle.o libahci.o libahci_platform.o obj-$(CONFIG_SATA_AHCI_PLATFORM) += ahci_platform.o libahci.o libahci_platform.o obj-$(CONFIG_SATA_FSL) += sata_fsl.o obj-$(CONFIG_SATA_GEMINI) += sata_gemini.o -obj-$(CONFIG_SATA_INIC162X) += sata_inic162x.o obj-$(CONFIG_SATA_SIL24) += sata_sil24.o obj-$(CONFIG_SATA_DWC) += sata_dwc_460ex.o obj-$(CONFIG_SATA_HIGHBANK) += sata_highbank.o libahci.o diff --git a/drivers/ata/ahci.c b/drivers/ata/ahci.c index 85b7bd605d7d..bd543a5ee8c4 100644 --- a/drivers/ata/ahci.c +++ b/drivers/ata/ahci.c @@ -111,17 +111,17 @@ static const struct scsi_host_template ahci_sht = { static struct ata_port_operations ahci_vt8251_ops = { .inherits = &ahci_ops, - .hardreset = ahci_vt8251_hardreset, + .reset.hardreset = ahci_vt8251_hardreset, }; static struct ata_port_operations ahci_p5wdh_ops = { .inherits = &ahci_ops, - .hardreset = ahci_p5wdh_hardreset, + .reset.hardreset = ahci_p5wdh_hardreset, }; static struct ata_port_operations ahci_avn_ops = { .inherits = &ahci_ops, - .hardreset = ahci_avn_hardreset, + .reset.hardreset = ahci_avn_hardreset, }; static const struct ata_port_info ahci_port_info[] = { diff --git a/drivers/ata/ahci_da850.c b/drivers/ata/ahci_da850.c index 55a6627d5450..2b331770ac52 100644 --- a/drivers/ata/ahci_da850.c +++ b/drivers/ata/ahci_da850.c @@ -137,13 +137,13 @@ static int ahci_da850_hardreset(struct ata_link *link, static struct ata_port_operations ahci_da850_port_ops = { .inherits = &ahci_platform_ops, - .softreset = ahci_da850_softreset, + .reset.softreset = ahci_da850_softreset, /* * No need to override .pmp_softreset - it's only used for actual * PMP-enabled ports. */ - .hardreset = ahci_da850_hardreset, - .pmp_hardreset = ahci_da850_hardreset, + .reset.hardreset = ahci_da850_hardreset, + .pmp_reset.hardreset = ahci_da850_hardreset, }; static const struct ata_port_info ahci_da850_port_info = { diff --git a/drivers/ata/ahci_dm816.c b/drivers/ata/ahci_dm816.c index 4cb70064fb99..188d45716d3a 100644 --- a/drivers/ata/ahci_dm816.c +++ b/drivers/ata/ahci_dm816.c @@ -124,7 +124,7 @@ static int ahci_dm816_softreset(struct ata_link *link, static struct ata_port_operations ahci_dm816_port_ops = { .inherits = &ahci_platform_ops, - .softreset = ahci_dm816_softreset, + .reset.softreset = ahci_dm816_softreset, }; static const struct ata_port_info ahci_dm816_port_info = { diff --git a/drivers/ata/ahci_imx.c b/drivers/ata/ahci_imx.c index 6f955e9105e8..91c72a58f91c 100644 --- a/drivers/ata/ahci_imx.c +++ b/drivers/ata/ahci_imx.c @@ -642,18 +642,19 @@ static int ahci_imx_softreset(struct ata_link *link, unsigned int *class, int ret; if (imxpriv->type == AHCI_IMX53) - ret = ahci_pmp_retry_srst_ops.softreset(link, class, deadline); + ret = ahci_pmp_retry_srst_ops.reset.softreset(link, class, + deadline); else - ret = ahci_ops.softreset(link, class, deadline); + ret = ahci_ops.reset.softreset(link, class, deadline); return ret; } static struct ata_port_operations ahci_imx_ops = { - .inherits = &ahci_ops, - .host_stop = ahci_imx_host_stop, - .error_handler = ahci_imx_error_handler, - .softreset = ahci_imx_softreset, + .inherits = &ahci_ops, + .host_stop = ahci_imx_host_stop, + .error_handler = ahci_imx_error_handler, + .reset.softreset = ahci_imx_softreset, }; static const struct ata_port_info ahci_imx_port_info = { diff --git a/drivers/ata/ahci_qoriq.c b/drivers/ata/ahci_qoriq.c index b1a4e57578e2..36e367a17d59 100644 --- a/drivers/ata/ahci_qoriq.c +++ b/drivers/ata/ahci_qoriq.c @@ -146,8 +146,8 @@ static int ahci_qoriq_hardreset(struct ata_link *link, unsigned int *class, } static struct ata_port_operations ahci_qoriq_ops = { - .inherits = &ahci_ops, - .hardreset = ahci_qoriq_hardreset, + .inherits = &ahci_ops, + .reset.hardreset = ahci_qoriq_hardreset, }; static const struct ata_port_info ahci_qoriq_port_info = { diff --git a/drivers/ata/ahci_xgene.c b/drivers/ata/ahci_xgene.c index 81a1d838c0fc..d51cd210be13 100644 --- a/drivers/ata/ahci_xgene.c +++ b/drivers/ata/ahci_xgene.c @@ -613,11 +613,11 @@ static irqreturn_t xgene_ahci_irq_intr(int irq, void *dev_instance) static struct ata_port_operations xgene_ahci_v1_ops = { .inherits = &ahci_ops, .host_stop = xgene_ahci_host_stop, - .hardreset = xgene_ahci_hardreset, + .reset.hardreset = xgene_ahci_hardreset, + .reset.softreset = xgene_ahci_softreset, + .pmp_reset.softreset = xgene_ahci_pmp_softreset, .read_id = xgene_ahci_read_id, .qc_issue = xgene_ahci_qc_issue, - .softreset = xgene_ahci_softreset, - .pmp_softreset = xgene_ahci_pmp_softreset }; static const struct ata_port_info xgene_ahci_v1_port_info = { @@ -630,7 +630,7 @@ static const struct ata_port_info xgene_ahci_v1_port_info = { static struct ata_port_operations xgene_ahci_v2_ops = { .inherits = &ahci_ops, .host_stop = xgene_ahci_host_stop, - .hardreset = xgene_ahci_hardreset, + .reset.hardreset = xgene_ahci_hardreset, .read_id = xgene_ahci_read_id, }; diff --git a/drivers/ata/ata_piix.c b/drivers/ata/ata_piix.c index d3cda803ae06..a78b662d3322 100644 --- a/drivers/ata/ata_piix.c +++ b/drivers/ata/ata_piix.c @@ -1074,7 +1074,7 @@ static struct ata_port_operations piix_pata_ops = { .cable_detect = ata_cable_40wire, .set_piomode = piix_set_piomode, .set_dmamode = piix_set_dmamode, - .prereset = piix_pata_prereset, + .reset.prereset = piix_pata_prereset, }; static struct ata_port_operations piix_vmw_ops = { @@ -1103,7 +1103,7 @@ static const struct scsi_host_template piix_sidpr_sht = { static struct ata_port_operations piix_sidpr_sata_ops = { .inherits = &piix_sata_ops, - .hardreset = sata_std_hardreset, + .reset.hardreset = sata_std_hardreset, .scr_read = piix_sidpr_scr_read, .scr_write = piix_sidpr_scr_write, .set_lpm = piix_sidpr_set_lpm, diff --git a/drivers/ata/libahci.c b/drivers/ata/libahci.c index 3fd193fcbd6e..cefb5013186e 100644 --- a/drivers/ata/libahci.c +++ b/drivers/ata/libahci.c @@ -163,10 +163,10 @@ struct ata_port_operations ahci_ops = { .freeze = ahci_freeze, .thaw = ahci_thaw, - .softreset = ahci_softreset, - .hardreset = ahci_hardreset, - .postreset = ahci_postreset, - .pmp_softreset = ahci_softreset, + .reset.softreset = ahci_softreset, + .reset.hardreset = ahci_hardreset, + .reset.postreset = ahci_postreset, + .pmp_reset.softreset = ahci_softreset, .error_handler = ahci_error_handler, .post_internal_cmd = ahci_post_internal_cmd, .dev_config = ahci_dev_config, @@ -193,7 +193,7 @@ EXPORT_SYMBOL_GPL(ahci_ops); struct ata_port_operations ahci_pmp_retry_srst_ops = { .inherits = &ahci_ops, - .softreset = ahci_pmp_retry_softreset, + .reset.softreset = ahci_pmp_retry_softreset, }; EXPORT_SYMBOL_GPL(ahci_pmp_retry_srst_ops); diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c index 0db6821cc3aa..65fb2cfc0e29 100644 --- a/drivers/ata/libata-core.c +++ b/drivers/ata/libata-core.c @@ -65,8 +65,8 @@ #include "libata-transport.h" const struct ata_port_operations ata_base_port_ops = { - .prereset = ata_std_prereset, - .postreset = ata_std_postreset, + .reset.prereset = ata_std_prereset, + .reset.postreset = ata_std_postreset, .error_handler = ata_std_error_handler, .sched_eh = ata_std_sched_eh, .end_eh = ata_std_end_eh, diff --git a/drivers/ata/libata-eh.c b/drivers/ata/libata-eh.c index 32ec1cfd648e..b78aa5371c14 100644 --- a/drivers/ata/libata-eh.c +++ b/drivers/ata/libata-eh.c @@ -2632,14 +2632,16 @@ static int ata_eh_followup_srst_needed(struct ata_link *link, int rc) return 0; } -int ata_eh_reset(struct ata_link *link, int classify, - ata_prereset_fn_t prereset, ata_reset_fn_t softreset, - ata_reset_fn_t hardreset, ata_postreset_fn_t postreset) +int ata_eh_reset(struct ata_port *ap, struct ata_link *link, int classify, + struct ata_reset_operations *reset_ops) { - struct ata_port *ap = link->ap; struct ata_link *slave = ap->slave_link; struct ata_eh_context *ehc = &link->eh_context; struct ata_eh_context *sehc = slave ? &slave->eh_context : NULL; + ata_reset_fn_t hardreset = reset_ops->hardreset; + ata_reset_fn_t softreset = reset_ops->softreset; + ata_prereset_fn_t prereset = reset_ops->prereset; + ata_postreset_fn_t postreset = reset_ops->postreset; unsigned int *classes = ehc->classes; unsigned int lflags = link->flags; int verbose = !(ehc->i.flags & ATA_EHI_QUIET); @@ -3248,12 +3250,12 @@ static int ata_eh_revalidate_and_attach(struct ata_link *link, } /** - * ata_set_mode - Program timings and issue SET FEATURES - XFER + * ata_eh_set_mode - Program timings and issue SET FEATURES - XFER * @link: link on which timings will be programmed * @r_failed_dev: out parameter for failed device * * Set ATA device disk transfer mode (PIO3, UDMA6, etc.). If - * ata_set_mode() fails, pointer to the failing device is + * ata_eh_set_mode() fails, pointer to the failing device is * returned in @r_failed_dev. * * LOCKING: @@ -3262,7 +3264,8 @@ static int ata_eh_revalidate_and_attach(struct ata_link *link, * RETURNS: * 0 on success, negative errno otherwise */ -int ata_set_mode(struct ata_link *link, struct ata_device **r_failed_dev) +static int ata_eh_set_mode(struct ata_link *link, + struct ata_device **r_failed_dev) { struct ata_port *ap = link->ap; struct ata_device *dev; @@ -3729,10 +3732,7 @@ static int ata_eh_handle_dev_fail(struct ata_device *dev, int err) /** * ata_eh_recover - recover host port after error * @ap: host port to recover - * @prereset: prereset method (can be NULL) - * @softreset: softreset method (can be NULL) - * @hardreset: hardreset method (can be NULL) - * @postreset: postreset method (can be NULL) + * @reset_ops: The set of reset operations to use * @r_failed_link: out parameter for failed link * * This is the alpha and omega, eum and yang, heart and soul of @@ -3748,9 +3748,7 @@ static int ata_eh_handle_dev_fail(struct ata_device *dev, int err) * RETURNS: * 0 on success, -errno on failure. */ -int ata_eh_recover(struct ata_port *ap, ata_prereset_fn_t prereset, - ata_reset_fn_t softreset, ata_reset_fn_t hardreset, - ata_postreset_fn_t postreset, +int ata_eh_recover(struct ata_port *ap, struct ata_reset_operations *reset_ops, struct ata_link **r_failed_link) { struct ata_link *link; @@ -3818,8 +3816,8 @@ int ata_eh_recover(struct ata_port *ap, ata_prereset_fn_t prereset, if (!(ehc->i.action & ATA_EH_RESET)) continue; - rc = ata_eh_reset(link, ata_link_nr_vacant(link), - prereset, softreset, hardreset, postreset); + rc = ata_eh_reset(ap, link, ata_link_nr_vacant(link), + reset_ops); if (rc) { ata_link_err(link, "reset failed, giving up\n"); goto out; @@ -3900,7 +3898,7 @@ int ata_eh_recover(struct ata_port *ap, ata_prereset_fn_t prereset, /* configure transfer mode if necessary */ if (ehc->i.flags & ATA_EHI_SETMODE) { - rc = ata_set_mode(link, &dev); + rc = ata_eh_set_mode(link, &dev); if (rc) goto rest_fail; ehc->i.flags &= ~ATA_EHI_SETMODE; @@ -4039,59 +4037,39 @@ void ata_eh_finish(struct ata_port *ap) } /** - * ata_do_eh - do standard error handling + * ata_std_error_handler - standard error handler * @ap: host port to handle error for * - * @prereset: prereset method (can be NULL) - * @softreset: softreset method (can be NULL) - * @hardreset: hardreset method (can be NULL) - * @postreset: postreset method (can be NULL) - * * Perform standard error handling sequence. * * LOCKING: * Kernel thread context (may sleep). */ -void ata_do_eh(struct ata_port *ap, ata_prereset_fn_t prereset, - ata_reset_fn_t softreset, ata_reset_fn_t hardreset, - ata_postreset_fn_t postreset) +void ata_std_error_handler(struct ata_port *ap) { - struct ata_device *dev; + struct ata_reset_operations *reset_ops = &ap->ops->reset; + struct ata_link *link = &ap->link; int rc; + /* Ignore built-in hardresets if SCR access is not available */ + if ((reset_ops->hardreset == sata_std_hardreset || + reset_ops->hardreset == sata_sff_hardreset) && + !sata_scr_valid(link)) + link->flags |= ATA_LFLAG_NO_HRST; + ata_eh_autopsy(ap); ata_eh_report(ap); - rc = ata_eh_recover(ap, prereset, softreset, hardreset, postreset, - NULL); + rc = ata_eh_recover(ap, reset_ops, NULL); if (rc) { - ata_for_each_dev(dev, &ap->link, ALL) + struct ata_device *dev; + + ata_for_each_dev(dev, link, ALL) ata_dev_disable(dev); } ata_eh_finish(ap); } - -/** - * ata_std_error_handler - standard error handler - * @ap: host port to handle error for - * - * Standard error handler - * - * LOCKING: - * Kernel thread context (may sleep). - */ -void ata_std_error_handler(struct ata_port *ap) -{ - struct ata_port_operations *ops = ap->ops; - ata_reset_fn_t hardreset = ops->hardreset; - - /* ignore built-in hardreset if SCR access is not available */ - if (hardreset == sata_std_hardreset && !sata_scr_valid(&ap->link)) - hardreset = NULL; - - ata_do_eh(ap, ops->prereset, ops->softreset, hardreset, ops->postreset); -} EXPORT_SYMBOL_GPL(ata_std_error_handler); #ifdef CONFIG_PM diff --git a/drivers/ata/libata-pmp.c b/drivers/ata/libata-pmp.c index 20dd3a5a818a..c3b3af28d17c 100644 --- a/drivers/ata/libata-pmp.c +++ b/drivers/ata/libata-pmp.c @@ -15,9 +15,9 @@ const struct ata_port_operations sata_pmp_port_ops = { .inherits = &sata_port_ops, - .pmp_prereset = ata_std_prereset, - .pmp_hardreset = sata_std_hardreset, - .pmp_postreset = ata_std_postreset, + .pmp_reset.prereset = ata_std_prereset, + .pmp_reset.hardreset = sata_std_hardreset, + .pmp_reset.postreset = ata_std_postreset, .error_handler = sata_pmp_error_handler, }; @@ -482,6 +482,20 @@ static void sata_pmp_quirks(struct ata_port *ap) if (link->pmp == 4) link->flags |= ATA_LFLAG_DISABLED; } + } else if (vendor == 0x1b4b && devid == 0x9705) { + /* + * Marvell 88SM9705 quirks + * + * 1-to-5 PMP with an extra SEMB port. It reports 6 ports and + * port 5 is the SEMB, which never answers SRST: EH spends about + * a minute per round retrying it, and disk discovery waits for + * that. Seen behind a Marvell 88SE9235 in the Synology DX1215 + * and DX1222. + */ + ata_for_each_link(link, ap, EDGE) { + if (link->pmp == 5) + link->flags |= ATA_LFLAG_DISABLED; + } } } @@ -746,10 +760,7 @@ static int sata_pmp_revalidate_quick(struct ata_device *dev) /** * sata_pmp_eh_recover_pmp - recover PMP * @ap: ATA port PMP is attached to - * @prereset: prereset method (can be NULL) - * @softreset: softreset method - * @hardreset: hardreset method - * @postreset: postreset method (can be NULL) + * @reset_ops: The set of reset operations to use * * Recover PMP attached to @ap. Recovery procedure is somewhat * similar to that of ata_eh_recover() except that reset should @@ -763,8 +774,7 @@ static int sata_pmp_revalidate_quick(struct ata_device *dev) * 0 on success, -errno on failure. */ static int sata_pmp_eh_recover_pmp(struct ata_port *ap, - ata_prereset_fn_t prereset, ata_reset_fn_t softreset, - ata_reset_fn_t hardreset, ata_postreset_fn_t postreset) + struct ata_reset_operations *reset_ops) { struct ata_link *link = &ap->link; struct ata_eh_context *ehc = &link->eh_context; @@ -786,8 +796,7 @@ static int sata_pmp_eh_recover_pmp(struct ata_port *ap, struct ata_link *tlink; /* reset */ - rc = ata_eh_reset(link, 0, prereset, softreset, hardreset, - postreset); + rc = ata_eh_reset(ap, link, 0, reset_ops); if (rc) { ata_link_err(link, "failed to reset PMP, giving up\n"); goto fail; @@ -951,8 +960,7 @@ static int sata_pmp_eh_recover(struct ata_port *ap) retry: /* PMP attached? */ if (!sata_pmp_attached(ap)) { - rc = ata_eh_recover(ap, ops->prereset, ops->softreset, - ops->hardreset, ops->postreset, NULL); + rc = ata_eh_recover(ap, &ops->reset, NULL); if (rc) { ata_for_each_dev(dev, &ap->link, ALL) ata_dev_disable(dev); @@ -970,8 +978,7 @@ static int sata_pmp_eh_recover(struct ata_port *ap) } /* recover pmp */ - rc = sata_pmp_eh_recover_pmp(ap, ops->prereset, ops->softreset, - ops->hardreset, ops->postreset); + rc = sata_pmp_eh_recover_pmp(ap, &ops->reset); if (rc) goto pmp_fail; @@ -997,8 +1004,7 @@ static int sata_pmp_eh_recover(struct ata_port *ap) goto pmp_fail; /* recover links */ - rc = ata_eh_recover(ap, ops->pmp_prereset, ops->pmp_softreset, - ops->pmp_hardreset, ops->pmp_postreset, &link); + rc = ata_eh_recover(ap, &ops->pmp_reset, &link); if (rc) goto link_fail; diff --git a/drivers/ata/libata-sata.c b/drivers/ata/libata-sata.c index fc64fae9e23f..0b6cfdfc2759 100644 --- a/drivers/ata/libata-sata.c +++ b/drivers/ata/libata-sata.c @@ -1735,6 +1735,6 @@ const struct ata_port_operations sata_port_ops = { .inherits = &ata_base_port_ops, .qc_defer = ata_std_qc_defer, - .hardreset = sata_std_hardreset, + .reset.hardreset = sata_std_hardreset, }; EXPORT_SYMBOL_GPL(sata_port_ops); diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c index 619819262414..0e309707f882 100644 --- a/drivers/ata/libata-scsi.c +++ b/drivers/ata/libata-scsi.c @@ -1989,6 +1989,18 @@ static int ata_scsi_translate(struct ata_device *dev, struct scsi_cmnd *cmd, if (xlat_func(qc)) goto done; + /* + * A data-out request may contain pages pinned without write access. + * Reject a translated data-in command before issuing it, since its + * completion handler may write to the buffer even if the device fails. + */ + if (unlikely(cmd->sc_data_direction == DMA_TO_DEVICE && + ata_is_data(qc->tf.protocol) && + !(qc->tf.flags & ATA_TFLAG_WRITE))) { + ata_scsi_set_sense(dev, cmd, ABORTED_COMMAND, 0x4b, 0x0a); + goto done; + } + return ata_scsi_qc_issue(ap, qc); done: @@ -2020,6 +2032,12 @@ static void ata_scsi_rbuf_fill(struct ata_device *dev, struct scsi_cmnd *cmd, unsigned long flags; unsigned int len; + /* A data-out request may contain pages pinned without write access. */ + if (cmd->sc_data_direction == DMA_TO_DEVICE) { + ata_scsi_set_sense(dev, cmd, ABORTED_COMMAND, 0x4b, 0x0a); + return; + } + spin_lock_irqsave(&ata_scsi_rbuf_lock, flags); memset(ata_scsi_rbuf, 0, ATA_SCSI_RBUF_SIZE); @@ -3077,6 +3095,13 @@ static unsigned int atapi_xlat(struct ata_queued_cmd *qc) qc->complete_fn = atapi_qc_complete; + /* atapi_fixup_inquiry() writes to the buffer on successful completion. */ + if (scmd->sc_data_direction == DMA_TO_DEVICE && + scmd->cmnd[0] == INQUIRY) { + ata_scsi_set_sense(dev, scmd, ABORTED_COMMAND, 0x4b, 0x0a); + return 1; + } + qc->tf.flags |= ATA_TFLAG_ISADDR | ATA_TFLAG_DEVICE; if (scmd->sc_data_direction == DMA_TO_DEVICE) { qc->tf.flags |= ATA_TFLAG_WRITE; diff --git a/drivers/ata/libata-sff.c b/drivers/ata/libata-sff.c index 83cb44ca5a97..95a3ddf36eff 100644 --- a/drivers/ata/libata-sff.c +++ b/drivers/ata/libata-sff.c @@ -31,10 +31,10 @@ const struct ata_port_operations ata_sff_port_ops = { .freeze = ata_sff_freeze, .thaw = ata_sff_thaw, - .prereset = ata_sff_prereset, - .softreset = ata_sff_softreset, - .hardreset = sata_sff_hardreset, - .postreset = ata_sff_postreset, + .reset.prereset = ata_sff_prereset, + .reset.softreset = ata_sff_softreset, + .reset.hardreset = sata_sff_hardreset, + .reset.postreset = ata_sff_postreset, .error_handler = ata_sff_error_handler, .sff_dev_select = ata_sff_dev_select, @@ -2054,8 +2054,6 @@ EXPORT_SYMBOL_GPL(ata_sff_drain_fifo); */ void ata_sff_error_handler(struct ata_port *ap) { - ata_reset_fn_t softreset = ap->ops->softreset; - ata_reset_fn_t hardreset = ap->ops->hardreset; struct ata_queued_cmd *qc; unsigned long flags; @@ -2077,13 +2075,7 @@ void ata_sff_error_handler(struct ata_port *ap) spin_unlock_irqrestore(ap->lock, flags); - /* ignore built-in hardresets if SCR access is not available */ - if ((hardreset == sata_std_hardreset || - hardreset == sata_sff_hardreset) && !sata_scr_valid(&ap->link)) - hardreset = NULL; - - ata_do_eh(ap, ap->ops->prereset, softreset, hardreset, - ap->ops->postreset); + ata_std_error_handler(ap); } EXPORT_SYMBOL_GPL(ata_sff_error_handler); diff --git a/drivers/ata/libata.h b/drivers/ata/libata.h index 4b3e246227e6..f98a3eb5d51e 100644 --- a/drivers/ata/libata.h +++ b/drivers/ata/libata.h @@ -196,13 +196,10 @@ extern void ata_eh_done(struct ata_link *link, struct ata_device *dev, extern void ata_eh_autopsy(struct ata_port *ap); const char *ata_get_cmd_name(u8 command); extern void ata_eh_report(struct ata_port *ap); -extern int ata_eh_reset(struct ata_link *link, int classify, - ata_prereset_fn_t prereset, ata_reset_fn_t softreset, - ata_reset_fn_t hardreset, ata_postreset_fn_t postreset); -extern int ata_set_mode(struct ata_link *link, struct ata_device **r_failed_dev); -extern int ata_eh_recover(struct ata_port *ap, ata_prereset_fn_t prereset, - ata_reset_fn_t softreset, ata_reset_fn_t hardreset, - ata_postreset_fn_t postreset, +extern int ata_eh_reset(struct ata_port *ap, struct ata_link *link, + int classify, struct ata_reset_operations *reset_ops); +extern int ata_eh_recover(struct ata_port *ap, + struct ata_reset_operations *reset_ops, struct ata_link **r_failed_disk); extern void ata_eh_finish(struct ata_port *ap); extern int ata_ering_map(struct ata_ering *ering, diff --git a/drivers/ata/pata_acpi.c b/drivers/ata/pata_acpi.c index ab38871b5e00..23fff10af2ac 100644 --- a/drivers/ata/pata_acpi.c +++ b/drivers/ata/pata_acpi.c @@ -216,7 +216,7 @@ static struct ata_port_operations pacpi_ops = { .mode_filter = pacpi_mode_filter, .set_piomode = pacpi_set_piomode, .set_dmamode = pacpi_set_dmamode, - .prereset = pacpi_pre_reset, + .reset.prereset = pacpi_pre_reset, .port_start = pacpi_port_start, }; diff --git a/drivers/ata/pata_ali.c b/drivers/ata/pata_ali.c index bb790edd6036..9d5cb9c34c52 100644 --- a/drivers/ata/pata_ali.c +++ b/drivers/ata/pata_ali.c @@ -392,11 +392,11 @@ static struct ata_port_operations ali_20_port_ops = { * Port operations for DMA capable ALi with cable detect */ static struct ata_port_operations ali_c2_port_ops = { - .inherits = &ali_dma_base_ops, - .check_atapi_dma = ali_check_atapi_dma, - .cable_detect = ali_c2_cable_detect, - .dev_config = ali_lock_sectors, - .postreset = ali_c2_c3_postreset, + .inherits = &ali_dma_base_ops, + .check_atapi_dma = ali_check_atapi_dma, + .cable_detect = ali_c2_cable_detect, + .dev_config = ali_lock_sectors, + .reset.postreset = ali_c2_c3_postreset, }; /* diff --git a/drivers/ata/pata_amd.c b/drivers/ata/pata_amd.c index 5b02b89748b7..a2fecadc927d 100644 --- a/drivers/ata/pata_amd.c +++ b/drivers/ata/pata_amd.c @@ -394,7 +394,7 @@ static const struct scsi_host_template amd_sht = { static const struct ata_port_operations amd_base_port_ops = { .inherits = &ata_bmdma32_port_ops, - .prereset = amd_pre_reset, + .reset.prereset = amd_pre_reset, }; static struct ata_port_operations amd33_port_ops = { @@ -429,7 +429,7 @@ static const struct ata_port_operations nv_base_port_ops = { .inherits = &ata_bmdma_port_ops, .cable_detect = ata_cable_ignore, .mode_filter = nv_mode_filter, - .prereset = nv_pre_reset, + .reset.prereset = nv_pre_reset, .host_stop = nv_host_stop, }; diff --git a/drivers/ata/pata_artop.c b/drivers/ata/pata_artop.c index 40544282f455..6160414172a3 100644 --- a/drivers/ata/pata_artop.c +++ b/drivers/ata/pata_artop.c @@ -301,7 +301,7 @@ static struct ata_port_operations artop6210_ops = { .cable_detect = ata_cable_40wire, .set_piomode = artop6210_set_piomode, .set_dmamode = artop6210_set_dmamode, - .prereset = artop62x0_pre_reset, + .reset.prereset = artop62x0_pre_reset, .qc_defer = artop6210_qc_defer, }; @@ -310,7 +310,7 @@ static struct ata_port_operations artop6260_ops = { .cable_detect = artop6260_cable_detect, .set_piomode = artop6260_set_piomode, .set_dmamode = artop6260_set_dmamode, - .prereset = artop62x0_pre_reset, + .reset.prereset = artop62x0_pre_reset, }; static void atp8xx_fixup(struct pci_dev *pdev) diff --git a/drivers/ata/pata_atiixp.c b/drivers/ata/pata_atiixp.c index 8c5cc803aab3..4c612f9543f6 100644 --- a/drivers/ata/pata_atiixp.c +++ b/drivers/ata/pata_atiixp.c @@ -264,7 +264,7 @@ static struct ata_port_operations atiixp_port_ops = { .bmdma_start = atiixp_bmdma_start, .bmdma_stop = atiixp_bmdma_stop, - .prereset = atiixp_prereset, + .reset.prereset = atiixp_prereset, .cable_detect = atiixp_cable_detect, .set_piomode = atiixp_set_piomode, .set_dmamode = atiixp_set_dmamode, diff --git a/drivers/ata/pata_efar.c b/drivers/ata/pata_efar.c index 2e6eccf2902f..6fe49b303fee 100644 --- a/drivers/ata/pata_efar.c +++ b/drivers/ata/pata_efar.c @@ -243,7 +243,7 @@ static struct ata_port_operations efar_ops = { .cable_detect = efar_cable_detect, .set_piomode = efar_set_piomode, .set_dmamode = efar_set_dmamode, - .prereset = efar_pre_reset, + .reset.prereset = efar_pre_reset, }; diff --git a/drivers/ata/pata_ep93xx.c b/drivers/ata/pata_ep93xx.c index 3ee975309791..f469c40a8916 100644 --- a/drivers/ata/pata_ep93xx.c +++ b/drivers/ata/pata_ep93xx.c @@ -891,8 +891,8 @@ static const struct scsi_host_template ep93xx_pata_sht = { static struct ata_port_operations ep93xx_pata_port_ops = { .inherits = &ata_bmdma_port_ops, - .softreset = ep93xx_pata_softreset, - .hardreset = ATA_OP_NULL, + .reset.softreset = ep93xx_pata_softreset, + .reset.hardreset = ATA_OP_NULL, .sff_dev_select = ep93xx_pata_dev_select, .sff_set_devctl = ep93xx_pata_set_devctl, diff --git a/drivers/ata/pata_hpt366.c b/drivers/ata/pata_hpt366.c index 5280e9960025..b96e8bd2a3f8 100644 --- a/drivers/ata/pata_hpt366.c +++ b/drivers/ata/pata_hpt366.c @@ -322,7 +322,7 @@ static const struct scsi_host_template hpt36x_sht = { static struct ata_port_operations hpt366_port_ops = { .inherits = &ata_bmdma_port_ops, - .prereset = hpt366_prereset, + .reset.prereset = hpt366_prereset, .cable_detect = hpt36x_cable_detect, .mode_filter = hpt366_filter, .set_piomode = hpt366_set_piomode, diff --git a/drivers/ata/pata_hpt37x.c b/drivers/ata/pata_hpt37x.c index 4af22b819416..07e3a984cbb1 100644 --- a/drivers/ata/pata_hpt37x.c +++ b/drivers/ata/pata_hpt37x.c @@ -543,7 +543,7 @@ static struct ata_port_operations hpt370_port_ops = { .cable_detect = hpt37x_cable_detect, .set_piomode = hpt37x_set_piomode, .set_dmamode = hpt37x_set_dmamode, - .prereset = hpt37x_pre_reset, + .reset.prereset = hpt37x_pre_reset, }; /* @@ -567,7 +567,7 @@ static struct ata_port_operations hpt302_port_ops = { .cable_detect = hpt37x_cable_detect, .set_piomode = hpt37x_set_piomode, .set_dmamode = hpt37x_set_dmamode, - .prereset = hpt37x_pre_reset, + .reset.prereset = hpt37x_pre_reset, }; /* diff --git a/drivers/ata/pata_hpt3x2n.c b/drivers/ata/pata_hpt3x2n.c index 5b1ecccf3c83..2cc57fcf2c46 100644 --- a/drivers/ata/pata_hpt3x2n.c +++ b/drivers/ata/pata_hpt3x2n.c @@ -356,7 +356,7 @@ static struct ata_port_operations hpt3xxn_port_ops = { .cable_detect = hpt3x2n_cable_detect, .set_piomode = hpt3x2n_set_piomode, .set_dmamode = hpt3x2n_set_dmamode, - .prereset = hpt3x2n_pre_reset, + .reset.prereset = hpt3x2n_pre_reset, }; /* diff --git a/drivers/ata/pata_icside.c b/drivers/ata/pata_icside.c index 61d8760f09d9..70f056e47e6b 100644 --- a/drivers/ata/pata_icside.c +++ b/drivers/ata/pata_icside.c @@ -336,7 +336,7 @@ static struct ata_port_operations pata_icside_port_ops = { .cable_detect = ata_cable_40wire, .set_dmamode = pata_icside_set_dmamode, - .postreset = pata_icside_postreset, + .reset.postreset = pata_icside_postreset, .port_start = ATA_OP_NULL, /* don't need PRD table */ }; diff --git a/drivers/ata/pata_it8213.c b/drivers/ata/pata_it8213.c index b7ac56103c8a..5ea75933cb8b 100644 --- a/drivers/ata/pata_it8213.c +++ b/drivers/ata/pata_it8213.c @@ -238,7 +238,7 @@ static struct ata_port_operations it8213_ops = { .cable_detect = it8213_cable_detect, .set_piomode = it8213_set_piomode, .set_dmamode = it8213_set_dmamode, - .prereset = it8213_pre_reset, + .reset.prereset = it8213_pre_reset, }; diff --git a/drivers/ata/pata_jmicron.c b/drivers/ata/pata_jmicron.c index f51fb8219762..b885f33e8980 100644 --- a/drivers/ata/pata_jmicron.c +++ b/drivers/ata/pata_jmicron.c @@ -113,7 +113,7 @@ static const struct scsi_host_template jmicron_sht = { static struct ata_port_operations jmicron_ops = { .inherits = &ata_bmdma_port_ops, - .prereset = jmicron_pre_reset, + .reset.prereset = jmicron_pre_reset, }; diff --git a/drivers/ata/pata_marvell.c b/drivers/ata/pata_marvell.c index 8119caaad605..deab67328388 100644 --- a/drivers/ata/pata_marvell.c +++ b/drivers/ata/pata_marvell.c @@ -99,7 +99,7 @@ static const struct scsi_host_template marvell_sht = { static struct ata_port_operations marvell_ops = { .inherits = &ata_bmdma_port_ops, .cable_detect = marvell_cable_detect, - .prereset = marvell_pre_reset, + .reset.prereset = marvell_pre_reset, }; diff --git a/drivers/ata/pata_mpiix.c b/drivers/ata/pata_mpiix.c index 69e4baf27d72..ce310ae7c93a 100644 --- a/drivers/ata/pata_mpiix.c +++ b/drivers/ata/pata_mpiix.c @@ -145,7 +145,7 @@ static struct ata_port_operations mpiix_port_ops = { .qc_issue = mpiix_qc_issue, .cable_detect = ata_cable_40wire, .set_piomode = mpiix_set_piomode, - .prereset = mpiix_pre_reset, + .reset.prereset = mpiix_pre_reset, .sff_data_xfer = ata_sff_data_xfer32, }; diff --git a/drivers/ata/pata_ns87410.c b/drivers/ata/pata_ns87410.c index 44cc24d21d5f..bdb55c1a3280 100644 --- a/drivers/ata/pata_ns87410.c +++ b/drivers/ata/pata_ns87410.c @@ -123,7 +123,7 @@ static struct ata_port_operations ns87410_port_ops = { .qc_issue = ns87410_qc_issue, .cable_detect = ata_cable_40wire, .set_piomode = ns87410_set_piomode, - .prereset = ns87410_pre_reset, + .reset.prereset = ns87410_pre_reset, }; static int ns87410_init_one(struct pci_dev *dev, const struct pci_device_id *id) diff --git a/drivers/ata/pata_octeon_cf.c b/drivers/ata/pata_octeon_cf.c index 0bb9607e7348..dc61a3004ba9 100644 --- a/drivers/ata/pata_octeon_cf.c +++ b/drivers/ata/pata_octeon_cf.c @@ -942,7 +942,7 @@ static int octeon_cf_probe(struct platform_device *pdev) /* 16 bit but not True IDE */ base = cs0 + 0x800; octeon_cf_ops.sff_data_xfer = octeon_cf_data_xfer16; - octeon_cf_ops.softreset = octeon_cf_softreset16; + octeon_cf_ops.reset.softreset = octeon_cf_softreset16; octeon_cf_ops.sff_check_status = octeon_cf_check_status16; octeon_cf_ops.sff_tf_read = octeon_cf_tf_read16; octeon_cf_ops.sff_tf_load = octeon_cf_tf_load16; diff --git a/drivers/ata/pata_oldpiix.c b/drivers/ata/pata_oldpiix.c index dca82d92b004..659f27aa3464 100644 --- a/drivers/ata/pata_oldpiix.c +++ b/drivers/ata/pata_oldpiix.c @@ -214,7 +214,7 @@ static struct ata_port_operations oldpiix_pata_ops = { .cable_detect = ata_cable_40wire, .set_piomode = oldpiix_set_piomode, .set_dmamode = oldpiix_set_dmamode, - .prereset = oldpiix_pre_reset, + .reset.prereset = oldpiix_pre_reset, }; diff --git a/drivers/ata/pata_opti.c b/drivers/ata/pata_opti.c index 3d23f57eb128..3db1b95d1404 100644 --- a/drivers/ata/pata_opti.c +++ b/drivers/ata/pata_opti.c @@ -156,7 +156,7 @@ static struct ata_port_operations opti_port_ops = { .inherits = &ata_sff_port_ops, .cable_detect = ata_cable_40wire, .set_piomode = opti_set_piomode, - .prereset = opti_pre_reset, + .reset.prereset = opti_pre_reset, }; static int opti_init_one(struct pci_dev *dev, const struct pci_device_id *id) diff --git a/drivers/ata/pata_optidma.c b/drivers/ata/pata_optidma.c index dfc36b4ec9c6..50124e205ece 100644 --- a/drivers/ata/pata_optidma.c +++ b/drivers/ata/pata_optidma.c @@ -344,7 +344,7 @@ static struct ata_port_operations optidma_port_ops = { .set_piomode = optidma_set_pio_mode, .set_dmamode = optidma_set_dma_mode, .set_mode = optidma_set_mode, - .prereset = optidma_pre_reset, + .reset.prereset = optidma_pre_reset, }; static struct ata_port_operations optiplus_port_ops = { diff --git a/drivers/ata/pata_parport/pata_parport.c b/drivers/ata/pata_parport/pata_parport.c index 93ebf566b54e..22bd3ff6b7ae 100644 --- a/drivers/ata/pata_parport/pata_parport.c +++ b/drivers/ata/pata_parport/pata_parport.c @@ -321,8 +321,8 @@ static void pata_parport_drain_fifo(struct ata_queued_cmd *qc) static struct ata_port_operations pata_parport_port_ops = { .inherits = &ata_sff_port_ops, - .softreset = pata_parport_softreset, - .hardreset = NULL, + .reset.softreset = pata_parport_softreset, + .reset.hardreset = NULL, .sff_dev_select = pata_parport_dev_select, .sff_set_devctl = pata_parport_set_devctl, diff --git a/drivers/ata/pata_pdc2027x.c b/drivers/ata/pata_pdc2027x.c index 6820c5597b14..29df7d501b94 100644 --- a/drivers/ata/pata_pdc2027x.c +++ b/drivers/ata/pata_pdc2027x.c @@ -130,7 +130,7 @@ static struct ata_port_operations pdc2027x_pata100_ops = { .inherits = &ata_bmdma_port_ops, .check_atapi_dma = pdc2027x_check_atapi_dma, .cable_detect = pdc2027x_cable_detect, - .prereset = pdc2027x_prereset, + .reset.prereset = pdc2027x_prereset, }; static struct ata_port_operations pdc2027x_pata133_ops = { diff --git a/drivers/ata/pata_rdc.c b/drivers/ata/pata_rdc.c index 0a9689862f71..6d8b276c979d 100644 --- a/drivers/ata/pata_rdc.c +++ b/drivers/ata/pata_rdc.c @@ -276,7 +276,7 @@ static struct ata_port_operations rdc_pata_ops = { .cable_detect = rdc_pata_cable_detect, .set_piomode = rdc_set_piomode, .set_dmamode = rdc_set_dmamode, - .prereset = rdc_pata_prereset, + .reset.prereset = rdc_pata_prereset, }; static const struct ata_port_info rdc_port_info = { diff --git a/drivers/ata/pata_sis.c b/drivers/ata/pata_sis.c index 31de06b66221..2b751e393771 100644 --- a/drivers/ata/pata_sis.c +++ b/drivers/ata/pata_sis.c @@ -552,7 +552,7 @@ static struct ata_port_operations sis_133_for_sata_ops = { static struct ata_port_operations sis_base_ops = { .inherits = &ata_bmdma_port_ops, - .prereset = sis_pre_reset, + .reset.prereset = sis_pre_reset, }; static struct ata_port_operations sis_133_ops = { diff --git a/drivers/ata/pata_sl82c105.c b/drivers/ata/pata_sl82c105.c index 2a5ce46af5a0..25bce3fe2a10 100644 --- a/drivers/ata/pata_sl82c105.c +++ b/drivers/ata/pata_sl82c105.c @@ -248,7 +248,7 @@ static struct ata_port_operations sl82c105_port_ops = { .bmdma_stop = sl82c105_bmdma_stop, .cable_detect = ata_cable_40wire, .set_piomode = sl82c105_set_piomode, - .prereset = sl82c105_pre_reset, + .reset.prereset = sl82c105_pre_reset, .sff_irq_check = sl82c105_sff_irq_check, }; diff --git a/drivers/ata/pata_triflex.c b/drivers/ata/pata_triflex.c index 26d448a869e2..596e86a031b3 100644 --- a/drivers/ata/pata_triflex.c +++ b/drivers/ata/pata_triflex.c @@ -170,7 +170,7 @@ static struct ata_port_operations triflex_port_ops = { .bmdma_stop = triflex_bmdma_stop, .cable_detect = ata_cable_40wire, .set_piomode = triflex_set_piomode, - .prereset = triflex_prereset, + .reset.prereset = triflex_prereset, }; static int triflex_init_one(struct pci_dev *dev, const struct pci_device_id *id) diff --git a/drivers/ata/pata_via.c b/drivers/ata/pata_via.c index bb80e7800dcb..a8c9cf685b4b 100644 --- a/drivers/ata/pata_via.c +++ b/drivers/ata/pata_via.c @@ -451,7 +451,7 @@ static struct ata_port_operations via_port_ops = { .cable_detect = via_cable_detect, .set_piomode = via_set_piomode, .set_dmamode = via_set_dmamode, - .prereset = via_pre_reset, + .reset.prereset = via_pre_reset, .sff_tf_load = via_tf_load, .port_start = via_port_start, .mode_filter = via_mode_filter, diff --git a/drivers/ata/pdc_adma.c b/drivers/ata/pdc_adma.c index 8e6b2599f0d5..17a5a59861c3 100644 --- a/drivers/ata/pdc_adma.c +++ b/drivers/ata/pdc_adma.c @@ -140,7 +140,7 @@ static struct ata_port_operations adma_ata_ops = { .freeze = adma_freeze, .thaw = adma_thaw, - .prereset = adma_prereset, + .reset.prereset = adma_prereset, .port_start = adma_port_start, .port_stop = adma_port_stop, diff --git a/drivers/ata/sata_dwc_460ex.c b/drivers/ata/sata_dwc_460ex.c index 66db8dd8ec26..23753c4ae1b5 100644 --- a/drivers/ata/sata_dwc_460ex.c +++ b/drivers/ata/sata_dwc_460ex.c @@ -1087,7 +1087,7 @@ static struct ata_port_operations sata_dwc_ops = { .inherits = &ata_sff_port_ops, .error_handler = sata_dwc_error_handler, - .hardreset = sata_dwc_hardreset, + .reset.hardreset = sata_dwc_hardreset, .qc_issue = sata_dwc_qc_issue, diff --git a/drivers/ata/sata_fsl.c b/drivers/ata/sata_fsl.c index c64c9d99cd6c..56247db5d1d2 100644 --- a/drivers/ata/sata_fsl.c +++ b/drivers/ata/sata_fsl.c @@ -1395,9 +1395,9 @@ static struct ata_port_operations sata_fsl_ops = { .freeze = sata_fsl_freeze, .thaw = sata_fsl_thaw, - .softreset = sata_fsl_softreset, - .hardreset = sata_fsl_hardreset, - .pmp_softreset = sata_fsl_softreset, + .reset.softreset = sata_fsl_softreset, + .reset.hardreset = sata_fsl_hardreset, + .pmp_reset.softreset = sata_fsl_softreset, .error_handler = sata_fsl_error_handler, .post_internal_cmd = sata_fsl_post_internal_cmd, diff --git a/drivers/ata/sata_highbank.c b/drivers/ata/sata_highbank.c index 596c6d294da9..6349026ed027 100644 --- a/drivers/ata/sata_highbank.c +++ b/drivers/ata/sata_highbank.c @@ -428,7 +428,7 @@ static int ahci_highbank_hardreset(struct ata_link *link, unsigned int *class, static struct ata_port_operations ahci_highbank_ops = { .inherits = &ahci_ops, - .hardreset = ahci_highbank_hardreset, + .reset.hardreset = ahci_highbank_hardreset, .transmit_led_message = ecx_transmit_led_message, }; diff --git a/drivers/ata/sata_inic162x.c b/drivers/ata/sata_inic162x.c deleted file mode 100644 index db9c255dc9f2..000000000000 --- a/drivers/ata/sata_inic162x.c +++ /dev/null @@ -1,902 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * sata_inic162x.c - Driver for Initio 162x SATA controllers - * - * Copyright 2006 SUSE Linux Products GmbH - * Copyright 2006 Tejun Heo - * - * **** WARNING **** - * - * This driver never worked properly and unfortunately data corruption is - * relatively common. There isn't anyone working on the driver and there's - * no support from the vendor. Do not use this driver in any production - * environment. - * - * http://thread.gmane.org/gmane.linux.debian.devel.bugs.rc/378525/focus=54491 - * https://bugzilla.kernel.org/show_bug.cgi?id=60565 - * - * ***************** - * - * This controller is eccentric and easily locks up if something isn't - * right. Documentation is available at initio's website but it only - * documents registers (not programming model). - * - * This driver has interesting history. The first version was written - * from the documentation and a 2.4 IDE driver posted on a Taiwan - * company, which didn't use any IDMA features and couldn't handle - * LBA48. The resulting driver couldn't handle LBA48 devices either - * making it pretty useless. - * - * After a while, initio picked the driver up, renamed it to - * sata_initio162x, updated it to use IDMA for ATA DMA commands and - * posted it on their website. It only used ATA_PROT_DMA for IDMA and - * attaching both devices and issuing IDMA and !IDMA commands - * simultaneously broke it due to PIRQ masking interaction but it did - * show how to use the IDMA (ADMA + some initio specific twists) - * engine. - * - * Then, I picked up their changes again and here's the usable driver - * which uses IDMA for everything. Everything works now including - * LBA48, CD/DVD burning, suspend/resume and hotplug. There are some - * issues tho. Result Tf is not resported properly, NCQ isn't - * supported yet and CD/DVD writing works with DMA assisted PIO - * protocol (which, for native SATA devices, shouldn't cause any - * noticeable difference). - * - * Anyways, so, here's finally a working driver for inic162x. Enjoy! - * - * initio: If you guys wanna improve the driver regarding result TF - * access and other stuff, please feel free to contact me. I'll be - * happy to assist. - */ - -#include -#include -#include -#include -#include -#include -#include -#include - -#define DRV_NAME "sata_inic162x" -#define DRV_VERSION "0.4" - -enum { - MMIO_BAR_PCI = 5, - MMIO_BAR_CARDBUS = 1, - - NR_PORTS = 2, - - IDMA_CPB_TBL_SIZE = 4 * 32, - - INIC_DMA_BOUNDARY = 0xffffff, - - HOST_ACTRL = 0x08, - HOST_CTL = 0x7c, - HOST_STAT = 0x7e, - HOST_IRQ_STAT = 0xbc, - HOST_IRQ_MASK = 0xbe, - - PORT_SIZE = 0x40, - - /* registers for ATA TF operation */ - PORT_TF_DATA = 0x00, - PORT_TF_FEATURE = 0x01, - PORT_TF_NSECT = 0x02, - PORT_TF_LBAL = 0x03, - PORT_TF_LBAM = 0x04, - PORT_TF_LBAH = 0x05, - PORT_TF_DEVICE = 0x06, - PORT_TF_COMMAND = 0x07, - PORT_TF_ALT_STAT = 0x08, - PORT_IRQ_STAT = 0x09, - PORT_IRQ_MASK = 0x0a, - PORT_PRD_CTL = 0x0b, - PORT_PRD_ADDR = 0x0c, - PORT_PRD_XFERLEN = 0x10, - PORT_CPB_CPBLAR = 0x18, - PORT_CPB_PTQFIFO = 0x1c, - - /* IDMA register */ - PORT_IDMA_CTL = 0x14, - PORT_IDMA_STAT = 0x16, - - PORT_RPQ_FIFO = 0x1e, - PORT_RPQ_CNT = 0x1f, - - PORT_SCR = 0x20, - - /* HOST_CTL bits */ - HCTL_LEDEN = (1 << 3), /* enable LED operation */ - HCTL_IRQOFF = (1 << 8), /* global IRQ off */ - HCTL_FTHD0 = (1 << 10), /* fifo threshold 0 */ - HCTL_FTHD1 = (1 << 11), /* fifo threshold 1*/ - HCTL_PWRDWN = (1 << 12), /* power down PHYs */ - HCTL_SOFTRST = (1 << 13), /* global reset (no phy reset) */ - HCTL_RPGSEL = (1 << 15), /* register page select */ - - HCTL_KNOWN_BITS = HCTL_IRQOFF | HCTL_PWRDWN | HCTL_SOFTRST | - HCTL_RPGSEL, - - /* HOST_IRQ_(STAT|MASK) bits */ - HIRQ_PORT0 = (1 << 0), - HIRQ_PORT1 = (1 << 1), - HIRQ_SOFT = (1 << 14), - HIRQ_GLOBAL = (1 << 15), /* STAT only */ - - /* PORT_IRQ_(STAT|MASK) bits */ - PIRQ_OFFLINE = (1 << 0), /* device unplugged */ - PIRQ_ONLINE = (1 << 1), /* device plugged */ - PIRQ_COMPLETE = (1 << 2), /* completion interrupt */ - PIRQ_FATAL = (1 << 3), /* fatal error */ - PIRQ_ATA = (1 << 4), /* ATA interrupt */ - PIRQ_REPLY = (1 << 5), /* reply FIFO not empty */ - PIRQ_PENDING = (1 << 7), /* port IRQ pending (STAT only) */ - - PIRQ_ERR = PIRQ_OFFLINE | PIRQ_ONLINE | PIRQ_FATAL, - PIRQ_MASK_DEFAULT = PIRQ_REPLY | PIRQ_ATA, - PIRQ_MASK_FREEZE = 0xff, - - /* PORT_PRD_CTL bits */ - PRD_CTL_START = (1 << 0), - PRD_CTL_WR = (1 << 3), - PRD_CTL_DMAEN = (1 << 7), /* DMA enable */ - - /* PORT_IDMA_CTL bits */ - IDMA_CTL_RST_ATA = (1 << 2), /* hardreset ATA bus */ - IDMA_CTL_RST_IDMA = (1 << 5), /* reset IDMA machinery */ - IDMA_CTL_GO = (1 << 7), /* IDMA mode go */ - IDMA_CTL_ATA_NIEN = (1 << 8), /* ATA IRQ disable */ - - /* PORT_IDMA_STAT bits */ - IDMA_STAT_PERR = (1 << 0), /* PCI ERROR MODE */ - IDMA_STAT_CPBERR = (1 << 1), /* ADMA CPB error */ - IDMA_STAT_LGCY = (1 << 3), /* ADMA legacy */ - IDMA_STAT_UIRQ = (1 << 4), /* ADMA unsolicited irq */ - IDMA_STAT_STPD = (1 << 5), /* ADMA stopped */ - IDMA_STAT_PSD = (1 << 6), /* ADMA pause */ - IDMA_STAT_DONE = (1 << 7), /* ADMA done */ - - IDMA_STAT_ERR = IDMA_STAT_PERR | IDMA_STAT_CPBERR, - - /* CPB Control Flags*/ - CPB_CTL_VALID = (1 << 0), /* CPB valid */ - CPB_CTL_QUEUED = (1 << 1), /* queued command */ - CPB_CTL_DATA = (1 << 2), /* data, rsvd in datasheet */ - CPB_CTL_IEN = (1 << 3), /* PCI interrupt enable */ - CPB_CTL_DEVDIR = (1 << 4), /* device direction control */ - - /* CPB Response Flags */ - CPB_RESP_DONE = (1 << 0), /* ATA command complete */ - CPB_RESP_REL = (1 << 1), /* ATA release */ - CPB_RESP_IGNORED = (1 << 2), /* CPB ignored */ - CPB_RESP_ATA_ERR = (1 << 3), /* ATA command error */ - CPB_RESP_SPURIOUS = (1 << 4), /* ATA spurious interrupt error */ - CPB_RESP_UNDERFLOW = (1 << 5), /* APRD deficiency length error */ - CPB_RESP_OVERFLOW = (1 << 6), /* APRD exccess length error */ - CPB_RESP_CPB_ERR = (1 << 7), /* CPB error flag */ - - /* PRD Control Flags */ - PRD_DRAIN = (1 << 1), /* ignore data excess */ - PRD_CDB = (1 << 2), /* atapi packet command pointer */ - PRD_DIRECT_INTR = (1 << 3), /* direct interrupt */ - PRD_DMA = (1 << 4), /* data transfer method */ - PRD_WRITE = (1 << 5), /* data dir, rsvd in datasheet */ - PRD_IOM = (1 << 6), /* io/memory transfer */ - PRD_END = (1 << 7), /* APRD chain end */ -}; - -/* Comman Parameter Block */ -struct inic_cpb { - u8 resp_flags; /* Response Flags */ - u8 error; /* ATA Error */ - u8 status; /* ATA Status */ - u8 ctl_flags; /* Control Flags */ - __le32 len; /* Total Transfer Length */ - __le32 prd; /* First PRD pointer */ - u8 rsvd[4]; - /* 16 bytes */ - u8 feature; /* ATA Feature */ - u8 hob_feature; /* ATA Ex. Feature */ - u8 device; /* ATA Device/Head */ - u8 mirctl; /* Mirror Control */ - u8 nsect; /* ATA Sector Count */ - u8 hob_nsect; /* ATA Ex. Sector Count */ - u8 lbal; /* ATA Sector Number */ - u8 hob_lbal; /* ATA Ex. Sector Number */ - u8 lbam; /* ATA Cylinder Low */ - u8 hob_lbam; /* ATA Ex. Cylinder Low */ - u8 lbah; /* ATA Cylinder High */ - u8 hob_lbah; /* ATA Ex. Cylinder High */ - u8 command; /* ATA Command */ - u8 ctl; /* ATA Control */ - u8 slave_error; /* Slave ATA Error */ - u8 slave_status; /* Slave ATA Status */ - /* 32 bytes */ -} __packed; - -/* Physical Region Descriptor */ -struct inic_prd { - __le32 mad; /* Physical Memory Address */ - __le16 len; /* Transfer Length */ - u8 rsvd; - u8 flags; /* Control Flags */ -} __packed; - -struct inic_pkt { - struct inic_cpb cpb; - struct inic_prd prd[LIBATA_MAX_PRD + 1]; /* + 1 for cdb */ - u8 cdb[ATAPI_CDB_LEN]; -} __packed; - -struct inic_host_priv { - void __iomem *mmio_base; - u16 cached_hctl; -}; - -struct inic_port_priv { - struct inic_pkt *pkt; - dma_addr_t pkt_dma; - u32 *cpb_tbl; - dma_addr_t cpb_tbl_dma; -}; - -static const struct scsi_host_template inic_sht = { - ATA_BASE_SHT(DRV_NAME), - .sg_tablesize = LIBATA_MAX_PRD, /* maybe it can be larger? */ - - /* - * This controller is braindamaged. dma_boundary is 0xffff like others - * but it will lock up the whole machine HARD if 65536 byte PRD entry - * is fed. Reduce maximum segment size. - */ - .dma_boundary = INIC_DMA_BOUNDARY, - .max_segment_size = 65536 - 512, -}; - -static const int scr_map[] = { - [SCR_STATUS] = 0, - [SCR_ERROR] = 1, - [SCR_CONTROL] = 2, -}; - -static void __iomem *inic_port_base(struct ata_port *ap) -{ - struct inic_host_priv *hpriv = ap->host->private_data; - - return hpriv->mmio_base + ap->port_no * PORT_SIZE; -} - -static void inic_reset_port(void __iomem *port_base) -{ - void __iomem *idma_ctl = port_base + PORT_IDMA_CTL; - - /* stop IDMA engine */ - readw(idma_ctl); /* flush */ - msleep(1); - - /* mask IRQ and assert reset */ - writew(IDMA_CTL_RST_IDMA, idma_ctl); - readw(idma_ctl); /* flush */ - msleep(1); - - /* release reset */ - writew(0, idma_ctl); - - /* clear irq */ - writeb(0xff, port_base + PORT_IRQ_STAT); -} - -static int inic_scr_read(struct ata_link *link, unsigned sc_reg, u32 *val) -{ - void __iomem *scr_addr = inic_port_base(link->ap) + PORT_SCR; - - if (unlikely(sc_reg >= ARRAY_SIZE(scr_map))) - return -EINVAL; - - *val = readl(scr_addr + scr_map[sc_reg] * 4); - - /* this controller has stuck DIAG.N, ignore it */ - if (sc_reg == SCR_ERROR) - *val &= ~SERR_PHYRDY_CHG; - return 0; -} - -static int inic_scr_write(struct ata_link *link, unsigned sc_reg, u32 val) -{ - void __iomem *scr_addr = inic_port_base(link->ap) + PORT_SCR; - - if (unlikely(sc_reg >= ARRAY_SIZE(scr_map))) - return -EINVAL; - - writel(val, scr_addr + scr_map[sc_reg] * 4); - return 0; -} - -static void inic_stop_idma(struct ata_port *ap) -{ - void __iomem *port_base = inic_port_base(ap); - - readb(port_base + PORT_RPQ_FIFO); - readb(port_base + PORT_RPQ_CNT); - writew(0, port_base + PORT_IDMA_CTL); -} - -static void inic_host_err_intr(struct ata_port *ap, u8 irq_stat, u16 idma_stat) -{ - struct ata_eh_info *ehi = &ap->link.eh_info; - struct inic_port_priv *pp = ap->private_data; - struct inic_cpb *cpb = &pp->pkt->cpb; - bool freeze = false; - - ata_ehi_clear_desc(ehi); - ata_ehi_push_desc(ehi, "irq_stat=0x%x idma_stat=0x%x", - irq_stat, idma_stat); - - inic_stop_idma(ap); - - if (irq_stat & (PIRQ_OFFLINE | PIRQ_ONLINE)) { - ata_ehi_push_desc(ehi, "hotplug"); - ata_ehi_hotplugged(ehi); - freeze = true; - } - - if (idma_stat & IDMA_STAT_PERR) { - ata_ehi_push_desc(ehi, "PCI error"); - freeze = true; - } - - if (idma_stat & IDMA_STAT_CPBERR) { - ata_ehi_push_desc(ehi, "CPB error"); - - if (cpb->resp_flags & CPB_RESP_IGNORED) { - __ata_ehi_push_desc(ehi, " ignored"); - ehi->err_mask |= AC_ERR_INVALID; - freeze = true; - } - - if (cpb->resp_flags & CPB_RESP_ATA_ERR) - ehi->err_mask |= AC_ERR_DEV; - - if (cpb->resp_flags & CPB_RESP_SPURIOUS) { - __ata_ehi_push_desc(ehi, " spurious-intr"); - ehi->err_mask |= AC_ERR_HSM; - freeze = true; - } - - if (cpb->resp_flags & - (CPB_RESP_UNDERFLOW | CPB_RESP_OVERFLOW)) { - __ata_ehi_push_desc(ehi, " data-over/underflow"); - ehi->err_mask |= AC_ERR_HSM; - freeze = true; - } - } - - if (freeze) - ata_port_freeze(ap); - else - ata_port_abort(ap); -} - -static void inic_host_intr(struct ata_port *ap) -{ - void __iomem *port_base = inic_port_base(ap); - struct ata_queued_cmd *qc = ata_qc_from_tag(ap, ap->link.active_tag); - u8 irq_stat; - u16 idma_stat; - - /* read and clear IRQ status */ - irq_stat = readb(port_base + PORT_IRQ_STAT); - writeb(irq_stat, port_base + PORT_IRQ_STAT); - idma_stat = readw(port_base + PORT_IDMA_STAT); - - if (unlikely((irq_stat & PIRQ_ERR) || (idma_stat & IDMA_STAT_ERR))) - inic_host_err_intr(ap, irq_stat, idma_stat); - - if (unlikely(!qc)) - goto spurious; - - if (likely(idma_stat & IDMA_STAT_DONE)) { - inic_stop_idma(ap); - - /* Depending on circumstances, device error - * isn't reported by IDMA, check it explicitly. - */ - if (unlikely(readb(port_base + PORT_TF_COMMAND) & - (ATA_DF | ATA_ERR))) - qc->err_mask |= AC_ERR_DEV; - - ata_qc_complete(qc); - return; - } - - spurious: - ata_port_warn(ap, "unhandled interrupt: cmd=0x%x irq_stat=0x%x idma_stat=0x%x\n", - qc ? qc->tf.command : 0xff, irq_stat, idma_stat); -} - -static irqreturn_t inic_interrupt(int irq, void *dev_instance) -{ - struct ata_host *host = dev_instance; - struct inic_host_priv *hpriv = host->private_data; - u16 host_irq_stat; - int i, handled = 0; - - host_irq_stat = readw(hpriv->mmio_base + HOST_IRQ_STAT); - - if (unlikely(!(host_irq_stat & HIRQ_GLOBAL))) - goto out; - - spin_lock(&host->lock); - - for (i = 0; i < NR_PORTS; i++) - if (host_irq_stat & (HIRQ_PORT0 << i)) { - inic_host_intr(host->ports[i]); - handled++; - } - - spin_unlock(&host->lock); - - out: - return IRQ_RETVAL(handled); -} - -static int inic_check_atapi_dma(struct ata_queued_cmd *qc) -{ - /* For some reason ATAPI_PROT_DMA doesn't work for some - * commands including writes and other misc ops. Use PIO - * protocol instead, which BTW is driven by the DMA engine - * anyway, so it shouldn't make much difference for native - * SATA devices. - */ - if (atapi_cmd_type(qc->cdb[0]) == READ) - return 0; - return 1; -} - -static void inic_fill_sg(struct inic_prd *prd, struct ata_queued_cmd *qc) -{ - struct scatterlist *sg; - unsigned int si; - u8 flags = 0; - - if (qc->tf.flags & ATA_TFLAG_WRITE) - flags |= PRD_WRITE; - - if (ata_is_dma(qc->tf.protocol)) - flags |= PRD_DMA; - - for_each_sg(qc->sg, sg, qc->n_elem, si) { - prd->mad = cpu_to_le32(sg_dma_address(sg)); - prd->len = cpu_to_le16(sg_dma_len(sg)); - prd->flags = flags; - prd++; - } - - WARN_ON(!si); - prd[-1].flags |= PRD_END; -} - -static enum ata_completion_errors inic_qc_prep(struct ata_queued_cmd *qc) -{ - struct inic_port_priv *pp = qc->ap->private_data; - struct inic_pkt *pkt = pp->pkt; - struct inic_cpb *cpb = &pkt->cpb; - struct inic_prd *prd = pkt->prd; - bool is_atapi = ata_is_atapi(qc->tf.protocol); - bool is_data = ata_is_data(qc->tf.protocol); - unsigned int cdb_len = 0; - - if (is_atapi) - cdb_len = qc->dev->cdb_len; - - /* prepare packet, based on initio driver */ - memset(pkt, 0, sizeof(struct inic_pkt)); - - cpb->ctl_flags = CPB_CTL_VALID | CPB_CTL_IEN; - if (is_atapi || is_data) - cpb->ctl_flags |= CPB_CTL_DATA; - - cpb->len = cpu_to_le32(qc->nbytes + cdb_len); - cpb->prd = cpu_to_le32(pp->pkt_dma + offsetof(struct inic_pkt, prd)); - - cpb->device = qc->tf.device; - cpb->feature = qc->tf.feature; - cpb->nsect = qc->tf.nsect; - cpb->lbal = qc->tf.lbal; - cpb->lbam = qc->tf.lbam; - cpb->lbah = qc->tf.lbah; - - if (qc->tf.flags & ATA_TFLAG_LBA48) { - cpb->hob_feature = qc->tf.hob_feature; - cpb->hob_nsect = qc->tf.hob_nsect; - cpb->hob_lbal = qc->tf.hob_lbal; - cpb->hob_lbam = qc->tf.hob_lbam; - cpb->hob_lbah = qc->tf.hob_lbah; - } - - cpb->command = qc->tf.command; - /* don't load ctl - dunno why. it's like that in the initio driver */ - - /* setup PRD for CDB */ - if (is_atapi) { - memcpy(pkt->cdb, qc->cdb, ATAPI_CDB_LEN); - prd->mad = cpu_to_le32(pp->pkt_dma + - offsetof(struct inic_pkt, cdb)); - prd->len = cpu_to_le16(cdb_len); - prd->flags = PRD_CDB | PRD_WRITE; - if (!is_data) - prd->flags |= PRD_END; - prd++; - } - - /* setup sg table */ - if (is_data) - inic_fill_sg(prd, qc); - - pp->cpb_tbl[0] = pp->pkt_dma; - - return AC_ERR_OK; -} - -static unsigned int inic_qc_issue(struct ata_queued_cmd *qc) -{ - struct ata_port *ap = qc->ap; - void __iomem *port_base = inic_port_base(ap); - - /* fire up the ADMA engine */ - writew(HCTL_FTHD0 | HCTL_LEDEN, port_base + HOST_CTL); - writew(IDMA_CTL_GO, port_base + PORT_IDMA_CTL); - writeb(0, port_base + PORT_CPB_PTQFIFO); - - return 0; -} - -static void inic_tf_read(struct ata_port *ap, struct ata_taskfile *tf) -{ - void __iomem *port_base = inic_port_base(ap); - - tf->error = readb(port_base + PORT_TF_FEATURE); - tf->nsect = readb(port_base + PORT_TF_NSECT); - tf->lbal = readb(port_base + PORT_TF_LBAL); - tf->lbam = readb(port_base + PORT_TF_LBAM); - tf->lbah = readb(port_base + PORT_TF_LBAH); - tf->device = readb(port_base + PORT_TF_DEVICE); - tf->status = readb(port_base + PORT_TF_COMMAND); -} - -static void inic_qc_fill_rtf(struct ata_queued_cmd *qc) -{ - struct ata_taskfile *rtf = &qc->result_tf; - struct ata_taskfile tf; - - /* FIXME: Except for status and error, result TF access - * doesn't work. I tried reading from BAR0/2, CPB and BAR5. - * None works regardless of which command interface is used. - * For now return true iff status indicates device error. - * This means that we're reporting bogus sector for RW - * failures. Eeekk.... - */ - inic_tf_read(qc->ap, &tf); - - if (tf.status & ATA_ERR) { - rtf->status = tf.status; - rtf->error = tf.error; - } -} - -static void inic_freeze(struct ata_port *ap) -{ - void __iomem *port_base = inic_port_base(ap); - - writeb(PIRQ_MASK_FREEZE, port_base + PORT_IRQ_MASK); - writeb(0xff, port_base + PORT_IRQ_STAT); -} - -static void inic_thaw(struct ata_port *ap) -{ - void __iomem *port_base = inic_port_base(ap); - - writeb(0xff, port_base + PORT_IRQ_STAT); - writeb(PIRQ_MASK_DEFAULT, port_base + PORT_IRQ_MASK); -} - -static int inic_check_ready(struct ata_link *link) -{ - void __iomem *port_base = inic_port_base(link->ap); - - return ata_check_ready(readb(port_base + PORT_TF_COMMAND)); -} - -/* - * SRST and SControl hardreset don't give valid signature on this - * controller. Only controller specific hardreset mechanism works. - */ -static int inic_hardreset(struct ata_link *link, unsigned int *class, - unsigned long deadline) -{ - struct ata_port *ap = link->ap; - void __iomem *port_base = inic_port_base(ap); - void __iomem *idma_ctl = port_base + PORT_IDMA_CTL; - const unsigned int *timing = sata_ehc_deb_timing(&link->eh_context); - int rc; - - /* hammer it into sane state */ - inic_reset_port(port_base); - - writew(IDMA_CTL_RST_ATA, idma_ctl); - readw(idma_ctl); /* flush */ - ata_msleep(ap, 1); - writew(0, idma_ctl); - - rc = sata_link_resume(link, timing, deadline); - if (rc) { - ata_link_warn(link, - "failed to resume link after reset (errno=%d)\n", - rc); - return rc; - } - - *class = ATA_DEV_NONE; - if (ata_link_online(link)) { - struct ata_taskfile tf; - - /* wait for link to become ready */ - rc = ata_wait_after_reset(link, deadline, inic_check_ready); - /* link occupied, -ENODEV too is an error */ - if (rc) { - ata_link_warn(link, - "device not ready after hardreset (errno=%d)\n", - rc); - return rc; - } - - inic_tf_read(ap, &tf); - *class = ata_port_classify(ap, &tf); - } - - return 0; -} - -static void inic_error_handler(struct ata_port *ap) -{ - void __iomem *port_base = inic_port_base(ap); - - inic_reset_port(port_base); - ata_std_error_handler(ap); -} - -static void inic_post_internal_cmd(struct ata_queued_cmd *qc) -{ - /* make DMA engine forget about the failed command */ - if (qc->flags & ATA_QCFLAG_EH) - inic_reset_port(inic_port_base(qc->ap)); -} - -static void init_port(struct ata_port *ap) -{ - void __iomem *port_base = inic_port_base(ap); - struct inic_port_priv *pp = ap->private_data; - - /* clear packet and CPB table */ - memset(pp->pkt, 0, sizeof(struct inic_pkt)); - memset(pp->cpb_tbl, 0, IDMA_CPB_TBL_SIZE); - - /* setup CPB lookup table addresses */ - writel(pp->cpb_tbl_dma, port_base + PORT_CPB_CPBLAR); -} - -static int inic_port_resume(struct ata_port *ap) -{ - init_port(ap); - return 0; -} - -static int inic_port_start(struct ata_port *ap) -{ - struct device *dev = ap->host->dev; - struct inic_port_priv *pp; - - /* alloc and initialize private data */ - pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL); - if (!pp) - return -ENOMEM; - ap->private_data = pp; - - /* Alloc resources */ - pp->pkt = dmam_alloc_coherent(dev, sizeof(struct inic_pkt), - &pp->pkt_dma, GFP_KERNEL); - if (!pp->pkt) - return -ENOMEM; - - pp->cpb_tbl = dmam_alloc_coherent(dev, IDMA_CPB_TBL_SIZE, - &pp->cpb_tbl_dma, GFP_KERNEL); - if (!pp->cpb_tbl) - return -ENOMEM; - - init_port(ap); - - return 0; -} - -static struct ata_port_operations inic_port_ops = { - .inherits = &sata_port_ops, - - .check_atapi_dma = inic_check_atapi_dma, - .qc_prep = inic_qc_prep, - .qc_issue = inic_qc_issue, - .qc_fill_rtf = inic_qc_fill_rtf, - - .freeze = inic_freeze, - .thaw = inic_thaw, - .hardreset = inic_hardreset, - .error_handler = inic_error_handler, - .post_internal_cmd = inic_post_internal_cmd, - - .scr_read = inic_scr_read, - .scr_write = inic_scr_write, - - .port_resume = inic_port_resume, - .port_start = inic_port_start, -}; - -static const struct ata_port_info inic_port_info = { - .flags = ATA_FLAG_SATA | ATA_FLAG_PIO_DMA, - .pio_mask = ATA_PIO4, - .mwdma_mask = ATA_MWDMA2, - .udma_mask = ATA_UDMA6, - .port_ops = &inic_port_ops -}; - -static int init_controller(void __iomem *mmio_base, u16 hctl) -{ - int i; - u16 val; - - hctl &= ~HCTL_KNOWN_BITS; - - /* Soft reset whole controller. Spec says reset duration is 3 - * PCI clocks, be generous and give it 10ms. - */ - writew(hctl | HCTL_SOFTRST, mmio_base + HOST_CTL); - readw(mmio_base + HOST_CTL); /* flush */ - - for (i = 0; i < 10; i++) { - msleep(1); - val = readw(mmio_base + HOST_CTL); - if (!(val & HCTL_SOFTRST)) - break; - } - - if (val & HCTL_SOFTRST) - return -EIO; - - /* mask all interrupts and reset ports */ - for (i = 0; i < NR_PORTS; i++) { - void __iomem *port_base = mmio_base + i * PORT_SIZE; - - writeb(0xff, port_base + PORT_IRQ_MASK); - inic_reset_port(port_base); - } - - /* port IRQ is masked now, unmask global IRQ */ - writew(hctl & ~HCTL_IRQOFF, mmio_base + HOST_CTL); - val = readw(mmio_base + HOST_IRQ_MASK); - val &= ~(HIRQ_PORT0 | HIRQ_PORT1); - writew(val, mmio_base + HOST_IRQ_MASK); - - return 0; -} - -#ifdef CONFIG_PM_SLEEP -static int inic_pci_device_resume(struct pci_dev *pdev) -{ - struct ata_host *host = pci_get_drvdata(pdev); - struct inic_host_priv *hpriv = host->private_data; - int rc; - - rc = ata_pci_device_do_resume(pdev); - if (rc) - return rc; - - if (pdev->dev.power.power_state.event == PM_EVENT_SUSPEND) { - rc = init_controller(hpriv->mmio_base, hpriv->cached_hctl); - if (rc) - return rc; - } - - ata_host_resume(host); - - return 0; -} -#endif - -static int inic_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) -{ - const struct ata_port_info *ppi[] = { &inic_port_info, NULL }; - struct ata_host *host; - struct inic_host_priv *hpriv; - void __iomem * const *iomap; - int mmio_bar; - int i, rc; - - ata_print_version_once(&pdev->dev, DRV_VERSION); - - dev_alert(&pdev->dev, "inic162x support is broken with common data corruption issues and will be disabled by default, contact linux-ide@vger.kernel.org if in production use\n"); - - /* alloc host */ - host = ata_host_alloc_pinfo(&pdev->dev, ppi, NR_PORTS); - hpriv = devm_kzalloc(&pdev->dev, sizeof(*hpriv), GFP_KERNEL); - if (!host || !hpriv) - return -ENOMEM; - - host->private_data = hpriv; - - /* Acquire resources and fill host. Note that PCI and cardbus - * use different BARs. - */ - rc = pcim_enable_device(pdev); - if (rc) - return rc; - - if (pci_resource_flags(pdev, MMIO_BAR_PCI) & IORESOURCE_MEM) - mmio_bar = MMIO_BAR_PCI; - else - mmio_bar = MMIO_BAR_CARDBUS; - - rc = pcim_iomap_regions(pdev, 1 << mmio_bar, DRV_NAME); - if (rc) - return rc; - host->iomap = iomap = pcim_iomap_table(pdev); - hpriv->mmio_base = iomap[mmio_bar]; - hpriv->cached_hctl = readw(hpriv->mmio_base + HOST_CTL); - - for (i = 0; i < NR_PORTS; i++) { - struct ata_port *ap = host->ports[i]; - - ata_port_pbar_desc(ap, mmio_bar, -1, "mmio"); - ata_port_pbar_desc(ap, mmio_bar, i * PORT_SIZE, "port"); - } - - /* Set dma_mask. This devices doesn't support 64bit addressing. */ - rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); - if (rc) { - dev_err(&pdev->dev, "32-bit DMA enable failed\n"); - return rc; - } - - rc = init_controller(hpriv->mmio_base, hpriv->cached_hctl); - if (rc) { - dev_err(&pdev->dev, "failed to initialize controller\n"); - return rc; - } - - pci_set_master(pdev); - return ata_host_activate(host, pdev->irq, inic_interrupt, IRQF_SHARED, - &inic_sht); -} - -static const struct pci_device_id inic_pci_tbl[] = { - { PCI_VDEVICE(INIT, 0x1622), }, - { }, -}; - -static struct pci_driver inic_pci_driver = { - .name = DRV_NAME, - .id_table = inic_pci_tbl, -#ifdef CONFIG_PM_SLEEP - .suspend = ata_pci_device_suspend, - .resume = inic_pci_device_resume, -#endif - .probe = inic_init_one, - .remove = ata_pci_remove_one, -}; - -module_pci_driver(inic_pci_driver); - -MODULE_AUTHOR("Tejun Heo"); -MODULE_DESCRIPTION("low-level driver for Initio 162x SATA"); -MODULE_LICENSE("GPL v2"); -MODULE_DEVICE_TABLE(pci, inic_pci_tbl); -MODULE_VERSION(DRV_VERSION); diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c index 40fdc422940d..7e1bd5610c3c 100644 --- a/drivers/ata/sata_mv.c +++ b/drivers/ata/sata_mv.c @@ -687,7 +687,7 @@ static struct ata_port_operations mv5_ops = { .freeze = mv_eh_freeze, .thaw = mv_eh_thaw, - .hardreset = mv_hardreset, + .reset.hardreset = mv_hardreset, .scr_read = mv5_scr_read, .scr_write = mv5_scr_write, @@ -709,10 +709,10 @@ static struct ata_port_operations mv6_ops = { .freeze = mv_eh_freeze, .thaw = mv_eh_thaw, - .hardreset = mv_hardreset, - .softreset = mv_softreset, - .pmp_hardreset = mv_pmp_hardreset, - .pmp_softreset = mv_softreset, + .reset.hardreset = mv_hardreset, + .reset.softreset = mv_softreset, + .pmp_reset.hardreset = mv_pmp_hardreset, + .pmp_reset.softreset = mv_softreset, .error_handler = mv_pmp_error_handler, .scr_read = mv_scr_read, diff --git a/drivers/ata/sata_nv.c b/drivers/ata/sata_nv.c index 36d99043ef50..d1e6e80d0103 100644 --- a/drivers/ata/sata_nv.c +++ b/drivers/ata/sata_nv.c @@ -462,7 +462,7 @@ static struct ata_port_operations nv_generic_ops = { .lost_interrupt = ATA_OP_NULL, .scr_read = nv_scr_read, .scr_write = nv_scr_write, - .hardreset = nv_hardreset, + .reset.hardreset = nv_hardreset, }; static struct ata_port_operations nv_nf2_ops = { diff --git a/drivers/ata/sata_promise.c b/drivers/ata/sata_promise.c index 2df1a070b25a..2a005aede123 100644 --- a/drivers/ata/sata_promise.c +++ b/drivers/ata/sata_promise.c @@ -188,7 +188,7 @@ static struct ata_port_operations pdc_sata_ops = { .scr_read = pdc_sata_scr_read, .scr_write = pdc_sata_scr_write, .port_start = pdc_sata_port_start, - .hardreset = pdc_sata_hardreset, + .reset.hardreset = pdc_sata_hardreset, }; /* First-generation chips need a more restrictive ->check_atapi_dma op, @@ -206,7 +206,7 @@ static struct ata_port_operations pdc_pata_ops = { .freeze = pdc_freeze, .thaw = pdc_thaw, .port_start = pdc_common_port_start, - .softreset = pdc_pata_softreset, + .reset.softreset = pdc_pata_softreset, }; static const struct ata_port_info pdc_port_info[] = { diff --git a/drivers/ata/sata_qstor.c b/drivers/ata/sata_qstor.c index 90289560e7cd..b2be1ce87878 100644 --- a/drivers/ata/sata_qstor.c +++ b/drivers/ata/sata_qstor.c @@ -123,8 +123,8 @@ static struct ata_port_operations qs_ata_ops = { .freeze = qs_freeze, .thaw = qs_thaw, - .prereset = qs_prereset, - .softreset = ATA_OP_NULL, + .reset.prereset = qs_prereset, + .reset.softreset = ATA_OP_NULL, .error_handler = qs_error_handler, .lost_interrupt = ATA_OP_NULL, diff --git a/drivers/ata/sata_rcar.c b/drivers/ata/sata_rcar.c index c1469d076880..146a231f8fe4 100644 --- a/drivers/ata/sata_rcar.c +++ b/drivers/ata/sata_rcar.c @@ -624,7 +624,7 @@ static struct ata_port_operations sata_rcar_port_ops = { .freeze = sata_rcar_freeze, .thaw = sata_rcar_thaw, - .softreset = sata_rcar_softreset, + .reset.softreset = sata_rcar_softreset, .scr_read = sata_rcar_scr_read, .scr_write = sata_rcar_scr_write, diff --git a/drivers/ata/sata_sil24.c b/drivers/ata/sata_sil24.c index ca24c294bf46..92bb693c32d7 100644 --- a/drivers/ata/sata_sil24.c +++ b/drivers/ata/sata_sil24.c @@ -394,10 +394,10 @@ static struct ata_port_operations sil24_ops = { .freeze = sil24_freeze, .thaw = sil24_thaw, - .softreset = sil24_softreset, - .hardreset = sil24_hardreset, - .pmp_softreset = sil24_softreset, - .pmp_hardreset = sil24_pmp_hardreset, + .reset.softreset = sil24_softreset, + .reset.hardreset = sil24_hardreset, + .pmp_reset.softreset = sil24_softreset, + .pmp_reset.hardreset = sil24_pmp_hardreset, .error_handler = sil24_error_handler, .post_internal_cmd = sil24_post_internal_cmd, .dev_config = sil24_dev_config, diff --git a/drivers/ata/sata_svw.c b/drivers/ata/sata_svw.c index 598a872f6a08..c5d6aa36c9c3 100644 --- a/drivers/ata/sata_svw.c +++ b/drivers/ata/sata_svw.c @@ -340,8 +340,8 @@ static const struct scsi_host_template k2_sata_sht = { static struct ata_port_operations k2_sata_ops = { .inherits = &ata_bmdma_port_ops, - .softreset = k2_sata_softreset, - .hardreset = k2_sata_hardreset, + .reset.softreset = k2_sata_softreset, + .reset.hardreset = k2_sata_hardreset, .sff_tf_load = k2_sata_tf_load, .sff_tf_read = k2_sata_tf_read, .sff_check_status = k2_stat_check_status, diff --git a/drivers/ata/sata_sx4.c b/drivers/ata/sata_sx4.c index c3042eca6332..1c2ba6d30c2d 100644 --- a/drivers/ata/sata_sx4.c +++ b/drivers/ata/sata_sx4.c @@ -241,7 +241,7 @@ static struct ata_port_operations pdc_20621_ops = { .freeze = pdc_freeze, .thaw = pdc_thaw, - .softreset = pdc_softreset, + .reset.softreset = pdc_softreset, .error_handler = pdc_error_handler, .lost_interrupt = ATA_OP_NULL, .post_internal_cmd = pdc_post_internal_cmd, diff --git a/drivers/ata/sata_uli.c b/drivers/ata/sata_uli.c index 60ea45926cd1..6ad4c00688df 100644 --- a/drivers/ata/sata_uli.c +++ b/drivers/ata/sata_uli.c @@ -67,7 +67,7 @@ static struct ata_port_operations uli_ops = { .inherits = &ata_bmdma_port_ops, .scr_read = uli_scr_read, .scr_write = uli_scr_write, - .hardreset = ATA_OP_NULL, + .reset.hardreset = ATA_OP_NULL, }; static const struct ata_port_info uli_port_info = { diff --git a/drivers/ata/sata_via.c b/drivers/ata/sata_via.c index 57cbf2cef618..59fb2c1ebcb0 100644 --- a/drivers/ata/sata_via.c +++ b/drivers/ata/sata_via.c @@ -119,7 +119,7 @@ static struct ata_port_operations svia_base_ops = { static struct ata_port_operations vt6420_sata_ops = { .inherits = &svia_base_ops, .freeze = svia_noop_freeze, - .prereset = vt6420_prereset, + .reset.prereset = vt6420_prereset, .bmdma_start = vt6420_bmdma_start, }; @@ -139,7 +139,7 @@ static struct ata_port_operations vt6421_sata_ops = { static struct ata_port_operations vt8251_ops = { .inherits = &svia_base_ops, - .hardreset = sata_std_hardreset, + .reset.hardreset = sata_std_hardreset, .scr_read = vt8251_scr_read, .scr_write = vt8251_scr_write, }; diff --git a/drivers/block/amiflop.c b/drivers/block/amiflop.c index 49ced65bef4c..7a53ee1d18d8 100644 --- a/drivers/block/amiflop.c +++ b/drivers/block/amiflop.c @@ -265,6 +265,8 @@ static struct fd_data_type data_types[] = { static struct amiga_floppy_struct unit[FD_MAX_UNITS]; static struct timer_list flush_track_timer[FD_MAX_UNITS]; +static DEFINE_SPINLOCK(flush_track_timer_lock); +static bool flush_track_timer_stopped[FD_MAX_UNITS]; static struct timer_list post_write_timer; static unsigned long post_write_timer_drive; static struct timer_list motor_on_timer; @@ -1370,11 +1372,22 @@ static void flush_track_callback(struct timer_list *timer) sizeof(flush_track_timer[0]); nr&=3; + spin_lock(&flush_track_timer_lock); + if (flush_track_timer_stopped[nr]) { + spin_unlock(&flush_track_timer_lock); + return; + } + spin_unlock(&flush_track_timer_lock); + writefromint = 1; if (!try_fdc(nr)) { /* we might block in an interrupt, so try again later */ - flush_track_timer[nr].expires = jiffies + 1; - add_timer(flush_track_timer + nr); + spin_lock(&flush_track_timer_lock); + if (!flush_track_timer_stopped[nr]) { + flush_track_timer[nr].expires = jiffies + 1; + add_timer(flush_track_timer + nr); + } + spin_unlock(&flush_track_timer_lock); return; } get_fdc(nr); @@ -1421,6 +1434,35 @@ static int non_int_flush_track (unsigned long nr) return 1; } +static void flush_track_timer_stop(unsigned int drive) +{ + unsigned long flags; + + spin_lock_irqsave(&flush_track_timer_lock, flags); + flush_track_timer_stopped[drive] = true; + spin_unlock_irqrestore(&flush_track_timer_lock, flags); + + timer_delete_sync(flush_track_timer + drive); +} + +static void flush_track_timer_start(unsigned int drive) +{ + unsigned long flags; + + spin_lock_irqsave(&flush_track_timer_lock, flags); + flush_track_timer_stopped[drive] = false; + if (unit[drive].dirty) + mod_timer(flush_track_timer + drive, jiffies + 1); + spin_unlock_irqrestore(&flush_track_timer_lock, flags); +} + +static void flush_track_timer_sync(unsigned int drive) +{ + flush_track_timer_stop(drive); + non_int_flush_track(drive); + flush_track_timer_start(drive); +} + static int get_track(int drive, int track) { int error, errcnt; @@ -1435,8 +1477,7 @@ static int get_track(int drive, int track) } if (unit[drive].dirty == 1) { - del_timer (flush_track_timer + drive); - non_int_flush_track (drive); + flush_track_timer_sync(drive); } errcnt = 0; while (errcnt < MAX_ERRORS) { @@ -1562,6 +1603,12 @@ static int fd_locked_ioctl(struct block_device *bdev, blk_mode_t mode, case FDFMTTRK: if (param < p->type->tracks * p->type->heads) { + flush_track_timer_stop(drive); + if (unit[drive].dirty == 1 && + !non_int_flush_track(drive)) { + flush_track_timer_start(drive); + return -EIO; + } get_fdc(drive); if (fd_seek(drive,param) != 0){ memset(p->trackbuf, FD_FILL_BYTE, @@ -1570,6 +1617,7 @@ static int fd_locked_ioctl(struct block_device *bdev, blk_mode_t mode, } floppy_off(drive); rel_fdc(); + flush_track_timer_start(drive); } else return -EINVAL; @@ -1591,8 +1639,7 @@ static int fd_locked_ioctl(struct block_device *bdev, blk_mode_t mode, case FDDEFPRM: return -EINVAL; case FDFLUSH: /* unconditionally, even if not needed */ - del_timer (flush_track_timer + drive); - non_int_flush_track(drive); + flush_track_timer_sync(drive); break; #ifdef RAW_IOCTL case IOCTL_RAW_TRACK: @@ -1611,9 +1658,14 @@ static int fd_ioctl(struct block_device *bdev, blk_mode_t mode, unsigned int cmd, unsigned long param) { int ret; + bool freeze_queue = cmd == FDFMTTRK || cmd == FDFLUSH; mutex_lock(&amiflop_mutex); + if (freeze_queue) + blk_mq_freeze_queue(bdev->bd_disk->queue); ret = fd_locked_ioctl(bdev, mode, cmd, param); + if (freeze_queue) + blk_mq_unfreeze_queue(bdev->bd_disk->queue); mutex_unlock(&amiflop_mutex); return ret; @@ -1713,9 +1765,9 @@ static void floppy_release(struct gendisk *disk) int drive = p - unit; mutex_lock(&amiflop_mutex); + blk_mq_freeze_queue(disk->queue); if (unit[drive].dirty == 1) { - del_timer (flush_track_timer + drive); - non_int_flush_track (drive); + flush_track_timer_sync(drive); } if (!fd_ref[drive]--) { @@ -1725,6 +1777,7 @@ static void floppy_release(struct gendisk *disk) #ifdef MODULE floppy_off (drive); #endif + blk_mq_unfreeze_queue(disk->queue); mutex_unlock(&amiflop_mutex); } diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c index 085a4d624659..f89b36017d97 100644 --- a/drivers/block/ublk_drv.c +++ b/drivers/block/ublk_drv.c @@ -1589,8 +1589,10 @@ static void ublk_cancel_cmd(struct ublk_queue *ubq, unsigned tag, io->flags |= UBLK_IO_FLAG_CANCELED; spin_unlock(&ubq->cancel_lock); + /* dependency ordered against smp_wmb() in ublk_prep_cancel() */ if (!done) - io_uring_cmd_done(io->cmd, UBLK_IO_RES_ABORT, 0, issue_flags); + io_uring_cmd_done(READ_ONCE(io->cmd), UBLK_IO_RES_ABORT, 0, + issue_flags); } /* @@ -1807,6 +1809,12 @@ static inline void ublk_fill_io_cmd(struct ublk_io *io, io->addr = buf_addr; } +/* + * Call before ublk_fill_io_cmd() publishes @cmd in io->cmd: a control-path + * cancel may complete any command found there, and io_uring_cmd_done() only + * takes it off the cancelable list if it is marked already. The handlers + * hold uring_lock, so marking takes no lock. + */ static inline void ublk_prep_cancel(struct io_uring_cmd *cmd, unsigned int issue_flags, struct ublk_queue *ubq, unsigned int tag) @@ -1820,6 +1828,8 @@ static inline void ublk_prep_cancel(struct io_uring_cmd *cmd, pdu->ubq = ubq; pdu->tag = tag; io_uring_cmd_mark_cancelable(cmd, issue_flags); + /* pairs with the cancel loading cmd from io->cmd, then cmd->flags */ + smp_wmb(); } static int __ublk_ch_uring_cmd(struct io_uring_cmd *cmd, @@ -1899,6 +1909,7 @@ static int __ublk_ch_uring_cmd(struct io_uring_cmd *cmd, goto out; } + ublk_prep_cancel(cmd, issue_flags, ubq, tag); ublk_fill_io_cmd(io, cmd, ub_cmd->addr); ublk_mark_io_ready(ub, ubq); break; @@ -1925,19 +1936,20 @@ static int __ublk_ch_uring_cmd(struct io_uring_cmd *cmd, goto out; } + ublk_prep_cancel(cmd, issue_flags, ubq, tag); ublk_fill_io_cmd(io, cmd, ub_cmd->addr); ublk_commit_completion(ub, ub_cmd); break; case UBLK_IO_NEED_GET_DATA: if (!(io->flags & UBLK_IO_FLAG_OWNED_BY_SRV)) goto out; + ublk_prep_cancel(cmd, issue_flags, ubq, tag); ublk_fill_io_cmd(io, cmd, ub_cmd->addr); ublk_handle_need_get_data(ub, ub_cmd->q_id, ub_cmd->tag); break; default: goto out; } - ublk_prep_cancel(cmd, issue_flags, ubq, tag); return -EIOCBQUEUED; out: @@ -3037,6 +3049,7 @@ static int ublk_ctrl_uring_cmd_permission(struct ublk_device *ub, case UBLK_CMD_STOP_DEV: case UBLK_CMD_ADD_DEV: case UBLK_CMD_DEL_DEV: + case UBLK_CMD_DEL_DEV_ASYNC: case UBLK_CMD_SET_PARAMS: case UBLK_CMD_START_USER_RECOVERY: case UBLK_CMD_END_USER_RECOVERY: diff --git a/drivers/char/hw_random/virtio-rng.c b/drivers/char/hw_random/virtio-rng.c index 2933f8cb769e..fcb60d4deaac 100644 --- a/drivers/char/hw_random/virtio-rng.c +++ b/drivers/char/hw_random/virtio-rng.c @@ -187,6 +187,7 @@ static int probe_common(struct virtio_device *vdev) err_find: ida_free(&rng_index_ida, index); err_ida: + vdev->priv = NULL; kfree(vi); return err; } @@ -195,6 +196,11 @@ static void remove_common(struct virtio_device *vdev) { struct virtrng_info *vi = vdev->priv; + if (!vi) { + virtio_reset_device(vdev); + return; + } + vi->hwrng_removed = true; vi->data_avail = 0; vi->data_idx = 0; @@ -204,6 +210,7 @@ static void remove_common(struct virtio_device *vdev) virtio_reset_device(vdev); vdev->config->del_vqs(vdev); ida_free(&rng_index_ida, vi->index); + vdev->priv = NULL; kfree(vi); } diff --git a/drivers/char/tpm/tpm2-cmd.c b/drivers/char/tpm/tpm2-cmd.c index 63d26aebf55d..2e4652d43bca 100644 --- a/drivers/char/tpm/tpm2-cmd.c +++ b/drivers/char/tpm/tpm2-cmd.c @@ -551,9 +551,10 @@ int tpm2_probe(struct tpm_chip *chip) out = (struct tpm_header *)buf.data; if (be16_to_cpu(out->tag) == TPM2_ST_NO_SESSIONS) chip->flags |= TPM_CHIP_FLAG_TPM2; + rc = 0; } tpm_buf_destroy(&buf); - return 0; + return rc; } EXPORT_SYMBOL_GPL(tpm2_probe); diff --git a/drivers/clk/nuvoton/clk-ma35d1.c b/drivers/clk/nuvoton/clk-ma35d1.c index 9f65d0623517..c914079cee2d 100644 --- a/drivers/clk/nuvoton/clk-ma35d1.c +++ b/drivers/clk/nuvoton/clk-ma35d1.c @@ -17,6 +17,7 @@ static DEFINE_SPINLOCK(ma35d1_lock); #define PLL_MAX_NUM 5 +#define MA35D1_CLK_COUNT 238 /* Clock Control Registers Offset */ #define REG_CLK_PWRCTL 0x00 @@ -467,12 +468,12 @@ static int ma35d1_clocks_probe(struct platform_device *pdev) int ret; ma35d1_hw_data = devm_kzalloc(dev, - struct_size(ma35d1_hw_data, hws, CLK_MAX_IDX), + struct_size(ma35d1_hw_data, hws, MA35D1_CLK_COUNT), GFP_KERNEL); if (!ma35d1_hw_data) return -ENOMEM; - ma35d1_hw_data->num = CLK_MAX_IDX; + ma35d1_hw_data->num = MA35D1_CLK_COUNT; hws = ma35d1_hw_data->hws; clk_base = devm_platform_ioremap_resource(pdev, 0); @@ -543,6 +544,8 @@ static int ma35d1_clocks_probe(struct platform_device *pdev) hws[HCLK3] = ma35d1_clk_fixed_factor(dev, "hclk3", "sysclk1_mux", 1, 2); hws[PCLK3] = ma35d1_clk_fixed_factor(dev, "pclk3", "sysclk1_mux", 1, 2); hws[PCLK4] = ma35d1_clk_fixed_factor(dev, "pclk4", "sysclk1_mux", 1, 2); + hws[PCLK3_DIV4096] = ma35d1_clk_fixed_factor(dev, "pclk3_div4096", "pclk3", 1, 4096); + hws[PCLK4_DIV4096] = ma35d1_clk_fixed_factor(dev, "pclk4_div4096", "pclk4", 1, 4096); hws[USBPHY0] = ma35d1_clk_fixed("usbphy0", 480000000); hws[USBPHY1] = ma35d1_clk_fixed("usbphy1", 480000000); diff --git a/drivers/cpufreq/amd-pstate.c b/drivers/cpufreq/amd-pstate.c index 88dffbd65fcc..7d51877c536f 100644 --- a/drivers/cpufreq/amd-pstate.c +++ b/drivers/cpufreq/amd-pstate.c @@ -1285,7 +1285,7 @@ static ssize_t amd_pstate_show_status(char *buf) int amd_pstate_update_status(const char *buf, size_t size) { - int mode_idx; + int cpu, mode_idx, ret = 0; if (size > strlen("passive") || size < strlen("active")) return -EINVAL; @@ -1295,10 +1295,25 @@ int amd_pstate_update_status(const char *buf, size_t size) if (mode_idx < 0 || mode_idx >= AMD_PSTATE_MAX) return -EINVAL; - if (mode_state_machine[cppc_state][mode_idx]) - return mode_state_machine[cppc_state][mode_idx](mode_idx); + if (!mode_state_machine[cppc_state][mode_idx]) + return 0; - return 0; + if (mode_idx == AMD_PSTATE_PASSIVE && + !cpu_feature_enabled(X86_FEATURE_CPPC)) { + cpus_read_lock(); + /* Check offline CPUs too, before changing or removing the driver. */ + for_each_present_cpu(cpu) { + if (cppc_auto_sel_is_immutable(cpu)) { + ret = -EOPNOTSUPP; + break; + } + } + cpus_read_unlock(); + if (ret) + return ret; + } + + return mode_state_machine[cppc_state][mode_idx](mode_idx); } EXPORT_SYMBOL_GPL(amd_pstate_update_status); diff --git a/drivers/crypto/omap-aes.c b/drivers/crypto/omap-aes.c index bad1adacbc84..df2fc4e1c31d 100644 --- a/drivers/crypto/omap-aes.c +++ b/drivers/crypto/omap-aes.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "omap-crypto.h" #include "omap-aes.h" @@ -222,7 +223,7 @@ static void omap_aes_dma_out_callback(void *data) struct omap_aes_dev *dd = data; /* dma_lch_out - completed */ - tasklet_schedule(&dd->done_task); + queue_work(system_bh_wq, &dd->done_task); } static int omap_aes_dma_init(struct omap_aes_dev *dd) @@ -496,9 +497,9 @@ static void omap_aes_copy_ivout(struct omap_aes_dev *dd, u8 *ivbuf) ((u32 *)ivbuf)[i] = omap_aes_read(dd, AES_REG_IV(dd, i)); } -static void omap_aes_done_task(unsigned long data) +static void omap_aes_done_task(struct work_struct *t) { - struct omap_aes_dev *dd = (struct omap_aes_dev *)data; + struct omap_aes_dev *dd = from_work(dd, t, done_task); pr_debug("enter done_task\n"); @@ -932,7 +933,7 @@ static irqreturn_t omap_aes_irq(int irq, void *dev_id) if (!dd->total) /* All bytes read! */ - tasklet_schedule(&dd->done_task); + queue_work(system_bh_wq, &dd->done_task); else /* Enable DATA_IN interrupt for next block */ omap_aes_write(dd, AES_REG_IRQ_ENABLE(dd), 0x2); @@ -1150,7 +1151,7 @@ static int omap_aes_probe(struct platform_device *pdev) (reg & dd->pdata->major_mask) >> dd->pdata->major_shift, (reg & dd->pdata->minor_mask) >> dd->pdata->minor_shift); - tasklet_init(&dd->done_task, omap_aes_done_task, (unsigned long)dd); + INIT_WORK(&dd->done_task, omap_aes_done_task); err = omap_aes_dma_init(dd); if (err == -EPROBE_DEFER) { @@ -1245,7 +1246,7 @@ static int omap_aes_probe(struct platform_device *pdev) omap_aes_dma_cleanup(dd); err_irq: - tasklet_kill(&dd->done_task); + cancel_work_sync(&dd->done_task); err_pm_disable: pm_runtime_disable(dev); err_res: @@ -1280,7 +1281,7 @@ static void omap_aes_remove(struct platform_device *pdev) crypto_engine_exit(dd->engine); - tasklet_kill(&dd->done_task); + cancel_work_sync(&dd->done_task); omap_aes_dma_cleanup(dd); pm_runtime_disable(dd->dev); diff --git a/drivers/crypto/omap-aes.h b/drivers/crypto/omap-aes.h index 0f35c9164764..d999d94e2314 100644 --- a/drivers/crypto/omap-aes.h +++ b/drivers/crypto/omap-aes.h @@ -161,7 +161,7 @@ struct omap_aes_dev { unsigned long flags; int err; - struct tasklet_struct done_task; + struct work_struct done_task; struct aead_queue aead_queue; spinlock_t lock; diff --git a/drivers/crypto/omap-des.c b/drivers/crypto/omap-des.c index d2c016eafd46..cccd275ace67 100644 --- a/drivers/crypto/omap-des.c +++ b/drivers/crypto/omap-des.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "omap-crypto.h" @@ -133,7 +134,7 @@ struct omap_des_dev { unsigned long flags; int err; - struct tasklet_struct done_task; + struct work_struct done_task; struct skcipher_request *req; struct crypto_engine *engine; @@ -328,7 +329,7 @@ static void omap_des_dma_out_callback(void *data) struct omap_des_dev *dd = data; /* dma_lch_out - completed */ - tasklet_schedule(&dd->done_task); + queue_work(system_bh_wq, &dd->done_task); } static int omap_des_dma_init(struct omap_des_dev *dd) @@ -584,9 +585,9 @@ static int omap_des_crypt_req(struct crypto_engine *engine, omap_des_crypt_dma_start(dd); } -static void omap_des_done_task(unsigned long data) +static void omap_des_done_task(struct work_struct *t) { - struct omap_des_dev *dd = (struct omap_des_dev *)data; + struct omap_des_dev *dd = from_work(dd, t, done_task); int i; pr_debug("enter done_task\n"); @@ -899,7 +900,7 @@ static irqreturn_t omap_des_irq(int irq, void *dev_id) if (!dd->total) /* All bytes read! */ - tasklet_schedule(&dd->done_task); + queue_work(system_bh_wq, &dd->done_task); else /* Enable DATA_IN interrupt for next block */ omap_des_write(dd, DES_REG_IRQ_ENABLE(dd), 0x2); @@ -1009,7 +1010,7 @@ static int omap_des_probe(struct platform_device *pdev) (reg & dd->pdata->major_mask) >> dd->pdata->major_shift, (reg & dd->pdata->minor_mask) >> dd->pdata->minor_shift); - tasklet_init(&dd->done_task, omap_des_done_task, (unsigned long)dd); + INIT_WORK(&dd->done_task, omap_des_done_task); err = omap_des_dma_init(dd); if (err == -EPROBE_DEFER) { @@ -1072,7 +1073,7 @@ static int omap_des_probe(struct platform_device *pdev) omap_des_dma_cleanup(dd); err_irq: - tasklet_kill(&dd->done_task); + cancel_work_sync(&dd->done_task); err_get: pm_runtime_disable(dev); err_res: @@ -1092,7 +1093,9 @@ static void omap_des_remove(struct platform_device *pdev) omap_des_unregister_algs(dd->pdata); - tasklet_kill(&dd->done_task); + crypto_engine_exit(dd->engine); + + cancel_work_sync(&dd->done_task); omap_des_dma_cleanup(dd); pm_runtime_disable(dd->dev); } diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c index 5bcd9ab0f72a..4f5e36197d3d 100644 --- a/drivers/crypto/omap-sham.c +++ b/drivers/crypto/omap-sham.c @@ -37,6 +37,7 @@ #include #include #include +#include #define MD5_DIGEST_SIZE 16 @@ -217,7 +218,7 @@ struct omap_sham_dev { int irq; int err; struct dma_chan *dma_lch; - struct tasklet_struct done_task; + struct work_struct done_task; u8 polling_mode; u8 xmit_buf[BUFLEN] OMAP_ALIGNED; @@ -561,7 +562,7 @@ static void omap_sham_dma_callback(void *param) struct omap_sham_dev *dd = param; set_bit(FLAGS_DMA_READY, &dd->flags); - tasklet_schedule(&dd->done_task); + queue_work(system_bh_wq, &dd->done_task); } static int omap_sham_xmit_dma(struct omap_sham_dev *dd, size_t length, @@ -1704,9 +1705,9 @@ static struct ahash_engine_alg algs_sha384_sha512[] = { }, }; -static void omap_sham_done_task(unsigned long data) +static void omap_sham_done_task(struct work_struct *t) { - struct omap_sham_dev *dd = (struct omap_sham_dev *)data; + struct omap_sham_dev *dd = from_work(dd, t, done_task); int err = 0; dev_dbg(dd->dev, "%s: flags=%lx\n", __func__, dd->flags); @@ -1740,7 +1741,7 @@ static void omap_sham_done_task(unsigned long data) static irqreturn_t omap_sham_irq_common(struct omap_sham_dev *dd) { set_bit(FLAGS_OUTPUT_READY, &dd->flags); - tasklet_schedule(&dd->done_task); + queue_work(system_bh_wq, &dd->done_task); return IRQ_HANDLED; } @@ -2044,6 +2045,20 @@ static const struct attribute_group omap_sham_attr_group = { .attrs = omap_sham_attrs, }; +static void omap_sham_unregister_algs(const struct omap_sham_pdata *pdata) +{ + struct omap_sham_algs_info *alg_info; + int i; + + for (i = pdata->algs_info_size - 1; i >= 0; i--) { + alg_info = &pdata->algs_info[i]; + + crypto_engine_unregister_ahashes(alg_info->algs_list, + alg_info->registered); + alg_info->registered = 0; + } +} + static int omap_sham_probe(struct platform_device *pdev) { struct omap_sham_dev *dd; @@ -2063,7 +2078,7 @@ static int omap_sham_probe(struct platform_device *pdev) platform_set_drvdata(pdev, dd); INIT_LIST_HEAD(&dd->list); - tasklet_init(&dd->done_task, omap_sham_done_task, (unsigned long)dd); + INIT_WORK(&dd->done_task, omap_sham_done_task); crypto_init_queue(&dd->queue, OMAP_SHAM_QUEUE_LENGTH); err = (dev->of_node) ? omap_sham_get_res_of(dd, dev, &res) : @@ -2167,10 +2182,7 @@ static int omap_sham_probe(struct platform_device *pdev) return 0; err_algs: - for (i = dd->pdata->algs_info_size - 1; i >= 0; i--) - for (j = dd->pdata->algs_info[i].registered - 1; j >= 0; j--) - crypto_engine_unregister_ahash( - &dd->pdata->algs_info[i].algs_list[j]); + omap_sham_unregister_algs(dd->pdata); err_engine_start: crypto_engine_exit(dd->engine); err_engine: @@ -2191,20 +2203,15 @@ static int omap_sham_probe(struct platform_device *pdev) static void omap_sham_remove(struct platform_device *pdev) { struct omap_sham_dev *dd; - int i, j; dd = platform_get_drvdata(pdev); spin_lock_bh(&sham.lock); list_del(&dd->list); spin_unlock_bh(&sham.lock); - for (i = dd->pdata->algs_info_size - 1; i >= 0; i--) - for (j = dd->pdata->algs_info[i].registered - 1; j >= 0; j--) { - crypto_engine_unregister_ahash( - &dd->pdata->algs_info[i].algs_list[j]); - dd->pdata->algs_info[i].registered--; - } - tasklet_kill(&dd->done_task); + omap_sham_unregister_algs(dd->pdata); + crypto_engine_exit(dd->engine); + cancel_work_sync(&dd->done_task); pm_runtime_dont_use_autosuspend(&pdev->dev); pm_runtime_disable(&pdev->dev); diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c index cf647662ae81..d06f95f7868c 100644 --- a/drivers/dma/qcom/bam_dma.c +++ b/drivers/dma/qcom/bam_dma.c @@ -1322,8 +1322,8 @@ static int bam_dma_probe(struct platform_device *pdev) for (i = 0; i < bdev->num_channels; i++) bam_channel_init(bdev, &bdev->channels[i], i); - ret = devm_request_irq(bdev->dev, bdev->irq, bam_dma_irq, - IRQF_TRIGGER_HIGH, "bam_dma", bdev); + ret = request_irq(bdev->irq, bam_dma_irq, IRQF_TRIGGER_NONE, + "bam_dma", bdev); if (ret) goto err_bam_channel_exit; @@ -1356,7 +1356,7 @@ static int bam_dma_probe(struct platform_device *pdev) ret = dma_async_device_register(&bdev->common); if (ret) { dev_err(bdev->dev, "failed to register dma async device\n"); - goto err_bam_channel_exit; + goto err_free_irq; } ret = of_dma_controller_register(pdev->dev.of_node, bam_dma_xlate, @@ -1375,6 +1375,8 @@ static int bam_dma_probe(struct platform_device *pdev) err_unregister_dma: dma_async_device_unregister(&bdev->common); +err_free_irq: + free_irq(bdev->irq, bdev); err_bam_channel_exit: for (i = 0; i < bdev->num_channels; i++) tasklet_kill(&bdev->channels[i].vc.task); @@ -1400,7 +1402,7 @@ static void bam_dma_remove(struct platform_device *pdev) /* mask all interrupts for this execution environment */ writel_relaxed(0, bam_addr(bdev, 0, BAM_IRQ_SRCS_MSK_EE)); - devm_free_irq(bdev->dev, bdev->irq, bdev); + free_irq(bdev->irq, bdev); for (i = 0; i < bdev->num_channels; i++) { bam_dma_terminate_all(&bdev->channels[i].vc.chan); diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile index 1141cd06011f..d33c02e58717 100644 --- a/drivers/firmware/efi/libstub/Makefile +++ b/drivers/firmware/efi/libstub/Makefile @@ -22,7 +22,7 @@ cflags-$(CONFIG_X86) += -m$(BITS) -D__KERNEL__ -std=gnu11 \ # arm64 uses the full KBUILD_CFLAGS so it's necessary to explicitly # disable the stackleak plugin -cflags-$(CONFIG_ARM64) += -fpie $(DISABLE_STACKLEAK_PLUGIN) \ +cflags-$(CONFIG_ARM64) += -fpie $(DISABLE_STACKLEAK_PLUGIN) -ffixed-x18 \ -fno-unwind-tables -fno-asynchronous-unwind-tables cflags-$(CONFIG_ARM) += -DEFI_HAVE_STRLEN -DEFI_HAVE_STRNLEN \ -DEFI_HAVE_MEMCHR -DEFI_HAVE_STRRCHR \ diff --git a/drivers/gpio/gpio-pca953x.c b/drivers/gpio/gpio-pca953x.c index 72a1819713ed..db29342a7676 100644 --- a/drivers/gpio/gpio-pca953x.c +++ b/drivers/gpio/gpio-pca953x.c @@ -742,13 +742,18 @@ static int pca953x_irq_set_wake(struct irq_data *d, unsigned int on) { struct gpio_chip *gc = irq_data_get_irq_chip_data(d); struct pca953x_chip *chip = gpiochip_get_data(gc); + int ret; + + ret = irq_set_irq_wake(chip->client->irq, on); + if (ret) + return ret; if (on) atomic_inc(&chip->wakeup_path); else atomic_dec(&chip->wakeup_path); - return irq_set_irq_wake(chip->client->irq, on); + return 0; } static void pca953x_irq_bus_lock(struct irq_data *d) diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c index b85ba9989b14..e7416a018a54 100644 --- a/drivers/gpio/gpiolib-cdev.c +++ b/drivers/gpio/gpiolib-cdev.c @@ -1367,11 +1367,6 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 flags, assign_bit(FLAG_ACTIVE_LOW, flagsp, flags & GPIO_V2_LINE_FLAG_ACTIVE_LOW); - if (flags & GPIO_V2_LINE_FLAG_OUTPUT) - set_bit(FLAG_IS_OUT, flagsp); - else if (flags & GPIO_V2_LINE_FLAG_INPUT) - clear_bit(FLAG_IS_OUT, flagsp); - assign_bit(FLAG_EDGE_RISING, flagsp, flags & GPIO_V2_LINE_FLAG_EDGE_RISING); assign_bit(FLAG_EDGE_FALLING, flagsp, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index d79c431de223..b921587d428c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -103,6 +103,7 @@ struct amdgpu_kfd_dev { struct kfd_dev *dev; int64_t vram_used[MAX_XCP]; uint64_t vram_used_aligned[MAX_XCP]; + atomic64_t vram_pinned; bool init_complete; struct work_struct reset_work; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index bc284939cf66..55834719e984 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -1500,6 +1500,11 @@ static int amdgpu_amdkfd_gpuvm_pin_bo(struct amdgpu_bo *bo, u32 domain) pr_err("Error in Pinning BO to domain: %d\n", domain); amdgpu_bo_sync_wait(bo, AMDGPU_FENCE_OWNER_KFD, false); + + if (!ret && bo->tbo.resource->mem_type == TTM_PL_VRAM) + atomic64_add(amdgpu_bo_size(bo), + &amdgpu_ttm_adev(bo->tbo.bdev)->kfd.vram_pinned); + out: amdgpu_bo_unreserve(bo); return ret; @@ -1522,6 +1527,11 @@ static void amdgpu_amdkfd_gpuvm_unpin_bo(struct amdgpu_bo *bo) return; amdgpu_bo_unpin(bo); + + if (bo->tbo.resource->mem_type == TTM_PL_VRAM) + atomic64_sub(amdgpu_bo_size(bo), + &amdgpu_ttm_adev(bo->tbo.bdev)->kfd.vram_pinned); + amdgpu_bo_unreserve(bo); } @@ -1681,6 +1691,7 @@ size_t amdgpu_amdkfd_get_available_memory(struct amdgpu_device *adev, vram_available = KFD_XCP_MEMORY_SIZE(adev, xcp_id) - adev->kfd.vram_used_aligned[xcp_id] - atomic64_read(&adev->vram_pin_size) + + atomic64_read(&adev->kfd.vram_pinned) - reserved_for_pt - reserved_for_ras; @@ -2441,6 +2452,9 @@ static int import_obj_create(struct amdgpu_device *adev, err_remove_mem: remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info); + amdgpu_sync_free(&(*mem)->sync); + mutex_destroy(&(*mem)->lock); + dma_buf_put(dma_buf); drm_vma_node_revoke(&obj->vma_node, drm_priv); err_free_mem: kfree(*mem); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index 528f0aeac36c..c49aabb343a7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -160,7 +160,12 @@ static int amdgpu_cs_p1_user_fence(struct amdgpu_cs_parser *p, if (size != PAGE_SIZE || data->offset > (size - 8)) return -EINVAL; - if (amdgpu_ttm_tt_get_usermm(p->uf_bo->tbo.ttm)) + if (amdgpu_ttm_tt_is_userptr(p->uf_bo->tbo.ttm)) + return -EINVAL; + + /* The GPU writes the user fence through the GART */ + if (p->uf_bo->preferred_domains != AMDGPU_GEM_DOMAIN_GTT && + p->uf_bo->allowed_domains != AMDGPU_GEM_DOMAIN_GTT) return -EINVAL; *offset = data->offset; @@ -198,25 +203,17 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p, struct amdgpu_fpriv *fpriv = p->filp->driver_priv; unsigned int num_ibs[AMDGPU_CS_GANG_SIZE] = { }; struct amdgpu_vm *vm = &fpriv->vm; - uint64_t *chunk_array_user; uint64_t *chunk_array; uint32_t uf_offset = 0; size_t size; int ret; int i; - chunk_array = kvmalloc_array(cs->in.num_chunks, sizeof(uint64_t), - GFP_KERNEL); - if (!chunk_array) - return -ENOMEM; - - /* get chunks */ - chunk_array_user = u64_to_user_ptr(cs->in.chunks); - if (copy_from_user(chunk_array, chunk_array_user, - sizeof(uint64_t)*cs->in.num_chunks)) { - ret = -EFAULT; - goto free_chunk; - } + chunk_array = memdup_array_user(u64_to_user_ptr(cs->in.chunks), + cs->in.num_chunks, + sizeof(uint64_t)); + if (IS_ERR(chunk_array)) + return PTR_ERR(chunk_array); p->nchunks = cs->in.num_chunks; p->chunks = kvmalloc_array(p->nchunks, sizeof(struct amdgpu_cs_chunk), @@ -229,7 +226,6 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p, for (i = 0; i < p->nchunks; i++) { struct drm_amdgpu_cs_chunk __user *chunk_ptr = NULL; struct drm_amdgpu_cs_chunk user_chunk; - uint32_t __user *cdata; chunk_ptr = u64_to_user_ptr(chunk_array[i]); if (copy_from_user(&user_chunk, chunk_ptr, @@ -242,20 +238,22 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p, p->chunks[i].length_dw = user_chunk.length_dw; size = p->chunks[i].length_dw; - cdata = u64_to_user_ptr(user_chunk.chunk_data); - p->chunks[i].kdata = kvmalloc_array(size, sizeof(uint32_t), - GFP_KERNEL); - if (p->chunks[i].kdata == NULL) { - ret = -ENOMEM; + if (size > INT_MAX / sizeof(uint32_t)) { + ret = -EINVAL; i--; goto free_partial_kdata; } - size *= sizeof(uint32_t); - if (copy_from_user(p->chunks[i].kdata, cdata, size)) { - ret = -EFAULT; + + p->chunks[i].kdata = vmemdup_array_user(u64_to_user_ptr(user_chunk.chunk_data), + size, + sizeof(uint32_t)); + if (IS_ERR(p->chunks[i].kdata)) { + ret = PTR_ERR(p->chunks[i].kdata); + i--; goto free_partial_kdata; } + size *= sizeof(uint32_t); /* Assume the worst on the following checks */ ret = -EINVAL; @@ -1086,7 +1084,7 @@ static int amdgpu_cs_patch_ibs(struct amdgpu_cs_parser *p, if (r) return r; - kptr += va_start - (m->start * AMDGPU_GPU_PAGE_SIZE); + kptr += va_start - (m->start * AMDGPU_GPU_PAGE_SIZE) + m->offset; if (ring->funcs->parse_cs) { memcpy(ib->ptr, kptr, ib->length_dw * 4); @@ -1320,10 +1318,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p, fence = &p->jobs[i]->base.s_fence->scheduled; dma_fence_get(fence); r = drm_sched_job_add_dependency(&leader->base, fence); - if (r) { - dma_fence_put(fence); + if (r) return r; - } } if (p->gang_size > 1) { diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index a8f8bb342884..037cb9ce69e4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -2680,6 +2680,13 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) break; } + /* Check for IP version 9.4.3 with A0 hardware */ + if (amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 4, 3) && + !amdgpu_device_get_rev_id(adev)) { + dev_err(adev->dev, "Unsupported A0 hardware\n"); + return -ENODEV; /* device unsupported - no device error */ + } + if (amdgpu_has_atpx() && (amdgpu_is_atpx_hybrid() || amdgpu_has_atpx_dgpu_power_cntl()) && @@ -2692,7 +2699,6 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) adev->has_pr3 = parent ? pci_pr3_present(parent) : false; } - adev->pm.pp_feature = amdgpu_pp_feature_mask; if (amdgpu_sriov_vf(adev) || sched_policy == KFD_SCHED_POLICY_NO_HWS) adev->pm.pp_feature &= ~PP_GFXOFF_MASK; @@ -3301,6 +3307,12 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) amdgpu_device_set_cg_state(adev, AMD_CG_STATE_GATE); amdgpu_device_set_pg_state(adev, AMD_PG_STATE_GATE); + r = amdgpu_dpm_reapply_umd_profile(adev); + if (r) { + dev_err(adev->dev, "Failed to restore UMD profile state: %d\n", r); + return r; + } + amdgpu_device_fill_reset_magic(adev); r = amdgpu_device_enable_mgpu_fan_boost(); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 1a112a361757..7c9fee6a2edb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -2838,19 +2838,6 @@ static int amdgpu_pmops_runtime_suspend(struct device *dev) return 0; } -static void amdgpu_restore_umd_profile_pstate_after_runpm(struct amdgpu_device *adev) -{ - enum amd_dpm_forced_level level; - uint32_t profile_mode_mask = AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD | - AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK | - AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK | - AMD_DPM_FORCED_LEVEL_PROFILE_PEAK; - - level = amdgpu_dpm_get_performance_level(adev); - if (level & profile_mode_mask) - amdgpu_asic_update_umd_stable_pstate(adev, true); -} - static int amdgpu_pmops_runtime_resume(struct device *dev) { struct pci_dev *pdev = to_pci_dev(dev); @@ -2896,7 +2883,6 @@ static int amdgpu_pmops_runtime_resume(struct device *dev) if (adev->pm.rpm_mode == AMDGPU_RUNPM_PX) drm_dev->switch_power_state = DRM_SWITCH_POWER_ON; - amdgpu_restore_umd_profile_pstate_after_runpm(adev); adev->in_runpm = false; return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c index 86c17a8946f5..8fe497411601 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c @@ -423,10 +423,8 @@ int amdgpu_sync_push_to_job(struct amdgpu_sync *sync, struct amdgpu_job *job) dma_fence_get(f); r = drm_sched_job_add_dependency(&job->base, f); - if (r) { - dma_fence_put(f); + if (r) return r; - } } return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c index 902d0a538e41..dd72891624eb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c @@ -234,7 +234,6 @@ static int amdgpu_vm_sdma_update(struct amdgpu_vm_update_params *p, dma_fence_get(fence); r = drm_sched_job_add_dependency(&p->job->base, fence); if (r) { - dma_fence_put(fence); dma_resv_iter_end(&cursor); return r; } diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c index daed4a8439cc..8bfa0b07dec7 100644 --- a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c +++ b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c @@ -1736,8 +1736,9 @@ static void dce_v10_0_afmt_setmode(struct drm_encoder *encoder, WREG32(mmAFMT_RAMP_CONTROL2 + dig->afmt->offset, 0x00000001); WREG32(mmAFMT_RAMP_CONTROL3 + dig->afmt->offset, 0x00000001); - /* enable audio after to setting up hw */ - dce_v10_0_audio_enable(adev, dig->afmt->pin, true); + /* Only enable audio if the display supports it */ + if (connector->display_info.has_audio) + dce_v10_0_audio_enable(adev, dig->afmt->pin, true); } static void dce_v10_0_afmt_enable(struct drm_encoder *encoder, bool enable) diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c index 1e15348a5c7d..9b39518a3e11 100644 --- a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c +++ b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c @@ -1721,8 +1721,9 @@ static void dce_v6_0_afmt_setmode(struct drm_encoder *encoder, dce_v6_0_audio_dp_enable(encoder, 1); } - /* enable audio after setting up hw */ - dce_v6_0_audio_enable(adev, dig->afmt->pin, true); + /* Only enable audio if the display supports it */ + if (connector->display_info.has_audio) + dce_v6_0_audio_enable(adev, dig->afmt->pin, true); } static void dce_v6_0_afmt_enable(struct drm_encoder *encoder, bool enable) diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c index 7e92f322bc33..c22bb261d0ff 100644 --- a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c @@ -1683,8 +1683,9 @@ static void dce_v8_0_afmt_setmode(struct drm_encoder *encoder, WREG32(mmAFMT_RAMP_CONTROL2 + offset, 0x00000001); WREG32(mmAFMT_RAMP_CONTROL3 + offset, 0x00000001); - /* enable audio after setting up hw */ - dce_v8_0_audio_enable(adev, dig->afmt->pin, true); + /* Only enable audio if the display supports it */ + if (connector->display_info.has_audio) + dce_v8_0_audio_enable(adev, dig->afmt->pin, true); } static void dce_v8_0_afmt_enable(struct drm_encoder *encoder, bool enable) diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c index a076a314dc67..5282c2de1ec8 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c @@ -3828,6 +3828,7 @@ static void gfx_v12_0_update_spm_vmid(struct amdgpu_device *adev, { u32 reg, data; + amdgpu_gfx_off_ctrl(adev, false); reg = SOC15_REG_OFFSET(GC, 0, regRLC_SPM_MC_CNTL); if (amdgpu_sriov_is_pp_one_vf(adev)) data = RREG32_NO_KIQ(reg); @@ -3841,6 +3842,7 @@ static void gfx_v12_0_update_spm_vmid(struct amdgpu_device *adev, WREG32_SOC15_NO_KIQ(GC, 0, regRLC_SPM_MC_CNTL, data); else WREG32_SOC15(GC, 0, regRLC_SPM_MC_CNTL, data); + amdgpu_gfx_off_ctrl(adev, true); if (ring && amdgpu_sriov_is_pp_one_vf(adev) diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c index d82f90f48024..06e4b393d1ce 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c @@ -1139,12 +1139,13 @@ static void gmc_v9_0_get_coherence_flags(struct amdgpu_device *adev, bool uncached = bo->flags & AMDGPU_GEM_CREATE_UNCACHED; struct amdgpu_vm *vm = mapping->bo_va->base.vm; unsigned int mtype_local, mtype; + uint32_t gc_ip_version = amdgpu_ip_version(adev, GC_HWIP, 0); bool snoop = false; bool is_local; dma_resv_assert_held(bo->tbo.base.resv); - switch (amdgpu_ip_version(adev, GC_HWIP, 0)) { + switch (gc_ip_version) { case IP_VERSION(9, 4, 1): case IP_VERSION(9, 4, 2): if (is_vram) { @@ -1158,10 +1159,7 @@ static void gmc_v9_0_get_coherence_flags(struct amdgpu_device *adev, /* FIXME: is this still needed? Or does * amdgpu_ttm_tt_pde_flags already handle this? */ - if ((amdgpu_ip_version(adev, GC_HWIP, 0) == - IP_VERSION(9, 4, 2) || - amdgpu_ip_version(adev, GC_HWIP, 0) == - IP_VERSION(9, 4, 3)) && + if (gc_ip_version == IP_VERSION(9, 4, 2) && adev->gmc.xgmi.connected_to_cpu) snoop = true; } else { @@ -1209,20 +1207,17 @@ static void gmc_v9_0_get_coherence_flags(struct amdgpu_device *adev, if (uncached) { mtype = MTYPE_UC; } else if (ext_coherent) { - if (adev->rev_id) - mtype = is_local ? MTYPE_CC : MTYPE_UC; - else - mtype = MTYPE_UC; + mtype = is_local ? MTYPE_CC : MTYPE_UC; } else if (adev->flags & AMD_IS_APU) { mtype = is_local ? mtype_local : MTYPE_NC; } else { /* dGPU */ if (is_local) mtype = mtype_local; - else if (is_vram) - mtype = MTYPE_NC; - else + else if (gc_ip_version < IP_VERSION(9, 5, 0) && !is_vram) mtype = MTYPE_UC; + else + mtype = MTYPE_NC; } break; @@ -1332,7 +1327,7 @@ static void gmc_v9_0_override_vm_pte_flags(struct amdgpu_device *adev, mtype_local = MTYPE_CC; *flags = AMDGPU_PTE_MTYPE_VG10(*flags, mtype_local); - } else if (adev->rev_id) { + } else { /* MTYPE_UC case */ *flags = AMDGPU_PTE_MTYPE_VG10(*flags, MTYPE_CC); } @@ -2329,17 +2324,22 @@ static int gmc_v9_0_hw_init(void *handle) * (type 2), which flushes both. Due to a race condition with * concurrent memory accesses using the same TLB cache line, we still * need a second TLB flush after this. + * + * GC 9.4.3, 9.4.4 and 9.5.0 hit a similar race. */ - adev->gmc.flush_tlb_needs_extra_type_2 = - amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 4, 0) && - adev->gmc.xgmi.num_physical_nodes; - /* - * TODO: This workaround is badly documented and had a buggy - * implementation. We should probably verify what we do here. - */ - adev->gmc.flush_tlb_needs_extra_type_0 = - amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 4, 3) && - adev->rev_id == 0; + switch (amdgpu_ip_version(adev, GC_HWIP, 0)) { + case IP_VERSION(9, 4, 0): + adev->gmc.flush_tlb_needs_extra_type_2 = + adev->gmc.xgmi.num_physical_nodes; + break; + case IP_VERSION(9, 4, 3): + case IP_VERSION(9, 4, 4): + case IP_VERSION(9, 5, 0): + adev->gmc.flush_tlb_needs_extra_type_2 = true; + break; + default: + break; + } /* The sequence of these two function calls matters.*/ gmc_v9_0_init_golden_registers(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c index fa6c7e5fbbe3..78775b8cfbc6 100644 --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c @@ -168,8 +168,19 @@ static int ih_v6_0_toggle_ring_interrupts(struct amdgpu_device *adev, tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0)); if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) { - if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) + /* + * PSP programming of IH_RB_CNTL is slow enough for MSI to + * arrive before this call returns. Publish ih->enabled first + * so amdgpu_ih_process() consumes that IV and rearms. + */ + if (enable) { + ih->enabled = true; + smp_wmb(); + } + if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) { + ih->enabled = false; return -ETIMEDOUT; + } } else { WREG32(ih_regs->ih_rb_cntl, tmp); } diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c index 236fedc5b183..63762ea6f87c 100644 --- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c @@ -201,8 +201,10 @@ static int mes_v11_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes, r = amdgpu_fence_wait_polling(ring, seq - ring->fence_drv.num_fences_mask, timeout); - if (r < 1) + if (r < 1) { + r = -ETIMEDOUT; goto error_undo; + } api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off); api_status->api_completion_fence_addr = status_gpu_addr; diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c index c432a2a3405d..98c706fa97fe 100644 --- a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c +++ b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c @@ -187,8 +187,10 @@ static int mes_v12_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes, r = amdgpu_fence_wait_polling(ring, seq - ring->fence_drv.num_fences_mask, timeout); - if (r < 1) + if (r < 1) { + r = -ETIMEDOUT; goto error_undo; + } api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off); api_status->api_completion_fence_addr = status_gpu_addr; diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 17aab897f86b..454a161a189f 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -161,13 +161,52 @@ static int navi10_ih_toggle_ring_interrupts(struct amdgpu_device *adev, tmp = RREG32(ih_regs->ih_rb_cntl); tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_ENABLE, (enable ? 1 : 0)); tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_GPU_TS_ENABLE, 1); + + if (enable) { + /* Unset the CLEAR_OVERFLOW bit to make sure the next step + * is switching the bit from 0 to 1 + */ + tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 0); + if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) { + if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) + return -ETIMEDOUT; + } else { + WREG32_NO_KIQ(ih_regs->ih_rb_cntl, tmp); + } + + /* Clear RB_OVERFLOW bit */ + tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 1); + if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) { + if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) + return -ETIMEDOUT; + } else { + WREG32_NO_KIQ(ih_regs->ih_rb_cntl, tmp); + } + + /* Unset the CLEAR_OVERFLOW bit immediately so new overflows + * can be detected. + */ + tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 0); + } + /* enable_intr field is only valid in ring0 */ if (ih == &adev->irq.ih) tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0)); if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) { - if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) + /* + * PSP programming of IH_RB_CNTL is slow enough for MSI to + * arrive before this call returns. Publish ih->enabled first + * so amdgpu_ih_process() consumes that IV and rearms. + */ + if (enable) { + ih->enabled = true; + smp_wmb(); + } + if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) { + ih->enabled = false; return -ETIMEDOUT; + } } else { WREG32(ih_regs->ih_rb_cntl, tmp); } diff --git a/drivers/gpu/drm/amd/amdgpu/si.c b/drivers/gpu/drm/amd/amdgpu/si.c index 93c68abf447a..dc0102db4633 100644 --- a/drivers/gpu/drm/amd/amdgpu/si.c +++ b/drivers/gpu/drm/amd/amdgpu/si.c @@ -1509,6 +1509,16 @@ static bool si_need_full_reset(struct amdgpu_device *adev) static bool si_need_reset_on_init(struct amdgpu_device *adev) { + /* + * MacBookPros with switchable graphics put VRAM at 0 when the iGPU + * is enabled. Any later post (S3 resume, GPU reset) moves it back to + * the vbios default, so reset and post on load to start with the + * vbios layout. + */ + if (adev->pdev->subsystem_vendor == PCI_VENDOR_ID_APPLE && + !(RREG32(mmMC_VM_FB_LOCATION) & 0xFFFF)) + return true; + return false; } diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c index fa95bde45f3c..297148b68b86 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c @@ -347,11 +347,6 @@ struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf) f2g = &aldebaran_kfd2kgd; break; case IP_VERSION(9, 4, 3): - gfx_target_version = adev->rev_id >= 1 ? 90402 - : adev->flags & AMD_IS_APU ? 90400 - : 90401; - f2g = &gc_9_4_3_kfd2kgd; - break; case IP_VERSION(9, 4, 4): gfx_target_version = 90402; f2g = &gc_9_4_3_kfd2kgd; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_queue.c index e64eba0a9263..7dd458dcab5d 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_queue.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_queue.c @@ -424,7 +424,7 @@ static u32 kfd_get_vgpr_size_per_cu(u32 gfxv) { u32 vgpr_size = 0x40000; - if ((gfxv / 100 * 100) == 90400 || /* GFX_VERSION_AQUA_VANJARAM */ + if (gfxv == 90402 || /* GFX_VERSION_AQUA_VANJARAM */ gfxv == 90010 || /* GFX_VERSION_ALDEBARAN */ gfxv == 90008) /* GFX_VERSION_ARCTURUS */ vgpr_size = 0x80000; @@ -483,7 +483,7 @@ void kfd_queue_ctx_save_restore_size(struct kfd_topology_device *dev) if (gfxv == 80002) /* GFX_VERSION_TONGA */ props->eop_buffer_size = 0x8000; - else if ((gfxv / 100 * 100) == 90400) /* GFX_VERSION_AQUA_VANJARAM */ + else if (gfxv == 90402) /* GFX_VERSION_AQUA_VANJARAM */ props->eop_buffer_size = 4096; else if (gfxv >= 80000) props->eop_buffer_size = 4096; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c index 7bbeb9768a56..8f114e85ffbb 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c @@ -1224,6 +1224,7 @@ svm_range_get_pte_flags(struct kfd_node *node, struct kfd_node *bo_node; uint32_t flags = prange->flags; uint32_t mapping_flags = 0; + uint32_t gc_ip_version = KFD_GC_VERSION(node); uint64_t pte_flags; bool snoop = (domain != SVM_RANGE_VRAM_DOMAIN); bool coherent = flags & (KFD_IOCTL_SVM_FLAG_COHERENT | KFD_IOCTL_SVM_FLAG_EXT_COHERENT); @@ -1233,7 +1234,7 @@ svm_range_get_pte_flags(struct kfd_node *node, if (domain == SVM_RANGE_VRAM_DOMAIN) bo_node = prange->svm_bo->node; - switch (amdgpu_ip_version(node->adev, GC_HWIP, 0)) { + switch (gc_ip_version) { case IP_VERSION(9, 4, 1): if (domain == SVM_RANGE_VRAM_DOMAIN) { if (bo_node == node) { @@ -1271,7 +1272,7 @@ svm_range_get_pte_flags(struct kfd_node *node, case IP_VERSION(9, 4, 3): case IP_VERSION(9, 4, 4): if (ext_coherent) - mtype_local = node->adev->rev_id ? AMDGPU_VM_MTYPE_CC : AMDGPU_VM_MTYPE_UC; + mtype_local = AMDGPU_VM_MTYPE_CC; else mtype_local = amdgpu_mtype_local == 1 ? AMDGPU_VM_MTYPE_NC : amdgpu_mtype_local == 2 ? AMDGPU_VM_MTYPE_CC : AMDGPU_VM_MTYPE_RW; @@ -1286,9 +1287,13 @@ svm_range_get_pte_flags(struct kfd_node *node, */ else if (svm_nodes_in_same_hive(bo_node, node) && !ext_coherent) mapping_flags |= AMDGPU_VM_MTYPE_NC; - /* PCIe P2P or extended system scope coherence */ - else + /* PCIe P2P on GPUs pre-9.5.0 */ + else if (gc_ip_version < IP_VERSION(9, 5, 0) && + !svm_nodes_in_same_hive(bo_node, node)) mapping_flags |= AMDGPU_VM_MTYPE_UC; + /* Other remote memory */ + else + mapping_flags |= ext_coherent ? AMDGPU_VM_MTYPE_UC : AMDGPU_VM_MTYPE_NC; /* system memory accessed by the APU */ } else if (node->adev->flags & AMD_IS_APU) { /* On NUMA systems, locality is determined per-page @@ -1300,7 +1305,10 @@ svm_range_get_pte_flags(struct kfd_node *node, mapping_flags |= ext_coherent ? AMDGPU_VM_MTYPE_UC : AMDGPU_VM_MTYPE_NC; /* system memory accessed by the dGPU */ } else { - mapping_flags |= AMDGPU_VM_MTYPE_UC; + if (gc_ip_version < IP_VERSION(9, 5, 0)) + mapping_flags |= AMDGPU_VM_MTYPE_UC; + else + mapping_flags |= AMDGPU_VM_MTYPE_NC; } break; case IP_VERSION(12, 0, 0): @@ -1322,7 +1330,7 @@ svm_range_get_pte_flags(struct kfd_node *node, pte_flags = AMDGPU_PTE_VALID; pte_flags |= (domain == SVM_RANGE_VRAM_DOMAIN) ? 0 : AMDGPU_PTE_SYSTEM; pte_flags |= snoop ? AMDGPU_PTE_SNOOPED : 0; - if (KFD_GC_VERSION(node) >= IP_VERSION(12, 0, 0)) + if (gc_ip_version >= IP_VERSION(12, 0, 0)) pte_flags |= AMDGPU_PTE_IS_PTE; pte_flags |= amdgpu_gem_va_map_flags(node->adev, mapping_flags); diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c index 2ee034879f9f..24b1103c88c8 100644 --- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c +++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c @@ -511,6 +511,9 @@ static void dcn401_update_clocks_update_dtb_dto(struct clk_mgr_internal *clk_mgr otg_master = resource_get_otg_master_for_stream( &context->res_ctx, context->streams[i]); ASSERT(otg_master); + if (!otg_master) + continue; + ASSERT(otg_master->clock_source); ASSERT(otg_master->clock_source->funcs->program_pix_clk); ASSERT(otg_master->stream_res.pix_clk_params.controller_id >= CONTROLLER_ID_D0); diff --git a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c index fa29eb177331..896c69352482 100644 --- a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c +++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c @@ -2328,12 +2328,8 @@ bool dp_is_sink_present(struct dc_link *link) * which indicates we need additional delay */ - if (dal_ddc_open(ddc, GPIO_MODE_INPUT, - GPIO_DDC_CONFIG_TYPE_MODE_I2C) != GPIO_RESULT_OK) { - dal_ddc_close(ddc); - + if (dal_ddc_open(ddc, GPIO_MODE_INPUT, GPIO_DDC_CONFIG_TYPE_MODE_I2C) != GPIO_RESULT_OK) return present; - } /* * Read GPIO: DP sink is present if both clock and data pins are zero diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c b/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c index b4694985a40a..d70b14f54519 100644 --- a/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c +++ b/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c @@ -452,6 +452,13 @@ void optc2_lock_doublebuffer_disable(struct timing_generator *optc) void optc2_setup_manual_trigger(struct timing_generator *optc) { struct optc *optc1 = DCN10TG_FROM_TG(optc); + uint32_t v_total, v_total_max; + + REG_GET(OTG_V_TOTAL_MAX, OTG_V_TOTAL_MAX, &v_total_max); + if (!v_total_max) { + REG_GET(OTG_V_TOTAL, OTG_V_TOTAL, &v_total); + optc->funcs->set_vtotal_min_max(optc, v_total, v_total); + } /* Set the min/max selectors unconditionally so that * DMCUB fw may change OTG timings when necessary diff --git a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c index 1fa4d32ce0be..c79a175d4d9a 100644 --- a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c +++ b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c @@ -1775,8 +1775,21 @@ static bool dml1_validate(struct dc *dc, struct dc_state *context, bool fast_val out = dcn32_internal_validate_bw(dc, context, pipes, &pipe_cnt, &vlevel, fast_validate); DC_FP_END(); - if (pipe_cnt == 0) + if (pipe_cnt == 0) { + /* The context is a copy of the previous state, so without this the + * clocks the last display needed stay applied with all displays off. + */ + context->bw_ctx.bw.dcn.clk.socclk_khz = 0; + context->bw_ctx.bw.dcn.clk.dppclk_khz = 0; + context->bw_ctx.bw.dcn.clk.dcfclk_khz = 0; + context->bw_ctx.bw.dcn.clk.dcfclk_deep_sleep_khz = 0; + context->bw_ctx.bw.dcn.clk.dramclk_khz = 0; + context->bw_ctx.bw.dcn.clk.fclk_khz = 0; + context->bw_ctx.bw.dcn.clk.p_state_change_support = true; + context->bw_ctx.bw.dcn.clk.fclk_p_state_change_support = true; + context->bw_ctx.bw.dcn.clk.num_ways = 0; goto validate_out; + } if (!out) goto validate_fail; diff --git a/drivers/gpu/drm/amd/include/kgd_pp_interface.h b/drivers/gpu/drm/amd/include/kgd_pp_interface.h index a97b03232c0b..a42e6c8710a2 100644 --- a/drivers/gpu/drm/amd/include/kgd_pp_interface.h +++ b/drivers/gpu/drm/amd/include/kgd_pp_interface.h @@ -372,6 +372,8 @@ struct amd_pm_funcs { int (*print_clock_levels)(void *handle, enum pp_clock_type type, char *buf); int (*emit_clock_levels)(void *handle, enum pp_clock_type type, char *buf, int *offset); int (*force_performance_level)(void *handle, enum amd_dpm_forced_level level); + int (*reapply_performance_level)(void *handle, + enum amd_dpm_forced_level level); int (*get_sclk_od)(void *handle); int (*set_sclk_od)(void *handle, uint32_t value); int (*get_mclk_od)(void *handle); diff --git a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c index 0f37d5ccf246..e096efd77e73 100644 --- a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c +++ b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c @@ -974,6 +974,109 @@ enum amd_dpm_forced_level amdgpu_dpm_get_performance_level(struct amdgpu_device return level; } +static void amdgpu_dpm_enter_umd_state(struct amdgpu_device *adev) +{ + /* enter UMD Pstate */ + amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_GFX, + AMD_PG_STATE_UNGATE); + amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_GFX, + AMD_CG_STATE_UNGATE); +} + +static void amdgpu_dpm_exit_umd_state(struct amdgpu_device *adev) +{ + /* exit UMD Pstate */ + amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_GFX, + AMD_CG_STATE_GATE); + amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_GFX, + AMD_PG_STATE_GATE); +} + +/** + * amdgpu_dpm_reapply_umd_profile - Restore the active UMD profile + * @adev: amdgpu device + * + * Resume and reset retain the selected DPM level in software, but the + * underlying GFX and PMFW state is lost. Reapply a retained UMD profile + * after the common resume path has restored its default CG and PG state. + */ +int amdgpu_dpm_reapply_umd_profile(struct amdgpu_device *adev) +{ + const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs; + enum amd_dpm_forced_level level; + int ret, fallback_ret; + + if (!pp_funcs || !pp_funcs->reapply_performance_level || + !pp_funcs->force_performance_level) + return 0; + + /* Serialize the retained level with the GFX and PMFW reprogramming. */ + mutex_lock(&adev->pm.mutex); + if (adev->pm.dpm.thermal_active) { + dev_warn(adev->dev, + "Cannot restore UMD profile while thermal control is active\n"); + ret = 0; + goto unlock; + } + + if (pp_funcs->get_performance_level) + level = pp_funcs->get_performance_level(adev->powerplay.pp_handle); + else + level = adev->pm.dpm.forced_level; + + switch (level) { + case AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD: + case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK: + case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK: + case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK: + break; + default: + ret = 0; + goto unlock; + } + + /* The active profile already owns the GFX PG reference. */ + ret = amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_GFX, + AMD_CG_STATE_UNGATE); + if (ret) + goto fallback_auto; + + ret = amdgpu_asic_update_umd_stable_pstate(adev, true); + if (ret) + goto fallback_auto; + + ret = pp_funcs->reapply_performance_level(adev->powerplay.pp_handle, + level); + if (ret) + goto fallback_auto; + +unlock: + mutex_unlock(&adev->pm.mutex); + + return ret; + +fallback_auto: + dev_warn(adev->dev, "Failed to restore UMD profile state: %d\n", ret); + + /* Restore normal CG before releasing the profile's PG reference. */ + amdgpu_asic_update_umd_stable_pstate(adev, false); + amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_GFX, + AMD_CG_STATE_GATE); + mutex_unlock(&adev->pm.mutex); + + mutex_lock(&adev->pm.stable_pstate_ctx_lock); + fallback_ret = amdgpu_dpm_force_performance_level(adev, + AMD_DPM_FORCED_LEVEL_AUTO); + if (fallback_ret) + dev_warn(adev->dev, "Failed to fall back to AUTO performance level: %d\n", + fallback_ret); + else + adev->pm.stable_pstate_ctx = NULL; + mutex_unlock(&adev->pm.stable_pstate_ctx_lock); + + return 0; +} + int amdgpu_dpm_force_performance_level(struct amdgpu_device *adev, enum amd_dpm_forced_level level) { @@ -994,6 +1097,10 @@ int amdgpu_dpm_force_performance_level(struct amdgpu_device *adev, if (current_level == level) return 0; + if (!(current_level & profile_mode_mask) && + (level == AMD_DPM_FORCED_LEVEL_PROFILE_EXIT)) + return -EINVAL; + if (adev->asic_type == CHIP_RAVEN) { if (!(adev->apu_flags & AMD_APU_IS_RAVEN2)) { if (current_level != AMD_DPM_FORCED_LEVEL_MANUAL && @@ -1005,35 +1112,25 @@ int amdgpu_dpm_force_performance_level(struct amdgpu_device *adev, } } - if (!(current_level & profile_mode_mask) && - (level == AMD_DPM_FORCED_LEVEL_PROFILE_EXIT)) - return -EINVAL; - - if (!(current_level & profile_mode_mask) && - (level & profile_mode_mask)) { - /* enter UMD Pstate */ - amdgpu_device_ip_set_powergating_state(adev, - AMD_IP_BLOCK_TYPE_GFX, - AMD_PG_STATE_UNGATE); - amdgpu_device_ip_set_clockgating_state(adev, - AMD_IP_BLOCK_TYPE_GFX, - AMD_CG_STATE_UNGATE); - } else if ((current_level & profile_mode_mask) && - !(level & profile_mode_mask)) { - /* exit UMD Pstate */ - amdgpu_device_ip_set_clockgating_state(adev, - AMD_IP_BLOCK_TYPE_GFX, - AMD_CG_STATE_GATE); - amdgpu_device_ip_set_powergating_state(adev, - AMD_IP_BLOCK_TYPE_GFX, - AMD_PG_STATE_GATE); - } + if (!(current_level & profile_mode_mask) && (level & profile_mode_mask)) + amdgpu_dpm_enter_umd_state(adev); + else if ((current_level & profile_mode_mask) && + !(level & profile_mode_mask)) + amdgpu_dpm_exit_umd_state(adev); mutex_lock(&adev->pm.mutex); if (pp_funcs->force_performance_level(adev->powerplay.pp_handle, level)) { mutex_unlock(&adev->pm.mutex); + /* If new level failed, retain the umd state as before */ + if (!(current_level & profile_mode_mask) && + (level & profile_mode_mask)) + amdgpu_dpm_exit_umd_state(adev); + else if ((current_level & profile_mode_mask) && + !(level & profile_mode_mask)) + amdgpu_dpm_enter_umd_state(adev); + return -EINVAL; } diff --git a/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h b/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h index 3e27050ccd12..ff49e03626c6 100644 --- a/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h +++ b/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h @@ -480,6 +480,7 @@ void amdgpu_dpm_set_power_state(struct amdgpu_device *adev, enum amd_dpm_forced_level amdgpu_dpm_get_performance_level(struct amdgpu_device *adev); int amdgpu_dpm_force_performance_level(struct amdgpu_device *adev, enum amd_dpm_forced_level level); +int amdgpu_dpm_reapply_umd_profile(struct amdgpu_device *adev); int amdgpu_dpm_get_pp_num_states(struct amdgpu_device *adev, struct pp_states_info *states); int amdgpu_dpm_dispatch_task(struct amdgpu_device *adev, diff --git a/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c b/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c index b1526f80c972..58dc4a5920a5 100644 --- a/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c +++ b/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c @@ -383,24 +383,46 @@ static void pp_dpm_en_umd_pstate(struct pp_hwmgr *hwmgr, } } -static int pp_dpm_force_performance_level(void *handle, - enum amd_dpm_forced_level level) +static int pp_dpm_apply_performance_level(struct pp_hwmgr *hwmgr, + enum amd_dpm_forced_level level, + bool reapply) { - struct pp_hwmgr *hwmgr = handle; + int ret; if (!hwmgr || !hwmgr->pm_en) return -EINVAL; - if (level == hwmgr->dpm_level) + if (!reapply && level == hwmgr->dpm_level) return 0; pp_dpm_en_umd_pstate(hwmgr, &level); hwmgr->request_dpm_level = level; - hwmgr_handle_task(hwmgr, AMD_PP_TASK_READJUST_POWER_STATE, NULL); + ret = hwmgr_handle_task(hwmgr, AMD_PP_TASK_READJUST_POWER_STATE, NULL); + if (ret) + return ret; + + ret = phm_force_dpm_levels(hwmgr, level); + if (ret) + return ret; + + hwmgr->dpm_level = level; return 0; } +static int pp_dpm_force_performance_level(void *handle, + enum amd_dpm_forced_level level) +{ + return pp_dpm_apply_performance_level(handle, level, false); +} + +static int +pp_dpm_reapply_performance_level(void *handle, + enum amd_dpm_forced_level level) +{ + return pp_dpm_apply_performance_level(handle, level, true); +} + static enum amd_dpm_forced_level pp_dpm_get_performance_level( void *handle) { @@ -1583,6 +1605,7 @@ static const struct amd_pm_funcs pp_dpm_funcs = { .load_firmware = pp_dpm_load_fw, .wait_for_fw_loading_complete = pp_dpm_fw_loading_complete, .force_performance_level = pp_dpm_force_performance_level, + .reapply_performance_level = pp_dpm_reapply_performance_level, .get_performance_level = pp_dpm_get_performance_level, .get_current_power_state = pp_dpm_get_current_power_state, .dispatch_tasks = pp_dpm_dispatch_tasks, diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c index 83f666dfe7bd..1963653a5c60 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c +++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c @@ -2214,6 +2214,13 @@ static int smu_set_powergating_state(void *handle, return 0; } +static void smu_enter_umd_pstate(struct smu_context *smu) +{ + smu_gpo_control(smu, false); + smu_gfx_ulv_control(smu, false); + smu_deep_sleep_control(smu, false); +} + static int smu_enable_umd_pstate(void *handle, enum amd_dpm_forced_level *level) { @@ -2232,9 +2239,7 @@ static int smu_enable_umd_pstate(void *handle, /* enter umd pstate, save current level, disable gfx cg*/ if (*level & profile_mode_mask) { smu_dpm_ctx->saved_dpm_level = smu_dpm_ctx->dpm_level; - smu_gpo_control(smu, false); - smu_gfx_ulv_control(smu, false); - smu_deep_sleep_control(smu, false); + smu_enter_umd_pstate(smu); amdgpu_asic_update_umd_stable_pstate(smu->adev, true); } } else { @@ -2293,7 +2298,8 @@ static void smu_power_profile_mode_put(struct smu_context *smu, static int smu_adjust_power_state_dynamic(struct smu_context *smu, enum amd_dpm_forced_level level, - bool skip_display_settings) + bool skip_display_settings, + bool force_performance_level) { int ret = 0; struct smu_dpm_context *smu_dpm_ctx = &(smu->smu_dpm); @@ -2320,7 +2326,7 @@ static int smu_adjust_power_state_dynamic(struct smu_context *smu, } } - if (smu_dpm_ctx->dpm_level != level) { + if (force_performance_level || smu_dpm_ctx->dpm_level != level) { ret = smu_asic_set_performance_level(smu, level); if (ret) { dev_err(smu->adev->dev, "Failed to set performance level!"); @@ -2338,6 +2344,39 @@ static int smu_adjust_power_state_dynamic(struct smu_context *smu, return ret; } +static int +smu_reapply_power_state_dynamic(struct smu_context *smu, + enum amd_dpm_forced_level level) +{ + struct smu_dpm_context *smu_dpm_ctx = &smu->smu_dpm; + int ret; + + ret = smu_apply_clocks_adjust_rules(smu); + if (ret) { + dev_err(smu->adev->dev, "Failed to apply clocks adjust rules!"); + return ret; + } + + ret = smu_asic_set_performance_level(smu, level); + if (ret) { + if (ret == -EOPNOTSUPP) + dev_info(smu->adev->dev, "set performance level %d not supported", + level); + else + dev_err(smu->adev->dev, "Failed to set performance level %d", + level); + return ret; + } + + smu_dpm_ctx->dpm_level = level; + + if (smu_dpm_ctx->dpm_level == AMD_DPM_FORCED_LEVEL_MANUAL || + smu_dpm_ctx->dpm_level == AMD_DPM_FORCED_LEVEL_PERF_DETERMINISM) + return 0; + + return smu_bump_power_profile_mode(smu, NULL, 0); +} + static int smu_handle_task(struct smu_context *smu, enum amd_dpm_forced_level level, enum amd_pp_task task_id) @@ -2352,13 +2391,13 @@ static int smu_handle_task(struct smu_context *smu, ret = smu_pre_display_config_changed(smu); if (ret) return ret; - ret = smu_adjust_power_state_dynamic(smu, level, false); + ret = smu_adjust_power_state_dynamic(smu, level, false, false); break; case AMD_PP_TASK_COMPLETE_INIT: - ret = smu_adjust_power_state_dynamic(smu, level, true); + ret = smu_adjust_power_state_dynamic(smu, level, true, false); break; case AMD_PP_TASK_READJUST_POWER_STATE: - ret = smu_adjust_power_state_dynamic(smu, level, true); + ret = smu_adjust_power_state_dynamic(smu, level, true, false); break; default: break; @@ -2454,6 +2493,37 @@ static int smu_force_performance_level(void *handle, return ret; } +static int +smu_reapply_performance_level(void *handle, + enum amd_dpm_forced_level level) +{ + struct smu_context *smu = handle; + struct smu_dpm_context *smu_dpm_ctx = &smu->smu_dpm; + + if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled) + return -EOPNOTSUPP; + + if (!smu->is_apu && !smu_dpm_ctx->dpm_context) + return -EINVAL; + + switch (level) { + case AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD: + case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK: + case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK: + case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK: + break; + default: + return -EINVAL; + } + + if (smu_dpm_ctx->dpm_level != level) + return -EINVAL; + + smu_enter_umd_pstate(smu); + + return smu_reapply_power_state_dynamic(smu, level); +} + static int smu_set_display_count(void *handle, uint32_t count) { struct smu_context *smu = handle; @@ -3711,6 +3781,7 @@ static const struct amd_pm_funcs swsmu_pm_funcs = { .print_clock_levels = smu_print_ppclk_levels, .emit_clock_levels = smu_emit_ppclk_levels, .force_performance_level = smu_force_performance_level, + .reapply_performance_level = smu_reapply_performance_level, .read_sensor = smu_read_sensor, .get_apu_thermal_limit = smu_get_apu_thermal_limit, .set_apu_thermal_limit = smu_set_apu_thermal_limit, diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h index 0145aff45c38..9521be67373a 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h +++ b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h @@ -358,6 +358,8 @@ struct smu_table_context { void *boot_overdrive_table; void *user_overdrive_table; + uint32_t user_od_feature_mask; + uint32_t gpu_metrics_table_size; void *gpu_metrics_table; }; diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c index 6d178621ade1..5793f85d16fe 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c +++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c @@ -1610,7 +1610,6 @@ static int smu_v13_0_0_od_edit_dpm_table(struct smu_context *smu, struct amdgpu_device *adev = smu->adev; uint32_t offset_of_voltageoffset; int32_t minimum, maximum; - uint32_t feature_ctrlmask; int i, ret = 0; switch (type) { @@ -1890,21 +1889,16 @@ static int smu_v13_0_0_od_edit_dpm_table(struct smu_context *smu, if (ret) return ret; } else { - feature_ctrlmask = od_table->OverDriveTable.FeatureCtrlMask; memcpy(od_table, table_context->boot_overdrive_table, sizeof(OverDriveTableExternal_t)); - od_table->OverDriveTable.FeatureCtrlMask = feature_ctrlmask; + od_table->OverDriveTable.FeatureCtrlMask = 0; + table_context->user_od_feature_mask = 0; } fallthrough; case PP_OD_COMMIT_DPM_TABLE: - /* - * The member below instructs PMFW the settings focused in - * this single operation. - * `uint32_t FeatureCtrlMask;` - * It does not contain actual informations about user's custom - * settings. Thus we do not cache it. - */ + table_context->user_od_feature_mask = + od_table->OverDriveTable.FeatureCtrlMask; offset_of_voltageoffset = offsetof(OverDriveTable_t, VoltageOffsetPerZoneBoundary); if (memcmp((u8 *)od_table + offset_of_voltageoffset, table_context->user_overdrive_table + offset_of_voltageoffset, @@ -1917,7 +1911,6 @@ static int smu_v13_0_0_od_edit_dpm_table(struct smu_context *smu, return ret; } - od_table->OverDriveTable.FeatureCtrlMask = 0; memcpy(table_context->user_overdrive_table + offset_of_voltageoffset, (u8 *)od_table + offset_of_voltageoffset, sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset); @@ -2191,6 +2184,7 @@ static int smu_v13_0_0_set_default_od_settings(struct smu_context *smu) boot_od_table, sizeof(OverDriveTableExternal_t)); smu->user_dpm_profile.user_od = false; + smu->smu_table.user_od_feature_mask = 0; } else if (smu->user_dpm_profile.user_od) { memcpy(&user_od_table_bak, user_od_table, @@ -2236,11 +2230,8 @@ static int smu_v13_0_0_restore_user_od_settings(struct smu_context *smu) OverDriveTableExternal_t *od_table = table_context->overdrive_table; OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table; int res; - - user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) | - BIT(PP_OD_FEATURE_UCLK_BIT) | - BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) | - BIT(PP_OD_FEATURE_FAN_CURVE_BIT); + user_od_table->OverDriveTable.FeatureCtrlMask = + table_context->user_od_feature_mask; res = smu_v13_0_0_upload_overdrive_table(smu, user_od_table); user_od_table->OverDriveTable.FeatureCtrlMask = 0; if (res == 0) @@ -3060,8 +3051,10 @@ static int smu_v13_0_0_set_power_limit(struct smu_context *smu, if (limit <= msg_limit) { if (smu->current_power_limit > msg_limit) { od_table->OverDriveTable.Ppt = 0; - od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT; - + table_context->user_od_feature_mask &= + ~(1U << PP_OD_FEATURE_PPT_BIT); + od_table->OverDriveTable.FeatureCtrlMask = + table_context->user_od_feature_mask; ret = smu_v13_0_0_upload_overdrive_table(smu, od_table); if (ret) { dev_err(smu->adev->dev, "Failed to upload overdrive table!\n"); @@ -3075,8 +3068,9 @@ static int smu_v13_0_0_set_power_limit(struct smu_context *smu, return ret; od_table->OverDriveTable.Ppt = (limit * 100) / msg_limit - 100; - od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT; - + table_context->user_od_feature_mask |= 1U << PP_OD_FEATURE_PPT_BIT; + od_table->OverDriveTable.FeatureCtrlMask = + table_context->user_od_feature_mask; ret = smu_v13_0_0_upload_overdrive_table(smu, od_table); if (ret) { dev_err(smu->adev->dev, "Failed to upload overdrive table!\n"); diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c index 7a0e82a96a1c..9467db88969c 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c +++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c @@ -1598,7 +1598,6 @@ static int smu_v13_0_7_od_edit_dpm_table(struct smu_context *smu, struct amdgpu_device *adev = smu->adev; uint32_t offset_of_voltageoffset; int32_t minimum, maximum; - uint32_t feature_ctrlmask; int i, ret = 0; switch (type) { @@ -1878,22 +1877,17 @@ static int smu_v13_0_7_od_edit_dpm_table(struct smu_context *smu, if (ret) return ret; } else { - feature_ctrlmask = od_table->OverDriveTable.FeatureCtrlMask; memcpy(od_table, table_context->boot_overdrive_table, sizeof(OverDriveTableExternal_t)); - od_table->OverDriveTable.FeatureCtrlMask = feature_ctrlmask; + od_table->OverDriveTable.FeatureCtrlMask = 0; + table_context->user_od_feature_mask = 0; } fallthrough; case PP_OD_COMMIT_DPM_TABLE: - /* - * The member below instructs PMFW the settings focused in - * this single operation. - * `uint32_t FeatureCtrlMask;` - * It does not contain actual informations about user's custom - * settings. Thus we do not cache it. - */ + table_context->user_od_feature_mask = + od_table->OverDriveTable.FeatureCtrlMask; offset_of_voltageoffset = offsetof(OverDriveTable_t, VoltageOffsetPerZoneBoundary); if (memcmp((u8 *)od_table + offset_of_voltageoffset, table_context->user_overdrive_table + offset_of_voltageoffset, @@ -1906,7 +1900,6 @@ static int smu_v13_0_7_od_edit_dpm_table(struct smu_context *smu, return ret; } - od_table->OverDriveTable.FeatureCtrlMask = 0; memcpy(table_context->user_overdrive_table + offset_of_voltageoffset, (u8 *)od_table + offset_of_voltageoffset, sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset); @@ -2174,6 +2167,7 @@ static int smu_v13_0_7_set_default_od_settings(struct smu_context *smu) boot_od_table, sizeof(OverDriveTableExternal_t)); smu->user_dpm_profile.user_od = false; + smu->smu_table.user_od_feature_mask = 0; } else if (smu->user_dpm_profile.user_od) { memcpy(&user_od_table_bak, user_od_table, @@ -2220,10 +2214,8 @@ static int smu_v13_0_7_restore_user_od_settings(struct smu_context *smu) OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table; int res; - user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) | - BIT(PP_OD_FEATURE_UCLK_BIT) | - BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) | - BIT(PP_OD_FEATURE_FAN_CURVE_BIT); + user_od_table->OverDriveTable.FeatureCtrlMask = + table_context->user_od_feature_mask; res = smu_v13_0_7_upload_overdrive_table(smu, user_od_table); user_od_table->OverDriveTable.FeatureCtrlMask = 0; if (res == 0) @@ -2641,7 +2633,10 @@ static int smu_v13_0_7_set_power_limit(struct smu_context *smu, if (limit <= msg_limit) { if (smu->current_power_limit > msg_limit) { od_table->OverDriveTable.Ppt = 0; - od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT; + table_context->user_od_feature_mask &= + ~(1U << PP_OD_FEATURE_PPT_BIT); + od_table->OverDriveTable.FeatureCtrlMask = + table_context->user_od_feature_mask; ret = smu_v13_0_7_upload_overdrive_table(smu, od_table); if (ret) { @@ -2656,7 +2651,9 @@ static int smu_v13_0_7_set_power_limit(struct smu_context *smu, return ret; od_table->OverDriveTable.Ppt = (limit * 100) / msg_limit - 100; - od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT; + table_context->user_od_feature_mask |= 1U << PP_OD_FEATURE_PPT_BIT; + od_table->OverDriveTable.FeatureCtrlMask = + table_context->user_od_feature_mask; ret = smu_v13_0_7_upload_overdrive_table(smu, od_table); if (ret) { diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c index b2c32b2d2e51..33da78b53a86 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c +++ b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c @@ -2350,6 +2350,7 @@ static int smu_v14_0_2_set_default_od_settings(struct smu_context *smu) boot_od_table, sizeof(OverDriveTableExternal_t)); smu->user_dpm_profile.user_od = false; + smu->smu_table.user_od_feature_mask = 0; } else if (smu->user_dpm_profile.user_od) { memcpy(&user_od_table_bak, user_od_table, @@ -2394,11 +2395,8 @@ static int smu_v14_0_2_restore_user_od_settings(struct smu_context *smu) OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table; int res; - user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) | - BIT(PP_OD_FEATURE_UCLK_BIT) | - BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) | - BIT(PP_OD_FEATURE_FAN_CURVE_BIT) | - BIT(PP_OD_FEATURE_ZERO_FAN_BIT); + user_od_table->OverDriveTable.FeatureCtrlMask = + table_context->user_od_feature_mask; res = smu_v14_0_2_upload_overdrive_table(smu, user_od_table); user_od_table->OverDriveTable.FeatureCtrlMask = 0; if (res == 0) @@ -2473,7 +2471,6 @@ static int smu_v14_0_2_od_edit_dpm_table(struct smu_context *smu, struct amdgpu_device *adev = smu->adev; uint32_t offset_of_voltageoffset; int32_t minimum, maximum; - uint32_t feature_ctrlmask; int i, ret = 0; switch (type) { @@ -2720,21 +2717,26 @@ static int smu_v14_0_2_od_edit_dpm_table(struct smu_context *smu, if (ret) return ret; } else { - feature_ctrlmask = od_table->OverDriveTable.FeatureCtrlMask; memcpy(od_table, table_context->boot_overdrive_table, sizeof(OverDriveTableExternal_t)); - od_table->OverDriveTable.FeatureCtrlMask = feature_ctrlmask; + od_table->OverDriveTable.FeatureCtrlMask = 0; + table_context->user_od_feature_mask = 0; } fallthrough; case PP_OD_COMMIT_DPM_TABLE: /* - * The member below instructs PMFW the settings focused in - * this single operation. - * `uint32_t FeatureCtrlMask;` - * It does not contain actual informations about user's custom - * settings. Thus we do not cache it. + * PMFW interprets FeatureCtrlMask as the complete set of OD + * features that are currently overridden, not as a delta + * describing this single operation. Any feature whose bit is + * clear gets reset to its default on import. od_table's mask + * has accumulated every feature the user touched since the + * last full restore, so record it and keep restating it on + * subsequent uploads. */ + table_context->user_od_feature_mask = + od_table->OverDriveTable.FeatureCtrlMask; + offset_of_voltageoffset = offsetof(OverDriveTable_t, VoltageOffsetPerZoneBoundary); if (memcmp((u8 *)od_table + offset_of_voltageoffset, table_context->user_overdrive_table + offset_of_voltageoffset, @@ -2747,7 +2749,6 @@ static int smu_v14_0_2_od_edit_dpm_table(struct smu_context *smu, return ret; } - od_table->OverDriveTable.FeatureCtrlMask = 0; memcpy(table_context->user_overdrive_table + offset_of_voltageoffset, (u8 *)od_table + offset_of_voltageoffset, sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset); @@ -2785,7 +2786,10 @@ static int smu_v14_0_2_set_power_limit(struct smu_context *smu, if (limit <= msg_limit) { if (smu->current_power_limit > msg_limit) { od_table->OverDriveTable.Ppt = 0; - od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT; + table_context->user_od_feature_mask &= + ~(1U << PP_OD_FEATURE_PPT_BIT); + od_table->OverDriveTable.FeatureCtrlMask = + table_context->user_od_feature_mask; ret = smu_v14_0_2_upload_overdrive_table(smu, od_table); if (ret) { @@ -2800,7 +2804,9 @@ static int smu_v14_0_2_set_power_limit(struct smu_context *smu, return ret; od_table->OverDriveTable.Ppt = (limit * 100) / msg_limit - 100; - od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT; + table_context->user_od_feature_mask |= 1U << PP_OD_FEATURE_PPT_BIT; + od_table->OverDriveTable.FeatureCtrlMask = + table_context->user_od_feature_mask; ret = smu_v14_0_2_upload_overdrive_table(smu, od_table); if (ret) { diff --git a/drivers/gpu/drm/bridge/nwl-dsi.c b/drivers/gpu/drm/bridge/nwl-dsi.c index 5f05647a3bea..00622febf01c 100644 --- a/drivers/gpu/drm/bridge/nwl-dsi.c +++ b/drivers/gpu/drm/bridge/nwl-dsi.c @@ -110,6 +110,8 @@ struct nwl_dsi { int error; struct nwl_dsi_transfer *xfer; + + unsigned int endpoint; }; static const struct regmap_config nwl_dsi_regmap_config = { @@ -822,10 +824,13 @@ static int nwl_dsi_bridge_atomic_check(struct drm_bridge *bridge, struct drm_connector_state *conn_state) { struct drm_display_mode *adjusted_mode = &crtc_state->adjusted_mode; + struct nwl_dsi *dsi = bridge_to_dsi(bridge); /* At least LCDIF + NWL needs active high sync */ - adjusted_mode->flags |= (DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_PVSYNC); - adjusted_mode->flags &= ~(DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC); + if (dsi->endpoint == NWL_DSI_ENDPOINT_LCDIF) { + adjusted_mode->flags |= (DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_PVSYNC); + adjusted_mode->flags &= ~(DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC); + } /* * Do a full modeset if crtc_state->active is changed to be true. @@ -1089,13 +1094,12 @@ static int nwl_dsi_parse_dt(struct nwl_dsi *dsi) static int nwl_dsi_select_input(struct nwl_dsi *dsi) { struct device_node *remote; - u32 use_dcss = 1; int ret; remote = of_graph_get_remote_node(dsi->dev->of_node, 0, NWL_DSI_ENDPOINT_LCDIF); if (remote) { - use_dcss = 0; + dsi->endpoint = NWL_DSI_ENDPOINT_LCDIF; } else { remote = of_graph_get_remote_node(dsi->dev->of_node, 0, NWL_DSI_ENDPOINT_DCSS); @@ -1104,11 +1108,12 @@ static int nwl_dsi_select_input(struct nwl_dsi *dsi) "No valid input endpoint found\n"); return -EINVAL; } + dsi->endpoint = NWL_DSI_ENDPOINT_DCSS; } DRM_DEV_INFO(dsi->dev, "Using %s as input source\n", - (use_dcss) ? "DCSS" : "LCDIF"); - ret = mux_control_try_select(dsi->mux, use_dcss); + (dsi->endpoint == NWL_DSI_ENDPOINT_DCSS) ? "DCSS" : "LCDIF"); + ret = mux_control_try_select(dsi->mux, dsi->endpoint); if (ret < 0) DRM_DEV_ERROR(dsi->dev, "Failed to select input: %d\n", ret); diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c index 3488ff067c69..fdeed38cec03 100644 --- a/drivers/gpu/drm/drm_crtc.c +++ b/drivers/gpu/drm/drm_crtc.c @@ -476,14 +476,23 @@ EXPORT_SYMBOL(__drmm_crtc_alloc_with_planes); * drm_crtc_cleanup - Clean up the core crtc usage * @crtc: CRTC to cleanup * - * This function cleans up @crtc and removes it from the DRM mode setting - * core. Note that the function does *not* free the crtc structure itself, - * this is the responsibility of the caller. + * This function cleans up @crtc and removes it from the DRM mode setting core, + * after first waiting an RCU grace period to ensure @crtc->dev can safely be + * dereferenced by our dma_fence_ops. + * + * Note that the function does *not* free the crtc structure itself, this is the + * responsibility of the caller. */ void drm_crtc_cleanup(struct drm_crtc *crtc) { struct drm_device *dev = crtc->dev; + /* Ensure our dma_fence_ops remain valid for an RCU grace period after + * the fence is signaled. This is necessary because our dma_fence_ops + * dereference crtc->dev. + */ + synchronize_rcu(); + /* Note that the crtc_list is considered to be static; should we * remove the drm_crtc at runtime we would have to decrement all * the indices on the drm_crtc after us in the crtc_list. diff --git a/drivers/gpu/drm/drm_panel.c b/drivers/gpu/drm/drm_panel.c index fd8fa2e0ef6f..7443aaacdd6c 100644 --- a/drivers/gpu/drm/drm_panel.c +++ b/drivers/gpu/drm/drm_panel.c @@ -316,6 +316,31 @@ int drm_panel_get_modes(struct drm_panel *panel, } EXPORT_SYMBOL(drm_panel_get_modes); +void *__devm_drm_panel_alloc(struct device *dev, size_t size, size_t offset, + const struct drm_panel_funcs *funcs, + int connector_type) +{ + void *container; + struct drm_panel *panel; + + if (!funcs) { + dev_warn(dev, "Missing funcs pointer\n"); + return ERR_PTR(-EINVAL); + } + + container = devm_kzalloc(dev, size, GFP_KERNEL); + if (!container) + return ERR_PTR(-ENOMEM); + + panel = container + offset; + panel->funcs = funcs; + + drm_panel_init(panel, dev, funcs, connector_type); + + return container; +} +EXPORT_SYMBOL(__devm_drm_panel_alloc); + #ifdef CONFIG_OF /** * of_drm_find_panel - look up a panel using a device tree node diff --git a/drivers/gpu/drm/drm_panic.c b/drivers/gpu/drm/drm_panic.c index 0aa87eafdacd..f4662f88e217 100644 --- a/drivers/gpu/drm/drm_panic.c +++ b/drivers/gpu/drm/drm_panic.c @@ -395,7 +395,8 @@ static void draw_panic_static_kmsg(struct drm_scanout_buffer *sb) const struct font_desc *font = get_default_font(sb->width, sb->height, NULL, NULL); struct drm_rect r_screen = DRM_RECT_INIT(0, 0, sb->width, sb->height); struct kmsg_dump_iter iter; - char kmsg_buf[512]; + /* the size printk formats one record into, too big for the stack */ + static char kmsg_buf[2048]; size_t kmsg_len; struct drm_panic_line line; int yoffset; @@ -413,6 +414,10 @@ static void draw_panic_static_kmsg(struct drm_scanout_buffer *sb) char *start; char *end; + /* a record that does not fit comes back empty, every time */ + if (!kmsg_len) + break; + /* ignore terminating NUL and newline */ start = kmsg_buf + kmsg_len - 2; end = kmsg_buf + kmsg_len - 1; @@ -502,7 +507,8 @@ static int drm_panic_get_qr_code_url(u8 **qr_image) char *kmsg; int max_qr_data_size, url_len; - url_len = snprintf(url, sizeof(url), CONFIG_DRM_PANIC_SCREEN_QR_CODE_URL "?a=%s&v=%s&zl=", + url_len = snprintf(url, sizeof(url), "%s?a=%s&v=%s&zl=", + CONFIG_DRM_PANIC_SCREEN_QR_CODE_URL, utsname()->machine, utsname()->release); max_qr_data_size = drm_panic_qr_max_data_size(panic_qr_version, url_len); diff --git a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c index 88385dc3b30d..4f22e373ed34 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c @@ -230,6 +230,17 @@ u16 etnaviv_buffer_config_pta(struct etnaviv_gpu *gpu, unsigned short id) return buffer->user_size / 8; } +static void etnaviv_cmd_flush_ts(struct etnaviv_cmdbuf *buffer, bool has_blt) +{ + CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE, VIVS_TS_FLUSH_CACHE_FLUSH); + + if (has_blt) { + CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1); + CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1); + CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0); + } +} + void etnaviv_buffer_end(struct etnaviv_gpu *gpu) { struct etnaviv_cmdbuf *buffer = &gpu->buffer; @@ -253,7 +264,7 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu) unsigned int dwords = 7; if (has_blt) - dwords += 10; + dwords += 11; link_target = etnaviv_buffer_reserve(gpu, buffer, dwords); @@ -266,16 +277,8 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu) CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0); } CMD_LOAD_STATE(buffer, VIVS_GL_FLUSH_CACHE, flush); - if (gpu->exec_state == ETNA_PIPE_3D) { - if (has_blt) { - CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1); - CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1); - CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0); - } else { - CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE, - VIVS_TS_FLUSH_CACHE_FLUSH); - } - } + if (gpu->exec_state == ETNA_PIPE_3D) + etnaviv_cmd_flush_ts(buffer, has_blt); CMD_SEM(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE); CMD_STALL(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE); if (has_blt) { @@ -462,12 +465,12 @@ void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, u32 exec_state, return_dwords = 7; /* - * When the BLT engine is present we need 6 more dwords in the return + * When the BLT engine is present we need 7 more dwords in the return * target: 3 enable/flush/disable + 4 enable/semaphore stall/disable, - * but we don't need the normal TS flush state. + * on top of the normal TS flush state. */ if (has_blt) - return_dwords += 6; + return_dwords += 7; return_target = etnaviv_buffer_reserve(gpu, buffer, return_dwords); CMD_LINK(cmdbuf, return_dwords, return_target); @@ -484,14 +487,7 @@ void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, u32 exec_state, VIVS_GL_FLUSH_CACHE_DEPTH | VIVS_GL_FLUSH_CACHE_COLOR | VIVS_GL_FLUSH_CACHE_SHADER_L1); - if (has_blt) { - CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1); - CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1); - CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0); - } else { - CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE, - VIVS_TS_FLUSH_CACHE_FLUSH); - } + etnaviv_cmd_flush_ts(buffer, has_blt); } CMD_SEM(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE); CMD_STALL(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE); diff --git a/drivers/gpu/drm/i915/display/intel_psr.c b/drivers/gpu/drm/i915/display/intel_psr.c index df16587ea964..97f82949a3a5 100644 --- a/drivers/gpu/drm/i915/display/intel_psr.c +++ b/drivers/gpu/drm/i915/display/intel_psr.c @@ -2406,23 +2406,25 @@ static u32 psr2_pipe_srcsz_early_tpt_calc(struct intel_crtc_state *crtc_state, } static void clip_area_update(struct drm_rect *overlap_damage_area, - struct drm_rect *damage_area, - struct drm_rect *display_area) + const struct drm_rect *damage_area, + const struct drm_rect *display_area) { - if (!drm_rect_intersect(damage_area, display_area)) + struct drm_rect inter = *damage_area; + + if (!drm_rect_intersect(&inter, display_area)) return; if (overlap_damage_area->y1 == -1) { - overlap_damage_area->y1 = damage_area->y1; - overlap_damage_area->y2 = damage_area->y2; + overlap_damage_area->y1 = inter.y1; + overlap_damage_area->y2 = inter.y2; return; } - if (damage_area->y1 < overlap_damage_area->y1) - overlap_damage_area->y1 = damage_area->y1; + if (inter.y1 < overlap_damage_area->y1) + overlap_damage_area->y1 = inter.y1; - if (damage_area->y2 > overlap_damage_area->y2) - overlap_damage_area->y2 = damage_area->y2; + if (inter.y2 > overlap_damage_area->y2) + overlap_damage_area->y2 = inter.y2; } static bool intel_psr2_sel_fetch_pipe_alignment(struct intel_crtc_state *crtc_state) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c index dbe9fc1e8a10..a89e61dd3940 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c @@ -2789,7 +2789,6 @@ add_timeline_fence_array(struct i915_execbuffer *eb, u64 __user *user_values; struct eb_fence *f; u64 nfences; - int err = 0; nfences = timeline_fences->fence_count; if (!nfences) @@ -2827,6 +2826,7 @@ add_timeline_fence_array(struct i915_execbuffer *eb, struct drm_syncobj *syncobj; struct dma_fence *fence = NULL; u64 point; + int err = 0; if (__copy_from_user(&user_fence, user_fences++, diff --git a/drivers/gpu/drm/loongson/lsdc_gem.c b/drivers/gpu/drm/loongson/lsdc_gem.c index a720d8f53209..5eb6b4f92302 100644 --- a/drivers/gpu/drm/loongson/lsdc_gem.c +++ b/drivers/gpu/drm/loongson/lsdc_gem.c @@ -155,14 +155,18 @@ struct drm_gem_object *lsdc_gem_object_create(struct drm_device *ddev, return ERR_PTR(ret); } + gobj = &lbo->tbo.base; + gobj->funcs = &lsdc_gem_object_funcs; + if (!sg) { /* VRAM is filled with random data */ - lsdc_bo_clear(lbo); + ret = lsdc_bo_clear(lbo); + if (ret) { + drm_gem_object_put(gobj); + return ERR_PTR(ret); + } } - gobj = &lbo->tbo.base; - gobj->funcs = &lsdc_gem_object_funcs; - /* tracking the BOs we created */ mutex_lock(&ldev->gem.mutex); list_add_tail(&lbo->list, &ldev->gem.objects); diff --git a/drivers/gpu/drm/loongson/lsdc_plane.c b/drivers/gpu/drm/loongson/lsdc_plane.c index d227a2c1dcf1..df6cc5d11d79 100644 --- a/drivers/gpu/drm/loongson/lsdc_plane.c +++ b/drivers/gpu/drm/loongson/lsdc_plane.c @@ -227,7 +227,7 @@ static void lsdc_cursor_plane_atomic_async_update(struct drm_plane *plane, new_state = drm_atomic_get_new_plane_state(state, plane); - new_fb = plane->state->fb; + new_fb = new_state->fb; plane->state->crtc_x = new_state->crtc_x; plane->state->crtc_y = new_state->crtc_y; diff --git a/drivers/gpu/drm/loongson/lsdc_ttm.c b/drivers/gpu/drm/loongson/lsdc_ttm.c index 2e42c6970c9f..17b1998254e2 100644 --- a/drivers/gpu/drm/loongson/lsdc_ttm.c +++ b/drivers/gpu/drm/loongson/lsdc_ttm.c @@ -387,9 +387,13 @@ void lsdc_bo_kunmap(struct lsdc_bo *lbo) ttm_bo_kunmap(&lbo->kmap); } -void lsdc_bo_clear(struct lsdc_bo *lbo) +int lsdc_bo_clear(struct lsdc_bo *lbo) { - lsdc_bo_kmap(lbo); + int ret; + + ret = lsdc_bo_kmap(lbo); + if (ret) + return ret; if (lbo->is_iomem) memset_io((void __iomem *)lbo->kptr, 0, lbo->size); @@ -397,6 +401,8 @@ void lsdc_bo_clear(struct lsdc_bo *lbo) memset(lbo->kptr, 0, lbo->size); lsdc_bo_kunmap(lbo); + + return 0; } int lsdc_bo_evict_vram(struct drm_device *ddev) @@ -474,10 +480,8 @@ struct lsdc_bo *lsdc_bo_create(struct drm_device *ddev, ret = ttm_bo_init_validate(bdev, tbo, bo_type, &lbo->placement, 0, false, sg, resv, lsdc_bo_destroy); - if (ret) { - kfree(lbo); + if (ret) return ERR_PTR(ret); - } return lbo; } diff --git a/drivers/gpu/drm/loongson/lsdc_ttm.h b/drivers/gpu/drm/loongson/lsdc_ttm.h index 843e1475064e..df47d9bb0d03 100644 --- a/drivers/gpu/drm/loongson/lsdc_ttm.h +++ b/drivers/gpu/drm/loongson/lsdc_ttm.h @@ -89,7 +89,7 @@ size_t lsdc_bo_size(struct lsdc_bo *lbo); int lsdc_bo_kmap(struct lsdc_bo *lbo); void lsdc_bo_kunmap(struct lsdc_bo *lbo); -void lsdc_bo_clear(struct lsdc_bo *lbo); +int lsdc_bo_clear(struct lsdc_bo *lbo); int lsdc_bo_evict_vram(struct drm_device *ddev); diff --git a/drivers/gpu/drm/msm/dsi/dsi_host.c b/drivers/gpu/drm/msm/dsi/dsi_host.c index cd605c75cef8..ebd9dfca6459 100644 --- a/drivers/gpu/drm/msm/dsi/dsi_host.c +++ b/drivers/gpu/drm/msm/dsi/dsi_host.c @@ -1205,6 +1205,8 @@ int dsi_tx_buf_alloc_6g(struct msm_dsi_host *msm_host, int size) if (IS_ERR(data)) { msm_host->tx_gem_obj = NULL; + msm_gem_address_space_put(msm_host->aspace); + msm_host->aspace = NULL; return PTR_ERR(data); } diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h index 3c103101d5fc..f21d6f1cf4fe 100644 --- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h +++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h @@ -46,6 +46,7 @@ int nv4c_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct n int g84_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **); int g92_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **); int g94_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **); +int mcp79_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **); int gf100_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **); int gf106_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **); int gk104_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **); diff --git a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c index 6c20e827a069..3ceca50fd8d4 100644 --- a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c +++ b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c @@ -145,6 +145,7 @@ nvkm_subdev_init_(struct nvkm_subdev *subdev) ret = subdev->func->init(subdev); if (ret) { nvkm_error(subdev, "init failed, %d\n", ret); + subdev->use.enabled = false; return ret; } } diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c index 9093d89b16f3..6f13b8b0fc77 100644 --- a/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c +++ b/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c @@ -1237,7 +1237,7 @@ nvac_chipset = { .mc = { 0x00000001, g98_mc_new }, .mmu = { 0x00000001, mcp77_mmu_new }, .mxm = { 0x00000001, nv50_mxm_new }, - .pci = { 0x00000001, g94_pci_new }, + .pci = { 0x00000001, mcp79_pci_new }, .therm = { 0x00000001, g84_therm_new }, .timer = { 0x00000001, nv41_timer_new }, .volt = { 0x00000001, nv40_volt_new }, diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c index 5368e705e7fd..186e09ea0cb1 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c @@ -27,6 +27,7 @@ #include #include #include +#include int gf100_devinit_pll_set(struct nvkm_devinit *init, u32 type, u32 freq) @@ -99,6 +100,25 @@ gf100_devinit_preinit(struct nvkm_devinit *base) * can use it as a reliable way to know whether we should run devinit. */ base->post = ((nvkm_rd32(device, 0x2240c) & BIT(1)) == 0); + + /* + * However some VBIOS init sequences miss this bit. So fall back to + * the nv50 method of checking if the bit is not set. Running the + * VBIOS multiple times may have detrimental effects. + */ + if (base->post) { + u64 disable = nvkm_devinit_disable(base); + /* magic to detect whether or not x86 vbios code has + * executed the devinit scripts to initialise the + * board. only works if there's a display engine. + */ + if (!(disable & (1ULL << NVKM_ENGINE_DISP))) { + if (nvkm_rdvgac(device, 0, 0x00) || + nvkm_rdvgac(device, 0, 0x1a)) { + base->post = false; + } + } + } } static const struct nvkm_devinit_func diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pci/Kbuild b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/Kbuild index 174bdf995271..c5911d7d08ea 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/pci/Kbuild +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/Kbuild @@ -9,6 +9,7 @@ nvkm-y += nvkm/subdev/pci/nv4c.o nvkm-y += nvkm/subdev/pci/g84.o nvkm-y += nvkm/subdev/pci/g92.o nvkm-y += nvkm/subdev/pci/g94.o +nvkm-y += nvkm/subdev/pci/mcp79.o nvkm-y += nvkm/subdev/pci/gf100.o nvkm-y += nvkm/subdev/pci/gf106.o nvkm-y += nvkm/subdev/pci/gk104.o diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pci/mcp79.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/mcp79.c new file mode 100644 index 000000000000..98d85a856b88 --- /dev/null +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/mcp79.c @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: MIT +/* + * Copyright 2026 Marek Czernohous + * + * MCP79/MCP7A (NVAC): like g94, but MSI re-arm goes through real PCI + * config space. The MMIO-mirror re-arm is unreliable on this IGP and a + * missed re-arm kills the interrupt line (see the nv46 comment; g92 + * already re-arms through config space for the same reason). + */ +#include "priv.h" + +static const struct nvkm_pci_func +mcp79_pci_func = { + .init = g84_pci_init, + .rd32 = nv40_pci_rd32, + .wr08 = nv40_pci_wr08, + .wr32 = nv40_pci_wr32, + .msi_rearm = nv46_pci_msi_rearm, + + .pcie.init = g84_pcie_init, + .pcie.set_link = g84_pcie_set_link, + + .pcie.max_speed = g84_pcie_max_speed, + .pcie.cur_speed = g84_pcie_cur_speed, + + .pcie.set_version = g84_pcie_set_version, + .pcie.version = g84_pcie_version, + .pcie.version_supported = g92_pcie_version_supported, +}; + +int +mcp79_pci_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst, + struct nvkm_pci **ppci) +{ + return nvkm_pci_new_(&mcp79_pci_func, device, type, inst, ppci); +} diff --git a/drivers/gpu/drm/panel/panel-ebbg-ft8719.c b/drivers/gpu/drm/panel/panel-ebbg-ft8719.c index e85d63a176d0..c97825177693 100644 --- a/drivers/gpu/drm/panel/panel-ebbg-ft8719.c +++ b/drivers/gpu/drm/panel/panel-ebbg-ft8719.c @@ -198,9 +198,11 @@ static int ebbg_ft8719_probe(struct mipi_dsi_device *dsi) struct ebbg_ft8719 *ctx; int i, ret; - ctx = devm_kzalloc(dev, sizeof(*ctx), GFP_KERNEL); - if (!ctx) - return -ENOMEM; + ctx = devm_drm_panel_alloc(dev, struct ebbg_ft8719, panel, + &ebbg_ft8719_panel_funcs, + DRM_MODE_CONNECTOR_DSI); + if (IS_ERR(ctx)) + return PTR_ERR(ctx); for (i = 0; i < ARRAY_SIZE(ctx->supplies); i++) ctx->supplies[i].supply = regulator_names[i]; @@ -231,8 +233,7 @@ static int ebbg_ft8719_probe(struct mipi_dsi_device *dsi) dsi->mode_flags = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_BURST | MIPI_DSI_CLOCK_NON_CONTINUOUS; - drm_panel_init(&ctx->panel, dev, &ebbg_ft8719_panel_funcs, - DRM_MODE_CONNECTOR_DSI); + ctx->panel.prepare_prev_first = true; ret = drm_panel_of_backlight(&ctx->panel); if (ret) diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c index 70bd566cf6d4..2dd678fd8607 100644 --- a/drivers/gpu/drm/v3d/v3d_sched.c +++ b/drivers/gpu/drm/v3d/v3d_sched.c @@ -395,6 +395,8 @@ v3d_csd_job_run(struct drm_sched_job *sched_job) dma_fence_put(job->base.irq_fence); job->base.irq_fence = dma_fence_get(fence); + job->timedout_batches = job->args.cfg[4]; + trace_v3d_submit_csd(dev, to_v3d_fence(fence)->seqno); v3d_job_start_stats(&job->base, V3D_CSD); diff --git a/drivers/gpu/drm/vc4/vc4_gem.c b/drivers/gpu/drm/vc4/vc4_gem.c index bec799d41d62..f7df302e5c67 100644 --- a/drivers/gpu/drm/vc4/vc4_gem.c +++ b/drivers/gpu/drm/vc4/vc4_gem.c @@ -706,8 +706,11 @@ vc4_queue_submit(struct drm_device *dev, struct vc4_exec_info *exec, fence->seqno = exec->seqno; exec->fence = &fence->base; + /* The job can complete and drop exec->fence as soon as job_lock is + * released, so hold our own reference. + */ if (out_sync) - drm_syncobj_replace_fence(out_sync, exec->fence); + dma_fence_get(&fence->base); vc4_update_bo_seqnos(exec, seqno); @@ -729,6 +732,11 @@ vc4_queue_submit(struct drm_device *dev, struct vc4_exec_info *exec, spin_unlock_irqrestore(&vc4->job_lock, irqflags); + if (out_sync) { + drm_syncobj_replace_fence(out_sync, &fence->base); + dma_fence_put(&fence->base); + } + return 0; } diff --git a/drivers/gpu/drm/vc4/vc4_hdmi.c b/drivers/gpu/drm/vc4/vc4_hdmi.c index 7ea1622377b5..2fa0e0f4bcd2 100644 --- a/drivers/gpu/drm/vc4/vc4_hdmi.c +++ b/drivers/gpu/drm/vc4/vc4_hdmi.c @@ -806,8 +806,7 @@ static void vc4_hdmi_disable_scrambling(struct drm_encoder *encoder) vc4_hdmi->scdc_enabled = false; - if (delayed_work_pending(&vc4_hdmi->scrambling_work)) - cancel_delayed_work_sync(&vc4_hdmi->scrambling_work); + cancel_delayed_work_sync(&vc4_hdmi->scrambling_work); if (!drm_dev_enter(drm, &idx)) return; @@ -870,8 +869,6 @@ static void vc4_hdmi_encoder_post_crtc_disable(struct drm_encoder *encoder, HDMI_READ(HDMI_VID_CTL) & ~VC4_HD_VID_CTL_ENABLE); spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags); - vc4_hdmi_disable_scrambling(encoder); - drm_dev_exit(idx); out: @@ -900,6 +897,8 @@ static void vc4_hdmi_encoder_post_crtc_powerdown(struct drm_encoder *encoder, if (vc4_hdmi->variant->phy_disable) vc4_hdmi->variant->phy_disable(vc4_hdmi); + vc4_hdmi_disable_scrambling(encoder); + clk_disable_unprepare(vc4_hdmi->pixel_bvb_clock); clk_disable_unprepare(vc4_hdmi->pixel_clock); @@ -1510,6 +1509,8 @@ static void vc4_hdmi_encoder_pre_crtc_configure(struct drm_encoder *encoder, goto err_disable_pixel_clock; } + vc4_hdmi_enable_scrambling(encoder); + if (vc4_hdmi->variant->phy_init) vc4_hdmi->variant->phy_init(vc4_hdmi, conn_state); @@ -1641,7 +1642,6 @@ static void vc4_hdmi_encoder_post_crtc_enable(struct drm_encoder *encoder, } vc4_hdmi_recenter_fifo(vc4_hdmi); - vc4_hdmi_enable_scrambling(encoder); drm_dev_exit(idx); diff --git a/drivers/hte/hte-tegra194-test.c b/drivers/hte/hte-tegra194-test.c index df631b5100d2..0bec696774c1 100644 --- a/drivers/hte/hte-tegra194-test.c +++ b/drivers/hte/hte-tegra194-test.c @@ -218,10 +218,10 @@ static void tegra_hte_test_remove(struct platform_device *pdev) { (void)pdev; + timer_shutdown_sync(&hte.timer); free_irq(hte.gpio_in_irq, &hte); gpiod_put(hte.gpio_in); gpiod_put(hte.gpio_out); - del_timer_sync(&hte.timer); } static struct platform_driver tegra_hte_test_driver = { diff --git a/drivers/hwmon/acpi_power_meter.c b/drivers/hwmon/acpi_power_meter.c index 6c8a9c863528..eedcfc9d96c6 100644 --- a/drivers/hwmon/acpi_power_meter.c +++ b/drivers/hwmon/acpi_power_meter.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -31,7 +32,12 @@ #define POWER_METER_CAN_NOTIFY (1 << 3) #define POWER_METER_IS_BATTERY (1 << 8) #define UNKNOWN_HYSTERESIS 0xFFFFFFFF -#define UNKNOWN_POWER 0xFFFFFFFF +/* + * ACPI integers can either be 32-bit or 64-bit wide, so + * we have to check for both values. + */ +#define UNKNOWN_POWER_32 U32_MAX +#define UNKNOWN_POWER_64 U64_MAX #define METER_NOTIFY_CONFIG 0x80 #define METER_NOTIFY_TRIP 0x81 @@ -349,7 +355,7 @@ static ssize_t show_power(struct device *dev, update_meter(resource); mutex_unlock(&resource->lock); - if (resource->power == UNKNOWN_POWER) + if (resource->power == UNKNOWN_POWER_32 || resource->power == UNKNOWN_POWER_64) return -ENODATA; return sprintf(buf, "%llu\n", resource->power * 1000); diff --git a/drivers/hwmon/aquacomputer_d5next.c b/drivers/hwmon/aquacomputer_d5next.c index 0dcb8a3a691d..95ec74c6e67e 100644 --- a/drivers/hwmon/aquacomputer_d5next.c +++ b/drivers/hwmon/aquacomputer_d5next.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -133,6 +134,7 @@ static u8 aquaero_secondary_ctrl_report[] = { #define AQUAERO_FAN_CURRENT_OFFSET 0x06 #define AQUAERO_FAN_POWER_OFFSET 0x08 #define AQUAERO_FAN_SPEED_OFFSET 0x00 +#define AQUAERO_MIN_STATUS_REPORT_SIZE (0x18B + AQUAERO_FAN_POWER_OFFSET + sizeof(u16)) static u16 aquaero_sensor_fan_offsets[] = { 0x167, 0x173, 0x17f, 0x18B }; /* Control report offsets for the Aquaero fan controllers */ @@ -156,6 +158,8 @@ static u16 aquaero_ctrl_fan_offsets[] = { 0x20c, 0x220, 0x234, 0x248 }; #define D5NEXT_5V_VOLTAGE 0x39 #define D5NEXT_12V_VOLTAGE 0x37 #define D5NEXT_VIRTUAL_SENSORS_START 0x3f +#define D5NEXT_MIN_STATUS_REPORT_SIZE (D5NEXT_PUMP_OFFSET + AQC_FAN_SPEED_OFFSET + \ + sizeof(u16)) static u16 d5next_sensor_fan_offsets[] = { D5NEXT_PUMP_OFFSET, D5NEXT_FAN_OFFSET }; /* Control report offsets for the D5 Next pump */ @@ -180,11 +184,14 @@ static u16 d5next_ctrl_fan_offsets[] = { 0x97, 0x42 }; /* Pump and fan speed (fr #define AQUASTREAMULT_FAN_CURRENT_OFFSET 0x00 #define AQUASTREAMULT_FAN_POWER_OFFSET 0x04 #define AQUASTREAMULT_FAN_SPEED_OFFSET 0x06 +#define AQUASTREAMULT_MIN_STATUS_REPORT_SIZE (AQUASTREAMULT_PRESSURE_OFFSET + sizeof(u16)) static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET }; /* Spec and sensor report offset for the Farbwerk RGB controller */ #define FARBWERK_NUM_SENSORS 4 #define FARBWERK_SENSOR_START 0x2f +#define FARBWERK_MIN_STATUS_REPORT_SIZE (FARBWERK_SENSOR_START + \ + FARBWERK_NUM_SENSORS * AQC_SENSOR_SIZE) /* Specs of the Farbwerk 360 RGB controller */ #define FARBWERK360_NUM_SENSORS 4 @@ -194,6 +201,8 @@ static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET }; /* Sensor report offsets for the Farbwerk 360 */ #define FARBWERK360_SENSOR_START 0x32 #define FARBWERK360_VIRTUAL_SENSORS_START 0x3a +#define FARBWERK360_MIN_STATUS_REPORT_SIZE (FARBWERK360_VIRTUAL_SENSORS_START + \ + FARBWERK360_NUM_VIRTUAL_SENSORS * AQC_SENSOR_SIZE) /* Control report offsets for the Farbwerk 360 */ #define FARBWERK360_TEMP_CTRL_OFFSET 0x8 @@ -210,6 +219,7 @@ static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET }; #define OCTO_SENSOR_START 0x3D #define OCTO_VIRTUAL_SENSORS_START 0x45 #define OCTO_FLOW_SENSOR_OFFSET 0x7B +#define OCTO_MIN_STATUS_REPORT_SIZE (0xD8 + AQC_FAN_SPEED_OFFSET + sizeof(u16)) static u16 octo_sensor_fan_offsets[] = { 0x7D, 0x8A, 0x97, 0xA4, 0xB1, 0xBE, 0xCB, 0xD8 }; /* Control report offsets for the Octo */ @@ -230,6 +240,7 @@ static u16 octo_ctrl_fan_offsets[] = { 0x5B, 0xB0, 0x105, 0x15A, 0x1AF, 0x204, 0 #define QUADRO_SENSOR_START 0x34 #define QUADRO_VIRTUAL_SENSORS_START 0x3c #define QUADRO_FLOW_SENSOR_OFFSET 0x6e +#define QUADRO_MIN_STATUS_REPORT_SIZE (0x97 + AQC_FAN_SPEED_OFFSET + sizeof(u16)) static u16 quadro_sensor_fan_offsets[] = { 0x70, 0x7D, 0x8A, 0x97 }; /* Control report offsets for the Quadro */ @@ -249,6 +260,7 @@ static u16 quadro_ctrl_fan_offsets[] = { 0x37, 0x8c, 0xe1, 0x136 }; /* Fan speed #define HIGHFLOWNEXT_CONDUCTIVITY 95 #define HIGHFLOWNEXT_5V_VOLTAGE 97 #define HIGHFLOWNEXT_5V_VOLTAGE_USB 99 +#define HIGHFLOWNEXT_MIN_STATUS_REPORT_SIZE (HIGHFLOWNEXT_5V_VOLTAGE_USB + sizeof(u16)) /* Specs of the Leakshield */ #define LEAKSHIELD_NUM_SENSORS 2 @@ -264,6 +276,7 @@ static u16 quadro_ctrl_fan_offsets[] = { 0x37, 0x8c, 0xe1, 0x136 }; /* Fan speed #define LEAKSHIELD_FLOW_IN 111 #define LEAKSHIELD_RESERVOIR_VOLUME 313 #define LEAKSHIELD_RESERVOIR_FILLED 311 +#define LEAKSHIELD_MIN_STATUS_REPORT_SIZE (LEAKSHIELD_RESERVOIR_VOLUME + sizeof(u16)) /* Specs of the Aquastream XT pump */ #define AQUASTREAMXT_SERIAL_START 0x3a @@ -556,6 +569,7 @@ struct aqc_data { const char *name; int status_report_id; /* Used for legacy devices, report is stored in buffer */ + u16 min_report_size; int ctrl_report_id; int secondary_ctrl_report_id; int secondary_ctrl_report_size; @@ -601,9 +615,9 @@ struct aqc_data { u32 speed_input_min[1]; u32 speed_input_target[1]; u32 speed_input_max[1]; - u32 power_input[8]; + long power_input[8]; u16 voltage_input[8]; - u16 current_input[8]; + u32 current_input[8]; /* Label values */ const char *const *temp_label; @@ -996,7 +1010,7 @@ static int aqc_legacy_read(struct aqc_data *priv) /* Calculation derived from linear regression */ sensor_value = get_unaligned_le16(priv->buffer + AQUASTREAMXT_PUMP_CURR_OFFSET); - priv->current_input[0] = DIV_ROUND_CLOSEST(sensor_value * 176, 100) - 52; + priv->current_input[0] = max(DIV_ROUND_CLOSEST(sensor_value * 176, 100) - 52, 0); sensor_value = get_unaligned_le16(priv->buffer + AQUASTREAMXT_PUMP_VOLTAGE_OFFSET); priv->voltage_input[0] = DIV_ROUND_CLOSEST(sensor_value * 1000, 61); @@ -1029,6 +1043,7 @@ static int aqc_legacy_read(struct aqc_data *priv) static int aqc_read(struct device *dev, enum hwmon_sensor_types type, u32 attr, int channel, long *val) { + long power; int ret; struct aqc_data *priv = dev_get_drvdata(dev); @@ -1093,7 +1108,10 @@ static int aqc_read(struct device *dev, enum hwmon_sensor_types type, u32 attr, } break; case hwmon_power: - *val = priv->power_input[channel]; + power = READ_ONCE(priv->power_input[channel]); + if (power == -ENODATA) + return -ENODATA; + *val = power; break; case hwmon_pwm: switch (priv->kind) { @@ -1356,6 +1374,8 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8 return 0; priv = hid_get_drvdata(hdev); + if (!priv->min_report_size || size < priv->min_report_size) + return 0; /* Info provided with every report */ priv->serial_number[0] = get_unaligned_be16(data + priv->serial_number_start_offset); @@ -1391,9 +1411,9 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8 priv->speed_input[i] = get_unaligned_be16(data + priv->fan_sensor_offsets[i] + priv->fan_structure->speed); - priv->power_input[i] = - get_unaligned_be16(data + priv->fan_sensor_offsets[i] + - priv->fan_structure->power) * 10000; + WRITE_ONCE(priv->power_input[i], + get_unaligned_be16(data + priv->fan_sensor_offsets[i] + + priv->fan_structure->power) * 10000); priv->voltage_input[i] = get_unaligned_be16(data + priv->fan_sensor_offsets[i] + priv->fan_structure->voltage) * 10; @@ -1433,7 +1453,8 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8 priv->speed_input[2] = get_unaligned_be16(data + AQUASTREAMULT_PRESSURE_OFFSET); priv->speed_input[3] = get_unaligned_be16(data + AQUASTREAMULT_FLOW_SENSOR_OFFSET); - priv->power_input[1] = get_unaligned_be16(data + AQUASTREAMULT_PUMP_POWER) * 10000; + WRITE_ONCE(priv->power_input[1], + get_unaligned_be16(data + AQUASTREAMULT_PUMP_POWER) * 10000); priv->voltage_input[1] = get_unaligned_be16(data + AQUASTREAMULT_PUMP_VOLTAGE) * 10; @@ -1446,10 +1467,11 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8 case highflownext: /* If external temp sensor is not connected, its power reading is also N/A */ if (priv->temp_input[1] == -ENODATA) - priv->power_input[0] = -ENODATA; + WRITE_ONCE(priv->power_input[0], -ENODATA); else - priv->power_input[0] = - get_unaligned_be16(data + HIGHFLOWNEXT_POWER) * 1000000; + WRITE_ONCE(priv->power_input[0], + min_t(u64, get_unaligned_be16(data + HIGHFLOWNEXT_POWER) * + 1000000ULL, LONG_MAX)); priv->voltage_input[0] = get_unaligned_be16(data + HIGHFLOWNEXT_5V_VOLTAGE) * 10; priv->voltage_input[1] = @@ -1541,6 +1563,7 @@ static void aqc_debugfs_init(struct aqc_data *priv) static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) { struct aqc_data *priv; + u16 min_report_size = 0; int ret; priv = devm_kzalloc(&hdev->dev, sizeof(*priv), GFP_KERNEL); @@ -1577,7 +1600,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) * they present. The two other devices have the type of the second element in * their respective collections set to 1, while the real device has it set to 0. */ - if (hdev->collection[1].type != 0) { + if (hdev->maxcollection < 2 || hdev->collection[1].type != 0) { ret = -ENODEV; goto fail_and_close; } @@ -1608,6 +1631,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->power_label = label_fan_power; priv->voltage_label = label_fan_voltage; priv->current_label = label_fan_current; + min_report_size = AQUAERO_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_D5NEXT: priv->kind = d5next; @@ -1633,6 +1657,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->power_label = label_d5next_power; priv->voltage_label = label_d5next_voltages; priv->current_label = label_d5next_current; + min_report_size = D5NEXT_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_FARBWERK: priv->kind = farbwerk; @@ -1643,6 +1668,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->temp_sensor_start_offset = FARBWERK_SENSOR_START; priv->temp_label = label_temp_sensors; + min_report_size = FARBWERK_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_FARBWERK360: priv->kind = farbwerk360; @@ -1659,6 +1685,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->temp_label = label_temp_sensors; priv->virtual_temp_label = label_virtual_temp_sensors; + min_report_size = FARBWERK360_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_OCTO: priv->kind = octo; @@ -1688,6 +1715,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->power_label = label_fan_power; priv->voltage_label = label_fan_voltage; priv->current_label = label_fan_current; + min_report_size = OCTO_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_QUADRO: priv->kind = quadro; @@ -1717,6 +1745,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->power_label = label_fan_power; priv->voltage_label = label_fan_voltage; priv->current_label = label_fan_current; + min_report_size = QUADRO_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_HIGHFLOWNEXT: priv->kind = highflownext; @@ -1734,6 +1763,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->speed_label = label_highflownext_fan_speed; priv->power_label = label_highflownext_power; priv->voltage_label = label_highflownext_voltage; + min_report_size = HIGHFLOWNEXT_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_LEAKSHIELD: /* @@ -1753,6 +1783,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->temp_label = label_leakshield_temp_sensors; priv->speed_label = label_leakshield_fan_speed; + min_report_size = LEAKSHIELD_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_AQUASTREAMXT: priv->kind = aquastreamxt; @@ -1783,6 +1814,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) priv->power_label = label_aquastreamult_power; priv->voltage_label = label_aquastreamult_voltages; priv->current_label = label_aquastreamult_current; + min_report_size = AQUASTREAMULT_MIN_STATUS_REPORT_SIZE; break; case USB_PRODUCT_ID_POWERADJUST3: priv->kind = poweradjust3; @@ -1856,6 +1888,8 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id) break; } + priv->min_report_size = min_report_size; + if (priv->buffer_size != 0) { priv->checksum_start = 0x01; priv->checksum_length = priv->buffer_size - 3; diff --git a/drivers/hwmon/asus_wmi_sensors.c b/drivers/hwmon/asus_wmi_sensors.c index c2dd7ff882f2..502996e97cf1 100644 --- a/drivers/hwmon/asus_wmi_sensors.c +++ b/drivers/hwmon/asus_wmi_sensors.c @@ -164,6 +164,7 @@ struct asus_wmi_wmi_info { struct asus_wmi_sensors { struct asus_wmi_wmi_info wmi; + struct hwmon_chip_info chip_info; /* lock access to internal cache */ struct mutex lock; }; @@ -496,11 +497,6 @@ static const struct hwmon_ops asus_wmi_hwmon_ops = { .read_string = asus_wmi_hwmon_read_string, }; -static struct hwmon_chip_info asus_wmi_chip_info = { - .ops = &asus_wmi_hwmon_ops, - .info = NULL, -}; - static int asus_wmi_configure_sensor_setup(struct device *dev, struct asus_wmi_sensors *sensor_data) { @@ -508,7 +504,7 @@ static int asus_wmi_configure_sensor_setup(struct device *dev, struct hwmon_channel_info *asus_wmi_hwmon_chan; int nr_count[hwmon_max] = {}, nr_types = 0; struct asus_wmi_sensor_info *temp_sensor; - const struct hwmon_chip_info *chip_info; + struct hwmon_chip_info *chip_info; enum hwmon_sensor_types type; struct device *hwdev; int i, idx; @@ -549,8 +545,9 @@ static int asus_wmi_configure_sensor_setup(struct device *dev, if (!ptr_asus_wmi_ci) return -ENOMEM; - asus_wmi_chip_info.info = ptr_asus_wmi_ci; - chip_info = &asus_wmi_chip_info; + chip_info = &sensor_data->chip_info; + chip_info->ops = &asus_wmi_hwmon_ops; + chip_info->info = ptr_asus_wmi_ci; sensor_data->wmi.info_by_id = devm_kcalloc(dev, sensor_data->wmi.sensor_count, sizeof(*sensor_data->wmi.info_by_id), diff --git a/drivers/hwmon/chipcap2.c b/drivers/hwmon/chipcap2.c index d9adb9e1f9ab..46a3e22f43ad 100644 --- a/drivers/hwmon/chipcap2.c +++ b/drivers/hwmon/chipcap2.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #define CC2_START_CM 0xA0 @@ -87,6 +88,7 @@ struct cc2_data { int irq_low; int irq_high; bool process_irqs; + bool regulator_enabled; }; /* %RH as a per cent mille from a register value */ @@ -116,8 +118,10 @@ static int cc2_enable(struct cc2_data *data) int ret; /* exclusive regulator, check in case a disable failed */ - if (regulator_is_enabled(data->regulator)) + if (data->regulator_enabled) { + data->process_irqs = true; return 0; + } /* clear any pending completion */ try_wait_for_completion(&data->complete); @@ -126,6 +130,8 @@ static int cc2_enable(struct cc2_data *data) if (ret < 0) return ret; + data->regulator_enabled = true; + usleep_range(CC2_STARTUP_TIME_US, CC2_STARTUP_TIME_US + 125); data->process_irqs = true; @@ -137,15 +143,27 @@ static void cc2_disable(struct cc2_data *data) { int err; + /* let the alarms already raised by the last measurement be processed */ + if (data->irq_low > 0) + synchronize_irq(data->irq_low); + + if (data->irq_high > 0) + synchronize_irq(data->irq_high); + /* ignore alarms triggered by voltage toggling when powering up */ data->process_irqs = false; /* exclusive regulator, check in case an enable failed */ - if (regulator_is_enabled(data->regulator)) { - err = regulator_disable(data->regulator); - if (err) - dev_dbg(&data->client->dev, "Failed to disable device"); + if (!data->regulator_enabled) + return; + + err = regulator_disable(data->regulator); + if (err) { + dev_dbg(&data->client->dev, "Failed to disable device"); + return; } + + data->regulator_enabled = false; } static int cc2_cmd_response_diagnostic(struct device *dev, u8 status) @@ -164,13 +182,13 @@ static int cc2_cmd_response_diagnostic(struct device *dev, u8 status) case CC2_RESPONSE_BUSY: return -EBUSY; case CC2_RESPONSE_NACK: - if (resp & CC2_ERR_CORR_EEPROM) + if (status & CC2_ERR_CORR_EEPROM) dev_dbg(dev, "Command failed: corrected EEPROM\n"); - if (resp & CC2_ERR_UNCORR_EEPROM) + if (status & CC2_ERR_UNCORR_EEPROM) dev_dbg(dev, "Command failed: uncorrected EEPROM\n"); - if (resp & CC2_ERR_RAM_PARITY) + if (status & CC2_ERR_RAM_PARITY) dev_dbg(dev, "Command failed: RAM parity\n"); - if (resp & CC2_ERR_RAM_PARITY) + if (status & CC2_ERR_CONFIG_LOAD) dev_dbg(dev, "Command failed: configuration error\n"); return -ENODATA; default: @@ -223,7 +241,7 @@ static int cc2_command_mode_start(struct cc2_data *data) 2 * CC2_RESP_START_CM_US); } ret = cc2_read_command_status(data->client); - if (ret != -ETIMEDOUT || i == CC2_CM_RETRIES) + if (ret != -ETIMEDOUT || i == CC2_CM_RETRIES - 1) break; /* command window missed, prepare for a retry */ @@ -257,8 +275,7 @@ static int cc2_write_reg(struct cc2_data *data, u8 reg, u16 val) if (ret < 0) goto disable; - cpu_to_be16s(&val); - ret = i2c_smbus_write_word_data(data->client, reg, val); + ret = i2c_smbus_write_word_swapped(data->client, reg, val); if (ret < 0) goto disable; @@ -308,7 +325,7 @@ static int cc2_read_reg(struct cc2_data *data, u8 reg, u16 *val) if (ret != CC2_EEPROM_DATA_LEN) return ret < 0 ? ret : -EIO; - *val = be16_to_cpup((__be16 *)&buf[1]); + *val = get_unaligned_be16(&buf[1]); return cc2_read_command_status(data->client); } @@ -348,10 +365,10 @@ static int cc2_data_fetch(struct i2c_client *client, switch (type) { case hwmon_humidity: - *val = cc2_rh_convert(be16_to_cpup((__be16 *)&data[0])); + *val = cc2_rh_convert(get_unaligned_be16(&data[0])); break; case hwmon_temp: - *val = cc2_temp_convert(be16_to_cpup((__be16 *)&data[2])); + *val = cc2_temp_convert(get_unaligned_be16(&data[2])); break; default: return -EINVAL; @@ -651,6 +668,8 @@ static int cc2_request_ready_irq(struct cc2_data *data, struct device *dev) int ret = 0; data->irq_ready = fwnode_irq_get_byname(dev_fwnode(dev), "ready"); + if (data->irq_ready == -EPROBE_DEFER) + return data->irq_ready; if (data->irq_ready > 0) { init_completion(&data->complete); ret = devm_request_threaded_irq(dev, data->irq_ready, NULL, @@ -668,6 +687,8 @@ static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev) int ret = 0; data->irq_low = fwnode_irq_get_byname(dev_fwnode(dev), "low"); + if (data->irq_low == -EPROBE_DEFER) + return data->irq_low; if (data->irq_low > 0) { ret = devm_request_threaded_irq(dev, data->irq_low, NULL, cc2_low_interrupt, @@ -681,6 +702,8 @@ static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev) } data->irq_high = fwnode_irq_get_byname(dev_fwnode(dev), "high"); + if (data->irq_high == -EPROBE_DEFER) + return data->irq_high; if (data->irq_high > 0) { ret = devm_request_threaded_irq(dev, data->irq_high, NULL, cc2_high_interrupt, diff --git a/drivers/hwmon/coretemp.c b/drivers/hwmon/coretemp.c index 04b20cf272ac..cc81b849c09e 100644 --- a/drivers/hwmon/coretemp.c +++ b/drivers/hwmon/coretemp.c @@ -61,6 +61,8 @@ enum coretemp_attr_index { /* * Per-Core Temperature Data + * @temp_valid: Whether the CPU reported @temp as valid. @temp is not + * reported when this is false. * @tjmax: The static tjmax value when tjmax cannot be retrieved from * IA32_TEMPERATURE_TARGET MSR. * @last_updated: The time when the current temperature value was updated @@ -73,6 +75,7 @@ enum coretemp_attr_index { */ struct temp_data { int temp; + bool temp_valid; int tjmax; unsigned long last_updated; unsigned int cpu; @@ -397,7 +400,8 @@ static ssize_t show_temp(struct device *dev, { u32 eax, edx; struct temp_data *tdata = container_of(devattr, struct temp_data, sd_attrs[ATTR_TEMP]); - int tjmax; + int tjmax, temp; + bool valid; mutex_lock(&tdata->update_lock); @@ -406,17 +410,22 @@ static ssize_t show_temp(struct device *dev, if (time_after(jiffies, tdata->last_updated + HZ)) { rdmsr_on_cpu(tdata->cpu, tdata->status_reg, &eax, &edx); /* - * Ignore the valid bit. In all observed cases the register - * value is either low or zero if the valid bit is 0. - * Return it instead of reporting an error which doesn't - * really help at all. + * Bit 31 indicates whether the reading is valid. Some CPUs + * clear it while the temperature is too low to be measured. + * Such a reading is not a temperature. Do not report it. */ + tdata->temp_valid = (eax >> 31) & 1; tdata->temp = tjmax - ((eax >> 16) & 0xff) * 1000; tdata->last_updated = jiffies; } + valid = tdata->temp_valid; + temp = tdata->temp; mutex_unlock(&tdata->update_lock); - return sprintf(buf, "%d\n", tdata->temp); + + if (!valid) + return -ENODATA; + return sprintf(buf, "%d\n", temp); } static int create_core_attrs(struct temp_data *tdata, struct device *dev) @@ -692,6 +701,9 @@ static int coretemp_cpu_online(unsigned int cpu) if (!housekeeping_cpu(cpu, HK_TYPE_MISC)) return 0; + if (!pdev) + return 0; + pdata = platform_get_drvdata(pdev); if (!pdata->hwmon_dev) { struct device *hwmon; @@ -743,6 +755,9 @@ static int coretemp_cpu_offline(unsigned int cpu) return 0; /* If the physical CPU device does not exist, just return */ + if (!pdev) + return 0; + pd = platform_get_drvdata(pdev); if (!pd->hwmon_dev) return 0; diff --git a/drivers/hwmon/corsair-cpro.c b/drivers/hwmon/corsair-cpro.c index 17bb080eb9bb..1c152a6db380 100644 --- a/drivers/hwmon/corsair-cpro.c +++ b/drivers/hwmon/corsair-cpro.c @@ -215,10 +215,11 @@ static int set_target(struct ccp_device *ccp, int channel, long val) int ret; val = clamp_val(val, 0, 0xFFFF); - ccp->target[channel] = val; mutex_lock(&ccp->mutex); ret = send_usb_cmd(ccp, CTL_SET_FAN_TARGET, channel, val >> 8, val); + if (!ret) + ccp->target[channel] = val; mutex_unlock(&ccp->mutex); return ret; diff --git a/drivers/hwmon/hp-wmi-sensors.c b/drivers/hwmon/hp-wmi-sensors.c index 82210fef828f..d2f8f93ee96d 100644 --- a/drivers/hwmon/hp-wmi-sensors.c +++ b/drivers/hwmon/hp-wmi-sensors.c @@ -390,6 +390,7 @@ struct hp_wmi_info { */ struct hp_wmi_sensors { struct wmi_device *wdev; + struct hwmon_chip_info chip_info; struct hp_wmi_info **info_map[hwmon_max]; u8 channel_count[hwmon_max]; bool has_intrusion; @@ -724,15 +725,21 @@ static long scale_numeric_sensor(const struct hp_wmi_numeric_sensor *nsensor) val = current_reading; - for (; unit_modifier < target_modifier; unit_modifier++) + for (; val && unit_modifier < target_modifier; unit_modifier++) val = DIV_ROUND_CLOSEST(val, 10); - for (; unit_modifier > target_modifier; unit_modifier--) { - if (val > LONG_MAX / 10) { - val = LONG_MAX; - break; + if (val) { + for (; unit_modifier > target_modifier; unit_modifier--) { + if (val > LONG_MAX / 10) { + val = LONG_MAX; + break; + } + if (val < LONG_MIN / 10) { + val = LONG_MIN; + break; + } + val *= 10; } - val *= 10; } if (sensor_type == HP_WMI_TYPE_TEMPERATURE) { @@ -1517,11 +1524,6 @@ static const struct hwmon_ops hp_wmi_hwmon_ops = { .write = hp_wmi_hwmon_write, }; -static struct hwmon_chip_info hp_wmi_chip_info = { - .ops = &hp_wmi_hwmon_ops, - .info = NULL, -}; - static struct hp_wmi_info *match_fan_event(struct hp_wmi_sensors *state, const char *event_description) { @@ -1948,7 +1950,8 @@ static int make_chip_info(struct hp_wmi_sensors *state, bool has_events) if (!ptr_channel_info) return -ENOMEM; - hp_wmi_chip_info.info = ptr_channel_info; + state->chip_info.ops = &hp_wmi_hwmon_ops; + state->chip_info.info = ptr_channel_info; for (type = hwmon_chip; type < hwmon_max; type++) { count = channel_count[type]; @@ -2039,7 +2042,7 @@ static int hp_wmi_sensors_init(struct hp_wmi_sensors *state) return err; hwdev = devm_hwmon_device_register_with_info(dev, "hp_wmi_sensors", - state, &hp_wmi_chip_info, + state, &state->chip_info, NULL); return PTR_ERR_OR_ZERO(hwdev); } diff --git a/drivers/hwmon/lm70.c b/drivers/hwmon/lm70.c index 0d5a250cb672..ee06c5641870 100644 --- a/drivers/hwmon/lm70.c +++ b/drivers/hwmon/lm70.c @@ -97,21 +97,21 @@ static ssize_t temp1_input_show(struct device *dev, */ switch (p_lm70->chip) { case LM70_CHIP_LM70: - val = ((int)raw / 32) * 250; + val = ((int)raw >> 5) * 250; break; case LM70_CHIP_TMP121: case LM70_CHIP_TMP122: case LM70_CHIP_LM74: - val = ((int)raw / 8) * 625 / 10; + val = ((int)raw >> 3) * 625 / 10; break; case LM70_CHIP_LM71: - val = ((int)raw / 4) * 3125 / 100; + val = ((int)raw >> 2) * 3125 / 100; break; case LM70_CHIP_TMP125: - val = (sign_extend32(raw, 14) / 32) * 250; + val = (sign_extend32(raw, 14) >> 5) * 250; break; } diff --git a/drivers/hwmon/lm87.c b/drivers/hwmon/lm87.c index d2d970e73c61..bdd633feb2ba 100644 --- a/drivers/hwmon/lm87.c +++ b/drivers/hwmon/lm87.c @@ -600,13 +600,16 @@ static DEVICE_ATTR_RO(cpu0_vid); static ssize_t vrm_show(struct device *dev, struct device_attribute *attr, char *buf) { - struct lm87_data *data = dev_get_drvdata(dev); + struct i2c_client *client = dev_get_drvdata(dev); + struct lm87_data *data = i2c_get_clientdata(client); + return sprintf(buf, "%d\n", data->vrm); } static ssize_t vrm_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t count) { - struct lm87_data *data = dev_get_drvdata(dev); + struct i2c_client *client = dev_get_drvdata(dev); + struct lm87_data *data = i2c_get_clientdata(client); unsigned long val; int err; diff --git a/drivers/hwmon/nct6683.c b/drivers/hwmon/nct6683.c index f71615e06a8f..e091551688a0 100644 --- a/drivers/hwmon/nct6683.c +++ b/drivers/hwmon/nct6683.c @@ -914,7 +914,7 @@ static ssize_t show_pwm(struct device *dev, struct device_attribute *attr, char *buf) { struct nct6683_data *data = nct6683_update_device(dev); - struct sensor_device_attribute_2 *sattr = to_sensor_dev_attr_2(attr); + struct sensor_device_attribute *sattr = to_sensor_dev_attr(attr); int index = sattr->index; return sprintf(buf, "%d\n", data->pwm[index]); @@ -924,7 +924,7 @@ static ssize_t store_pwm(struct device *dev, struct device_attribute *attr, const char *buf, size_t count) { - struct sensor_device_attribute_2 *sattr = to_sensor_dev_attr_2(attr); + struct sensor_device_attribute *sattr = to_sensor_dev_attr(attr); struct nct6683_data *data = dev_get_drvdata(dev); int index = sattr->index; unsigned long val; diff --git a/drivers/hwmon/nct6775-platform.c b/drivers/hwmon/nct6775-platform.c index ece183a0d4e9..4b0003280f11 100644 --- a/drivers/hwmon/nct6775-platform.c +++ b/drivers/hwmon/nct6775-platform.c @@ -1158,6 +1158,7 @@ static const char * const asus_wmi_boards[] = { "Pro A520M-C", "Pro A520M-C II", "Pro B550M-C", + "Pro WS W480-ACE", "Pro WS X570-ACE", "ProArt B550-CREATOR", "ProArt X570-CREATOR WIFI", @@ -1257,6 +1258,7 @@ static const char * const asus_wmi_boards[] = { "TUF Z390-PRO GAMING", "TUF Z390M-PRO GAMING", "TUF Z390M-PRO GAMING (WI-FI)", + "W480/SYS", "WS Z390 PRO", "Z490-GUNDAM (WI-FI)", }; @@ -1269,6 +1271,7 @@ static const char * const asus_msi_boards[] = { "EX-B760M-V5 D4", "EX-H510M-V3", "EX-H610M-V3 D4", + "G15CE", "G15CF", "PRIME A620M-A", "PRIME B560-PLUS", @@ -1319,6 +1322,8 @@ static const char * const asus_msi_boards[] = { "PRIME X670-P", "PRIME X670-P WIFI", "PRIME X670E-PRO WIFI", + "PRIME X870-P", + "PRIME X870-P WIFI", "PRIME Z590-A", "PRIME Z590-P", "PRIME Z590-P WIFI", @@ -1361,11 +1366,18 @@ static const char * const asus_msi_boards[] = { "ProArt B660-CREATOR D4", "ProArt B760-CREATOR D4", "ProArt X670E-CREATOR WIFI", + "ProArt X870E-CREATOR WIFI", "ProArt Z690-CREATOR WIFI", "ProArt Z790-CREATOR WIFI", "ROG CROSSHAIR X670E EXTREME", "ROG CROSSHAIR X670E GENE", "ROG CROSSHAIR X670E HERO", + "ROG CROSSHAIR X870E APEX", + "ROG CROSSHAIR X870E DARK HERO", + "ROG CROSSHAIR X870E EXTREME", + "ROG CROSSHAIR X870E GLACIAL", + "ROG CROSSHAIR X870E HERO", + "ROG CROSSHAIR X870E HERO BTF", "ROG MAXIMUS XIII APEX", "ROG MAXIMUS XIII EXTREME", "ROG MAXIMUS XIII EXTREME GLACIAL", @@ -1403,6 +1415,12 @@ static const char * const asus_msi_boards[] = { "ROG STRIX X670E-E GAMING WIFI", "ROG STRIX X670E-F GAMING WIFI", "ROG STRIX X670E-I GAMING WIFI", + "ROG STRIX X870-A GAMING WIFI", + "ROG STRIX X870-F GAMING WIFI", + "ROG STRIX X870-I GAMING WIFI", + "ROG STRIX X870E-E GAMING WIFI", + "ROG STRIX X870E-E GAMING WIFI7 R2", + "ROG STRIX X870E-H GAMING WIFI7", "ROG STRIX Z590-A GAMING WIFI", "ROG STRIX Z590-A GAMING WIFI II", "ROG STRIX Z590-E GAMING WIFI", @@ -1449,6 +1467,9 @@ static const char * const asus_msi_boards[] = { "TUF GAMING H770-PRO WIFI", "TUF GAMING X670E-PLUS", "TUF GAMING X670E-PLUS WIFI", + "TUF GAMING X870-PLUS WIFI", + "TUF GAMING X870-PRO WIFI7 W NEO", + "TUF GAMING X870E-PLUS WIFI7", "TUF GAMING Z590-PLUS", "TUF GAMING Z590-PLUS WIFI", "TUF GAMING Z690-PLUS", @@ -1458,6 +1479,9 @@ static const char * const asus_msi_boards[] = { "TUF GAMING Z790-PLUS D4", "TUF GAMING Z790-PLUS WIFI", "TUF GAMING Z790-PLUS WIFI D4", + "X870 AYW GAMING WIFI W", + "X870 MAX GAMING WIFI7", + "X870 MAX GAMING WIFI7 W", "Z590 WIFI GUNDAM EDITION", }; diff --git a/drivers/hwmon/peci/dimmtemp.c b/drivers/hwmon/peci/dimmtemp.c index b7b09780c7b0..671ea8d3b8ad 100644 --- a/drivers/hwmon/peci/dimmtemp.c +++ b/drivers/hwmon/peci/dimmtemp.c @@ -4,7 +4,6 @@ #include #include #include -#include #include #include #include @@ -331,11 +330,13 @@ static int create_dimm_temp_info(struct peci_dimmtemp *priv) if (ret == -ENODEV) { dev_dbg(priv->dev, "No DIMMs found\n"); return 0; - } else if (ret) { + } + if (ret == -EAGAIN) { schedule_delayed_work(&priv->detect_work, DIMM_MASK_CHECK_DELAY_JIFFIES); dev_dbg(priv->dev, "Deferred populating DIMM temp info\n"); - return ret; } + if (ret) + return ret; channels = priv->gen_info->chan_rank_max * priv->gen_info->dimm_idx_max; @@ -405,10 +406,8 @@ static int peci_dimmtemp_probe(struct auxiliary_device *adev, const struct auxil "Unexpected PECI revision %#x, some features may be unavailable\n", peci_dev->info.peci_revision); - ret = devm_delayed_work_autocancel(priv->dev, &priv->detect_work, - create_dimm_temp_info_delayed); - if (ret) - return ret; + auxiliary_set_drvdata(adev, priv); + INIT_DELAYED_WORK(&priv->detect_work, create_dimm_temp_info_delayed); ret = create_dimm_temp_info(priv); if (ret && ret != -EAGAIN) { @@ -419,6 +418,13 @@ static int peci_dimmtemp_probe(struct auxiliary_device *adev, const struct auxil return 0; } +static void peci_dimmtemp_remove(struct auxiliary_device *adev) +{ + struct peci_dimmtemp *priv = auxiliary_get_drvdata(adev); + + cancel_delayed_work_sync(&priv->detect_work); +} + static int read_thresholds_hsx(struct peci_dimmtemp *priv, int dimm_order, int chan_rank, u32 *data) { @@ -655,6 +661,7 @@ MODULE_DEVICE_TABLE(auxiliary, peci_dimmtemp_ids); static struct auxiliary_driver peci_dimmtemp_driver = { .probe = peci_dimmtemp_probe, + .remove = peci_dimmtemp_remove, .id_table = peci_dimmtemp_ids, }; diff --git a/drivers/hwmon/pmbus/pmbus_core.c b/drivers/hwmon/pmbus/pmbus_core.c index ea79cff0e8d0..596b7fd568bf 100644 --- a/drivers/hwmon/pmbus/pmbus_core.c +++ b/drivers/hwmon/pmbus/pmbus_core.c @@ -1420,11 +1420,14 @@ static int pmbus_thermal_add_sensor(struct pmbus_data *pmbus_data, tzd = devm_thermal_of_zone_register(dev, index, tdata, &pmbus_thermal_ops); /* - * If CONFIG_THERMAL_OF is disabled, this returns -ENODEV, - * so ignore that error but forward any other error. + * If CONFIG_THERMAL_OF is disabled, devm_thermal_of_zone_register() + * returns -ENOTSUPP. If no thermal zone is configured for the sensor, + * it returns -ENODEV. Ignore those errors but forward any other error. */ - if (IS_ERR(tzd) && (PTR_ERR(tzd) != -ENODEV)) + if (IS_ERR(tzd) && PTR_ERR(tzd) != -ENODEV && PTR_ERR(tzd) != -ENOTSUPP) { + devm_kfree(dev, tdata); return PTR_ERR(tzd); + } return 0; } @@ -1439,10 +1442,11 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data, { struct sensor_device_attribute *a; struct pmbus_sensor *sensor; + int err; sensor = devm_kzalloc(data->dev, sizeof(*sensor), GFP_KERNEL); if (!sensor) - return NULL; + return ERR_PTR(-ENOMEM); a = &sensor->attribute; if (type) @@ -1467,14 +1471,17 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data, pmbus_set_sensor, -1); if (pmbus_add_attribute(data, &a->dev_attr.attr)) - return NULL; + return ERR_PTR(-ENOMEM); sensor->next = data->sensors; data->sensors = sensor; /* temperature sensors with _input values are registered with thermal */ - if (class == PSC_TEMPERATURE && strcmp(type, "input") == 0) - pmbus_thermal_add_sensor(data, sensor, seq); + if (class == PSC_TEMPERATURE && strcmp(type, "input") == 0) { + err = pmbus_thermal_add_sensor(data, sensor, seq); + if (err) + return ERR_PTR(err); + } return sensor; } @@ -1576,8 +1583,8 @@ static int pmbus_add_limit_attrs(struct i2c_client *client, page, 0xff, l->reg, attr->class, attr->update || l->update, l->readonly, l->writeonly, true); - if (!curr) - return -ENOMEM; + if (IS_ERR(curr)) + return PTR_ERR(curr); if (l->sbit && (info->func[page] & attr->sfunc)) { ret = pmbus_add_boolean(data, name, l->alarm, index, @@ -1616,8 +1623,8 @@ static int pmbus_add_sensor_attrs_one(struct i2c_client *client, } base = pmbus_add_sensor(data, name, "input", index, page, phase, attr->reg, attr->class, true, true, false, true); - if (!base) - return -ENOMEM; + if (IS_ERR(base)) + return PTR_ERR(base); /* No limit and alarm attributes for phase specific sensors */ if (attr->sfunc && phase == 0xff) { ret = pmbus_add_limit_attrs(client, data, info, name, @@ -2290,8 +2297,8 @@ static int pmbus_add_fan_ctrl(struct i2c_client *client, 0xff, PMBUS_VIRT_FAN_TARGET_1 + id, PSC_FAN, false, false, false, true); - if (!sensor) - return -ENOMEM; + if (IS_ERR(sensor)) + return PTR_ERR(sensor); if (!((data->info->func[page] & PMBUS_HAVE_PWM12) || (data->info->func[page] & PMBUS_HAVE_PWM34))) @@ -2301,15 +2308,15 @@ static int pmbus_add_fan_ctrl(struct i2c_client *client, 0xff, PMBUS_VIRT_PWM_1 + id, PSC_PWM, false, false, false, true); - if (!sensor) - return -ENOMEM; + if (IS_ERR(sensor)) + return PTR_ERR(sensor); sensor = pmbus_add_sensor(data, "pwm", "enable", index, page, 0xff, PMBUS_VIRT_PWM_ENABLE_1 + id, PSC_PWM, true, false, false, false); - if (!sensor) - return -ENOMEM; + if (IS_ERR(sensor)) + return PTR_ERR(sensor); return 0; } @@ -2326,6 +2333,7 @@ static int pmbus_add_fan_attributes(struct i2c_client *client, int f; for (f = 0; f < ARRAY_SIZE(pmbus_fan_registers); f++) { + struct pmbus_sensor *sensor; int regval; if (!(info->func[page] & pmbus_fan_flags[f])) @@ -2346,10 +2354,11 @@ static int pmbus_add_fan_attributes(struct i2c_client *client, (!(regval & (PB_FAN_1_INSTALLED >> ((f & 1) * 4))))) continue; - if (pmbus_add_sensor(data, "fan", "input", index, - page, 0xff, pmbus_fan_registers[f], - PSC_FAN, true, true, false, true) == NULL) - return -ENOMEM; + sensor = pmbus_add_sensor(data, "fan", "input", index, + page, 0xff, pmbus_fan_registers[f], + PSC_FAN, true, true, false, true); + if (IS_ERR(sensor)) + return PTR_ERR(sensor); /* Fan control */ if (pmbus_check_word_register(client, page, diff --git a/drivers/hwmon/pmbus/q54sj108a2.c b/drivers/hwmon/pmbus/q54sj108a2.c index d9fd1095ae7e..857bf6b6141c 100644 --- a/drivers/hwmon/pmbus/q54sj108a2.c +++ b/drivers/hwmon/pmbus/q54sj108a2.c @@ -343,6 +343,8 @@ static int q54sj108a2_probe(struct i2c_client *client) psu->client = client; debugfs = pmbus_get_debugfs_dir(client); + if (!debugfs) + return 0; q54sj108a2_dir = debugfs_create_dir(client->name, debugfs); diff --git a/drivers/hwmon/tmp102.c b/drivers/hwmon/tmp102.c index 8af44a33055f..967d2e7e8ccf 100644 --- a/drivers/hwmon/tmp102.c +++ b/drivers/hwmon/tmp102.c @@ -54,7 +54,7 @@ struct tmp102 { struct regmap *regmap; u16 config_orig; - unsigned long ready_time; + u64 ready_time; }; /* convert left adjusted 13-bit TMP102 register value to milliCelsius */ @@ -79,7 +79,7 @@ static int tmp102_read(struct device *dev, enum hwmon_sensor_types type, switch (attr) { case hwmon_temp_input: /* Is it too early to return a conversion ? */ - if (time_before(jiffies, tmp102->ready_time)) { + if (time_before64(get_jiffies_64(), tmp102->ready_time)) { dev_dbg(dev, "%s: Conversion not ready yet..\n", __func__); return -EAGAIN; } @@ -245,7 +245,7 @@ static int tmp102_probe(struct i2c_client *client) * Mark that we are not ready with data until the first * conversion is complete */ - tmp102->ready_time = jiffies + msecs_to_jiffies(CONVERSION_TIME_MS); + tmp102->ready_time = get_jiffies_64() + msecs_to_jiffies(CONVERSION_TIME_MS); hwmon_dev = devm_hwmon_device_register_with_info(dev, client->name, tmp102, @@ -278,7 +278,7 @@ static int tmp102_resume(struct device *dev) err = regmap_update_bits(tmp102->regmap, TMP102_CONF_REG, TMP102_CONF_SD, 0); - tmp102->ready_time = jiffies + msecs_to_jiffies(CONVERSION_TIME_MS); + tmp102->ready_time = get_jiffies_64() + msecs_to_jiffies(CONVERSION_TIME_MS); return err; } diff --git a/drivers/hwmon/tmp108.c b/drivers/hwmon/tmp108.c index a82bbc959eb1..18e7c9d9bcbb 100644 --- a/drivers/hwmon/tmp108.c +++ b/drivers/hwmon/tmp108.c @@ -73,7 +73,7 @@ struct tmp108 { struct regmap *regmap; u16 orig_config; - unsigned long ready_time; + u64 ready_time; }; /* convert 12-bit TMP108 register value to milliCelsius */ @@ -124,7 +124,7 @@ static int tmp108_read(struct device *dev, enum hwmon_sensor_types type, switch (attr) { case hwmon_temp_input: /* Is it too early to return a conversion ? */ - if (time_before(jiffies, tmp108->ready_time)) { + if (time_before64(get_jiffies_64(), tmp108->ready_time)) { dev_dbg(dev, "%s: Conversion not ready yet..\n", __func__); return -EAGAIN; @@ -371,7 +371,7 @@ static int tmp108_probe(struct i2c_client *client) return err; } - tmp108->ready_time = jiffies; + tmp108->ready_time = get_jiffies_64(); if ((tmp108->orig_config & TMP108_CONF_MODE_MASK) == TMP108_MODE_SHUTDOWN) tmp108->ready_time += @@ -405,7 +405,7 @@ static int tmp108_resume(struct device *dev) err = regmap_update_bits(tmp108->regmap, TMP108_REG_CONF, TMP108_CONF_MODE_MASK, TMP108_MODE_CONTINUOUS); - tmp108->ready_time = jiffies + + tmp108->ready_time = get_jiffies_64() + msecs_to_jiffies(TMP108_CONVERSION_TIME_MS); return err; } diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 9127bc025552..41307719f144 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1082,6 +1082,9 @@ static void etm4_disable_sysfs(struct coresight_device *csdev) smp_call_function_single(drvdata->cpu, etm4_disable_hw, drvdata, 1); spin_unlock(&drvdata->spinlock); + + cscfg_csdev_disable_active_config(csdev); + cpus_read_unlock(); /* diff --git a/drivers/iio/adc/ad7192.c b/drivers/iio/adc/ad7192.c index 6fe32f866765..ef8268ad353e 100644 --- a/drivers/iio/adc/ad7192.c +++ b/drivers/iio/adc/ad7192.c @@ -1395,7 +1395,7 @@ static int ad7192_probe(struct spi_device *spi) st->chip_info = spi_get_device_match_data(spi); if (!st->chip_info) - return -ENODEV; + return -ENODATA; indio_dev->name = st->chip_info->name; indio_dev->modes = INDIO_DIRECT_MODE; diff --git a/drivers/iio/adc/ti-ads1100.c b/drivers/iio/adc/ti-ads1100.c index 1e46f07a9ca6..bea727ff9d6e 100644 --- a/drivers/iio/adc/ti-ads1100.c +++ b/drivers/iio/adc/ti-ads1100.c @@ -14,10 +14,12 @@ #include #include #include +#include #include #include #include #include +#include #include #include @@ -42,6 +44,9 @@ static const int ads1100_data_rate[] = { 128, 32, 16, 8 }; static const int ads1100_data_rate_bits[] = { 12, 14, 15, 16 }; +/* Timeout based on the minimum sample rate of 8 SPS (7500ms) */ +#define ADS1100_MAX_DRDY_TIMEOUT_US (7500 * USEC_PER_MSEC) + struct ads1100_data { struct i2c_client *client; struct regulator *reg_vdd; @@ -123,10 +128,86 @@ static int ads1100_get_adc_result(struct ads1100_data *data, int chan, int *val) return 0; } +static int ads1100_conversion_busy(struct ads1100_data *data) +{ + u8 buffer[3]; + int ret; + + ret = i2c_master_recv(data->client, (char *)&buffer, sizeof(buffer)); + if (ret < 0) { + dev_err(&data->client->dev, "I2C read fail: %d\n", ret); + return ret; + } + + return FIELD_GET(ADS1100_CFG_ST_BSY, buffer[2]); +} + +static int ads1100_wait_single_conversion(struct ads1100_data *data) +{ + int data_rate_index = FIELD_GET(ADS1100_DR_MASK, data->config); + int data_rate_Hz = ads1100_data_rate[data_rate_index]; + unsigned long poll_us = DIV_ROUND_CLOSEST(USEC_PER_SEC, data_rate_Hz) / 4; + int busy; + int ret; + + ret = readx_poll_timeout(ads1100_conversion_busy, data, + busy, busy <= 0, + poll_us, ADS1100_MAX_DRDY_TIMEOUT_US); + if (busy < 0) + return busy; + + return ret; +} + +static int ads1100_start_single_conversion(struct ads1100_data *data) +{ + u8 config = data->config | ADS1100_CFG_SC; + int ret; + + ret = i2c_master_send(data->client, &config, sizeof(config)); + if (ret < 0) { + dev_err(&data->client->dev, "I2C write fail: %d\n", ret); + return ret; + } + /* Need to wait because of change from continuous to single mode */ + ret = ads1100_wait_single_conversion(data); + if (ret) + return ret; + + config |= ADS1100_CFG_ST_BSY; + + ret = i2c_master_send(data->client, &config, sizeof(config)); + if (ret < 0) { + dev_err(&data->client->dev, "I2C write fail: %d\n", ret); + return ret; + } + + /* No need to cache it, it's status bit */ + data->config = config & ~ADS1100_CFG_ST_BSY; + + return 0; +} + +static int ads1100_poll_data_ready(struct ads1100_data *data) +{ + int ret; + + ret = ads1100_start_single_conversion(data); + if (ret) + return ret; + + ret = ads1100_wait_single_conversion(data); + if (ret) + return ret; + + return ads1100_set_config_bits(data, ADS1100_CFG_SC, ADS1100_CONTINUOUS); +} + static int ads1100_set_scale(struct ads1100_data *data, int val, int val2) { int microvolts; int gain; + int ret; /* With Vdd between 2.7 and 5V, the scale is always below 1 */ if (val) @@ -135,6 +216,10 @@ static int ads1100_set_scale(struct ads1100_data *data, int val, int val2) if (!val2) return -EINVAL; + ret = pm_runtime_resume_and_get(&data->client->dev); + if (ret) + return ret; + microvolts = regulator_get_voltage(data->reg_vdd); /* * val2 is in 'micro' units, n = val2 / 1000000 @@ -144,27 +229,51 @@ static int ads1100_set_scale(struct ads1100_data *data, int val, int val2) * bitshift so everything fits in 32-bits yields this formula. */ gain = DIV_ROUND_CLOSEST(microvolts, BIT(15)) * MILLI / val2; - if (gain < BIT(0) || gain > BIT(3)) - return -EINVAL; + if (gain < BIT(0) || gain > BIT(3)) { + ret = -EINVAL; + goto out; + } ads1100_set_config_bits(data, ADS1100_PGA_MASK, ffs(gain) - 1); - return 0; + ret = ads1100_poll_data_ready(data); +out: + pm_runtime_mark_last_busy(&data->client->dev); + pm_runtime_put_autosuspend(&data->client->dev); + + return ret; } static int ads1100_set_data_rate(struct ads1100_data *data, int chan, int rate) { unsigned int i; unsigned int size; + int ret; size = data->supports_data_rate ? ARRAY_SIZE(ads1100_data_rate) : 1; for (i = 0; i < size; i++) { if (ads1100_data_rate[i] == rate) - return ads1100_set_config_bits(data, ADS1100_DR_MASK, - FIELD_PREP(ADS1100_DR_MASK, i)); + break; } - return -EINVAL; + if (i == size) + return -EINVAL; + + ret = pm_runtime_resume_and_get(&data->client->dev); + if (ret) + return ret; + + ret = ads1100_set_config_bits(data, ADS1100_DR_MASK, + FIELD_PREP(ADS1100_DR_MASK, i)); + if (ret) + goto out; + + ret = ads1100_poll_data_ready(data); +out: + pm_runtime_mark_last_busy(&data->client->dev); + pm_runtime_put_autosuspend(&data->client->dev); + + return ret; } static int ads1100_get_vdd_millivolts(struct ads1100_data *data) diff --git a/drivers/iio/dac/mcp4821.c b/drivers/iio/dac/mcp4821.c index c1a59bbbba3c..9e4a3cb2a712 100644 --- a/drivers/iio/dac/mcp4821.c +++ b/drivers/iio/dac/mcp4821.c @@ -172,9 +172,13 @@ static const struct iio_info mcp4821_info = { static int mcp4821_probe(struct spi_device *spi) { + const struct mcp4821_chip_info *info; struct iio_dev *indio_dev; struct mcp4821_state *state; - const struct mcp4821_chip_info *info; + + info = spi_get_device_match_data(spi); + if (!info) + return -ENODATA; indio_dev = devm_iio_device_alloc(&spi->dev, sizeof(*state)); if (indio_dev == NULL) @@ -183,7 +187,6 @@ static int mcp4821_probe(struct spi_device *spi) state = iio_priv(indio_dev); state->spi = spi; - info = spi_get_device_match_data(spi); indio_dev->name = info->name; indio_dev->info = &mcp4821_info; indio_dev->modes = INDIO_DIRECT_MODE; diff --git a/drivers/iio/light/isl29028.c b/drivers/iio/light/isl29028.c index a23b2af287d5..12708bc35150 100644 --- a/drivers/iio/light/isl29028.c +++ b/drivers/iio/light/isl29028.c @@ -407,14 +407,9 @@ static int isl29028_write_raw(struct iio_dev *indio_dev, mutex_unlock(&chip->lock); - if (ret < 0) - return ret; + isl29028_set_pm_runtime_busy(chip, false); - ret = isl29028_set_pm_runtime_busy(chip, false); - if (ret < 0) - return ret; - - return 0; + return ret; } static int isl29028_read_raw(struct iio_dev *indio_dev, @@ -423,7 +418,7 @@ static int isl29028_read_raw(struct iio_dev *indio_dev, { struct isl29028_chip *chip = iio_priv(indio_dev); struct device *dev = regmap_get_device(chip->regmap); - int ret, pm_ret; + int ret; ret = isl29028_set_pm_runtime_busy(chip, true); if (ret < 0) @@ -476,17 +471,7 @@ static int isl29028_read_raw(struct iio_dev *indio_dev, mutex_unlock(&chip->lock); - if (ret < 0) - return ret; - - /** - * Preserve the ret variable if the call to - * isl29028_set_pm_runtime_busy() is successful so the reading - * (if applicable) is returned to user space. - */ - pm_ret = isl29028_set_pm_runtime_busy(chip, false); - if (pm_ret < 0) - return pm_ret; + isl29028_set_pm_runtime_busy(chip, false); return ret; } diff --git a/drivers/iio/light/ltr501.c b/drivers/iio/light/ltr501.c index 8c0b616815b2..3c9e1dc01951 100644 --- a/drivers/iio/light/ltr501.c +++ b/drivers/iio/light/ltr501.c @@ -1558,7 +1558,7 @@ static int ltr501_probe(struct i2c_client *client) if (ret) { dev_err(&client->dev, "request irq (%d) failed\n", client->irq); - return ret; + goto powerdown_on_error; } } else { indio_dev->info = data->chip_info->info_no_irq; diff --git a/drivers/iio/pressure/dps310.c b/drivers/iio/pressure/dps310.c index 5a986d5d6b66..aa2284b38878 100644 --- a/drivers/iio/pressure/dps310.c +++ b/drivers/iio/pressure/dps310.c @@ -50,9 +50,9 @@ #define DPS310_CFG_REG 0x09 #define DPS310_INT_HL BIT(7) #define DPS310_TMP_SHIFT_EN BIT(3) -#define DPS310_PRS_SHIFT_EN BIT(4) -#define DPS310_FIFO_EN BIT(5) -#define DPS310_SPI_EN BIT(6) +#define DPS310_PRS_SHIFT_EN BIT(2) +#define DPS310_FIFO_EN BIT(1) +#define DPS310_SPI_EN BIT(0) #define DPS310_RESET 0x0c #define DPS310_RESET_MAGIC 0x09 #define DPS310_COEF_BASE 0x10 diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c index db96a7547fb7..721f0d367281 100644 --- a/drivers/infiniband/hw/hfi1/chip.c +++ b/drivers/infiniband/hw/hfi1/chip.c @@ -5576,7 +5576,7 @@ static int init_rcverr(struct hfi1_devdata *dd) static void free_rcverr(struct hfi1_devdata *dd) { if (dd->rcverr_timer.function) - del_timer_sync(&dd->rcverr_timer); + timer_shutdown_sync(&dd->rcverr_timer); } static void handle_rxe_err(struct hfi1_devdata *dd, u32 unused, u64 reg) @@ -12308,7 +12308,7 @@ static void free_cntrs(struct hfi1_devdata *dd) int i; if (dd->synth_stats_timer.function) - del_timer_sync(&dd->synth_stats_timer); + timer_shutdown_sync(&dd->synth_stats_timer); cancel_work_sync(&dd->update_cntr_work); ppd = (struct hfi1_pportdata *)(dd + 1); for (i = 0; i < dd->num_pports; i++, ppd++) { diff --git a/drivers/input/input.c b/drivers/input/input.c index c51858f1cdc5..05366b82f722 100644 --- a/drivers/input/input.c +++ b/drivers/input/input.c @@ -392,7 +392,7 @@ void input_event(struct input_dev *dev, { unsigned long flags; - if (is_event_supported(type, dev->evbit, EV_MAX)) { + if (type == EV_SYN || is_event_supported(type, dev->evbit, EV_MAX)) { spin_lock_irqsave(&dev->event_lock, flags); input_handle_event(dev, type, code, value); @@ -419,7 +419,7 @@ void input_inject_event(struct input_handle *handle, struct input_handle *grab; unsigned long flags; - if (is_event_supported(type, dev->evbit, EV_MAX)) { + if (type == EV_SYN || is_event_supported(type, dev->evbit, EV_MAX)) { spin_lock_irqsave(&dev->event_lock, flags); rcu_read_lock(); diff --git a/drivers/input/joystick/xpad.c b/drivers/input/joystick/xpad.c index 3086fc4a0642..5e83766a308f 100644 --- a/drivers/input/joystick/xpad.c +++ b/drivers/input/joystick/xpad.c @@ -432,6 +432,7 @@ static const struct xpad_device { { 0x3537, 0x1004, "GameSir T4 Kaleid", 0, XTYPE_XBOX360 }, { 0x3537, 0x100f, "GameSir Nova 2 Lite", 0, XTYPE_XBOX360 }, { 0x3537, 0x1010, "GameSir G7 SE", 0, XTYPE_XBOXONE }, + { 0x3537, 0x1040, "GameSir Nova Lite", 0, XTYPE_XBOX360 }, { 0x3767, 0x0101, "Fanatec Speedster 3 Forceshock Wheel", 0, XTYPE_XBOX }, { 0x413d, 0x2104, "Black Shark Green Ghost Gamepad", 0, XTYPE_XBOX360 }, { 0xffff, 0xffff, "Chinese-made Xbox Controller", 0, XTYPE_XBOX }, diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index 54aa09e1174d..835b24524aa8 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -1579,6 +1579,21 @@ static int __init init_iommu_from_acpi(struct amd_iommu *iommu, return 0; } +/* + * Apply workaround for Zen1 to Zen5(BRH) generation of processors. + * References: CVE-2024-21963 + */ +static bool __init amd_iommu_force_max_dev_table(void) +{ + if (boot_cpu_data.x86 >= 0x17 && boot_cpu_data.x86 <= 0x1a) { + if (cpu_feature_enabled(X86_FEATURE_ZEN6)) + return false; + else + return true; + } + return false; +} + /* Allocate PCI segment data structure */ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id, struct acpi_table_header *ivrs_base) @@ -1601,7 +1616,13 @@ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id, pci_seg->last_bdf = last_bdf; DUMP_printk("PCI segment : 0x%0x, last bdf : 0x%04x\n", id, last_bdf); - pci_seg->dev_table_size = tbl_size(DEV_TABLE_ENTRY_SIZE, last_bdf); + if (amd_iommu_force_max_dev_table()) { + pr_info("Force device table to maximum size\n"); + pci_seg->dev_table_size = SZ_2M; + } else { + pci_seg->dev_table_size = tbl_size(DEV_TABLE_ENTRY_SIZE, + last_bdf); + } pci_seg->alias_table_size = tbl_size(ALIAS_TABLE_ENTRY_SIZE, last_bdf); pci_seg->rlookup_table_size = tbl_size(RLOOKUP_TABLE_ENTRY_SIZE, last_bdf); @@ -2570,6 +2591,15 @@ static int __init init_unity_map_range(struct ivmd_header *m, return 0; } +static bool ivrs_block_type_ivmd(u8 type) +{ + if (type == ACPI_IVMD_TYPE || + type == ACPI_IVMD_TYPE_ALL || + type == ACPI_IVMD_TYPE_RANGE) + return true; + return false; +} + /* iterates over all memory definitions we find in the ACPI table */ static int __init init_memory_definitions(struct acpi_table_header *table) { @@ -2581,10 +2611,11 @@ static int __init init_memory_definitions(struct acpi_table_header *table) while (p < end) { m = (struct ivmd_header *)p; + p += m->length; + if (!ivrs_block_type_ivmd(m->type)) + continue; if (m->flags & (IVMD_FLAG_UNITY_MAP | IVMD_FLAG_EXCL_RANGE)) init_unity_map_range(m, table); - - p += m->length; } return 0; diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c index 177e8e12f189..382150147f4d 100644 --- a/drivers/iommu/iommufd/io_pagetable.c +++ b/drivers/iommu/iommufd/io_pagetable.c @@ -505,7 +505,7 @@ iommu_read_and_clear_dirty(struct iommu_domain *domain, arg.iopt = iopt; arg.domain = domain; arg.dirty = &dirty; - iova_bitmap_for_each(iter, &arg, __iommu_read_and_clear_dirty); + ret = iova_bitmap_for_each(iter, &arg, __iommu_read_and_clear_dirty); if (!(flags & IOMMU_DIRTY_NO_CLEAR)) iommu_iotlb_sync(domain, &gather); diff --git a/drivers/iommu/iommufd/iova_bitmap.c b/drivers/iommu/iommufd/iova_bitmap.c index 0943a351fb91..e12a36bf6aed 100644 --- a/drivers/iommu/iommufd/iova_bitmap.c +++ b/drivers/iommu/iommufd/iova_bitmap.c @@ -116,6 +116,9 @@ struct iova_bitmap { /* length of the IOVA range for the whole bitmap */ size_t length; + + /* first error encountered while recording dirty IOVAs */ + int err; }; /* @@ -384,7 +387,9 @@ static int iova_bitmap_advance_to(struct iova_bitmap *bitmap, int iova_bitmap_for_each(struct iova_bitmap *bitmap, void *opaque, iova_bitmap_fn_t fn) { - return fn(bitmap, bitmap->iova, bitmap->length, opaque); + int ret = fn(bitmap, bitmap->iova, bitmap->length, opaque); + + return ret ? ret : bitmap->err; } EXPORT_SYMBOL_NS_GPL(iova_bitmap_for_each, IOMMUFD); @@ -404,6 +409,18 @@ void iova_bitmap_set(struct iova_bitmap *bitmap, struct iova_bitmap_map *mapped = &bitmap->mapped; unsigned long cur_bit, last_bit, last_page_idx; + /* + * If the range cannot be recorded then the system call returns failure. + * This is to prevent loosing dirties if userspace is expecting to step + * the bitmap through the IOVA space. + */ + if (!length || length > bitmap->length || iova < bitmap->iova || + iova - bitmap->iova > bitmap->length - length) { + if (!bitmap->err) + bitmap->err = -EPROTO; + return; + } + update_indexes: if (unlikely(!iova_bitmap_mapped_range(mapped, iova, length))) { diff --git a/drivers/iommu/virtio-iommu.c b/drivers/iommu/virtio-iommu.c index b85ce6310ddb..f84a50596f4d 100644 --- a/drivers/iommu/virtio-iommu.c +++ b/drivers/iommu/virtio-iommu.c @@ -1191,12 +1191,12 @@ static int viommu_probe(struct virtio_device *vdev) /* Populate the event queue with buffers */ ret = viommu_fill_evtq(viommu); if (ret) - goto err_free_vqs; + goto err_reset_vdev; ret = iommu_device_sysfs_add(&viommu->iommu, dev, NULL, "%s", virtio_bus_name(vdev)); if (ret) - goto err_free_vqs; + goto err_reset_vdev; iommu_device_register(&viommu->iommu, &viommu_ops, parent_dev); @@ -1208,6 +1208,8 @@ static int viommu_probe(struct virtio_device *vdev) return 0; +err_reset_vdev: + virtio_reset_device(vdev); err_free_vqs: vdev->config->del_vqs(vdev); diff --git a/drivers/irqchip/irq-atmel-aic.c b/drivers/irqchip/irq-atmel-aic.c index 3839ad79ad31..46c1b380c604 100644 --- a/drivers/irqchip/irq-atmel-aic.c +++ b/drivers/irqchip/irq-atmel-aic.c @@ -125,11 +125,12 @@ static void aic_resume(struct irq_data *d) static void aic_pm_shutdown(struct irq_data *d) { struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d); + unsigned long flags; - irq_gc_lock(gc); + irq_gc_lock_irqsave(gc, flags); irq_reg_writel(gc, 0xffffffff, AT91_AIC_IDCR); irq_reg_writel(gc, 0xffffffff, AT91_AIC_ICCR); - irq_gc_unlock(gc); + irq_gc_unlock_irqrestore(gc, flags); } #else #define aic_suspend NULL diff --git a/drivers/irqchip/irq-atmel-aic5.c b/drivers/irqchip/irq-atmel-aic5.c index c0f55dc7b050..28dd53df0d7e 100644 --- a/drivers/irqchip/irq-atmel-aic5.c +++ b/drivers/irqchip/irq-atmel-aic5.c @@ -223,15 +223,16 @@ static void aic5_pm_shutdown(struct irq_data *d) struct irq_domain_chip_generic *dgc = domain->gc; struct irq_chip_generic *bgc = irq_get_domain_generic_chip(domain, 0); struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d); + unsigned long flags; int i; - irq_gc_lock(bgc); + irq_gc_lock_irqsave(bgc, flags); for (i = 0; i < dgc->irqs_per_chip; i++) { irq_reg_writel(bgc, i + gc->irq_base, AT91_AIC5_SSR); irq_reg_writel(bgc, 1, AT91_AIC5_IDCR); irq_reg_writel(bgc, 1, AT91_AIC5_ICCR); } - irq_gc_unlock(bgc); + irq_gc_unlock_irqrestore(bgc, flags); } #else #define aic5_suspend NULL diff --git a/drivers/mailbox/mailbox-altera.c b/drivers/mailbox/mailbox-altera.c index afb320e9d69c..e81c8bac651a 100644 --- a/drivers/mailbox/mailbox-altera.c +++ b/drivers/mailbox/mailbox-altera.c @@ -270,7 +270,7 @@ static void altera_mbox_shutdown(struct mbox_chan *chan) writel_relaxed(~0, mbox->mbox_base + MAILBOX_INTMASK_REG); free_irq(mbox->irq, chan); } else if (!mbox->is_sender) { - del_timer_sync(&mbox->rxpoll_timer); + timer_shutdown_sync(&mbox->rxpoll_timer); } } diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c index 601c3e3fe56b..d2a4fc35c838 100644 --- a/drivers/md/dm-bufio.c +++ b/drivers/md/dm-bufio.c @@ -2526,6 +2526,13 @@ struct dm_bufio_client *dm_bufio_client_create(struct block_device *bdev, unsign init_waitqueue_head(&c->free_buffer_wait); c->async_write_error = 0; + /* + * A client with no buffers yet is not holding the oldest buffer in + * the system; leaving this zeroed would claim exactly that and keep + * the client pinned at the front of dm_bufio_all_clients. + */ + c->oldest_buffer = jiffies; + c->dm_io = dm_io_client_create(); if (IS_ERR(c->dm_io)) { r = PTR_ERR(c->dm_io); @@ -2861,9 +2868,23 @@ static unsigned long __evict_a_few(unsigned long nr_buffers) count = __evict_many(c, ¶ms, LIST_CLEAN, nr_buffers); dm_bufio_unlock(c); - if (count) + if (count) { c->oldest_buffer = params.last_accessed; - __insert_client(c); + __insert_client(c); + } else { + /* + * We evicted nothing, so 'oldest_buffer' tells us nothing new + * and the client keeps its old, possibly very stale, value. + * Requeue it at the back rather than by that value, otherwise + * __insert_client() would put it straight back at the front + * and every subsequent call would pop this same client again, + * never reaching any of the others. + * + * This leaves the list slightly out of order, which is fine - + * it is only ever an approximation. + */ + list_add_tail(&c->client_list, &dm_bufio_all_clients); + } return count; } @@ -2888,11 +2909,23 @@ static void evict_old(void) { unsigned long threshold = dm_bufio_cache_size - dm_bufio_cache_size / DM_BUFIO_LOW_WATERMARK_RATIO; + int unproductive = 0; mutex_lock(&dm_bufio_clients_lock); while (dm_bufio_current_allocated > threshold) { - if (!__evict_a_few(64)) - break; + if (__evict_a_few(64)) { + unproductive = 0; + } else { + /* + * This client had nothing to give. Try the others + * before giving up: one idle client, or one whose + * buffers are all held, must not stop us evicting + * from everybody else. Stop once a whole sweep of + * the client list has freed nothing. + */ + if (++unproductive >= dm_bufio_client_count) + break; + } cond_resched(); } mutex_unlock(&dm_bufio_clients_lock); diff --git a/drivers/media/common/cypress_firmware.c b/drivers/media/common/cypress_firmware.c index cdc7050ed3ac..3468eff508b0 100644 --- a/drivers/media/common/cypress_firmware.c +++ b/drivers/media/common/cypress_firmware.c @@ -59,6 +59,8 @@ static int cypress_get_hexline(const struct firmware *fw, if (hx->type == 0x04) { /* b[4] and b[5] are the Extended linear address record data * field */ + if (hx->len != 2) + return -EINVAL; hx->addr |= (b[4] << 24) | (b[5] << 16); } diff --git a/drivers/media/i2c/adv748x/adv748x-core.c b/drivers/media/i2c/adv748x/adv748x-core.c index 3eb6d5e8f082..70594d001804 100644 --- a/drivers/media/i2c/adv748x/adv748x-core.c +++ b/drivers/media/i2c/adv748x/adv748x-core.c @@ -675,9 +675,6 @@ static int adv748x_parse_dt(struct adv748x_state *state) continue; } - of_node_get(ep_np); - state->endpoints[ep.port] = ep_np; - /* * At least one input endpoint and one output endpoint shall * be defined. @@ -689,8 +686,13 @@ static int adv748x_parse_dt(struct adv748x_state *state) /* Store number of CSI-2 lanes used for TXA and TXB. */ ret = adv748x_parse_csi2_lanes(state, ep.port, ep_np); - if (ret) + if (ret) { + of_node_put(ep_np); return ret; + } + + of_node_get(ep_np); + state->endpoints[ep.port] = ep_np; } return in_found && out_found ? 0 : -ENODEV; @@ -739,7 +741,7 @@ static int adv748x_probe(struct i2c_client *client) ret = adv748x_parse_dt(state); if (ret) { adv_err(state, "Failed to parse device tree"); - goto err_free_mutex; + goto err_cleanup_dt; } /* Configure IO Regmap region */ @@ -809,7 +811,6 @@ static int adv748x_probe(struct i2c_client *client) adv748x_unregister_clients(state); err_cleanup_dt: adv748x_dt_cleanup(state); -err_free_mutex: mutex_destroy(&state->mutex); return ret; diff --git a/drivers/media/i2c/og01a1b.c b/drivers/media/i2c/og01a1b.c index 68573122cd6e..cc8ff6ffd29a 100644 --- a/drivers/media/i2c/og01a1b.c +++ b/drivers/media/i2c/og01a1b.c @@ -442,9 +442,6 @@ struct og01a1b { /* Current mode */ const struct og01a1b_mode *cur_mode; - - /* To serialize asynchronus callbacks */ - struct mutex mutex; }; static u64 to_pixel_rate(u32 f_index) @@ -616,7 +613,6 @@ static int og01a1b_init_controls(struct og01a1b *og01a1b) if (ret) return ret; - ctrl_hdlr->lock = &og01a1b->mutex; og01a1b->link_freq = v4l2_ctrl_new_int_menu(ctrl_hdlr, &og01a1b_ctrl_ops, V4L2_CID_LINK_FREQ, @@ -686,74 +682,66 @@ static void og01a1b_update_pad_format(const struct og01a1b_mode *mode, fmt->field = V4L2_FIELD_NONE; } -static int og01a1b_start_streaming(struct og01a1b *og01a1b) +static int og01a1b_enable_streams(struct v4l2_subdev *sd, + struct v4l2_subdev_state *state, u32 pad, + u64 streams_mask) { + struct og01a1b *og01a1b = to_og01a1b(sd); + unsigned int link_freq_index = og01a1b->cur_mode->link_freq_index; const struct og01a1b_reg_list *reg_list; - int link_freq_index, ret; + int ret; - link_freq_index = og01a1b->cur_mode->link_freq_index; - reg_list = &link_freq_configs[link_freq_index].reg_list; + ret = pm_runtime_resume_and_get(og01a1b->dev); + if (ret) + return ret; + reg_list = &link_freq_configs[link_freq_index].reg_list; ret = og01a1b_write_reg_list(og01a1b, reg_list); if (ret) { - dev_err(og01a1b->dev, "failed to set plls"); - return ret; + dev_err(og01a1b->dev, "failed to set plls: %d\n", ret); + goto error; } reg_list = &og01a1b->cur_mode->reg_list; ret = og01a1b_write_reg_list(og01a1b, reg_list); if (ret) { - dev_err(og01a1b->dev, "failed to set mode"); - return ret; + dev_err(og01a1b->dev, "failed to set mode: %d\n", ret); + goto error; } ret = __v4l2_ctrl_handler_setup(og01a1b->sd.ctrl_handler); if (ret) - return ret; + goto error; ret = og01a1b_write_reg(og01a1b, OG01A1B_REG_MODE_SELECT, OG01A1B_REG_VALUE_08BIT, OG01A1B_MODE_STREAMING); if (ret) { - dev_err(og01a1b->dev, "failed to set stream"); - return ret; + dev_err(og01a1b->dev, "failed to start streaming: %d\n", ret); + goto error; } return 0; -} -static void og01a1b_stop_streaming(struct og01a1b *og01a1b) -{ - if (og01a1b_write_reg(og01a1b, OG01A1B_REG_MODE_SELECT, - OG01A1B_REG_VALUE_08BIT, OG01A1B_MODE_STANDBY)) - dev_err(og01a1b->dev, "failed to set stream"); +error: + pm_runtime_put_autosuspend(og01a1b->dev); + + return ret; } -static int og01a1b_set_stream(struct v4l2_subdev *sd, int enable) +static int og01a1b_disable_streams(struct v4l2_subdev *sd, + struct v4l2_subdev_state *state, u32 pad, + u64 streams_mask) { struct og01a1b *og01a1b = to_og01a1b(sd); - int ret = 0; - - mutex_lock(&og01a1b->mutex); - if (enable) { - ret = pm_runtime_resume_and_get(og01a1b->dev); - if (ret) { - mutex_unlock(&og01a1b->mutex); - return ret; - } + int ret; - ret = og01a1b_start_streaming(og01a1b); - if (ret) { - enable = 0; - og01a1b_stop_streaming(og01a1b); - pm_runtime_put(og01a1b->dev); - } - } else { - og01a1b_stop_streaming(og01a1b); - pm_runtime_put(og01a1b->dev); - } + ret = og01a1b_write_reg(og01a1b, OG01A1B_REG_MODE_SELECT, + OG01A1B_REG_VALUE_08BIT, OG01A1B_MODE_STANDBY); + if (ret) + dev_err(og01a1b->dev, "failed to stop streaming: %d\n", ret); - mutex_unlock(&og01a1b->mutex); + pm_runtime_put_autosuspend(og01a1b->dev); return ret; } @@ -771,7 +759,6 @@ static int og01a1b_set_format(struct v4l2_subdev *sd, height, fmt->format.width, fmt->format.height); - mutex_lock(&og01a1b->mutex); og01a1b_update_pad_format(mode, &fmt->format); if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) { *v4l2_subdev_state_get_format(sd_state, fmt->pad) = fmt->format; @@ -794,8 +781,6 @@ static int og01a1b_set_format(struct v4l2_subdev *sd, h_blank); } - mutex_unlock(&og01a1b->mutex); - return 0; } @@ -805,15 +790,12 @@ static int og01a1b_get_format(struct v4l2_subdev *sd, { struct og01a1b *og01a1b = to_og01a1b(sd); - mutex_lock(&og01a1b->mutex); if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) fmt->format = *v4l2_subdev_state_get_format(sd_state, fmt->pad); else og01a1b_update_pad_format(og01a1b->cur_mode, &fmt->format); - mutex_unlock(&og01a1b->mutex); - return 0; } @@ -849,18 +831,14 @@ static int og01a1b_enum_frame_size(struct v4l2_subdev *sd, static int og01a1b_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh) { - struct og01a1b *og01a1b = to_og01a1b(sd); - - mutex_lock(&og01a1b->mutex); og01a1b_update_pad_format(&supported_modes[0], v4l2_subdev_state_get_format(fh->state, 0)); - mutex_unlock(&og01a1b->mutex); return 0; } static const struct v4l2_subdev_video_ops og01a1b_video_ops = { - .s_stream = og01a1b_set_stream, + .s_stream = v4l2_subdev_s_stream_helper, }; static const struct v4l2_subdev_pad_ops og01a1b_pad_ops = { @@ -868,6 +846,8 @@ static const struct v4l2_subdev_pad_ops og01a1b_pad_ops = { .get_fmt = og01a1b_get_format, .enum_mbus_code = og01a1b_enum_mbus_code, .enum_frame_size = og01a1b_enum_frame_size, + .enable_streams = og01a1b_enable_streams, + .disable_streams = og01a1b_disable_streams, }; static const struct v4l2_subdev_ops og01a1b_subdev_ops = { @@ -1062,7 +1042,11 @@ static void og01a1b_remove(struct i2c_client *client) media_entity_cleanup(&sd->entity); v4l2_ctrl_handler_free(sd->ctrl_handler); pm_runtime_disable(og01a1b->dev); - mutex_destroy(&og01a1b->mutex); + + if (!pm_runtime_status_suspended(og01a1b->dev)) { + og01a1b_power_off(og01a1b->dev); + pm_runtime_set_suspended(og01a1b->dev); + } } static int og01a1b_probe(struct i2c_client *client) @@ -1146,7 +1130,6 @@ static int og01a1b_probe(struct i2c_client *client) goto power_off; } - mutex_init(&og01a1b->mutex); og01a1b->cur_mode = &supported_modes[0]; ret = og01a1b_init_controls(og01a1b); if (ret) { @@ -1154,6 +1137,7 @@ static int og01a1b_probe(struct i2c_client *client) goto probe_error_v4l2_ctrl_handler_free; } + og01a1b->sd.state_lock = og01a1b->ctrl_handler.lock; og01a1b->sd.internal_ops = &og01a1b_internal_ops; og01a1b->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE; og01a1b->sd.entity.ops = &og01a1b_subdev_entity_ops; @@ -1194,7 +1178,6 @@ static int og01a1b_probe(struct i2c_client *client) probe_error_v4l2_ctrl_handler_free: v4l2_ctrl_handler_free(og01a1b->sd.ctrl_handler); - mutex_destroy(&og01a1b->mutex); power_off: og01a1b_power_off(og01a1b->dev); diff --git a/drivers/media/i2c/ov5647.c b/drivers/media/i2c/ov5647.c index 305677913a0c..3e7fd8f28df3 100644 --- a/drivers/media/i2c/ov5647.c +++ b/drivers/media/i2c/ov5647.c @@ -1481,6 +1481,10 @@ static void ov5647_remove(struct i2c_client *client) v4l2_ctrl_handler_free(&sensor->ctrls); v4l2_device_unregister_subdev(sd); pm_runtime_disable(&client->dev); + if (!pm_runtime_status_suspended(&client->dev)) { + ov5647_power_off(&client->dev); + pm_runtime_set_suspended(&client->dev); + } mutex_destroy(&sensor->lock); } diff --git a/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c b/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c index 62ed92ff1d30..51d02649d7a7 100644 --- a/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c +++ b/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c @@ -218,7 +218,6 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys, unsigned int max_send_queues, max_sram_blocks, max_devq_size; struct ipu6_fw_syscom_queue_config *input_queue_cfg; struct ipu6_fw_syscom_queue_config *output_queue_cfg; - struct device *dev = &isys->adev->auxdev.dev; int type_proxy = IPU6_FW_ISYS_QUEUE_TYPE_PROXY; int type_dev = IPU6_FW_ISYS_QUEUE_TYPE_DEV; int type_msg = IPU6_FW_ISYS_QUEUE_TYPE_MSG; @@ -228,7 +227,6 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys, struct ipu6_fw_isys_fw_config *isys_fw_cfg; u32 num_in_message_queues; unsigned int max_streams; - unsigned int size; unsigned int i; max_streams = isys->pdata->ipdata->max_streams; @@ -236,7 +234,7 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys, max_sram_blocks = isys->pdata->ipdata->max_sram_blocks; max_devq_size = isys->pdata->ipdata->max_devq_size; num_in_message_queues = clamp(num_streams, 1U, max_streams); - isys_fw_cfg = devm_kzalloc(dev, sizeof(*isys_fw_cfg), GFP_KERNEL); + isys_fw_cfg = kzalloc_obj(*isys_fw_cfg); if (!isys_fw_cfg) return -ENOMEM; @@ -248,15 +246,14 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys, isys_fw_cfg->num_recv_queues[type_dev] = 0; isys_fw_cfg->num_recv_queues[type_msg] = 1; - size = sizeof(*input_queue_cfg) * max_send_queues; - input_queue_cfg = devm_kzalloc(dev, size, GFP_KERNEL); + input_queue_cfg = kzalloc_objs(*input_queue_cfg, max_send_queues); if (!input_queue_cfg) - return -ENOMEM; + goto err_free_fw_cfg; - size = sizeof(*output_queue_cfg) * IPU6_N_MAX_RECV_QUEUES; - output_queue_cfg = devm_kzalloc(dev, size, GFP_KERNEL); + output_queue_cfg = kzalloc_objs(*output_queue_cfg, + IPU6_N_MAX_RECV_QUEUES); if (!output_queue_cfg) - return -ENOMEM; + goto err_free_input_queue_cfg; fwcom->input = input_queue_cfg; fwcom->output = output_queue_cfg; @@ -319,6 +316,13 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys, fwcom->specific_size = sizeof(*isys_fw_cfg); return 0; + +err_free_input_queue_cfg: + kfree(input_queue_cfg); +err_free_fw_cfg: + kfree(isys_fw_cfg); + + return -ENOMEM; } int ipu6_fw_isys_init(struct ipu6_isys *isys, unsigned int num_streams) @@ -332,10 +336,15 @@ int ipu6_fw_isys_init(struct ipu6_isys *isys, unsigned int num_streams) }; int ret; - ipu6_isys_fwcom_cfg_init(isys, &fwcom, num_streams); + ret = ipu6_isys_fwcom_cfg_init(isys, &fwcom, num_streams); + if (ret) + return ret; isys->fwcom = ipu6_fw_com_prepare(&fwcom, isys->adev, isys->pdata->base); + kfree(fwcom.specific_addr); + kfree(fwcom.output); + kfree(fwcom.input); if (!isys->fwcom) { dev_err(dev, "isys fw com prepare failed\n"); return -EIO; diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu.c b/drivers/media/platform/chips-media/wave5/wave5-vpu.c index 5ec14a7c70c7..d38be4761d4e 100644 --- a/drivers/media/platform/chips-media/wave5/wave5-vpu.c +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu.c @@ -300,6 +300,13 @@ static int wave5_vpu_probe(struct platform_device *pdev) dev_err(&pdev->dev, "failed to get irq resource, falling back to polling\n"); sema_init(&dev->irq_sem, 1); dev->irq_thread = kthread_run(irq_thread, dev, "irq thread"); + if (IS_ERR(dev->irq_thread)) { + ret = PTR_ERR(dev->irq_thread); + dev_err(&pdev->dev, "failed to create irq thread\n"); + dev->irq_thread = NULL; + goto err_vdi_release; + } + hrtimer_init(&dev->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL_PINNED); dev->hrtimer.function = &wave5_vpu_timer_callback; dev->worker = kthread_create_worker(0, "vpu_irq_thread"); diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c index 3a7027459731..a69ca8fc5701 100644 --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c @@ -1584,7 +1584,7 @@ static irqreturn_t mtk_jpeg_enc_done(struct mtk_jpeg_dev *jpeg) v4l2_m2m_buf_done(src_buf, buf_state); v4l2_m2m_buf_done(dst_buf, buf_state); v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx); - pm_runtime_put(ctx->jpeg->dev); + pm_runtime_put(jpeg->dev); return IRQ_HANDLED; } @@ -1871,7 +1871,7 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv) v4l2_m2m_buf_done(src_buf, buf_state); v4l2_m2m_buf_done(dst_buf, buf_state); v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx); - pm_runtime_put(ctx->jpeg->dev); + pm_runtime_put(jpeg->dev); return IRQ_HANDLED; } diff --git a/drivers/media/platform/microchip/microchip-isc-base.c b/drivers/media/platform/microchip/microchip-isc-base.c index 97cd1f74ff2b..9c8d02722f92 100644 --- a/drivers/media/platform/microchip/microchip-isc-base.c +++ b/drivers/media/platform/microchip/microchip-isc-base.c @@ -8,6 +8,7 @@ * Author: Eugen Hristev * */ +#include #include #include #include @@ -62,17 +63,17 @@ static inline void isc_update_awb_ctrls(struct isc_device *isc) /* In here we set our actual hw pipeline config */ regmap_write(isc->regmap, ISC_WB_O_RGR, - ((ctrls->offset[ISC_HIS_CFG_MODE_R])) | - ((ctrls->offset[ISC_HIS_CFG_MODE_GR]) << 16)); + FIELD_PREP(ISC_WB_O_LO, ctrls->offset[ISC_HIS_CFG_MODE_R]) | + FIELD_PREP(ISC_WB_O_HI, ctrls->offset[ISC_HIS_CFG_MODE_GR])); regmap_write(isc->regmap, ISC_WB_O_BGB, - ((ctrls->offset[ISC_HIS_CFG_MODE_B])) | - ((ctrls->offset[ISC_HIS_CFG_MODE_GB]) << 16)); + FIELD_PREP(ISC_WB_O_LO, ctrls->offset[ISC_HIS_CFG_MODE_B]) | + FIELD_PREP(ISC_WB_O_HI, ctrls->offset[ISC_HIS_CFG_MODE_GB])); regmap_write(isc->regmap, ISC_WB_G_RGR, - ctrls->gain[ISC_HIS_CFG_MODE_R] | - (ctrls->gain[ISC_HIS_CFG_MODE_GR] << 16)); + FIELD_PREP(ISC_WB_G_LO, ctrls->gain[ISC_HIS_CFG_MODE_R]) | + FIELD_PREP(ISC_WB_G_HI, ctrls->gain[ISC_HIS_CFG_MODE_GR])); regmap_write(isc->regmap, ISC_WB_G_BGB, - ctrls->gain[ISC_HIS_CFG_MODE_B] | - (ctrls->gain[ISC_HIS_CFG_MODE_GB] << 16)); + FIELD_PREP(ISC_WB_G_LO, ctrls->gain[ISC_HIS_CFG_MODE_B]) | + FIELD_PREP(ISC_WB_G_HI, ctrls->gain[ISC_HIS_CFG_MODE_GB])); } static inline void isc_reset_awb_ctrls(struct isc_device *isc) @@ -291,7 +292,8 @@ static int isc_configure(struct isc_device *isc) struct isc_subdev_entity *subdev = isc->current_subdev; int ret; - pfe_cfg0 = isc->config.sd_format->pfe_cfg0_bps; + pfe_cfg0 = FIELD_PREP(ISC_PFE_CFG0_BPS_MASK, + isc->config.sd_format->pfe_cfg0_bps); pipeline = isc->config.bits_pipeline; dcfg = isc->config.dcfg_imode | isc->dcfg; diff --git a/drivers/media/platform/microchip/microchip-isc-regs.h b/drivers/media/platform/microchip/microchip-isc-regs.h index e77e1d9a1db8..fe145b142b82 100644 --- a/drivers/media/platform/microchip/microchip-isc-regs.h +++ b/drivers/media/platform/microchip/microchip-isc-regs.h @@ -31,11 +31,11 @@ #define ISC_PFE_CFG0_MODE_PROGRESSIVE (0x0 << 4) #define ISC_PFE_CFG0_MODE_MASK GENMASK(6, 4) -#define ISC_PFE_CFG0_BPS_EIGHT (0x4 << 28) -#define ISC_PFG_CFG0_BPS_NINE (0x3 << 28) -#define ISC_PFG_CFG0_BPS_TEN (0x2 << 28) -#define ISC_PFG_CFG0_BPS_ELEVEN (0x1 << 28) -#define ISC_PFG_CFG0_BPS_TWELVE (0x0 << 28) +#define ISC_PFE_CFG0_BPS_EIGHT 0x4 +#define ISC_PFE_CFG0_BPS_NINE 0x3 +#define ISC_PFE_CFG0_BPS_TEN 0x2 +#define ISC_PFE_CFG0_BPS_ELEVEN 0x1 +#define ISC_PFE_CFG0_BPS_TWELVE 0x0 #define ISC_PFE_CFG0_BPS_MASK GENMASK(30, 28) #define ISC_PFE_CFG0_COLEN BIT(12) @@ -149,6 +149,12 @@ /* ISC White Balance Gain for B, GB Register */ #define ISC_WB_G_BGB 0x0000006c +/* Each WB offset/gain register packs two 13-bit fields, low and high */ +#define ISC_WB_O_LO GENMASK(12, 0) /* R or B offset [12:0] */ +#define ISC_WB_O_HI GENMASK(28, 16) /* GR or GB offset [28:16] */ +#define ISC_WB_G_LO GENMASK(12, 0) /* R or B gain [12:0] */ +#define ISC_WB_G_HI GENMASK(28, 16) /* GR or GB gain [28:16] */ + /* ISC Color Filter Array Control Register */ #define ISC_CFA_CTRL 0x00000070 diff --git a/drivers/media/platform/microchip/microchip-isc.h b/drivers/media/platform/microchip/microchip-isc.h index f5e322c2e36b..d7bcd74efff9 100644 --- a/drivers/media/platform/microchip/microchip-isc.h +++ b/drivers/media/platform/microchip/microchip-isc.h @@ -62,7 +62,9 @@ struct isc_subdev_entity { * @mbus_code: V4L2 media bus format code. * @cfa_baycfg: If this format is RAW BAYER, indicate the type of bayer. this is either BGBG, RGRG, etc. - * @pfe_cfg0_bps: Number of hardware data lines connected to the ISC + * @pfe_cfg0_bps: ISC_PFE_CFG0 BPS field value (e.g. ISC_PFE_CFG0_BPS_EIGHT), + written into PFE_CFG0 with FIELD_PREP(ISC_PFE_CFG0_BPS_MASK) + at configure time. * @raw: If the format is raw bayer. */ diff --git a/drivers/media/platform/microchip/microchip-sama5d2-isc.c b/drivers/media/platform/microchip/microchip-sama5d2-isc.c index 583d3c4dca54..a5d9d37b4c40 100644 --- a/drivers/media/platform/microchip/microchip-sama5d2-isc.c +++ b/drivers/media/platform/microchip/microchip-sama5d2-isc.c @@ -147,49 +147,49 @@ static struct isc_format sama5d2_formats_list[] = { { .fourcc = V4L2_PIX_FMT_SBGGR10, .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, - .cfa_baycfg = ISC_BAY_CFG_RGRG, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, + .cfa_baycfg = ISC_BAY_CFG_BGBG, }, { .fourcc = V4L2_PIX_FMT_SGBRG10, .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, .cfa_baycfg = ISC_BAY_CFG_GBGB, }, { .fourcc = V4L2_PIX_FMT_SGRBG10, .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, .cfa_baycfg = ISC_BAY_CFG_GRGR, }, { .fourcc = V4L2_PIX_FMT_SRGGB10, .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, .cfa_baycfg = ISC_BAY_CFG_RGRG, }, { .fourcc = V4L2_PIX_FMT_SBGGR12, .mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_BGBG, }, { .fourcc = V4L2_PIX_FMT_SGBRG12, .mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_GBGB, }, { .fourcc = V4L2_PIX_FMT_SGRBG12, .mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_GRGR, }, { .fourcc = V4L2_PIX_FMT_SRGGB12, .mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_RGRG, }, { @@ -210,7 +210,7 @@ static struct isc_format sama5d2_formats_list[] = { { .fourcc = V4L2_PIX_FMT_Y10, .mbus_code = MEDIA_BUS_FMT_Y10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, }, }; diff --git a/drivers/media/platform/microchip/microchip-sama7g5-isc.c b/drivers/media/platform/microchip/microchip-sama7g5-isc.c index d2e1899869c4..ee4f66526bd6 100644 --- a/drivers/media/platform/microchip/microchip-sama7g5-isc.c +++ b/drivers/media/platform/microchip/microchip-sama7g5-isc.c @@ -156,49 +156,49 @@ static struct isc_format sama7g5_formats_list[] = { { .fourcc = V4L2_PIX_FMT_SBGGR10, .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, - .cfa_baycfg = ISC_BAY_CFG_RGRG, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, + .cfa_baycfg = ISC_BAY_CFG_BGBG, }, { .fourcc = V4L2_PIX_FMT_SGBRG10, .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, .cfa_baycfg = ISC_BAY_CFG_GBGB, }, { .fourcc = V4L2_PIX_FMT_SGRBG10, .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, .cfa_baycfg = ISC_BAY_CFG_GRGR, }, { .fourcc = V4L2_PIX_FMT_SRGGB10, .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, .cfa_baycfg = ISC_BAY_CFG_RGRG, }, { .fourcc = V4L2_PIX_FMT_SBGGR12, .mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_BGBG, }, { .fourcc = V4L2_PIX_FMT_SGBRG12, .mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_GBGB, }, { .fourcc = V4L2_PIX_FMT_SGRBG12, .mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_GRGR, }, { .fourcc = V4L2_PIX_FMT_SRGGB12, .mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE, .cfa_baycfg = ISC_BAY_CFG_RGRG, }, { @@ -224,7 +224,7 @@ static struct isc_format sama7g5_formats_list[] = { { .fourcc = V4L2_PIX_FMT_Y10, .mbus_code = MEDIA_BUS_FMT_Y10_1X10, - .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN, + .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN, }, }; diff --git a/drivers/media/platform/qcom/camss/camss-vfe-17x.c b/drivers/media/platform/qcom/camss/camss-vfe-17x.c index bf9aab765858..62c2405359a0 100644 --- a/drivers/media/platform/qcom/camss/camss-vfe-17x.c +++ b/drivers/media/platform/qcom/camss/camss-vfe-17x.c @@ -14,8 +14,6 @@ #include "camss.h" #include "camss-vfe.h" -#define VFE_HW_VERSION (0x000) - #define VFE_GLOBAL_RESET_CMD (0x018) #define GLOBAL_RESET_CMD_CORE BIT(0) #define GLOBAL_RESET_CMD_CAMIF BIT(1) @@ -176,20 +174,6 @@ #define VFE_BUS_WM_FRAME_INC(n) (0x2258 + (n) * 0x100) #define VFE_BUS_WM_BURST_LIMIT(n) (0x225c + (n) * 0x100) -static u32 vfe_hw_version(struct vfe_device *vfe) -{ - u32 hw_version = readl_relaxed(vfe->base + VFE_HW_VERSION); - - u32 gen = (hw_version >> 28) & 0xF; - u32 rev = (hw_version >> 16) & 0xFFF; - u32 step = hw_version & 0xFFFF; - - dev_dbg(vfe->camss->dev, "VFE HW Version = %u.%u.%u\n", - gen, rev, step); - - return hw_version; -} - static inline void vfe_reg_set(struct vfe_device *vfe, u32 reg, u32 set_bits) { u32 bits = readl_relaxed(vfe->base + reg); @@ -398,101 +382,6 @@ static int vfe_halt(struct vfe_device *vfe) return 0; } -static int vfe_get_output(struct vfe_line *line) -{ - struct vfe_device *vfe = to_vfe(line); - struct vfe_output *output; - unsigned long flags; - int wm_idx; - - spin_lock_irqsave(&vfe->output_lock, flags); - - output = &line->output; - if (output->state > VFE_OUTPUT_RESERVED) { - dev_err(vfe->camss->dev, "Output is running\n"); - goto error; - } - - output->wm_num = 1; - - wm_idx = vfe_reserve_wm(vfe, line->id); - if (wm_idx < 0) { - dev_err(vfe->camss->dev, "Can not reserve wm\n"); - goto error_get_wm; - } - output->wm_idx[0] = wm_idx; - - output->drop_update_idx = 0; - - spin_unlock_irqrestore(&vfe->output_lock, flags); - - return 0; - -error_get_wm: - vfe_release_wm(vfe, output->wm_idx[0]); - output->state = VFE_OUTPUT_OFF; -error: - spin_unlock_irqrestore(&vfe->output_lock, flags); - - return -EINVAL; -} - -static int vfe_enable_output(struct vfe_line *line) -{ - struct vfe_device *vfe = to_vfe(line); - struct vfe_output *output = &line->output; - const struct vfe_hw_ops *ops = vfe->res->hw_ops; - struct media_entity *sensor; - unsigned long flags; - unsigned int frame_skip = 0; - unsigned int i; - - sensor = camss_find_sensor(&line->subdev.entity); - if (sensor) { - struct v4l2_subdev *subdev = media_entity_to_v4l2_subdev(sensor); - - v4l2_subdev_call(subdev, sensor, g_skip_frames, &frame_skip); - /* Max frame skip is 29 frames */ - if (frame_skip > VFE_FRAME_DROP_VAL - 1) - frame_skip = VFE_FRAME_DROP_VAL - 1; - } - - spin_lock_irqsave(&vfe->output_lock, flags); - - ops->reg_update_clear(vfe, line->id); - - if (output->state > VFE_OUTPUT_RESERVED) { - dev_err(vfe->camss->dev, "Output is not in reserved state %d\n", - output->state); - spin_unlock_irqrestore(&vfe->output_lock, flags); - return -EINVAL; - } - - WARN_ON(output->gen2.active_num); - - output->state = VFE_OUTPUT_ON; - - output->sequence = 0; - output->wait_reg_update = 0; - reinit_completion(&output->reg_update); - - vfe_wm_start(vfe, output->wm_idx[0], line); - - for (i = 0; i < 2; i++) { - output->buf[i] = vfe_buf_get_pending(output); - if (!output->buf[i]) - break; - output->gen2.active_num++; - vfe_wm_update(vfe, output->wm_idx[0], output->buf[i]->addr[0], line); - } - - ops->reg_update(vfe, line->id); - - spin_unlock_irqrestore(&vfe->output_lock, flags); - - return 0; -} - /* * vfe_enable - Enable streaming on VFE line * @line: VFE line @@ -513,11 +402,11 @@ static int vfe_enable(struct vfe_line *line) mutex_unlock(&vfe->stream_lock); - ret = vfe_get_output(line); + ret = vfe_get_output_v2(line); if (ret < 0) goto error_get_output; - ret = vfe_enable_output(line); + ret = vfe_enable_output_v2(line); if (ret < 0) goto error_enable_output; @@ -626,40 +515,6 @@ static void vfe_isr_wm_done(struct vfe_device *vfe, u8 wm) spin_unlock_irqrestore(&vfe->output_lock, flags); } -/* - * vfe_queue_buffer - Add empty buffer - * @vid: Video device structure - * @buf: Buffer to be enqueued - * - * Add an empty buffer - depending on the current number of buffers it will be - * put in pending buffer queue or directly given to the hardware to be filled. - * - * Return 0 on success or a negative error code otherwise - */ -static int vfe_queue_buffer(struct camss_video *vid, - struct camss_buffer *buf) -{ - struct vfe_line *line = container_of(vid, struct vfe_line, video_out); - struct vfe_device *vfe = to_vfe(line); - struct vfe_output *output; - unsigned long flags; - - output = &line->output; - - spin_lock_irqsave(&vfe->output_lock, flags); - - if (output->state == VFE_OUTPUT_ON && output->gen2.active_num < 2) { - output->buf[output->gen2.active_num++] = buf; - vfe_wm_update(vfe, output->wm_idx[0], buf->addr[0], line); - } else { - vfe_buf_add_pending(output, buf); - } - - spin_unlock_irqrestore(&vfe->output_lock, flags); - - return 0; -} - static const struct vfe_isr_ops vfe_isr_ops_170 = { .reset_ack = vfe_isr_reset_ack, .halt_ack = vfe_isr_halt_ack, @@ -670,7 +525,7 @@ static const struct vfe_isr_ops vfe_isr_ops_170 = { }; static const struct camss_video_ops vfe_video_ops_170 = { - .queue_buffer = vfe_queue_buffer, + .queue_buffer = vfe_queue_buffer_v2, .flush_buffers = vfe_flush_buffers, }; @@ -694,5 +549,7 @@ const struct vfe_hw_ops vfe_ops_170 = { .vfe_enable = vfe_enable, .vfe_halt = vfe_halt, .violation_read = vfe_violation_read, + .vfe_wm_start = vfe_wm_start, .vfe_wm_stop = vfe_wm_stop, + .vfe_wm_update = vfe_wm_update, }; diff --git a/drivers/media/platform/qcom/camss/camss-vfe-4-1.c b/drivers/media/platform/qcom/camss/camss-vfe-4-1.c index 1bd3a6ef1d04..6930799f77c2 100644 --- a/drivers/media/platform/qcom/camss/camss-vfe-4-1.c +++ b/drivers/media/platform/qcom/camss/camss-vfe-4-1.c @@ -210,15 +210,6 @@ #define MSM_VFE_VFE0_UB_SIZE 1023 #define MSM_VFE_VFE0_UB_SIZE_RDI (MSM_VFE_VFE0_UB_SIZE / 3) -static u32 vfe_hw_version(struct vfe_device *vfe) -{ - u32 hw_version = readl_relaxed(vfe->base + VFE_0_HW_VERSION); - - dev_dbg(vfe->camss->dev, "VFE HW Version = 0x%08x\n", hw_version); - - return hw_version; -} - static u16 vfe_get_ub_size(u8 vfe_id) { if (vfe_id == 0) diff --git a/drivers/media/platform/qcom/camss/camss-vfe-4-7.c b/drivers/media/platform/qcom/camss/camss-vfe-4-7.c index ce0719106bd3..76729607db02 100644 --- a/drivers/media/platform/qcom/camss/camss-vfe-4-7.c +++ b/drivers/media/platform/qcom/camss/camss-vfe-4-7.c @@ -18,8 +18,6 @@ #include "camss-vfe-gen1.h" -#define VFE_0_HW_VERSION 0x000 - #define VFE_0_GLOBAL_RESET_CMD 0x018 #define VFE_0_GLOBAL_RESET_CMD_CORE BIT(0) #define VFE_0_GLOBAL_RESET_CMD_CAMIF BIT(1) @@ -254,15 +252,6 @@ #define MSM_VFE_VFE1_UB_SIZE 1535 #define MSM_VFE_VFE1_UB_SIZE_RDI (MSM_VFE_VFE1_UB_SIZE / 3) -static u32 vfe_hw_version(struct vfe_device *vfe) -{ - u32 hw_version = readl_relaxed(vfe->base + VFE_0_HW_VERSION); - - dev_dbg(vfe->camss->dev, "VFE HW Version = 0x%08x\n", hw_version); - - return hw_version; -} - static u16 vfe_get_ub_size(u8 vfe_id) { if (vfe_id == 0) diff --git a/drivers/media/platform/qcom/camss/camss-vfe-4-8.c b/drivers/media/platform/qcom/camss/camss-vfe-4-8.c index 6b59c8107a3c..b2f7d855d8dd 100644 --- a/drivers/media/platform/qcom/camss/camss-vfe-4-8.c +++ b/drivers/media/platform/qcom/camss/camss-vfe-4-8.c @@ -17,8 +17,6 @@ #include "camss-vfe.h" #include "camss-vfe-gen1.h" -#define VFE_0_HW_VERSION 0x000 - #define VFE_0_GLOBAL_RESET_CMD 0x018 #define VFE_0_GLOBAL_RESET_CMD_CORE BIT(0) #define VFE_0_GLOBAL_RESET_CMD_CAMIF BIT(1) @@ -247,15 +245,6 @@ #define MSM_VFE_VFE1_UB_SIZE 1535 #define MSM_VFE_VFE1_UB_SIZE_RDI (MSM_VFE_VFE1_UB_SIZE / 3) -static u32 vfe_hw_version(struct vfe_device *vfe) -{ - u32 hw_version = readl_relaxed(vfe->base + VFE_0_HW_VERSION); - - dev_dbg(vfe->camss->dev, "VFE HW Version = 0x%08x\n", hw_version); - - return hw_version; -} - static inline void vfe_reg_clr(struct vfe_device *vfe, u32 reg, u32 clr_bits) { u32 bits = readl_relaxed(vfe->base + reg); diff --git a/drivers/media/platform/qcom/camss/camss-vfe-480.c b/drivers/media/platform/qcom/camss/camss-vfe-480.c index 003e1900d3c9..d73f733fde04 100644 --- a/drivers/media/platform/qcom/camss/camss-vfe-480.c +++ b/drivers/media/platform/qcom/camss/camss-vfe-480.c @@ -15,8 +15,6 @@ #include "camss.h" #include "camss-vfe.h" -#define VFE_HW_VERSION (0x00) - #define VFE_GLOBAL_RESET_CMD (vfe_is_lite(vfe) ? 0x0c : 0x1c) #define GLOBAL_RESET_HW_AND_REG (vfe_is_lite(vfe) ? BIT(1) : BIT(0)) @@ -92,19 +90,6 @@ static inline int bus_irq_mask_0_comp_done(struct vfe_device *vfe, int n) #define MAX_VFE_OUTPUT_LINES 4 -static u32 vfe_hw_version(struct vfe_device *vfe) -{ - u32 hw_version = readl_relaxed(vfe->base + VFE_HW_VERSION); - - u32 gen = (hw_version >> 28) & 0xF; - u32 rev = (hw_version >> 16) & 0xFFF; - u32 step = hw_version & 0xFFFF; - - dev_dbg(vfe->camss->dev, "VFE HW Version = %u.%u.%u\n", gen, rev, step); - - return hw_version; -} - static void vfe_global_reset(struct vfe_device *vfe) { writel_relaxed(IRQ_MASK_0_RESET_ACK, vfe->base + VFE_IRQ_MASK(0)); @@ -167,18 +152,16 @@ static inline void vfe_reg_update_clear(struct vfe_device *vfe, vfe->reg_update &= ~REG_UPDATE_RDI(vfe, line_id); } -static void vfe_enable_irq_common(struct vfe_device *vfe) -{ - /* enable reset ack IRQ and top BUS status IRQ */ - writel_relaxed(IRQ_MASK_0_RESET_ACK | IRQ_MASK_0_BUS_TOP_IRQ, - vfe->base + VFE_IRQ_MASK(0)); -} - -static void vfe_enable_lines_irq(struct vfe_device *vfe) +static void vfe_enable_irq(struct vfe_device *vfe) { int i; u32 bus_irq_mask = 0; + if (!vfe->stream_count) + /* enable reset ack IRQ and top BUS status IRQ */ + writel(IRQ_MASK_0_RESET_ACK | IRQ_MASK_0_BUS_TOP_IRQ, + vfe->base + VFE_IRQ_MASK(0)); + for (i = 0; i < MAX_VFE_OUTPUT_LINES; i++) { /* Enable IRQ for newly added lines, but also keep already running lines's IRQ */ if (vfe->line[i].output.state == VFE_OUTPUT_RESERVED || @@ -188,11 +171,10 @@ static void vfe_enable_lines_irq(struct vfe_device *vfe) } } - writel_relaxed(bus_irq_mask, vfe->base + VFE_BUS_IRQ_MASK(0)); + writel(bus_irq_mask, vfe->base + VFE_BUS_IRQ_MASK(0)); } static void vfe_isr_reg_update(struct vfe_device *vfe, enum vfe_line_id line_id); -static void vfe_isr_wm_done(struct vfe_device *vfe, u8 wm); /* * vfe_isr - VFE module interrupt handler @@ -228,7 +210,7 @@ static irqreturn_t vfe_isr(int irq, void *dev) /* Loop through all WMs IRQs */ for (i = 0; i < MSM_VFE_IMAGE_MASTERS_NUM; i++) { if (status & BUS_IRQ_MASK_0_COMP_DONE(vfe, RDI_COMP_GROUP(i))) - vfe_isr_wm_done(vfe, i); + vfe_buf_done(vfe, i); } } @@ -247,132 +229,6 @@ static int vfe_halt(struct vfe_device *vfe) return 0; } -static int vfe_get_output(struct vfe_line *line) -{ - struct vfe_device *vfe = to_vfe(line); - struct vfe_output *output; - unsigned long flags; - - spin_lock_irqsave(&vfe->output_lock, flags); - - output = &line->output; - if (output->state > VFE_OUTPUT_RESERVED) { - dev_err(vfe->camss->dev, "Output is running\n"); - goto error; - } - - output->wm_num = 1; - - /* Correspondence between VFE line number and WM number. - * line 0 -> RDI 0, line 1 -> RDI1, line 2 -> RDI2, line 3 -> PIX/RDI3 - * Note this 1:1 mapping will not work for PIX streams. - */ - output->wm_idx[0] = line->id; - vfe->wm_output_map[line->id] = line->id; - - output->drop_update_idx = 0; - - spin_unlock_irqrestore(&vfe->output_lock, flags); - - return 0; - -error: - spin_unlock_irqrestore(&vfe->output_lock, flags); - output->state = VFE_OUTPUT_OFF; - - return -EINVAL; -} - -static int vfe_enable_output(struct vfe_line *line) -{ - struct vfe_device *vfe = to_vfe(line); - struct vfe_output *output = &line->output; - unsigned long flags; - unsigned int i; - - spin_lock_irqsave(&vfe->output_lock, flags); - - vfe_reg_update_clear(vfe, line->id); - - if (output->state > VFE_OUTPUT_RESERVED) { - dev_err(vfe->camss->dev, "Output is not in reserved state %d\n", - output->state); - spin_unlock_irqrestore(&vfe->output_lock, flags); - return -EINVAL; - } - - WARN_ON(output->gen2.active_num); - - output->state = VFE_OUTPUT_ON; - - output->sequence = 0; - output->wait_reg_update = 0; - reinit_completion(&output->reg_update); - - vfe_wm_start(vfe, output->wm_idx[0], line); - - for (i = 0; i < 2; i++) { - output->buf[i] = vfe_buf_get_pending(output); - if (!output->buf[i]) - break; - output->gen2.active_num++; - vfe_wm_update(vfe, output->wm_idx[0], output->buf[i]->addr[0], line); - } - - vfe_reg_update(vfe, line->id); - - spin_unlock_irqrestore(&vfe->output_lock, flags); - - return 0; -} - -/* - * vfe_enable - Enable streaming on VFE line - * @line: VFE line - * - * Return 0 on success or a negative error code otherwise - */ -static int vfe_enable(struct vfe_line *line) -{ - struct vfe_device *vfe = to_vfe(line); - int ret; - - mutex_lock(&vfe->stream_lock); - - if (!vfe->stream_count) - vfe_enable_irq_common(vfe); - - vfe->stream_count++; - - vfe_enable_lines_irq(vfe); - - mutex_unlock(&vfe->stream_lock); - - ret = vfe_get_output(line); - if (ret < 0) - goto error_get_output; - - ret = vfe_enable_output(line); - if (ret < 0) - goto error_enable_output; - - vfe->was_streaming = 1; - - return 0; - -error_enable_output: - vfe_put_output(line); - -error_get_output: - mutex_lock(&vfe->stream_lock); - - vfe->stream_count--; - - mutex_unlock(&vfe->stream_lock); - - return ret; -} - /* * vfe_isr_reg_update - Process reg update interrupt * @vfe: VFE Device @@ -396,114 +252,48 @@ static void vfe_isr_reg_update(struct vfe_device *vfe, enum vfe_line_id line_id) spin_unlock_irqrestore(&vfe->output_lock, flags); } -/* - * vfe_isr_wm_done - Process write master done interrupt - * @vfe: VFE Device - * @wm: Write master id - */ -static void vfe_isr_wm_done(struct vfe_device *vfe, u8 wm) -{ - struct vfe_line *line = &vfe->line[vfe->wm_output_map[wm]]; - struct camss_buffer *ready_buf; - struct vfe_output *output; - unsigned long flags; - u32 index; - u64 ts = ktime_get_ns(); - - spin_lock_irqsave(&vfe->output_lock, flags); - - if (vfe->wm_output_map[wm] == VFE_LINE_NONE) { - dev_err_ratelimited(vfe->camss->dev, - "Received wm done for unmapped index\n"); - goto out_unlock; - } - output = &vfe->line[vfe->wm_output_map[wm]].output; - - ready_buf = output->buf[0]; - if (!ready_buf) { - dev_err_ratelimited(vfe->camss->dev, - "Missing ready buf %d!\n", output->state); - goto out_unlock; - } - - ready_buf->vb.vb2_buf.timestamp = ts; - ready_buf->vb.sequence = output->sequence++; - - index = 0; - output->buf[0] = output->buf[1]; - if (output->buf[0]) - index = 1; - - output->buf[index] = vfe_buf_get_pending(output); - - if (output->buf[index]) - vfe_wm_update(vfe, output->wm_idx[0], output->buf[index]->addr[0], line); - else - output->gen2.active_num--; - - spin_unlock_irqrestore(&vfe->output_lock, flags); - - vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE); - - return; +static const struct camss_video_ops vfe_video_ops_480 = { + .queue_buffer = vfe_queue_buffer_v2, + .flush_buffers = vfe_flush_buffers, +}; -out_unlock: - spin_unlock_irqrestore(&vfe->output_lock, flags); +static void vfe_subdev_init(struct device *dev, struct vfe_device *vfe) +{ + vfe->video_ops = vfe_video_ops_480; } -/* - * vfe_queue_buffer - Add empty buffer - * @vid: Video device structure - * @buf: Buffer to be enqueued - * - * Add an empty buffer - depending on the current number of buffers it will be - * put in pending buffer queue or directly given to the hardware to be filled. - * - * Return 0 on success or a negative error code otherwise - */ -static int vfe_queue_buffer(struct camss_video *vid, - struct camss_buffer *buf) +static void vfe_isr_read(struct vfe_device *vfe, u32 *value0, u32 *value1) { - struct vfe_line *line = container_of(vid, struct vfe_line, video_out); - struct vfe_device *vfe = to_vfe(line); - struct vfe_output *output; - unsigned long flags; - - output = &line->output; - - spin_lock_irqsave(&vfe->output_lock, flags); - - if (output->state == VFE_OUTPUT_ON && output->gen2.active_num < 2) { - output->buf[output->gen2.active_num++] = buf; - vfe_wm_update(vfe, output->wm_idx[0], buf->addr[0], line); - } else { - vfe_buf_add_pending(output, buf); - } - - spin_unlock_irqrestore(&vfe->output_lock, flags); - - return 0; + /* nop */ } -static const struct camss_video_ops vfe_video_ops_480 = { - .queue_buffer = vfe_queue_buffer, - .flush_buffers = vfe_flush_buffers, -}; +static void vfe_violation_read(struct vfe_device *vfe) +{ + /* nop */ +} -static void vfe_subdev_init(struct device *dev, struct vfe_device *vfe) +static void vfe_buf_done_480(struct vfe_device *vfe, int port_id) { - vfe->video_ops = vfe_video_ops_480; + /* nop */ } const struct vfe_hw_ops vfe_ops_480 = { + .enable_irq = vfe_enable_irq, .global_reset = vfe_global_reset, .hw_version = vfe_hw_version, .isr = vfe_isr, + .isr_read = vfe_isr_read, + .reg_update = vfe_reg_update, + .reg_update_clear = vfe_reg_update_clear, .pm_domain_off = vfe_pm_domain_off, .pm_domain_on = vfe_pm_domain_on, .subdev_init = vfe_subdev_init, .vfe_disable = vfe_disable, - .vfe_enable = vfe_enable, + .vfe_enable = vfe_enable_v2, .vfe_halt = vfe_halt, + .violation_read = vfe_violation_read, + .vfe_wm_start = vfe_wm_start, .vfe_wm_stop = vfe_wm_stop, + .vfe_buf_done = vfe_buf_done_480, + .vfe_wm_update = vfe_wm_update, }; diff --git a/drivers/media/platform/qcom/camss/camss-vfe.c b/drivers/media/platform/qcom/camss/camss-vfe.c index 0eeba59dd933..e86e648d8a0d 100644 --- a/drivers/media/platform/qcom/camss/camss-vfe.c +++ b/drivers/media/platform/qcom/camss/camss-vfe.c @@ -32,6 +32,11 @@ #define SCALER_RATIO_MAX 16 +#define VFE_HW_VERSION 0x0 +#define HW_VERSION_STEPPING 0 +#define HW_VERSION_REVISION 16 +#define HW_VERSION_GENERATION 28 + static const struct camss_format_info formats_rdi_8x16[] = { { MEDIA_BUS_FMT_UYVY8_1X16, 8, V4L2_PIX_FMT_UYVY, 1, PER_PLANE_DATA(0, 1, 1, 1, 1, 16) }, @@ -406,6 +411,269 @@ static u32 vfe_src_pad_code(struct vfe_line *line, u32 sink_code, return 0; } +/* + * vfe_hw_version - Process write master done interrupt + * @vfe: VFE Device + * + * Return vfe hw version + */ +u32 vfe_hw_version(struct vfe_device *vfe) +{ + u32 hw_version = readl_relaxed(vfe->base + VFE_HW_VERSION); + + u32 gen = (hw_version >> HW_VERSION_GENERATION) & 0xF; + u32 rev = (hw_version >> HW_VERSION_REVISION) & 0xFFF; + u32 step = (hw_version >> HW_VERSION_STEPPING) & 0xFFFF; + + dev_dbg(vfe->camss->dev, "VFE:%d HW Version = %u.%u.%u\n", + vfe->id, gen, rev, step); + + return hw_version; +} + +/* + * vfe_buf_done - Process write master done interrupt + * @vfe: VFE Device + * @wm: Write master id + */ +void vfe_buf_done(struct vfe_device *vfe, int wm) +{ + struct vfe_line *line = &vfe->line[vfe->wm_output_map[wm]]; + const struct vfe_hw_ops *ops = vfe->res->hw_ops; + struct camss_buffer *ready_buf; + struct vfe_output *output; + unsigned long flags; + u32 index; + u64 ts = ktime_get_ns(); + + spin_lock_irqsave(&vfe->output_lock, flags); + + if (vfe->wm_output_map[wm] == VFE_LINE_NONE) { + dev_err_ratelimited(vfe->camss->dev, + "Received wm done for unmapped index\n"); + goto out_unlock; + } + output = &vfe->line[vfe->wm_output_map[wm]].output; + + ready_buf = output->buf[0]; + if (!ready_buf) { + dev_err_ratelimited(vfe->camss->dev, + "Missing ready buf %d!\n", output->state); + goto out_unlock; + } + + ready_buf->vb.vb2_buf.timestamp = ts; + ready_buf->vb.sequence = output->sequence++; + + index = 0; + output->buf[0] = output->buf[1]; + if (output->buf[0]) + index = 1; + + output->buf[index] = vfe_buf_get_pending(output); + + if (output->buf[index]) { + ops->vfe_wm_update(vfe, output->wm_idx[0], + output->buf[index]->addr[0], + line); + ops->reg_update(vfe, line->id); + } else { + output->gen2.active_num--; + } + + spin_unlock_irqrestore(&vfe->output_lock, flags); + + vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE); + + return; + +out_unlock: + spin_unlock_irqrestore(&vfe->output_lock, flags); +} + +int vfe_enable_output_v2(struct vfe_line *line) +{ + struct vfe_device *vfe = to_vfe(line); + struct vfe_output *output = &line->output; + const struct vfe_hw_ops *ops = vfe->res->hw_ops; + struct media_entity *sensor; + unsigned long flags; + unsigned int frame_skip = 0; + unsigned int i; + + sensor = camss_find_sensor(&line->subdev.entity); + if (sensor) { + struct v4l2_subdev *subdev = media_entity_to_v4l2_subdev(sensor); + + v4l2_subdev_call(subdev, sensor, g_skip_frames, &frame_skip); + /* Max frame skip is 29 frames */ + if (frame_skip > VFE_FRAME_DROP_VAL - 1) + frame_skip = VFE_FRAME_DROP_VAL - 1; + } + + spin_lock_irqsave(&vfe->output_lock, flags); + + ops->reg_update_clear(vfe, line->id); + + if (output->state > VFE_OUTPUT_RESERVED) { + dev_err(vfe->camss->dev, + "Output is not in reserved state %d\n", + output->state); + spin_unlock_irqrestore(&vfe->output_lock, flags); + return -EINVAL; + } + + WARN_ON(output->gen2.active_num); + + output->state = VFE_OUTPUT_ON; + + output->sequence = 0; + output->wait_reg_update = 0; + reinit_completion(&output->reg_update); + + ops->vfe_wm_start(vfe, output->wm_idx[0], line); + + for (i = 0; i < 2; i++) { + output->buf[i] = vfe_buf_get_pending(output); + if (!output->buf[i]) + break; + output->gen2.active_num++; + ops->vfe_wm_update(vfe, output->wm_idx[0], + output->buf[i]->addr[0], line); + ops->reg_update(vfe, line->id); + } + + spin_unlock_irqrestore(&vfe->output_lock, flags); + + return 0; +} + +/* + * vfe_queue_buffer_v2 - Add empty buffer + * @vid: Video device structure + * @buf: Buffer to be enqueued + * + * Add an empty buffer - depending on the current number of buffers it will be + * put in pending buffer queue or directly given to the hardware to be filled. + * + * Return 0 on success or a negative error code otherwise + */ +int vfe_queue_buffer_v2(struct camss_video *vid, + struct camss_buffer *buf) +{ + struct vfe_line *line = container_of(vid, struct vfe_line, video_out); + struct vfe_device *vfe = to_vfe(line); + const struct vfe_hw_ops *ops = vfe->res->hw_ops; + struct vfe_output *output; + unsigned long flags; + + output = &line->output; + + spin_lock_irqsave(&vfe->output_lock, flags); + + if (output->state == VFE_OUTPUT_ON && + output->gen2.active_num < 2) { + output->buf[output->gen2.active_num++] = buf; + ops->vfe_wm_update(vfe, output->wm_idx[0], + buf->addr[0], line); + ops->reg_update(vfe, line->id); + } else { + vfe_buf_add_pending(output, buf); + } + + spin_unlock_irqrestore(&vfe->output_lock, flags); + + return 0; +} + +/* + * vfe_enable_v2 - Enable streaming on VFE line + * @line: VFE line + * + * Return 0 on success or a negative error code otherwise + */ +int vfe_enable_v2(struct vfe_line *line) +{ + struct vfe_device *vfe = to_vfe(line); + const struct vfe_hw_ops *ops = vfe->res->hw_ops; + int ret; + + mutex_lock(&vfe->stream_lock); + + if (vfe->res->hw_ops->enable_irq) + ops->enable_irq(vfe); + + vfe->stream_count++; + + mutex_unlock(&vfe->stream_lock); + + ret = vfe_get_output_v2(line); + if (ret < 0) + goto error_get_output; + + ret = vfe_enable_output_v2(line); + if (ret < 0) + goto error_enable_output; + + vfe->was_streaming = 1; + + return 0; + +error_enable_output: + vfe_put_output(line); + +error_get_output: + mutex_lock(&vfe->stream_lock); + + vfe->stream_count--; + + mutex_unlock(&vfe->stream_lock); + + return ret; +} + +/* + * vfe_get_output_v2 - Get vfe output port for corresponding VFE line + * @line: VFE line + * + * Return 0 on success or a negative error code otherwise + */ +int vfe_get_output_v2(struct vfe_line *line) +{ + struct vfe_device *vfe = to_vfe(line); + struct vfe_output *output; + unsigned long flags; + + spin_lock_irqsave(&vfe->output_lock, flags); + + output = &line->output; + if (output->state > VFE_OUTPUT_RESERVED) { + dev_err(vfe->camss->dev, "Output is running\n"); + goto error; + } + + output->wm_num = 1; + + /* Correspondence between VFE line number and WM number. + * line 0 -> RDI 0, line 1 -> RDI1, line 2 -> RDI2, line 3 -> PIX/RDI3 + * Note this 1:1 mapping will not work for PIX streams. + */ + output->wm_idx[0] = line->id; + vfe->wm_output_map[line->id] = line->id; + + output->drop_update_idx = 0; + + spin_unlock_irqrestore(&vfe->output_lock, flags); + + return 0; + +error: + spin_unlock_irqrestore(&vfe->output_lock, flags); + output->state = VFE_OUTPUT_OFF; + + return -EINVAL; +} + int vfe_reset(struct vfe_device *vfe) { unsigned long time; diff --git a/drivers/media/platform/qcom/camss/camss-vfe.h b/drivers/media/platform/qcom/camss/camss-vfe.h index 10e2cc3c0b83..fcbf4f609129 100644 --- a/drivers/media/platform/qcom/camss/camss-vfe.h +++ b/drivers/media/platform/qcom/camss/camss-vfe.h @@ -99,7 +99,7 @@ struct vfe_line { struct vfe_device; struct vfe_hw_ops { - void (*enable_irq_common)(struct vfe_device *vfe); + void (*enable_irq)(struct vfe_device *vfe); void (*global_reset)(struct vfe_device *vfe); u32 (*hw_version)(struct vfe_device *vfe); irqreturn_t (*isr)(int irq, void *dev); @@ -114,7 +114,12 @@ struct vfe_hw_ops { int (*vfe_enable)(struct vfe_line *line); int (*vfe_halt)(struct vfe_device *vfe); void (*violation_read)(struct vfe_device *vfe); + void (*vfe_wm_start)(struct vfe_device *vfe, u8 wm, + struct vfe_line *line); void (*vfe_wm_stop)(struct vfe_device *vfe, u8 wm); + void (*vfe_buf_done)(struct vfe_device *vfe, int port_id); + void (*vfe_wm_update)(struct vfe_device *vfe, u8 wm, u32 addr, + struct vfe_line *line); }; struct vfe_isr_ops { @@ -252,4 +257,55 @@ void vfe_put(struct vfe_device *vfe); */ bool vfe_is_lite(struct vfe_device *vfe); +/* + * vfe_hw_version - Process write master done interrupt + * @vfe: VFE Device + * + * Return vfe hw version + */ +u32 vfe_hw_version(struct vfe_device *vfe); +/* + * vfe_enable - Enable streaming on VFE line + * @line: VFE line + * + * Return 0 on success or a negative error code otherwise + */ +int vfe_enable_v2(struct vfe_line *line); + +/* + * vfe_buf_done - Process write master done interrupt + * @vfe: VFE Device + * @wm: Write master id + */ +void vfe_buf_done(struct vfe_device *vfe, int wm); + +/* + * vfe_get_output_v2 - Get vfe output line + * line: VFE line + * + * Return 0 on success or a negative error code otherwise + */ +int vfe_get_output_v2(struct vfe_line *line); + +/* + * vfe_enable_output_v2 - Enable vfe output line + * line: VFE line + * + * Return 0 on success or a negative error code otherwise + */ +int vfe_enable_output_v2(struct vfe_line *line); + +/* + * vfe_queue_buffer_v2 - Add empty buffer + * @vid: Video device structure + * @buf: Buffer to be enqueued + * + * Add an empty buffer - depending on the current number of buffers it will be + * put in pending buffer queue or directly given to the hardware to be filled. + * + * Return 0 on success or a negative error code otherwise + */ +int vfe_queue_buffer_v2(struct camss_video *vid, + struct camss_buffer *buf); + #endif /* QC_MSM_CAMSS_VFE_H */ diff --git a/drivers/media/platform/rockchip/rga/rga.c b/drivers/media/platform/rockchip/rga/rga.c index de532b7ecd74..dd512785649a 100644 --- a/drivers/media/platform/rockchip/rga/rga.c +++ b/drivers/media/platform/rockchip/rga/rga.c @@ -704,48 +704,10 @@ static const struct video_device rga_videodev = { .device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING, }; -static int rga_enable_clocks(struct rockchip_rga *rga) -{ - int ret; - - ret = clk_prepare_enable(rga->sclk); - if (ret) { - dev_err(rga->dev, "Cannot enable rga sclk: %d\n", ret); - return ret; - } - - ret = clk_prepare_enable(rga->aclk); - if (ret) { - dev_err(rga->dev, "Cannot enable rga aclk: %d\n", ret); - goto err_disable_sclk; - } - - ret = clk_prepare_enable(rga->hclk); - if (ret) { - dev_err(rga->dev, "Cannot enable rga hclk: %d\n", ret); - goto err_disable_aclk; - } - - return 0; - -err_disable_aclk: - clk_disable_unprepare(rga->aclk); -err_disable_sclk: - clk_disable_unprepare(rga->sclk); - - return ret; -} - -static void rga_disable_clocks(struct rockchip_rga *rga) -{ - clk_disable_unprepare(rga->sclk); - clk_disable_unprepare(rga->hclk); - clk_disable_unprepare(rga->aclk); -} - static int rga_parse_dt(struct rockchip_rga *rga) { struct reset_control *core_rst, *axi_rst, *ahb_rst; + int ret; core_rst = devm_reset_control_get(rga->dev, "core"); if (IS_ERR(core_rst)) { @@ -777,23 +739,12 @@ static int rga_parse_dt(struct rockchip_rga *rga) udelay(1); reset_control_deassert(ahb_rst); - rga->sclk = devm_clk_get(rga->dev, "sclk"); - if (IS_ERR(rga->sclk)) { - dev_err(rga->dev, "failed to get sclk clock\n"); - return PTR_ERR(rga->sclk); - } - - rga->aclk = devm_clk_get(rga->dev, "aclk"); - if (IS_ERR(rga->aclk)) { - dev_err(rga->dev, "failed to get aclk clock\n"); - return PTR_ERR(rga->aclk); - } - - rga->hclk = devm_clk_get(rga->dev, "hclk"); - if (IS_ERR(rga->hclk)) { - dev_err(rga->dev, "failed to get hclk clock\n"); - return PTR_ERR(rga->hclk); + ret = devm_clk_bulk_get_all(rga->dev, &rga->clks); + if (ret < 0) { + dev_err(rga->dev, "failed to get clocks\n"); + return ret; } + rga->num_clks = ret; return 0; } @@ -833,6 +784,7 @@ static int rga_probe(struct platform_device *pdev) ret = irq; goto err_put_clk; } + rga->irq = irq; ret = devm_request_irq(rga->dev, irq, rga_isr, 0, dev_name(rga->dev), rga); @@ -930,8 +882,9 @@ static void rga_remove(struct platform_device *pdev) v4l2_info(&rga->v4l2_dev, "Removing\n"); - v4l2_m2m_release(rga->m2m_dev); video_unregister_device(rga->vfd); + devm_free_irq(rga->dev, rga->irq, rga); + v4l2_m2m_release(rga->m2m_dev); v4l2_device_unregister(&rga->v4l2_dev); pm_runtime_disable(rga->dev); @@ -941,7 +894,7 @@ static int __maybe_unused rga_runtime_suspend(struct device *dev) { struct rockchip_rga *rga = dev_get_drvdata(dev); - rga_disable_clocks(rga); + clk_bulk_disable_unprepare(rga->num_clks, rga->clks); return 0; } @@ -950,7 +903,7 @@ static int __maybe_unused rga_runtime_resume(struct device *dev) { struct rockchip_rga *rga = dev_get_drvdata(dev); - return rga_enable_clocks(rga); + return clk_bulk_prepare_enable(rga->num_clks, rga->clks); } static const struct dev_pm_ops rga_pm = { diff --git a/drivers/media/platform/rockchip/rga/rga.h b/drivers/media/platform/rockchip/rga/rga.h index 8105bb2efe57..8f0185d59210 100644 --- a/drivers/media/platform/rockchip/rga/rga.h +++ b/drivers/media/platform/rockchip/rga/rga.h @@ -6,6 +6,7 @@ #ifndef __RGA_H__ #define __RGA_H__ +#include #include #include #include @@ -76,9 +77,9 @@ struct rockchip_rga { struct device *dev; struct regmap *grf; void __iomem *regs; - struct clk *sclk; - struct clk *aclk; - struct clk *hclk; + int irq; + struct clk_bulk_data *clks; + int num_clks; struct rockchip_rga_version version; /* vfd lock */ diff --git a/drivers/media/rc/ene_ir.c b/drivers/media/rc/ene_ir.c index 3fd51a41c3b2..70e28bd4bdb9 100644 --- a/drivers/media/rc/ene_ir.c +++ b/drivers/media/rc/ene_ir.c @@ -1103,15 +1103,15 @@ static void ene_remove(struct pnp_dev *pnp_dev) unsigned long flags; rc_unregister_device(dev->rdev); - del_timer_sync(&dev->tx_sim_timer); spin_lock_irqsave(&dev->hw_lock, flags); ene_rx_disable(dev); ene_rx_restore_hw_buffer(dev); spin_unlock_irqrestore(&dev->hw_lock, flags); - rc_free_device(dev->rdev); free_irq(dev->irq, dev); + timer_delete_sync(&dev->tx_sim_timer); release_region(dev->hw_io, ENE_IO_SIZE); + rc_free_device(dev->rdev); kfree(dev); } diff --git a/drivers/media/rc/redrat3.c b/drivers/media/rc/redrat3.c index c9a5c030f772..080207aa888c 100644 --- a/drivers/media/rc/redrat3.c +++ b/drivers/media/rc/redrat3.c @@ -773,17 +773,9 @@ static int redrat3_transmit_ir(struct rc_dev *rcdev, unsigned *txbuf, u8 curlencheck = 0; unsigned i, sendbuf_len; - if (rr3->transmitting) { - dev_warn(dev, "%s: transmitter already in use\n", __func__); - return -EAGAIN; - } - if (count > RR3_MAX_SIG_SIZE - RR3_TX_TRAILER_LEN) return -EINVAL; - /* rr3 will disable rc detector on transmit */ - rr3->transmitting = true; - sample_lens = kcalloc(RR3_DRIVER_MAXLENS, sizeof(*sample_lens), GFP_KERNEL); @@ -796,6 +788,9 @@ static int redrat3_transmit_ir(struct rc_dev *rcdev, unsigned *txbuf, goto out; } + /* rr3 will disable rc detector on transmit */ + rr3->transmitting = true; + for (i = 0; i < count; i++) { cur_sample_len = redrat3_us_to_len(txbuf[i]); if (cur_sample_len > 0xffff) { diff --git a/drivers/media/rc/serial_ir.c b/drivers/media/rc/serial_ir.c index fc5fd3927177..f6702b3df9a0 100644 --- a/drivers/media/rc/serial_ir.c +++ b/drivers/media/rc/serial_ir.c @@ -798,7 +798,7 @@ static int __init serial_ir_init_module(void) static void __exit serial_ir_exit_module(void) { - del_timer_sync(&serial_ir.timeout_timer); + timer_shutdown_sync(&serial_ir.timeout_timer); serial_ir_exit(); } diff --git a/drivers/media/usb/dvb-usb/dvb-usb-firmware.c b/drivers/media/usb/dvb-usb/dvb-usb-firmware.c index 0fb3fa6100e4..675d9d1d4f47 100644 --- a/drivers/media/usb/dvb-usb/dvb-usb-firmware.c +++ b/drivers/media/usb/dvb-usb/dvb-usb-firmware.c @@ -141,6 +141,8 @@ int dvb_usb_get_hexline(const struct firmware *fw, struct hexline *hx, if (hx->type == 0x04) { /* b[4] and b[5] are the Extended linear address record data field */ + if (hx->len != 2) + return -EINVAL; hx->addr |= (b[4] << 24) | (b[5] << 16); /* hx->len -= 2; data_offs += 2; */ diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c index d3fa27bf6551..44ce06268065 100644 --- a/drivers/media/usb/uvc/uvc_driver.c +++ b/drivers/media/usb/uvc/uvc_driver.c @@ -2828,6 +2828,26 @@ static const struct usb_device_id uvc_ids[] = { .bInterfaceSubClass = 1, .bInterfaceProtocol = 0, .driver_info = (kernel_ulong_t)&uvc_quirk_fix_bandwidth }, + /* Avermedia GC515 */ + { .match_flags = USB_DEVICE_ID_MATCH_DEVICE + | USB_DEVICE_ID_MATCH_INT_INFO, + .idVendor = 0x07ca, + .idProduct = 0x1515, + .bInterfaceClass = USB_CLASS_VIDEO, + .bInterfaceSubClass = 1, + .bInterfaceProtocol = UVC_PC_PROTOCOL_15, + .driver_info = (kernel_ulong_t)&(const struct uvc_device_info){ + .uvc_version = 0x0110, + } }, + /* AVerMedia Live Gamer HD 2 capture card */ + { .match_flags = USB_DEVICE_ID_MATCH_DEVICE + | USB_DEVICE_ID_MATCH_INT_INFO, + .idVendor = 0x07ca, + .idProduct = 0x1570, + .bInterfaceClass = USB_CLASS_VIDEO, + .bInterfaceSubClass = 1, + .bInterfaceProtocol = 0, + .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_DISABLE_AUTOSUSPEND) }, /* ViMicro Vega */ { .match_flags = USB_DEVICE_ID_MATCH_DEVICE | USB_DEVICE_ID_MATCH_INT_INFO, diff --git a/drivers/mfd/ab8500-core.c b/drivers/mfd/ab8500-core.c index 17c90850946b..53d2b117058c 100644 --- a/drivers/mfd/ab8500-core.c +++ b/drivers/mfd/ab8500-core.c @@ -1195,39 +1195,41 @@ static int ab8500_probe(struct platform_device *pdev) return ret; if (is_ab9540(ab8500)) - ret = mfd_add_devices(ab8500->dev, 0, ab9540_devs, - ARRAY_SIZE(ab9540_devs), NULL, - 0, ab8500->domain); + ret = devm_mfd_add_devices(ab8500->dev, 0, ab9540_devs, + ARRAY_SIZE(ab9540_devs), NULL, + 0, ab8500->domain); else if (is_ab8540(ab8500)) { - ret = mfd_add_devices(ab8500->dev, 0, ab8540_devs, - ARRAY_SIZE(ab8540_devs), NULL, - 0, ab8500->domain); + ret = devm_mfd_add_devices(ab8500->dev, 0, ab8540_devs, + ARRAY_SIZE(ab8540_devs), NULL, + 0, ab8500->domain); if (ret) return ret; if (is_ab8540_1p2_or_earlier(ab8500)) - ret = mfd_add_devices(ab8500->dev, 0, ab8540_cut1_devs, - ARRAY_SIZE(ab8540_cut1_devs), NULL, - 0, ab8500->domain); + ret = devm_mfd_add_devices(ab8500->dev, 0, + ab8540_cut1_devs, + ARRAY_SIZE(ab8540_cut1_devs), + NULL, 0, ab8500->domain); else /* ab8540 >= cut2 */ - ret = mfd_add_devices(ab8500->dev, 0, ab8540_cut2_devs, - ARRAY_SIZE(ab8540_cut2_devs), NULL, - 0, ab8500->domain); + ret = devm_mfd_add_devices(ab8500->dev, 0, + ab8540_cut2_devs, + ARRAY_SIZE(ab8540_cut2_devs), + NULL, 0, ab8500->domain); } else if (is_ab8505(ab8500)) - ret = mfd_add_devices(ab8500->dev, 0, ab8505_devs, - ARRAY_SIZE(ab8505_devs), NULL, - 0, ab8500->domain); + ret = devm_mfd_add_devices(ab8500->dev, 0, ab8505_devs, + ARRAY_SIZE(ab8505_devs), NULL, + 0, ab8500->domain); else - ret = mfd_add_devices(ab8500->dev, 0, ab8500_devs, - ARRAY_SIZE(ab8500_devs), NULL, - 0, ab8500->domain); + ret = devm_mfd_add_devices(ab8500->dev, 0, ab8500_devs, + ARRAY_SIZE(ab8500_devs), NULL, + 0, ab8500->domain); if (ret) return ret; /* Add battery management devices */ - ret = mfd_add_devices(ab8500->dev, 0, ab8500_bm_devs, - ARRAY_SIZE(ab8500_bm_devs), NULL, - 0, ab8500->domain); + ret = devm_mfd_add_devices(ab8500->dev, 0, ab8500_bm_devs, + ARRAY_SIZE(ab8500_bm_devs), NULL, + 0, ab8500->domain); if (ret) dev_err(ab8500->dev, "error adding bm devices\n"); diff --git a/drivers/mfd/twl-core.c b/drivers/mfd/twl-core.c index c130ffef182f..8334066f8e22 100644 --- a/drivers/mfd/twl-core.c +++ b/drivers/mfd/twl-core.c @@ -715,6 +715,11 @@ static const struct mfd_cell twl6032_cells[] = { { .name = "twl6032-clk" }, }; +static void twl_unregister_device(void *data) +{ + platform_device_unregister(data); +} + /* NOTE: This driver only handles a single twl4030/tps659x0 chip */ static int twl_probe(struct i2c_client *client) @@ -750,17 +755,22 @@ twl_probe(struct i2c_client *client) return status; } - if (i2c_check_functionality(client->adapter, I2C_FUNC_I2C) == 0) { + status = devm_add_action_or_reset(&client->dev, twl_unregister_device, + pdev); + if (status) + return status; + + if (!i2c_check_functionality(client->adapter, I2C_FUNC_I2C)) { dev_dbg(&client->dev, "can't talk I2C?\n"); status = -EIO; - goto free; + goto out; } twl_priv = devm_kzalloc(&client->dev, sizeof(struct twl_private), GFP_KERNEL); if (!twl_priv) { status = -ENOMEM; - goto free; + goto out; } if ((id->driver_data) & TWL6030_CLASS) { @@ -780,7 +790,7 @@ twl_probe(struct i2c_client *client) GFP_KERNEL); if (!twl_priv->twl_modules) { status = -ENOMEM; - goto free; + goto out; } for (i = 0; i < num_slaves; i++) { @@ -868,7 +878,7 @@ twl_probe(struct i2c_client *client) ARRAY_SIZE(twl6032_cells), NULL, 0, NULL); if (status < 0) - goto free; + goto out; } if (twl_class_is_6030()) { @@ -886,10 +896,7 @@ twl_probe(struct i2c_client *client) fail: if (status < 0) twl_remove(client); -free: - if (status < 0) - platform_device_unregister(pdev); - +out: return status; } diff --git a/drivers/misc/fastrpc.c b/drivers/misc/fastrpc.c index 576cd9e5b807..a4577ef5a394 100644 --- a/drivers/misc/fastrpc.c +++ b/drivers/misc/fastrpc.c @@ -109,7 +109,7 @@ static const char *domains[FASTRPC_DEV_MAX] = { "adsp", "mdsp", "sdsp", "cdsp", "cdsp1" }; struct fastrpc_phy_page { - u64 addr; /* physical address */ + dma_addr_t addr; /* dma address */ u64 size; /* size of contiguous region */ }; @@ -174,7 +174,7 @@ struct fastrpc_msg { u64 ctx; /* invoke caller context */ u32 handle; /* handle to invoke */ u32 sc; /* scalars structure describing the data */ - u64 addr; /* physical address */ + dma_addr_t addr; /* dma address */ u64 size; /* size of contiguous region */ }; @@ -197,7 +197,7 @@ struct fastrpc_buf { struct dma_buf *dmabuf; struct device *dev; void *virt; - u64 phys; + dma_addr_t dma_addr; u64 size; /* Lock for dma buf attachments */ struct mutex lock; @@ -220,7 +220,7 @@ struct fastrpc_map { struct dma_buf *buf; struct sg_table *table; struct dma_buf_attachment *attach; - u64 phys; + dma_addr_t dma_addr; u64 size; void *va; u64 len; @@ -325,11 +325,12 @@ static void fastrpc_free_map(struct kref *ref) perm.vmid = QCOM_SCM_VMID_HLOS; perm.perm = QCOM_SCM_PERM_RWX; - err = qcom_scm_assign_mem(map->phys, map->len, + err = qcom_scm_assign_mem(map->dma_addr, map->len, &src_perms, &perm, 1); if (err) { - dev_err(map->fl->sctx->dev, "Failed to assign memory phys 0x%llx size 0x%llx err %d\n", - map->phys, map->len, err); + dev_err(map->fl->sctx->dev, + "Failed to assign memory dma_addr %pad size 0x%llx err %d\n", + &map->dma_addr, map->len, err); return; } } @@ -400,7 +401,7 @@ static int fastrpc_map_lookup(struct fastrpc_user *fl, int fd, static void fastrpc_buf_free(struct fastrpc_buf *buf) { dma_free_coherent(buf->dev, buf->size, buf->virt, - FASTRPC_PHYS(buf->phys)); + FASTRPC_PHYS(buf->dma_addr)); kfree(buf); } @@ -419,12 +420,12 @@ static int __fastrpc_buf_alloc(struct fastrpc_user *fl, struct device *dev, buf->fl = fl; buf->virt = NULL; - buf->phys = 0; + buf->dma_addr = 0; buf->size = size; buf->dev = dev; buf->raddr = 0; - buf->virt = dma_alloc_coherent(dev, buf->size, (dma_addr_t *)&buf->phys, + buf->virt = dma_alloc_coherent(dev, buf->size, &buf->dma_addr, GFP_KERNEL); if (!buf->virt) { mutex_destroy(&buf->lock); @@ -450,7 +451,7 @@ static int fastrpc_buf_alloc(struct fastrpc_user *fl, struct device *dev, buf = *obuf; if (fl->sctx && fl->sctx->sid) - buf->phys += ((u64)fl->sctx->sid << 32); + buf->dma_addr += ((u64)fl->sctx->sid << 32); return 0; } @@ -743,7 +744,7 @@ static int fastrpc_dma_buf_attach(struct dma_buf *dmabuf, return -ENOMEM; ret = dma_get_sgtable(buffer->dev, &a->sgt, buffer->virt, - FASTRPC_PHYS(buffer->phys), buffer->size); + FASTRPC_PHYS(buffer->dma_addr), buffer->size); if (ret < 0) { dev_err(buffer->dev, "failed to get scatterlist from DMA API\n"); kfree(a); @@ -792,7 +793,7 @@ static int fastrpc_mmap(struct dma_buf *dmabuf, dma_resv_assert_held(dmabuf->resv); return dma_mmap_coherent(buf->dev, vma, buf->virt, - FASTRPC_PHYS(buf->phys), size); + FASTRPC_PHYS(buf->dma_addr), size); } static const struct dma_buf_ops fastrpc_dma_buf_ops = { @@ -844,10 +845,10 @@ static int fastrpc_map_attach(struct fastrpc_user *fl, int fd, map->table = table; if (attr & FASTRPC_ATTR_SECUREMAP) { - map->phys = sg_phys(map->table->sgl); + map->dma_addr = sg_phys(map->table->sgl); } else { - map->phys = sg_dma_address(map->table->sgl); - map->phys += ((u64)fl->sctx->sid << 32); + map->dma_addr = sg_dma_address(map->table->sgl); + map->dma_addr += ((u64)fl->sctx->sid << 32); } for_each_sg(map->table->sgl, sgl, map->table->nents, sgl_index) @@ -856,7 +857,7 @@ static int fastrpc_map_attach(struct fastrpc_user *fl, int fd, dev_dbg(sess->dev, "Bad size passed len 0x%llx map size 0x%llx\n", len, map->size); err = -EINVAL; - goto map_err; + goto get_err; } map->va = sg_virt(map->table->sgl); map->len = len; @@ -874,11 +875,12 @@ static int fastrpc_map_attach(struct fastrpc_user *fl, int fd, dst_perms[1].vmid = fl->cctx->vmperms[0].vmid; dst_perms[1].perm = QCOM_SCM_PERM_RWX; map->attr = attr; - err = qcom_scm_assign_mem(map->phys, (u64)map->len, &src_perms, dst_perms, 2); + err = qcom_scm_assign_mem(map->dma_addr, (u64)map->len, &src_perms, dst_perms, 2); if (err) { - dev_err(sess->dev, "Failed to assign memory with phys 0x%llx size 0x%llx err %d\n", - map->phys, map->len, err); - goto map_err; + dev_err(sess->dev, + "Failed to assign memory with dma_addr %pad size 0x%llx err %d\n", + &map->dma_addr, map->len, err); + goto get_err; } } spin_lock(&fl->lock); @@ -1059,7 +1061,7 @@ static int fastrpc_get_args(u32 kernel, struct fastrpc_invoke_ctx *ctx) struct vm_area_struct *vma = NULL; rpra[i].buf.pv = (u64) ctx->args[i].ptr; - pages[i].addr = ctx->maps[i]->phys; + pages[i].addr = ctx->maps[i]->dma_addr; mmap_read_lock(current->mm); vma = vma_lookup(current->mm, ctx->args[i].ptr); @@ -1086,7 +1088,7 @@ static int fastrpc_get_args(u32 kernel, struct fastrpc_invoke_ctx *ctx) goto bail; rpra[i].buf.pv = args - ctx->olaps[oix].offset; - pages[i].addr = ctx->buf->phys - + pages[i].addr = ctx->buf->dma_addr - ctx->olaps[oix].offset + (pkt_size - rlen); pages[i].addr = pages[i].addr & PAGE_MASK; @@ -1118,7 +1120,7 @@ static int fastrpc_get_args(u32 kernel, struct fastrpc_invoke_ctx *ctx) list[i].num = ctx->args[i].length ? 1 : 0; list[i].pgidx = i; if (ctx->maps[i]) { - pages[i].addr = ctx->maps[i]->phys; + pages[i].addr = ctx->maps[i]->dma_addr; pages[i].size = ctx->maps[i]->size; } rpra[i].dma.fd = ctx->args[i].fd; @@ -1200,7 +1202,7 @@ static int fastrpc_invoke_send(struct fastrpc_session_ctx *sctx, msg->ctx = ctx->ctxid | fl->pd; msg->handle = handle; msg->sc = ctx->sc; - msg->addr = ctx->buf ? ctx->buf->phys : 0; + msg->addr = ctx->buf ? ctx->buf->dma_addr : 0; msg->size = roundup(ctx->msg_sz, PAGE_SIZE); fastrpc_context_get(ctx); @@ -1358,13 +1360,15 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl, if (fl->cctx->vmcount) { u64 src_perms = BIT(QCOM_SCM_VMID_HLOS); - err = qcom_scm_assign_mem(fl->cctx->remote_heap->phys, + err = qcom_scm_assign_mem(fl->cctx->remote_heap->dma_addr, (u64)fl->cctx->remote_heap->size, &src_perms, fl->cctx->vmperms, fl->cctx->vmcount); if (err) { - dev_err(fl->sctx->dev, "Failed to assign memory with phys 0x%llx size 0x%llx err %d\n", - fl->cctx->remote_heap->phys, fl->cctx->remote_heap->size, err); + dev_err(fl->sctx->dev, + "Failed to assign memory with dma_addr %pad size 0x%llx err %d\n", + &fl->cctx->remote_heap->dma_addr, + fl->cctx->remote_heap->size, err); goto err_map; } scm_done = true; @@ -1384,7 +1388,7 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl, args[1].length = inbuf.namelen; args[1].fd = -1; - pages[0].addr = fl->cctx->remote_heap->phys; + pages[0].addr = fl->cctx->remote_heap->dma_addr; pages[0].size = fl->cctx->remote_heap->size; args[2].ptr = (u64)(uintptr_t) pages; @@ -1413,12 +1417,12 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl, dst_perms.vmid = QCOM_SCM_VMID_HLOS; dst_perms.perm = QCOM_SCM_PERM_RWX; - err = qcom_scm_assign_mem(fl->cctx->remote_heap->phys, + err = qcom_scm_assign_mem(fl->cctx->remote_heap->dma_addr, (u64)fl->cctx->remote_heap->size, &src_perms, &dst_perms, 1); if (err) - dev_err(fl->sctx->dev, "Failed to assign memory phys 0x%llx size 0x%llx err %d\n", - fl->cctx->remote_heap->phys, fl->cctx->remote_heap->size, err); + dev_err(fl->sctx->dev, "Failed to assign memory dma_addr %pad size 0x%llx err %d\n", + &fl->cctx->remote_heap->dma_addr, fl->cctx->remote_heap->size, err); } err_map: fastrpc_buf_free(fl->cctx->remote_heap); @@ -1508,7 +1512,7 @@ static int fastrpc_init_create_process(struct fastrpc_user *fl, args[2].length = inbuf.filelen; args[2].fd = init.filefd; - pages[0].addr = imem->phys; + pages[0].addr = imem->dma_addr; pages[0].size = imem->size; args[3].ptr = (u64)(uintptr_t) pages; @@ -1962,7 +1966,7 @@ static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp) args[0].ptr = (u64) (uintptr_t) &req_msg; args[0].length = sizeof(req_msg); - pages.addr = buf->phys; + pages.addr = buf->dma_addr; pages.size = buf->size; args[1].ptr = (u64) (uintptr_t) &pages; @@ -1990,11 +1994,12 @@ static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp) if (req.flags == ADSP_MMAP_REMOTE_HEAP_ADDR && fl->cctx->vmcount) { u64 src_perms = BIT(QCOM_SCM_VMID_HLOS); - err = qcom_scm_assign_mem(buf->phys, (u64)buf->size, + err = qcom_scm_assign_mem(buf->dma_addr, (u64)buf->size, &src_perms, fl->cctx->vmperms, fl->cctx->vmcount); if (err) { - dev_err(fl->sctx->dev, "Failed to assign memory phys 0x%llx size 0x%llx err %d", - buf->phys, buf->size, err); + dev_err(fl->sctx->dev, + "Failed to assign memory dma_addr %pad size 0x%llx err %d", + &buf->dma_addr, buf->size, err); goto err_assign; } } @@ -2108,7 +2113,7 @@ static int fastrpc_req_mem_map(struct fastrpc_user *fl, char __user *argp) args[0].ptr = (u64) (uintptr_t) &req_msg; args[0].length = sizeof(req_msg); - pages.addr = map->phys; + pages.addr = map->dma_addr; pages.size = map->len; args[1].ptr = (u64) (uintptr_t) &pages; diff --git a/drivers/mtd/nand/raw/Kconfig b/drivers/mtd/nand/raw/Kconfig index d0aaccf72d78..91fae26a3d91 100644 --- a/drivers/mtd/nand/raw/Kconfig +++ b/drivers/mtd/nand/raw/Kconfig @@ -71,7 +71,8 @@ config MTD_NAND_AU1550 config MTD_NAND_NDFC tristate "IBM/MCC 4xx NAND controller" - depends on 4xx + depends on 44x || COMPILE_TEST + depends on OF select MTD_NAND_ECC_SW_HAMMING select MTD_NAND_ECC_SW_HAMMING_SMC help diff --git a/drivers/mtd/nand/raw/ndfc.c b/drivers/mtd/nand/raw/ndfc.c index 3bb32a7c6d67..677d05d628a5 100644 --- a/drivers/mtd/nand/raw/ndfc.c +++ b/drivers/mtd/nand/raw/ndfc.c @@ -44,13 +44,13 @@ static void ndfc_select_chip(struct nand_chip *nchip, int chip) uint32_t ccr; struct ndfc_controller *ndfc = nand_get_controller_data(nchip); - ccr = in_be32(ndfc->ndfcbase + NDFC_CCR); + ccr = ioread32be(ndfc->ndfcbase + NDFC_CCR); if (chip >= 0) { ccr &= ~NDFC_CCR_BS_MASK; ccr |= NDFC_CCR_BS(chip + ndfc->chip_select); } else ccr |= NDFC_CCR_RESET_CE; - out_be32(ndfc->ndfcbase + NDFC_CCR, ccr); + iowrite32be(ccr, ndfc->ndfcbase + NDFC_CCR); } static void ndfc_hwcontrol(struct nand_chip *chip, int cmd, unsigned int ctrl) @@ -70,7 +70,7 @@ static int ndfc_ready(struct nand_chip *chip) { struct ndfc_controller *ndfc = nand_get_controller_data(chip); - return in_be32(ndfc->ndfcbase + NDFC_STAT) & NDFC_STAT_IS_READY; + return ioread32be(ndfc->ndfcbase + NDFC_STAT) & NDFC_STAT_IS_READY; } static void ndfc_enable_hwecc(struct nand_chip *chip, int mode) @@ -78,9 +78,9 @@ static void ndfc_enable_hwecc(struct nand_chip *chip, int mode) uint32_t ccr; struct ndfc_controller *ndfc = nand_get_controller_data(chip); - ccr = in_be32(ndfc->ndfcbase + NDFC_CCR); + ccr = ioread32be(ndfc->ndfcbase + NDFC_CCR); ccr |= NDFC_CCR_RESET_ECC; - out_be32(ndfc->ndfcbase + NDFC_CCR, ccr); + iowrite32be(ccr, ndfc->ndfcbase + NDFC_CCR); wmb(); } @@ -92,7 +92,7 @@ static int ndfc_calculate_ecc(struct nand_chip *chip, uint8_t *p = (uint8_t *)&ecc; wmb(); - ecc = in_be32(ndfc->ndfcbase + NDFC_ECC); + ecc = ioread32be(ndfc->ndfcbase + NDFC_ECC); /* The NDFC uses Smart Media (SMC) bytes order */ ecc_code[0] = p[1]; ecc_code[1] = p[2]; @@ -114,7 +114,7 @@ static void ndfc_read_buf(struct nand_chip *chip, uint8_t *buf, int len) uint32_t *p = (uint32_t *) buf; for(;len > 0; len -= 4) - *p++ = in_be32(ndfc->ndfcbase + NDFC_DATA); + *p++ = ioread32be(ndfc->ndfcbase + NDFC_DATA); } static void ndfc_write_buf(struct nand_chip *chip, const uint8_t *buf, int len) @@ -123,7 +123,7 @@ static void ndfc_write_buf(struct nand_chip *chip, const uint8_t *buf, int len) uint32_t *p = (uint32_t *) buf; for(;len > 0; len -= 4) - out_be32(ndfc->ndfcbase + NDFC_DATA, *p++); + iowrite32be(*p++, ndfc->ndfcbase + NDFC_DATA); } /* @@ -185,10 +185,15 @@ static int ndfc_chip_init(struct ndfc_controller *ndfc, static int ndfc_probe(struct platform_device *ofdev) { struct ndfc_controller *ndfc; + void __iomem *ndfcbase; const __be32 *reg; u32 ccr; u32 cs; - int err, len; + int len = 0; + + ndfcbase = devm_platform_ioremap_resource(ofdev, 0); + if (IS_ERR(ndfcbase)) + return PTR_ERR(ndfcbase); /* Read the reg property to get the chip select */ reg = of_get_property(ofdev->dev.of_node, "reg", &len); @@ -210,11 +215,7 @@ static int ndfc_probe(struct platform_device *ofdev) ndfc->ofdev = ofdev; dev_set_drvdata(&ofdev->dev, ndfc); - ndfc->ndfcbase = of_iomap(ofdev->dev.of_node, 0); - if (!ndfc->ndfcbase) { - dev_err(&ofdev->dev, "failed to get memory\n"); - return -EIO; - } + ndfc->ndfcbase = ndfcbase; ccr = NDFC_CCR_BS(ndfc->chip_select); @@ -223,22 +224,16 @@ static int ndfc_probe(struct platform_device *ofdev) if (reg) ccr |= be32_to_cpup(reg); - out_be32(ndfc->ndfcbase + NDFC_CCR, ccr); + iowrite32be(ccr, ndfc->ndfcbase + NDFC_CCR); /* Set the bank settings if given */ reg = of_get_property(ofdev->dev.of_node, "bank-settings", NULL); if (reg) { int offset = NDFC_BCFG0 + (ndfc->chip_select << 2); - out_be32(ndfc->ndfcbase + offset, be32_to_cpup(reg)); + iowrite32be(be32_to_cpup(reg), ndfc->ndfcbase + offset); } - err = ndfc_chip_init(ndfc, ofdev->dev.of_node); - if (err) { - iounmap(ndfc->ndfcbase); - return err; - } - - return 0; + return ndfc_chip_init(ndfc, ofdev->dev.of_node); } static void ndfc_remove(struct platform_device *ofdev) diff --git a/drivers/net/can/dev/netlink.c b/drivers/net/can/dev/netlink.c index 77d165ed0d53..884313a5cdd3 100644 --- a/drivers/net/can/dev/netlink.c +++ b/drivers/net/can/dev/netlink.c @@ -454,7 +454,7 @@ static size_t can_get_size(const struct net_device *dev) size += nla_total_size(sizeof(u32)); /* IFLA_CAN_STATE */ size += nla_total_size(sizeof(struct can_ctrlmode)); /* IFLA_CAN_CTRLMODE */ size += nla_total_size(sizeof(u32)); /* IFLA_CAN_RESTART_MS */ - if (priv->do_get_berr_counter) /* IFLA_CAN_BERR_COUNTER */ + if (priv->do_get_berr_counter && netif_running(dev)) /* IFLA_CAN_BERR_COUNTER */ size += nla_total_size(sizeof(struct can_berr_counter)); if (priv->data_bittiming.bitrate) /* IFLA_CAN_DATA_BITTIMING */ size += nla_total_size(sizeof(struct can_bittiming)); @@ -574,7 +574,7 @@ static int can_fill_info(struct sk_buff *skb, const struct net_device *dev) nla_put(skb, IFLA_CAN_CTRLMODE, sizeof(cm), &cm) || nla_put_u32(skb, IFLA_CAN_RESTART_MS, priv->restart_ms) || - (priv->do_get_berr_counter && + (priv->do_get_berr_counter && netif_running(dev) && !priv->do_get_berr_counter(dev, &bec) && nla_put(skb, IFLA_CAN_BERR_COUNTER, sizeof(bec), &bec)) || diff --git a/drivers/net/can/dev/skb.c b/drivers/net/can/dev/skb.c index 3ebd4f779b9b..6fb1f9b2a9e2 100644 --- a/drivers/net/can/dev/skb.c +++ b/drivers/net/can/dev/skb.c @@ -52,6 +52,7 @@ int can_put_echo_skb(struct sk_buff *skb, struct net_device *dev, if (idx >= priv->echo_skb_max) { netdev_err(dev, "%s: BUG! Trying to access can_priv::echo_skb out of bounds (%u/max %u)\n", __func__, idx, priv->echo_skb_max); + dev_kfree_skb_any(skb); return -EINVAL; } diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c index 6ebfbc0753d9..7954937e828b 100644 --- a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c +++ b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c @@ -754,7 +754,7 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) u16 rx_msg_type, rx_msg_size; size_t rx_msg_min_size; - if (msg_end - msg_ptr < sizeof(*rx_msg)) { + if (msg_end - msg_ptr < sizeof(rx_msg->size)) { err = -EBADMSG; break; } @@ -765,6 +765,11 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) break; } + if (msg_end - msg_ptr < sizeof(*rx_msg)) { + err = -EBADMSG; + break; + } + rx_msg_size = le16_to_cpu(rx_msg->size); rx_msg_type = le16_to_cpu(rx_msg->type); diff --git a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c index 96152e5a920a..5af78b3de09d 100644 --- a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c +++ b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c @@ -1934,6 +1934,39 @@ static int atl1c_clean_rx(struct napi_struct *napi, int budget) break; } atl1c_clean_rfd(rfd_ring, rrs, rfd_num); + + /* A link partner whose l2mtu is larger than ours can hand us a + * frame that does not fit the buffer the hardware was given: + * the descriptor reports the untruncated length while at most + * buffer_info->length bytes were written. skb_put() of such a + * length would pull in whatever the tail of the buffer holds + * and, past the end of the skb, trips skb_over_panic and takes + * the host down. Bound it by what the device actually wrote + * rather than by skb_tailroom(): the skb carries 64 bytes of + * slack for the DMA erratum, so the two are not the same. + * + * Frames shorter than an Ethernet header plus its FCS are + * rejected by the same test; before this, a length below + * ETH_FCS_LEN underflowed in skb_put(). + */ + if (unlikely(length < ETH_HLEN + ETH_FCS_LEN || + length > buffer_info->length)) { + dev_core_stats_rx_dropped_inc(netdev); + if (netif_msg_rx_err(adapter)) + dev_warn_ratelimited(&pdev->dev, + "dropped rx frame: %u bytes, buffer %u\n", + length, buffer_info->length); + kfree_skb_reason(skb, SKB_DROP_REASON_PKT_TOO_BIG); + /* atl1c_clean_rfd() has already recycled the RFD, so + * count the drop as work: the refill at the end of the + * poll only runs when work_done is non-zero, and + * sustained oversized traffic would otherwise drain the + * ring and stop RX for good. + */ + work_done++; + continue; + } + skb_put(skb, length - ETH_FCS_LEN); skb->protocol = eth_type_trans(skb, netdev); atl1c_rx_checksum(adapter, skb, rrs); diff --git a/drivers/net/ethernet/broadcom/bgmac.c b/drivers/net/ethernet/broadcom/bgmac.c index 6ffdc4229407..258b3b5a1cdb 100644 --- a/drivers/net/ethernet/broadcom/bgmac.c +++ b/drivers/net/ethernet/broadcom/bgmac.c @@ -1183,9 +1183,6 @@ static int bgmac_open(struct net_device *net_dev) if (err) return err; - /* Specs say about reclaiming rings here, but we do that in DMA init */ - bgmac_chip_init(bgmac); - err = request_irq(bgmac->irq, bgmac_interrupt, IRQF_SHARED, net_dev->name, net_dev); if (err < 0) { @@ -1195,6 +1192,9 @@ static int bgmac_open(struct net_device *net_dev) } napi_enable(&bgmac->napi); + /* Specs say about reclaiming rings here, but we do that in DMA init */ + bgmac_chip_init(bgmac); + phy_start(net_dev->phydev); netif_start_queue(net_dev); @@ -1609,10 +1609,10 @@ int bgmac_enet_resume(struct bgmac *bgmac) if (rc) return rc; - bgmac_chip_init(bgmac); - napi_enable(&bgmac->napi); + bgmac_chip_init(bgmac); + netif_tx_lock(bgmac->net_dev); netif_device_attach(bgmac->net_dev); netif_tx_unlock(bgmac->net_dev); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c index 2c16348d3b8c..fcdffdc3ffa8 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c @@ -380,12 +380,34 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames, return nxmit; } +static void bnxt_xdp_apply_cfg(struct bnxt *bp, int tx_xdp) +{ + struct net_device *dev = bp->dev; + int tc = bp->num_tc ? : 1; + int tx_cp; + + if (bp->xdp_prog) { + bnxt_set_rx_skb_mode(bp, true); + xdp_features_set_redirect_target(dev, true); + } else { + xdp_features_clear_redirect_target(dev); + bnxt_set_rx_skb_mode(bp, false); + } + bp->tx_nr_rings_xdp = tx_xdp; + bp->tx_nr_rings = bp->tx_nr_rings_per_tc * tc + tx_xdp; + tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings); + bp->cp_nr_rings = max_t(int, tx_cp, bp->rx_nr_rings); + bnxt_set_tpa_flags(bp); + bnxt_set_ring_params(bp); +} + /* Under rtnl_lock */ static int bnxt_xdp_set(struct bnxt *bp, struct bpf_prog *prog) { struct net_device *dev = bp->dev; - int tx_xdp = 0, tx_cp, rc, tc; + int tx_xdp = 0, rc, tc; struct bpf_prog *old; + int old_tx_xdp; if (prog && !prog->aux->xdp_has_frags && bp->dev->mtu > BNXT_MAX_PAGE_MODE_MTU) { @@ -412,26 +434,22 @@ static int bnxt_xdp_set(struct bnxt *bp, struct bpf_prog *prog) if (netif_running(dev)) bnxt_close_nic(bp, true, false); + old_tx_xdp = bp->tx_nr_rings_xdp; old = xchg(&bp->xdp_prog, prog); - if (old) - bpf_prog_put(old); - - if (prog) { - bnxt_set_rx_skb_mode(bp, true); - xdp_features_set_redirect_target(dev, true); - } else { - xdp_features_clear_redirect_target(dev); - bnxt_set_rx_skb_mode(bp, false); + bnxt_xdp_apply_cfg(bp, tx_xdp); + + if (netif_running(dev)) { + rc = bnxt_open_nic(bp, true, false); + /* a detach cannot be undone, only an attach */ + if (rc && prog) { + WRITE_ONCE(bp->xdp_prog, old); + bnxt_xdp_apply_cfg(bp, old_tx_xdp); + return rc; + } } - bp->tx_nr_rings_xdp = tx_xdp; - bp->tx_nr_rings = bp->tx_nr_rings_per_tc * tc + tx_xdp; - tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings); - bp->cp_nr_rings = max_t(int, tx_cp, bp->rx_nr_rings); - bnxt_set_tpa_flags(bp); - bnxt_set_ring_params(bp); - if (netif_running(dev)) - return bnxt_open_nic(bp, true, false); + if (old) + bpf_prog_put(old); return 0; } diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c index 7f143e388d42..190e7f622e7f 100644 --- a/drivers/net/ethernet/broadcom/tg3.c +++ b/drivers/net/ethernet/broadcom/tg3.c @@ -18170,41 +18170,17 @@ static int tg3_resume(struct device *device) static SIMPLE_DEV_PM_OPS(tg3_pm_ops, tg3_suspend, tg3_resume); -/* Systems where ACPI _PTS (Prepare To Sleep) S5 will result in a fatal - * PCIe AER event on the tg3 device if the tg3 device is not, or cannot - * be, powered down. +/* Systems where powering down a tg3 device on restarts triggers + * NMIs and fatal hangs during reboot if the network adapters were + * initialized under SNP. */ -static const struct dmi_system_id tg3_restart_aer_quirk_table[] = { - { - .matches = { - DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."), - DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R440"), - }, - }, - { - .matches = { - DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."), - DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R540"), - }, - }, - { - .matches = { - DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."), - DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R640"), - }, - }, +static const struct dmi_system_id tg3_restart_snp_quirk_table[] = { { .matches = { DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."), DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R650"), }, }, - { - .matches = { - DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."), - DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R740"), - }, - }, { .matches = { DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."), @@ -18228,14 +18204,18 @@ static void tg3_shutdown(struct pci_dev *pdev) if (netif_running(dev)) dev_close(dev); - if (system_state == SYSTEM_POWER_OFF) + if (!dmi_first_match(tg3_restart_snp_quirk_table) || + system_state == SYSTEM_POWER_OFF) { tg3_power_down(tp); - else if (system_state == SYSTEM_RESTART && - dmi_first_match(tg3_restart_aer_quirk_table) && + } else if (system_state == SYSTEM_RESTART && pdev->current_state != PCI_D3cold && pdev->current_state != PCI_UNKNOWN) { /* Disable PCIe AER on the tg3 to avoid a fatal * error during this system restart. + * Needed since on restarts for systems in + * tg3_restart_snp_quirk_table skipping on powering down the + * tg3 device causes a fatal AER error during ACPI _PTS + * (Prepare To Sleep). */ pcie_capability_clear_word(pdev, PCI_EXP_DEVCTL, PCI_EXP_DEVCTL_CERE | diff --git a/drivers/net/ethernet/jme.c b/drivers/net/ethernet/jme.c index d8be0e4dcb07..6cfcfda05554 100644 --- a/drivers/net/ethernet/jme.c +++ b/drivers/net/ethernet/jme.c @@ -1502,7 +1502,8 @@ jme_intr_msi(struct jme_adapter *jme, u32 intrstat) * all other events are ignored */ jwrite32(jme, JME_IEVE, intrstat); - schedule_work(&jme->linkch_task); + if (!test_bit(JME_FLAG_WOL_QUIESCE, &jme->flags)) + schedule_work(&jme->linkch_task); goto out_reenable; } @@ -1546,7 +1547,8 @@ jme_intr_msi(struct jme_adapter *jme, u32 intrstat) /* * Re-enable interrupt */ - jwrite32f(jme, JME_IENS, INTR_ENABLE); + if (!test_bit(JME_FLAG_WOL_QUIESCE, &jme->flags)) + jwrite32f(jme, JME_IENS, INTR_ENABLE); } static irqreturn_t @@ -1892,6 +1894,7 @@ jme_powersave_phy(struct jme_adapter *jme) jme_set_100m_half(jme); if (jme->reg_pmcs & (PMCS_LFEN | PMCS_LREN)) jme_wait_link(jme); + jme_mac_rxclk_on(jme); jme_clear_pm_enable_wol(jme); } else { jme_phy_off(jme); @@ -3158,6 +3161,21 @@ jme_shutdown(struct pci_dev *pdev) struct net_device *netdev = pci_get_drvdata(pdev); struct jme_adapter *jme = netdev_priv(netdev); + if (netif_running(netdev)) { + set_bit(JME_FLAG_WOL_QUIESCE, &jme->flags); + netif_device_detach(netdev); + netif_stop_queue(netdev); + jme_stop_irq(jme); + synchronize_irq(pdev->irq); + jme_stop_irq(jme); + cancel_work_sync(&jme->linkch_task); + tasklet_disable(&jme->pcc_task); + napi_disable(&jme->napi); + tasklet_disable(&jme->txclean_task); + tasklet_disable(&jme->rxclean_task); + tasklet_disable(&jme->rxempty_task); + } + jme_powersave_phy(jme); pci_pme_active(pdev, true); } @@ -3172,11 +3190,16 @@ jme_suspend(struct device *dev) if (!netif_running(netdev)) return 0; - atomic_dec(&jme->link_changing); - + set_bit(JME_FLAG_WOL_QUIESCE, &jme->flags); netif_device_detach(netdev); netif_stop_queue(netdev); jme_stop_irq(jme); + synchronize_irq(jme->pdev->irq); + jme_stop_irq(jme); + cancel_work_sync(&jme->linkch_task); + tasklet_disable(&jme->pcc_task); + napi_disable(&jme->napi); + atomic_dec(&jme->link_changing); tasklet_disable(&jme->txclean_task); tasklet_disable(&jme->rxclean_task); @@ -3215,6 +3238,7 @@ jme_resume(struct device *dev) return 0; jme_clear_pm_disable_wol(jme); + jme_mac_rxclk_off(jme); jme_phy_on(jme); if (test_bit(JME_FLAG_SSET, &jme->flags)) jme_set_link_ksettings(netdev, &jme->old_cmd); @@ -3226,8 +3250,11 @@ jme_resume(struct device *dev) atomic_inc(&jme->link_changing); - jme_reset_link(jme); + napi_enable(&jme->napi); + tasklet_enable(&jme->pcc_task); + clear_bit(JME_FLAG_WOL_QUIESCE, &jme->flags); + jme_reset_link(jme); jme_start_irq(jme); return 0; diff --git a/drivers/net/ethernet/jme.h b/drivers/net/ethernet/jme.h index 860494ff3714..ec5dfddf7322 100644 --- a/drivers/net/ethernet/jme.h +++ b/drivers/net/ethernet/jme.h @@ -450,6 +450,7 @@ enum jme_flags_bits { JME_FLAG_SSET = 2, JME_FLAG_POLL = 5, JME_FLAG_SHUTDOWN = 6, + JME_FLAG_WOL_QUIESCE = 7, }; #define TX_TIMEOUT (5 * HZ) diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.c b/drivers/net/ethernet/mellanox/mlxsw/core.c index 4a79c0d7e7ad..feaeb4118f0b 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core.c +++ b/drivers/net/ethernet/mellanox/mlxsw/core.c @@ -2002,7 +2002,7 @@ mlxsw_core_health_fw_fatal_test(struct devlink_health_reporter *reporter, struct netlink_ext_ack *extack) { struct mlxsw_core *mlxsw_core = devlink_health_reporter_priv(reporter); - char mfgd_pl[MLXSW_REG_MFGD_LEN]; + char mfgd_pl[MLXSW_REG_MFGD_LEN] = {0}; int err; /* Read the register first to make sure no other bits are changed. */ @@ -2023,7 +2023,7 @@ mlxsw_core_health_fw_fatal_ops = { static int mlxsw_core_health_fw_fatal_config(struct mlxsw_core *mlxsw_core, bool enable) { - char mfgd_pl[MLXSW_REG_MFGD_LEN]; + char mfgd_pl[MLXSW_REG_MFGD_LEN] = {0}; int err; /* Read the register first to make sure no other bits are changed. */ diff --git a/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c b/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c index 9c12e1feb643..e39a8b0a5709 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c +++ b/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c @@ -120,7 +120,7 @@ static ssize_t mlxsw_hwmon_temp_rst_store(struct device *dev, container_of(attr, struct mlxsw_hwmon_attr, dev_attr); struct mlxsw_hwmon_dev *mlxsw_hwmon_dev = mlxsw_hwmon_attr->mlxsw_hwmon_dev; struct mlxsw_hwmon *mlxsw_hwmon = mlxsw_hwmon_dev->hwmon; - char mtmp_pl[MLXSW_REG_MTMP_LEN]; + char mtmp_pl[MLXSW_REG_MTMP_LEN] = {0}; unsigned long val; int index; int err; @@ -177,7 +177,7 @@ static ssize_t mlxsw_hwmon_fan_fault_show(struct device *dev, container_of(attr, struct mlxsw_hwmon_attr, dev_attr); struct mlxsw_hwmon_dev *mlxsw_hwmon_dev = mlxsw_hwmon_attr->mlxsw_hwmon_dev; struct mlxsw_hwmon *mlxsw_hwmon = mlxsw_hwmon_dev->hwmon; - char fore_pl[MLXSW_REG_FORE_LEN]; + char fore_pl[MLXSW_REG_FORE_LEN] = {0}; bool fault; int err; diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c index 32d2e61f2b82..91fb77c8e20d 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c @@ -1720,7 +1720,7 @@ static int mlxsw_sp2_span_init(struct mlxsw_sp *mlxsw_sp) static int mlxsw_sp2_span_policer_id_base_set(struct mlxsw_sp *mlxsw_sp, u16 policer_id_base) { - char mogcr_pl[MLXSW_REG_MOGCR_LEN]; + char mogcr_pl[MLXSW_REG_MOGCR_LEN] = {0}; int err; err = mlxsw_reg_query(mlxsw_sp->core, MLXSW_REG(mogcr), mogcr_pl); diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.c b/drivers/net/ethernet/ti/am65-cpsw-nuss.c index e9d9d69e2ba9..12860cf2fb93 100644 --- a/drivers/net/ethernet/ti/am65-cpsw-nuss.c +++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.c @@ -121,6 +121,7 @@ #define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_319 BIT(21) #define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_320 BIT(22) #define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO BIT(23) +#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_UNI_EN BIT(24) /* The PTP event messages - Sync, Delay_Req, Pdelay_Req, and Pdelay_Resp. */ #define AM65_CPSW_TS_EVENT_MSG_TYPE_BITS (BIT(0) | BIT(1) | BIT(2) | BIT(3)) @@ -1830,7 +1831,8 @@ static int am65_cpsw_nuss_hwtstamp_set(struct net_device *ndev, AM65_CPSW_PN_TS_CTL_LTYPE2_TS_132 | AM65_CPSW_PN_TS_CTL_LTYPE2_TS_319 | AM65_CPSW_PN_TS_CTL_LTYPE2_TS_320 | - AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO; + AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO | + AM65_CPSW_PN_TS_CTL_LTYPE2_TS_UNI_EN; ts_ctrl = AM65_CPSW_TS_EVENT_MSG_TYPE_BITS << AM65_CPSW_PN_TS_CTL_MSG_TYPE_EN_SHIFT; diff --git a/drivers/net/hamradio/6pack.c b/drivers/net/hamradio/6pack.c index 437e5e391a77..5168658ed2c9 100644 --- a/drivers/net/hamradio/6pack.c +++ b/drivers/net/hamradio/6pack.c @@ -256,6 +256,12 @@ static int sp_open_dev(struct net_device *dev) if (sp->tty == NULL) return -ENODEV; + + /* sp_close() stopped the queue: restart it, or the interface + * never transmits again after it is set down and up. + */ + netif_start_queue(dev); + return 0; } diff --git a/drivers/net/hamradio/bpqether.c b/drivers/net/hamradio/bpqether.c index bac1bb69d63a..f027205347d3 100644 --- a/drivers/net/hamradio/bpqether.c +++ b/drivers/net/hamradio/bpqether.c @@ -287,11 +287,10 @@ static netdev_tx_t bpq_xmit(struct sk_buff *skb, struct net_device *dev) skb->protocol = ax25_type_trans(skb, dev); skb_reset_network_header(skb); dev_hard_header(skb, dev, ETH_P_BPQ, bpq->dest_addr, NULL, 0); - dev->stats.tx_packets++; - dev->stats.tx_bytes+=skb->len; - + orig_dev->stats.tx_packets++; + orig_dev->stats.tx_bytes += skb->len; + dev_queue_xmit(skb); - netif_wake_queue(dev); return NETDEV_TX_OK; } @@ -387,6 +386,7 @@ static void *bpq_seq_start(struct seq_file *seq, loff_t *pos) list_for_each_entry_rcu(bpqdev, &bpq_devices, bpq_list) { if (i == *pos) return bpqdev; + i++; } return NULL; } diff --git a/drivers/net/hamradio/mkiss.c b/drivers/net/hamradio/mkiss.c index 5f38a002bd9e..8b0e2ca5d0ea 100644 --- a/drivers/net/hamradio/mkiss.c +++ b/drivers/net/hamradio/mkiss.c @@ -30,6 +30,7 @@ #include #define AX_MTU 236 +#define AX_MAX_MTU 65535 /* some arch define END as assembly function ending, just undef it */ #undef END @@ -355,13 +356,19 @@ static int ax_set_mac_address(struct net_device *dev, void *addr) /*---------------------------------------------------------------------------*/ -static void ax_changedmtu(struct mkiss *ax) -{ - struct net_device *dev = ax->dev; - unsigned char *xbuff, *rbuff, *oxbuff, *orbuff; - int len; +/* + * Length of the frame buffers for the current MTU. + * + * ax_encaps() accepts frames of up to dev->mtu + 73 bytes and KISS escaping + * can double every one of them. The buffers are allocated AX_BUF_SLACK bytes + * larger than this, for the two END delimiters and the two CRC bytes, which + * can need escaping as well. + */ +#define AX_BUF_SLACK 6 - len = dev->mtu * 2; +static int ax_buflen(const struct net_device *dev) +{ + int len = (dev->mtu + 73) * 2; /* * allow for arrival of larger UDP packets, even if we say not to @@ -371,8 +378,19 @@ static void ax_changedmtu(struct mkiss *ax) if (len < 576 * 2) len = 576 * 2; - xbuff = kmalloc(len + 4, GFP_ATOMIC); - rbuff = kmalloc(len + 4, GFP_ATOMIC); + return len; +} + +static void ax_changedmtu(struct mkiss *ax) +{ + struct net_device *dev = ax->dev; + unsigned char *xbuff, *rbuff, *oxbuff, *orbuff; + int len; + + len = ax_buflen(dev); + + xbuff = kmalloc(len + AX_BUF_SLACK, GFP_ATOMIC); + rbuff = kmalloc(len + AX_BUF_SLACK, GFP_ATOMIC); if (xbuff == NULL || rbuff == NULL) { printk(KERN_ERR "mkiss: %s: unable to grow ax25 buffers, " @@ -508,6 +526,14 @@ static void ax_encaps(struct net_device *dev, unsigned char *icp, int len) set_bit(TTY_DO_WRITE_WAKEUP, &ax->tty->flags); actual = ax->tty->ops->write(ax->tty, ax->xbuff, count); + if (actual < 0) { + /* The tty refused the frame: drop it. */ + clear_bit(TTY_DO_WRITE_WAKEUP, &ax->tty->flags); + dev->stats.tx_errors++; + ax->xleft = 0; + netif_start_queue(dev); + return; + } dev->stats.tx_packets++; dev->stats.tx_bytes += actual; @@ -563,6 +589,11 @@ static int ax_open_dev(struct net_device *dev) if (ax->tty == NULL) return -ENODEV; + /* ax_close() stopped the queue: restart it, or the interface + * never transmits again after it is set down and up. + */ + netif_start_queue(dev); + return 0; } @@ -581,20 +612,12 @@ static int ax_open(struct net_device *dev) * rbuff Receive buffer. * xbuff Transmit buffer. */ - len = dev->mtu * 2; + len = ax_buflen(dev); - /* - * allow for arrival of larger UDP packets, even if we say not to - * also fixes a bug in which SunOS sends 512-byte packets even with - * an MSS of 128 - */ - if (len < 576 * 2) - len = 576 * 2; - - if ((ax->rbuff = kmalloc(len + 4, GFP_KERNEL)) == NULL) + if ((ax->rbuff = kmalloc(len + AX_BUF_SLACK, GFP_KERNEL)) == NULL) goto norbuff; - if ((ax->xbuff = kmalloc(len + 4, GFP_KERNEL)) == NULL) + if ((ax->xbuff = kmalloc(len + AX_BUF_SLACK, GFP_KERNEL)) == NULL) goto noxbuff; ax->mtu = dev->mtu + 73; @@ -640,6 +663,7 @@ static void ax_setup(struct net_device *dev) { /* Finish setting up the DEVICE info. */ dev->mtu = AX_MTU; + dev->max_mtu = AX_MAX_MTU; dev->hard_header_len = AX25_MAX_HEADER_LEN; dev->addr_len = AX25_ADDR_LEN; dev->type = ARPHRD_AX25; @@ -928,6 +952,14 @@ static void mkiss_write_wakeup(struct tty_struct *tty) } actual = tty->ops->write(tty, ax->xhead, ax->xleft); + if (actual < 0) { + /* Give up on the rest of this frame. */ + clear_bit(TTY_DO_WRITE_WAKEUP, &tty->flags); + ax->dev->stats.tx_errors++; + ax->xleft = 0; + netif_wake_queue(ax->dev); + goto out; + } ax->xleft -= actual; ax->xhead += actual; diff --git a/drivers/net/ppp/pppoe.c b/drivers/net/ppp/pppoe.c index 8ff2024dde60..293b6fb1c1e9 100644 --- a/drivers/net/ppp/pppoe.c +++ b/drivers/net/ppp/pppoe.c @@ -194,18 +194,16 @@ static int __set_item(struct pppoe_net *pn, struct pppox_sock *po) return 0; } -static void __delete_item(struct pppoe_net *pn, __be16 sid, - char *addr, int ifindex) +static void __delete_item(struct pppoe_net *pn, struct pppox_sock *po) { - int hash = hash_item(sid, addr); + int hash = hash_item(po->pppoe_pa.sid, po->pppoe_pa.remote); struct pppox_sock *ret, **src; ret = pn->hash_table[hash]; src = &pn->hash_table[hash]; while (ret) { - if (cmp_addr(&ret->pppoe_pa, sid, addr) && - ret->pppoe_ifindex == ifindex) { + if (ret == po) { *src = ret->next; break; } @@ -255,11 +253,10 @@ static inline struct pppox_sock *get_item_by_addr(struct net *net, return pppox_sock; } -static inline void delete_item(struct pppoe_net *pn, __be16 sid, - char *addr, int ifindex) +static void delete_item(struct pppoe_net *pn, struct pppox_sock *po) { write_lock_bh(&pn->hash_lock); - __delete_item(pn, sid, addr, ifindex); + __delete_item(pn, po); write_unlock_bh(&pn->hash_lock); } @@ -599,8 +596,7 @@ static int pppoe_release(struct socket *sock) * protect "po" from concurrent updates * on pppoe_flush_dev */ - delete_item(pn, po->pppoe_pa.sid, po->pppoe_pa.remote, - po->pppoe_ifindex); + delete_item(pn, po); sock_orphan(sk); sock->sk = NULL; @@ -651,8 +647,7 @@ static int pppoe_connect(struct socket *sock, struct sockaddr *uservaddr, if (stage_session(po->pppoe_pa.sid)) { pppox_unbind_sock(sk); pn = pppoe_pernet(sock_net(sk)); - delete_item(pn, po->pppoe_pa.sid, - po->pppoe_pa.remote, po->pppoe_ifindex); + delete_item(pn, po); if (po->pppoe_dev) { dev_put(po->pppoe_dev); po->pppoe_dev = NULL; @@ -703,8 +698,7 @@ static int pppoe_connect(struct socket *sock, struct sockaddr *uservaddr, error = ppp_register_net_channel(dev_net(dev), &po->chan); if (error) { - delete_item(pn, po->pppoe_pa.sid, - po->pppoe_pa.remote, po->pppoe_ifindex); + delete_item(pn, po); goto err_put; } diff --git a/drivers/net/veth.c b/drivers/net/veth.c index 564bf57bc897..22387f66bf90 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -1371,9 +1371,10 @@ static int veth_set_channels(struct net_device *dev, * to identify the range we have to disable */ veth_disable_range_safe(dev, new_rx_count, old_rx_count); - netif_carrier_on(dev); - if (peer) + if (peer && (peer->flags & IFF_UP)) { + netif_carrier_on(dev); netif_carrier_on(peer); + } } /* update XDP supported features */ diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c index e70a0b657570..caba78398d92 100644 --- a/drivers/net/vxlan/vxlan_core.c +++ b/drivers/net/vxlan/vxlan_core.c @@ -1006,12 +1006,6 @@ static int vxlan_fdb_update_existing(struct vxlan_dev *vxlan, return -EOPNOTSUPP; } - if (rcu_access_pointer(f->nh) && - !(state & (NUD_PERMANENT | NUD_NOARP))) { - NL_SET_ERR_MSG(extack, "Cannot make a nexthop fdb dynamic"); - return -EOPNOTSUPP; - } - /* Do not allow an externally learned entry to take over an entry added * by the user. */ @@ -1279,11 +1273,6 @@ static int vxlan_fdb_add(struct ndmsg *ndm, struct nlattr *tb[], if (err) return err; - if (nhid && !(ndm->ndm_state & (NUD_PERMANENT | NUD_NOARP))) { - NL_SET_ERR_MSG(extack, "A nexthop fdb cannot be dynamic"); - return -EINVAL; - } - if (vxlan->default_dst.remote_ip.sa.sa_family != ip.sa.sa_family) return -EAFNOSUPPORT; @@ -2896,6 +2885,9 @@ static void vxlan_cleanup(struct timer_list *t) if (f->flags & NTF_EXT_LEARNED) continue; + if (rcu_access_pointer(f->nh)) + continue; + timeout = READ_ONCE(f->used) + vxlan->cfg.age_interval * HZ; if (time_before_eq(timeout, jiffies)) { netdev_dbg(vxlan->dev, diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c index c3a3a0817910..a2a0cc43bc72 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c @@ -6424,6 +6424,7 @@ brcmf_bss_connect_done(struct brcmf_cfg80211_info *cfg, struct brcmf_cfg80211_profile *profile = &ifp->vif->profile; struct brcmf_cfg80211_connect_info *conn_info = cfg_to_conn(cfg); struct cfg80211_connect_resp_params conn_params; + bool authorized; brcmf_dbg(TRACE, "Enter\n"); @@ -6448,7 +6449,13 @@ brcmf_bss_connect_done(struct brcmf_cfg80211_info *cfg, conn_params.req_ie_len = conn_info->req_ie_len; conn_params.resp_ie = conn_info->resp_ie; conn_params.resp_ie_len = conn_info->resp_ie_len; + authorized = completed && + (profile->use_fwsup == BRCMF_PROFILE_FWSUP_SAE || + profile->use_fwsup == BRCMF_PROFILE_FWSUP_PSK); cfg80211_connect_done(ndev, &conn_params, GFP_KERNEL); + if (authorized) + cfg80211_port_authorized(ndev, profile->bssid, NULL, 0, + GFP_KERNEL); brcmf_dbg(CONN, "Report connect result - connection %s\n", completed ? "succeeded" : "failed"); } diff --git a/drivers/net/wireless/mediatek/mt76/mac80211.c b/drivers/net/wireless/mediatek/mt76/mac80211.c index cd02dff7ab34..13e659494518 100644 --- a/drivers/net/wireless/mediatek/mt76/mac80211.c +++ b/drivers/net/wireless/mediatek/mt76/mac80211.c @@ -973,6 +973,20 @@ void mt76_update_survey(struct mt76_phy *phy) } EXPORT_SYMBOL_GPL(mt76_update_survey); +void mt76_phy_chandef_set(struct mt76_phy *phy, + struct cfg80211_chan_def *chandef) +{ + if (!chandef->chan) + return; + + mt76_update_survey(phy); + + phy->chandef = *chandef; + phy->main_chan = chandef->chan; + phy->chan_state = mt76_channel_state(phy, chandef->chan); +} +EXPORT_SYMBOL_GPL(mt76_phy_chandef_set); + int mt76_set_channel(struct mt76_phy *phy, struct cfg80211_chan_def *chandef, bool offchannel) { diff --git a/drivers/net/wireless/mediatek/mt76/mt76.h b/drivers/net/wireless/mediatek/mt76/mt76.h index 8addbffa97c5..d709b7415968 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76.h +++ b/drivers/net/wireless/mediatek/mt76/mt76.h @@ -1400,6 +1400,8 @@ void mt76_release_buffered_frames(struct ieee80211_hw *hw, bool mt76_has_tx_pending(struct mt76_phy *phy); int mt76_update_channel(struct mt76_phy *phy); void mt76_update_survey(struct mt76_phy *phy); +void mt76_phy_chandef_set(struct mt76_phy *phy, + struct cfg80211_chan_def *chandef); void mt76_update_survey_active_time(struct mt76_phy *phy, ktime_t time); int mt76_get_survey(struct ieee80211_hw *hw, int idx, struct survey_info *survey); diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/main.c b/drivers/net/wireless/mediatek/mt76/mt7921/main.c index 99561094640f..18eb934da570 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7921/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7921/main.c @@ -1394,6 +1394,7 @@ mt7921_change_chanctx(struct ieee80211_hw *hw, mt7921_mcu_config_sniffer(mvif, ctx); else mt76_connac_mcu_uni_set_chctx(mvif->phy->mt76, &mvif->bss_conf.mt76, ctx); + mt76_phy_chandef_set(mvif->phy->mt76, &ctx->def); mt792x_mutex_release(phy->dev); } diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/main.c b/drivers/net/wireless/mediatek/mt76/mt7925/main.c index 9a52607dc5a8..dba0f4892207 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7925/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7925/main.c @@ -1873,6 +1873,7 @@ mt7925_change_chanctx(struct ieee80211_hw *hw, link_conf, ctx); } } + mt76_phy_chandef_set(mvif->phy->mt76, &ctx->def); mt792x_mutex_release(phy->dev); } @@ -2170,6 +2171,7 @@ static int mt7925_assign_vif_chanctx(struct ieee80211_hw *hw, mconf->mt76.ctx = ctx; mctx->bss_conf = mconf; + mt76_phy_chandef_set(mvif->phy->mt76, &ctx->def); mutex_unlock(&dev->mt76.mutex); return 0; diff --git a/drivers/net/wireless/mediatek/mt76/mt792x_core.c b/drivers/net/wireless/mediatek/mt76/mt792x_core.c index c6afdf90fade..86b45807481f 100644 --- a/drivers/net/wireless/mediatek/mt76/mt792x_core.c +++ b/drivers/net/wireless/mediatek/mt76/mt792x_core.c @@ -308,6 +308,7 @@ int mt792x_assign_vif_chanctx(struct ieee80211_hw *hw, mutex_lock(&dev->mt76.mutex); mvif->bss_conf.mt76.ctx = ctx; mctx->bss_conf = &mvif->bss_conf; + mt76_phy_chandef_set(&dev->mphy, &ctx->def); mutex_unlock(&dev->mt76.mutex); return 0; @@ -658,7 +659,7 @@ int mt792x_init_wiphy(struct ieee80211_hw *hw) } EXPORT_SYMBOL_GPL(mt792x_init_wiphy); -static u8 +static int mt792x_get_offload_capability(struct device *dev, const char *fw_wm) { const struct mt76_connac2_fw_trailer *hdr; @@ -720,13 +721,20 @@ mt792x_get_mac80211_ops(struct device *dev, void *drv_data, u8 *fw_features) { struct ieee80211_ops *ops; + int ret; ops = devm_kmemdup(dev, mac80211_ops, sizeof(struct ieee80211_ops), GFP_KERNEL); if (!ops) return NULL; - *fw_features = mt792x_get_offload_capability(dev, drv_data); + ret = mt792x_get_offload_capability(dev, drv_data); + if (ret < 0) { + dev_warn(dev, "failed to read firmware offload caps: %d\n", ret); + *fw_features = 0; + } else { + *fw_features = ret; + } if (!(*fw_features & MT792x_FW_CAP_CNM)) { ops->remain_on_channel = NULL; ops->cancel_remain_on_channel = NULL; diff --git a/drivers/net/wireless/realtek/rtw88/mac80211.c b/drivers/net/wireless/realtek/rtw88/mac80211.c index 910431591420..bf21937f25b2 100644 --- a/drivers/net/wireless/realtek/rtw88/mac80211.c +++ b/drivers/net/wireless/realtek/rtw88/mac80211.c @@ -466,6 +466,7 @@ static int rtw_ops_start_ap(struct ieee80211_hw *hw, mutex_lock(&rtwdev->mutex); rtw_write32_set(rtwdev, REG_TCR, BIT_TCR_UPDATE_HGQMD); + rtw_write16(rtwdev, REG_ATIMWND, ATIMWND_AP); rtwdev->ap_active = true; rtw_store_op_chan(rtwdev, true); chip->ops->phy_calibration(rtwdev); @@ -482,6 +483,7 @@ static void rtw_ops_stop_ap(struct ieee80211_hw *hw, mutex_lock(&rtwdev->mutex); rtw_write32_clr(rtwdev, REG_TCR, BIT_TCR_UPDATE_HGQMD); + rtw_write16(rtwdev, REG_ATIMWND, ATIMWND_DEFAULT); rtwdev->ap_active = false; if (!rtw_core_check_sta_active(rtwdev)) rtw_clear_op_chan(rtwdev); diff --git a/drivers/net/wireless/realtek/rtw88/reg.h b/drivers/net/wireless/realtek/rtw88/reg.h index 2708ee2f12a4..6ff593f62066 100644 --- a/drivers/net/wireless/realtek/rtw88/reg.h +++ b/drivers/net/wireless/realtek/rtw88/reg.h @@ -431,6 +431,8 @@ #define REG_DRVERLYINT 0x0558 #define REG_BCNDMATIM 0x0559 #define REG_ATIMWND 0x055A +#define ATIMWND_DEFAULT 0x2 +#define ATIMWND_AP 0xa #define REG_USTIME_TSF 0x055C #define REG_BCN_MAX_ERR 0x055D #define REG_RXTSF_OFFSET_CCK 0x055E diff --git a/drivers/net/wireless/realtek/rtw88/tx.c b/drivers/net/wireless/realtek/rtw88/tx.c index 0c8817bf46c7..959ae66776aa 100644 --- a/drivers/net/wireless/realtek/rtw88/tx.c +++ b/drivers/net/wireless/realtek/rtw88/tx.c @@ -191,7 +191,7 @@ void rtw_tx_report_purge_timer(struct timer_list *t) void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn) { struct rtw_tx_report *tx_report = &rtwdev->tx_report; - unsigned long timeout = RTW_TX_PROBE_TIMEOUT; + unsigned long timeout; unsigned long flags; u8 *drv_data; @@ -203,9 +203,10 @@ void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn) __skb_queue_tail(&tx_report->queue, skb); spin_unlock_irqrestore(&tx_report->q_lock, flags); - if (rtwdev->chip->id == RTW_CHIP_TYPE_8723D && - rtwdev->hci.type == RTW_HCI_TYPE_USB) - timeout = msecs_to_jiffies(2500); + if (rtwdev->hci.type == RTW_HCI_TYPE_PCIE) + timeout = RTW_TX_PROBE_TIMEOUT_SHORT; + else + timeout = RTW_TX_PROBE_TIMEOUT_LONG; mod_timer(&tx_report->purge_timer, jiffies + timeout); } diff --git a/drivers/net/wireless/realtek/rtw88/tx.h b/drivers/net/wireless/realtek/rtw88/tx.h index 3d544fd7f60f..f361713ece9f 100644 --- a/drivers/net/wireless/realtek/rtw88/tx.h +++ b/drivers/net/wireless/realtek/rtw88/tx.h @@ -7,7 +7,8 @@ #define RTK_TX_MAX_AGG_NUM_MASK 0x1f -#define RTW_TX_PROBE_TIMEOUT msecs_to_jiffies(500) +#define RTW_TX_PROBE_TIMEOUT_SHORT msecs_to_jiffies(500) +#define RTW_TX_PROBE_TIMEOUT_LONG msecs_to_jiffies(2500) struct rtw_tx_desc { __le32 w0; diff --git a/drivers/net/wireless/realtek/rtw88/usb.c b/drivers/net/wireless/realtek/rtw88/usb.c index b3e1e9a0fda6..e1df0ad1852b 100644 --- a/drivers/net/wireless/realtek/rtw88/usb.c +++ b/drivers/net/wireless/realtek/rtw88/usb.c @@ -2,9 +2,11 @@ /* Copyright(c) 2018-2019 Realtek Corporation */ +#include #include -#include #include +#include +#include #include "main.h" #include "debug.h" #include "reg.h" @@ -555,9 +557,61 @@ static int rtw_usb_write_data_h2c(struct rtw_dev *rtwdev, u8 *buf, u32 size) return rtw_usb_write_data(rtwdev, &pkt_info, buf); } -static u8 rtw_usb_tx_queue_mapping_to_qsel(struct sk_buff *skb) +static bool rtw_usb_bmc_needs_dtim(struct sk_buff *skb) +{ + struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb); + struct udphdr *udphdr; + + if (info->control.flags & IEEE80211_TX_CTRL_PORT_CTRL_PROTO) + return true; + + if (skb->protocol == htons(ETH_P_ARP)) + return true; + + if (skb->protocol == htons(ETH_P_IP) && + ip_hdr(skb)->protocol == IPPROTO_UDP) { + udphdr = udp_hdr(skb); + + return udphdr->dest == htons(67) || udphdr->dest == htons(68); + } + + return false; +} + +#define RTW_USB_HIQ_REFILL_INTERVAL (HZ / 10) /* one unit of budget per 100 ms */ +#define RTW_USB_HIQ_BUDGET_MAX 16 + +static bool rtw_usb_hiq_take_budget(struct rtw_usb *rtwusb) +{ + unsigned long flags, elapsed, add; + bool ok; + + spin_lock_irqsave(&rtwusb->hiq_lock, flags); + + elapsed = jiffies - rtwusb->hiq_refill; + add = elapsed / RTW_USB_HIQ_REFILL_INTERVAL; + if (add) { + rtwusb->hiq_budget = min_t(unsigned long, rtwusb->hiq_budget + add, + RTW_USB_HIQ_BUDGET_MAX); + + /* keep the unfinished part of the interval for the next unit */ + rtwusb->hiq_refill = jiffies - elapsed % RTW_USB_HIQ_REFILL_INTERVAL; + } + + ok = rtwusb->hiq_budget > 0; + if (ok) + rtwusb->hiq_budget--; + + spin_unlock_irqrestore(&rtwusb->hiq_lock, flags); + + return ok; +} + +static u8 rtw_usb_tx_queue_mapping_to_qsel(struct rtw_usb *rtwusb, + struct sk_buff *skb) { struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data; + struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb); __le16 fc = hdr->frame_control; u8 qsel; @@ -565,7 +619,9 @@ static u8 rtw_usb_tx_queue_mapping_to_qsel(struct sk_buff *skb) qsel = TX_DESC_QSEL_MGMT; else if (is_broadcast_ether_addr(hdr->addr1) || is_multicast_ether_addr(hdr->addr1)) - qsel = TX_DESC_QSEL_HIGH; + qsel = (info->flags & IEEE80211_TX_CTL_SEND_AFTER_DTIM) && + rtw_usb_bmc_needs_dtim(skb) && rtw_usb_hiq_take_budget(rtwusb) ? + TX_DESC_QSEL_HIGH : skb->priority; else if (skb_get_queue_mapping(skb) <= IEEE80211_AC_BK) qsel = skb->priority; else @@ -584,7 +640,7 @@ static int rtw_usb_tx_write(struct rtw_dev *rtwdev, u8 *pkt_desc; int ep; - pkt_info->qsel = rtw_usb_tx_queue_mapping_to_qsel(skb); + pkt_info->qsel = rtw_usb_tx_queue_mapping_to_qsel(rtwusb, skb); pkt_desc = skb_push(skb, chip->tx_pkt_desc_sz); memset(pkt_desc, 0, chip->tx_pkt_desc_sz); ep = qsel_to_ep(rtwusb, pkt_info->qsel); @@ -935,6 +991,10 @@ static int rtw_usb_init_tx(struct rtw_dev *rtwdev) struct rtw_usb *rtwusb = rtw_get_usb_priv(rtwdev); int i; + spin_lock_init(&rtwusb->hiq_lock); + rtwusb->hiq_budget = RTW_USB_HIQ_BUDGET_MAX; + rtwusb->hiq_refill = jiffies; + rtwusb->txwq = create_singlethread_workqueue("rtw88_usb: tx wq"); if (!rtwusb->txwq) { rtw_err(rtwdev, "failed to create TX work queue\n"); diff --git a/drivers/net/wireless/realtek/rtw88/usb.h b/drivers/net/wireless/realtek/rtw88/usb.h index 86697a5c0103..36a57a0e0df6 100644 --- a/drivers/net/wireless/realtek/rtw88/usb.h +++ b/drivers/net/wireless/realtek/rtw88/usb.h @@ -74,6 +74,11 @@ struct rtw_usb { u8 out_ep[RTW_USB_EP_MAX]; int qsel_to_ep[TX_DESC_QSEL_MAX]; + /* protects hiq_budget and hiq_refill */ + spinlock_t hiq_lock; + u32 hiq_budget; + unsigned long hiq_refill; + struct workqueue_struct *txwq, *rxwq; struct sk_buff_head tx_queue[RTW_USB_EP_MAX]; diff --git a/drivers/net/wireless/rsi/rsi_91x_core.c b/drivers/net/wireless/rsi/rsi_91x_core.c index ead4d4e04328..3281fc94e107 100644 --- a/drivers/net/wireless/rsi/rsi_91x_core.c +++ b/drivers/net/wireless/rsi/rsi_91x_core.c @@ -458,7 +458,7 @@ void rsi_core_xmit(struct rsi_common *common, struct sk_buff *skb) tx_params->sta_id = 0; } - if (rsta) { + if (rsta && tid < IEEE80211_NUM_TIDS) { /* Start aggregation if not done for this tid */ if (!rsta->start_tx_aggr[tid]) { rsta->start_tx_aggr[tid] = true; diff --git a/drivers/nvme/common/keyring.c b/drivers/nvme/common/keyring.c index ed5167f942d8..9f8c72c7bba7 100644 --- a/drivers/nvme/common/keyring.c +++ b/drivers/nvme/common/keyring.c @@ -36,6 +36,7 @@ struct key *nvme_tls_key_lookup(key_serial_t key_id) } if (nvme_tls_psk_revoked(key)) { pr_err("key id %08x revoked\n", key_id); + key_put(key); return ERR_PTR(-EKEYREVOKED); } return key; diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index cf79a6c45ccd..37dc942af0d2 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -4375,9 +4375,8 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl) nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING_NOIO); mutex_lock(&ctrl->namespaces_lock); - list_splice_init_rcu(&ctrl->namespaces, &ns_list, synchronize_rcu); + list_splice_init_srcu(&ctrl->namespaces, &ns_list, &ctrl->srcu); mutex_unlock(&ctrl->namespaces_lock); - synchronize_srcu(&ctrl->srcu); list_for_each_entry_safe(ns, next, &ns_list, list) nvme_ns_remove(ns); diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index df08ebe13ea0..8cc6023e478e 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -3683,6 +3683,8 @@ static const struct pci_device_id nvme_id_table[] = { .driver_data = NVME_QUIRK_BOGUS_NID, }, { PCI_DEVICE(0x1344, 0x6004), /* Micron 4100AT */ .driver_data = NVME_QUIRK_DMAPOOL_ALIGN_512, }, + { PCI_DEVICE(0x1e0f, 0x0032), /* KIOXIA XG10d */ + .driver_data = NVME_QUIRK_DMAPOOL_ALIGN_512, }, { PCI_DEVICE(0x1c5c, 0x1504), /* SK Hynix PC400 */ .driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, }, { PCI_DEVICE(0x1c5c, 0x174a), /* SK Hynix P31 SSD */ diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c index 67d02cfb47ec..951506216714 100644 --- a/drivers/nvme/host/tcp.c +++ b/drivers/nvme/host/tcp.c @@ -6,6 +6,7 @@ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt #include #include +#include #include #include #include @@ -47,13 +48,34 @@ MODULE_PARM_DESC(wq_unbound, "Use unbound workqueue for nvme-tcp IO context (def /* * TLS handshake timeout */ -static int tls_handshake_timeout = 10; +static unsigned int tls_handshake_timeout = 10; #ifdef CONFIG_NVME_TCP_TLS -module_param(tls_handshake_timeout, int, 0644); +static int tls_handshake_timeout_set(const char *val, + const struct kernel_param *kp) +{ + return param_set_uint_minmax(val, kp, 0, + min_t(unsigned long, UINT_MAX / MSEC_PER_SEC, + MAX_JIFFY_OFFSET / HZ)); +} + +static const struct kernel_param_ops tls_handshake_timeout_ops = { + .set = tls_handshake_timeout_set, + .get = param_get_uint, +}; + +module_param_cb(tls_handshake_timeout, &tls_handshake_timeout_ops, + &tls_handshake_timeout, 0644); +__MODULE_PARM_TYPE(tls_handshake_timeout, "uint"); MODULE_PARM_DESC(tls_handshake_timeout, "nvme TLS handshake timeout in seconds (default 10)"); #endif +/* read a TLS control record in chunks of this size */ +#define NVME_TCP_TLS_REC_CHUNK 256 + +/* TLS records read while waiting for the ICResp, the ICResp included */ +#define NVME_TCP_ICRESP_MAX_RECORDS 8 + static atomic_t nvme_tcp_cpu_queues[NR_CPUS]; #ifdef CONFIG_DEBUG_LOCK_ALLOC @@ -1446,9 +1468,11 @@ static void nvme_tcp_io_work(struct work_struct *w) do { bool pending = false; + bool got_send_mutex = false; int result; if (mutex_trylock(&queue->send_mutex)) { + got_send_mutex = true; result = nvme_tcp_try_send(queue); mutex_unlock(&queue->send_mutex); if (result > 0) @@ -1463,8 +1487,13 @@ static void nvme_tcp_io_work(struct work_struct *w) else if (unlikely(result < 0)) return; - /* did we get some space after spending time in recv? */ - if (nvme_tcp_queue_has_pending(queue) && + /* + * did we get some space after spending time in recv? + * Only check this if we could take send_mutex. Otherwise + * the task holding it sends the requests and queues io_work + * again after it releases the mutex. + */ + if (got_send_mutex && nvme_tcp_queue_has_pending(queue) && sk_stream_is_writeable(queue->sock->sk)) pending = true; @@ -1562,12 +1591,143 @@ static void nvme_tcp_free_queue(struct nvme_ctrl *nctrl, int qid) mutex_destroy(&queue->pf_cache_lock); } +static int nvme_tcp_recv_tls_alert(struct nvme_tcp_queue *queue, + struct msghdr *msg, int len) +{ + struct sock *sk = queue->sock->sk; + u8 level, description; + + if (len < 2) { + dev_err(queue->ctrl->ctrl.device, + "queue %d: truncated TLS alert\n", + nvme_tcp_queue_id(queue)); + return -ENOTCONN; + } + + /* tls_alert_recv() reads through the iterator */ + iov_iter_revert(&msg->msg_iter, len); + tls_alert_recv(sk, msg, &level, &description); + + if (level == TLS_ALERT_LEVEL_FATAL) { + dev_err(queue->ctrl->ctrl.device, + "queue %d: fatal TLS alert %u\n", + nvme_tcp_queue_id(queue), description); + return -ENOTCONN; + } + + /* a peer shutting down normally is not worth a warning */ + if (description == TLS_ALERT_DESC_CLOSE_NOTIFY) + dev_dbg(queue->ctrl->ctrl.device, + "queue %d: TLS close_notify\n", + nvme_tcp_queue_id(queue)); + else + dev_warn(queue->ctrl->ctrl.device, + "queue %d: TLS alert %u\n", + nvme_tcp_queue_id(queue), description); + + return 0; +} + +/* + * Discard a TLS record that arrived in place of the ICResp. A peer may send + * a NewSessionTicket as soon as the handshake completes. + * + * Returns 0 when the record was discarded and the ICResp can be read, or a + * negative errno if the connection cannot continue. + */ +static int nvme_tcp_skip_tls_record(struct nvme_tcp_queue *queue, + struct msghdr *msg, u8 ctype, int len) +{ + u8 buf[NVME_TCP_TLS_REC_CHUNK]; + int ret; + + switch (ctype) { + case TLS_RECORD_TYPE_HANDSHAKE: + break; + case TLS_RECORD_TYPE_ALERT: + ret = nvme_tcp_recv_tls_alert(queue, msg, len); + return ret ? : -ENOTCONN; + default: + pr_err("queue %d: unhandled TLS record %u\n", + nvme_tcp_queue_id(queue), ctype); + return -ENOTCONN; + } + + /* the ICResp is in a later record, drain this one first */ + while (!(msg->msg_flags & MSG_EOR)) { + struct kvec iov = { + .iov_base = buf, + .iov_len = sizeof(buf), + }; + char cbuf[CMSG_LEN(sizeof(char))] = {}; + /* kTLS fails a control record read without a cmsg buffer */ + struct msghdr drain = { + .msg_control = cbuf, + .msg_controllen = sizeof(cbuf), + }; + + ret = kernel_recvmsg(queue->sock, &drain, &iov, 1, sizeof(buf), + 0); + if (ret <= 0) { + pr_warn("queue %d: failed to drain TLS record: %d\n", + nvme_tcp_queue_id(queue), ret); + return ret ? : -ECONNRESET; + } + msg->msg_flags = drain.msg_flags; + } + + return 0; +} + +/* + * Receive the ICResp. Over TLS, skip the records that a peer may send + * ahead of it. + * + * Returns the length received, or a negative errno. + */ +static int nvme_tcp_recv_icresp(struct nvme_tcp_queue *queue, + struct nvme_tcp_icresp_pdu *icresp) +{ + char cbuf[CMSG_LEN(sizeof(char))]; + struct msghdr msg; + struct kvec iov; + u8 ctype; + int i, ret; + + for (i = 0; i < NVME_TCP_ICRESP_MAX_RECORDS; i++) { + memset(&msg, 0, sizeof(msg)); + memset(cbuf, 0, sizeof(cbuf)); + iov.iov_base = icresp; + iov.iov_len = sizeof(*icresp); + if (nvme_tcp_queue_tls(queue)) { + msg.msg_control = cbuf; + msg.msg_controllen = sizeof(cbuf); + } + msg.msg_flags = MSG_WAITALL; + ret = kernel_recvmsg(queue->sock, &msg, &iov, 1, + iov.iov_len, msg.msg_flags); + if (ret <= 0 || !nvme_tcp_queue_tls(queue)) + return ret; + + ctype = tls_get_record_type(queue->sock->sk, + (struct cmsghdr *)cbuf); + if (ctype == TLS_RECORD_TYPE_DATA) + return ret; + + ret = nvme_tcp_skip_tls_record(queue, &msg, ctype, ret); + if (ret < 0) + return ret; + } + + pr_err("queue %d: too many TLS records before the ICResp\n", + nvme_tcp_queue_id(queue)); + return -EPROTO; +} + static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue) { struct nvme_tcp_icreq_pdu *icreq; struct nvme_tcp_icresp_pdu *icresp; - char cbuf[CMSG_LEN(sizeof(char))] = {}; - u8 ctype; struct msghdr msg = {}; struct kvec iov; bool ctrl_hdgst, ctrl_ddgst; @@ -1605,16 +1765,7 @@ static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue) goto free_icresp; } - memset(&msg, 0, sizeof(msg)); - iov.iov_base = icresp; - iov.iov_len = sizeof(*icresp); - if (nvme_tcp_queue_tls(queue)) { - msg.msg_control = cbuf; - msg.msg_controllen = sizeof(cbuf); - } - msg.msg_flags = MSG_WAITALL; - ret = kernel_recvmsg(queue->sock, &msg, &iov, 1, - iov.iov_len, msg.msg_flags); + ret = nvme_tcp_recv_icresp(queue, icresp); if (ret >= 0 && ret < sizeof(*icresp)) ret = -ECONNRESET; if (ret < 0) { @@ -1622,16 +1773,6 @@ static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue) nvme_tcp_queue_id(queue), ret); goto free_icresp; } - ret = -ENOTCONN; - if (nvme_tcp_queue_tls(queue)) { - ctype = tls_get_record_type(queue->sock->sk, - (struct cmsghdr *)cbuf); - if (ctype != TLS_RECORD_TYPE_DATA) { - pr_err("queue %d: unhandled TLS record %d\n", - nvme_tcp_queue_id(queue), ctype); - goto free_icresp; - } - } ret = -EINVAL; if (icresp->hdr.type != nvme_tcp_icresp) { pr_err("queue %d: bad type returned %d\n", @@ -1821,8 +1962,10 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl, { int qid = nvme_tcp_queue_id(queue); int ret; + long wait_ret; struct tls_handshake_args args; - unsigned long tmo = tls_handshake_timeout * HZ; + unsigned int timeout = READ_ONCE(tls_handshake_timeout); + unsigned long tmo = (unsigned long)timeout * HZ; key_serial_t keyring = nvme_keyring_id(); dev_dbg(nctrl->device, "queue %d: start TLS with key %x\n", @@ -1836,7 +1979,7 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl, if (nctrl->opts->keyring) keyring = key_serial(nctrl->opts->keyring); args.ta_keyring = keyring; - args.ta_timeout_ms = tls_handshake_timeout * 1000; + args.ta_timeout_ms = timeout * MSEC_PER_SEC; queue->tls_err = -EOPNOTSUPP; init_completion(&queue->tls_complete); ret = tls_client_hello_psk(&args, GFP_KERNEL); @@ -1845,10 +1988,10 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl, qid, ret); return ret; } - ret = wait_for_completion_interruptible_timeout(&queue->tls_complete, tmo); - if (ret <= 0) { - if (ret == 0) - ret = -ETIMEDOUT; + wait_ret = wait_for_completion_interruptible_timeout(&queue->tls_complete, + tmo); + if (wait_ret <= 0) { + ret = wait_ret < 0 ? -ERESTARTSYS : -ETIMEDOUT; dev_err(nctrl->device, "queue %d: TLS handshake failed, error %d\n", diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c index c8ff53faddbc..1cc2f57b53ff 100644 --- a/drivers/nvme/target/admin-cmd.c +++ b/drivers/nvme/target/admin-cmd.c @@ -282,6 +282,15 @@ static void nvmet_execute_get_log_page_ana(struct nvmet_req *req) u16 ngrps = 0; u16 status; + /* + * The header and one descriptor per enabled group are the only bytes + * written below; zero the whole transfer first so the unwritten tail + * does not leak stale kernel memory to the host. + */ + status = nvmet_zero_sgl(req, 0, req->transfer_len); + if (status) + goto out; + status = NVME_SC_INTERNAL; desc = kmalloc(struct_size(desc, nsids, NVMET_MAX_NAMESPACES), GFP_KERNEL); diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c index 3b066601fd35..0e12f800beb3 100644 --- a/drivers/nvme/target/configfs.c +++ b/drivers/nvme/target/configfs.c @@ -148,7 +148,7 @@ static ssize_t nvmet_addr_traddr_store(struct config_item *item, if (nvmet_is_port_enabled(port, __func__)) return -EACCES; - if (sscanf(page, "%s\n", port->disc_addr.traddr) != 1) + if (sscanf(page, "%255s", port->disc_addr.traddr) != 1) return -EINVAL; return count; } @@ -240,7 +240,7 @@ static ssize_t nvmet_addr_trsvcid_store(struct config_item *item, if (nvmet_is_port_enabled(port, __func__)) return -EACCES; - if (sscanf(page, "%s\n", port->disc_addr.trsvcid) != 1) + if (sscanf(page, "%31s", port->disc_addr.trsvcid) != 1) return -EINVAL; return count; } diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c index 01b7722afb5e..290b51fef0b9 100644 --- a/drivers/nvme/target/rdma.c +++ b/drivers/nvme/target/rdma.c @@ -864,7 +864,8 @@ static u16 nvmet_rdma_map_sgl_inline(struct nvmet_rdma_rsp *rsp) return NVME_SC_INVALID_FIELD | NVME_STATUS_DNR; } - if (off + len > rsp->queue->dev->inline_data_size) { + if (off > (u64)rsp->queue->dev->inline_data_size || + len > (u64)rsp->queue->dev->inline_data_size - off) { pr_err("invalid inline data offset!\n"); return NVME_SC_SGL_INVALID_OFFSET | NVME_STATUS_DNR; } diff --git a/drivers/nvmem/rockchip-otp.c b/drivers/nvmem/rockchip-otp.c index d88f12c53242..1475ce11a9c7 100644 --- a/drivers/nvmem/rockchip-otp.c +++ b/drivers/nvmem/rockchip-otp.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -77,9 +78,11 @@ struct rockchip_data { struct rockchip_otp { struct device *dev; void __iomem *base; - struct clk_bulk_data *clks; struct reset_control *rst; const struct rockchip_data *data; + /* Serializes access to the OTP controller state machine */ + struct mutex mutex; + struct clk_bulk_data clks[]; }; static int rockchip_otp_reset(struct rockchip_otp *otp) @@ -239,16 +242,21 @@ static int rockchip_otp_read(void *context, unsigned int offset, if (!otp->data || !otp->data->reg_read) return -EINVAL; + mutex_lock(&otp->mutex); + ret = clk_bulk_prepare_enable(otp->data->num_clks, otp->clks); if (ret < 0) { dev_err(otp->dev, "failed to prepare/enable clks\n"); - return ret; + goto unlock; } ret = otp->data->reg_read(context, offset, val, bytes); clk_bulk_disable_unprepare(otp->data->num_clks, otp->clks); +unlock: + mutex_unlock(&otp->mutex); + return ret; } @@ -327,23 +335,23 @@ static int rockchip_otp_probe(struct platform_device *pdev) if (!data) return dev_err_probe(dev, -EINVAL, "failed to get match data\n"); - otp = devm_kzalloc(&pdev->dev, sizeof(struct rockchip_otp), + otp = devm_kzalloc(&pdev->dev, struct_size(otp, clks, data->num_clks), GFP_KERNEL); if (!otp) return -ENOMEM; otp->data = data; otp->dev = dev; + + ret = devm_mutex_init(dev, &otp->mutex); + if (ret) + return ret; + otp->base = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(otp->base)) return dev_err_probe(dev, PTR_ERR(otp->base), "failed to ioremap resource\n"); - otp->clks = devm_kcalloc(dev, data->num_clks, sizeof(*otp->clks), - GFP_KERNEL); - if (!otp->clks) - return -ENOMEM; - for (i = 0; i < data->num_clks; ++i) otp->clks[i].id = data->clks[i]; diff --git a/drivers/pci/controller/cadence/pcie-cadence-plat.c b/drivers/pci/controller/cadence/pcie-cadence-plat.c index 0456845dabb9..c1ec619ab0a4 100644 --- a/drivers/pci/controller/cadence/pcie-cadence-plat.c +++ b/drivers/pci/controller/cadence/pcie-cadence-plat.c @@ -14,14 +14,6 @@ #define CDNS_PLAT_CPU_TO_BUS_ADDR 0x0FFFFFFF -/** - * struct cdns_plat_pcie - private data for this PCIe platform driver - * @pcie: Cadence PCIe controller - */ -struct cdns_plat_pcie { - struct cdns_pcie *pcie; -}; - struct cdns_plat_pcie_of_data { bool is_rc; }; @@ -40,9 +32,9 @@ static const struct cdns_pcie_ops cdns_plat_ops = { static int cdns_plat_pcie_probe(struct platform_device *pdev) { const struct cdns_plat_pcie_of_data *data; - struct cdns_plat_pcie *cdns_plat_pcie; struct device *dev = &pdev->dev; struct pci_host_bridge *bridge; + struct cdns_pcie *pcie; struct cdns_pcie_ep *ep; struct cdns_pcie_rc *rc; int phy_count; @@ -56,11 +48,6 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev) is_rc = data->is_rc; pr_debug(" Started %s with is_rc: %d\n", __func__, is_rc); - cdns_plat_pcie = devm_kzalloc(dev, sizeof(*cdns_plat_pcie), GFP_KERNEL); - if (!cdns_plat_pcie) - return -ENOMEM; - - platform_set_drvdata(pdev, cdns_plat_pcie); if (is_rc) { if (!IS_ENABLED(CONFIG_PCIE_CADENCE_PLAT_HOST)) return -ENODEV; @@ -72,9 +59,9 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev) rc = pci_host_bridge_priv(bridge); rc->pcie.dev = dev; rc->pcie.ops = &cdns_plat_ops; - cdns_plat_pcie->pcie = &rc->pcie; + pcie = &rc->pcie; - ret = cdns_pcie_init_phy(dev, cdns_plat_pcie->pcie); + ret = cdns_pcie_init_phy(dev, pcie); if (ret) { dev_err(dev, "failed to init phy\n"); return ret; @@ -99,9 +86,9 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev) ep->pcie.dev = dev; ep->pcie.ops = &cdns_plat_ops; - cdns_plat_pcie->pcie = &ep->pcie; + pcie = &ep->pcie; - ret = cdns_pcie_init_phy(dev, cdns_plat_pcie->pcie); + ret = cdns_pcie_init_phy(dev, pcie); if (ret) { dev_err(dev, "failed to init phy\n"); return ret; @@ -119,18 +106,20 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev) goto err_init; } + platform_set_drvdata(pdev, pcie); + return 0; err_init: err_get_sync: pm_runtime_put_sync(dev); pm_runtime_disable(dev); - cdns_pcie_disable_phy(cdns_plat_pcie->pcie); - phy_count = cdns_plat_pcie->pcie->phy_count; + cdns_pcie_disable_phy(pcie); + phy_count = pcie->phy_count; while (phy_count--) - device_link_del(cdns_plat_pcie->pcie->link[phy_count]); + device_link_del(pcie->link[phy_count]); - return 0; + return ret; } static void cdns_plat_pcie_shutdown(struct platform_device *pdev) diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c index bc3d6269f33b..1dad7d69de91 100644 --- a/drivers/pci/controller/dwc/pcie-designware.c +++ b/drivers/pci/controller/dwc/pcie-designware.c @@ -514,6 +514,8 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci, /* The data-less messages only for now */ val |= PCIE_ATU_INHIBIT_PAYLOAD | atu->code; } + if (dw_pcie_ver_is_ge(pci, 460A)) + val |= PCIE_ATU_DMA_BYPASS; dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_REGION_CTRL2, val); /* diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h index a5f58ac8ea94..728f15cebf9e 100644 --- a/drivers/pci/controller/dwc/pcie-designware.h +++ b/drivers/pci/controller/dwc/pcie-designware.h @@ -173,6 +173,7 @@ #define PCIE_ATU_REGION_CTRL2 0x004 #define PCIE_ATU_ENABLE BIT(31) #define PCIE_ATU_BAR_MODE_ENABLE BIT(30) +#define PCIE_ATU_DMA_BYPASS BIT(27) #define PCIE_ATU_INHIBIT_PAYLOAD BIT(22) #define PCIE_ATU_FUNC_NUM_MATCH_EN BIT(19) #define PCIE_ATU_LOWER_BASE 0x008 diff --git a/drivers/pci/controller/dwc/pcie-rcar-gen4.c b/drivers/pci/controller/dwc/pcie-rcar-gen4.c index 8dcb63f813be..a5f750709e2c 100644 --- a/drivers/pci/controller/dwc/pcie-rcar-gen4.c +++ b/drivers/pci/controller/dwc/pcie-rcar-gen4.c @@ -305,6 +305,57 @@ static struct rcar_gen4_pcie *rcar_gen4_pcie_alloc(struct platform_device *pdev) return rcar; } +static int rcar_gen4_pcie_enable_device(struct pci_host_bridge *bridge, + struct pci_dev *dev) +{ + /* + * R-Car Gen4 PCIe controller has a hardware limitation of 256 Bytes + * Max_Payload_Size (MPS). PCIe specification indicates that the MPS + * must not exceed minimum MPS of any element along the packet path. + * The controller reports Max_Payload_Size_Supported (MPSS) 256 Bytes + * for header type 0 and 128 Bytes for header type 1. The PCIe core + * will not allow MPS to be set higher than MPSS; warn here in case + * something went very wrong in the core. + * + * For details, refer to chapter "104.1.1 Features" in either of: + * R-Car S4 R19UH0161EJ0140 Rev.1.40 Jul. 31, 2026 or + * R-Car V4H R19UH0186EJ0140 Rev.1.40 Aug. 7, 2026 or + * R-Car V4M R19UH0217EJ0110 Rev.1.10 Jun. 30, 2026. + */ + WARN_ON(pcie_get_mps(dev) > 256); + + /* + * R-Car Gen4 Reference Manual, chapter 104.4.8 Usage notes for + * MRRS (Max Read Request Size) states: + * + * Please set "Max Read Request Size" to 128 bytes or 256 bytes. + * If "Max Read Request Size" is set to anything other than the + * above, the transferred data will not match the expected value. + * + * This limitation also seems to apply to devices issuing MRd TLPs. + * This limitation can be triggered by using non-HMB NVMe SSD with + * Max_Read_Request_Size 512 Bytes, for example Crucial P5 Plus. + * Any write into the SSD (MRd TLP issued by the SSD) longer than + * 256 Bytes wraps around at 256 Byte boundary, and the same data + * are written into the SSD starting at offset 0 and at 256 Bytes. + * + * Limit Max_Read_Request_Size to at most 256 Bytes for each + * device connected to this PCIe controller to avoid this behavior. + * + * For details, refer to aforementioned chapter in either of: + * R-Car S4 R19UH0161EJ0140 Rev.1.40 Jul. 31, 2026 or + * R-Car V4H R19UH0186EJ0140 Rev.1.40 Aug. 7, 2026 or + * R-Car V4M R19UH0217EJ0110 Rev.1.10 Jun. 30, 2026. + */ + bridge->no_inc_mrrs = 1; + if (pcie_get_readrq(dev) > 256) { + pci_info(dev, "Limiting MRRS to 256 bytes\n"); + pcie_set_readrq(dev, 256); + } + + return 0; +} + /* Host mode */ static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp) { @@ -313,6 +364,9 @@ static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp) int ret; u32 val; + if (pp->bridge) + pp->bridge->enable_device = rcar_gen4_pcie_enable_device; + gpiod_set_value_cansleep(dw->pe_rst, 1); ret = rcar_gen4_pcie_common_init(rcar); diff --git a/drivers/pci/controller/pcie-rockchip.c b/drivers/pci/controller/pcie-rockchip.c index c07d7129f1c7..6a970dc766d4 100644 --- a/drivers/pci/controller/pcie-rockchip.c +++ b/drivers/pci/controller/pcie-rockchip.c @@ -66,8 +66,10 @@ int rockchip_pcie_parse_dt(struct rockchip_pcie *rockchip) } rockchip->link_gen = of_pci_get_max_link_speed(node); - if (rockchip->link_gen < 0 || rockchip->link_gen > 2) - rockchip->link_gen = 2; + if (rockchip->link_gen < 0 || rockchip->link_gen >= 2) { + rockchip->link_gen = 1; + dev_warn(dev, "invalid max-link-speed, limited to 2.5 GT/s\n"); + } rockchip->core_rst = devm_reset_control_get_exclusive(dev, "core"); if (IS_ERR(rockchip->core_rst)) { @@ -237,12 +239,8 @@ int rockchip_pcie_init_port(struct rockchip_pcie *rockchip) goto err_exit_phy; } - if (rockchip->link_gen == 2) - rockchip_pcie_write(rockchip, PCIE_CLIENT_GEN_SEL_2, - PCIE_CLIENT_CONFIG); - else - rockchip_pcie_write(rockchip, PCIE_CLIENT_GEN_SEL_1, - PCIE_CLIENT_CONFIG); + rockchip_pcie_write(rockchip, PCIE_CLIENT_GEN_SEL_1, + PCIE_CLIENT_CONFIG); regs = PCIE_CLIENT_LINK_TRAIN_ENABLE | PCIE_CLIENT_ARI_ENABLE | PCIE_CLIENT_CONF_LANE_NUM(rockchip->lanes); diff --git a/drivers/pci/controller/pcie-rockchip.h b/drivers/pci/controller/pcie-rockchip.h index d916fcc8badb..5629ac4a7439 100644 --- a/drivers/pci/controller/pcie-rockchip.h +++ b/drivers/pci/controller/pcie-rockchip.h @@ -37,7 +37,6 @@ #define PCIE_CLIENT_MODE_RC HIWORD_UPDATE_BIT(0x0040) #define PCIE_CLIENT_MODE_EP HIWORD_UPDATE(0x0040, 0) #define PCIE_CLIENT_GEN_SEL_1 HIWORD_UPDATE(0x0080, 0) -#define PCIE_CLIENT_GEN_SEL_2 HIWORD_UPDATE_BIT(0x0080) #define PCIE_CLIENT_LEGACY_INT_CTRL (PCIE_CLIENT_BASE + 0x0c) #define PCIE_CLIENT_INT_IN_ASSERT HIWORD_UPDATE_BIT(0x0002) #define PCIE_CLIENT_INT_IN_DEASSERT HIWORD_UPDATE(0x0002, 0) @@ -183,8 +182,6 @@ (((x) & PCIE_CLIENT_DEBUG_LTSSM_MASK) == PCIE_CLIENT_DEBUG_LTSSM_L2) #define PCIE_LINK_UP(x) \ (((x) & PCIE_CLIENT_LINK_STATUS_MASK) == PCIE_CLIENT_LINK_STATUS_UP) -#define PCIE_LINK_IS_GEN2(x) \ - (((x) & PCIE_CORE_PL_CONF_SPEED_MASK) == PCIE_CORE_PL_CONF_SPEED_5G) #define RC_REGION_0_ADDR_TRANS_H 0x00000000 #define RC_REGION_0_ADDR_TRANS_L 0x00000000 diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c index 963a1a1f983d..710bddc30cdf 100644 --- a/drivers/pci/endpoint/functions/pci-epf-vntb.c +++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c @@ -116,7 +116,7 @@ struct epf_ntb_ctrl { } __packed; struct epf_ntb { - struct ntb_dev ntb; + struct ntb_dev *ntb; struct pci_epf *epf; struct config_group group; @@ -142,10 +142,15 @@ struct epf_ntb { void __iomem *vpci_mw_addr[MAX_MW]; struct delayed_work cmd_handler; + struct pci_bus *vpci_bus; }; #define to_epf_ntb(epf_group) container_of((epf_group), struct epf_ntb, group) -#define ntb_ndev(__ntb) container_of(__ntb, struct epf_ntb, ntb) + +static struct epf_ntb *ntb_ndev(struct ntb_dev *ntb) +{ + return ntb->pdev->sysdata; +} static struct pci_epf_header epf_ntb_header = { .vendorid = PCI_ANY_ID, @@ -171,7 +176,7 @@ static int epf_ntb_link_up(struct epf_ntb *ntb, bool link_up) else ntb->reg->link_status &= ~LINK_STATUS_UP; - ntb_link_event(&ntb->ntb); + ntb_link_event(ntb->ntb); return 0; } @@ -259,7 +264,7 @@ static void epf_ntb_cmd_handler(struct work_struct *work) for (i = 1; i < ntb->db_count; i++) { if (ntb->epf_db[i]) { ntb->db |= 1 << (i - 1); - ntb_db_event(&ntb->ntb, i); + ntb_db_event(ntb->ntb, i); ntb->epf_db[i] = 0; } } @@ -760,7 +765,6 @@ static int epf_ntb_epc_init(struct epf_ntb *ntb) } INIT_DELAYED_WORK(&ntb->cmd_handler, epf_ntb_cmd_handler); - queue_work(kpcintb_workqueue, &ntb->cmd_handler.work); return 0; @@ -1009,6 +1013,7 @@ static int vpci_scan_bus(void *sysdata) pr_err("create pci bus failed\n"); return -EINVAL; } + ndev->vpci_bus = vpci_bus; pci_bus_add_devices(vpci_bus); @@ -1053,7 +1058,7 @@ static int vntb_epf_mw_set_trans(struct ntb_dev *ndev, int pidx, int idx, int ret; struct device *dev; - dev = &ntb->ntb.dev; + dev = &ndev->dev; barno = ntb->epf_ntb_bar[BAR_MW0 + idx]; epf_bar = &ntb->epf->bar[barno]; epf_bar->phys_addr = addr; @@ -1153,7 +1158,7 @@ static int vntb_epf_peer_db_set(struct ntb_dev *ndev, u64 db_bits) ret = pci_epc_raise_irq(ntb->epf->epc, func_no, vfunc_no, PCI_IRQ_MSI, interrupt_num + 1); if (ret) - dev_err(&ntb->ntb.dev, "Failed to raise IRQ\n"); + dev_err(&ntb->epf->dev, "Failed to raise IRQ\n"); return ret; } @@ -1236,13 +1241,18 @@ static const struct ntb_dev_ops vntb_epf_ops = { static int pci_vntb_probe(struct pci_dev *pdev, const struct pci_device_id *id) { - int ret; struct epf_ntb *ndev = (struct epf_ntb *)pdev->sysdata; struct device *dev = &pdev->dev; + struct ntb_dev *ntb; + int ret; - ndev->ntb.pdev = pdev; - ndev->ntb.topo = NTB_TOPO_NONE; - ndev->ntb.ops = &vntb_epf_ops; + ntb = devm_kzalloc(dev, sizeof(*ntb), GFP_KERNEL); + if (!ntb) + return -ENOMEM; + + ntb->pdev = pdev; + ntb->topo = NTB_TOPO_NONE; + ntb->ops = &vntb_epf_ops; ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)); if (ret) { @@ -1250,16 +1260,33 @@ static int pci_vntb_probe(struct pci_dev *pdev, const struct pci_device_id *id) return ret; } - ret = ntb_register_device(&ndev->ntb); + ret = ntb_register_device(ntb); if (ret) { dev_err(dev, "Failed to register NTB device\n"); return ret; } + /* Publish after ntb_register_device() succeeds */ + smp_store_release(&ndev->ntb, ntb); + queue_delayed_work(kpcintb_workqueue, &ndev->cmd_handler, 0); + dev_dbg(dev, "PCI Virtual NTB driver loaded\n"); return 0; } +static void pci_vntb_remove(struct pci_dev *pdev) +{ + struct epf_ntb *ndev = pdev->sysdata; + struct ntb_dev *ntb; + + /* Stop the work reader */ + cancel_delayed_work_sync(&ndev->cmd_handler); + + ntb = xchg(&ndev->ntb, NULL); + + ntb_unregister_device(ntb); +} + static struct pci_device_id pci_vntb_table[] = { { PCI_DEVICE(0xffff, 0xffff), @@ -1271,6 +1298,7 @@ static struct pci_driver vntb_pci_driver = { .name = "pci-vntb", .id_table = pci_vntb_table, .probe = pci_vntb_probe, + .remove = pci_vntb_remove, }; /* ============ PCIe EPF Driver Bind ====================*/ @@ -1351,12 +1379,25 @@ static int epf_ntb_bind(struct pci_epf *epf) */ static void epf_ntb_unbind(struct pci_epf *epf) { + struct pci_host_bridge *bridge; struct epf_ntb *ntb = epf_get_drvdata(epf); + pci_unregister_driver(&vntb_pci_driver); + + if (ntb->vpci_bus) { + bridge = to_pci_host_bridge(ntb->vpci_bus->bridge); + + pci_lock_rescan_remove(); + pci_stop_root_bus(ntb->vpci_bus); + pci_remove_root_bus(ntb->vpci_bus); + ntb->vpci_bus = NULL; + pci_unlock_rescan_remove(); + + pci_free_host_bridge(bridge); + } + epf_ntb_epc_cleanup(ntb); epf_ntb_config_spad_bar_free(ntb); - - pci_unregister_driver(&vntb_pci_driver); } // EPF driver probe diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 2f575af94e09..b53ee0920bdc 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -2090,6 +2090,28 @@ int __weak pcibios_enable_device(struct pci_dev *dev, int bars) return pci_enable_resources(dev, bars); } +static int pci_host_bridge_enable_device(struct pci_dev *dev) +{ + struct pci_host_bridge *host_bridge = pci_find_host_bridge(dev->bus); + int err; + + if (host_bridge && host_bridge->enable_device) { + err = host_bridge->enable_device(host_bridge, dev); + if (err) + return err; + } + + return 0; +} + +static void pci_host_bridge_disable_device(struct pci_dev *dev) +{ + struct pci_host_bridge *host_bridge = pci_find_host_bridge(dev->bus); + + if (host_bridge && host_bridge->disable_device) + host_bridge->disable_device(host_bridge, dev); +} + static int do_pci_enable_device(struct pci_dev *dev, int bars) { int err; @@ -2105,9 +2127,13 @@ static int do_pci_enable_device(struct pci_dev *dev, int bars) if (bridge) pcie_aspm_powersave_config_link(bridge); + err = pci_host_bridge_enable_device(dev); + if (err) + return err; + err = pcibios_enable_device(dev, bars); if (err < 0) - return err; + goto err_enable; pci_fixup_device(pci_fixup_enable, dev); if (dev->msi_enabled || dev->msix_enabled) @@ -2122,6 +2148,12 @@ static int do_pci_enable_device(struct pci_dev *dev, int bars) } return 0; + +err_enable: + pci_host_bridge_disable_device(dev); + + return err; + } /** @@ -2305,6 +2337,8 @@ void pci_disable_device(struct pci_dev *dev) if (atomic_dec_return(&dev->enable_cnt) != 0) return; + pci_host_bridge_disable_device(dev); + do_pci_disable_device(dev); dev->is_busmaster = 0; diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index 8cfa7a3f55cb..5b266583acb5 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -93,8 +93,12 @@ int pcie_failed_link_retrain(struct pci_dev *dev) pcie_capability_read_word(dev, PCI_EXP_LNKCTL2, &lnkctl2); pcie_capability_read_word(dev, PCI_EXP_LNKSTA, &lnksta); oldlnkctl2 = lnkctl2; - if ((lnksta & (PCI_EXP_LNKSTA_LBMS | PCI_EXP_LNKSTA_DLLLA)) == - PCI_EXP_LNKSTA_LBMS) { + if (lnksta & PCI_EXP_LNKSTA_DLLLA) { + ; + } else if ((lnkctl2 & PCI_EXP_LNKCTL2_TLS) == + PCI_EXP_LNKCTL2_TLS_2_5GT) { + return ret; + } else if (lnksta & PCI_EXP_LNKSTA_LBMS) { pci_info(dev, "broken device, retraining non-functional downstream link at 2.5GT/s\n"); lnkctl2 &= ~PCI_EXP_LNKCTL2_TLS; diff --git a/drivers/platform/x86/amd/pmf/core.c b/drivers/platform/x86/amd/pmf/core.c index 496fea1389fc..2039c9d90f87 100644 --- a/drivers/platform/x86/amd/pmf/core.c +++ b/drivers/platform/x86/amd/pmf/core.c @@ -467,6 +467,7 @@ static const struct acpi_device_id amd_pmf_acpi_ids[] = { {"AMDI0105", 0}, {"AMDI0107", 0}, {"AMDI0108", 0}, + {"AMDI0112", 0}, { } }; MODULE_DEVICE_TABLE(acpi, amd_pmf_acpi_ids); diff --git a/drivers/platform/x86/asus-wmi.c b/drivers/platform/x86/asus-wmi.c index 1465057b3e3d..e71d2be12e78 100644 --- a/drivers/platform/x86/asus-wmi.c +++ b/drivers/platform/x86/asus-wmi.c @@ -15,6 +15,7 @@ #include #include +#include #include #include #include @@ -27,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -337,6 +339,20 @@ struct asus_wmi { /* WMI ************************************************************************/ +/* + * Concurrent evaluations of ASUS WMI methods can re-enter firmware + * SMI/EC mailbox handling and corrupt the mailbox. + */ +static DEFINE_MUTEX(asus_wmi_eval_lock); + +static acpi_status asus_wmi_evaluate_method_locked(u32 method_id, + struct acpi_buffer *input, + struct acpi_buffer *output) +{ + guard(mutex)(&asus_wmi_eval_lock); + return wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id, input, output); +} + static int asus_wmi_evaluate_method3(u32 method_id, u32 arg0, u32 arg1, u32 arg2, u32 *retval) { @@ -351,8 +367,7 @@ static int asus_wmi_evaluate_method3(u32 method_id, union acpi_object *obj; u32 tmp = 0; - status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id, - &input, &output); + status = asus_wmi_evaluate_method_locked(method_id, &input, &output); pr_debug("%s called (0x%08x) with args: 0x%08x, 0x%08x, 0x%08x\n", __func__, method_id, arg0, arg1, arg2); @@ -403,8 +418,7 @@ static int asus_wmi_evaluate_method5(u32 method_id, union acpi_object *obj; u32 tmp = 0; - status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id, - &input, &output); + status = asus_wmi_evaluate_method_locked(method_id, &input, &output); pr_debug("%s called (0x%08x) with args: 0x%08x, 0x%08x, 0x%08x, 0x%08x, 0x%08x\n", __func__, method_id, arg0, arg1, arg2, arg3, arg4); @@ -451,8 +465,7 @@ static int asus_wmi_evaluate_method_buf(u32 method_id, union acpi_object *obj; int err = 0; - status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id, - &input, &output); + status = asus_wmi_evaluate_method_locked(method_id, &input, &output); pr_debug("%s called (0x%08x) with args: 0x%08x, 0x%08x\n", __func__, method_id, arg0, arg1); @@ -4632,9 +4645,8 @@ static int show_call(struct seq_file *m, void *data) union acpi_object *obj; acpi_status status; - status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID, - 0, asus->debug.method_id, - &input, &output); + status = asus_wmi_evaluate_method_locked(asus->debug.method_id, + &input, &output); if (ACPI_FAILURE(status)) return -EIO; diff --git a/drivers/platform/x86/intel/int3472/discrete.c b/drivers/platform/x86/intel/int3472/discrete.c index 9e69ac9cfb92..06191446e38b 100644 --- a/drivers/platform/x86/intel/int3472/discrete.c +++ b/drivers/platform/x86/intel/int3472/discrete.c @@ -237,7 +237,6 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares, u8 active_value, pin, type; union acpi_object *obj; struct gpio_desc *gpio; - const char *err_msg; const char *func; unsigned long gpio_flags; int ret; @@ -283,7 +282,8 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares, case INT3472_GPIO_TYPE_POWERDOWN: ret = skl_int3472_map_gpio_to_sensor(int3472, agpio, func, gpio_flags); if (ret) - err_msg = "Failed to map GPIO pin to sensor\n"; + dev_err_probe(int3472->dev, ret, + "Failed to map GPIO pin to sensor\n"); break; case INT3472_GPIO_TYPE_CLK_ENABLE: @@ -292,7 +292,7 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares, gpio = skl_int3472_gpiod_get_from_temp_lookup(int3472, agpio, func, gpio_flags); if (IS_ERR(gpio)) { ret = PTR_ERR(gpio); - err_msg = "Failed to get GPIO\n"; + dev_err_probe(int3472->dev, ret, "Failed to get GPIO\n"); break; } @@ -300,19 +300,22 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares, case INT3472_GPIO_TYPE_CLK_ENABLE: ret = skl_int3472_register_gpio_clock(int3472, gpio); if (ret) - err_msg = "Failed to register clock\n"; + dev_err_probe(int3472->dev, ret, + "Failed to register clock\n"); break; case INT3472_GPIO_TYPE_PRIVACY_LED: ret = skl_int3472_register_pled(int3472, gpio); if (ret) - err_msg = "Failed to register LED\n"; + dev_err_probe(int3472->dev, ret, + "Failed to register LED\n"); break; case INT3472_GPIO_TYPE_POWER_ENABLE: ret = skl_int3472_register_regulator(int3472, gpio); if (ret) - err_msg = "Failed to map regulator to sensor\n"; + dev_err_probe(int3472->dev, ret, + "Failed to map regulator to sensor\n"); break; default: /* Never reached */ @@ -331,9 +334,6 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares, int3472->ngpios++; ACPI_FREE(obj); - if (ret < 0) - return dev_err_probe(int3472->dev, ret, err_msg); - return ret; } diff --git a/drivers/power/supply/Makefile b/drivers/power/supply/Makefile index 8dcb41545317..6673860ff26c 100644 --- a/drivers/power/supply/Makefile +++ b/drivers/power/supply/Makefile @@ -115,5 +115,5 @@ obj-$(CONFIG_BATTERY_ACER_A500) += acer_a500_battery.o obj-$(CONFIG_BATTERY_SURFACE) += surface_battery.o obj-$(CONFIG_CHARGER_SURFACE) += surface_charger.o obj-$(CONFIG_BATTERY_UG3105) += ug3105_battery.o -obj-$(CONFIG_CHARGER_QCOM_SMB2) += qcom_pmi8998_charger.o +obj-$(CONFIG_CHARGER_QCOM_SMB2) += qcom_smbx.o obj-$(CONFIG_FUEL_GAUGE_MM8013) += mm8013.o diff --git a/drivers/power/supply/bq27xxx_battery.c b/drivers/power/supply/bq27xxx_battery.c index 19c73aad769a..59d9e2d99c3f 100644 --- a/drivers/power/supply/bq27xxx_battery.c +++ b/drivers/power/supply/bq27xxx_battery.c @@ -43,6 +43,7 @@ */ #include +#include #include #include #include @@ -2150,26 +2151,17 @@ static void bq27xxx_external_power_changed(struct power_supply *psy) mod_delayed_work(system_wq, &di->work, HZ / 2); } -static void bq27xxx_battery_mutex_destroy(void *data) -{ - struct mutex *lock = data; - - mutex_destroy(lock); -} - int bq27xxx_battery_setup(struct bq27xxx_device_info *di) { struct power_supply_desc *psy_desc; struct power_supply_config psy_cfg = { .of_node = di->dev->of_node, .drv_data = di, + .no_wakeup_source = true, }; int ret; - INIT_DELAYED_WORK(&di->work, bq27xxx_battery_poll); - mutex_init(&di->lock); - ret = devm_add_action_or_reset(di->dev, bq27xxx_battery_mutex_destroy, - &di->lock); + ret = devm_mutex_init(di->dev, &di->lock); if (ret) return ret; @@ -2189,7 +2181,13 @@ int bq27xxx_battery_setup(struct bq27xxx_device_info *di) psy_desc->get_property = bq27xxx_battery_get_property; psy_desc->external_power_changed = bq27xxx_external_power_changed; - di->bat = devm_power_supply_register_no_ws(di->dev, psy_desc, &psy_cfg); + /* Cancel the poll work after the power_supply is unregistered. */ + ret = devm_delayed_work_autocancel(di->dev, &di->work, + bq27xxx_battery_poll); + if (ret) + return ret; + + di->bat = devm_power_supply_register(di->dev, psy_desc, &psy_cfg); if (IS_ERR(di->bat)) return dev_err_probe(di->dev, PTR_ERR(di->bat), "failed to register battery\n"); @@ -2215,8 +2213,6 @@ void bq27xxx_battery_teardown(struct bq27xxx_device_info *di) mutex_lock(&di->lock); di->removed = true; mutex_unlock(&di->lock); - - cancel_delayed_work_sync(&di->work); } EXPORT_SYMBOL_GPL(bq27xxx_battery_teardown); diff --git a/drivers/power/supply/cros_charge-control.c b/drivers/power/supply/cros_charge-control.c index 9b0a7500296b..2c16ce4cf6b7 100644 --- a/drivers/power/supply/cros_charge-control.c +++ b/drivers/power/supply/cros_charge-control.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #define EC_CHARGE_CONTROL_BEHAVIOURS (BIT(POWER_SUPPLY_CHARGE_BEHAVIOUR_AUTO) | \ @@ -322,6 +323,7 @@ static int cros_chctl_probe(struct platform_device *pdev) dev_dbg(dev, "Command version: %u\n", (unsigned int)priv->cmd_version); priv->cros_ec = cros_ec; + dev_set_drvdata(dev, priv); priv->device_attrs[CROS_CHCTL_ATTR_START_THRESHOLD] = (struct device_attribute)__ATTR_RW(charge_control_start_threshold); priv->device_attrs[CROS_CHCTL_ATTR_END_THRESHOLD] = @@ -352,13 +354,38 @@ static int cros_chctl_probe(struct platform_device *pdev) return devm_battery_hook_register(dev, &priv->battery_hook); } +static int cros_chctl_resume(struct device *dev) +{ + struct cros_chctl_priv *priv = dev_get_drvdata(dev); + int ret; + + /* + * The EC resets its charge control state when the AP powers off, so the + * configuration is lost across hibernation, while the values exposed to + * userspace still describe it. The cached state is the only remaining + * record of that configuration: reprogram the EC from it rather than + * re-reading the EC, which would adopt the reset defaults instead. + */ + guard(mutex)(&priv->lock); + ret = cros_chctl_configure_ec(priv); + if (ret) + dev_err(dev, "failed to restore charge control state: %d\n", ret); + + return ret; +} + +static DEFINE_SIMPLE_DEV_PM_OPS(cros_chctl_pm_ops, NULL, cros_chctl_resume); + static const struct platform_device_id cros_chctl_id[] = { { "cros-charge-control", 0 }, {} }; static struct platform_driver cros_chctl_driver = { - .driver.name = "cros-charge-control", + .driver = { + .name = "cros-charge-control", + .pm = pm_sleep_ptr(&cros_chctl_pm_ops), + }, .probe = cros_chctl_probe, .id_table = cros_chctl_id, }; diff --git a/drivers/power/supply/ltc2941-battery-gauge.c b/drivers/power/supply/ltc2941-battery-gauge.c index a1ddc4b060ce..bf0b268b10bf 100644 --- a/drivers/power/supply/ltc2941-battery-gauge.c +++ b/drivers/power/supply/ltc2941-battery-gauge.c @@ -464,9 +464,11 @@ static int ltc294x_i2c_probe(struct i2c_client *client) /* r_sense can be negative, when sense+ is connected to the battery * instead of the sense-. This results in reversed measurements. */ ret = of_property_read_u32(np, "lltc,resistor-sense", &r_sense); - if (ret < 0) + if (ret < 0) { + of_node_put(np); return dev_err_probe(&client->dev, ret, "Could not find lltc,resistor-sense in devicetree\n"); + } info->r_sense = r_sense; ret = of_property_read_u32(np, "lltc,prescaler-exponent", @@ -477,6 +479,8 @@ static int ltc294x_i2c_probe(struct i2c_client *client) prescaler_exp = LTC2941_MAX_PRESCALER_EXP; } + of_node_put(np); + if (info->id == LTC2943_ID) { if (prescaler_exp > LTC2943_MAX_PRESCALER_EXP) prescaler_exp = LTC2943_MAX_PRESCALER_EXP; diff --git a/drivers/power/supply/mt6370-charger.c b/drivers/power/supply/mt6370-charger.c index ad8793bf997e..16ffa927147f 100644 --- a/drivers/power/supply/mt6370-charger.c +++ b/drivers/power/supply/mt6370-charger.c @@ -761,13 +761,6 @@ static int mt6370_chg_init_psy(struct mt6370_priv *priv) return PTR_ERR_OR_ZERO(priv->psy); } -static void mt6370_chg_destroy_attach_lock(void *data) -{ - struct mutex *attach_lock = data; - - mutex_destroy(attach_lock); -} - static void mt6370_chg_destroy_wq(void *data) { struct workqueue_struct *wq = data; @@ -895,9 +888,7 @@ static int mt6370_chg_probe(struct platform_device *pdev) if (ret) return dev_err_probe(dev, ret, "Failed to init psy\n"); - mutex_init(&priv->attach_lock); - ret = devm_add_action_or_reset(dev, mt6370_chg_destroy_attach_lock, - &priv->attach_lock); + ret = devm_mutex_init(dev, &priv->attach_lock); if (ret) return dev_err_probe(dev, ret, "Failed to init attach lock\n"); diff --git a/drivers/power/supply/power_supply_core.c b/drivers/power/supply/power_supply_core.c index edc7b3bd9b5c..9610ea1c341d 100644 --- a/drivers/power/supply/power_supply_core.c +++ b/drivers/power/supply/power_supply_core.c @@ -66,10 +66,9 @@ static bool __power_supply_is_supplied_by(struct power_supply *supplier, return false; } -static int __power_supply_changed_work(struct device *dev, void *data) +static int __power_supply_changed_work(struct power_supply *pst, void *data) { struct power_supply *psy = data; - struct power_supply *pst = dev_get_drvdata(dev); if (__power_supply_is_supplied_by(psy, pst)) { if (pst->desc->external_power_changed) @@ -98,7 +97,7 @@ static void power_supply_changed_work(struct work_struct *work) if (likely(psy->changed)) { psy->changed = false; spin_unlock_irqrestore(&psy->changed_lock, flags); - power_supply_for_each_device(psy, __power_supply_changed_work); + power_supply_for_each_psy(psy, __power_supply_changed_work); power_supply_update_leds(psy); blocking_notifier_call_chain(&power_supply_notifier, PSY_EVENT_PROP_CHANGED, psy); @@ -122,6 +121,30 @@ int power_supply_for_each_device(void *data, int (*fn)(struct device *dev, void } EXPORT_SYMBOL_GPL(power_supply_for_each_device); +struct psy_for_each_psy_cb_data { + int (*fn)(struct power_supply *psy, void *data); + void *data; +}; + +static int psy_for_each_psy_cb(struct device *dev, void *data) +{ + struct psy_for_each_psy_cb_data *cb_data = data; + struct power_supply *psy = dev_get_drvdata(dev); + + return cb_data->fn(psy, cb_data->data); +} + +int power_supply_for_each_psy(void *data, int (*fn)(struct power_supply *psy, void *data)) +{ + struct psy_for_each_psy_cb_data cb_data = { + .fn = fn, + .data = data, + }; + + return power_supply_for_each_device(&cb_data, psy_for_each_psy_cb); +} +EXPORT_SYMBOL_GPL(power_supply_for_each_psy); + void power_supply_changed(struct power_supply *psy) { unsigned long flags; @@ -165,21 +188,48 @@ static void power_supply_deferred_register_work(struct work_struct *work) mutex_unlock(&psy->dev.parent->mutex); } -#ifdef CONFIG_OF -static int __power_supply_populate_supplied_from(struct device *dev, +static int power_supply_check_supplies_by_name(struct power_supply *psy) +{ + struct device *parent = psy->dev.parent; + int nval, ret; + + if (!parent) + return 0; + + nval = device_property_string_array_count(parent, "supplied-from"); + if (nval <= 0) + return 0; + + psy->supplied_from = devm_kmalloc_array(&psy->dev, nval, + sizeof(*psy->supplied_from), + GFP_KERNEL); + if (!psy->supplied_from) + return -ENOMEM; + + ret = device_property_read_string_array(parent, "supplied-from", + (const char **)psy->supplied_from, + nval); + if (ret < 0) + return ret; + + psy->num_supplies = nval; + + return 0; +} + +static int __power_supply_populate_supplied_from(struct power_supply *epsy, void *data) { struct power_supply *psy = data; - struct power_supply *epsy = dev_get_drvdata(dev); struct device_node *np; int i = 0; do { - np = of_parse_phandle(psy->of_node, "power-supplies", i++); + np = of_parse_phandle(psy->dev.of_node, "power-supplies", i++); if (!np) break; - if (np == epsy->of_node) { + if (np == epsy->dev.of_node) { dev_dbg(&psy->dev, "%s: Found supply : %s\n", psy->desc->name, epsy->desc->name); psy->supplied_from[i-1] = (char *)epsy->desc->name; @@ -197,21 +247,20 @@ static int power_supply_populate_supplied_from(struct power_supply *psy) { int error; - error = power_supply_for_each_device(psy, __power_supply_populate_supplied_from); + error = power_supply_for_each_psy(psy, __power_supply_populate_supplied_from); dev_dbg(&psy->dev, "%s %d\n", __func__, error); return error; } -static int __power_supply_find_supply_from_node(struct device *dev, +static int __power_supply_find_supply_from_node(struct power_supply *epsy, void *data) { struct device_node *np = data; - struct power_supply *epsy = dev_get_drvdata(dev); - /* returning non-zero breaks out of power_supply_for_each_device loop */ - if (epsy->of_node == np) + /* returning non-zero breaks out of power_supply_for_each_psy loop */ + if (epsy->dev.of_node == np) return 1; return 0; @@ -222,99 +271,70 @@ static int power_supply_find_supply_from_node(struct device_node *supply_node) int error; /* - * power_supply_for_each_device() either returns its own errors or values + * power_supply_for_each_psy() either returns its own errors or values * returned by __power_supply_find_supply_from_node(). * * __power_supply_find_supply_from_node() will return 0 (no match) * or 1 (match). * - * We return 0 if power_supply_for_each_device() returned 1, -EPROBE_DEFER if + * We return 0 if power_supply_for_each_psy() returned 1, -EPROBE_DEFER if * it returned 0, or error as returned by it. */ - error = power_supply_for_each_device(supply_node, __power_supply_find_supply_from_node); + error = power_supply_for_each_psy(supply_node, __power_supply_find_supply_from_node); return error ? (error == 1 ? 0 : error) : -EPROBE_DEFER; } static int power_supply_check_supplies(struct power_supply *psy) { - struct device_node *np; - int cnt = 0; + struct of_phandle_args args; + int cnt, ret; /* If there is already a list honor it */ if (psy->supplied_from && psy->num_supplies > 0) return 0; - /* No device node found, nothing to do */ - if (!psy->of_node) - return 0; - - do { - int ret; + for (cnt = 0; psy->dev.of_node; cnt++) { + ret = of_parse_phandle_with_fixed_args(psy->dev.of_node, + "power-supplies", 0, cnt, &args); + if (ret) { + if (ret != -ENOENT) + return ret; - np = of_parse_phandle(psy->of_node, "power-supplies", cnt++); - if (!np) break; + } - ret = power_supply_find_supply_from_node(np); - of_node_put(np); + ret = power_supply_find_supply_from_node(args.np); + of_node_put(args.np); if (ret) { dev_dbg(&psy->dev, "Failed to find supply!\n"); return ret; } - } while (np); + } - /* Missing valid "power-supplies" entries */ - if (cnt == 1) - return 0; + /* Fall back to the name-based property if no references were specified. */ + if (!cnt) + return power_supply_check_supplies_by_name(psy); /* All supplies found, allocate char * array for filling */ psy->supplied_from = devm_kcalloc(&psy->dev, - cnt - 1, sizeof(*psy->supplied_from), + cnt, sizeof(*psy->supplied_from), GFP_KERNEL); if (!psy->supplied_from) return -ENOMEM; return power_supply_populate_supplied_from(psy); } -#else -static int power_supply_check_supplies(struct power_supply *psy) -{ - int nval, ret; - - if (!psy->dev.parent) - return 0; - - nval = device_property_string_array_count(psy->dev.parent, "supplied-from"); - if (nval <= 0) - return 0; - - psy->supplied_from = devm_kmalloc_array(&psy->dev, nval, - sizeof(char *), GFP_KERNEL); - if (!psy->supplied_from) - return -ENOMEM; - - ret = device_property_read_string_array(psy->dev.parent, - "supplied-from", (const char **)psy->supplied_from, nval); - if (ret < 0) - return ret; - - psy->num_supplies = nval; - - return 0; -} -#endif struct psy_am_i_supplied_data { struct power_supply *psy; unsigned int count; }; -static int __power_supply_am_i_supplied(struct device *dev, void *_data) +static int __power_supply_am_i_supplied(struct power_supply *epsy, void *_data) { union power_supply_propval ret = {0,}; - struct power_supply *epsy = dev_get_drvdata(dev); struct psy_am_i_supplied_data *data = _data; if (__power_supply_is_supplied_by(epsy, data->psy)) { @@ -332,7 +352,7 @@ int power_supply_am_i_supplied(struct power_supply *psy) struct psy_am_i_supplied_data data = { psy, 0 }; int error; - error = power_supply_for_each_device(&data, __power_supply_am_i_supplied); + error = power_supply_for_each_psy(&data, __power_supply_am_i_supplied); dev_dbg(&psy->dev, "%s count %u err %d\n", __func__, data.count, error); @@ -343,10 +363,9 @@ int power_supply_am_i_supplied(struct power_supply *psy) } EXPORT_SYMBOL_GPL(power_supply_am_i_supplied); -static int __power_supply_is_system_supplied(struct device *dev, void *data) +static int __power_supply_is_system_supplied(struct power_supply *psy, void *data) { union power_supply_propval ret = {0,}; - struct power_supply *psy = dev_get_drvdata(dev); unsigned int *count = data; if (!psy->desc->get_property(psy, POWER_SUPPLY_PROP_SCOPE, &ret)) @@ -367,7 +386,7 @@ int power_supply_is_system_supplied(void) int error; unsigned int count = 0; - error = power_supply_for_each_device(&count, __power_supply_is_system_supplied); + error = power_supply_for_each_psy(&count, __power_supply_is_system_supplied); /* * If no system scope power class device was found at all, most probably we @@ -386,9 +405,8 @@ struct psy_get_supplier_prop_data { union power_supply_propval *val; }; -static int __power_supply_get_supplier_property(struct device *dev, void *_data) +static int __power_supply_get_supplier_property(struct power_supply *epsy, void *_data) { - struct power_supply *epsy = dev_get_drvdata(dev); struct psy_get_supplier_prop_data *data = _data; if (__power_supply_is_supplied_by(epsy, data->psy)) @@ -413,7 +431,7 @@ int power_supply_get_property_from_supplier(struct power_supply *psy, * This function is not intended for use with a supply with multiple * suppliers, we simply pick the first supply to report the psp. */ - ret = power_supply_for_each_device(&data, __power_supply_get_supplier_property); + ret = power_supply_for_each_psy(&data, __power_supply_get_supplier_property); if (ret < 0) return ret; if (ret == 0) @@ -580,8 +598,8 @@ int power_supply_get_battery_info(struct power_supply *psy, const __be32 *list; u32 min_max[2]; - if (psy->of_node) { - battery_np = of_parse_phandle(psy->of_node, "monitored-battery", 0); + if (psy->dev.of_node) { + battery_np = of_parse_phandle(psy->dev.of_node, "monitored-battery", 0); if (!battery_np) return -ENODEV; @@ -1366,9 +1384,8 @@ __power_supply_register(struct device *parent, if (cfg) { dev->groups = cfg->attr_grp; psy->drv_data = cfg->drv_data; - psy->of_node = + dev->of_node = cfg->fwnode ? to_of_node(cfg->fwnode) : cfg->of_node; - dev->of_node = psy->of_node; psy->supplied_to = cfg->supplied_to; psy->num_supplicants = cfg->num_supplicants; } @@ -1403,6 +1420,9 @@ __power_supply_register(struct device *parent, if (rc) goto device_add_failed; + if (cfg && cfg->no_wakeup_source) + ws = false; + rc = device_init_wakeup(dev, ws); if (rc) goto wakeup_init_failed; diff --git a/drivers/power/supply/qcom_battmgr.c b/drivers/power/supply/qcom_battmgr.c index d55982553863..3674afb0d49d 100644 --- a/drivers/power/supply/qcom_battmgr.c +++ b/drivers/power/supply/qcom_battmgr.c @@ -998,6 +998,23 @@ static unsigned int qcom_battmgr_sc8280xp_convert_temp(unsigned int temperature) return DIV_ROUND_CLOSEST(temperature, 10); } +/* + * The charger supplies' online state comes from the battery status response, + * and the firmware sends no notification of its own when it changes, so + * announce the change here. A supply that is not registered yet announces its + * state when it is. + */ +static void qcom_battmgr_sc8280xp_set_online(struct power_supply *psy, + bool *online, bool value) +{ + if (*online == value) + return; + + *online = value; + if (psy) + power_supply_changed(psy); +} + static void qcom_battmgr_sc8280xp_callback(struct qcom_battmgr *battmgr, const struct qcom_battmgr_message *resp, size_t len) @@ -1067,9 +1084,12 @@ static void qcom_battmgr_sc8280xp_callback(struct qcom_battmgr *battmgr, battmgr->status.temperature = qcom_battmgr_sc8280xp_convert_temp(le32_to_cpu(resp->status.temperature)); source = le32_to_cpu(resp->status.charging_source); - battmgr->ac.online = source == BATTMGR_CHARGING_SOURCE_AC; - battmgr->usb.online = source == BATTMGR_CHARGING_SOURCE_USB; - battmgr->wireless.online = source == BATTMGR_CHARGING_SOURCE_WIRELESS; + qcom_battmgr_sc8280xp_set_online(battmgr->ac_psy, &battmgr->ac.online, + source == BATTMGR_CHARGING_SOURCE_AC); + qcom_battmgr_sc8280xp_set_online(battmgr->usb_psy, &battmgr->usb.online, + source == BATTMGR_CHARGING_SOURCE_USB); + qcom_battmgr_sc8280xp_set_online(battmgr->wls_psy, &battmgr->wireless.online, + source == BATTMGR_CHARGING_SOURCE_WIRELESS); break; case BATTMGR_BAT_DISCHARGE_TIME: battmgr->status.discharge_time = le32_to_cpu(resp->time); diff --git a/drivers/power/supply/qcom_pmi8998_charger.c b/drivers/power/supply/qcom_pmi8998_charger.c deleted file mode 100644 index d4c9f85654da..000000000000 --- a/drivers/power/supply/qcom_pmi8998_charger.c +++ /dev/null @@ -1,1053 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * Copyright (c) 2016-2019 The Linux Foundation. All rights reserved. - * Copyright (c) 2023, Linaro Ltd. - * Author: Caleb Connolly - * - * This driver is for the switch-mode battery charger and boost - * hardware found in pmi8998 and related PMICs. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -/* clang-format off */ -#define BATTERY_CHARGER_STATUS_1 0x06 -#define BVR_INITIAL_RAMP_BIT BIT(7) -#define CC_SOFT_TERMINATE_BIT BIT(6) -#define STEP_CHARGING_STATUS_SHIFT 3 -#define STEP_CHARGING_STATUS_MASK GENMASK(5, 3) -#define BATTERY_CHARGER_STATUS_MASK GENMASK(2, 0) - -#define BATTERY_CHARGER_STATUS_2 0x07 -#define INPUT_CURRENT_LIMITED_BIT BIT(7) -#define CHARGER_ERROR_STATUS_SFT_EXPIRE_BIT BIT(6) -#define CHARGER_ERROR_STATUS_BAT_OV_BIT BIT(5) -#define CHARGER_ERROR_STATUS_BAT_TERM_MISSING_BIT BIT(4) -#define BAT_TEMP_STATUS_MASK GENMASK(3, 0) -#define BAT_TEMP_STATUS_SOFT_LIMIT_MASK GENMASK(3, 2) -#define BAT_TEMP_STATUS_HOT_SOFT_LIMIT_BIT BIT(3) -#define BAT_TEMP_STATUS_COLD_SOFT_LIMIT_BIT BIT(2) -#define BAT_TEMP_STATUS_TOO_HOT_BIT BIT(1) -#define BAT_TEMP_STATUS_TOO_COLD_BIT BIT(0) - -#define BATTERY_CHARGER_STATUS_4 0x0A -#define CHARGE_CURRENT_POST_JEITA_MASK GENMASK(7, 0) - -#define BATTERY_CHARGER_STATUS_7 0x0D -#define ENABLE_TRICKLE_BIT BIT(7) -#define ENABLE_PRE_CHARGING_BIT BIT(6) -#define ENABLE_FAST_CHARGING_BIT BIT(5) -#define ENABLE_FULLON_MODE_BIT BIT(4) -#define TOO_COLD_ADC_BIT BIT(3) -#define TOO_HOT_ADC_BIT BIT(2) -#define HOT_SL_ADC_BIT BIT(1) -#define COLD_SL_ADC_BIT BIT(0) - -#define CHARGING_ENABLE_CMD 0x42 -#define CHARGING_ENABLE_CMD_BIT BIT(0) - -#define CHGR_CFG2 0x51 -#define CHG_EN_SRC_BIT BIT(7) -#define CHG_EN_POLARITY_BIT BIT(6) -#define PRETOFAST_TRANSITION_CFG_BIT BIT(5) -#define BAT_OV_ECC_BIT BIT(4) -#define I_TERM_BIT BIT(3) -#define AUTO_RECHG_BIT BIT(2) -#define EN_ANALOG_DROP_IN_VBATT_BIT BIT(1) -#define CHARGER_INHIBIT_BIT BIT(0) - -#define PRE_CHARGE_CURRENT_CFG 0x60 -#define PRE_CHARGE_CURRENT_SETTING_MASK GENMASK(5, 0) - -#define FAST_CHARGE_CURRENT_CFG 0x61 -#define FAST_CHARGE_CURRENT_SETTING_MASK GENMASK(7, 0) - -#define FLOAT_VOLTAGE_CFG 0x70 -#define FLOAT_VOLTAGE_SETTING_MASK GENMASK(7, 0) - -#define FG_UPDATE_CFG_2_SEL 0x7D -#define SOC_LT_OTG_THRESH_SEL_BIT BIT(3) -#define SOC_LT_CHG_RECHARGE_THRESH_SEL_BIT BIT(2) -#define VBT_LT_CHG_RECHARGE_THRESH_SEL_BIT BIT(1) -#define IBT_LT_CHG_TERM_THRESH_SEL_BIT BIT(0) - -#define JEITA_EN_CFG 0x90 -#define JEITA_EN_HARDLIMIT_BIT BIT(4) -#define JEITA_EN_HOT_SL_FCV_BIT BIT(3) -#define JEITA_EN_COLD_SL_FCV_BIT BIT(2) -#define JEITA_EN_HOT_SL_CCC_BIT BIT(1) -#define JEITA_EN_COLD_SL_CCC_BIT BIT(0) - -#define INT_RT_STS 0x310 -#define TYPE_C_CHANGE_RT_STS_BIT BIT(7) -#define USBIN_ICL_CHANGE_RT_STS_BIT BIT(6) -#define USBIN_SOURCE_CHANGE_RT_STS_BIT BIT(5) -#define USBIN_PLUGIN_RT_STS_BIT BIT(4) -#define USBIN_OV_RT_STS_BIT BIT(3) -#define USBIN_UV_RT_STS_BIT BIT(2) -#define USBIN_LT_3P6V_RT_STS_BIT BIT(1) -#define USBIN_COLLAPSE_RT_STS_BIT BIT(0) - -#define OTG_CFG 0x153 -#define OTG_RESERVED_MASK GENMASK(7, 6) -#define DIS_OTG_ON_TLIM_BIT BIT(5) -#define QUICKSTART_OTG_FASTROLESWAP_BIT BIT(4) -#define INCREASE_DFP_TIME_BIT BIT(3) -#define ENABLE_OTG_IN_DEBUG_MODE_BIT BIT(2) -#define OTG_EN_SRC_CFG_BIT BIT(1) -#define CONCURRENT_MODE_CFG_BIT BIT(0) - -#define OTG_ENG_OTG_CFG 0x1C0 -#define ENG_BUCKBOOST_HALT1_8_MODE_BIT BIT(0) - -#define APSD_STATUS 0x307 -#define APSD_STATUS_7_BIT BIT(7) -#define HVDCP_CHECK_TIMEOUT_BIT BIT(6) -#define SLOW_PLUGIN_TIMEOUT_BIT BIT(5) -#define ENUMERATION_DONE_BIT BIT(4) -#define VADP_CHANGE_DONE_AFTER_AUTH_BIT BIT(3) -#define QC_AUTH_DONE_STATUS_BIT BIT(2) -#define QC_CHARGER_BIT BIT(1) -#define APSD_DTC_STATUS_DONE_BIT BIT(0) - -#define APSD_RESULT_STATUS 0x308 -#define ICL_OVERRIDE_LATCH_BIT BIT(7) -#define APSD_RESULT_STATUS_MASK GENMASK(6, 0) -#define QC_3P0_BIT BIT(6) -#define QC_2P0_BIT BIT(5) -#define FLOAT_CHARGER_BIT BIT(4) -#define DCP_CHARGER_BIT BIT(3) -#define CDP_CHARGER_BIT BIT(2) -#define OCP_CHARGER_BIT BIT(1) -#define SDP_CHARGER_BIT BIT(0) - -#define TYPE_C_STATUS_1 0x30B -#define UFP_TYPEC_MASK GENMASK(7, 5) -#define UFP_TYPEC_RDSTD_BIT BIT(7) -#define UFP_TYPEC_RD1P5_BIT BIT(6) -#define UFP_TYPEC_RD3P0_BIT BIT(5) -#define UFP_TYPEC_FMB_255K_BIT BIT(4) -#define UFP_TYPEC_FMB_301K_BIT BIT(3) -#define UFP_TYPEC_FMB_523K_BIT BIT(2) -#define UFP_TYPEC_FMB_619K_BIT BIT(1) -#define UFP_TYPEC_OPEN_OPEN_BIT BIT(0) - -#define TYPE_C_STATUS_2 0x30C -#define DFP_RA_OPEN_BIT BIT(7) -#define TIMER_STAGE_BIT BIT(6) -#define EXIT_UFP_MODE_BIT BIT(5) -#define EXIT_DFP_MODE_BIT BIT(4) -#define DFP_TYPEC_MASK GENMASK(3, 0) -#define DFP_RD_OPEN_BIT BIT(3) -#define DFP_RD_RA_VCONN_BIT BIT(2) -#define DFP_RD_RD_BIT BIT(1) -#define DFP_RA_RA_BIT BIT(0) - -#define TYPE_C_STATUS_3 0x30D -#define ENABLE_BANDGAP_BIT BIT(7) -#define U_USB_GND_NOVBUS_BIT BIT(6) -#define U_USB_FLOAT_NOVBUS_BIT BIT(5) -#define U_USB_GND_BIT BIT(4) -#define U_USB_FMB1_BIT BIT(3) -#define U_USB_FLOAT1_BIT BIT(2) -#define U_USB_FMB2_BIT BIT(1) -#define U_USB_FLOAT2_BIT BIT(0) - -#define TYPE_C_STATUS_4 0x30E -#define UFP_DFP_MODE_STATUS_BIT BIT(7) -#define TYPEC_VBUS_STATUS_BIT BIT(6) -#define TYPEC_VBUS_ERROR_STATUS_BIT BIT(5) -#define TYPEC_DEBOUNCE_DONE_STATUS_BIT BIT(4) -#define TYPEC_UFP_AUDIO_ADAPT_STATUS_BIT BIT(3) -#define TYPEC_VCONN_OVERCURR_STATUS_BIT BIT(2) -#define CC_ORIENTATION_BIT BIT(1) -#define CC_ATTACHED_BIT BIT(0) - -#define TYPE_C_STATUS_5 0x30F -#define TRY_SOURCE_FAILED_BIT BIT(6) -#define TRY_SINK_FAILED_BIT BIT(5) -#define TIMER_STAGE_2_BIT BIT(4) -#define TYPEC_LEGACY_CABLE_STATUS_BIT BIT(3) -#define TYPEC_NONCOMP_LEGACY_CABLE_STATUS_BIT BIT(2) -#define TYPEC_TRYSOURCE_DETECT_STATUS_BIT BIT(1) -#define TYPEC_TRYSINK_DETECT_STATUS_BIT BIT(0) - -#define CMD_APSD 0x341 -#define ICL_OVERRIDE_BIT BIT(1) -#define APSD_RERUN_BIT BIT(0) - -#define TYPE_C_CFG 0x358 -#define APSD_START_ON_CC_BIT BIT(7) -#define WAIT_FOR_APSD_BIT BIT(6) -#define FACTORY_MODE_DETECTION_EN_BIT BIT(5) -#define FACTORY_MODE_ICL_3A_4A_BIT BIT(4) -#define FACTORY_MODE_DIS_CHGING_CFG_BIT BIT(3) -#define SUSPEND_NON_COMPLIANT_CFG_BIT BIT(2) -#define VCONN_OC_CFG_BIT BIT(1) -#define TYPE_C_OR_U_USB_BIT BIT(0) - -#define TYPE_C_CFG_2 0x359 -#define TYPE_C_DFP_CURRSRC_MODE_BIT BIT(7) -#define DFP_CC_1P4V_OR_1P6V_BIT BIT(6) -#define VCONN_SOFTSTART_CFG_MASK GENMASK(5, 4) -#define EN_TRY_SOURCE_MODE_BIT BIT(3) -#define USB_FACTORY_MODE_ENABLE_BIT BIT(2) -#define TYPE_C_UFP_MODE_BIT BIT(1) -#define EN_80UA_180UA_CUR_SOURCE_BIT BIT(0) - -#define TYPE_C_CFG_3 0x35A -#define TVBUS_DEBOUNCE_BIT BIT(7) -#define TYPEC_LEGACY_CABLE_INT_EN_BIT BIT(6) -#define TYPEC_NONCOMPLIANT_LEGACY_CABLE_INT_EN_B BIT(5) -#define TYPEC_TRYSOURCE_DETECT_INT_EN_BIT BIT(4) -#define TYPEC_TRYSINK_DETECT_INT_EN_BIT BIT(3) -#define EN_TRYSINK_MODE_BIT BIT(2) -#define EN_LEGACY_CABLE_DETECTION_BIT BIT(1) -#define ALLOW_PD_DRING_UFP_TCCDB_BIT BIT(0) - -#define USBIN_OPTIONS_1_CFG 0x362 -#define CABLE_R_SEL_BIT BIT(7) -#define HVDCP_AUTH_ALG_EN_CFG_BIT BIT(6) -#define HVDCP_AUTONOMOUS_MODE_EN_CFG_BIT BIT(5) -#define INPUT_PRIORITY_BIT BIT(4) -#define AUTO_SRC_DETECT_BIT BIT(3) -#define HVDCP_EN_BIT BIT(2) -#define VADP_INCREMENT_VOLTAGE_LIMIT_BIT BIT(1) -#define VADP_TAPER_TIMER_EN_BIT BIT(0) - -#define USBIN_OPTIONS_2_CFG 0x363 -#define WIPWR_RST_EUD_CFG_BIT BIT(7) -#define SWITCHER_START_CFG_BIT BIT(6) -#define DCD_TIMEOUT_SEL_BIT BIT(5) -#define OCD_CURRENT_SEL_BIT BIT(4) -#define SLOW_PLUGIN_TIMER_EN_CFG_BIT BIT(3) -#define FLOAT_OPTIONS_MASK GENMASK(2, 0) -#define FLOAT_DIS_CHGING_CFG_BIT BIT(2) -#define SUSPEND_FLOAT_CFG_BIT BIT(1) -#define FORCE_FLOAT_SDP_CFG_BIT BIT(0) - -#define TAPER_TIMER_SEL_CFG 0x364 -#define TYPEC_SPARE_CFG_BIT BIT(7) -#define TYPEC_DRP_DFP_TIME_CFG_BIT BIT(5) -#define TAPER_TIMER_SEL_MASK GENMASK(1, 0) - -#define USBIN_LOAD_CFG 0x365 -#define USBIN_OV_CH_LOAD_OPTION_BIT BIT(7) -#define ICL_OVERRIDE_AFTER_APSD_BIT BIT(4) - -#define USBIN_ICL_OPTIONS 0x366 -#define CFG_USB3P0_SEL_BIT BIT(2) -#define USB51_MODE_BIT BIT(1) -#define USBIN_MODE_CHG_BIT BIT(0) - -#define TYPE_C_INTRPT_ENB_SOFTWARE_CTRL 0x368 -#define EXIT_SNK_BASED_ON_CC_BIT BIT(7) -#define VCONN_EN_ORIENTATION_BIT BIT(6) -#define TYPEC_VCONN_OVERCURR_INT_EN_BIT BIT(5) -#define VCONN_EN_SRC_BIT BIT(4) -#define VCONN_EN_VALUE_BIT BIT(3) -#define TYPEC_POWER_ROLE_CMD_MASK GENMASK(2, 0) -#define UFP_EN_CMD_BIT BIT(2) -#define DFP_EN_CMD_BIT BIT(1) -#define TYPEC_DISABLE_CMD_BIT BIT(0) - -#define USBIN_CURRENT_LIMIT_CFG 0x370 -#define USBIN_CURRENT_LIMIT_MASK GENMASK(7, 0) - -#define USBIN_AICL_OPTIONS_CFG 0x380 -#define SUSPEND_ON_COLLAPSE_USBIN_BIT BIT(7) -#define USBIN_AICL_HDC_EN_BIT BIT(6) -#define USBIN_AICL_START_AT_MAX_BIT BIT(5) -#define USBIN_AICL_RERUN_EN_BIT BIT(4) -#define USBIN_AICL_ADC_EN_BIT BIT(3) -#define USBIN_AICL_EN_BIT BIT(2) -#define USBIN_HV_COLLAPSE_RESPONSE_BIT BIT(1) -#define USBIN_LV_COLLAPSE_RESPONSE_BIT BIT(0) - -#define USBIN_5V_AICL_THRESHOLD_CFG 0x381 -#define USBIN_5V_AICL_THRESHOLD_CFG_MASK GENMASK(2, 0) - -#define USBIN_CONT_AICL_THRESHOLD_CFG 0x384 -#define USBIN_CONT_AICL_THRESHOLD_CFG_MASK GENMASK(5, 0) - -#define DC_ENG_SSUPPLY_CFG2 0x4C1 -#define ENG_SSUPPLY_IVREF_OTG_SS_MASK GENMASK(2, 0) -#define OTG_SS_SLOW 0x3 - -#define DCIN_AICL_REF_SEL_CFG 0x481 -#define DCIN_CONT_AICL_THRESHOLD_CFG_MASK GENMASK(5, 0) - -#define WI_PWR_OPTIONS 0x495 -#define CHG_OK_BIT BIT(7) -#define WIPWR_UVLO_IRQ_OPT_BIT BIT(6) -#define BUCK_HOLDOFF_ENABLE_BIT BIT(5) -#define CHG_OK_HW_SW_SELECT_BIT BIT(4) -#define WIPWR_RST_ENABLE_BIT BIT(3) -#define DCIN_WIPWR_IRQ_SELECT_BIT BIT(2) -#define AICL_SWITCH_ENABLE_BIT BIT(1) -#define ZIN_ICL_ENABLE_BIT BIT(0) - -#define ICL_STATUS 0x607 -#define INPUT_CURRENT_LIMIT_MASK GENMASK(7, 0) - -#define POWER_PATH_STATUS 0x60B -#define P_PATH_INPUT_SS_DONE_BIT BIT(7) -#define P_PATH_USBIN_SUSPEND_STS_BIT BIT(6) -#define P_PATH_DCIN_SUSPEND_STS_BIT BIT(5) -#define P_PATH_USE_USBIN_BIT BIT(4) -#define P_PATH_USE_DCIN_BIT BIT(3) -#define P_PATH_POWER_PATH_MASK GENMASK(2, 1) -#define P_PATH_VALID_INPUT_POWER_SOURCE_STS_BIT BIT(0) - -#define BARK_BITE_WDOG_PET 0x643 -#define BARK_BITE_WDOG_PET_BIT BIT(0) - -#define WD_CFG 0x651 -#define WATCHDOG_TRIGGER_AFP_EN_BIT BIT(7) -#define BARK_WDOG_INT_EN_BIT BIT(6) -#define BITE_WDOG_INT_EN_BIT BIT(5) -#define SFT_AFTER_WDOG_IRQ_MASK GENMASK(4, 3) -#define WDOG_IRQ_SFT_BIT BIT(2) -#define WDOG_TIMER_EN_ON_PLUGIN_BIT BIT(1) -#define WDOG_TIMER_EN_BIT BIT(0) - -#define SNARL_BARK_BITE_WD_CFG 0x653 -#define BITE_WDOG_DISABLE_CHARGING_CFG_BIT BIT(7) -#define SNARL_WDOG_TIMEOUT_MASK GENMASK(6, 4) -#define BARK_WDOG_TIMEOUT_MASK GENMASK(3, 2) -#define BITE_WDOG_TIMEOUT_MASK GENMASK(1, 0) - -#define AICL_RERUN_TIME_CFG 0x661 -#define AICL_RERUN_TIME_MASK GENMASK(1, 0) - -#define STAT_CFG 0x690 -#define STAT_SW_OVERRIDE_VALUE_BIT BIT(7) -#define STAT_SW_OVERRIDE_CFG_BIT BIT(6) -#define STAT_PARALLEL_OFF_DG_CFG_MASK GENMASK(5, 4) -#define STAT_POLARITY_CFG_BIT BIT(3) -#define STAT_PARALLEL_CFG_BIT BIT(2) -#define STAT_FUNCTION_CFG_BIT BIT(1) -#define STAT_IRQ_PULSING_EN_BIT BIT(0) - -#define SDP_CURRENT_UA 500000 -#define CDP_CURRENT_UA 1500000 -#define DCP_CURRENT_UA 1500000 -#define CURRENT_MAX_UA DCP_CURRENT_UA - -/* pmi8998 registers represent current in increments of 1/40th of an amp */ -#define CURRENT_SCALE_FACTOR 25000 -#define SMB2_FLOAT_VOLTAGE_MIN_UV 3487500 -#define SMB2_FLOAT_VOLTAGE_STEP_UV 7500 -/* clang-format on */ - -enum charger_status { - TRICKLE_CHARGE = 0, - PRE_CHARGE, - FAST_CHARGE, - FULLON_CHARGE, - TAPER_CHARGE, - TERMINATE_CHARGE, - INHIBIT_CHARGE, - DISABLE_CHARGE, -}; - -struct smb2_register { - u16 addr; - u8 mask; - u8 val; -}; - -/** - * struct smb2_chip - smb2 chip structure - * @dev: Device reference for power_supply - * @name: The platform device name - * @base: Base address for smb2 registers - * @regmap: Register map - * @batt_info: Battery data from DT - * @status_change_work: Worker to handle plug/unplug events - * @cable_irq: USB plugin IRQ - * @wakeup_enabled: If the cable IRQ will cause a wakeup - * @usb_in_i_chan: USB_IN current measurement channel - * @usb_in_v_chan: USB_IN voltage measurement channel - * @chg_psy: Charger power supply instance - */ -struct smb2_chip { - struct device *dev; - const char *name; - unsigned int base; - struct regmap *regmap; - struct power_supply_battery_info *batt_info; - - struct delayed_work status_change_work; - int cable_irq; - bool wakeup_enabled; - - struct iio_channel *usb_in_i_chan; - struct iio_channel *usb_in_v_chan; - - struct power_supply *chg_psy; -}; - -static enum power_supply_property smb2_properties[] = { - POWER_SUPPLY_PROP_MANUFACTURER, - POWER_SUPPLY_PROP_MODEL_NAME, - POWER_SUPPLY_PROP_CURRENT_MAX, - POWER_SUPPLY_PROP_CURRENT_NOW, - POWER_SUPPLY_PROP_VOLTAGE_NOW, - POWER_SUPPLY_PROP_STATUS, - POWER_SUPPLY_PROP_HEALTH, - POWER_SUPPLY_PROP_ONLINE, - POWER_SUPPLY_PROP_USB_TYPE, -}; - -static int smb2_get_prop_usb_online(struct smb2_chip *chip, int *val) -{ - unsigned int stat; - int rc; - - rc = regmap_read(chip->regmap, chip->base + POWER_PATH_STATUS, &stat); - if (rc < 0) { - dev_err(chip->dev, "Couldn't read power path status: %d\n", rc); - return rc; - } - - *val = (stat & P_PATH_USE_USBIN_BIT) && - (stat & P_PATH_VALID_INPUT_POWER_SOURCE_STS_BIT); - return 0; -} - -/* - * Qualcomm "automatic power source detection" aka APSD - * tells us what type of charger we're connected to. - */ -static int smb2_apsd_get_charger_type(struct smb2_chip *chip, int *val) -{ - unsigned int apsd_stat, stat; - int usb_online = 0; - int rc; - - rc = smb2_get_prop_usb_online(chip, &usb_online); - if (!usb_online) { - *val = POWER_SUPPLY_USB_TYPE_UNKNOWN; - return rc; - } - - rc = regmap_read(chip->regmap, chip->base + APSD_STATUS, &apsd_stat); - if (rc < 0) { - dev_err(chip->dev, "Failed to read apsd status, rc = %d", rc); - return rc; - } - if (!(apsd_stat & APSD_DTC_STATUS_DONE_BIT)) { - dev_dbg(chip->dev, "Apsd not ready"); - return -EAGAIN; - } - - rc = regmap_read(chip->regmap, chip->base + APSD_RESULT_STATUS, &stat); - if (rc < 0) { - dev_err(chip->dev, "Failed to read apsd result, rc = %d", rc); - return rc; - } - - stat &= APSD_RESULT_STATUS_MASK; - - if (stat & CDP_CHARGER_BIT) - *val = POWER_SUPPLY_USB_TYPE_CDP; - else if (stat & (DCP_CHARGER_BIT | OCP_CHARGER_BIT | FLOAT_CHARGER_BIT)) - *val = POWER_SUPPLY_USB_TYPE_DCP; - else /* SDP_CHARGER_BIT (or others) */ - *val = POWER_SUPPLY_USB_TYPE_SDP; - - return 0; -} - -static int smb2_get_prop_status(struct smb2_chip *chip, int *val) -{ - unsigned char stat[2]; - int usb_online = 0; - int rc; - - rc = smb2_get_prop_usb_online(chip, &usb_online); - if (!usb_online) { - *val = POWER_SUPPLY_STATUS_DISCHARGING; - return rc; - } - - rc = regmap_bulk_read(chip->regmap, - chip->base + BATTERY_CHARGER_STATUS_1, &stat, 2); - if (rc < 0) { - dev_err(chip->dev, "Failed to read charging status ret=%d\n", - rc); - return rc; - } - - if (stat[1] & CHARGER_ERROR_STATUS_BAT_OV_BIT) { - *val = POWER_SUPPLY_STATUS_NOT_CHARGING; - return 0; - } - - stat[0] = stat[0] & BATTERY_CHARGER_STATUS_MASK; - - switch (stat[0]) { - case TRICKLE_CHARGE: - case PRE_CHARGE: - case FAST_CHARGE: - case FULLON_CHARGE: - case TAPER_CHARGE: - *val = POWER_SUPPLY_STATUS_CHARGING; - return rc; - case DISABLE_CHARGE: - *val = POWER_SUPPLY_STATUS_NOT_CHARGING; - return rc; - case TERMINATE_CHARGE: - case INHIBIT_CHARGE: - *val = POWER_SUPPLY_STATUS_FULL; - return rc; - default: - *val = POWER_SUPPLY_STATUS_UNKNOWN; - return rc; - } -} - -static inline int smb2_get_current_limit(struct smb2_chip *chip, - unsigned int *val) -{ - int rc = regmap_read(chip->regmap, chip->base + ICL_STATUS, val); - - if (rc >= 0) - *val *= CURRENT_SCALE_FACTOR; - return rc; -} - -static int smb2_set_current_limit(struct smb2_chip *chip, unsigned int val) -{ - unsigned char val_raw; - - if (val > 4800000) { - dev_err(chip->dev, - "Can't set current limit higher than 4800000uA"); - return -EINVAL; - } - val_raw = val / CURRENT_SCALE_FACTOR; - - return regmap_write(chip->regmap, chip->base + USBIN_CURRENT_LIMIT_CFG, - val_raw); -} - -static void smb2_status_change_work(struct work_struct *work) -{ - unsigned int charger_type, current_ua; - int usb_online = 0; - int count, rc; - struct smb2_chip *chip; - - chip = container_of(work, struct smb2_chip, status_change_work.work); - - smb2_get_prop_usb_online(chip, &usb_online); - if (!usb_online) - return; - - for (count = 0; count < 3; count++) { - dev_dbg(chip->dev, "get charger type retry %d\n", count); - rc = smb2_apsd_get_charger_type(chip, &charger_type); - if (rc != -EAGAIN) - break; - msleep(100); - } - - if (rc < 0 && rc != -EAGAIN) { - dev_err(chip->dev, "get charger type failed: %d\n", rc); - return; - } - - if (rc < 0) { - rc = regmap_update_bits(chip->regmap, chip->base + CMD_APSD, - APSD_RERUN_BIT, APSD_RERUN_BIT); - schedule_delayed_work(&chip->status_change_work, - msecs_to_jiffies(1000)); - dev_dbg(chip->dev, "get charger type failed, rerun apsd\n"); - return; - } - - switch (charger_type) { - case POWER_SUPPLY_USB_TYPE_CDP: - current_ua = CDP_CURRENT_UA; - break; - case POWER_SUPPLY_USB_TYPE_DCP: - current_ua = DCP_CURRENT_UA; - break; - case POWER_SUPPLY_USB_TYPE_SDP: - default: - current_ua = SDP_CURRENT_UA; - break; - } - - smb2_set_current_limit(chip, current_ua); - power_supply_changed(chip->chg_psy); -} - -static int smb2_get_iio_chan(struct smb2_chip *chip, struct iio_channel *chan, - int *val) -{ - int rc; - union power_supply_propval status; - - rc = power_supply_get_property(chip->chg_psy, POWER_SUPPLY_PROP_STATUS, - &status); - if (rc < 0 || status.intval != POWER_SUPPLY_STATUS_CHARGING) { - *val = 0; - return 0; - } - - if (IS_ERR(chan)) { - dev_err(chip->dev, "Failed to chan, err = %li", PTR_ERR(chan)); - return PTR_ERR(chan); - } - - return iio_read_channel_processed(chan, val); -} - -static int smb2_get_prop_health(struct smb2_chip *chip, int *val) -{ - int rc; - unsigned int stat; - - rc = regmap_read(chip->regmap, chip->base + BATTERY_CHARGER_STATUS_2, - &stat); - if (rc < 0) { - dev_err(chip->dev, "Couldn't read charger status rc=%d\n", rc); - return rc; - } - - if (stat & CHARGER_ERROR_STATUS_BAT_OV_BIT) - *val = POWER_SUPPLY_HEALTH_OVERVOLTAGE; - else if (stat & BAT_TEMP_STATUS_TOO_COLD_BIT) - *val = POWER_SUPPLY_HEALTH_COLD; - else if (stat & BAT_TEMP_STATUS_TOO_HOT_BIT) - *val = POWER_SUPPLY_HEALTH_OVERHEAT; - else if (stat & BAT_TEMP_STATUS_COLD_SOFT_LIMIT_BIT) - *val = POWER_SUPPLY_HEALTH_COOL; - else if (stat & BAT_TEMP_STATUS_HOT_SOFT_LIMIT_BIT) - *val = POWER_SUPPLY_HEALTH_WARM; - else - *val = POWER_SUPPLY_HEALTH_GOOD; - - return 0; -} - -static int smb2_get_property(struct power_supply *psy, - enum power_supply_property psp, - union power_supply_propval *val) -{ - struct smb2_chip *chip = power_supply_get_drvdata(psy); - - switch (psp) { - case POWER_SUPPLY_PROP_MANUFACTURER: - val->strval = "Qualcomm"; - return 0; - case POWER_SUPPLY_PROP_MODEL_NAME: - val->strval = chip->name; - return 0; - case POWER_SUPPLY_PROP_CURRENT_MAX: - return smb2_get_current_limit(chip, &val->intval); - case POWER_SUPPLY_PROP_CURRENT_NOW: - return smb2_get_iio_chan(chip, chip->usb_in_i_chan, - &val->intval); - case POWER_SUPPLY_PROP_VOLTAGE_NOW: - return smb2_get_iio_chan(chip, chip->usb_in_v_chan, - &val->intval); - case POWER_SUPPLY_PROP_ONLINE: - return smb2_get_prop_usb_online(chip, &val->intval); - case POWER_SUPPLY_PROP_STATUS: - return smb2_get_prop_status(chip, &val->intval); - case POWER_SUPPLY_PROP_HEALTH: - return smb2_get_prop_health(chip, &val->intval); - case POWER_SUPPLY_PROP_USB_TYPE: - return smb2_apsd_get_charger_type(chip, &val->intval); - default: - dev_err(chip->dev, "invalid property: %d\n", psp); - return -EINVAL; - } -} - -static int smb2_set_property(struct power_supply *psy, - enum power_supply_property psp, - const union power_supply_propval *val) -{ - struct smb2_chip *chip = power_supply_get_drvdata(psy); - - switch (psp) { - case POWER_SUPPLY_PROP_CURRENT_MAX: - return smb2_set_current_limit(chip, val->intval); - default: - dev_err(chip->dev, "No setter for property: %d\n", psp); - return -EINVAL; - } -} - -static int smb2_property_is_writable(struct power_supply *psy, - enum power_supply_property psp) -{ - switch (psp) { - case POWER_SUPPLY_PROP_CURRENT_MAX: - return 1; - default: - return 0; - } -} - -static irqreturn_t smb2_handle_batt_overvoltage(int irq, void *data) -{ - struct smb2_chip *chip = data; - unsigned int status; - int rc; - - rc = regmap_read(chip->regmap, - chip->base + BATTERY_CHARGER_STATUS_2, &status); - if (rc < 0) { - dev_err(chip->dev, "Couldn't read charger status: %d\n", rc); - return IRQ_HANDLED; - } - - if (status & CHARGER_ERROR_STATUS_BAT_OV_BIT) { - /* The hardware stops charging automatically */ - dev_err(chip->dev, "battery overvoltage detected\n"); - } - - power_supply_changed(chip->chg_psy); - - return IRQ_HANDLED; -} - -static irqreturn_t smb2_handle_usb_plugin(int irq, void *data) -{ - struct smb2_chip *chip = data; - - power_supply_changed(chip->chg_psy); - - schedule_delayed_work(&chip->status_change_work, - msecs_to_jiffies(1500)); - - return IRQ_HANDLED; -} - -static irqreturn_t smb2_handle_usb_icl_change(int irq, void *data) -{ - struct smb2_chip *chip = data; - - power_supply_changed(chip->chg_psy); - - return IRQ_HANDLED; -} - -static irqreturn_t smb2_handle_wdog_bark(int irq, void *data) -{ - struct smb2_chip *chip = data; - int rc; - - power_supply_changed(chip->chg_psy); - - rc = regmap_write(chip->regmap, chip->base + BARK_BITE_WDOG_PET, - BARK_BITE_WDOG_PET_BIT); - if (rc < 0) - dev_err(chip->dev, "Couldn't pet the dog rc=%d\n", rc); - - return IRQ_HANDLED; -} - -static const struct power_supply_desc smb2_psy_desc = { - .name = "pmi8998_charger", - .type = POWER_SUPPLY_TYPE_USB, - .usb_types = BIT(POWER_SUPPLY_USB_TYPE_SDP) | - BIT(POWER_SUPPLY_USB_TYPE_CDP) | - BIT(POWER_SUPPLY_USB_TYPE_DCP) | - BIT(POWER_SUPPLY_USB_TYPE_UNKNOWN), - .properties = smb2_properties, - .num_properties = ARRAY_SIZE(smb2_properties), - .get_property = smb2_get_property, - .set_property = smb2_set_property, - .property_is_writeable = smb2_property_is_writable, -}; - -/* Init sequence derived from vendor downstream driver */ -static const struct smb2_register smb2_init_seq[] = { - { .addr = AICL_RERUN_TIME_CFG, .mask = AICL_RERUN_TIME_MASK, .val = 0 }, - /* - * By default configure us as an upstream facing port - * FIXME: This will be handled by the type-c driver - */ - { .addr = TYPE_C_INTRPT_ENB_SOFTWARE_CTRL, - .mask = TYPEC_POWER_ROLE_CMD_MASK | VCONN_EN_SRC_BIT | - VCONN_EN_VALUE_BIT, - .val = VCONN_EN_SRC_BIT }, - /* - * Disable Type-C factory mode and stay in Attached.SRC state when VCONN - * over-current happens - */ - { .addr = TYPE_C_CFG, - .mask = FACTORY_MODE_DETECTION_EN_BIT | VCONN_OC_CFG_BIT, - .val = 0 }, - /* Configure VBUS for software control */ - { .addr = OTG_CFG, .mask = OTG_EN_SRC_CFG_BIT, .val = 0 }, - /* - * Use VBAT to determine the recharge threshold when battery is full - * rather than the state of charge. - */ - { .addr = FG_UPDATE_CFG_2_SEL, - .mask = SOC_LT_CHG_RECHARGE_THRESH_SEL_BIT | - VBT_LT_CHG_RECHARGE_THRESH_SEL_BIT, - .val = VBT_LT_CHG_RECHARGE_THRESH_SEL_BIT }, - /* Enable charging */ - { .addr = USBIN_OPTIONS_1_CFG, .mask = HVDCP_EN_BIT, .val = 0 }, - { .addr = CHARGING_ENABLE_CMD, - .mask = CHARGING_ENABLE_CMD_BIT, - .val = CHARGING_ENABLE_CMD_BIT }, - /* - * Match downstream defaults - * CHG_EN_SRC_BIT - charger enable is controlled by software - * CHG_EN_POLARITY_BIT - polarity of charge enable pin when in HW control - * pulled low on OnePlus 6 and SHIFT6mq - * PRETOFAST_TRANSITION_CFG_BIT - - * BAT_OV_ECC_BIT - - * I_TERM_BIT - Current termination ?? 0 = enabled - * AUTO_RECHG_BIT - Enable automatic recharge when battery is full - * 0 = enabled - * EN_ANALOG_DROP_IN_VBATT_BIT - * CHARGER_INHIBIT_BIT - Inhibit charging based on battery voltage - * instead of ?? - */ - { .addr = CHGR_CFG2, - .mask = CHG_EN_SRC_BIT | CHG_EN_POLARITY_BIT | - PRETOFAST_TRANSITION_CFG_BIT | BAT_OV_ECC_BIT | I_TERM_BIT | - AUTO_RECHG_BIT | EN_ANALOG_DROP_IN_VBATT_BIT | - CHARGER_INHIBIT_BIT, - .val = CHARGER_INHIBIT_BIT }, - /* STAT pin software override, match downstream. Parallell charging? */ - { .addr = STAT_CFG, - .mask = STAT_SW_OVERRIDE_CFG_BIT, - .val = STAT_SW_OVERRIDE_CFG_BIT }, - /* Set the default SDP charger type to a 500ma USB 2.0 port */ - { .addr = USBIN_ICL_OPTIONS, - .mask = USB51_MODE_BIT | USBIN_MODE_CHG_BIT, - .val = USB51_MODE_BIT }, - /* Disable watchdog */ - { .addr = SNARL_BARK_BITE_WD_CFG, .mask = 0xff, .val = 0 }, - { .addr = WD_CFG, - .mask = WATCHDOG_TRIGGER_AFP_EN_BIT | WDOG_TIMER_EN_ON_PLUGIN_BIT | - BARK_WDOG_INT_EN_BIT, - .val = 0 }, - /* These bits aren't documented anywhere */ - { .addr = USBIN_5V_AICL_THRESHOLD_CFG, - .mask = USBIN_5V_AICL_THRESHOLD_CFG_MASK, - .val = 0x3 }, - { .addr = USBIN_CONT_AICL_THRESHOLD_CFG, - .mask = USBIN_CONT_AICL_THRESHOLD_CFG_MASK, - .val = 0x3 }, - /* - * Enable Automatic Input Current Limit, this will slowly ramp up the current - * When connected to a wall charger, and automatically stop when it detects - * the charger current limit (voltage drop?) or it reaches the programmed limit. - */ - { .addr = USBIN_AICL_OPTIONS_CFG, - .mask = USBIN_AICL_START_AT_MAX_BIT | USBIN_AICL_ADC_EN_BIT | - USBIN_AICL_EN_BIT | SUSPEND_ON_COLLAPSE_USBIN_BIT | - USBIN_HV_COLLAPSE_RESPONSE_BIT | - USBIN_LV_COLLAPSE_RESPONSE_BIT, - .val = USBIN_HV_COLLAPSE_RESPONSE_BIT | - USBIN_LV_COLLAPSE_RESPONSE_BIT | USBIN_AICL_EN_BIT }, - /* - * Set pre charge current to default, the OnePlus 6 bootloader - * sets this very conservatively. - */ - { .addr = PRE_CHARGE_CURRENT_CFG, - .mask = PRE_CHARGE_CURRENT_SETTING_MASK, - .val = 500000 / CURRENT_SCALE_FACTOR }, - /* - * This overrides all of the current limit options exposed to userspace - * and prevents the device from pulling more than ~1A. This is done - * to minimise potential fire hazard risks. - */ - { .addr = FAST_CHARGE_CURRENT_CFG, - .mask = FAST_CHARGE_CURRENT_SETTING_MASK, - .val = 1000000 / CURRENT_SCALE_FACTOR }, -}; - -static int smb2_init_hw(struct smb2_chip *chip) -{ - int rc, i; - - for (i = 0; i < ARRAY_SIZE(smb2_init_seq); i++) { - dev_dbg(chip->dev, "%d: Writing 0x%02x to 0x%02x\n", i, - smb2_init_seq[i].val, smb2_init_seq[i].addr); - rc = regmap_update_bits(chip->regmap, - chip->base + smb2_init_seq[i].addr, - smb2_init_seq[i].mask, - smb2_init_seq[i].val); - if (rc < 0) - return dev_err_probe(chip->dev, rc, - "%s: init command %d failed\n", - __func__, i); - } - - return 0; -} - -static int smb2_init_irq(struct smb2_chip *chip, int *irq, const char *name, - irqreturn_t (*handler)(int irq, void *data)) -{ - int irqnum; - int rc; - - irqnum = platform_get_irq_byname(to_platform_device(chip->dev), name); - if (irqnum < 0) - return irqnum; - - rc = devm_request_threaded_irq(chip->dev, irqnum, NULL, handler, - IRQF_ONESHOT, name, chip); - if (rc < 0) - return dev_err_probe(chip->dev, rc, "Couldn't request irq %s\n", - name); - - if (irq) - *irq = irqnum; - - return 0; -} - -static int smb2_probe(struct platform_device *pdev) -{ - struct power_supply_config supply_config = {}; - struct power_supply_desc *desc; - struct smb2_chip *chip; - int rc, irq; - - chip = devm_kzalloc(&pdev->dev, sizeof(*chip), GFP_KERNEL); - if (!chip) - return -ENOMEM; - - chip->dev = &pdev->dev; - chip->name = pdev->name; - - chip->regmap = dev_get_regmap(pdev->dev.parent, NULL); - if (!chip->regmap) - return dev_err_probe(chip->dev, -ENODEV, - "failed to locate the regmap\n"); - - rc = device_property_read_u32(chip->dev, "reg", &chip->base); - if (rc < 0) - return dev_err_probe(chip->dev, rc, - "Couldn't read base address\n"); - - chip->usb_in_v_chan = devm_iio_channel_get(chip->dev, "usbin_v"); - if (IS_ERR(chip->usb_in_v_chan)) - return dev_err_probe(chip->dev, PTR_ERR(chip->usb_in_v_chan), - "Couldn't get usbin_v IIO channel\n"); - - chip->usb_in_i_chan = devm_iio_channel_get(chip->dev, "usbin_i"); - if (IS_ERR(chip->usb_in_i_chan)) { - return dev_err_probe(chip->dev, PTR_ERR(chip->usb_in_i_chan), - "Couldn't get usbin_i IIO channel\n"); - } - - rc = smb2_init_hw(chip); - if (rc < 0) - return rc; - - supply_config.drv_data = chip; - supply_config.of_node = pdev->dev.of_node; - - desc = devm_kzalloc(chip->dev, sizeof(smb2_psy_desc), GFP_KERNEL); - if (!desc) - return -ENOMEM; - memcpy(desc, &smb2_psy_desc, sizeof(smb2_psy_desc)); - desc->name = - devm_kasprintf(chip->dev, GFP_KERNEL, "%s-charger", - (const char *)device_get_match_data(chip->dev)); - if (!desc->name) - return -ENOMEM; - - chip->chg_psy = - devm_power_supply_register(chip->dev, desc, &supply_config); - if (IS_ERR(chip->chg_psy)) - return dev_err_probe(chip->dev, PTR_ERR(chip->chg_psy), - "failed to register power supply\n"); - - rc = power_supply_get_battery_info(chip->chg_psy, &chip->batt_info); - if (rc) - return dev_err_probe(chip->dev, rc, - "Failed to get battery info\n"); - - rc = devm_delayed_work_autocancel(chip->dev, &chip->status_change_work, - smb2_status_change_work); - if (rc) - return dev_err_probe(chip->dev, rc, - "Failed to init status change work\n"); - - rc = (chip->batt_info->voltage_max_design_uv - - SMB2_FLOAT_VOLTAGE_MIN_UV) / SMB2_FLOAT_VOLTAGE_STEP_UV; - rc = regmap_update_bits(chip->regmap, chip->base + FLOAT_VOLTAGE_CFG, - FLOAT_VOLTAGE_SETTING_MASK, rc); - if (rc < 0) - return dev_err_probe(chip->dev, rc, "Couldn't set vbat max\n"); - - rc = smb2_init_irq(chip, &irq, "bat-ov", smb2_handle_batt_overvoltage); - if (rc < 0) - return rc; - - rc = smb2_init_irq(chip, &chip->cable_irq, "usb-plugin", - smb2_handle_usb_plugin); - if (rc < 0) - return rc; - - rc = smb2_init_irq(chip, &irq, "usbin-icl-change", - smb2_handle_usb_icl_change); - if (rc < 0) - return rc; - rc = smb2_init_irq(chip, &irq, "wdog-bark", smb2_handle_wdog_bark); - if (rc < 0) - return rc; - - rc = dev_pm_set_wake_irq(chip->dev, chip->cable_irq); - if (rc < 0) - return dev_err_probe(chip->dev, rc, "Couldn't set wake irq\n"); - - platform_set_drvdata(pdev, chip); - - /* Initialise charger state */ - schedule_delayed_work(&chip->status_change_work, 0); - - return 0; -} - -static const struct of_device_id smb2_match_id_table[] = { - { .compatible = "qcom,pmi8998-charger", .data = "pmi8998" }, - { .compatible = "qcom,pm660-charger", .data = "pm660" }, - { /* sentinal */ } -}; -MODULE_DEVICE_TABLE(of, smb2_match_id_table); - -static struct platform_driver qcom_spmi_smb2 = { - .probe = smb2_probe, - .driver = { - .name = "qcom-pmi8998/pm660-charger", - .of_match_table = smb2_match_id_table, - }, -}; - -module_platform_driver(qcom_spmi_smb2); - -MODULE_AUTHOR("Caleb Connolly "); -MODULE_DESCRIPTION("Qualcomm SMB2 Charger Driver"); -MODULE_LICENSE("GPL"); diff --git a/drivers/power/supply/qcom_smbx.c b/drivers/power/supply/qcom_smbx.c new file mode 100644 index 000000000000..099d1385ea12 --- /dev/null +++ b/drivers/power/supply/qcom_smbx.c @@ -0,0 +1,1108 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2016-2019 The Linux Foundation. All rights reserved. + * Copyright (c) 2023, Linaro Ltd. + * Author: Caleb Connolly + * + * This driver is for the switch-mode battery charger and boost + * hardware found in pmi8998 and related PMICs. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* clang-format off */ +#define BATTERY_CHARGER_STATUS_1 0x06 +#define BVR_INITIAL_RAMP_BIT BIT(7) +#define CC_SOFT_TERMINATE_BIT BIT(6) +#define STEP_CHARGING_STATUS_SHIFT 3 +#define STEP_CHARGING_STATUS_MASK GENMASK(5, 3) +#define BATTERY_CHARGER_STATUS_MASK GENMASK(2, 0) + +#define BATTERY_CHARGER_STATUS_2 0x07 +#define INPUT_CURRENT_LIMITED_BIT BIT(7) +#define CHARGER_ERROR_STATUS_SFT_EXPIRE_BIT BIT(6) +#define CHARGER_ERROR_STATUS_BAT_OV_BIT BIT(5) +#define CHARGER_ERROR_STATUS_BAT_TERM_MISSING_BIT BIT(4) +#define BAT_TEMP_STATUS_MASK GENMASK(3, 0) +#define BAT_TEMP_STATUS_SOFT_LIMIT_MASK GENMASK(3, 2) +#define BAT_TEMP_STATUS_HOT_SOFT_LIMIT_BIT BIT(3) +#define BAT_TEMP_STATUS_COLD_SOFT_LIMIT_BIT BIT(2) +#define BAT_TEMP_STATUS_TOO_HOT_BIT BIT(1) +#define BAT_TEMP_STATUS_TOO_COLD_BIT BIT(0) + +#define BATTERY_CHARGER_STATUS_4 0x0A +#define CHARGE_CURRENT_POST_JEITA_MASK GENMASK(7, 0) + +#define BATTERY_CHARGER_STATUS_7 0x0D +#define ENABLE_TRICKLE_BIT BIT(7) +#define ENABLE_PRE_CHARGING_BIT BIT(6) +#define ENABLE_FAST_CHARGING_BIT BIT(5) +#define ENABLE_FULLON_MODE_BIT BIT(4) +#define TOO_COLD_ADC_BIT BIT(3) +#define TOO_HOT_ADC_BIT BIT(2) +#define HOT_SL_ADC_BIT BIT(1) +#define COLD_SL_ADC_BIT BIT(0) + +#define CHARGING_ENABLE_CMD 0x42 +#define CHARGING_ENABLE_CMD_BIT BIT(0) + +#define CHGR_CFG2 0x51 +#define CHG_EN_SRC_BIT BIT(7) +#define CHG_EN_POLARITY_BIT BIT(6) +#define PRETOFAST_TRANSITION_CFG_BIT BIT(5) +#define BAT_OV_ECC_BIT BIT(4) +#define I_TERM_BIT BIT(3) +#define AUTO_RECHG_BIT BIT(2) +#define EN_ANALOG_DROP_IN_VBATT_BIT BIT(1) +#define CHARGER_INHIBIT_BIT BIT(0) + +#define PRE_CHARGE_CURRENT_CFG 0x60 +#define PRE_CHARGE_CURRENT_SETTING_MASK GENMASK(5, 0) + +#define FAST_CHARGE_CURRENT_CFG 0x61 +#define FAST_CHARGE_CURRENT_SETTING_MASK GENMASK(7, 0) + +#define FLOAT_VOLTAGE_CFG 0x70 +#define FLOAT_VOLTAGE_SETTING_MASK GENMASK(7, 0) + +#define FG_UPDATE_CFG_2_SEL 0x7D +#define SOC_LT_OTG_THRESH_SEL_BIT BIT(3) +#define SOC_LT_CHG_RECHARGE_THRESH_SEL_BIT BIT(2) +#define VBT_LT_CHG_RECHARGE_THRESH_SEL_BIT BIT(1) +#define IBT_LT_CHG_TERM_THRESH_SEL_BIT BIT(0) + +#define JEITA_EN_CFG 0x90 +#define JEITA_EN_HARDLIMIT_BIT BIT(4) +#define JEITA_EN_HOT_SL_FCV_BIT BIT(3) +#define JEITA_EN_COLD_SL_FCV_BIT BIT(2) +#define JEITA_EN_HOT_SL_CCC_BIT BIT(1) +#define JEITA_EN_COLD_SL_CCC_BIT BIT(0) + +#define INT_RT_STS 0x310 +#define TYPE_C_CHANGE_RT_STS_BIT BIT(7) +#define USBIN_ICL_CHANGE_RT_STS_BIT BIT(6) +#define USBIN_SOURCE_CHANGE_RT_STS_BIT BIT(5) +#define USBIN_PLUGIN_RT_STS_BIT BIT(4) +#define USBIN_OV_RT_STS_BIT BIT(3) +#define USBIN_UV_RT_STS_BIT BIT(2) +#define USBIN_LT_3P6V_RT_STS_BIT BIT(1) +#define USBIN_COLLAPSE_RT_STS_BIT BIT(0) + +#define OTG_CFG 0x153 +#define OTG_RESERVED_MASK GENMASK(7, 6) +#define DIS_OTG_ON_TLIM_BIT BIT(5) +#define QUICKSTART_OTG_FASTROLESWAP_BIT BIT(4) +#define INCREASE_DFP_TIME_BIT BIT(3) +#define ENABLE_OTG_IN_DEBUG_MODE_BIT BIT(2) +#define OTG_EN_SRC_CFG_BIT BIT(1) +#define CONCURRENT_MODE_CFG_BIT BIT(0) + +#define OTG_ENG_OTG_CFG 0x1C0 +#define ENG_BUCKBOOST_HALT1_8_MODE_BIT BIT(0) + +#define APSD_STATUS 0x307 +#define APSD_STATUS_7_BIT BIT(7) +#define HVDCP_CHECK_TIMEOUT_BIT BIT(6) +#define SLOW_PLUGIN_TIMEOUT_BIT BIT(5) +#define ENUMERATION_DONE_BIT BIT(4) +#define VADP_CHANGE_DONE_AFTER_AUTH_BIT BIT(3) +#define QC_AUTH_DONE_STATUS_BIT BIT(2) +#define QC_CHARGER_BIT BIT(1) +#define APSD_DTC_STATUS_DONE_BIT BIT(0) + +#define APSD_RESULT_STATUS 0x308 +#define ICL_OVERRIDE_LATCH_BIT BIT(7) +#define APSD_RESULT_STATUS_MASK GENMASK(6, 0) +#define QC_3P0_BIT BIT(6) +#define QC_2P0_BIT BIT(5) +#define FLOAT_CHARGER_BIT BIT(4) +#define DCP_CHARGER_BIT BIT(3) +#define CDP_CHARGER_BIT BIT(2) +#define OCP_CHARGER_BIT BIT(1) +#define SDP_CHARGER_BIT BIT(0) + +#define TYPE_C_STATUS_1 0x30B +#define UFP_TYPEC_MASK GENMASK(7, 5) +#define UFP_TYPEC_RDSTD_BIT BIT(7) +#define UFP_TYPEC_RD1P5_BIT BIT(6) +#define UFP_TYPEC_RD3P0_BIT BIT(5) +#define UFP_TYPEC_FMB_255K_BIT BIT(4) +#define UFP_TYPEC_FMB_301K_BIT BIT(3) +#define UFP_TYPEC_FMB_523K_BIT BIT(2) +#define UFP_TYPEC_FMB_619K_BIT BIT(1) +#define UFP_TYPEC_OPEN_OPEN_BIT BIT(0) + +#define TYPE_C_STATUS_2 0x30C +#define DFP_RA_OPEN_BIT BIT(7) +#define TIMER_STAGE_BIT BIT(6) +#define EXIT_UFP_MODE_BIT BIT(5) +#define EXIT_DFP_MODE_BIT BIT(4) +#define DFP_TYPEC_MASK GENMASK(3, 0) +#define DFP_RD_OPEN_BIT BIT(3) +#define DFP_RD_RA_VCONN_BIT BIT(2) +#define DFP_RD_RD_BIT BIT(1) +#define DFP_RA_RA_BIT BIT(0) + +#define TYPE_C_STATUS_3 0x30D +#define ENABLE_BANDGAP_BIT BIT(7) +#define U_USB_GND_NOVBUS_BIT BIT(6) +#define U_USB_FLOAT_NOVBUS_BIT BIT(5) +#define U_USB_GND_BIT BIT(4) +#define U_USB_FMB1_BIT BIT(3) +#define U_USB_FLOAT1_BIT BIT(2) +#define U_USB_FMB2_BIT BIT(1) +#define U_USB_FLOAT2_BIT BIT(0) + +#define TYPE_C_STATUS_4 0x30E +#define UFP_DFP_MODE_STATUS_BIT BIT(7) +#define TYPEC_VBUS_STATUS_BIT BIT(6) +#define TYPEC_VBUS_ERROR_STATUS_BIT BIT(5) +#define TYPEC_DEBOUNCE_DONE_STATUS_BIT BIT(4) +#define TYPEC_UFP_AUDIO_ADAPT_STATUS_BIT BIT(3) +#define TYPEC_VCONN_OVERCURR_STATUS_BIT BIT(2) +#define CC_ORIENTATION_BIT BIT(1) +#define CC_ATTACHED_BIT BIT(0) + +#define TYPE_C_STATUS_5 0x30F +#define TRY_SOURCE_FAILED_BIT BIT(6) +#define TRY_SINK_FAILED_BIT BIT(5) +#define TIMER_STAGE_2_BIT BIT(4) +#define TYPEC_LEGACY_CABLE_STATUS_BIT BIT(3) +#define TYPEC_NONCOMP_LEGACY_CABLE_STATUS_BIT BIT(2) +#define TYPEC_TRYSOURCE_DETECT_STATUS_BIT BIT(1) +#define TYPEC_TRYSINK_DETECT_STATUS_BIT BIT(0) + +#define CMD_APSD 0x341 +#define ICL_OVERRIDE_BIT BIT(1) +#define APSD_RERUN_BIT BIT(0) + +#define TYPE_C_CFG 0x358 +#define APSD_START_ON_CC_BIT BIT(7) +#define WAIT_FOR_APSD_BIT BIT(6) +#define FACTORY_MODE_DETECTION_EN_BIT BIT(5) +#define FACTORY_MODE_ICL_3A_4A_BIT BIT(4) +#define FACTORY_MODE_DIS_CHGING_CFG_BIT BIT(3) +#define SUSPEND_NON_COMPLIANT_CFG_BIT BIT(2) +#define VCONN_OC_CFG_BIT BIT(1) +#define TYPE_C_OR_U_USB_BIT BIT(0) + +#define TYPE_C_CFG_2 0x359 +#define TYPE_C_DFP_CURRSRC_MODE_BIT BIT(7) +#define DFP_CC_1P4V_OR_1P6V_BIT BIT(6) +#define VCONN_SOFTSTART_CFG_MASK GENMASK(5, 4) +#define EN_TRY_SOURCE_MODE_BIT BIT(3) +#define USB_FACTORY_MODE_ENABLE_BIT BIT(2) +#define TYPE_C_UFP_MODE_BIT BIT(1) +#define EN_80UA_180UA_CUR_SOURCE_BIT BIT(0) + +#define TYPE_C_CFG_3 0x35A +#define TVBUS_DEBOUNCE_BIT BIT(7) +#define TYPEC_LEGACY_CABLE_INT_EN_BIT BIT(6) +#define TYPEC_NONCOMPLIANT_LEGACY_CABLE_INT_EN_B BIT(5) +#define TYPEC_TRYSOURCE_DETECT_INT_EN_BIT BIT(4) +#define TYPEC_TRYSINK_DETECT_INT_EN_BIT BIT(3) +#define EN_TRYSINK_MODE_BIT BIT(2) +#define EN_LEGACY_CABLE_DETECTION_BIT BIT(1) +#define ALLOW_PD_DRING_UFP_TCCDB_BIT BIT(0) + +#define USBIN_OPTIONS_1_CFG 0x362 +#define CABLE_R_SEL_BIT BIT(7) +#define HVDCP_AUTH_ALG_EN_CFG_BIT BIT(6) +#define HVDCP_AUTONOMOUS_MODE_EN_CFG_BIT BIT(5) +#define INPUT_PRIORITY_BIT BIT(4) +#define AUTO_SRC_DETECT_BIT BIT(3) +#define HVDCP_EN_BIT BIT(2) +#define VADP_INCREMENT_VOLTAGE_LIMIT_BIT BIT(1) +#define VADP_TAPER_TIMER_EN_BIT BIT(0) + +#define USBIN_OPTIONS_2_CFG 0x363 +#define WIPWR_RST_EUD_CFG_BIT BIT(7) +#define SWITCHER_START_CFG_BIT BIT(6) +#define DCD_TIMEOUT_SEL_BIT BIT(5) +#define OCD_CURRENT_SEL_BIT BIT(4) +#define SLOW_PLUGIN_TIMER_EN_CFG_BIT BIT(3) +#define FLOAT_OPTIONS_MASK GENMASK(2, 0) +#define FLOAT_DIS_CHGING_CFG_BIT BIT(2) +#define SUSPEND_FLOAT_CFG_BIT BIT(1) +#define FORCE_FLOAT_SDP_CFG_BIT BIT(0) + +#define TAPER_TIMER_SEL_CFG 0x364 +#define TYPEC_SPARE_CFG_BIT BIT(7) +#define TYPEC_DRP_DFP_TIME_CFG_BIT BIT(5) +#define TAPER_TIMER_SEL_MASK GENMASK(1, 0) + +#define USBIN_LOAD_CFG 0x365 +#define USBIN_OV_CH_LOAD_OPTION_BIT BIT(7) +#define ICL_OVERRIDE_AFTER_APSD_BIT BIT(4) + +#define USBIN_ICL_OPTIONS 0x366 +#define CFG_USB3P0_SEL_BIT BIT(2) +#define USB51_MODE_BIT BIT(1) +#define USBIN_MODE_CHG_BIT BIT(0) + +#define TYPE_C_INTRPT_ENB_SOFTWARE_CTRL 0x368 +#define EXIT_SNK_BASED_ON_CC_BIT BIT(7) +#define VCONN_EN_ORIENTATION_BIT BIT(6) +#define TYPEC_VCONN_OVERCURR_INT_EN_BIT BIT(5) +#define VCONN_EN_SRC_BIT BIT(4) +#define VCONN_EN_VALUE_BIT BIT(3) +#define TYPEC_POWER_ROLE_CMD_MASK GENMASK(2, 0) +#define UFP_EN_CMD_BIT BIT(2) +#define DFP_EN_CMD_BIT BIT(1) +#define TYPEC_DISABLE_CMD_BIT BIT(0) + +#define USBIN_CURRENT_LIMIT_CFG 0x370 +#define USBIN_CURRENT_LIMIT_MASK GENMASK(7, 0) + +#define USBIN_AICL_OPTIONS_CFG 0x380 +#define SUSPEND_ON_COLLAPSE_USBIN_BIT BIT(7) +#define USBIN_AICL_HDC_EN_BIT BIT(6) +#define USBIN_AICL_START_AT_MAX_BIT BIT(5) +#define USBIN_AICL_RERUN_EN_BIT BIT(4) +#define USBIN_AICL_ADC_EN_BIT BIT(3) +#define USBIN_AICL_EN_BIT BIT(2) +#define USBIN_HV_COLLAPSE_RESPONSE_BIT BIT(1) +#define USBIN_LV_COLLAPSE_RESPONSE_BIT BIT(0) + +#define USBIN_5V_AICL_THRESHOLD_CFG 0x381 +#define USBIN_5V_AICL_THRESHOLD_CFG_MASK GENMASK(2, 0) + +#define USBIN_CONT_AICL_THRESHOLD_CFG 0x384 +#define USBIN_CONT_AICL_THRESHOLD_CFG_MASK GENMASK(5, 0) + +#define DC_ENG_SSUPPLY_CFG2 0x4C1 +#define ENG_SSUPPLY_IVREF_OTG_SS_MASK GENMASK(2, 0) +#define OTG_SS_SLOW 0x3 + +#define DCIN_AICL_REF_SEL_CFG 0x481 +#define DCIN_CONT_AICL_THRESHOLD_CFG_MASK GENMASK(5, 0) + +#define WI_PWR_OPTIONS 0x495 +#define CHG_OK_BIT BIT(7) +#define WIPWR_UVLO_IRQ_OPT_BIT BIT(6) +#define BUCK_HOLDOFF_ENABLE_BIT BIT(5) +#define CHG_OK_HW_SW_SELECT_BIT BIT(4) +#define WIPWR_RST_ENABLE_BIT BIT(3) +#define DCIN_WIPWR_IRQ_SELECT_BIT BIT(2) +#define AICL_SWITCH_ENABLE_BIT BIT(1) +#define ZIN_ICL_ENABLE_BIT BIT(0) + +#define ICL_STATUS 0x607 +#define INPUT_CURRENT_LIMIT_MASK GENMASK(7, 0) + +#define POWER_PATH_STATUS 0x60B +#define P_PATH_INPUT_SS_DONE_BIT BIT(7) +#define P_PATH_USBIN_SUSPEND_STS_BIT BIT(6) +#define P_PATH_DCIN_SUSPEND_STS_BIT BIT(5) +#define P_PATH_USE_USBIN_BIT BIT(4) +#define P_PATH_USE_DCIN_BIT BIT(3) +#define P_PATH_POWER_PATH_MASK GENMASK(2, 1) +#define P_PATH_VALID_INPUT_POWER_SOURCE_STS_BIT BIT(0) + +#define BARK_BITE_WDOG_PET 0x643 +#define BARK_BITE_WDOG_PET_BIT BIT(0) + +#define WD_CFG 0x651 +#define WATCHDOG_TRIGGER_AFP_EN_BIT BIT(7) +#define BARK_WDOG_INT_EN_BIT BIT(6) +#define BITE_WDOG_INT_EN_BIT BIT(5) +#define SFT_AFTER_WDOG_IRQ_MASK GENMASK(4, 3) +#define WDOG_IRQ_SFT_BIT BIT(2) +#define WDOG_TIMER_EN_ON_PLUGIN_BIT BIT(1) +#define WDOG_TIMER_EN_BIT BIT(0) + +#define SNARL_BARK_BITE_WD_CFG 0x653 +#define BITE_WDOG_DISABLE_CHARGING_CFG_BIT BIT(7) +#define SNARL_WDOG_TIMEOUT_MASK GENMASK(6, 4) +#define BARK_WDOG_TIMEOUT_MASK GENMASK(3, 2) +#define BITE_WDOG_TIMEOUT_MASK GENMASK(1, 0) + +#define AICL_RERUN_TIME_CFG 0x661 +#define AICL_RERUN_TIME_MASK GENMASK(1, 0) + +#define STAT_CFG 0x690 +#define STAT_SW_OVERRIDE_VALUE_BIT BIT(7) +#define STAT_SW_OVERRIDE_CFG_BIT BIT(6) +#define STAT_PARALLEL_OFF_DG_CFG_MASK GENMASK(5, 4) +#define STAT_POLARITY_CFG_BIT BIT(3) +#define STAT_PARALLEL_CFG_BIT BIT(2) +#define STAT_FUNCTION_CFG_BIT BIT(1) +#define STAT_IRQ_PULSING_EN_BIT BIT(0) + +#define SDP_CURRENT_UA 500000 +#define CDP_CURRENT_UA 1500000 +#define DCP_CURRENT_UA 1500000 +#define CURRENT_MAX_UA DCP_CURRENT_UA + +/* pmi8998 registers represent current in increments of 1/40th of an amp */ +#define CURRENT_SCALE_FACTOR 25000 +#define SMB2_FLOAT_VOLTAGE_MIN_UV 3487500 +#define SMB2_FLOAT_VOLTAGE_MAX_UV 4920000 +#define SMB2_FLOAT_VOLTAGE_STEP_UV 7500 +/* clang-format on */ + +enum charger_status { + TRICKLE_CHARGE = 0, + PRE_CHARGE, + FAST_CHARGE, + FULLON_CHARGE, + TAPER_CHARGE, + TERMINATE_CHARGE, + INHIBIT_CHARGE, + DISABLE_CHARGE, +}; + +struct smb_init_register { + u16 addr; + u8 mask; + u8 val; +}; + +/** + * struct smb_chip - smb chip structure + * @dev: Device reference for power_supply + * @name: The platform device name + * @base: Base address for smb registers + * @regmap: Register map + * @batt_info: Battery data from DT + * @initial_charge_enable: Charging enable state before hardware setup + * @status_change_work: Worker to handle plug/unplug events + * @cable_irq: USB plugin IRQ + * @wakeup_enabled: If the cable IRQ will cause a wakeup + * @usb_in_i_chan: USB_IN current measurement channel + * @usb_in_v_chan: USB_IN voltage measurement channel + * @chg_psy: Charger power supply instance + */ +struct smb_chip { + struct device *dev; + const char *name; + unsigned int base; + struct regmap *regmap; + struct power_supply_battery_info *batt_info; + u8 initial_charge_enable; + + struct delayed_work status_change_work; + int cable_irq; + bool wakeup_enabled; + + struct iio_channel *usb_in_i_chan; + struct iio_channel *usb_in_v_chan; + + struct power_supply *chg_psy; +}; + +static enum power_supply_property smb_properties[] = { + POWER_SUPPLY_PROP_MANUFACTURER, + POWER_SUPPLY_PROP_MODEL_NAME, + POWER_SUPPLY_PROP_CURRENT_MAX, + POWER_SUPPLY_PROP_CURRENT_NOW, + POWER_SUPPLY_PROP_VOLTAGE_NOW, + POWER_SUPPLY_PROP_STATUS, + POWER_SUPPLY_PROP_HEALTH, + POWER_SUPPLY_PROP_ONLINE, + POWER_SUPPLY_PROP_USB_TYPE, +}; + +static int smb_get_prop_usb_online(struct smb_chip *chip, int *val) +{ + unsigned int stat; + int rc; + + rc = regmap_read(chip->regmap, chip->base + POWER_PATH_STATUS, &stat); + if (rc < 0) { + dev_err(chip->dev, "Couldn't read power path status: %d\n", rc); + return rc; + } + + *val = (stat & P_PATH_USE_USBIN_BIT) && + (stat & P_PATH_VALID_INPUT_POWER_SOURCE_STS_BIT); + return 0; +} + +/* + * Qualcomm "automatic power source detection" aka APSD + * tells us what type of charger we're connected to. + */ +static int smb_apsd_get_charger_type(struct smb_chip *chip, int *val) +{ + unsigned int apsd_stat, stat; + int usb_online = 0; + int rc; + + rc = smb_get_prop_usb_online(chip, &usb_online); + if (!usb_online) { + *val = POWER_SUPPLY_USB_TYPE_UNKNOWN; + return rc; + } + + rc = regmap_read(chip->regmap, chip->base + APSD_STATUS, &apsd_stat); + if (rc < 0) { + dev_err(chip->dev, "Failed to read apsd status, rc = %d", rc); + return rc; + } + if (!(apsd_stat & APSD_DTC_STATUS_DONE_BIT)) { + dev_dbg(chip->dev, "Apsd not ready"); + return -EAGAIN; + } + + rc = regmap_read(chip->regmap, chip->base + APSD_RESULT_STATUS, &stat); + if (rc < 0) { + dev_err(chip->dev, "Failed to read apsd result, rc = %d", rc); + return rc; + } + + stat &= APSD_RESULT_STATUS_MASK; + + if (stat & CDP_CHARGER_BIT) + *val = POWER_SUPPLY_USB_TYPE_CDP; + else if (stat & (DCP_CHARGER_BIT | OCP_CHARGER_BIT | FLOAT_CHARGER_BIT)) + *val = POWER_SUPPLY_USB_TYPE_DCP; + else /* SDP_CHARGER_BIT (or others) */ + *val = POWER_SUPPLY_USB_TYPE_SDP; + + return 0; +} + +static int smb_get_prop_status(struct smb_chip *chip, int *val) +{ + unsigned char stat[2]; + int usb_online = 0; + int rc; + + rc = smb_get_prop_usb_online(chip, &usb_online); + if (!usb_online) { + *val = POWER_SUPPLY_STATUS_DISCHARGING; + return rc; + } + + rc = regmap_bulk_read(chip->regmap, + chip->base + BATTERY_CHARGER_STATUS_1, &stat, 2); + if (rc < 0) { + dev_err(chip->dev, "Failed to read charging status ret=%d\n", + rc); + return rc; + } + + if (stat[1] & CHARGER_ERROR_STATUS_BAT_OV_BIT) { + *val = POWER_SUPPLY_STATUS_NOT_CHARGING; + return 0; + } + + stat[0] = stat[0] & BATTERY_CHARGER_STATUS_MASK; + + switch (stat[0]) { + case TRICKLE_CHARGE: + case PRE_CHARGE: + case FAST_CHARGE: + case FULLON_CHARGE: + case TAPER_CHARGE: + *val = POWER_SUPPLY_STATUS_CHARGING; + return rc; + case DISABLE_CHARGE: + *val = POWER_SUPPLY_STATUS_NOT_CHARGING; + return rc; + case TERMINATE_CHARGE: + case INHIBIT_CHARGE: + *val = POWER_SUPPLY_STATUS_FULL; + return rc; + default: + *val = POWER_SUPPLY_STATUS_UNKNOWN; + return rc; + } +} + +static inline int smb_get_current_limit(struct smb_chip *chip, + unsigned int *val) +{ + int rc = regmap_read(chip->regmap, chip->base + ICL_STATUS, val); + + if (rc >= 0) + *val *= CURRENT_SCALE_FACTOR; + return rc; +} + +static int smb_set_current_limit(struct smb_chip *chip, unsigned int val) +{ + unsigned char val_raw; + + if (val > 4800000) { + dev_err(chip->dev, + "Can't set current limit higher than 4800000uA"); + return -EINVAL; + } + val_raw = val / CURRENT_SCALE_FACTOR; + + return regmap_write(chip->regmap, chip->base + USBIN_CURRENT_LIMIT_CFG, + val_raw); +} + +static void smb_status_change_work(struct work_struct *work) +{ + unsigned int charger_type, current_ua; + int usb_online = 0; + int count, rc; + struct smb_chip *chip; + + chip = container_of(work, struct smb_chip, status_change_work.work); + + smb_get_prop_usb_online(chip, &usb_online); + if (!usb_online) + return; + + for (count = 0; count < 3; count++) { + dev_dbg(chip->dev, "get charger type retry %d\n", count); + rc = smb_apsd_get_charger_type(chip, &charger_type); + if (rc != -EAGAIN) + break; + msleep(100); + } + + if (rc < 0 && rc != -EAGAIN) { + dev_err(chip->dev, "get charger type failed: %d\n", rc); + return; + } + + if (rc < 0) { + rc = regmap_update_bits(chip->regmap, chip->base + CMD_APSD, + APSD_RERUN_BIT, APSD_RERUN_BIT); + schedule_delayed_work(&chip->status_change_work, + msecs_to_jiffies(1000)); + dev_dbg(chip->dev, "get charger type failed, rerun apsd\n"); + return; + } + + switch (charger_type) { + case POWER_SUPPLY_USB_TYPE_CDP: + current_ua = CDP_CURRENT_UA; + break; + case POWER_SUPPLY_USB_TYPE_DCP: + current_ua = DCP_CURRENT_UA; + break; + case POWER_SUPPLY_USB_TYPE_SDP: + default: + current_ua = SDP_CURRENT_UA; + break; + } + + smb_set_current_limit(chip, current_ua); + power_supply_changed(chip->chg_psy); +} + +static int smb_get_iio_chan(struct smb_chip *chip, struct iio_channel *chan, + int *val) +{ + int rc; + union power_supply_propval status; + + rc = power_supply_get_property(chip->chg_psy, POWER_SUPPLY_PROP_STATUS, + &status); + if (rc < 0 || status.intval != POWER_SUPPLY_STATUS_CHARGING) { + *val = 0; + return 0; + } + + if (IS_ERR(chan)) { + dev_err(chip->dev, "Failed to chan, err = %li", PTR_ERR(chan)); + return PTR_ERR(chan); + } + + return iio_read_channel_processed(chan, val); +} + +static int smb_get_prop_health(struct smb_chip *chip, int *val) +{ + int rc; + unsigned int stat; + + rc = regmap_read(chip->regmap, chip->base + BATTERY_CHARGER_STATUS_2, + &stat); + if (rc < 0) { + dev_err(chip->dev, "Couldn't read charger status rc=%d\n", rc); + return rc; + } + + if (stat & CHARGER_ERROR_STATUS_BAT_OV_BIT) + *val = POWER_SUPPLY_HEALTH_OVERVOLTAGE; + else if (stat & BAT_TEMP_STATUS_TOO_COLD_BIT) + *val = POWER_SUPPLY_HEALTH_COLD; + else if (stat & BAT_TEMP_STATUS_TOO_HOT_BIT) + *val = POWER_SUPPLY_HEALTH_OVERHEAT; + else if (stat & BAT_TEMP_STATUS_COLD_SOFT_LIMIT_BIT) + *val = POWER_SUPPLY_HEALTH_COOL; + else if (stat & BAT_TEMP_STATUS_HOT_SOFT_LIMIT_BIT) + *val = POWER_SUPPLY_HEALTH_WARM; + else + *val = POWER_SUPPLY_HEALTH_GOOD; + + return 0; +} + +static int smb_get_property(struct power_supply *psy, + enum power_supply_property psp, + union power_supply_propval *val) +{ + struct smb_chip *chip = power_supply_get_drvdata(psy); + + switch (psp) { + case POWER_SUPPLY_PROP_MANUFACTURER: + val->strval = "Qualcomm"; + return 0; + case POWER_SUPPLY_PROP_MODEL_NAME: + val->strval = chip->name; + return 0; + case POWER_SUPPLY_PROP_CURRENT_MAX: + return smb_get_current_limit(chip, &val->intval); + case POWER_SUPPLY_PROP_CURRENT_NOW: + return smb_get_iio_chan(chip, chip->usb_in_i_chan, + &val->intval); + case POWER_SUPPLY_PROP_VOLTAGE_NOW: + return smb_get_iio_chan(chip, chip->usb_in_v_chan, + &val->intval); + case POWER_SUPPLY_PROP_ONLINE: + return smb_get_prop_usb_online(chip, &val->intval); + case POWER_SUPPLY_PROP_STATUS: + return smb_get_prop_status(chip, &val->intval); + case POWER_SUPPLY_PROP_HEALTH: + return smb_get_prop_health(chip, &val->intval); + case POWER_SUPPLY_PROP_USB_TYPE: + return smb_apsd_get_charger_type(chip, &val->intval); + default: + dev_err(chip->dev, "invalid property: %d\n", psp); + return -EINVAL; + } +} + +static int smb_set_property(struct power_supply *psy, + enum power_supply_property psp, + const union power_supply_propval *val) +{ + struct smb_chip *chip = power_supply_get_drvdata(psy); + + switch (psp) { + case POWER_SUPPLY_PROP_CURRENT_MAX: + return smb_set_current_limit(chip, val->intval); + default: + dev_err(chip->dev, "No setter for property: %d\n", psp); + return -EINVAL; + } +} + +static int smb_property_is_writable(struct power_supply *psy, + enum power_supply_property psp) +{ + switch (psp) { + case POWER_SUPPLY_PROP_CURRENT_MAX: + return 1; + default: + return 0; + } +} + +static irqreturn_t smb_handle_batt_overvoltage(int irq, void *data) +{ + struct smb_chip *chip = data; + unsigned int status; + int rc; + + rc = regmap_read(chip->regmap, + chip->base + BATTERY_CHARGER_STATUS_2, &status); + if (rc < 0) { + dev_err(chip->dev, "Couldn't read charger status: %d\n", rc); + return IRQ_HANDLED; + } + + if (status & CHARGER_ERROR_STATUS_BAT_OV_BIT) { + /* The hardware stops charging automatically */ + dev_err(chip->dev, "battery overvoltage detected\n"); + } + + power_supply_changed(chip->chg_psy); + + return IRQ_HANDLED; +} + +static irqreturn_t smb_handle_usb_plugin(int irq, void *data) +{ + struct smb_chip *chip = data; + + power_supply_changed(chip->chg_psy); + + schedule_delayed_work(&chip->status_change_work, + msecs_to_jiffies(1500)); + + return IRQ_HANDLED; +} + +static irqreturn_t smb_handle_usb_icl_change(int irq, void *data) +{ + struct smb_chip *chip = data; + + power_supply_changed(chip->chg_psy); + + return IRQ_HANDLED; +} + +static irqreturn_t smb_handle_wdog_bark(int irq, void *data) +{ + struct smb_chip *chip = data; + int rc; + + power_supply_changed(chip->chg_psy); + + rc = regmap_write(chip->regmap, chip->base + BARK_BITE_WDOG_PET, + BARK_BITE_WDOG_PET_BIT); + if (rc < 0) + dev_err(chip->dev, "Couldn't pet the dog rc=%d\n", rc); + + return IRQ_HANDLED; +} + +static const struct power_supply_desc smb_psy_desc = { + .name = "pmi8998_charger", + .type = POWER_SUPPLY_TYPE_USB, + .usb_types = BIT(POWER_SUPPLY_USB_TYPE_SDP) | + BIT(POWER_SUPPLY_USB_TYPE_CDP) | + BIT(POWER_SUPPLY_USB_TYPE_DCP) | + BIT(POWER_SUPPLY_USB_TYPE_UNKNOWN), + .properties = smb_properties, + .num_properties = ARRAY_SIZE(smb_properties), + .get_property = smb_get_property, + .set_property = smb_set_property, + .property_is_writeable = smb_property_is_writable, +}; + +/* Init sequence derived from vendor downstream driver */ +static const struct smb_init_register smb_init_seq[] = { + { .addr = AICL_RERUN_TIME_CFG, .mask = AICL_RERUN_TIME_MASK, .val = 0 }, + /* + * By default configure us as an upstream facing port + * FIXME: This will be handled by the type-c driver + */ + { .addr = TYPE_C_INTRPT_ENB_SOFTWARE_CTRL, + .mask = TYPEC_POWER_ROLE_CMD_MASK | VCONN_EN_SRC_BIT | + VCONN_EN_VALUE_BIT, + .val = VCONN_EN_SRC_BIT }, + /* + * Disable Type-C factory mode and stay in Attached.SRC state when VCONN + * over-current happens + */ + { .addr = TYPE_C_CFG, + .mask = FACTORY_MODE_DETECTION_EN_BIT | VCONN_OC_CFG_BIT, + .val = 0 }, + /* Configure VBUS for software control */ + { .addr = OTG_CFG, .mask = OTG_EN_SRC_CFG_BIT, .val = 0 }, + /* + * Use VBAT to determine the recharge threshold when battery is full + * rather than the state of charge. + */ + { .addr = FG_UPDATE_CFG_2_SEL, + .mask = SOC_LT_CHG_RECHARGE_THRESH_SEL_BIT | + VBT_LT_CHG_RECHARGE_THRESH_SEL_BIT, + .val = VBT_LT_CHG_RECHARGE_THRESH_SEL_BIT }, + /* Enable charging */ + { .addr = USBIN_OPTIONS_1_CFG, .mask = HVDCP_EN_BIT, .val = 0 }, + { .addr = CHARGING_ENABLE_CMD, + .mask = CHARGING_ENABLE_CMD_BIT, + .val = CHARGING_ENABLE_CMD_BIT }, + /* + * Match downstream defaults + * CHG_EN_SRC_BIT - charger enable is controlled by software + * CHG_EN_POLARITY_BIT - polarity of charge enable pin when in HW control + * pulled low on OnePlus 6 and SHIFT6mq + * PRETOFAST_TRANSITION_CFG_BIT - + * BAT_OV_ECC_BIT - + * I_TERM_BIT - Current termination ?? 0 = enabled + * AUTO_RECHG_BIT - Enable automatic recharge when battery is full + * 0 = enabled + * EN_ANALOG_DROP_IN_VBATT_BIT + * CHARGER_INHIBIT_BIT - Inhibit charging based on battery voltage + * instead of ?? + */ + { .addr = CHGR_CFG2, + .mask = CHG_EN_SRC_BIT | CHG_EN_POLARITY_BIT | + PRETOFAST_TRANSITION_CFG_BIT | BAT_OV_ECC_BIT | I_TERM_BIT | + AUTO_RECHG_BIT | EN_ANALOG_DROP_IN_VBATT_BIT | + CHARGER_INHIBIT_BIT, + .val = CHARGER_INHIBIT_BIT }, + /* STAT pin software override, match downstream. Parallell charging? */ + { .addr = STAT_CFG, + .mask = STAT_SW_OVERRIDE_CFG_BIT, + .val = STAT_SW_OVERRIDE_CFG_BIT }, + /* Set the default SDP charger type to a 500ma USB 2.0 port */ + { .addr = USBIN_ICL_OPTIONS, + .mask = USB51_MODE_BIT | USBIN_MODE_CHG_BIT, + .val = USB51_MODE_BIT }, + /* Disable watchdog */ + { .addr = SNARL_BARK_BITE_WD_CFG, .mask = 0xff, .val = 0 }, + { .addr = WD_CFG, + .mask = WATCHDOG_TRIGGER_AFP_EN_BIT | WDOG_TIMER_EN_ON_PLUGIN_BIT | + BARK_WDOG_INT_EN_BIT, + .val = 0 }, + /* These bits aren't documented anywhere */ + { .addr = USBIN_5V_AICL_THRESHOLD_CFG, + .mask = USBIN_5V_AICL_THRESHOLD_CFG_MASK, + .val = 0x3 }, + { .addr = USBIN_CONT_AICL_THRESHOLD_CFG, + .mask = USBIN_CONT_AICL_THRESHOLD_CFG_MASK, + .val = 0x3 }, + /* + * Enable Automatic Input Current Limit, this will slowly ramp up the current + * When connected to a wall charger, and automatically stop when it detects + * the charger current limit (voltage drop?) or it reaches the programmed limit. + */ + { .addr = USBIN_AICL_OPTIONS_CFG, + .mask = USBIN_AICL_START_AT_MAX_BIT | USBIN_AICL_ADC_EN_BIT | + USBIN_AICL_EN_BIT | SUSPEND_ON_COLLAPSE_USBIN_BIT | + USBIN_HV_COLLAPSE_RESPONSE_BIT | + USBIN_LV_COLLAPSE_RESPONSE_BIT, + .val = USBIN_HV_COLLAPSE_RESPONSE_BIT | + USBIN_LV_COLLAPSE_RESPONSE_BIT | USBIN_AICL_EN_BIT }, + /* + * Set pre charge current to default, the OnePlus 6 bootloader + * sets this very conservatively. + */ + { .addr = PRE_CHARGE_CURRENT_CFG, + .mask = PRE_CHARGE_CURRENT_SETTING_MASK, + .val = 500000 / CURRENT_SCALE_FACTOR }, + /* + * This overrides all of the current limit options exposed to userspace + * and prevents the device from pulling more than ~1A. This is done + * to minimise potential fire hazard risks. + */ + { .addr = FAST_CHARGE_CURRENT_CFG, + .mask = FAST_CHARGE_CURRENT_SETTING_MASK, + .val = 1000000 / CURRENT_SCALE_FACTOR }, +}; + +static int smb_init_hw(struct smb_chip *chip) +{ + int rc, i; + + for (i = 0; i < ARRAY_SIZE(smb_init_seq); i++) { + dev_dbg(chip->dev, "%d: Writing 0x%02x to 0x%02x\n", i, + smb_init_seq[i].val, smb_init_seq[i].addr); + rc = regmap_update_bits(chip->regmap, + chip->base + smb_init_seq[i].addr, + smb_init_seq[i].mask, + smb_init_seq[i].val); + if (rc < 0) + return dev_err_probe(chip->dev, rc, + "%s: init command %d failed\n", + __func__, i); + } + + return 0; +} + +static void smb_restore_charge_enable(void *data) +{ + struct smb_chip *chip = data; + int rc; + + rc = regmap_update_bits(chip->regmap, + chip->base + CHARGING_ENABLE_CMD, + CHARGING_ENABLE_CMD_BIT, + chip->initial_charge_enable); + if (rc < 0) + dev_err(chip->dev, "Couldn't restore charging state: %d\n", rc); +} + +static int smb_init_irq(struct smb_chip *chip, int *irq, const char *name, + irqreturn_t (*handler)(int irq, void *data)) +{ + int irqnum; + int rc; + + irqnum = platform_get_irq_byname(to_platform_device(chip->dev), name); + if (irqnum < 0) + return irqnum; + + rc = devm_request_threaded_irq(chip->dev, irqnum, NULL, handler, + IRQF_ONESHOT, name, chip); + if (rc < 0) + return dev_err_probe(chip->dev, rc, "Couldn't request irq %s\n", + name); + + if (irq) + *irq = irqnum; + + return 0; +} + +static int smb_probe(struct platform_device *pdev) +{ + struct power_supply_config supply_config = {}; + struct power_supply_desc *desc; + struct smb_chip *chip; + unsigned int charge_enable; + unsigned int float_voltage_sel; + int float_voltage_uv; + int rc, irq; + + chip = devm_kzalloc(&pdev->dev, sizeof(*chip), GFP_KERNEL); + if (!chip) + return -ENOMEM; + + chip->dev = &pdev->dev; + chip->name = pdev->name; + + chip->regmap = dev_get_regmap(pdev->dev.parent, NULL); + if (!chip->regmap) + return dev_err_probe(chip->dev, -ENODEV, + "failed to locate the regmap\n"); + + rc = device_property_read_u32(chip->dev, "reg", &chip->base); + if (rc < 0) + return dev_err_probe(chip->dev, rc, + "Couldn't read base address\n"); + + chip->usb_in_v_chan = devm_iio_channel_get(chip->dev, "usbin_v"); + if (IS_ERR(chip->usb_in_v_chan)) + return dev_err_probe(chip->dev, PTR_ERR(chip->usb_in_v_chan), + "Couldn't get usbin_v IIO channel\n"); + + chip->usb_in_i_chan = devm_iio_channel_get(chip->dev, "usbin_i"); + if (IS_ERR(chip->usb_in_i_chan)) { + return dev_err_probe(chip->dev, PTR_ERR(chip->usb_in_i_chan), + "Couldn't get usbin_i IIO channel\n"); + } + + rc = regmap_read(chip->regmap, chip->base + CHARGING_ENABLE_CMD, + &charge_enable); + if (rc < 0) + return dev_err_probe(chip->dev, rc, + "Couldn't read charging state\n"); + + chip->initial_charge_enable = charge_enable & CHARGING_ENABLE_CMD_BIT; + rc = devm_add_action_or_reset(chip->dev, smb_restore_charge_enable, chip); + if (rc) + return dev_err_probe(chip->dev, rc, + "Couldn't register charging state rollback\n"); + + rc = smb_init_hw(chip); + if (rc < 0) + return rc; + + supply_config.drv_data = chip; + supply_config.of_node = pdev->dev.of_node; + + desc = devm_kzalloc(chip->dev, sizeof(smb_psy_desc), GFP_KERNEL); + if (!desc) + return -ENOMEM; + memcpy(desc, &smb_psy_desc, sizeof(smb_psy_desc)); + desc->name = + devm_kasprintf(chip->dev, GFP_KERNEL, "%s-charger", + (const char *)device_get_match_data(chip->dev)); + if (!desc->name) + return -ENOMEM; + + chip->chg_psy = + devm_power_supply_register(chip->dev, desc, &supply_config); + if (IS_ERR(chip->chg_psy)) + return dev_err_probe(chip->dev, PTR_ERR(chip->chg_psy), + "failed to register power supply\n"); + + rc = power_supply_get_battery_info(chip->chg_psy, &chip->batt_info); + if (rc) + return dev_err_probe(chip->dev, rc, + "Failed to get battery info\n"); + + rc = devm_delayed_work_autocancel(chip->dev, &chip->status_change_work, + smb_status_change_work); + if (rc) + return dev_err_probe(chip->dev, rc, + "Failed to init status change work\n"); + + if (power_supply_battery_info_has_prop(chip->batt_info, + POWER_SUPPLY_PROP_CONSTANT_CHARGE_VOLTAGE_MAX)) { + float_voltage_uv = + chip->batt_info->constant_charge_voltage_max_uv; + } else if (power_supply_battery_info_has_prop(chip->batt_info, + POWER_SUPPLY_PROP_VOLTAGE_MAX_DESIGN)) { + float_voltage_uv = chip->batt_info->voltage_max_design_uv; + } else { + dev_warn(chip->dev, "No battery float voltage; preserving hardware setting\n"); + goto skip_float_voltage; + } + + if (float_voltage_uv < SMB2_FLOAT_VOLTAGE_MIN_UV || + float_voltage_uv > SMB2_FLOAT_VOLTAGE_MAX_UV) + return dev_err_probe(chip->dev, -EINVAL, + "float voltage %d uV outside %d-%d uV\n", + float_voltage_uv, + SMB2_FLOAT_VOLTAGE_MIN_UV, + SMB2_FLOAT_VOLTAGE_MAX_UV); + + float_voltage_sel = + (float_voltage_uv - SMB2_FLOAT_VOLTAGE_MIN_UV) / + SMB2_FLOAT_VOLTAGE_STEP_UV; + rc = regmap_update_bits(chip->regmap, chip->base + FLOAT_VOLTAGE_CFG, + FLOAT_VOLTAGE_SETTING_MASK, float_voltage_sel); + if (rc < 0) + return dev_err_probe(chip->dev, rc, "Couldn't set vbat max\n"); + +skip_float_voltage: + rc = smb_init_irq(chip, &irq, "bat-ov", smb_handle_batt_overvoltage); + if (rc < 0) + return rc; + + rc = smb_init_irq(chip, &chip->cable_irq, "usb-plugin", + smb_handle_usb_plugin); + if (rc < 0) + return rc; + + rc = smb_init_irq(chip, &irq, "usbin-icl-change", + smb_handle_usb_icl_change); + if (rc < 0) + return rc; + rc = smb_init_irq(chip, &irq, "wdog-bark", smb_handle_wdog_bark); + if (rc < 0) + return rc; + + rc = dev_pm_set_wake_irq(chip->dev, chip->cable_irq); + if (rc < 0) + return dev_err_probe(chip->dev, rc, "Couldn't set wake irq\n"); + + devm_remove_action(chip->dev, smb_restore_charge_enable, chip); + + platform_set_drvdata(pdev, chip); + + /* Initialise charger state */ + schedule_delayed_work(&chip->status_change_work, 0); + + return 0; +} + +static const struct of_device_id smb_match_id_table[] = { + { .compatible = "qcom,pmi8998-charger", .data = "pmi8998" }, + { .compatible = "qcom,pm660-charger", .data = "pm660" }, + { /* sentinal */ } +}; +MODULE_DEVICE_TABLE(of, smb_match_id_table); + +static struct platform_driver qcom_spmi_smb = { + .probe = smb_probe, + .driver = { + .name = "qcom-smbx-charger", + .of_match_table = smb_match_id_table, + }, +}; + +module_platform_driver(qcom_spmi_smb); + +MODULE_AUTHOR("Caleb Connolly "); +MODULE_DESCRIPTION("Qualcomm SMB2 Charger Driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/power/supply/rt9467-charger.c b/drivers/power/supply/rt9467-charger.c index be65b0f51721..be1ca7f61c0b 100644 --- a/drivers/power/supply/rt9467-charger.c +++ b/drivers/power/supply/rt9467-charger.c @@ -1145,27 +1145,6 @@ static int rt9467_reset_chip(struct rt9467_chg_data *data) return regmap_field_write(data->rm_field[F_RST], 1); } -static void rt9467_chg_destroy_adc_lock(void *data) -{ - struct mutex *adc_lock = data; - - mutex_destroy(adc_lock); -} - -static void rt9467_chg_destroy_attach_lock(void *data) -{ - struct mutex *attach_lock = data; - - mutex_destroy(attach_lock); -} - -static void rt9467_chg_destroy_ichg_ieoc_lock(void *data) -{ - struct mutex *ichg_ieoc_lock = data; - - mutex_destroy(ichg_ieoc_lock); -} - static void rt9467_chg_complete_aicl_done(void *data) { struct completion *aicl_done = data; @@ -1218,21 +1197,15 @@ static int rt9467_charger_probe(struct i2c_client *i2c) if (ret) return dev_err_probe(dev, ret, "Failed to add irq chip\n"); - mutex_init(&data->adc_lock); - ret = devm_add_action_or_reset(dev, rt9467_chg_destroy_adc_lock, - &data->adc_lock); + ret = devm_mutex_init(dev, &data->adc_lock); if (ret) return dev_err_probe(dev, ret, "Failed to init ADC lock\n"); - mutex_init(&data->attach_lock); - ret = devm_add_action_or_reset(dev, rt9467_chg_destroy_attach_lock, - &data->attach_lock); + ret = devm_mutex_init(dev, &data->attach_lock); if (ret) return dev_err_probe(dev, ret, "Failed to init attach lock\n"); - mutex_init(&data->ichg_ieoc_lock); - ret = devm_add_action_or_reset(dev, rt9467_chg_destroy_ichg_ieoc_lock, - &data->ichg_ieoc_lock); + ret = devm_mutex_init(dev, &data->ichg_ieoc_lock); if (ret) return dev_err_probe(dev, ret, "Failed to init ICHG/IEOC lock\n"); diff --git a/drivers/ptp/ptp_sysfs.c b/drivers/ptp/ptp_sysfs.c index 200eaf500696..0d4b081daf63 100644 --- a/drivers/ptp/ptp_sysfs.c +++ b/drivers/ptp/ptp_sysfs.c @@ -80,13 +80,17 @@ static ssize_t extts_fifo_show(struct device *dev, size_t qcnt; int cnt = 0; + spin_lock_irqsave(&ptp->tsevqs_lock, flags); cnt = list_count_nodes(&ptp->tsevqs); - if (cnt <= 0) + if (cnt <= 0) { + spin_unlock_irqrestore(&ptp->tsevqs_lock, flags); goto out; + } /* The sysfs fifo will always draw from the fist queue */ queue = list_first_entry(&ptp->tsevqs, struct timestamp_event_queue, qlist); + spin_unlock_irqrestore(&ptp->tsevqs_lock, flags); memset(&event, 0, sizeof(event)); spin_lock_irqsave(&queue->lock, flags); diff --git a/drivers/s390/char/vmcp.c b/drivers/s390/char/vmcp.c index c6d58335beb4..4fdc3f1dc8db 100644 --- a/drivers/s390/char/vmcp.c +++ b/drivers/s390/char/vmcp.c @@ -105,13 +105,11 @@ static int vmcp_open(struct inode *inode, struct file *file) if (!capable(CAP_SYS_ADMIN)) return -EPERM; - session = kmalloc(sizeof(*session), GFP_KERNEL); + session = kzalloc(sizeof(*session), GFP_KERNEL); if (!session) return -ENOMEM; session->bufsize = PAGE_SIZE; - session->response = NULL; - session->resp_size = 0; mutex_init(&session->mutex); file->private_data = session; return nonseekable_open(inode, file); diff --git a/drivers/scsi/aic7xxx/aic79xx_core.c b/drivers/scsi/aic7xxx/aic79xx_core.c index 3e3100dbfda3..287df8723810 100644 --- a/drivers/scsi/aic7xxx/aic79xx_core.c +++ b/drivers/scsi/aic7xxx/aic79xx_core.c @@ -8942,8 +8942,10 @@ ahd_handle_scsi_status(struct ahd_softc *ahd, struct scb *scb) } #endif - if (ahd_perform_autosense(scb) == 0) + if (ahd_perform_autosense(scb) == 0) { + ahd_done(ahd, scb); break; + } ahd_compile_devinfo(&devinfo, SCB_GET_OUR_ID(scb), SCB_GET_TARGET(ahd, scb), diff --git a/drivers/scsi/aic7xxx/aic79xx_osm.h b/drivers/scsi/aic7xxx/aic79xx_osm.h index 793fe19993a9..3bcfcfe18f08 100644 --- a/drivers/scsi/aic7xxx/aic79xx_osm.h +++ b/drivers/scsi/aic7xxx/aic79xx_osm.h @@ -499,6 +499,8 @@ int ahd_linux_show_info(struct seq_file *,struct Scsi_Host *); static inline void ahd_cmd_set_transaction_status(struct scsi_cmnd *cmd, uint32_t status) { + if (!cmd) + return; cmd->result &= ~(CAM_STATUS_MASK << 16); cmd->result |= status << 16; } @@ -512,6 +514,8 @@ void ahd_set_transaction_status(struct scb *scb, uint32_t status) static inline void ahd_cmd_set_scsi_status(struct scsi_cmnd *cmd, uint32_t status) { + if (!cmd) + return; cmd->result &= ~0xFFFF; cmd->result |= status; } @@ -525,6 +529,8 @@ void ahd_set_scsi_status(struct scb *scb, uint32_t status) static inline uint32_t ahd_cmd_get_transaction_status(struct scsi_cmnd *cmd) { + if (!cmd) + return CAM_REQ_INPROG; return ((cmd->result >> 16) & CAM_STATUS_MASK); } @@ -537,6 +543,8 @@ uint32_t ahd_get_transaction_status(struct scb *scb) static inline uint32_t ahd_cmd_get_scsi_status(struct scsi_cmnd *cmd) { + if (!cmd) + return SAM_STAT_GOOD; return (cmd->result & 0xFFFF); } @@ -564,13 +572,16 @@ u_long ahd_get_transfer_length(struct scb *scb) static inline int ahd_get_transfer_dir(struct scb *scb) { + if (!scb->io_ctx) + return DMA_NONE; return (scb->io_ctx->sc_data_direction); } static inline void ahd_set_residual(struct scb *scb, u_long resid) { - scsi_set_resid(scb->io_ctx, resid); + if (scb->io_ctx) + scsi_set_resid(scb->io_ctx, resid); } static inline @@ -582,6 +593,8 @@ void ahd_set_sense_residual(struct scb *scb, u_long resid) static inline u_long ahd_get_residual(struct scb *scb) { + if (!scb->io_ctx) + return 0; return scsi_get_resid(scb->io_ctx); } @@ -595,11 +608,12 @@ static inline int ahd_perform_autosense(struct scb *scb) { /* - * We always perform autosense in Linux. - * On other platforms this is set on a - * per-transaction basis. + * We always perform autosense in Linux, except for the device + * reset SCB, which has no command to return sense data to. */ - return (1); + if (!scb->io_ctx) + return 0; + return 1; } static inline uint32_t @@ -629,6 +643,8 @@ void ahd_platform_freeze_devq(struct ahd_softc *ahd, struct scb *scb); static inline void ahd_freeze_scb(struct scb *scb) { + if (!scb->io_ctx) + return; if ((scb->io_ctx->result & (CAM_DEV_QFRZN << 16)) == 0) { scb->io_ctx->result |= CAM_DEV_QFRZN << 16; scb->platform_data->dev->qfrozen++; diff --git a/drivers/scsi/aic7xxx/aic7xxx_core.c b/drivers/scsi/aic7xxx/aic7xxx_core.c index a396f048a031..d747d0b72a13 100644 --- a/drivers/scsi/aic7xxx/aic7xxx_core.c +++ b/drivers/scsi/aic7xxx/aic7xxx_core.c @@ -5266,9 +5266,13 @@ ahc_init(struct ahc_softc *ahc) /* * Assume we have a board at this stage and it has been reset. + * PCI controllers keep the host ID ahc_pci_config() chose. */ - if ((ahc->flags & AHC_USEDEFAULTS) != 0) - ahc->our_id = ahc->our_id_b = 7; + if ((ahc->flags & AHC_USEDEFAULTS) != 0) { + if ((ahc->chip & AHC_PCI) == 0) + ahc->our_id = 7; + ahc->our_id_b = 7; + } /* * Default to allowing initiator operations. diff --git a/drivers/scsi/aic7xxx/aic7xxx_pci.c b/drivers/scsi/aic7xxx/aic7xxx_pci.c index 2d4c85426dc3..5a04c7d55e4b 100644 --- a/drivers/scsi/aic7xxx/aic7xxx_pci.c +++ b/drivers/scsi/aic7xxx/aic7xxx_pci.c @@ -701,6 +701,21 @@ ahc_find_pci_device(ahc_dev_softc_t pci) return (NULL); } +/* + * The SGI O2 firmware runs its onboard controllers, in slots 1 and 2 of the + * root bus, without the Adaptec BIOS and at host ID 0. It leaves no settings + * in scratch RAM, which may still hold power-on contents with bad parity. + */ +static bool +ahc_pci_sgi_o2_onboard(struct ahc_softc *ahc) +{ + int slot = ahc_get_pci_slot(ahc->dev_softc); + + return IS_ENABLED(CONFIG_SGI_IP32) && + ahc_get_pci_bus(ahc->dev_softc) == 0 && + (slot == 1 || slot == 2); +} + int ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry) { @@ -782,6 +797,9 @@ ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry) ahc->bus_intr = ahc_pci_intr; ahc->bus_chip_init = ahc_pci_chip_init; + if (ahc_pci_sgi_o2_onboard(ahc)) + ahc->flags |= AHC_NO_BIOS_INIT; + /* Remember how the card was setup in case there is no SEEPROM */ if ((ahc_inb(ahc, HCNTRL) & POWRDN) == 0) { ahc_pause(ahc); @@ -898,7 +916,7 @@ ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry) * Assume only one connector and always turn * on termination. */ - our_id = 0x07; + our_id = ahc_pci_sgi_o2_onboard(ahc) ? 0 : 0x07; sxfrctl1 = STPWEN; } ahc_outb(ahc, SCSICONF, our_id|ENSPCHK|RESET_SCSI); diff --git a/drivers/scsi/libsas/sas_ata.c b/drivers/scsi/libsas/sas_ata.c index 7b4e7a61965a..adb9e7a94785 100644 --- a/drivers/scsi/libsas/sas_ata.c +++ b/drivers/scsi/libsas/sas_ata.c @@ -559,8 +559,8 @@ static int sas_ata_prereset(struct ata_link *link, unsigned long deadline) } static struct ata_port_operations sas_sata_ops = { - .prereset = sas_ata_prereset, - .hardreset = sas_ata_hard_reset, + .reset.prereset = sas_ata_prereset, + .reset.hardreset = sas_ata_hard_reset, .error_handler = ata_std_error_handler, .post_internal_cmd = sas_ata_post_internal, .qc_defer = ata_std_qc_defer, diff --git a/drivers/scsi/scsi_bsg.c b/drivers/scsi/scsi_bsg.c index a9a9ec086a7e..e2e4b8bb559f 100644 --- a/drivers/scsi/scsi_bsg.c +++ b/drivers/scsi/scsi_bsg.c @@ -15,7 +15,7 @@ static int scsi_bsg_sg_io_fn(struct request_queue *q, struct sg_io_v4 *hdr, struct scsi_cmnd *scmd; struct request *rq; struct bio *bio; - int ret; + int ret, unmap_ret; if (hdr->protocol != BSG_PROTOCOL_SCSI || hdr->subprotocol != BSG_SUB_PROTOCOL_SCSI_CMD) @@ -89,7 +89,9 @@ static int scsi_bsg_sg_io_fn(struct request_queue *q, struct sg_io_v4 *hdr, else hdr->dout_resid = scmd->resid_len; - blk_rq_unmap_user(bio); + unmap_ret = blk_rq_unmap_user(bio); + if (!ret) + ret = unmap_ret; out_put_request: blk_mq_free_request(rq); diff --git a/drivers/scsi/scsi_ioctl.c b/drivers/scsi/scsi_ioctl.c index 6f6c5973c3ea..1e969f78f060 100644 --- a/drivers/scsi/scsi_ioctl.c +++ b/drivers/scsi/scsi_ioctl.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -180,10 +181,14 @@ static int sg_set_timeout(struct scsi_device *sdev, int __user *p) { int timeout, err = get_user(timeout, p); - if (!err) - sdev->sg_timeout = clock_t_to_jiffies(timeout); + if (err) + return err; + if (timeout < 0) + return -EIO; - return err; + sdev->sg_timeout = min(clock_t_to_jiffies(timeout), + min(UINT_MAX, MAX_JIFFY_OFFSET)); + return 0; } static int sg_get_reserved_size(struct scsi_device *sdev, int __user *p) diff --git a/drivers/soc/qcom/pmic_glink_altmode.c b/drivers/soc/qcom/pmic_glink_altmode.c index db25f406878b..f650c8de9639 100644 --- a/drivers/soc/qcom/pmic_glink_altmode.c +++ b/drivers/soc/qcom/pmic_glink_altmode.c @@ -304,7 +304,7 @@ static void pmic_glink_altmode_sc8180xp_notify(struct pmic_glink_altmode *altmod alt_port->mode = mode; alt_port->hpd_state = hpd_state; alt_port->hpd_irq = hpd_irq; - schedule_work(&alt_port->work); + queue_work(system_freezable_wq, &alt_port->work); } #define SC8280XP_DPAM_MASK 0x3f @@ -347,7 +347,7 @@ static void pmic_glink_altmode_sc8280xp_notify(struct pmic_glink_altmode *altmod alt_port->mode = mode; alt_port->hpd_state = hpd_state; alt_port->hpd_irq = hpd_irq; - schedule_work(&alt_port->work); + queue_work(system_freezable_wq, &alt_port->work); } static void pmic_glink_altmode_callback(const void *data, size_t len, void *priv) diff --git a/drivers/spi/spi-cadence-xspi.c b/drivers/spi/spi-cadence-xspi.c index c3676b21e816..ca6af39e977a 100644 --- a/drivers/spi/spi-cadence-xspi.c +++ b/drivers/spi/spi-cadence-xspi.c @@ -2,7 +2,6 @@ // Cadence XSPI flash controller driver // Copyright (C) 2020-21 Cadence -#include #include #include #include @@ -21,6 +20,7 @@ #include #include #include +#include #define CDNS_XSPI_MAGIC_NUM_VALUE 0x6522 #define CDNS_XSPI_MAX_BANKS 8 @@ -316,39 +316,10 @@ struct cdns_xspi_driver_data { u32 rfile_phy_dll_slave_ctrl; }; -static struct cdns_xspi_driver_data marvell_driver_data = { - .mrvl_hw_overlay = true, - .dll_phy_ctrl = MARVELL_REGS_DLL_PHY_CTRL, - .ctb_rfile_phy_ctrl = MARVELL_CTB_RFILE_PHY_CTRL, - .rfile_phy_tsel = MARVELL_RFILE_PHY_TSEL, - .rfile_phy_dq_timing = MARVELL_RFILE_PHY_DQ_TIMING, - .rfile_phy_dqs_timing = MARVELL_RFILE_PHY_DQS_TIMING, - .rfile_phy_gate_lpbk_ctrl = MARVELL_RFILE_PHY_GATE_LPBK_CTRL, - .rfile_phy_dll_master_ctrl = MARVELL_RFILE_PHY_DLL_MASTER_CTRL, - .rfile_phy_dll_slave_ctrl = MARVELL_RFILE_PHY_DLL_SLAVE_CTRL, -}; - static struct cdns_xspi_driver_data cdns_driver_data = { .mrvl_hw_overlay = false, }; -static const int cdns_mrvl_xspi_clk_div_list[] = { - 4, //0x0 = Divide by 4. SPI clock is 200 MHz. - 6, //0x1 = Divide by 6. SPI clock is 133.33 MHz. - 8, //0x2 = Divide by 8. SPI clock is 100 MHz. - 10, //0x3 = Divide by 10. SPI clock is 80 MHz. - 12, //0x4 = Divide by 12. SPI clock is 66.666 MHz. - 16, //0x5 = Divide by 16. SPI clock is 50 MHz. - 18, //0x6 = Divide by 18. SPI clock is 44.44 MHz. - 20, //0x7 = Divide by 20. SPI clock is 40 MHz. - 24, //0x8 = Divide by 24. SPI clock is 33.33 MHz. - 32, //0x9 = Divide by 32. SPI clock is 25 MHz. - 40, //0xA = Divide by 40. SPI clock is 20 MHz. - 50, //0xB = Divide by 50. SPI clock is 16 MHz. - 64, //0xC = Divide by 64. SPI clock is 12.5 MHz. - 128 //0xD = Divide by 128. SPI clock is 6.25 MHz. -}; - struct cdns_xspi_dev { struct platform_device *pdev; struct device *dev; @@ -361,6 +332,7 @@ struct cdns_xspi_dev { int irq; int cur_cs; unsigned int sdmasize; + unsigned int sdma_xfer_len; struct completion cmd_complete; struct completion auto_cmd_complete; @@ -373,100 +345,13 @@ struct cdns_xspi_dev { u8 hw_num_banks; const struct cdns_xspi_driver_data *driver_data; - void (*sdma_handler)(struct cdns_xspi_dev *cdns_xspi); + int (*sdma_handler)(struct cdns_xspi_dev *cdns_xspi); void (*set_interrupts_handler)(struct cdns_xspi_dev *cdns_xspi, bool enabled); bool xfer_in_progress; int current_xfer_qword; }; -static void cdns_xspi_reset_dll(struct cdns_xspi_dev *cdns_xspi) -{ - u32 dll_cntrl = readl(cdns_xspi->iobase + - CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL); - - /* Reset DLL */ - dll_cntrl |= CDNS_XSPI_DLL_RST_N; - writel(dll_cntrl, cdns_xspi->iobase + - CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL); -} - -static bool cdns_xspi_is_dll_locked(struct cdns_xspi_dev *cdns_xspi) -{ - u32 dll_lock; - - return !readl_relaxed_poll_timeout(cdns_xspi->iobase + - CDNS_XSPI_INTR_STATUS_REG, - dll_lock, ((dll_lock & CDNS_XSPI_DLL_LOCK) == 1), 10, 10000); -} - -/* Static configuration of PHY */ -static bool cdns_xspi_configure_phy(struct cdns_xspi_dev *cdns_xspi) -{ - writel(cdns_xspi->driver_data->dll_phy_ctrl, - cdns_xspi->iobase + CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL); - writel(cdns_xspi->driver_data->ctb_rfile_phy_ctrl, - cdns_xspi->auxbase + CDNS_XSPI_PHY_CTB_RFILE_PHY_CTRL); - writel(cdns_xspi->driver_data->rfile_phy_tsel, - cdns_xspi->auxbase + CDNS_XSPI_PHY_CTB_RFILE_PHY_TSEL); - writel(cdns_xspi->driver_data->rfile_phy_dq_timing, - cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DQ_TIMING); - writel(cdns_xspi->driver_data->rfile_phy_dqs_timing, - cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DQS_TIMING); - writel(cdns_xspi->driver_data->rfile_phy_gate_lpbk_ctrl, - cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_GATE_LPBK_CTRL); - writel(cdns_xspi->driver_data->rfile_phy_dll_master_ctrl, - cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DLL_MASTER_CTRL); - writel(cdns_xspi->driver_data->rfile_phy_dll_slave_ctrl, - cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DLL_SLAVE_CTRL); - - cdns_xspi_reset_dll(cdns_xspi); - - return cdns_xspi_is_dll_locked(cdns_xspi); -} - -static bool cdns_mrvl_xspi_setup_clock(struct cdns_xspi_dev *cdns_xspi, - int requested_clk) -{ - int i = 0; - int clk_val; - u32 clk_reg; - bool update_clk = false; - - while (i < (ARRAY_SIZE(cdns_mrvl_xspi_clk_div_list) - 1)) { - clk_val = MRVL_XSPI_CLOCK_DIVIDED( - cdns_mrvl_xspi_clk_div_list[i]); - if (clk_val <= requested_clk) - break; - i++; - } - - dev_dbg(cdns_xspi->dev, "Found clk div: %d, clk val: %d\n", - cdns_mrvl_xspi_clk_div_list[i], - MRVL_XSPI_CLOCK_DIVIDED( - cdns_mrvl_xspi_clk_div_list[i])); - - clk_reg = readl(cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG); - - if (FIELD_GET(MRVL_XSPI_CLK_DIV, clk_reg) != i) { - clk_reg &= ~MRVL_XSPI_CLK_ENABLE; - writel(clk_reg, - cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG); - clk_reg = FIELD_PREP(MRVL_XSPI_CLK_DIV, i); - clk_reg &= ~MRVL_XSPI_CLK_DIV; - clk_reg |= FIELD_PREP(MRVL_XSPI_CLK_DIV, i); - clk_reg |= MRVL_XSPI_CLK_ENABLE; - clk_reg |= MRVL_XSPI_IRQ_ENABLE; - update_clk = true; - } - - if (update_clk) - writel(clk_reg, - cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG); - - return update_clk; -} - static int cdns_xspi_wait_for_controller_idle(struct cdns_xspi_dev *cdns_xspi) { u32 ctrl_stat; @@ -539,23 +424,6 @@ static void cdns_xspi_set_interrupts(struct cdns_xspi_dev *cdns_xspi, writel(intr_enable, cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG); } -static void marvell_xspi_set_interrupts(struct cdns_xspi_dev *cdns_xspi, - bool enabled) -{ - u32 intr_enable; - u32 irq_status; - - irq_status = readl(cdns_xspi->iobase + CDNS_XSPI_INTR_STATUS_REG); - writel(irq_status, cdns_xspi->iobase + CDNS_XSPI_INTR_STATUS_REG); - - intr_enable = readl(cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG); - if (enabled) - intr_enable |= CDNS_XSPI_INTR_MASK; - else - intr_enable &= ~CDNS_XSPI_INTR_MASK; - writel(intr_enable, cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG); -} - static int cdns_xspi_controller_init(struct cdns_xspi_dev *cdns_xspi) { u32 ctrl_ver; @@ -578,7 +446,7 @@ static int cdns_xspi_controller_init(struct cdns_xspi_dev *cdns_xspi) return 0; } -static void cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi) +static int cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi) { u32 sdma_size, sdma_trd_info; u8 sdma_dir; @@ -587,6 +455,13 @@ static void cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi) sdma_trd_info = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_TRD_INFO_REG); sdma_dir = FIELD_GET(CDNS_XSPI_SDMA_DIR, sdma_trd_info); + if (sdma_size > cdns_xspi->sdma_xfer_len) { + dev_err(cdns_xspi->dev, + "SDMA size %u exceeds the requested length %u\n", + sdma_size, cdns_xspi->sdma_xfer_len); + return -EINVAL; + } + switch (sdma_dir) { case CDNS_XSPI_SDMA_DIR_READ: ioread8_rep(cdns_xspi->sdmabase, @@ -598,78 +473,8 @@ static void cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi) cdns_xspi->out_buffer, sdma_size); break; } -} - -static void m_ioreadq(void __iomem *addr, void *buf, int len) -{ - if (IS_ALIGNED((long)buf, 8) && len >= 8) { - u64 full_ops = len / 8; - u64 *buffer = buf; - - len -= full_ops * 8; - buf += full_ops * 8; - - do { - u64 b = readq(addr); - *buffer++ = b; - } while (--full_ops); - } - - - while (len) { - u64 tmp_buf; - - tmp_buf = readq(addr); - memcpy(buf, &tmp_buf, min(len, 8)); - len = len > 8 ? len - 8 : 0; - buf += 8; - } -} - -static void m_iowriteq(void __iomem *addr, const void *buf, int len) -{ - if (IS_ALIGNED((long)buf, 8) && len >= 8) { - u64 full_ops = len / 8; - const u64 *buffer = buf; - - len -= full_ops * 8; - buf += full_ops * 8; - - do { - writeq(*buffer++, addr); - } while (--full_ops); - } - while (len) { - u64 tmp_buf; - - memcpy(&tmp_buf, buf, min(len, 8)); - writeq(tmp_buf, addr); - len = len > 8 ? len - 8 : 0; - buf += 8; - } -} - -static void marvell_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi) -{ - u32 sdma_size, sdma_trd_info; - u8 sdma_dir; - - sdma_size = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_SIZE_REG); - sdma_trd_info = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_TRD_INFO_REG); - sdma_dir = FIELD_GET(CDNS_XSPI_SDMA_DIR, sdma_trd_info); - - switch (sdma_dir) { - case CDNS_XSPI_SDMA_DIR_READ: - m_ioreadq(cdns_xspi->sdmabase, - cdns_xspi->in_buffer, sdma_size); - break; - - case CDNS_XSPI_SDMA_DIR_WRITE: - m_iowriteq(cdns_xspi->sdmabase, - cdns_xspi->out_buffer, sdma_size); - break; - } + return 0; } static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi, @@ -715,6 +520,7 @@ static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi, cdns_xspi->in_buffer = op->data.buf.in; cdns_xspi->out_buffer = op->data.buf.out; + cdns_xspi->sdma_xfer_len = op->data.nbytes; cdns_xspi_trigger_command(cdns_xspi, cmd_regs); @@ -723,7 +529,11 @@ static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi, cdns_xspi->set_interrupts_handler(cdns_xspi, false); return -EIO; } - cdns_xspi->sdma_handler(cdns_xspi); + ret = cdns_xspi->sdma_handler(cdns_xspi); + if (ret) { + cdns_xspi->set_interrupts_handler(cdns_xspi, false); + return ret; + } } wait_for_completion(&cdns_xspi->cmd_complete); @@ -761,33 +571,15 @@ static int cdns_xspi_mem_op_execute(struct spi_mem *mem, return ret; } -static int marvell_xspi_mem_op_execute(struct spi_mem *mem, - const struct spi_mem_op *op) -{ - struct cdns_xspi_dev *cdns_xspi = - spi_controller_get_devdata(mem->spi->controller); - int ret = 0; - - cdns_mrvl_xspi_setup_clock(cdns_xspi, mem->spi->max_speed_hz); - - ret = cdns_xspi_mem_op(cdns_xspi, mem, op); - - return ret; -} - -#ifdef CONFIG_ACPI static bool cdns_xspi_supports_op(struct spi_mem *mem, const struct spi_mem_op *op) { struct spi_device *spi = mem->spi; - const union acpi_object *obj; - struct acpi_device *adev; + struct device *dev = &spi->dev; + u32 value; - adev = ACPI_COMPANION(&spi->dev); - - if (!acpi_dev_get_property(adev, "spi-tx-bus-width", ACPI_TYPE_INTEGER, - &obj)) { - switch (obj->integer.value) { + if (!device_property_read_u32(dev, "spi-tx-bus-width", &value)) { + switch (value) { case 1: break; case 2: @@ -800,16 +592,13 @@ static bool cdns_xspi_supports_op(struct spi_mem *mem, spi->mode |= SPI_TX_OCTAL; break; default: - dev_warn(&spi->dev, - "spi-tx-bus-width %lld not supported\n", - obj->integer.value); + dev_warn(dev, "spi-tx-bus-width %u not supported\n", value); break; } } - if (!acpi_dev_get_property(adev, "spi-rx-bus-width", ACPI_TYPE_INTEGER, - &obj)) { - switch (obj->integer.value) { + if (!device_property_read_u32(dev, "spi-rx-bus-width", &value)) { + switch (value) { case 1: break; case 2: @@ -822,9 +611,7 @@ static bool cdns_xspi_supports_op(struct spi_mem *mem, spi->mode |= SPI_RX_OCTAL; break; default: - dev_warn(&spi->dev, - "spi-rx-bus-width %lld not supported\n", - obj->integer.value); + dev_warn(dev, "spi-rx-bus-width %u not supported\n", value); break; } } @@ -834,7 +621,6 @@ static bool cdns_xspi_supports_op(struct spi_mem *mem, return true; } -#endif static int cdns_xspi_adjust_mem_op_size(struct spi_mem *mem, struct spi_mem_op *op) { @@ -847,21 +633,11 @@ static int cdns_xspi_adjust_mem_op_size(struct spi_mem *mem, struct spi_mem_op * } static const struct spi_controller_mem_ops cadence_xspi_mem_ops = { -#ifdef CONFIG_ACPI - .supports_op = cdns_xspi_supports_op, -#endif + .supports_op = PTR_IF(IS_ENABLED(CONFIG_ACPI), cdns_xspi_supports_op), .exec_op = cdns_xspi_mem_op_execute, .adjust_op_size = cdns_xspi_adjust_mem_op_size, }; -static const struct spi_controller_mem_ops marvell_xspi_mem_ops = { -#ifdef CONFIG_ACPI - .supports_op = cdns_xspi_supports_op, -#endif - .exec_op = marvell_xspi_mem_op_execute, - .adjust_op_size = cdns_xspi_adjust_mem_op_size, -}; - static irqreturn_t cdns_xspi_irq_handler(int this_irq, void *dev) { struct cdns_xspi_dev *cdns_xspi = dev; @@ -943,6 +719,239 @@ static void cdns_xspi_print_phy_config(struct cdns_xspi_dev *cdns_xspi) readl(cdns_xspi->auxbase + CDNS_XSPI_CCP_PHY_DLL_SLAVE_CTRL)); } +static struct cdns_xspi_driver_data marvell_driver_data = { + .mrvl_hw_overlay = true, + .dll_phy_ctrl = MARVELL_REGS_DLL_PHY_CTRL, + .ctb_rfile_phy_ctrl = MARVELL_CTB_RFILE_PHY_CTRL, + .rfile_phy_tsel = MARVELL_RFILE_PHY_TSEL, + .rfile_phy_dq_timing = MARVELL_RFILE_PHY_DQ_TIMING, + .rfile_phy_dqs_timing = MARVELL_RFILE_PHY_DQS_TIMING, + .rfile_phy_gate_lpbk_ctrl = MARVELL_RFILE_PHY_GATE_LPBK_CTRL, + .rfile_phy_dll_master_ctrl = MARVELL_RFILE_PHY_DLL_MASTER_CTRL, + .rfile_phy_dll_slave_ctrl = MARVELL_RFILE_PHY_DLL_SLAVE_CTRL, +}; + +static const int cdns_mrvl_xspi_clk_div_list[] = { + 4, //0x0 = Divide by 4. SPI clock is 200 MHz. + 6, //0x1 = Divide by 6. SPI clock is 133.33 MHz. + 8, //0x2 = Divide by 8. SPI clock is 100 MHz. + 10, //0x3 = Divide by 10. SPI clock is 80 MHz. + 12, //0x4 = Divide by 12. SPI clock is 66.666 MHz. + 16, //0x5 = Divide by 16. SPI clock is 50 MHz. + 18, //0x6 = Divide by 18. SPI clock is 44.44 MHz. + 20, //0x7 = Divide by 20. SPI clock is 40 MHz. + 24, //0x8 = Divide by 24. SPI clock is 33.33 MHz. + 32, //0x9 = Divide by 32. SPI clock is 25 MHz. + 40, //0xA = Divide by 40. SPI clock is 20 MHz. + 50, //0xB = Divide by 50. SPI clock is 16 MHz. + 64, //0xC = Divide by 64. SPI clock is 12.5 MHz. + 128 //0xD = Divide by 128. SPI clock is 6.25 MHz. +}; + +static void cdns_xspi_reset_dll(struct cdns_xspi_dev *cdns_xspi) +{ + u32 dll_cntrl = readl(cdns_xspi->iobase + + CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL); + + /* Reset DLL */ + dll_cntrl |= CDNS_XSPI_DLL_RST_N; + writel(dll_cntrl, cdns_xspi->iobase + + CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL); +} + +static bool cdns_xspi_is_dll_locked(struct cdns_xspi_dev *cdns_xspi) +{ + u32 dll_lock; + + return !readl_relaxed_poll_timeout(cdns_xspi->iobase + + CDNS_XSPI_INTR_STATUS_REG, + dll_lock, ((dll_lock & CDNS_XSPI_DLL_LOCK) == 1), 10, 10000); +} + +/* Static configuration of PHY */ +static bool cdns_xspi_configure_phy(struct cdns_xspi_dev *cdns_xspi) +{ + writel(cdns_xspi->driver_data->dll_phy_ctrl, + cdns_xspi->iobase + CDNS_XSPI_RF_MINICTRL_REGS_DLL_PHY_CTRL); + writel(cdns_xspi->driver_data->ctb_rfile_phy_ctrl, + cdns_xspi->auxbase + CDNS_XSPI_PHY_CTB_RFILE_PHY_CTRL); + writel(cdns_xspi->driver_data->rfile_phy_tsel, + cdns_xspi->auxbase + CDNS_XSPI_PHY_CTB_RFILE_PHY_TSEL); + writel(cdns_xspi->driver_data->rfile_phy_dq_timing, + cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DQ_TIMING); + writel(cdns_xspi->driver_data->rfile_phy_dqs_timing, + cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DQS_TIMING); + writel(cdns_xspi->driver_data->rfile_phy_gate_lpbk_ctrl, + cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_GATE_LPBK_CTRL); + writel(cdns_xspi->driver_data->rfile_phy_dll_master_ctrl, + cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DLL_MASTER_CTRL); + writel(cdns_xspi->driver_data->rfile_phy_dll_slave_ctrl, + cdns_xspi->auxbase + CDNS_XSPI_PHY_DATASLICE_RFILE_PHY_DLL_SLAVE_CTRL); + + cdns_xspi_reset_dll(cdns_xspi); + + return cdns_xspi_is_dll_locked(cdns_xspi); +} + +static bool cdns_mrvl_xspi_setup_clock(struct cdns_xspi_dev *cdns_xspi, + int requested_clk) +{ + int i = 0; + int clk_val; + u32 clk_reg; + bool update_clk = false; + + while (i < (ARRAY_SIZE(cdns_mrvl_xspi_clk_div_list) - 1)) { + clk_val = MRVL_XSPI_CLOCK_DIVIDED( + cdns_mrvl_xspi_clk_div_list[i]); + if (clk_val <= requested_clk) + break; + i++; + } + + dev_dbg(cdns_xspi->dev, "Found clk div: %d, clk val: %d\n", + cdns_mrvl_xspi_clk_div_list[i], + MRVL_XSPI_CLOCK_DIVIDED( + cdns_mrvl_xspi_clk_div_list[i])); + + clk_reg = readl(cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG); + + if (FIELD_GET(MRVL_XSPI_CLK_DIV, clk_reg) != i) { + clk_reg &= ~MRVL_XSPI_CLK_ENABLE; + writel(clk_reg, + cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG); + clk_reg = FIELD_PREP(MRVL_XSPI_CLK_DIV, i); + FIELD_MODIFY(MRVL_XSPI_CLK_DIV, &clk_reg, i); + clk_reg |= MRVL_XSPI_CLK_ENABLE; + clk_reg |= MRVL_XSPI_IRQ_ENABLE; + update_clk = true; + } + + if (update_clk) + writel(clk_reg, + cdns_xspi->auxbase + MRVL_XSPI_CLK_CTRL_AUX_REG); + + return update_clk; +} + +static void marvell_xspi_set_interrupts(struct cdns_xspi_dev *cdns_xspi, + bool enabled) +{ + u32 intr_enable; + u32 irq_status; + + irq_status = readl(cdns_xspi->iobase + CDNS_XSPI_INTR_STATUS_REG); + writel(irq_status, cdns_xspi->iobase + CDNS_XSPI_INTR_STATUS_REG); + + intr_enable = readl(cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG); + if (enabled) + intr_enable |= CDNS_XSPI_INTR_MASK; + else + intr_enable &= ~CDNS_XSPI_INTR_MASK; + writel(intr_enable, cdns_xspi->iobase + CDNS_XSPI_INTR_ENABLE_REG); +} + +static int marvell_xspi_mem_op_execute(struct spi_mem *mem, + const struct spi_mem_op *op) +{ + struct cdns_xspi_dev *cdns_xspi = + spi_controller_get_devdata(mem->spi->controller); + int ret = 0; + + cdns_mrvl_xspi_setup_clock(cdns_xspi, mem->spi->max_speed_hz); + + ret = cdns_xspi_mem_op(cdns_xspi, mem, op); + + return ret; +} + +static void m_ioreadq(void __iomem *addr, void *buf, int len) +{ + if (IS_ALIGNED((long)buf, 8) && len >= 8) { + u64 full_ops = len / 8; + u64 *buffer = buf; + + len -= full_ops * 8; + buf += full_ops * 8; + + do { + u64 b = readq(addr); + *buffer++ = b; + } while (--full_ops); + } + + + while (len) { + u64 tmp_buf; + + tmp_buf = readq(addr); + memcpy(buf, &tmp_buf, min(len, 8)); + len = len > 8 ? len - 8 : 0; + buf += 8; + } +} + +static void m_iowriteq(void __iomem *addr, const void *buf, int len) +{ + if (IS_ALIGNED((long)buf, 8) && len >= 8) { + u64 full_ops = len / 8; + const u64 *buffer = buf; + + len -= full_ops * 8; + buf += full_ops * 8; + + do { + writeq(*buffer++, addr); + } while (--full_ops); + } + + while (len) { + u64 tmp_buf; + + memcpy(&tmp_buf, buf, min(len, 8)); + writeq(tmp_buf, addr); + len = len > 8 ? len - 8 : 0; + buf += 8; + } +} + +static int marvell_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi) +{ + u32 sdma_size, sdma_trd_info; + u8 sdma_dir; + + sdma_size = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_SIZE_REG); + sdma_trd_info = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_TRD_INFO_REG); + sdma_dir = FIELD_GET(CDNS_XSPI_SDMA_DIR, sdma_trd_info); + + if (sdma_size > cdns_xspi->sdma_xfer_len) { + dev_err(cdns_xspi->dev, + "SDMA size %u exceeds the requested length %u\n", + sdma_size, cdns_xspi->sdma_xfer_len); + return -EINVAL; + } + + switch (sdma_dir) { + case CDNS_XSPI_SDMA_DIR_READ: + m_ioreadq(cdns_xspi->sdmabase, + cdns_xspi->in_buffer, sdma_size); + break; + + case CDNS_XSPI_SDMA_DIR_WRITE: + m_iowriteq(cdns_xspi->sdmabase, + cdns_xspi->out_buffer, sdma_size); + break; + } + + return 0; +} + +static const struct spi_controller_mem_ops marvell_xspi_mem_ops = { + .supports_op = PTR_IF(IS_ENABLED(CONFIG_ACPI), cdns_xspi_supports_op), + .exec_op = marvell_xspi_mem_op_execute, + .adjust_op_size = cdns_xspi_adjust_mem_op_size, +}; + static int cdns_xspi_prepare_generic(int cs, const void *dout, int len, int glue, u32 *cmd_regs) { u8 *data = (u8 *)dout; @@ -1099,9 +1108,11 @@ static int cdns_xspi_transfer_one_message_b0(struct spi_controller *controller, cdns_xspi_prepare_transfer(cs, 1, current_transfer_len - 1, cmd_regs); cdns_xspi_trigger_command(cdns_xspi, cmd_regs); + cdns_xspi->sdma_xfer_len = current_transfer_len - 1; if (!cdns_xspi_is_sdma_ready(cdns_xspi, true)) return -EIO; - cdns_xspi->sdma_handler(cdns_xspi); + if (cdns_xspi->sdma_handler(cdns_xspi)) + return -EIO; if (!cdns_xspi_is_stig_ready(cdns_xspi, true)) return -EIO; @@ -1170,15 +1181,14 @@ static int cdns_xspi_probe(struct platform_device *pdev) return -ENODEV; } + host->mem_ops = &cadence_xspi_mem_ops; + cdns_xspi->sdma_handler = &cdns_xspi_sdma_handle; + cdns_xspi->set_interrupts_handler = &cdns_xspi_set_interrupts; if (cdns_xspi->driver_data->mrvl_hw_overlay) { host->mem_ops = &marvell_xspi_mem_ops; host->transfer_one_message = cdns_xspi_transfer_one_message_b0; cdns_xspi->sdma_handler = &marvell_xspi_sdma_handle; cdns_xspi->set_interrupts_handler = &marvell_xspi_set_interrupts; - } else { - host->mem_ops = &cadence_xspi_mem_ops; - cdns_xspi->sdma_handler = &cdns_xspi_sdma_handle; - cdns_xspi->set_interrupts_handler = &cdns_xspi_set_interrupts; } host->dev.of_node = pdev->dev.of_node; host->bus_num = -1; diff --git a/drivers/target/target_core_fabric_lib.c b/drivers/target/target_core_fabric_lib.c index 3af9a26cdefc..a1ee106971cb 100644 --- a/drivers/target/target_core_fabric_lib.c +++ b/drivers/target/target_core_fabric_lib.c @@ -370,6 +370,16 @@ static bool iscsi_parse_pr_out_transport_id( pr_err("Missing ISID in iSCSI Initiator port TransportID\n"); return false; } + /* + * The registration code reads the ISID as a C string, + * and its first 8 bytes with get_unaligned_be64(), so + * require 8 bytes after the separator and a NUL inside + * the rest of the descriptor. + */ + if (buf + tid_len - p < 8 || !memchr(p, '\0', buf + tid_len - p)) { + pr_err("Incomplete or unterminated ISID in iSCSI Initiator port TransportID\n"); + return false; + } *port_nexus_ptr = p; /* * Go ahead and do the lower case conversion of the received diff --git a/drivers/target/target_core_user.c b/drivers/target/target_core_user.c index 717931267bda..6b9adab3d11e 100644 --- a/drivers/target/target_core_user.c +++ b/drivers/target/target_core_user.c @@ -2514,6 +2514,11 @@ static int tcmu_set_data_pages_per_blk(struct tcmu_dev *udev, substring_t *arg) return ret; } + if (val <= 0) { + pr_err("Invalid data_pages_per_blk %d.\n", val); + return -EINVAL; + } + if (val > TCMU_MBS_TO_PAGES(udev->data_area_mb)) { pr_err("Invalid data_pages_per_blk %d: greater than max_data_area_mb %d -> %zd pages).\n", val, udev->data_area_mb, diff --git a/drivers/tty/serial/amba-pl011.c b/drivers/tty/serial/amba-pl011.c index 3e1932a3bfbb..a0e20bffefba 100644 --- a/drivers/tty/serial/amba-pl011.c +++ b/drivers/tty/serial/amba-pl011.c @@ -2224,8 +2224,21 @@ static int pl011_rs485_config(struct uart_port *port, struct ktermios *termios, struct uart_amba_port *uap = container_of(port, struct uart_amba_port, port); - if (port->rs485.flags & SER_RS485_ENABLED) + if ((port->rs485.flags & SER_RS485_ENABLED) && + !(rs485->flags & SER_RS485_ENABLED)) { + u32 cr = pl011_read(uap, REG_CR); + + /* + * Leaving RS485 mode, which only enables the transmitter while + * sending. Don't stop it, as that stalls a DMA transfer still + * in flight. Keep the transmitter and receiver enabled. + */ + uap->rs485_tx_started = false; + cr |= UART011_CR_TXE | UART011_CR_RXE; + pl011_write(cr, uap, REG_CR); + } else if (port->rs485.flags & SER_RS485_ENABLED) { pl011_rs485_tx_stop(uap); + } /* Make sure auto RTS is disabled */ if (rs485->flags & SER_RS485_ENABLED) { diff --git a/drivers/tty/serial/sc16is7xx.c b/drivers/tty/serial/sc16is7xx.c index 942d12f40a39..f104d6a7a3fd 100644 --- a/drivers/tty/serial/sc16is7xx.c +++ b/drivers/tty/serial/sc16is7xx.c @@ -535,10 +535,13 @@ EXPORT_SYMBOL_GPL(sc16is762_devtype); static bool sc16is7xx_regmap_volatile(struct device *dev, unsigned int reg) { switch (reg) { - case SC16IS7XX_RHR_REG: - case SC16IS7XX_IIR_REG: - case SC16IS7XX_LSR_REG: - case SC16IS7XX_MSR_REG: + case SC16IS7XX_RHR_REG: /* Shared address space with THR & DLL */ + case SC16IS7XX_IER_REG: /* Shared address space with DLH */ + case SC16IS7XX_IIR_REG: /* Shared address space with FCR & EFR */ + case SC16IS7XX_MCR_REG: /* Shared address space with XON1 */ + case SC16IS7XX_LSR_REG: /* Shared address space with XON2 */ + case SC16IS7XX_MSR_REG: /* Shared address space with TCR & XOFF1 */ + case SC16IS7XX_SPR_REG: /* Shared address space with TLR & XOFF2 */ case SC16IS7XX_TXLVL_REG: case SC16IS7XX_RXLVL_REG: case SC16IS7XX_IOSTATE_REG: @@ -781,6 +784,7 @@ static void sc16is7xx_update_mlines(struct sc16is7xx_one *one) unsigned long flags; unsigned int status, changed; + /* Lock required as MSR address is shared with TCR and XOFF1. */ lockdep_assert_held_once(&one->lock); status = sc16is7xx_get_hwmctrl(port); @@ -1615,7 +1619,7 @@ int sc16is7xx_probe(struct device *dev, const struct sc16is7xx_devtype *devtype, s->p[i].port.irq = irq; s->p[i].port.type = PORT_SC16IS7XX; s->p[i].port.fifosize = SC16IS7XX_FIFO_SIZE; - s->p[i].port.flags = UPF_FIXED_TYPE | UPF_LOW_LATENCY; + s->p[i].port.flags = UPF_FIXED_TYPE | UPF_LOW_LATENCY | UPF_SOFT_FLOW; s->p[i].port.iobase = i; /* * Use all ones as membase to make sure uart_configure_port() in diff --git a/drivers/ufs/core/ufs_bsg.c b/drivers/ufs/core/ufs_bsg.c index 0e0b08a3821c..59a473261ae1 100644 --- a/drivers/ufs/core/ufs_bsg.c +++ b/drivers/ufs/core/ufs_bsg.c @@ -79,6 +79,9 @@ static int ufs_bsg_exec_advanced_rpmb_req(struct ufs_hba *hba, struct bsg_job *j if (hba->ufs_version < ufshci_version(4, 0) || !hba->dev_info.b_advanced_rpmb_en) return -EINVAL; + if (job->request_len < sizeof(*rpmb_request)) + return -EINVAL; + if (rpmb_request->ehs_req.length != 2 || rpmb_request->ehs_req.ehs_type != 1) return -EINVAL; @@ -144,6 +147,11 @@ static int ufs_bsg_request(struct bsg_job *job) memset(job->reply, 0, job->reply_len); + if (job->request_len < sizeof(*bsg_request)) { + ret = -EINVAL; + goto out_complete; + } + ufshcd_rpm_get_sync(hba); msgcode = bsg_request->msgcode; @@ -170,6 +178,10 @@ static int ufs_bsg_request(struct bsg_job *job) break; case UPIU_TRANSACTION_UIC_CMD: memcpy(&uc, &bsg_request->upiu_req.uc, UIC_CMD_SIZE); + if (uc.argument2 & MASK_UIC_COMMAND_RESULT) { + ret = -EINVAL; + break; + } ret = ufshcd_send_bsg_uic_cmd(hba, &uc); if (ret) dev_err(hba->dev, "send uic cmd: error code %d\n", ret); @@ -193,6 +205,7 @@ static int ufs_bsg_request(struct bsg_job *job) out: ufshcd_rpm_put_sync(hba); kfree(buff); +out_complete: bsg_reply->result = ret; if (ret == 0) job->reply_len = rpmb ? sizeof(struct ufs_rpmb_reply) : diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 4a78a5d9cd8b..798ad0338cd9 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -5653,6 +5653,37 @@ static void ufshcd_clear_polled(struct ufs_hba *hba, } } +/* + * Return a bitmask of the transfer requests cleared from + * @hba->outstanding_reqs. + */ +static unsigned long __ufshcd_clear_outstanding_reqs(struct ufs_hba *hba, + unsigned long mask, + bool from_irq) + __must_hold(&hba->outstanding_lock) +{ + unsigned long completed_reqs = mask & hba->outstanding_reqs; + + if (from_irq) { + /* Do not complete polled requests from interrupt context. */ + ufshcd_clear_polled(hba, &completed_reqs); + } + hba->outstanding_reqs &= ~completed_reqs; + + return completed_reqs; +} + +/* + * Return a bitmask of the transfer requests cleared from + * @hba->outstanding_reqs. + */ +static unsigned long ufshcd_clear_outstanding_reqs(struct ufs_hba *hba, + unsigned long mask) +{ + guard(spinlock_irqsave)(&hba->outstanding_lock); + return __ufshcd_clear_outstanding_reqs(hba, mask, false); +} + /* * Return: > 0 if one or more commands have been completed or 0 if no * requests have been completed. @@ -5663,6 +5694,7 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num) unsigned long completed_reqs, flags; u32 tr_doorbell; struct ufs_hw_queue *hwq; + bool from_irq; if (hba->mcq_enabled) { hwq = &hba->uhq[queue_num]; @@ -5672,12 +5704,9 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num) spin_lock_irqsave(&hba->outstanding_lock, flags); tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); - completed_reqs = ~tr_doorbell & hba->outstanding_reqs; - if (queue_num == UFSHCD_POLL_FROM_INTERRUPT_CONTEXT) { - /* Do not complete polled requests from interrupt context. */ - ufshcd_clear_polled(hba, &completed_reqs); - } - hba->outstanding_reqs &= ~completed_reqs; + from_irq = queue_num == UFSHCD_POLL_FROM_INTERRUPT_CONTEXT; + completed_reqs = __ufshcd_clear_outstanding_reqs(hba, ~tr_doorbell, + from_irq); spin_unlock_irqrestore(&hba->outstanding_lock, flags); if (completed_reqs) @@ -7662,7 +7691,6 @@ static int ufshcd_abort(struct scsi_cmnd *cmd) struct ufshcd_lrb *lrbp = &hba->lrb[tag]; unsigned long flags; int err = FAILED; - bool outstanding; u32 reg; ufshcd_hold(hba); @@ -7743,11 +7771,7 @@ static int ufshcd_abort(struct scsi_cmnd *cmd) * Clear the corresponding bit from outstanding_reqs since the command * has been aborted successfully. */ - spin_lock_irqsave(&hba->outstanding_lock, flags); - outstanding = __test_and_clear_bit(tag, &hba->outstanding_reqs); - spin_unlock_irqrestore(&hba->outstanding_lock, flags); - - if (outstanding) + if (ufshcd_clear_outstanding_reqs(hba, BIT(tag))) ufshcd_release_scsi_cmd(hba, lrbp); err = SUCCESS; diff --git a/drivers/usb/core/config.c b/drivers/usb/core/config.c index 43f1419d6617..1dbff5398458 100644 --- a/drivers/usb/core/config.c +++ b/drivers/usb/core/config.c @@ -891,6 +891,18 @@ int usb_get_configuration(struct usb_device *dev) unsigned char *bigbuffer; struct usb_config_descriptor *desc; int result; + size_t usb_config_req_size; + + /* + * We usually start by grabbing the first 9-bytes descriptor so we know + * how long the whole configuration is. Some devices with quirky + * firmware will fail enumeration, so if the quirk is set, use 255 instead, + * mirroring the behavior of Windows. + */ + if (dev->quirks & USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE) + usb_config_req_size = 255; + else + usb_config_req_size = USB_DT_CONFIG_SIZE; if (ncfg > USB_MAXCONFIG) { dev_notice(ddev, "too many configurations: %d, " @@ -917,15 +929,13 @@ int usb_get_configuration(struct usb_device *dev) if (!dev->rawdescriptors) return -ENOMEM; - desc = kmalloc(USB_DT_CONFIG_SIZE, GFP_KERNEL); + desc = kmalloc(usb_config_req_size, GFP_KERNEL); if (!desc) return -ENOMEM; for (cfgno = 0; cfgno < ncfg; cfgno++) { - /* We grab just the first descriptor so we know how long - * the whole configuration is */ result = usb_get_descriptor(dev, USB_DT_CONFIG, cfgno, - desc, USB_DT_CONFIG_SIZE); + desc, usb_config_req_size); if (result < 0) { dev_err(ddev, "unable to read config index %d " "descriptor/%s: %d\n", cfgno, "start", result); @@ -935,16 +945,14 @@ int usb_get_configuration(struct usb_device *dev) dev->descriptor.bNumConfigurations = cfgno; break; } else if (result < 4) { - dev_err(ddev, "config index %d descriptor too short " - "(expected %i, got %i)\n", cfgno, - USB_DT_CONFIG_SIZE, result); + dev_err(ddev, "config index %d descriptor too short (asked for %zu, got %i)\n", + cfgno, usb_config_req_size, result); result = -EINVAL; goto err; } length = max((int) le16_to_cpu(desc->wTotalLength), USB_DT_CONFIG_SIZE); - /* Now that we know the length, get the whole thing */ bigbuffer = kmalloc(length, GFP_KERNEL); if (!bigbuffer) { result = -ENOMEM; @@ -954,6 +962,13 @@ int usb_get_configuration(struct usb_device *dev) if (dev->quirks & USB_QUIRK_DELAY_INIT) msleep(200); + /* Skip the second read if we already got everything */ + if (result >= length) { + memcpy(bigbuffer, desc, length); + goto store_and_parse; + } + + /* Get the whole thing */ result = usb_get_descriptor(dev, USB_DT_CONFIG, cfgno, bigbuffer, length); if (result < 0) { @@ -968,6 +983,7 @@ int usb_get_configuration(struct usb_device *dev) length = result; } +store_and_parse: dev->rawdescriptors[cfgno] = bigbuffer; result = usb_parse_configuration(dev, cfgno, diff --git a/drivers/usb/core/quirks.c b/drivers/usb/core/quirks.c index 6ed380599405..ee439a2ddf31 100644 --- a/drivers/usb/core/quirks.c +++ b/drivers/usb/core/quirks.c @@ -143,6 +143,10 @@ static int quirks_param_set(const char *value, const struct kernel_param *kp) break; case 'q': flags |= USB_QUIRK_FORCE_ONE_CONFIG; + break; + case 'r': + flags |= USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE; + break; /* Ignore unrecognized flag characters */ } } @@ -603,6 +607,9 @@ static const struct usb_device_id usb_quirk_list[] = { /* ezcap401 - BOS descriptor fetch hangs at SuperSpeed Plus */ { USB_DEVICE(0x32ed, 0x0401), .driver_info = USB_QUIRK_NO_BOS }, + /* GameSir G7 SE */ + { USB_DEVICE(0x3537, 0x10a0), .driver_info = USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE }, + /* DELL USB GEN2 */ { USB_DEVICE(0x413c, 0xb062), .driver_info = USB_QUIRK_NO_LPM | USB_QUIRK_RESET_RESUME }, diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c index 597a9a8c16b9..2c35cb444f73 100644 --- a/drivers/usb/host/xhci-ring.c +++ b/drivers/usb/host/xhci-ring.c @@ -3583,15 +3583,13 @@ static u32 xhci_td_remainder(struct xhci_hcd *xhci, int transferred, static int xhci_align_td(struct xhci_hcd *xhci, struct urb *urb, u32 enqd_len, - u32 *trb_buff_len, struct xhci_segment *seg) + u32 *trb_buff_len, struct xhci_segment *seg, u32 max_pkt) { struct device *dev = xhci_to_hcd(xhci)->self.sysdev; unsigned int unalign; - unsigned int max_pkt; u32 new_buff_len; size_t len; - max_pkt = usb_endpoint_maxp(&urb->ep->desc); unalign = (enqd_len + *trb_buff_len) % max_pkt; /* we got lucky, last normal TRB data on segment is packet aligned */ @@ -3736,9 +3734,8 @@ int xhci_queue_bulk_tx(struct xhci_hcd *xhci, gfp_t mem_flags, if (enqd_len + trb_buff_len < full_len) { field |= TRB_CHAIN; if (trb_is_link(ring->enqueue + 1)) { - if (xhci_align_td(xhci, urb, enqd_len, - &trb_buff_len, - ring->enq_seg)) { + if (xhci_align_td(xhci, urb, enqd_len, &trb_buff_len, + ring->enq_seg, ring->bounce_buf_len)) { send_addr = ring->enq_seg->bounce_dma; /* TD bounced at least, and last on this seg */ td->bounce_seg = ring->enq_seg; diff --git a/drivers/usb/typec/ucsi/ucsi_ccg.c b/drivers/usb/typec/ucsi/ucsi_ccg.c index 3a4e1b6f22ec..fbb65b734fc1 100644 --- a/drivers/usb/typec/ucsi/ucsi_ccg.c +++ b/drivers/usb/typec/ucsi/ucsi_ccg.c @@ -397,14 +397,28 @@ static int ucsi_ccg_init(struct ucsi_ccg *uc) return -ETIMEDOUT; } -static void ucsi_ccg_update_get_current_cam_cmd(struct ucsi_ccg *uc, u8 *data) +static int ucsi_ccg_update_get_current_cam_cmd(struct ucsi_ccg *uc, u8 *data) { u8 cam, new_cam; cam = data[0]; + if (cam >= UCSI_MAX_ALTMODES) { + dev_err(uc->dev, "PPM returned invalid alternate mode index %u\n", + cam); + return -EPROTO; + } + new_cam = uc->orig[cam].linked_idx; + if (new_cam >= UCSI_MAX_ALTMODES) { + dev_err(uc->dev, "invalid linked alternate mode index %u for %u\n", + new_cam, cam); + return -EPROTO; + } + uc->updated[new_cam].active_idx = cam; data[0] = new_cam; + + return 0; } static bool ucsi_ccg_update_altmodes(struct ucsi *ucsi, @@ -593,6 +607,7 @@ static int ucsi_ccg_read_message_in(struct ucsi *ucsi, void *val, size_t val_len struct ucsi_ccg *uc = ucsi_get_drvdata(ucsi); struct ucsi_capability *cap; struct ucsi_altmode *alt; + int ret = 0; spin_lock(&uc->op_lock); memcpy(val, uc->op_data.message_in, val_len); @@ -601,7 +616,7 @@ static int ucsi_ccg_read_message_in(struct ucsi *ucsi, void *val, size_t val_len switch (UCSI_COMMAND(uc->last_cmd_sent)) { case UCSI_GET_CURRENT_CAM: if (uc->has_multiple_dp) - ucsi_ccg_update_get_current_cam_cmd(uc, (u8 *)val); + ret = ucsi_ccg_update_get_current_cam_cmd(uc, (u8 *)val); break; case UCSI_GET_ALTERNATE_MODES: if (UCSI_ALTMODE_RECIPIENT(uc->last_cmd_sent) == @@ -622,7 +637,7 @@ static int ucsi_ccg_read_message_in(struct ucsi *ucsi, void *val, size_t val_len } uc->last_cmd_sent = 0; - return 0; + return ret; } static int ucsi_ccg_async_control(struct ucsi *ucsi, u64 command) diff --git a/drivers/vdpa/solidrun/snet_ctrl.c b/drivers/vdpa/solidrun/snet_ctrl.c index 3cef2571d15d..1c3b5f2e4499 100644 --- a/drivers/vdpa/solidrun/snet_ctrl.c +++ b/drivers/vdpa/solidrun/snet_ctrl.c @@ -291,7 +291,6 @@ static int snet_send_ctrl_msg(struct snet *snet, u16 opcode, u16 vq_idx) /* Check for errors */ val = snet_read_ctrl(regs); - ret = SNET_VAL_TO_ERR(val); /* Clear the chunk ready bit */ val &= ~SNET_CTRL_CHUNK_RDY_MASK; @@ -301,6 +300,8 @@ static int snet_send_ctrl_msg(struct snet *snet, u16 opcode, u16 vq_idx) if (ret) SNET_WARN(pdev, "Timeout waiting for DPU to complete a control command, err %d\n", ret); + else + ret = SNET_VAL_TO_ERR(val); exit: mutex_unlock(&snet->ctrl_lock); diff --git a/drivers/vdpa/vdpa_user/iova_domain.c b/drivers/vdpa/vdpa_user/iova_domain.c index e8b837df328f..b8b2eb44410d 100644 --- a/drivers/vdpa/vdpa_user/iova_domain.c +++ b/drivers/vdpa/vdpa_user/iova_domain.c @@ -76,6 +76,10 @@ int vduse_domain_set_map(struct vduse_iova_domain *domain, for (map = vhost_iotlb_itree_first(iotlb, start, last); map; map = vhost_iotlb_itree_next(map, start, last)) { map_file = (struct vdpa_map_file *)map->opaque; + if (!map_file || map_file->file == domain->file) { + ret = -EINVAL; + goto err; + } ret = vduse_iotlb_add_range(domain, map->start, map->last, map->addr, map->perm, map_file->file, @@ -226,6 +230,7 @@ vduse_domain_get_coherent_page(struct vduse_iova_domain *domain, u64 iova) { u64 start = iova & PAGE_MASK; u64 last = start + PAGE_SIZE - 1; + struct vdpa_map_file *map_file; struct vhost_iotlb_map *map; struct page *page = NULL; @@ -234,6 +239,10 @@ vduse_domain_get_coherent_page(struct vduse_iova_domain *domain, u64 iova) if (!map) goto out; + map_file = (struct vdpa_map_file *)map->opaque; + if (!map_file || map_file->file != domain->file) + goto out; + page = pfn_to_page((map->addr + iova - map->start) >> PAGE_SHIFT); get_page(page); out: @@ -579,7 +588,7 @@ static int vduse_domain_mmap(struct file *file, struct vm_area_struct *vma) { struct vduse_iova_domain *domain = file->private_data; - vm_flags_set(vma, VM_DONTDUMP | VM_DONTEXPAND); + vm_flags_set(vma, VM_IO | VM_DONTDUMP | VM_DONTEXPAND); vma->vm_private_data = domain; vma->vm_ops = &vduse_domain_mmap_ops; diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index a940718745de..33e3100b3836 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -1151,7 +1151,7 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, unsigned long list_size = PAGE_SIZE / sizeof(struct page *); unsigned int gup_flags = FOLL_LONGTERM; unsigned long npages, cur_base, map_pfn, last_pfn = 0; - unsigned long lock_limit, sz2pin, nchunks, i; + unsigned long lock_limit, sz2pin, i; unsigned long page_offset; u64 start = iova; long pinned; @@ -1187,7 +1187,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, cur_base = uaddr & PAGE_MASK; iova &= PAGE_MASK; - nchunks = 0; while (npages) { sz2pin = min_t(unsigned long, npages, list_size); @@ -1202,7 +1201,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, } goto out; } - nchunks++; if (!last_pfn) map_pfn = page_to_pfn(page_list[0]); @@ -1233,7 +1231,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, map_pfn = this_pfn; iova += csize; - nchunks = 0; } last_pfn = this_pfn; @@ -1248,7 +1245,7 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, PFN_PHYS(map_pfn), perm, NULL); out: if (ret) { - if (nchunks) { + if (last_pfn) { unsigned long pfn; /* @@ -1260,7 +1257,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, * the corresponding unpinning will be handled by * vdpa_unmap(). */ - WARN_ON(!last_pfn); for (pfn = map_pfn; pfn <= last_pfn; pfn++) unpin_user_page(pfn_to_page(pfn)); } diff --git a/drivers/vhost/vringh.c b/drivers/vhost/vringh.c index 781731eb95cf..db012d6b549e 100644 --- a/drivers/vhost/vringh.c +++ b/drivers/vhost/vringh.c @@ -197,8 +197,9 @@ static int move_to_indirect(const struct vringh *vrh, } len = vringh32_to_cpu(vrh, desc->len); - if (unlikely(len % sizeof(struct vring_desc))) { - vringh_bad("Strange indirect len %u", desc->len); + if (unlikely(len < sizeof(struct vring_desc) || + len % sizeof(struct vring_desc))) { + vringh_bad("Invalid indirect len %u", desc->len); return -EINVAL; } @@ -334,6 +335,17 @@ __vringh_iov(struct vringh *vrh, u16 i, if (unlikely(err)) goto fail; + if (up_next == -1) + count++; + else + indirect_count++; + + if (count > vrh->vring.num || indirect_count > desc_max) { + vringh_bad("Descriptor loop in %p", descs); + err = -ELOOP; + goto fail; + } + if (unlikely(desc.flags & cpu_to_vringh16(vrh, VRING_DESC_F_INDIRECT))) { u64 a = vringh64_to_cpu(vrh, desc.addr); @@ -359,17 +371,6 @@ __vringh_iov(struct vringh *vrh, u16 i, continue; } - if (up_next == -1) - count++; - else - indirect_count++; - - if (count > vrh->vring.num || indirect_count > desc_max) { - vringh_bad("Descriptor loop in %p", descs); - err = -ELOOP; - goto fail; - } - if (desc.flags & cpu_to_vringh16(vrh, VRING_DESC_F_WRITE)) iov = wiov; else { diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c index 65297f245b27..f00182e9efec 100644 --- a/drivers/vhost/vsock.c +++ b/drivers/vhost/vsock.c @@ -331,7 +331,7 @@ static struct sk_buff * vhost_vsock_alloc_skb(struct vhost_virtqueue *vq, unsigned int out, unsigned int in) { - struct virtio_vsock_hdr *hdr; + struct virtio_vsock_hdr hdr; struct iov_iter iov_iter; struct sk_buff *skb; size_t payload_len; @@ -349,34 +349,32 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq, len > VIRTIO_VSOCK_MAX_PKT_BUF_SIZE + VIRTIO_VSOCK_SKB_HEADROOM) return NULL; - /* len contains both payload and hdr */ - skb = virtio_vsock_alloc_skb(len, GFP_KERNEL); - if (!skb) - return NULL; - iov_iter_init(&iov_iter, ITER_SOURCE, vq->iov, out, len); - hdr = virtio_vsock_hdr(skb); - nbytes = copy_from_iter(hdr, sizeof(*hdr), &iov_iter); - if (nbytes != sizeof(*hdr)) { + nbytes = copy_from_iter(&hdr, sizeof(hdr), &iov_iter); + if (nbytes != sizeof(hdr)) { vq_err(vq, "Expected %zu bytes for pkt->hdr, got %zu bytes\n", - sizeof(*hdr), nbytes); - kfree_skb(skb); + sizeof(hdr), nbytes); return NULL; } - payload_len = le32_to_cpu(hdr->len); + payload_len = le32_to_cpu(hdr.len); + + /* The pkt is too big or the length in the header is invalid */ + if (payload_len > len - sizeof(hdr)) + return NULL; + + /* Allocate only for the payload declared in the header. */ + skb = virtio_vsock_alloc_skb(payload_len + sizeof(hdr), GFP_KERNEL); + if (!skb) + return NULL; + + memcpy(virtio_vsock_hdr(skb), &hdr, sizeof(hdr)); /* No payload */ if (!payload_len) return skb; - /* The pkt is too big or the length in the header is invalid */ - if (payload_len + sizeof(*hdr) > len) { - kfree_skb(skb); - return NULL; - } - virtio_vsock_skb_put(skb, payload_len); if (skb_copy_datagram_from_iter(skb, 0, &iov_iter, payload_len)) { diff --git a/drivers/video/fbdev/core/fb_info.c b/drivers/video/fbdev/core/fb_info.c index 4847ebe50d7d..9b36361bda6e 100644 --- a/drivers/video/fbdev/core/fb_info.c +++ b/drivers/video/fbdev/core/fb_info.c @@ -42,6 +42,7 @@ struct fb_info *framebuffer_alloc(size_t size, struct device *dev) info->device = dev; info->fbcon_rotate_hint = -1; + INIT_LIST_HEAD(&info->modelist); #if IS_ENABLED(CONFIG_FB_BACKLIGHT) mutex_init(&info->bl_curve_mutex); diff --git a/drivers/video/fbdev/core/fbcon_rotate.c b/drivers/video/fbdev/core/fbcon_rotate.c index a3f507825eed..3105682075c8 100644 --- a/drivers/video/fbdev/core/fbcon_rotate.c +++ b/drivers/video/fbdev/core/fbcon_rotate.c @@ -27,7 +27,8 @@ static int fbcon_rotate_font(struct fb_info *info, struct vc_data *vc) u8 *dst; if (vc->vc_font.data == par->fontdata && - par->p->con_rotate == par->cur_rotate) + par->p->con_rotate == par->cur_rotate && + par->fontbuffer) goto finished; src = par->fontdata = vc->vc_font.data; diff --git a/drivers/video/fbdev/nvidia/nvidia.c b/drivers/video/fbdev/nvidia/nvidia.c index 5f61d953e4c7..bd4a6dd573dd 100644 --- a/drivers/video/fbdev/nvidia/nvidia.c +++ b/drivers/video/fbdev/nvidia/nvidia.c @@ -1288,7 +1288,7 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent) NVTRACE_ENTER(); assert(pd != NULL); - if (pci_enable_device(pd)) { + if (pcim_enable_device(pd)) { printk(KERN_ERR PFX "cannot enable PCI device\n"); return -ENODEV; } @@ -1301,7 +1301,7 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent) nvidiafb_fix.mmio_start = pci_resource_start(pd, 0); nvidiafb_fix.mmio_len = pci_resource_len(pd, 0); - REGS = ioremap(nvidiafb_fix.mmio_start, nvidiafb_fix.mmio_len); + REGS = devm_ioremap(&pd->dev, nvidiafb_fix.mmio_start, nvidiafb_fix.mmio_len); if (!REGS) { printk(KERN_ERR PFX "cannot ioremap MMIO base\n"); return -ENODEV; @@ -1329,7 +1329,7 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent) if (info->pixmap.addr == NULL) goto err_out_kfree; - if (pci_request_regions(pd, "nvidiafb")) { + if (pcim_request_all_regions(pd, "nvidiafb")) { printk(KERN_ERR PFX "cannot request PCI regions\n"); goto err_out_enable; } @@ -1354,7 +1354,7 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent) sprintf(nvidiafb_fix.id, "NV%x", (pd->device & 0x0ff0) >> 4); if (NVCommonSetup(info)) - goto err_out_free_base0; + goto err_out_enable; par->FbAddress = nvidiafb_fix.smem_start; par->FbMapSize = par->RamAmountKBytes * 1024; @@ -1374,8 +1374,8 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent) par->ScratchBufferStart = par->FbUsableSize - par->ScratchBufferSize; par->CursorStart = par->FbUsableSize + (32 * 1024); - info->screen_base = ioremap_wc(nvidiafb_fix.smem_start, - par->FbMapSize); + info->screen_base = devm_ioremap_wc(&pd->dev, nvidiafb_fix.smem_start, + par->FbMapSize); info->screen_size = par->FbUsableSize; nvidiafb_fix.smem_len = par->RamAmountKBytes * 1024; @@ -1419,19 +1419,16 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent) return 0; err_out_iounmap_fb: - iounmap(info->screen_base); + arch_phys_wc_del(par->wc_cookie); fb_destroy_modelist(&info->modelist); err_out_free_base1: fb_destroy_modedb(info->monspecs.modedb); nvidia_delete_i2c_busses(par); -err_out_free_base0: - pci_release_regions(pd); err_out_enable: kfree(info->pixmap.addr); err_out_kfree: framebuffer_release(info); err_out: - iounmap(REGS); return -ENODEV; } @@ -1446,11 +1443,8 @@ static void nvidiafb_remove(struct pci_dev *pd) unregister_framebuffer(info); arch_phys_wc_del(par->wc_cookie); - iounmap(info->screen_base); fb_destroy_modedb(info->monspecs.modedb); nvidia_delete_i2c_busses(par); - iounmap(par->REGS); - pci_release_regions(pd); kfree(info->pixmap.addr); framebuffer_release(info); NVTRACE_LEAVE(); diff --git a/drivers/watchdog/hpwdt.c b/drivers/watchdog/hpwdt.c index ae30e394d176..32e9ac24e0d5 100644 --- a/drivers/watchdog/hpwdt.c +++ b/drivers/watchdog/hpwdt.c @@ -22,6 +22,7 @@ #include #ifdef CONFIG_HPWDT_NMI_DECODING #include +#include #endif #include @@ -158,24 +159,31 @@ static int hpwdt_set_pretimeout(struct watchdog_device *wdd, unsigned int req) return 0; } -static int hpwdt_my_nmi(void) -{ - return ioread8(hpwdt_nmistat) & 0x6; -} +#define NMISTAT_EASR BIT(0) +#define NMISTAT_EWDOG BIT(1) +#define NMISTAT_RTRAP BIT(2) +#define NMISTAT_DBELL BIT(3) +#define NMISTAT_EMSWDG BIT(4) +#define NMISTAT_GPI BIT(5) +#define NMISTAT_NMIOUT BIT(7) /* * NMI Handler */ static int hpwdt_pretimeout(unsigned int ulReason, struct pt_regs *regs) { - unsigned int mynmi = hpwdt_my_nmi(); - static char panic_msg[] = + u8 nmistat = ioread8(hpwdt_nmistat); + bool mynmi = (nmistat & (NMISTAT_EWDOG | NMISTAT_RTRAP)) != 0; + static char panic_msg_default[] = "00: An NMI occurred. Depending on your system the reason " "for the NMI is logged in any one of the following resources:\n" "1. Integrated Management Log (IML)\n" "2. OA Syslog\n" "3. OA Forward Progress Log\n" "4. iLO Event Log"; + static char panic_msg_uv[] = + "00: A watchdog NMI occurred."; + char *panic_msg = is_uv_system() ? panic_msg_uv : panic_msg_default; if (ulReason == NMI_UNKNOWN && !mynmi) return NMI_DONE; @@ -189,8 +197,8 @@ static int hpwdt_pretimeout(unsigned int ulReason, struct pt_regs *regs) hpwdt_ping_ticks(SECS_TO_TICKS(val)); } - hex_byte_pack(panic_msg, mynmi); - nmi_panic(regs, panic_msg); + hex_byte_pack(panic_msg, nmistat); + nmi_panic(regs, "%s", panic_msg); return NMI_HANDLED; } diff --git a/drivers/xen/gntdev-dmabuf.c b/drivers/xen/gntdev-dmabuf.c index 5ab973627d18..1072e781f12e 100644 --- a/drivers/xen/gntdev-dmabuf.c +++ b/drivers/xen/gntdev-dmabuf.c @@ -96,8 +96,10 @@ dmabuf_exp_wait_obj_new(struct gntdev_dmabuf_priv *priv, struct gntdev_dmabuf_wait_obj *obj; obj = kzalloc(sizeof(*obj), GFP_KERNEL); - if (!obj) + if (!obj) { + kref_put(&gntdev_dmabuf->u.exp.refcount, dmabuf_exp_release); return ERR_PTR(-ENOMEM); + } init_completion(&obj->completion); obj->gntdev_dmabuf = gntdev_dmabuf; diff --git a/fs/9p/v9fs.c b/fs/9p/v9fs.c index bc879d32cfcf..66b072152745 100644 --- a/fs/9p/v9fs.c +++ b/fs/9p/v9fs.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include @@ -555,21 +556,14 @@ static ssize_t caches_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { - ssize_t n = 0, count = 0, limit = PAGE_SIZE; + ssize_t count = 0; struct v9fs_session_info *v9ses; spin_lock(&v9fs_sessionlist_lock); list_for_each_entry(v9ses, &v9fs_sessionlist, slist) { - if (v9ses->cachetag) { - n = snprintf(buf + count, limit, "%s\n", v9ses->cachetag); - if (n < 0) { - count = n; - break; - } - - count += n; - limit -= n; - } + if (v9ses->cachetag) + count += sysfs_emit_at(buf, count, "%s\n", + v9ses->cachetag); } spin_unlock(&v9fs_sessionlist_lock); diff --git a/fs/9p/vfs_inode.c b/fs/9p/vfs_inode.c index 90fe3ea09595..7cb3f4b89dcc 100644 --- a/fs/9p/vfs_inode.c +++ b/fs/9p/vfs_inode.c @@ -493,6 +493,12 @@ static int v9fs_at_to_dotl_flags(int flags) * nothing and races with concurrent metadata fetches that may already * have observed the post-unlink value (nlink == 0). * + * The same applies to a regular file whose cached nlink has already been + * driven to zero by the server: in the cached modes the attributes come + * from GETATTR and are applied verbatim, so decrementing a link count the + * server does not maintain only trips the drop_nlink() WARN_ON and + * underflows i_nlink. + * * @inode: inode whose nlink is being dropped */ static void v9fs_dec_count(struct inode *inode) @@ -501,6 +507,8 @@ static void v9fs_dec_count(struct inode *inode) if (!(v9ses->cache & (CACHE_META | CACHE_LOOSE))) return; + if (inode->i_nlink == 0) + return; if (!S_ISDIR(inode->i_mode) || inode->i_nlink > 2) drop_nlink(inode); } diff --git a/fs/afs/rxrpc.c b/fs/afs/rxrpc.c index 55449c879142..934667e547dc 100644 --- a/fs/afs/rxrpc.c +++ b/fs/afs/rxrpc.c @@ -442,10 +442,8 @@ void afs_make_call(struct afs_call *call, gfp_t gfp) return; error_do_abort: - if (ret != -ECONNABORTED) - rxrpc_kernel_abort_call(call->net->socket, rxcall, - RX_USER_ABORT, ret, - afs_abort_send_data_error); + rxrpc_kernel_abort_call(call->net->socket, rxcall, + RX_USER_ABORT, ret, afs_abort_send_data_error); if (call->async) { afs_see_call(call, afs_call_trace_async_abort); return; @@ -853,6 +851,7 @@ void afs_send_empty_reply(struct afs_call *call) { struct afs_net *net = call->net; struct msghdr msg; + int ret; _enter(""); @@ -865,21 +864,13 @@ void afs_send_empty_reply(struct afs_call *call) msg.msg_controllen = 0; msg.msg_flags = 0; - switch (rxrpc_kernel_send_data(net->socket, call->rxcall, &msg, 0, - afs_notify_end_reply_tx)) { - case 0: - _leave(" [replied]"); - return; - - case -ENOMEM: - _debug("oom"); + ret = rxrpc_kernel_send_data(net->socket, call->rxcall, &msg, 0, + afs_notify_end_reply_tx); + if (ret < 0) { rxrpc_kernel_abort_call(net->socket, call->rxcall, - RXGEN_SS_MARSHAL, -ENOMEM, - afs_abort_oom); - fallthrough; - default: - _leave(" [error]"); - return; + RXGEN_SS_MARSHAL, ret, + afs_abort_send_error); + afs_set_call_complete(call, ret, RXGEN_SS_MARSHAL); } } @@ -891,7 +882,7 @@ void afs_send_simple_reply(struct afs_call *call, const void *buf, size_t len) struct afs_net *net = call->net; struct msghdr msg; struct kvec iov[1]; - int n; + int ret; _enter(""); @@ -906,21 +897,15 @@ void afs_send_simple_reply(struct afs_call *call, const void *buf, size_t len) msg.msg_controllen = 0; msg.msg_flags = 0; - n = rxrpc_kernel_send_data(net->socket, call->rxcall, &msg, len, - afs_notify_end_reply_tx); - if (n >= 0) { - /* Success */ - _leave(" [replied]"); - return; - } - - if (n == -ENOMEM) { - _debug("oom"); + ret = rxrpc_kernel_send_data(net->socket, call->rxcall, &msg, len, + afs_notify_end_reply_tx); + if (ret < 0) { rxrpc_kernel_abort_call(net->socket, call->rxcall, - RXGEN_SS_MARSHAL, -ENOMEM, - afs_abort_oom); + RXGEN_SS_MARSHAL, ret, + afs_abort_send_error); + afs_set_call_complete(call, ret, RXGEN_SS_MARSHAL); + _leave(" [error]"); } - _leave(" [error]"); } /* diff --git a/fs/backing-file.c b/fs/backing-file.c index cad40411c63e..2ed8f17aa5f4 100644 --- a/fs/backing-file.c +++ b/fs/backing-file.c @@ -83,6 +83,7 @@ struct backing_aio { refcount_t ref; struct kiocb *orig_iocb; /* used for aio completion */ + bool is_write; void (*end_write)(struct file *, loff_t, ssize_t); struct work_struct work; long res; @@ -123,7 +124,8 @@ static void backing_aio_rw_complete(struct kiocb *iocb, long res) struct backing_aio *aio = container_of(iocb, struct backing_aio, iocb); struct kiocb *orig_iocb = aio->orig_iocb; - if (iocb->ki_flags & IOCB_WRITE) + /* Pairs with kiocb_start_write() in vfs_iocb_iter_write() */ + if (aio->is_write) kiocb_end_write(iocb); backing_aio_cleanup(aio, res); @@ -256,6 +258,7 @@ ssize_t backing_file_write_iter(struct file *file, struct iov_iter *iter, goto out; aio->orig_iocb = iocb; + aio->is_write = true; aio->end_write = ctx->end_write; kiocb_clone(&aio->iocb, iocb, get_file(file)); aio->iocb.ki_flags = flags; diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c index 8f195e769ecf..70f5fd918cce 100644 --- a/fs/btrfs/sysfs.c +++ b/fs/btrfs/sysfs.c @@ -1750,6 +1750,7 @@ void btrfs_sysfs_add_block_group_type(struct btrfs_block_group *cache) if (space_info->block_group_kobjs[index]) { spin_unlock(&space_info->lock); kobject_put(&rkobj->kobj); + memalloc_nofs_restore(nofs_flag); return; } else { space_info->block_group_kobjs[index] = &rkobj->kobj; diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c index aa551d94be1c..63ead85dff26 100644 --- a/fs/ceph/addr.c +++ b/fs/ceph/addr.c @@ -2173,6 +2173,12 @@ void ceph_fill_inline_data(struct inode *inode, struct page *locked_page, doutc(cl, "%p %llx.%llx len %zu locked_page %p\n", inode, ceph_vinop(inode), len, locked_page); + if (len > PAGE_SIZE) { + pr_warn_ratelimited_client(cl, "oversized inline data %zu\n", + len); + len = PAGE_SIZE; + } + if (len > 0) { void *kaddr = kmap_atomic(page); memcpy(kaddr, data, len); diff --git a/fs/ceph/crypto.c b/fs/ceph/crypto.c index f955a37d6b17..97fa6b821d5e 100644 --- a/fs/ceph/crypto.c +++ b/fs/ceph/crypto.c @@ -389,6 +389,7 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname, struct fscrypt_str iname; char *name = fname->name; int name_len = fname->name_len; + int dst_len; int ret; if (WARN_ON_ONCE(tname && is_vmalloc_addr(tname->name))) @@ -464,6 +465,9 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname, iname.len = fname->ctext_len; } + /* oname->len is overwritten by the decryption below */ + dst_len = oname->len; + _oname.name = unlikely(is_vmalloc_addr(oname->name)) ? tname->name : oname->name; _oname.len = oname->len; @@ -481,6 +485,10 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname, name_len = snprintf(tmp_buf, sizeof(tmp_buf), "_%.*s_%ld", oname->len, oname->name, dir->i_ino); + if (name_len > dst_len) { + ret = -ENAMETOOLONG; + goto out; + } memcpy(oname->name, tmp_buf, name_len); oname->len = name_len; } diff --git a/fs/dax.c b/fs/dax.c index ae1e9497b416..e9f12b7de17a 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -2067,7 +2067,7 @@ int dax_dedupe_file_range_compare(struct inode *src, loff_t srcoff, compared = dax_range_compare_iter(&src_iter, &dst_iter, min(src_iter.len, dst_iter.len), same); if (compared < 0) - return ret; + return compared; src_iter.processed = dst_iter.processed = compared; } return ret; diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c index 08720efbb1a1..10b9aa2c2854 100644 --- a/fs/erofs/zdata.c +++ b/fs/erofs/zdata.c @@ -1386,15 +1386,6 @@ static void z_erofs_decompressqueue_kthread_work(struct kthread_work *work) } #endif -/* Use (kthread_)work in atomic contexts to minimize scheduling overhead */ -static inline bool z_erofs_in_atomic(void) -{ - if (IS_ENABLED(CONFIG_PREEMPTION) && rcu_preempt_depth()) - return true; - if (!IS_ENABLED(CONFIG_PREEMPT_COUNT)) - return true; - return !preemptible(); -} static void z_erofs_decompress_kickoff(struct z_erofs_decompressqueue *io, int bios) @@ -1411,7 +1402,7 @@ static void z_erofs_decompress_kickoff(struct z_erofs_decompressqueue *io, if (atomic_add_return(bios, &io->pending_bios)) return; - if (z_erofs_in_atomic()) { + if (bio_in_atomic()) { /* See `sync_decompress` in sysfs-fs-erofs for more details */ if (sbi->sync_decompress == EROFS_SYNC_DECOMPRESS_AUTO) sbi->sync_decompress = EROFS_SYNC_DECOMPRESS_FORCE_ON; diff --git a/fs/exfat/file.c b/fs/exfat/file.c index 033852efe5dc..b624dc527f67 100644 --- a/fs/exfat/file.c +++ b/fs/exfat/file.c @@ -581,23 +581,18 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter) struct file *file = iocb->ki_filp; struct inode *inode = file_inode(file); struct exfat_inode_info *ei = EXFAT_I(inode); - loff_t pos = iocb->ki_pos; - loff_t valid_size; + loff_t pos, valid_size; if (unlikely(exfat_forced_shutdown(inode->i_sb))) return -EIO; inode_lock(inode); - if (pos > i_size_read(inode)) - truncate_pagecache(inode, i_size_read(inode)); - - valid_size = ei->valid_size; - ret = generic_write_checks(iocb, iter); if (ret <= 0) goto unlock; + pos = iocb->ki_pos; if (iocb->ki_flags & IOCB_DIRECT) { unsigned long align = pos | iov_iter_alignment(iter); @@ -608,6 +603,10 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter) } } + if (pos > i_size_read(inode)) + truncate_pagecache(inode, i_size_read(inode)); + + valid_size = ei->valid_size; if (pos > valid_size) { ret = exfat_extend_valid_size(file, pos); if (ret < 0 && ret != -ENOSPC) { @@ -617,6 +616,8 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter) } if (ret < 0) goto unlock; + + pos = valid_size; } ret = __generic_file_write_iter(iocb, iter); @@ -625,9 +626,6 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter) inode_unlock(inode); - if (pos > valid_size) - pos = valid_size; - if (iocb->ki_pos > pos) { ssize_t err = generic_write_sync(iocb, iocb->ki_pos - pos); if (err < 0) diff --git a/fs/ext2/file.c b/fs/ext2/file.c index 908d4849874d..0f047c33e0fe 100644 --- a/fs/ext2/file.c +++ b/fs/ext2/file.c @@ -254,7 +254,8 @@ static ssize_t ext2_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) pos = iocb->ki_pos; status = generic_perform_write(iocb, from); if (unlikely(status < 0)) { - ret = status; + if (!ret) + ret = status; goto out_unlock; } diff --git a/fs/ext2/inode.c b/fs/ext2/inode.c index f3149022257a..e8a7b21eedb5 100644 --- a/fs/ext2/inode.c +++ b/fs/ext2/inode.c @@ -824,6 +824,12 @@ static int ext2_iomap_begin(struct inode *inode, loff_t offset, loff_t length, (first_block << blkbits) < i_size_read(inode)) create = 0; + ret = ext2_get_blocks(inode, first_block, max_blocks, + &bno, &new, &boundary, create); + if (ret < 0) + return ret; + + iomap->flags = 0; /* * Writes that span EOF might trigger an IO size update on completion, * so consider them to be dirty for the purposes of O_DSYNC even if @@ -831,13 +837,6 @@ static int ext2_iomap_begin(struct inode *inode, loff_t offset, loff_t length, */ if ((flags & IOMAP_WRITE) && offset + length > i_size_read(inode)) iomap->flags |= IOMAP_F_DIRTY; - - ret = ext2_get_blocks(inode, first_block, max_blocks, - &bno, &new, &boundary, create); - if (ret < 0) - return ret; - - iomap->flags = 0; iomap->offset = (u64)first_block << blkbits; if (flags & IOMAP_DAX) iomap->dax_dev = sbi->s_daxdev; diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c index dc9540400caa..eb928d644542 100644 --- a/fs/f2fs/checkpoint.c +++ b/fs/f2fs/checkpoint.c @@ -423,7 +423,7 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type, long nr_to_write, enum iostat_type io_type) { struct address_space *mapping = META_MAPPING(sbi); - pgoff_t index = 0, prev = ULONG_MAX; + pgoff_t index = 0, next = ULONG_MAX; struct folio_batch fbatch; long nwritten = 0; int nr_folios; @@ -444,9 +444,8 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type, for (i = 0; i < nr_folios; i++) { struct folio *folio = fbatch.folios[i]; - if (nr_to_write != LONG_MAX && i != 0 && - folio->index != prev + - folio_nr_pages(fbatch.folios[i-1])) { + if (nr_to_write != LONG_MAX && next != ULONG_MAX && + folio->index != next) { folio_batch_release(&fbatch); goto stop; } @@ -475,7 +474,7 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type, break; } nwritten += folio_nr_pages(folio); - prev = folio->index; + next = folio->index + folio_nr_pages(folio); if (unlikely(nwritten >= nr_to_write)) break; } diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index 47743a0bdede..b56e6a98e7f4 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -287,7 +287,7 @@ static void f2fs_read_end_io(struct bio *bio) { struct f2fs_sb_info *sbi = F2FS_P_SB(bio_first_page_all(bio)); struct bio_post_read_ctx *ctx; - bool intask = in_task() && !irqs_disabled(); + bool intask = !bio_in_atomic(); iostat_update_and_unbind_ctx(bio); ctx = bio->bi_private; @@ -460,6 +460,10 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages) struct bio *bio; bdev = f2fs_target_device(sbi, fio->new_blkaddr, §or); + /* avoid reversed IO if bio is split due to exceeding max_segments */ + if (npages > 1 && !is_read_io(fio->op) && + f2fs_is_sequential_zone_area(sbi, fio->new_blkaddr)) + npages = min_t(int, npages, bdev_max_segments(bdev)); bio = bio_alloc_bioset(bdev, npages, fio->op | fio->op_flags | f2fs_io_flags(fio), GFP_NOIO, &f2fs_bioset); @@ -738,7 +742,19 @@ static bool page_is_mergeable(struct f2fs_sb_info *sbi, struct bio *bio, return false; if (last_blkaddr + 1 != cur_blkaddr) return false; - return bio->bi_bdev == f2fs_target_device(sbi, cur_blkaddr, NULL); + if (bio->bi_bdev != f2fs_target_device(sbi, cur_blkaddr, NULL)) + return false; + /* avoid reversed IO if bio is split due to exceeding max_segments/sectors */ + if (f2fs_is_sequential_zone_area(sbi, cur_blkaddr)) { + struct request_queue *q = bdev_get_queue(bio->bi_bdev); + + if (bio->bi_vcnt >= queue_max_segments(q)) + return false; + if (bio_sectors(bio) + (F2FS_BLKSIZE >> SECTOR_SHIFT) > + queue_max_sectors(q)) + return false; + } + return true; } static bool io_type_is_mergeable(struct f2fs_bio_info *io, diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c index cd6e69f09e18..d0d47fc2762c 100644 --- a/fs/f2fs/file.c +++ b/fs/f2fs/file.c @@ -2383,7 +2383,7 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate) fi->original_i_size = isize; if (truncate) { set_inode_flag(inode, FI_ATOMIC_REPLACE); - truncate_inode_pages_final(inode->i_mapping); + truncate_inode_pages(inode->i_mapping, 0); f2fs_i_size_write(inode, 0); isize = 0; } diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c index 8a11623dbf1e..231a9281968e 100644 --- a/fs/f2fs/node.c +++ b/fs/f2fs/node.c @@ -785,8 +785,20 @@ int f2fs_get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode) if (!npage[0]) { npage[0] = f2fs_get_node_page(sbi, nids[0]); - if (IS_ERR(npage[0])) - return PTR_ERR(npage[0]); + if (IS_ERR(npage[0])) { + err = PTR_ERR(npage[0]); + /* + * cp_error drops dirty node caches, so the inode block + * of an inode that was never written reads as NEW_ADDR + * and returns -ENOENT. It is not a hole; the inode is + * gone. Callers treat -ENOENT as a hole and keep + * looking block by block (e.g. fiemap up to maxbytes). + */ + if (err == -ENOENT && (f2fs_cp_error(sbi) || + is_sbi_flag_set(sbi, SBI_IS_SHUTDOWN))) + err = -EIO; + return err; + } } /* if inline_data is set, should not report any block indices */ @@ -2767,6 +2779,45 @@ int f2fs_recover_inline_xattr(struct inode *inode, struct page *page) f2fs_wait_on_page_writeback(ipage, NODE, true, true); memcpy(dst_addr, src_addr, inline_size); update_inode: + /* + * The xattr node of the checkpoint was removed before fsync, e.g. the + * xattrs became small enough to fit inline. Release it as well, or + * the inode keeps pointing to the stale xattrs. + */ + if (!ri->i_xattr_nid && F2FS_I(inode)->i_xattr_nid) { + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); + struct node_info ni; + int err; + + err = f2fs_get_node_info(sbi, F2FS_I(inode)->i_xattr_nid, + &ni, false); + if (err) { + f2fs_put_page(ipage, 1); + return err; + } + + if (ni.blk_addr != NEW_ADDR && + !f2fs_is_valid_blkaddr(sbi, ni.blk_addr, + DATA_GENERIC_ENHANCE)) { + f2fs_err_ratelimited(sbi, + "nat entry is corrupted, run fsck to fix it, ino:%u, nid:%u, blkaddr:%u", + ni.ino, ni.nid, ni.blk_addr); + set_sbi_flag(sbi, SBI_NEED_FSCK); + f2fs_handle_error(sbi, ERROR_INCONSISTENT_NAT); + f2fs_put_page(ipage, 1); + return -EFSCORRUPTED; + } + + f2fs_invalidate_blocks(sbi, ni.blk_addr); + dec_valid_node_count(sbi, inode, false); + set_node_addr(sbi, &ni, NULL_ADDR, false); + /* drop the cached xattr node, which may be dirty */ + truncate_inode_pages_range(NODE_MAPPING(sbi), + F2FS_BLK_TO_BYTES((loff_t)ni.nid), + F2FS_BLK_END_BYTES((loff_t)ni.nid)); + f2fs_i_xnid_write(inode, 0); + } + f2fs_update_inode(inode, ipage); f2fs_put_page(ipage, 1); return 0; diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index 49146476882b..7559a85bb9df 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -193,7 +193,7 @@ void f2fs_abort_atomic_write(struct inode *inode, bool clean) return; if (clean) - truncate_inode_pages_final(inode->i_mapping); + truncate_inode_pages(inode->i_mapping, 0); release_atomic_write_cnt(inode); clear_inode_flag(inode, FI_ATOMIC_COMMITTED); @@ -2970,6 +2970,7 @@ static int change_curseg(struct f2fs_sb_info *sbi, int type) if (IS_ERR(sum_page)) { /* GC won't be able to use stale summary pages by cp_error */ memset(curseg->sum_blk, 0, SUM_ENTRY_SIZE); + reset_curseg_fields(curseg); return PTR_ERR(sum_page); } sum_node = (struct f2fs_summary_block *)page_address(sum_page); diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index 31724ab224d4..ec3e20cd6c5e 100644 --- a/fs/f2fs/super.c +++ b/fs/f2fs/super.c @@ -3773,6 +3773,7 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi) block_t avail_node_count, valid_node_count; unsigned int nat_blocks, nat_bits_bytes, nat_bits_blocks; unsigned int sit_blk_cnt; + unsigned long long ckpt_size, sit_end, nat_end; int i, j; total = le32_to_cpu(raw_super->segment_count); @@ -3904,6 +3905,29 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi) return 1; } + /* + * Nothing above ties the bitmap lengths to sbi->ckpt, which is + * (1 + cp_payload) blocks. Bound both ends the way __bitmap_ptr() + * lays them out. + */ + ckpt_size = (unsigned long long)(cp_payload + 1) * F2FS_BLKSIZE; + if (__is_set_ckpt_flags(ckpt, CP_LARGE_NAT_BITMAP_FLAG)) { + nat_end = CP_MIN_CHKSUM_OFFSET + sizeof(__le32) + nat_bitmap_size; + sit_end = nat_end + sit_bitmap_size; + } else if (cp_payload > 0) { + nat_end = CP_MIN_CHKSUM_OFFSET + nat_bitmap_size; + sit_end = F2FS_BLKSIZE + sit_bitmap_size; + } else { + sit_end = CP_MIN_CHKSUM_OFFSET + sit_bitmap_size; + nat_end = sit_end + nat_bitmap_size; + } + + if (sit_end > ckpt_size || nat_end > ckpt_size) { + f2fs_err(sbi, "Bitmaps do not fit the checkpoint pack: sit ends at %llu, nat at %llu, pack is %llu bytes", + sit_end, nat_end, ckpt_size); + return 1; + } + if (__is_set_ckpt_flags(ckpt, CP_LARGE_NAT_BITMAP_FLAG) && le32_to_cpu(ckpt->checksum_offset) != CP_MIN_CHKSUM_OFFSET) { f2fs_warn(sbi, "using deprecated layout of large_nat_bitmap, " diff --git a/fs/fat/inode.c b/fs/fat/inode.c index 3852bb66358c..1c9988efc8d2 100644 --- a/fs/fat/inode.c +++ b/fs/fat/inode.c @@ -215,7 +215,9 @@ static void fat_write_failed(struct address_space *mapping, loff_t to) if (to > inode->i_size) { truncate_pagecache(inode, inode->i_size); + down_write(&MSDOS_I(inode)->truncate_lock); fat_truncate_blocks(inode, inode->i_size); + up_write(&MSDOS_I(inode)->truncate_lock); } } diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index 435f14b7e2cc..3b67018191df 100644 --- a/fs/fs-writeback.c +++ b/fs/fs-writeback.c @@ -757,6 +757,13 @@ bool cleanup_offline_cgwb(struct bdi_writeback *wb) new_wb = &wb->bdi->wb; /* wb_get() is noop for bdi's wb */ nr = 0; + /* + * The scan moves walked inodes to the newest end of b_dirty_time, + * past ones it didn't reach. If one of those was dirtied after a + * sync's cutoff, the sync's queue_io() stops there and misses the + * moved ones. Exclude syncs as process_inode_switch_wbs() does. + */ + down_read(&wb->bdi->wb_switch_rwsem); /* * Paired with synchronize_rcu() in cgroup_writeback_umount(). * Holding rcu_read_lock across the SB_ACTIVE check, the inode grab @@ -779,6 +786,7 @@ bool cleanup_offline_cgwb(struct bdi_writeback *wb) restart = isw_prepare_wbs_switch(new_wb, isw, &wb->b_dirty_time, &nr); spin_unlock(&wb->list_lock); + up_read(&wb->bdi->wb_switch_rwsem); /* no attached inodes? bail out */ if (nr == 0) { diff --git a/fs/fuse/file.c b/fs/fuse/file.c index d242613a1c8a..cada359fd03a 100644 --- a/fs/fuse/file.c +++ b/fs/fuse/file.c @@ -2538,6 +2538,10 @@ static int fuse_write_end(struct file *file, struct address_space *mapping, static int fuse_launder_folio(struct folio *folio) { int err = 0; + + if (fuse_is_bad(folio->mapping->host)) + return -EIO; + if (folio_clear_dirty_for_io(folio)) { struct inode *inode = folio->mapping->host; @@ -3344,9 +3348,11 @@ static ssize_t __fuse_copy_file_range(struct file *file_in, loff_t pos_in, if (err) goto out; - truncate_inode_pages_range(inode_out->i_mapping, - ALIGN_DOWN(pos_out, PAGE_SIZE), - ALIGN(pos_out + outarg.size, PAGE_SIZE) - 1); + if (outarg.size) + truncate_inode_pages_range(inode_out->i_mapping, + ALIGN_DOWN(pos_out, PAGE_SIZE), + ALIGN(pos_out + outarg.size, + PAGE_SIZE) - 1); file_update_time(file_out); fuse_write_update_attr(inode_out, pos_out + outarg.size, outarg.size); diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index 06e59b820ffa..16baca6943ff 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -720,6 +720,10 @@ static void copy_args_from_argbuf(struct fuse_args *args, struct fuse_req *req) unsigned int num_out; unsigned int i; + /* fuse_request_end() returns this error; there are no args to copy. */ + if (req->out.h.error) + goto out; + remaining = req->out.h.len - sizeof(req->out.h); num_in = args->in_numargs - args->in_pages; num_out = args->out_numargs - args->out_pages; @@ -745,10 +749,57 @@ static void copy_args_from_argbuf(struct fuse_args *args, struct fuse_req *req) if (args->out_argvar) args->out_args[args->out_numargs - 1].size = remaining; +out: kfree(req->argbuf); req->argbuf = NULL; } +/* Verify that the server properly follows the FUSE protocol */ +static bool virtio_fs_verify_response(struct fuse_req *req, unsigned int len) +{ + struct fuse_args *args = req->args; + struct fuse_out_header *oh = &req->out.h; + unsigned int expected; + + if (len < sizeof(*oh)) { + pr_warn("virtio-fs: response too short (%u)\n", len); + return false; + } + if (oh->len != len) { + pr_warn("virtio-fs: oh.len mismatch (%u != %u)\n", oh->len, len); + return false; + } + if (oh->unique != req->in.h.unique) { + pr_warn("virtio-fs: oh.unique mismatch (%llu != %llu)\n", + oh->unique, req->in.h.unique); + return false; + } + if (oh->error <= -ERESTARTSYS || oh->error > 0) { + pr_warn("virtio-fs: invalid error value (%d)\n", oh->error); + return false; + } + + if (oh->error) { + if (len != sizeof(*oh)) { + pr_warn("virtio-fs: error response too long (%u)\n", len); + return false; + } + return true; + } + + expected = sizeof(*oh) + + fuse_len_args(args->out_numargs, args->out_args); + if (len > expected || + (len < expected && + (!args->out_argvar || + expected - len > args->out_args[args->out_numargs - 1].size))) { + pr_warn("virtio-fs: invalid response length (%u, expected %u)\n", + len, expected); + return false; + } + return true; +} + /* Work function for request completion */ static void virtio_fs_request_complete(struct fuse_req *req, struct virtio_fs_vq *fsvq) @@ -759,10 +810,6 @@ static void virtio_fs_request_complete(struct fuse_req *req, unsigned int len, i, thislen; struct page *page; - /* - * TODO verify that server properly follows FUSE protocol - * (oh.uniq, oh.len) - */ args = req->args; copy_args_from_argbuf(args, req); @@ -818,6 +865,10 @@ static void virtio_fs_requests_done_work(struct work_struct *work) virtqueue_disable_cb(vq); while ((req = virtqueue_get_buf(vq, &len)) != NULL) { + if (!virtio_fs_verify_response(req, len)) { + req->out.h.error = -EIO; + req->out.h.len = sizeof(struct fuse_out_header); + } spin_lock(&fpq->lock); list_move_tail(&req->list, &reqs); spin_unlock(&fpq->lock); diff --git a/fs/gfs2/glops.c b/fs/gfs2/glops.c index d13a050bcb9d..dc5160827201 100644 --- a/fs/gfs2/glops.c +++ b/fs/gfs2/glops.c @@ -395,6 +395,7 @@ static int gfs2_dinode_in(struct gfs2_inode *ip, const void *buf) umode_t mode = be32_to_cpu(str->di_mode); struct inode *inode = &ip->i_inode; bool is_new = inode->i_state & I_NEW; + u64 size; if (unlikely(ip->i_no_addr != be64_to_cpu(str->di_num.no_addr))) { gfs2_consist_inode(ip); @@ -420,7 +421,12 @@ static int gfs2_dinode_in(struct gfs2_inode *ip, const void *buf) i_uid_write(inode, be32_to_cpu(str->di_uid)); i_gid_write(inode, be32_to_cpu(str->di_gid)); set_nlink(inode, be32_to_cpu(str->di_nlink)); - i_size_write(inode, be64_to_cpu(str->di_size)); + size = be64_to_cpu(str->di_size); + if (unlikely(size > inode->i_sb->s_maxbytes)) { + gfs2_consist_inode(ip); + return -EIO; + } + i_size_write(inode, size); gfs2_set_inode_blocks(inode, be64_to_cpu(str->di_blocks)); atime.tv_sec = be64_to_cpu(str->di_atime); atime.tv_nsec = be32_to_cpu(str->di_atime_nsec); @@ -459,7 +465,7 @@ static int gfs2_dinode_in(struct gfs2_inode *ip, const void *buf) ip->i_depth = (u8)depth; ip->i_entries = be32_to_cpu(str->di_entries); - if (gfs2_is_stuffed(ip) && inode->i_size > gfs2_max_stuffed_size(ip)) { + if (gfs2_is_stuffed(ip) && size > gfs2_max_stuffed_size(ip)) { gfs2_consist_inode(ip); return -EIO; } diff --git a/fs/inode.c b/fs/inode.c index 7942673f686f..87396234d1e2 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -502,12 +502,11 @@ static void inode_pin_lru_isolating(struct inode *inode) static void inode_unpin_lru_isolating(struct inode *inode) { - spin_lock(&inode->i_lock); + lockdep_assert_held(&inode->i_lock); WARN_ON(!(inode->i_state & I_LRU_ISOLATING)); inode->i_state &= ~I_LRU_ISOLATING; /* Called with inode->i_lock which ensures memory ordering. */ inode_wake_up_bit(inode, __I_LRU_ISOLATING); - spin_unlock(&inode->i_lock); } static void inode_wait_for_lru_isolating(struct inode *inode) @@ -890,6 +889,12 @@ static enum lru_status inode_lru_isolate(struct list_head *item, if (!spin_trylock(&inode->i_lock)) return LRU_SKIP; + /* Another walker is dropping its page cache, leave it to that one */ + if (inode->i_state & I_LRU_ISOLATING) { + spin_unlock(&inode->i_lock); + return LRU_SKIP; + } + /* * Inodes can get referenced, redirtied, or repopulated while * they're already on the LRU, and this can make them @@ -930,7 +935,13 @@ static enum lru_status inode_lru_isolate(struct list_head *item, __count_vm_events(PGINODESTEAL, reap); mm_account_reclaimed_pages(reap); } + + spin_lock(&inode->i_lock); + /* The walk restarts here: don't retry leftovers right away */ + if (!mapping_empty(&inode->i_data)) + inode->i_state |= I_REFERENCED; inode_unpin_lru_isolating(inode); + spin_unlock(&inode->i_lock); spin_lock(lru_lock); return LRU_RETRY; } diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 02a5fc5105e7..23457bdfc20d 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -760,6 +760,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, if (iomap_block_needs_zeroing(iter, block_start)) { if (WARN_ON_ONCE(iter->flags & IOMAP_UNSHARE)) return -EIO; + /* uptodate once copied, see __iomap_write_end() */ folio_zero_segments(folio, poff, from, to, poff + plen); } else { int status; @@ -771,8 +772,8 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, poff, plen, srcmap); if (status) return status; + iomap_set_range_uptodate(folio, poff, plen); } - iomap_set_range_uptodate(folio, poff, plen); } while ((block_start += plen) < block_end); return 0; diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index c81cfcbb1bb6..92eb494aa0f0 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -250,19 +250,19 @@ void iomap_dio_bio_end_io(struct bio *bio) } EXPORT_SYMBOL_GPL(iomap_dio_bio_end_io); -static int iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio, +static void iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio, loff_t pos, unsigned len) { struct inode *inode = file_inode(dio->iocb->ki_filp); struct bio *bio; if (!len) - return 0; + return; /* * Max block size supported is 64k */ if (WARN_ON_ONCE(len > IOMAP_ZERO_PAGE_SIZE)) - return -EINVAL; + return; bio = iomap_dio_alloc_bio(iter, dio, 1, REQ_OP_WRITE | REQ_SYNC | REQ_IDLE); fscrypt_set_bio_crypt_ctx(bio, inode, pos >> inode->i_blkbits, @@ -273,7 +273,6 @@ static int iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio, __bio_add_page(bio, zero_page, len, 0); iomap_dio_submit_bio(iter, dio, bio, pos); - return 0; } /* @@ -382,10 +381,7 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter, if (need_zeroout) { /* zero out from the start of the block to the write offset */ pad = pos & (fs_block_size - 1); - - ret = iomap_dio_zero(iter, dio, pos - pad, pad); - if (ret) - goto out; + iomap_dio_zero(iter, dio, pos - pad, pad); } /* @@ -463,8 +459,7 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter, /* zero out from the end of the write to the end of the block */ pad = pos & (fs_block_size - 1); if (pad) - ret = iomap_dio_zero(iter, dio, pos, - fs_block_size - pad); + iomap_dio_zero(iter, dio, pos, fs_block_size - pad); } out: /* Undo iter limitation to current extent */ diff --git a/fs/nfs/callback_xdr.c b/fs/nfs/callback_xdr.c index 1aa6a6e14b05..34fa00f1a1c2 100644 --- a/fs/nfs/callback_xdr.c +++ b/fs/nfs/callback_xdr.c @@ -275,6 +275,13 @@ __be32 decode_devicenotify_args(struct svc_rqst *rqstp, if (n == 0) goto out; + /* sanity check the count against the remaining stream */ + if (n > xdr_stream_remaining(xdr) / + ((4 * sizeof(uint32_t)) + NFS4_DEVICEID4_SIZE)) { + status = htonl(NFS4ERR_BADXDR); + goto out; + } + args->devs = kmalloc_array(n, sizeof(*args->devs), GFP_KERNEL); if (!args->devs) { status = htonl(NFS4ERR_DELAY); diff --git a/fs/nfs/filelayout/filelayoutdev.c b/fs/nfs/filelayout/filelayoutdev.c index b878ea466361..7142958b1265 100644 --- a/fs/nfs/filelayout/filelayoutdev.c +++ b/fs/nfs/filelayout/filelayoutdev.c @@ -177,27 +177,14 @@ nfs4_fl_alloc_deviceid_node(struct nfs_server *server, struct pnfs_device *pdev, trace_fl_getdevinfo(server, &pdev->dev_id, dsaddr->ds_list[i]->ds_remotestr); /* If DS was already in cache, free ds addrs */ - while (!list_empty(&dsaddrs)) { - da = list_first_entry(&dsaddrs, - struct nfs4_pnfs_ds_addr, - da_node); - list_del_init(&da->da_node); - kfree(da->da_remotestr); - kfree(da); - } + nfs4_pnfs_ds_addr_list_free(&dsaddrs); } __free_page(scratch); return dsaddr; out_err_drain_dsaddrs: - while (!list_empty(&dsaddrs)) { - da = list_first_entry(&dsaddrs, struct nfs4_pnfs_ds_addr, - da_node); - list_del_init(&da->da_node); - kfree(da->da_remotestr); - kfree(da); - } + nfs4_pnfs_ds_addr_list_free(&dsaddrs); out_err_free_deviceid: nfs4_fl_free_deviceid(dsaddr); /* stripe_indicies was part of dsaddr */ diff --git a/fs/nfs/flexfilelayout/flexfilelayoutdev.c b/fs/nfs/flexfilelayout/flexfilelayoutdev.c index 53655581f9cb..f33e4f820dec 100644 --- a/fs/nfs/flexfilelayout/flexfilelayoutdev.c +++ b/fs/nfs/flexfilelayout/flexfilelayoutdev.c @@ -155,26 +155,13 @@ nfs4_ff_alloc_deviceid_node(struct nfs_server *server, struct pnfs_device *pdev, goto out_err_drain_dsaddrs; /* If DS was already in cache, free ds addrs */ - while (!list_empty(&dsaddrs)) { - da = list_first_entry(&dsaddrs, - struct nfs4_pnfs_ds_addr, - da_node); - list_del_init(&da->da_node); - kfree(da->da_remotestr); - kfree(da); - } + nfs4_pnfs_ds_addr_list_free(&dsaddrs); __free_page(scratch); return new_ds; out_err_drain_dsaddrs: - while (!list_empty(&dsaddrs)) { - da = list_first_entry(&dsaddrs, struct nfs4_pnfs_ds_addr, - da_node); - list_del_init(&da->da_node); - kfree(da->da_remotestr); - kfree(da); - } + nfs4_pnfs_ds_addr_list_free(&dsaddrs); kfree(ds_versions); out_scratch: diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 14d2a91250f2..9d612bed74e3 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -426,6 +426,7 @@ int nfs4_pnfs_ds_connect(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds, struct nfs4_pnfs_ds_addr *nfs4_decode_mp_ds_addr(struct net *net, struct xdr_stream *xdr, gfp_t gfp_flags); +void nfs4_pnfs_ds_addr_list_free(struct list_head *dsaddrs); void pnfs_layout_mark_request_commit(struct nfs_page *req, struct pnfs_layout_segment *lseg, struct nfs_commit_info *cinfo, diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 634ee58ced12..1e457dd3b8bd 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -632,23 +632,28 @@ static void nfs4_pnfs_ds_addr_free(struct nfs4_pnfs_ds_addr *da) kfree(da); } -static void destroy_ds(struct nfs4_pnfs_ds *ds) +void nfs4_pnfs_ds_addr_list_free(struct list_head *dsaddrs) { struct nfs4_pnfs_ds_addr *da; + while (!list_empty(dsaddrs)) { + da = list_first_entry(dsaddrs, struct nfs4_pnfs_ds_addr, + da_node); + list_del_init(&da->da_node); + nfs4_pnfs_ds_addr_free(da); + } +} +EXPORT_SYMBOL_GPL(nfs4_pnfs_ds_addr_list_free); + +static void destroy_ds(struct nfs4_pnfs_ds *ds) +{ dprintk("--> %s\n", __func__); ifdebug(FACILITY) print_ds(ds); nfs_put_client(ds->ds_clp); - while (!list_empty(&ds->ds_addrs)) { - da = list_first_entry(&ds->ds_addrs, - struct nfs4_pnfs_ds_addr, - da_node); - list_del_init(&da->da_node); - nfs4_pnfs_ds_addr_free(da); - } + nfs4_pnfs_ds_addr_list_free(&ds->ds_addrs); kfree(ds->ds_remotestr); kfree(ds); diff --git a/fs/nfsd/blocklayout.c b/fs/nfsd/blocklayout.c index bbd87cd16282..de789cefd30d 100644 --- a/fs/nfsd/blocklayout.c +++ b/fs/nfsd/blocklayout.c @@ -18,8 +18,8 @@ static __be32 -nfsd4_block_proc_layoutget(struct inode *inode, const struct svc_fh *fhp, - struct nfsd4_layoutget *args) +nfsd4_block_proc_layoutget(struct svc_rqst *rqstp, struct inode *inode, + const struct svc_fh *fhp, struct nfsd4_layoutget *args) { struct nfsd4_layout_seg *seg = &args->lg_seg; struct super_block *sb = inode->i_sb; diff --git a/fs/nfsd/flexfilelayout.c b/fs/nfsd/flexfilelayout.c index 3c4419da5e24..64b85b9fd9f4 100644 --- a/fs/nfsd/flexfilelayout.c +++ b/fs/nfsd/flexfilelayout.c @@ -14,19 +14,20 @@ #include #include "flexfilelayoutxdr.h" +#include "auth.h" #include "pnfs.h" #include "vfs.h" #define NFSDDBG_FACILITY NFSDDBG_PNFS static __be32 -nfsd4_ff_proc_layoutget(struct inode *inode, const struct svc_fh *fhp, - struct nfsd4_layoutget *args) +nfsd4_ff_proc_layoutget(struct svc_rqst *rqstp, struct inode *inode, + const struct svc_fh *fhp, struct nfsd4_layoutget *args) { + struct user_namespace *userns = nfsd_user_namespace(rqstp); struct nfsd4_layout_seg *seg = &args->lg_seg; u32 device_generation = 0; int error; - uid_t u; struct pnfs_ff_layout *fl; @@ -49,13 +50,16 @@ nfsd4_ff_proc_layoutget(struct inode *inode, const struct svc_fh *fhp, fl->flags = FF_FLAGS_NO_LAYOUTCOMMIT | FF_FLAGS_NO_IO_THRU_MDS | FF_FLAGS_NO_READ_IO; - /* Do not allow a IOMODE_READ segment to have write pemissions */ - if (seg->iomode == IOMODE_READ) { - u = from_kuid(&init_user_ns, inode->i_uid) + 1; - fl->uid = make_kuid(&init_user_ns, u); - } else - fl->uid = inode->i_uid; - fl->gid = inode->i_gid; + fl->uid = from_kuid_munged(userns, inode->i_uid); + fl->gid = from_kgid_munged(userns, inode->i_gid); + + /* + * Do not allow an IOMODE_READ segment to have write permissions. + * The group is left intact so group-readable files stay readable; + * nfsd_setuser() squashes an unmapped uid to the export's anon ID. + */ + if (seg->iomode == IOMODE_READ) + fl->uid++; error = nfsd4_set_deviceid(&fl->deviceid, fhp, device_generation); if (error) diff --git a/fs/nfsd/flexfilelayoutxdr.c b/fs/nfsd/flexfilelayoutxdr.c index 374e52d3064a..b80352f41c0d 100644 --- a/fs/nfsd/flexfilelayoutxdr.c +++ b/fs/nfsd/flexfilelayoutxdr.c @@ -32,8 +32,8 @@ nfsd4_ff_encode_layoutget(struct xdr_stream *xdr, fh_len = 4 + xdr_align_size(fl->fh.size); - uid.len = sprintf(uid.buf, "%u", from_kuid(&init_user_ns, fl->uid)); - gid.len = sprintf(gid.buf, "%u", from_kgid(&init_user_ns, fl->gid)); + uid.len = sprintf(uid.buf, "%u", fl->uid); + gid.len = sprintf(gid.buf, "%u", fl->gid); /* data server entry: deviceid + efficiency + stateid + fh list + * user + group + flags + stats_collect_hint diff --git a/fs/nfsd/flexfilelayoutxdr.h b/fs/nfsd/flexfilelayoutxdr.h index 6d5a1066a903..3e1876d49db2 100644 --- a/fs/nfsd/flexfilelayoutxdr.h +++ b/fs/nfsd/flexfilelayoutxdr.h @@ -35,8 +35,9 @@ struct pnfs_ff_device_addr { struct pnfs_ff_layout { u32 flags; u32 stats_collect_hint; - kuid_t uid; - kgid_t gid; + /* Values to encode; nfsd4_ff_proc_layoutget() has mapped these */ + u32 uid; + u32 gid; struct nfsd4_deviceid deviceid; stateid_t stateid; struct nfs_fh fh; diff --git a/fs/nfsd/nfs2acl.c b/fs/nfsd/nfs2acl.c index 2998640f259d..cf92e1f8df7d 100644 --- a/fs/nfsd/nfs2acl.c +++ b/fs/nfsd/nfs2acl.c @@ -15,6 +15,20 @@ #define NFSDDBG_FACILITY NFSDDBG_PROC +static __be32 nfsacld_map_status(__be32 status) +{ + switch (status) { + case nfserr_nofilehandle: + case nfserr_badhandle: + status = nfserr_stale; + break; + case nfserr_wrongsec: + status = nfserr_acces; + break; + } + return status; +} + /* * NULL call. */ @@ -79,6 +93,7 @@ static __be32 nfsacld_proc_getacl(struct svc_rqst *rqstp) /* resp->acl_{access,default} are released in nfssvc_release_getacl. */ out: + resp->status = nfsacld_map_status(resp->status); return rpc_success; fail: @@ -137,6 +152,7 @@ static __be32 nfsacld_proc_setacl(struct svc_rqst *rqstp) out: /* argp->acl_{access,default} are released in nfsaclsvc_release_setacl. */ + resp->status = nfsacld_map_status(resp->status); return rpc_success; out_drop_lock: @@ -163,6 +179,7 @@ static __be32 nfsacld_proc_getattr(struct svc_rqst *rqstp) goto out; resp->status = fh_getattr(&resp->fh, &resp->stat); out: + resp->status = nfsacld_map_status(resp->status); return rpc_success; } @@ -185,6 +202,7 @@ static __be32 nfsacld_proc_access(struct svc_rqst *rqstp) goto out; resp->status = fh_getattr(&resp->fh, &resp->stat); out: + resp->status = nfsacld_map_status(resp->status); return rpc_success; } diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c index 798fcc9c4092..47cfd4bd4ecc 100644 --- a/fs/nfsd/nfs4callback.c +++ b/fs/nfsd/nfs4callback.c @@ -1227,6 +1227,7 @@ static void nfsd41_destroy_cb(struct nfsd4_callback *cb) trace_nfsd_cb_destroy(clp, cb); nfsd41_cb_release_slot(cb); + clear_bit(NFSD4_CALLBACK_RUNNING, &cb->cb_flags); if (cb->cb_ops && cb->cb_ops->release) cb->cb_ops->release(cb); nfsd41_cb_inflight_end(clp); @@ -1575,6 +1576,7 @@ void nfsd4_init_cb(struct nfsd4_callback *cb, struct nfs4_client *clp, cb->cb_msg.rpc_proc = &nfs4_cb_procedures[op]; cb->cb_msg.rpc_argp = cb; cb->cb_msg.rpc_resp = cb; + cb->cb_flags = 0; cb->cb_ops = ops; INIT_WORK(&cb->cb_work, nfsd4_run_cb_work); cb->cb_status = 0; diff --git a/fs/nfsd/nfs4layouts.c b/fs/nfsd/nfs4layouts.c index 0ff899475263..1f37044db59f 100644 --- a/fs/nfsd/nfs4layouts.c +++ b/fs/nfsd/nfs4layouts.c @@ -125,6 +125,7 @@ nfsd4_set_deviceid(struct nfsd4_deviceid *id, const struct svc_fh *fhp, void nfsd4_setup_layout_type(struct svc_export *exp) { + struct nfsd_net *nn = net_generic(exp->cd->net, nfsd_net_id); #if defined(CONFIG_NFSD_BLOCKLAYOUT) || defined(CONFIG_NFSD_SCSILAYOUT) struct super_block *sb = exp->ex_path.mnt->mnt_sb; #endif @@ -132,9 +133,9 @@ void nfsd4_setup_layout_type(struct svc_export *exp) if (!(exp->ex_flags & NFSEXP_PNFS)) return; -#ifdef CONFIG_NFSD_FLEXFILELAYOUT - exp->ex_layout_types |= 1 << LAYOUT_FLEX_FILES; -#endif + if (IS_ENABLED(CONFIG_NFSD_FLEXFILELAYOUT) && + nfsd_vers(nn, 3, NFSD_TEST)) + exp->ex_layout_types |= 1 << LAYOUT_FLEX_FILES; #ifdef CONFIG_NFSD_BLOCKLAYOUT if (sb->s_export_op->get_uuid && sb->s_export_op->map_blocks && @@ -347,9 +348,10 @@ nfsd4_recall_file_layout(struct nfs4_layout_stateid *ls) atomic_inc(&ls->ls_stid.sc_file->fi_lo_recalls); trace_nfsd_layout_recall(&ls->ls_stid.sc_stateid); - refcount_inc(&ls->ls_stid.sc_count); - nfsd4_run_cb(&ls->ls_recall); - + if (!test_and_set_bit(NFSD4_CALLBACK_RUNNING, &ls->ls_recall.cb_flags)) { + refcount_inc(&ls->ls_stid.sc_count); + nfsd4_run_cb(&ls->ls_recall); + } out_unlock: spin_unlock(&ls->ls_lock); } diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c index 53ded5de593a..3d03ff6313d7 100644 --- a/fs/nfsd/nfs4proc.c +++ b/fs/nfsd/nfs4proc.c @@ -1769,7 +1769,7 @@ static void nfsd4_send_cb_offload(struct nfsd4_copy *copy) NFSPROC4_CLNT_CB_OFFLOAD); trace_nfsd_cb_offload(copy->cp_clp, &cbo->co_res.cb_stateid, &cbo->co_fh, copy->cp_count, copy->nfserr); - nfsd4_run_cb(&cbo->co_cb); + nfsd4_try_run_cb(&cbo->co_cb); } /** @@ -2359,7 +2359,7 @@ nfsd4_layoutget(struct svc_rqst *rqstp, if (atomic_read(&ls->ls_stid.sc_file->fi_lo_recalls)) goto out_put_stid; - nfserr = ops->proc_layoutget(d_inode(current_fh->fh_dentry), + nfserr = ops->proc_layoutget(rqstp, d_inode(current_fh->fh_dentry), current_fh, lgp); if (nfserr) goto out_put_stid; diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c index 104dafbe881b..ef0388d92aca 100644 --- a/fs/nfsd/nfs4recover.c +++ b/fs/nfsd/nfs4recover.c @@ -730,6 +730,7 @@ struct cld_upcall { struct completion cu_done; /* daemon has read the whole upcall; protected by cn_lock */ bool cu_inflight; + struct rpc_pipe_msg cu_pipe_msg; union { struct cld_msg_hdr cu_hdr; struct cld_msg cu_msg; @@ -741,22 +742,22 @@ static int __cld_pipe_upcall(struct rpc_pipe *pipe, void *cmsg, struct nfsd_net *nn) { int ret; - struct rpc_pipe_msg msg; struct cld_upcall *cup = container_of(cmsg, struct cld_upcall, cu_u); + struct rpc_pipe_msg *msg = &cup->cu_pipe_msg; - memset(&msg, 0, sizeof(msg)); - msg.data = cmsg; - msg.len = nn->client_tracking_ops->msglen; + memset(msg, 0, sizeof(*msg)); + msg->data = cmsg; + msg->len = nn->client_tracking_ops->msglen; - ret = rpc_queue_upcall(pipe, &msg); + ret = rpc_queue_upcall(pipe, msg); if (ret < 0) { goto out; } wait_for_completion(&cup->cu_done); - if (msg.errno < 0) - ret = msg.errno; + if (msg->errno < 0) + ret = msg->errno; out: return ret; } @@ -919,8 +920,11 @@ cld_pipe_downcall(struct file *filp, const char __user *src, size_t mlen) if (status == -EINPROGRESS) return __cld_pipe_inprogress_downcall(cmsg, nn); - if (copy_from_user(&cup->cu_u.cu_msg_v2, src, mlen) != 0) + if (copy_from_user(&cup->cu_u.cu_msg_v2, src, mlen) != 0) { + cup->cu_pipe_msg.errno = -EFAULT; + complete(&cup->cu_done); return -EFAULT; + } complete(&cup->cu_done); return mlen; @@ -929,9 +933,8 @@ cld_pipe_downcall(struct file *filp, const char __user *src, size_t mlen) static void cld_pipe_destroy_msg(struct rpc_pipe_msg *msg) { - struct cld_msg *cmsg = msg->data; - struct cld_upcall *cup = container_of(cmsg, struct cld_upcall, - cu_u.cu_msg); + struct cld_upcall *cup = container_of(msg, struct cld_upcall, + cu_pipe_msg); struct cld_net *cn = cup->cu_net; /* @@ -948,10 +951,34 @@ cld_pipe_destroy_msg(struct rpc_pipe_msg *msg) complete(&cup->cu_done); } +/* + * An upcall the daemon has consumed is off every pipe list, so the + * purge in rpc_pipe_release() and rpc_close_pipes() cannot reach it. + * Release its waiter here instead; no downcall can arrive now. + */ +static void +cld_release_pipe(struct inode *inode) +{ + struct nfsd_net *nn = net_generic(inode->i_sb->s_fs_info, nfsd_net_id); + struct cld_net *cn = nn->cld_net; + struct cld_upcall *cup; + + spin_lock(&cn->cn_lock); + list_for_each_entry(cup, &cn->cn_list, cu_list) { + if (!cup->cu_inflight) + continue; + cup->cu_inflight = false; + cup->cu_pipe_msg.errno = -EPIPE; + complete(&cup->cu_done); + } + spin_unlock(&cn->cn_lock); +} + static const struct rpc_pipe_ops cld_upcall_ops = { .upcall = rpc_pipe_generic_upcall, .downcall = cld_pipe_downcall, .destroy_msg = cld_pipe_destroy_msg, + .release_pipe = cld_release_pipe, }; static int diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 723a1ae68836..061ce98eb414 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -1420,6 +1420,7 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp) dp->dl_stid.sc_type = SC_TYPE_DELEG; list_add(&dp->dl_perfile, &fp->fi_delegations); list_add(&dp->dl_perclnt, &clp->cl_delegations); + clp->cl_deleg_count++; return 0; } @@ -1451,6 +1452,7 @@ unhash_delegation_locked(struct nfs4_delegation *dp, unsigned short statusmask) ++dp->dl_time; spin_lock(&fp->fi_lock); list_del_init(&dp->dl_perclnt); + dp->dl_stid.sc_client->cl_deleg_count--; list_del_init(&dp->dl_recall_lru); list_del_init(&dp->dl_perfile); spin_unlock(&fp->fi_lock); @@ -3246,7 +3248,6 @@ nfsd4_cb_recall_any_release(struct nfsd4_callback *cb) { struct nfs4_client *clp = cb->cb_clp; - clear_bit(NFSD4_CLIENT_CB_RECALL_ANY, &clp->cl_flags); drop_client(clp); } @@ -3303,8 +3304,10 @@ static void nfs4_cb_getattr(struct nfs4_cb_fattr *ncf) /* set to proper status when nfsd4_cb_getattr_done runs */ ncf->ncf_cb_status = NFS4ERR_IO; - refcount_inc(&dp->dl_stid.sc_count); - nfsd4_run_cb(&ncf->ncf_getattr); + if (!test_and_set_bit(NFSD4_CALLBACK_RUNNING, &ncf->ncf_getattr.cb_flags)) { + refcount_inc(&dp->dl_stid.sc_count); + nfsd4_run_cb(&ncf->ncf_getattr); + } } static struct nfs4_client *create_client(struct xdr_netobj name, @@ -5477,6 +5480,10 @@ static const struct nfsd4_callback_ops nfsd4_cb_recall_ops = { static void nfsd_break_one_deleg(struct nfs4_delegation *dp) { bool queued; + + if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &dp->dl_recall.cb_flags)) + return; + /* * We're assuming the state code never drops its reference * without first removing the lease. Since we're in this lease @@ -5487,8 +5494,10 @@ static void nfsd_break_one_deleg(struct nfs4_delegation *dp) refcount_inc(&dp->dl_stid.sc_count); queued = nfsd4_run_cb(&dp->dl_recall); WARN_ON_ONCE(!queued); - if (!queued) + if (!queued) { refcount_dec(&dp->dl_stid.sc_count); + clear_bit(NFSD4_CALLBACK_RUNNING, &dp->dl_recall.cb_flags); + } } /* Called from break_lease() with flc_lock held. */ @@ -6999,7 +7008,7 @@ deleg_reaper(struct nfsd_net *nn) { struct list_head *pos, *next; struct nfs4_client *clp; - LIST_HEAD(cblist); + unsigned int count; spin_lock(&nn->client_lock); list_for_each_safe(pos, next, &nn->client_lru) { @@ -7009,35 +7018,39 @@ deleg_reaper(struct nfsd_net *nn) continue; if (clp->cl_state != NFSD4_ACTIVE) continue; - if (list_empty(&clp->cl_delegations)) - continue; if (atomic_read(&clp->cl_delegs_in_recall)) continue; - if (test_bit(NFSD4_CLIENT_CB_RECALL_ANY, &clp->cl_flags)) - continue; if (ktime_get_boottime_seconds() - clp->cl_ra_time < 5) continue; if (clp->cl_cb_state != NFSD4_CB_UP) continue; - list_add(&clp->cl_ra_cblist, &cblist); + /* + * This read races with hash_delegation_locked() and + * unhash_delegation_locked() on other CPUs. A stale + * count only skews the keep value; the next + * laundromat pass sees a more current one. + */ + count = data_race(READ_ONCE(clp->cl_deleg_count)); + if (!count) + continue; + if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &clp->cl_ra->ra_cb.cb_flags)) + continue; /* release in nfsd4_cb_recall_any_release */ kref_get(&clp->cl_nfsdfs.cl_ref); - set_bit(NFSD4_CLIENT_CB_RECALL_ANY, &clp->cl_flags); clp->cl_ra_time = ktime_get_boottime_seconds(); - } - spin_unlock(&nn->client_lock); - - while (!list_empty(&cblist)) { - clp = list_first_entry(&cblist, struct nfs4_client, - cl_ra_cblist); - list_del_init(&clp->cl_ra_cblist); - clp->cl_ra->ra_keep = 0; + /* + * Ask for a single delegation. Recalling one before it + * is needed costs the client an OPEN when it next + * touches the file. + */ + clp->cl_ra->ra_keep = count - 1; clp->cl_ra->ra_bmval[0] = BIT(RCA4_TYPE_MASK_RDATA_DLG) | BIT(RCA4_TYPE_MASK_WDATA_DLG); trace_nfsd_cb_recall_any(clp->cl_ra); nfsd4_run_cb(&clp->cl_ra->ra_cb); } + spin_unlock(&nn->client_lock); } static void @@ -7977,7 +7990,7 @@ nfsd4_lm_notify(struct file_lock *fl) if (queue) { trace_nfsd_cb_notify_lock(lo, nbl); - nfsd4_run_cb(&nbl->nbl_cb); + nfsd4_try_run_cb(&nbl->nbl_cb); } } diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c index 2a225756ef0d..f2e1079bd043 100644 --- a/fs/nfsd/nfsctl.c +++ b/fs/nfsd/nfsctl.c @@ -2107,8 +2107,13 @@ static int nfsd_nl_validate_listeners(struct genl_info *info) int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info) { struct net *net = genl_info_net(info); + const struct nlattr *bad_attr = NULL; struct svc_xprt *xprt, *tmp; + const char *bad_xprt = NULL; + unsigned int rpcb_failures; const struct nlattr *attr; + bool skipped_rpcb = false; + bool bad_rpcb = false; struct svc_serv *serv; LIST_HEAD(permsocks); struct nfsd_net *nn; @@ -2198,13 +2203,16 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info) if (delete) svc_xprt_destroy_all(serv, net); + rpcb_failures = svc_rpcb_failure_count(serv); + /* walk list of addrs again, open any that still don't exist */ nlmsg_for_each_attr_type(attr, NFSD_A_SERVER_SOCK_ADDR, info->nlhdr, GENL_HDRLEN, rem) { struct nlattr *tb[NFSD_A_SOCK_MAX + 1]; const char *xcl_name; struct sockaddr *sa; - int ret; + bool hit_rpcb; + int flags, ret; /* validated up front in nfsd_nl_validate_listeners() */ if (nla_parse_nested(tb, NFSD_A_SOCK_MAX, attr, @@ -2223,11 +2231,53 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info) continue; } - ret = svc_xprt_create_from_sa(serv, xcl_name, net, sa, 0, + flags = skipped_rpcb ? SVC_SOCK_ANONYMOUS : 0; + ret = svc_xprt_create_from_sa(serv, xcl_name, net, sa, flags, current_cred()); + + hit_rpcb = false; + if (!skipped_rpcb && + svc_rpcb_failure_count(serv) != rpcb_failures) { + skipped_rpcb = true; + hit_rpcb = true; + if (ret < 0) + ret = svc_xprt_create_from_sa(serv, xcl_name, + net, sa, + SVC_SOCK_ANONYMOUS, + current_cred()); + } + /* always save the latest error */ - if (ret < 0) + if (ret < 0) { + bad_attr = attr; + bad_xprt = xcl_name; + bad_rpcb = hit_rpcb; err = ret; + } + } + + /* + * The ack carries the errno of the last entry that failed. Point at + * that entry as well, since several entries can share a transport + * name and the errno alone cannot tell them apart. + */ + if (err) { + NL_SET_BAD_ATTR(info->extack, bad_attr); + if (bad_rpcb) + NL_SET_ERR_MSG_FMT(info->extack, + "cannot create %s listener; rpcbind did not answer", + bad_xprt); + else if (skipped_rpcb) + NL_SET_ERR_MSG_FMT(info->extack, + "cannot create %s listener; rpcbind did not answer earlier", + bad_xprt); + else + NL_SET_ERR_MSG_FMT(info->extack, + "cannot create %s listener", + bad_xprt); + } else if (skipped_rpcb) { + NL_SET_ERR_MSG(info->extack, + "rpcbind did not answer, some listeners are not registered"); } if (!serv->sv_nrthreads && list_empty(&nn->nfsd_serv->sv_permsocks)) diff --git a/fs/nfsd/pnfs.h b/fs/nfsd/pnfs.h index dfd411d1f363..db9af780438b 100644 --- a/fs/nfsd/pnfs.h +++ b/fs/nfsd/pnfs.h @@ -29,8 +29,8 @@ struct nfsd4_layout_ops { __be32 (*encode_getdeviceinfo)(struct xdr_stream *xdr, const struct nfsd4_getdeviceinfo *gdevp); - __be32 (*proc_layoutget)(struct inode *, const struct svc_fh *fhp, - struct nfsd4_layoutget *lgp); + __be32 (*proc_layoutget)(struct svc_rqst *rqstp, struct inode *inode, + const struct svc_fh *fhp, struct nfsd4_layoutget *lgp); __be32 (*encode_layoutget)(struct xdr_stream *xdr, const struct nfsd4_layoutget *lgp); diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h index f39a425639b0..1909d744d4ac 100644 --- a/fs/nfsd/state.h +++ b/fs/nfsd/state.h @@ -67,6 +67,8 @@ typedef struct { struct nfsd4_callback { struct nfs4_client *cb_clp; struct rpc_message cb_msg; +#define NFSD4_CALLBACK_RUNNING (0) + unsigned long cb_flags; const struct nfsd4_callback_ops *cb_ops; struct work_struct cb_work; int cb_seq_status; @@ -410,7 +412,6 @@ struct nfs4_client { #define NFSD4_CLIENT_UPCALL_LOCK (5) /* upcall serialization */ #define NFSD4_CLIENT_CB_FLAG_MASK (1 << NFSD4_CLIENT_CB_UPDATE | \ 1 << NFSD4_CLIENT_CB_KILL) -#define NFSD4_CLIENT_CB_RECALL_ANY (6) unsigned long cl_flags; struct workqueue_struct *cl_callback_wq; @@ -456,10 +457,11 @@ struct nfs4_client { unsigned int cl_state; atomic_t cl_delegs_in_recall; + /* Length of cl_delegations, updated under nn->deleg_lock */ + unsigned int cl_deleg_count; struct nfsd4_cb_recall_any *cl_ra; time64_t cl_ra_time; - struct list_head cl_ra_cblist; }; /* struct nfs4_client_reset @@ -748,6 +750,13 @@ extern void nfsd4_change_callback(struct nfs4_client *clp, struct nfs4_cb_conn * extern void nfsd4_init_cb(struct nfsd4_callback *cb, struct nfs4_client *clp, const struct nfsd4_callback_ops *ops, enum nfsd4_cb_op op); extern bool nfsd4_run_cb(struct nfsd4_callback *cb); + +static inline void nfsd4_try_run_cb(struct nfsd4_callback *cb) +{ + if (!test_and_set_bit(NFSD4_CALLBACK_RUNNING, &cb->cb_flags)) + WARN_ON_ONCE(!nfsd4_run_cb(cb)); +} + extern void nfsd4_shutdown_callback(struct nfs4_client *); extern void nfsd4_shutdown_copy(struct nfs4_client *clp); extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(struct xdr_netobj name, diff --git a/fs/nilfs2/dir.c b/fs/nilfs2/dir.c index b0f182b1da8d..287e7c8e5dcc 100644 --- a/fs/nilfs2/dir.c +++ b/fs/nilfs2/dir.c @@ -95,7 +95,7 @@ static void nilfs_commit_chunk(struct folio *folio, unsigned int nr_dirty; int err; - nr_dirty = nilfs_page_count_clean_buffers(&folio->page, from, to); + nr_dirty = nilfs_page_count_clean_buffers(folio, from, to); copied = block_write_end(NULL, mapping, pos, len, len, folio, NULL); if (pos + copied > dir->i_size) i_size_write(dir, pos + copied); diff --git a/fs/nilfs2/gcinode.c b/fs/nilfs2/gcinode.c index 2dbb15767df1..561c220799c7 100644 --- a/fs/nilfs2/gcinode.c +++ b/fs/nilfs2/gcinode.c @@ -46,14 +46,11 @@ * specified by @pbn to the GC pagecache with the key @blkoff. * This function sets @vbn (@pbn if @vbn is zero) in b_blocknr of the buffer. * - * Return Value: On success, 0 is returned. On Error, one of the following - * negative error code is returned. - * - * %-EIO - I/O error. - * - * %-ENOMEM - Insufficient amount of memory available. - * - * %-ENOENT - The block specified with @pbn does not exist. + * Return: 0 on success, or one of the following negative error codes on + * failure: + * * %-EIO - I/O error (including metadata corruption). + * * %-ENOENT - The block specified with @pbn does not exist. + * * %-ENOMEM - Insufficient memory available. */ int nilfs_gccache_submit_read_data(struct inode *inode, sector_t blkoff, sector_t pbn, __u64 vbn, @@ -114,12 +111,11 @@ int nilfs_gccache_submit_read_data(struct inode *inode, sector_t blkoff, * specified by @vbn to the GC pagecache. @pbn can be supplied by the * caller to avoid translation of the disk block address. * - * Return Value: On success, 0 is returned. On Error, one of the following - * negative error code is returned. - * - * %-EIO - I/O error. - * - * %-ENOMEM - Insufficient amount of memory available. + * Return: 0 on success, or one of the following negative error codes on + * failure: + * * %-EIO - I/O error (including metadata corruption). + * * %-ENOENT - Invalid virtual block address. + * * %-ENOMEM - Insufficient memory available. */ int nilfs_gccache_submit_read_node(struct inode *inode, sector_t pbn, __u64 vbn, struct buffer_head **out_bh) diff --git a/fs/nilfs2/inode.c b/fs/nilfs2/inode.c index 6932f8daf7d9..7481912284fc 100644 --- a/fs/nilfs2/inode.c +++ b/fs/nilfs2/inode.c @@ -159,7 +159,7 @@ static int nilfs_writepages(struct address_space *mapping, int err = 0; if (sb_rdonly(inode->i_sb)) { - nilfs_clear_dirty_pages(mapping); + nilfs_clear_dirty_pages(mapping, false); return -EROFS; } @@ -170,37 +170,6 @@ static int nilfs_writepages(struct address_space *mapping, return err; } -static int nilfs_writepage(struct page *page, struct writeback_control *wbc) -{ - struct folio *folio = page_folio(page); - struct inode *inode = folio->mapping->host; - int err; - - if (sb_rdonly(inode->i_sb)) { - /* - * It means that filesystem was remounted in read-only - * mode because of error or metadata corruption. But we - * have dirty pages that try to be flushed in background. - * So, here we simply discard this dirty page. - */ - nilfs_clear_folio_dirty(folio); - folio_unlock(folio); - return -EROFS; - } - - folio_redirty_for_writepage(wbc, folio); - folio_unlock(folio); - - if (wbc->sync_mode == WB_SYNC_ALL) { - err = nilfs_construct_segment(inode->i_sb); - if (unlikely(err)) - return err; - } else if (wbc->for_reclaim) - nilfs_flush_segment(inode->i_sb, inode->i_ino); - - return 0; -} - static bool nilfs_dirty_folio(struct address_space *mapping, struct folio *folio) { @@ -273,7 +242,7 @@ static int nilfs_write_end(struct file *file, struct address_space *mapping, unsigned int nr_dirty; int err; - nr_dirty = nilfs_page_count_clean_buffers(&folio->page, start, + nr_dirty = nilfs_page_count_clean_buffers(folio, start, start + copied); copied = generic_write_end(file, mapping, pos, len, copied, folio, fsdata); @@ -295,7 +264,6 @@ nilfs_direct_IO(struct kiocb *iocb, struct iov_iter *iter) } const struct address_space_operations nilfs_aops = { - .writepage = nilfs_writepage, .read_folio = nilfs_read_folio, .writepages = nilfs_writepages, .dirty_folio = nilfs_dirty_folio, @@ -304,6 +272,7 @@ const struct address_space_operations nilfs_aops = { .write_end = nilfs_write_end, .invalidate_folio = block_invalidate_folio, .direct_IO = nilfs_direct_IO, + .migrate_folio = buffer_migrate_folio_norefs, .is_partially_uptodate = block_is_partially_uptodate, }; @@ -636,10 +605,7 @@ struct inode *nilfs_iget_for_gc(struct super_block *sb, unsigned long ino, * or does nothing if the inode already has it. This function allocates * an additional inode to maintain page cache of B-tree nodes one-on-one. * - * Return Value: On success, 0 is returned. On errors, one of the following - * negative error code is returned. - * - * %-ENOMEM - Insufficient memory available. + * Return: 0 on success, or %-ENOMEM if memory is insufficient. */ int nilfs_attach_btree_node_cache(struct inode *inode) { @@ -698,11 +664,8 @@ void nilfs_detach_btree_node_cache(struct inode *inode) * in one inode and the one for b-tree node pages is set up in the * other inode, which is attached to the former inode. * - * Return Value: On success, a pointer to the inode for data pages is - * returned. On errors, one of the following negative error code is returned - * in a pointer type. - * - * %-ENOMEM - Insufficient memory available. + * Return: a pointer to the inode for data pages on success, or %-ENOMEM + * if memory is insufficient. */ struct inode *nilfs_iget_for_shadow(struct inode *inode) { diff --git a/fs/nilfs2/ioctl.c b/fs/nilfs2/ioctl.c index 03062b6bc323..11ccf18c3999 100644 --- a/fs/nilfs2/ioctl.c +++ b/fs/nilfs2/ioctl.c @@ -972,10 +972,10 @@ static int nilfs_ioctl_clean_segments(struct inode *inode, struct file *filp, * Return: 0 on success, or one of the following negative error codes on * failure: * * %-EFAULT - Failure during execution of requested operation. + * * %-EINTR - Interrupted. * * %-EIO - I/O error. * * %-ENOMEM - Insufficient memory available. * * %-ENOSPC - No space left on device (only in a panic state). - * * %-ERESTARTSYS - Interrupted. * * %-EROFS - Read only filesystem. */ static int nilfs_ioctl_sync(struct inode *inode, struct file *filp, diff --git a/fs/nilfs2/mdt.c b/fs/nilfs2/mdt.c index 2db6350b5ac2..8e66f2c13ad6 100644 --- a/fs/nilfs2/mdt.c +++ b/fs/nilfs2/mdt.c @@ -410,7 +410,7 @@ nilfs_mdt_write_page(struct page *page, struct writeback_control *wbc) * have dirty folios that try to be flushed in background. * So, here we simply discard this dirty folio. */ - nilfs_clear_folio_dirty(folio); + nilfs_clear_folio_dirty(folio, false); folio_unlock(folio); return -EROFS; } @@ -637,10 +637,10 @@ void nilfs_mdt_restore_from_shadow_map(struct inode *inode) if (mi->mi_palloc_cache) nilfs_palloc_clear_cache(inode); - nilfs_clear_dirty_pages(inode->i_mapping); + nilfs_clear_dirty_pages(inode->i_mapping, true); nilfs_copy_back_pages(inode->i_mapping, shadow->inode->i_mapping); - nilfs_clear_dirty_pages(ii->i_assoc_inode->i_mapping); + nilfs_clear_dirty_pages(ii->i_assoc_inode->i_mapping, true); nilfs_copy_back_pages(ii->i_assoc_inode->i_mapping, NILFS_I(shadow->inode)->i_assoc_inode->i_mapping); diff --git a/fs/nilfs2/page.c b/fs/nilfs2/page.c index bc698bb66f79..2fc49a26d0b0 100644 --- a/fs/nilfs2/page.c +++ b/fs/nilfs2/page.c @@ -371,8 +371,9 @@ void nilfs_copy_back_pages(struct address_space *dmap, /** * nilfs_clear_dirty_pages - discard dirty pages in address space * @mapping: address space with dirty pages for discarding + * @force: whether to clear the dirty state regardless of busy buffer heads */ -void nilfs_clear_dirty_pages(struct address_space *mapping) +void nilfs_clear_dirty_pages(struct address_space *mapping, bool force) { struct folio_batch fbatch; unsigned int i; @@ -393,7 +394,7 @@ void nilfs_clear_dirty_pages(struct address_space *mapping) * was acquired. Skip processing in that case. */ if (likely(folio->mapping == mapping)) - nilfs_clear_folio_dirty(folio); + nilfs_clear_folio_dirty(folio, force); folio_unlock(folio); } @@ -405,13 +406,16 @@ void nilfs_clear_dirty_pages(struct address_space *mapping) /** * nilfs_clear_folio_dirty - discard dirty folio * @folio: dirty folio that will be discarded + * @force: whether to clear the states regardless of busy buffer heads * * nilfs_clear_folio_dirty() clears working states including dirty state for - * the folio and its buffers. If the folio has buffers, clear only if it is - * confirmed that none of the buffer heads are busy (none have valid - * references and none are locked). + * the folio and its buffers. If the folio has buffers and force is false, + * clear only if it is confirmed that none of the buffer heads are busy (none + * have valid references and none are locked). If force is true, the states + * are cleared unconditionally, the caller should guarantee that the folio is + * not being modified concurrently. */ -void nilfs_clear_folio_dirty(struct folio *folio) +void nilfs_clear_folio_dirty(struct folio *folio, bool force) { struct buffer_head *bh, *head; @@ -424,24 +428,28 @@ void nilfs_clear_folio_dirty(struct folio *folio) BIT(BH_Async_Write) | BIT(BH_NILFS_Volatile) | BIT(BH_NILFS_Checked) | BIT(BH_NILFS_Redirected) | BIT(BH_Delay)); - bool busy, invalidated = false; + + if (!force) { + bool busy, invalidated = false; recheck_buffers: - busy = false; - bh = head; - do { - if (atomic_read(&bh->b_count) | buffer_locked(bh)) { - busy = true; - break; + busy = false; + bh = head; + do { + if (atomic_read(&bh->b_count) | + buffer_locked(bh)) { + busy = true; + break; + } + } while (bh = bh->b_this_page, bh != head); + + if (busy) { + if (invalidated) + return; + invalidate_bh_lrus(); + invalidated = true; + goto recheck_buffers; } - } while (bh = bh->b_this_page, bh != head); - - if (busy) { - if (invalidated) - return; - invalidate_bh_lrus(); - invalidated = true; - goto recheck_buffers; } bh = head; @@ -458,14 +466,14 @@ void nilfs_clear_folio_dirty(struct folio *folio) __nilfs_clear_folio_dirty(folio); } -unsigned int nilfs_page_count_clean_buffers(struct page *page, +unsigned int nilfs_page_count_clean_buffers(struct folio *folio, unsigned int from, unsigned int to) { unsigned int block_start, block_end; struct buffer_head *bh, *head; unsigned int nc = 0; - for (bh = head = page_buffers(page), block_start = 0; + for (bh = head = folio_buffers(folio), block_start = 0; bh != head || !block_start; block_start = block_end, bh = bh->b_this_page) { block_end = block_start + bh->b_size; diff --git a/fs/nilfs2/page.h b/fs/nilfs2/page.h index 64521a03a19e..c3ba3468af5c 100644 --- a/fs/nilfs2/page.h +++ b/fs/nilfs2/page.h @@ -41,10 +41,10 @@ void nilfs_folio_bug(struct folio *); int nilfs_copy_dirty_pages(struct address_space *, struct address_space *); void nilfs_copy_back_pages(struct address_space *, struct address_space *); -void nilfs_clear_folio_dirty(struct folio *folio); -void nilfs_clear_dirty_pages(struct address_space *mapping); -unsigned int nilfs_page_count_clean_buffers(struct page *, unsigned int, - unsigned int); +void nilfs_clear_folio_dirty(struct folio *folio, bool force); +void nilfs_clear_dirty_pages(struct address_space *mapping, bool force); +unsigned int nilfs_page_count_clean_buffers(struct folio *folio, + unsigned int from, unsigned int to); unsigned long nilfs_find_uncommitted_extent(struct inode *inode, sector_t start_blk, sector_t *blkoff); diff --git a/fs/nilfs2/recovery.c b/fs/nilfs2/recovery.c index 21d81097a89f..e6e044f8f7b8 100644 --- a/fs/nilfs2/recovery.c +++ b/fs/nilfs2/recovery.c @@ -757,18 +757,13 @@ static void nilfs_abort_roll_forward(struct the_nilfs *nilfs) * @sb: super block instance * @ri: pointer to a nilfs_recovery_info struct to store search results. * - * Return Value: On success, 0 is returned. On error, one of the following - * negative error code is returned. - * - * %-EINVAL - Inconsistent filesystem state. - * - * %-EIO - I/O error - * - * %-ENOSPC - No space left on device (only in a panic state). - * - * %-ERESTARTSYS - Interrupted. - * - * %-ENOMEM - Insufficient memory available. + * Return: 0 on success, or one of the following negative error codes on + * failure: + * * %-EINTR - Interrupted. + * * %-EINVAL - Inconsistent filesystem state. + * * %-EIO - I/O error. + * * %-ENOMEM - Insufficient memory available. + * * %-ENOSPC - No space left on device (only in a panic state). */ int nilfs_salvage_orphan_logs(struct the_nilfs *nilfs, struct super_block *sb, @@ -833,14 +828,11 @@ int nilfs_salvage_orphan_logs(struct the_nilfs *nilfs, * segment pointed by the superblock. It sets up struct the_nilfs through * this search. It fills nilfs_recovery_info (ri) required for recovery. * - * Return Value: On success, 0 is returned. On error, one of the following - * negative error code is returned. - * - * %-EINVAL - No valid segment found - * - * %-EIO - I/O error - * - * %-ENOMEM - Insufficient memory available. + * Return: 0 on success, or one of the following negative error codes on + * failure: + * * %-EINVAL - No valid segment found. + * * %-EIO - I/O error. + * * %-ENOMEM - Insufficient memory available. */ int nilfs_search_super_root(struct the_nilfs *nilfs, struct nilfs_recovery_info *ri) diff --git a/fs/nilfs2/segbuf.c b/fs/nilfs2/segbuf.c index dc431b4c34c9..8b62234d5dd5 100644 --- a/fs/nilfs2/segbuf.c +++ b/fs/nilfs2/segbuf.c @@ -403,12 +403,7 @@ static int nilfs_segbuf_submit_bh(struct nilfs_segment_buffer *segbuf, * @segbuf: buffer storing a log to be written * @nilfs: nilfs object * - * Return Value: On Success, 0 is returned. On Error, one of the following - * negative error code is returned. - * - * %-EIO - I/O error - * - * %-ENOMEM - Insufficient memory available. + * Return: Always 0. */ static int nilfs_segbuf_write(struct nilfs_segment_buffer *segbuf, struct the_nilfs *nilfs) @@ -449,10 +444,7 @@ static int nilfs_segbuf_write(struct nilfs_segment_buffer *segbuf, * nilfs_segbuf_wait - wait for completion of requested BIOs * @segbuf: segment buffer * - * Return Value: On Success, 0 is returned. On Error, one of the following - * negative error code is returned. - * - * %-EIO - I/O error + * Return: 0 on success, or %-EIO if I/O error is detected. */ static int nilfs_segbuf_wait(struct nilfs_segment_buffer *segbuf) { diff --git a/fs/nilfs2/segment.c b/fs/nilfs2/segment.c index 063400f8a73c..9da544759f10 100644 --- a/fs/nilfs2/segment.c +++ b/fs/nilfs2/segment.c @@ -191,12 +191,10 @@ static int nilfs_prepare_segment_lock(struct super_block *sb, * When @vacancy_check flag is set, this function will check the amount of * free space, and will wait for the GC to reclaim disk space if low capacity. * - * Return Value: On success, 0 is returned. On error, one of the following - * negative error code is returned. - * - * %-ENOMEM - Insufficient memory available. - * - * %-ENOSPC - No space left on device + * Return: 0 on success, or one of the following negative error codes on + * failure: + * * %-ENOMEM - Insufficient memory available. + * * %-ENOSPC - No space left on device (if checking free space). */ int nilfs_transaction_begin(struct super_block *sb, struct nilfs_transaction_info *ti, @@ -2266,7 +2264,7 @@ static int nilfs_segctor_sync(struct nilfs_sc_info *sci) wake_up(&sci->sc_wait_daemon); for (;;) { - set_current_state(TASK_INTERRUPTIBLE); + set_current_state(TASK_KILLABLE); /* * Synchronize only while the log writer thread is alive. @@ -2280,11 +2278,11 @@ static int nilfs_segctor_sync(struct nilfs_sc_info *sci) err = wait_req.err; break; } - if (!signal_pending(current)) { + if (!fatal_signal_pending(current)) { schedule(); continue; } - err = -ERESTARTSYS; + err = -EINTR; break; } finish_wait(&sci->sc_wait_request, &wait_req.wq); @@ -2316,18 +2314,13 @@ static void nilfs_segctor_wakeup(struct nilfs_sc_info *sci, int err, bool force) * nilfs_construct_segment - construct a logical segment * @sb: super block * - * Return Value: On success, 0 is returned. On errors, one of the following - * negative error code is returned. - * - * %-EROFS - Read only filesystem. - * - * %-EIO - I/O error - * - * %-ENOSPC - No space left on device (only in a panic state). - * - * %-ERESTARTSYS - Interrupted. - * - * %-ENOMEM - Insufficient memory available. + * Return: 0 on success, or one of the following negative error codes on + * failure: + * * %-EINTR - Interrupted. + * * %-EIO - I/O error (including metadata corruption). + * * %-ENOMEM - Insufficient memory available. + * * %-ENOSPC - No space left on device (only in a panic state). + * * %-EROFS - Read only filesystem. */ int nilfs_construct_segment(struct super_block *sb) { @@ -2351,18 +2344,13 @@ int nilfs_construct_segment(struct super_block *sb) * @start: start byte offset * @end: end byte offset (inclusive) * - * Return Value: On success, 0 is returned. On errors, one of the following - * negative error code is returned. - * - * %-EROFS - Read only filesystem. - * - * %-EIO - I/O error - * - * %-ENOSPC - No space left on device (only in a panic state). - * - * %-ERESTARTSYS - Interrupted. - * - * %-ENOMEM - Insufficient memory available. + * Return: 0 on success, or one of the following negative error codes on + * failure: + * * %-EINTR - Interrupted. + * * %-EIO - I/O error (including metadata corruption). + * * %-ENOMEM - Insufficient memory available. + * * %-ENOSPC - No space left on device (only in a panic state). + * * %-EROFS - Read only filesystem. */ int nilfs_construct_dsync_segment(struct super_block *sb, struct inode *inode, loff_t start, loff_t end) diff --git a/fs/nilfs2/the_nilfs.c b/fs/nilfs2/the_nilfs.c index 575a543ee736..a0f14c1b3ef8 100644 --- a/fs/nilfs2/the_nilfs.c +++ b/fs/nilfs2/the_nilfs.c @@ -49,8 +49,8 @@ void nilfs_set_last_segment(struct the_nilfs *nilfs, * alloc_nilfs - allocate a nilfs object * @sb: super block instance * - * Return Value: On success, pointer to the_nilfs is returned. - * On error, NULL is returned. + * Return: a pointer to the allocated nilfs object on success, or NULL on + * failure. */ struct the_nilfs *alloc_nilfs(struct super_block *sb) { @@ -200,8 +200,7 @@ static int nilfs_store_log_cursor(struct the_nilfs *nilfs, * exponent information written in @sbp and stores it in @blocksize, * or aborts with an error message if it's too large. * - * Return Value: On success, 0 is returned. If the block size is too - * large, -EINVAL is returned. + * Return: 0 on success, or %-EINVAL if the block size is too large. */ static int nilfs_get_blocksize(struct super_block *sb, struct nilfs_super_block *sbp, int *blocksize) @@ -544,7 +543,7 @@ static int nilfs_valid_sb(struct nilfs_super_block *sbp) * area, or if the parameters themselves are not normal, it is * determined to be invalid. * - * Return Value: true if invalid, false if valid. + * Return: true if invalid, false if valid. */ static bool nilfs_sb2_bad_offset(struct nilfs_super_block *sbp, u64 offset) { @@ -690,8 +689,7 @@ static int nilfs_load_super_block(struct the_nilfs *nilfs, * reading the super block, getting disk layout information, initializing * shared fields in the_nilfs). * - * Return Value: On success, 0 is returned. On error, a negative error - * code is returned. + * Return: 0 on success, or a negative error code on failure. */ int init_nilfs(struct the_nilfs *nilfs, struct super_block *sb) { diff --git a/fs/notify/inotify/inotify_fsnotify.c b/fs/notify/inotify/inotify_fsnotify.c index 993375f0db67..ff0af29745a8 100644 --- a/fs/notify/inotify/inotify_fsnotify.c +++ b/fs/notify/inotify/inotify_fsnotify.c @@ -36,7 +36,7 @@ static bool event_compare(struct fsnotify_event *old_fsn, old = INOTIFY_E(old_fsn); new = INOTIFY_E(new_fsn); - if (old->mask & FS_IN_IGNORED) + if (old->mask & (FS_IN_IGNORED | FS_MOVE)) return false; if ((old->mask == new->mask) && (old->wd == new->wd) && diff --git a/fs/ntfs3/xattr.c b/fs/ntfs3/xattr.c index ee69f3447597..f929cef0f510 100644 --- a/fs/ntfs3/xattr.c +++ b/fs/ntfs3/xattr.c @@ -146,29 +146,32 @@ static int ntfs_read_ea(struct ntfs_inode *ni, struct EA_FULL **ea, for (off = 0; off < size; off += ea_size) { const struct EA_FULL *ef = Add2Ptr(ea_p, off); u32 bytes = size - off; - size_t need; - - /* Check if we can use field ea->size. */ - if (bytes < sizeof(ef->size)) - goto out1; + u32 packed_size; /* Check if we can use fields ef->name_len and ef->elength. */ if (bytes < offsetof(struct EA_FULL, name)) goto out1; - /* Size needed to hold this record's name and value. */ - need = struct_size(ef, name, - 1 + ef->name_len + le16_to_cpu(ef->elength)); + /* + * The name and the value have to fit in the entry whether or + * not ef->size is set, because ntfs_get_ea() copies elength + * bytes out of this buffer and checks them only against the + * caller supplied buffer size. + */ + packed_size = + ALIGN(struct_size(ef, name, + 1 + ef->name_len + + le16_to_cpu(ef->elength)), + 4); if (ef->size) { ea_size = le32_to_cpu(ef->size); - /* ef->size must fit the list and cover the record. */ - if (ea_size > bytes || ea_size < need) + if (ea_size > bytes || packed_size > ea_size) goto out1; continue; } - ea_size = ALIGN(need, 4); + ea_size = packed_size; if (ea_size > bytes) goto out1; } diff --git a/fs/ocfs2/alloc.c b/fs/ocfs2/alloc.c index eb6adb785e45..f33aa48a3bc5 100644 --- a/fs/ocfs2/alloc.c +++ b/fs/ocfs2/alloc.c @@ -1796,6 +1796,7 @@ static int __ocfs2_find_path(struct ocfs2_caching_info *ci, int i, ret = 0; u32 range; u64 blkno; + u32 prev_depth = OCFS2_MAX_PATH_DEPTH; struct buffer_head *bh = NULL; struct ocfs2_extent_block *eb; struct ocfs2_extent_list *el; @@ -1803,22 +1804,25 @@ static int __ocfs2_find_path(struct ocfs2_caching_info *ci, el = root_el; while (el->l_tree_depth) { - if (unlikely(le16_to_cpu(el->l_tree_depth) >= OCFS2_MAX_PATH_DEPTH)) { + if (unlikely(le16_to_cpu(el->l_tree_depth) >= prev_depth)) { ocfs2_error(ocfs2_metadata_cache_get_super(ci), - "Owner %llu has invalid tree depth %u in extent list\n", + "Owner %llu has invalid tree depth %u in extent list (max %u)\n", (unsigned long long)ocfs2_metadata_cache_owner(ci), - le16_to_cpu(el->l_tree_depth)); + le16_to_cpu(el->l_tree_depth), prev_depth - 1); ret = -EROFS; goto out; } - if (le16_to_cpu(el->l_next_free_rec) == 0) { + prev_depth = le16_to_cpu(el->l_tree_depth); + + if (!el->l_next_free_rec || !el->l_count) { ocfs2_error(ocfs2_metadata_cache_get_super(ci), - "Owner %llu has empty extent list at depth %u\n", + "Owner %llu has empty extent list at depth %u\n" + "(next free=%u count=%u)\n", (unsigned long long)ocfs2_metadata_cache_owner(ci), - le16_to_cpu(el->l_tree_depth)); + le16_to_cpu(el->l_tree_depth), + le16_to_cpu(el->l_next_free_rec), le16_to_cpu(el->l_count)); ret = -EROFS; goto out; - } for(i = 0; i < le16_to_cpu(el->l_next_free_rec) - 1; i++) { diff --git a/fs/ocfs2/cluster/tcp.c b/fs/ocfs2/cluster/tcp.c index dd4fb83d9cd8..a53b2c47f55d 100644 --- a/fs/ocfs2/cluster/tcp.c +++ b/fs/ocfs2/cluster/tcp.c @@ -391,6 +391,9 @@ static void sc_kref_release(struct kref *kref) sclog(sc, "releasing\n"); + /* Unlink before freeing anything the debugfs readers dereference. */ + o2net_debug_del_sc(sc); + if (sc->sc_sock) { sock_release(sc->sc_sock); sc->sc_sock = NULL; @@ -400,8 +403,6 @@ static void sc_kref_release(struct kref *kref) o2nm_node_put(sc->sc_node); sc->sc_node = NULL; - o2net_debug_del_sc(sc); - if (sc->sc_page) __free_page(sc->sc_page); kfree(sc); diff --git a/fs/ocfs2/dcache.h b/fs/ocfs2/dcache.h index 7f246c5692d8..046ab9a80769 100644 --- a/fs/ocfs2/dcache.h +++ b/fs/ocfs2/dcache.h @@ -17,9 +17,8 @@ struct ocfs2_dentry_lock { u64 dl_parent_blkno; /* - * The ocfs2_dentry_lock keeps an inode reference until - * dl_lockres has been destroyed. This is usually done in - * ->d_iput() anyway, so there should be minimal impact. + * The inode reference is dropped before dl_lockres is unlocked. + * Callbacks must not use it to find the mount context during teardown. */ struct inode *dl_inode; struct ocfs2_lock_res dl_lockres; diff --git a/fs/ocfs2/dlm/dlmmaster.c b/fs/ocfs2/dlm/dlmmaster.c index d84e816c70c5..c9c5e5bd7d88 100644 --- a/fs/ocfs2/dlm/dlmmaster.c +++ b/fs/ocfs2/dlm/dlmmaster.c @@ -2537,6 +2537,52 @@ static int dlm_is_lockres_migratable(struct dlm_ctxt *dlm, */ +/* + * Drop the sender's references on a migration MLE after the attempt has + * failed. The finishing assert from the target may already have unlinked + * the MLE and consumed its base reference; decide under the locks the + * assert handler holds, and only drop the base reference while the MLE is + * still hashed. Returns true when the assert got there first: the target + * has become the master, so the caller must complete the migration. + */ +static bool dlm_put_migration_mle(struct dlm_ctxt *dlm, + struct dlm_master_list_entry *mle) +{ + bool asserted; + + spin_lock(&dlm->spinlock); + spin_lock(&dlm->master_lock); + __dlm_mle_detach_hb_events(dlm, mle); + asserted = hlist_unhashed(&mle->master_hash_node); + if (!asserted) { + __dlm_unlink_mle(dlm, mle); + __dlm_put_mle(mle); + } + spin_unlock(&dlm->master_lock); + spin_unlock(&dlm->spinlock); + + dlm_put_mle_inuse(mle); + return asserted; +} + +/* + * The finishing assert got there first: it switched the owner to the + * target and cleared DLM_LOCK_RES_MIGRATING. Drop the other nodes' locks + * as a completed migration does, unless recovery has taken the lockres + * over since, and let the usage be recalculated. + */ +static void dlm_migrate_lockres_asserted(struct dlm_ctxt *dlm, + struct dlm_lock_resource *res, + u8 target) +{ + spin_lock(&res->spinlock); + if (res->owner == target) + dlm_remove_nonlocal_locks(dlm, res); + spin_unlock(&res->spinlock); + + dlm_lockres_calc_usage(dlm, res); +} + static int dlm_migrate_lockres(struct dlm_ctxt *dlm, struct dlm_lock_resource *res, u8 target) { @@ -2621,9 +2667,10 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm, if (ret < 0) { if (mle_added) { - dlm_mle_detach_hb_events(dlm, mle); - dlm_put_mle(mle); - dlm_put_mle_inuse(mle); + if (dlm_put_migration_mle(dlm, mle)) { + dlm_migrate_lockres_asserted(dlm, res, target); + ret = 0; + } } else if (mle) { kmem_cache_free(dlm_mle_cache, mle); mle = NULL; @@ -2652,9 +2699,11 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm, mlog(0, "migration to node %u failed with %d\n", target, ret); /* migration failed, detach and clean up mle */ - dlm_mle_detach_hb_events(dlm, mle); - dlm_put_mle(mle); - dlm_put_mle_inuse(mle); + if (dlm_put_migration_mle(dlm, mle)) { + dlm_migrate_lockres_asserted(dlm, res, target); + ret = 0; + goto leave; + } spin_lock(&res->spinlock); res->state &= ~DLM_LOCK_RES_MIGRATING; wake = 1; @@ -2698,9 +2747,12 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm, res->lockname.name, target); ret = -EINVAL; /* migration failed, detach and clean up mle */ - dlm_mle_detach_hb_events(dlm, mle); - dlm_put_mle(mle); - dlm_put_mle_inuse(mle); + if (dlm_put_migration_mle(dlm, mle)) { + dlm_migrate_lockres_asserted(dlm, res, + target); + ret = 0; + goto leave; + } spin_lock(&res->spinlock); res->state &= ~DLM_LOCK_RES_MIGRATING; wake = 1; diff --git a/fs/ocfs2/dlmglue.c b/fs/ocfs2/dlmglue.c index ec44ed33afdf..96d8f39a653a 100644 --- a/fs/ocfs2/dlmglue.c +++ b/fs/ocfs2/dlmglue.c @@ -56,7 +56,6 @@ struct ocfs2_mask_waiter { #endif }; -static struct ocfs2_super *ocfs2_get_dentry_osb(struct ocfs2_lock_res *lockres); static struct ocfs2_super *ocfs2_get_inode_osb(struct ocfs2_lock_res *lockres); static struct ocfs2_super *ocfs2_get_file_osb(struct ocfs2_lock_res *lockres); static struct ocfs2_super *ocfs2_get_qinfo_osb(struct ocfs2_lock_res *lockres); @@ -256,7 +255,6 @@ static const struct ocfs2_lock_res_ops ocfs2_orphan_scan_lops = { }; static const struct ocfs2_lock_res_ops ocfs2_dentry_lops = { - .get_osb = ocfs2_get_dentry_osb, .post_unlock = ocfs2_dentry_post_unlock, .downconvert_worker = ocfs2_dentry_convert_worker, .flags = 0, @@ -307,7 +305,7 @@ static inline struct ocfs2_dentry_lock *ocfs2_lock_res_dl(struct ocfs2_lock_res { BUG_ON(lockres->l_type != OCFS2_LOCK_TYPE_DENTRY); - return (struct ocfs2_dentry_lock *)lockres->l_priv; + return container_of(lockres, struct ocfs2_dentry_lock, dl_lockres); } static inline struct ocfs2_mem_dqinfo *ocfs2_lock_res_qinfo(struct ocfs2_lock_res *lockres) @@ -608,13 +606,6 @@ static __u64 ocfs2_get_dentry_lock_ino(struct ocfs2_lock_res *lockres) return be64_to_cpu(inode_blkno_be); } -static struct ocfs2_super *ocfs2_get_dentry_osb(struct ocfs2_lock_res *lockres) -{ - struct ocfs2_dentry_lock *dl = lockres->l_priv; - - return OCFS2_SB(dl->dl_inode->i_sb); -} - void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl, u64 parent, struct inode *inode) { @@ -622,6 +613,7 @@ void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl, u64 inode_blkno = OCFS2_I(inode)->ip_blkno; __be64 inode_blkno_be = cpu_to_be64(inode_blkno); struct ocfs2_lock_res *lockres = &dl->dl_lockres; + struct ocfs2_super *osb = OCFS2_SB(inode->i_sb); ocfs2_lock_res_init_once(lockres); @@ -650,9 +642,9 @@ void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl, memcpy(&lockres->l_name[OCFS2_DENTRY_LOCK_INO_START], &inode_blkno_be, sizeof(__be64)); - ocfs2_lock_res_init_common(OCFS2_SB(inode->i_sb), lockres, + ocfs2_lock_res_init_common(osb, lockres, OCFS2_LOCK_TYPE_DENTRY, &ocfs2_dentry_lops, - dl); + osb); } static void ocfs2_super_lock_res_init(struct ocfs2_lock_res *res, @@ -3336,6 +3328,14 @@ static void ocfs2_dlm_shutdown_debug(struct ocfs2_super *osb) { struct ocfs2_dlm_debug *dlm_debug = osb->osb_dlm_debug; + /* + * Drain the readers before the cluster stack goes away: the + * locking_state reader resolves ocfs2_dlm_lvb() through the + * stack. Every caller removes the parent directory later. + */ + debugfs_lookup_and_remove("locking_state", osb->osb_debug_root); + debugfs_lookup_and_remove("locking_filter", osb->osb_debug_root); + if (dlm_debug) ocfs2_put_dlm_debug(dlm_debug); } @@ -3350,8 +3350,6 @@ int ocfs2_dlm_init(struct ocfs2_super *osb) goto local; } - ocfs2_dlm_init_debug(osb); - /* launch downconvert thread */ osb->dc_task = kthread_run(ocfs2_downconvert_thread, osb, "ocfs2dc-%s", osb->uuid_str); @@ -3391,9 +3389,10 @@ int ocfs2_dlm_init(struct ocfs2_super *osb) ocfs2_orphan_scan_lock_res_init(&osb->osb_orphan_scan.os_lockres, osb); osb->cconn = conn; + if (conn) + ocfs2_dlm_init_debug(osb); bail: if (status < 0) { - ocfs2_dlm_shutdown_debug(osb); if (osb->dc_task) kthread_stop(osb->dc_task); } @@ -3423,10 +3422,11 @@ void ocfs2_dlm_shutdown(struct ocfs2_super *osb, ocfs2_lock_res_free(&osb->osb_orphan_scan.os_lockres); if (osb->cconn) { + /* The disconnect below may release the selected stack. */ + ocfs2_dlm_shutdown_debug(osb); + ocfs2_cluster_disconnect(osb->cconn, hangup_pending); osb->cconn = NULL; - - ocfs2_dlm_shutdown_debug(osb); } } diff --git a/fs/ocfs2/filecheck.c b/fs/ocfs2/filecheck.c index c04b55bfaa6d..40b59f00ed57 100644 --- a/fs/ocfs2/filecheck.c +++ b/fs/ocfs2/filecheck.c @@ -189,6 +189,8 @@ int ocfs2_filecheck_create_sysfs(struct ocfs2_super *osb) /* Mount failure still calls ocfs2_filecheck_remove_sysfs(). */ entry->fs_fcheck = NULL; kobject_put(&entry->fs_kobj); + /* Keep ocfs2_super alive until the kobject is released. */ + wait_for_completion(&entry->fs_kobj_unregister); kfree(fcheck); return ret; } diff --git a/fs/ocfs2/inode.c b/fs/ocfs2/inode.c index ce78ef998217..51e487ea432e 100644 --- a/fs/ocfs2/inode.c +++ b/fs/ocfs2/inode.c @@ -575,14 +575,18 @@ static int ocfs2_read_locked_inode(struct inode *inode, fe = (struct ocfs2_dinode *) bh->b_data; /* - * This is a code bug. Right now the caller needs to - * understand whether it is asking for a system file inode or - * not so the proper lock names can be built. + * The caller must know whether it is asking for a system file inode + * or not so the proper lock names can be built. Since i_flags comes + * from disk, a mismatch is filesystem corruption instead of a code + * bug, so handle it with ocfs2_error() rather than BUG_ON(). */ - mlog_bug_on_msg(!!(fe->i_flags & cpu_to_le32(OCFS2_SYSTEM_FL)) != - !!(args->fi_flags & OCFS2_FI_FLAG_SYSFILE), - "Inode %llu: system file state is ambiguous\n", - (unsigned long long)args->fi_blkno); + if (!!(fe->i_flags & cpu_to_le32(OCFS2_SYSTEM_FL)) != + !!(args->fi_flags & OCFS2_FI_FLAG_SYSFILE)) { + status = ocfs2_error(osb->sb, + "Inode %llu: system file state is ambiguous\n", + (unsigned long long)args->fi_blkno); + goto bail; + } if (S_ISCHR(le16_to_cpu(fe->i_mode)) || S_ISBLK(le16_to_cpu(fe->i_mode))) @@ -1599,8 +1603,23 @@ int ocfs2_validate_inode_block(struct super_block *sb, } } - if (le16_to_cpu(di->i_suballoc_slot) != (u16)OCFS2_INVALID_SLOT && - (u32)le16_to_cpu(di->i_suballoc_slot) > OCFS2_SB(sb)->max_slots - 1) { + /* + * Only system inodes created by mkfs.ocfs2 are allocated from the + * global allocator and thus legitimately carry OCFS2_INVALID_SLOT. + * Regular inodes are always allocated from a per-slot suballocator. + * If a regular inode with OCFS2_INVALID_SLOT was accepted here, + * deleting it would pass the slot to get_local_system_inode() via + * ocfs2_remove_inode() and trigger BUG_ON(slot == OCFS2_INVALID_SLOT). + */ + if (le16_to_cpu(di->i_suballoc_slot) == (u16)OCFS2_INVALID_SLOT) { + if (!(le32_to_cpu(di->i_flags) & OCFS2_SYSTEM_FL)) { + rc = ocfs2_error(sb, + "Invalid dinode %llu: suballoc slot %u for non-system inode\n", + (unsigned long long)bh->b_blocknr, + le16_to_cpu(di->i_suballoc_slot)); + goto bail; + } + } else if ((u32)le16_to_cpu(di->i_suballoc_slot) > OCFS2_SB(sb)->max_slots - 1) { rc = ocfs2_error(sb, "Invalid dinode %llu: suballoc slot %u\n", (unsigned long long)bh->b_blocknr, le16_to_cpu(di->i_suballoc_slot)); diff --git a/fs/ocfs2/journal.c b/fs/ocfs2/journal.c index 66427185a5d0..e8f6e3a554a3 100644 --- a/fs/ocfs2/journal.c +++ b/fs/ocfs2/journal.c @@ -230,13 +230,22 @@ void ocfs2_recovery_disable_quota(struct ocfs2_super *osb) ocfs2_recovery_disable(osb, OCFS2_REC_QUOTA_WANT_DISABLE); } +/* Stop producers and drain completion work without freeing recovery state. */ +void ocfs2_recovery_stop(struct ocfs2_super *osb) +{ + ocfs2_recovery_disable(osb, OCFS2_REC_WANT_DISABLE); +} + void ocfs2_recovery_exit(struct ocfs2_super *osb) { struct ocfs2_recovery_map *rm; - /* disable any new recovery threads and wait for any currently - * running ones to exit. Do this before setting the vol_state. */ - ocfs2_recovery_disable(osb, OCFS2_REC_WANT_DISABLE); + /* + * Disable producers and drain work before freeing recovery state. + * Do this before setting vol_state to VOLUME_DISMOUNTED: completion + * work may still be waiting in __ocfs2_wait_on_mount(). + */ + ocfs2_recovery_stop(osb); /* * Now that recovery is shut down, and the osb is about to be diff --git a/fs/ocfs2/journal.h b/fs/ocfs2/journal.h index 19fc920d26b1..4b52bf7b0938 100644 --- a/fs/ocfs2/journal.h +++ b/fs/ocfs2/journal.h @@ -147,6 +147,7 @@ void ocfs2_complete_recovery(struct work_struct *work); void ocfs2_wait_for_recovery(struct ocfs2_super *osb); int ocfs2_recovery_init(struct ocfs2_super *osb); +void ocfs2_recovery_stop(struct ocfs2_super *osb); void ocfs2_recovery_exit(struct ocfs2_super *osb); void ocfs2_recovery_disable_quota(struct ocfs2_super *osb); diff --git a/fs/ocfs2/super.c b/fs/ocfs2/super.c index 3535881de652..2a11202b5220 100644 --- a/fs/ocfs2/super.c +++ b/fs/ocfs2/super.c @@ -1884,6 +1884,9 @@ static void ocfs2_dismount_volume(struct super_block *sb, int mnt_err) /* Wait for worker to be done with the work structure in osb */ cancel_work_sync(&osb->dquot_drop_work); + /* Stop recovery producers before releasing their local allocators. */ + ocfs2_recovery_stop(osb); + ocfs2_shutdown_local_alloc(osb); ocfs2_truncate_log_shutdown(osb); diff --git a/fs/ocfs2/xattr.c b/fs/ocfs2/xattr.c index f73bfcfcde54..037ae2200c83 100644 --- a/fs/ocfs2/xattr.c +++ b/fs/ocfs2/xattr.c @@ -256,6 +256,7 @@ static int ocfs2_xattr_tree_list_index_block(struct inode *inode, size_t buffer_size); static int ocfs2_xattr_create_index_block(struct inode *inode, + struct ocfs2_xattr_info *xi, struct ocfs2_xattr_search *xs, struct ocfs2_xattr_set_ctxt *ctxt); @@ -3228,7 +3229,7 @@ static int ocfs2_xattr_block_set(struct inode *inode, else if ((ret != -ENOSPC) || ctxt->set_abort) goto end; else { - ret = ocfs2_xattr_create_index_block(inode, xs, ctxt); + ret = ocfs2_xattr_create_index_block(inode, xi, xs, ctxt); if (ret) goto end; } @@ -4510,6 +4511,21 @@ static void ocfs2_cp_xattr_block_to_bucket(struct inode *inode, for (i = 0; i < blks; i++) memset(bucket_block(bucket, i), 0, blocksize); + /* + * An empty block has no names or values to carry over, and the + * xh_entries[count - 1] below would be xh_entries[-1]. Leave the + * bucket empty, but carry over the hash ocfs2_xa_remove_entry() left + * in the block's xh_entries[0] so that the bucket still holds the key + * it is looked up by. + */ + if (!count) { + xh->xh_num_buckets = cpu_to_le16(1); + xh->xh_free_start = cpu_to_le16(OCFS2_XATTR_BUCKET_SIZE); + xh->xh_entries[0].xe_name_hash = + xb_xh->xh_entries[0].xe_name_hash; + return; + } + /* * Since the xe_name_offset is based on ocfs2_xattr_header, * there is a offset change corresponding to the change of @@ -4556,13 +4572,10 @@ static void ocfs2_cp_xattr_block_to_bucket(struct inode *inode, * real place of the xattr. */ static void ocfs2_xattr_update_xattr_search(struct inode *inode, - struct ocfs2_xattr_search *xs, - struct buffer_head *old_bh) + struct ocfs2_xattr_info *xi, + struct ocfs2_xattr_search *xs) { - char *buf = old_bh->b_data; - struct ocfs2_xattr_block *old_xb = (struct ocfs2_xattr_block *)buf; - struct ocfs2_xattr_header *old_xh = &old_xb->xb_attrs.xb_header; - int i; + int i, count; xs->header = bucket_xh(xs->bucket); xs->base = bucket_block(xs->bucket, 0); @@ -4571,11 +4584,47 @@ static void ocfs2_xattr_update_xattr_search(struct inode *inode, if (xs->not_found) return; - i = xs->here - old_xh->xh_entries; - xs->here = &xs->header->xh_entries[i]; + /* + * A block holds its entries in insertion order while a bucket holds + * them by name hash, so ocfs2_cp_xattr_block_to_bucket() sorted the + * copies and the index this entry had in the block no longer says + * where it is. It copied every entry and every name unchanged, and + * (type, name) pairs are unique within a block, so find it again by + * name. + * + * Search for the name being set, not for the name xs->here carries. + * A failed set may have removed that entry, and ocfs2_xa_remove_entry() + * then memmoves its successor into the slot. Comparing every entry + * rather than narrowing on the hash also finds an entry stored under + * the legacy signed hash, which ocfs2_find_xe_in_bucket() would skip. + */ + count = le16_to_cpu(xs->header->xh_count); + for (i = 0; i < count; i++) { + struct ocfs2_xattr_entry *xe = &xs->header->xh_entries[i]; + + if (ocfs2_xattr_get_type(xe) != xi->xi_name_index) + continue; + if (xe->xe_name_len != xi->xi_name_len) + continue; + if (!memcmp(xs->base + le16_to_cpu(xe->xe_name_offset), + xi->xi_name, xi->xi_name_len)) { + xs->here = xe; + return; + } + } + + /* + * The entry did not survive the failed set, so this has become an + * insert. Point xs->here back inside the bucket: it still refers to + * the block being converted, which is about to be rewritten as a tree + * root. + */ + xs->not_found = -ENODATA; + xs->here = xs->header->xh_entries; } static int ocfs2_xattr_create_index_block(struct inode *inode, + struct ocfs2_xattr_info *xi, struct ocfs2_xattr_search *xs, struct ocfs2_xattr_set_ctxt *ctxt) { @@ -4642,7 +4691,7 @@ static int ocfs2_xattr_create_index_block(struct inode *inode, ocfs2_cp_xattr_block_to_bucket(inode, xb_bh, xs->bucket); ocfs2_xattr_bucket_journal_dirty(handle, xs->bucket); - ocfs2_xattr_update_xattr_search(inode, xs, xb_bh); + ocfs2_xattr_update_xattr_search(inode, xi, xs); /* Change from ocfs2_xattr_header to ocfs2_xattr_tree_root */ memset(&xb->xb_attrs, 0, inode->i_sb->s_blocksize - @@ -4995,7 +5044,15 @@ static int ocfs2_divide_xattr_bucket(struct inode *inode, start = ocfs2_xattr_find_divide_pos(xh); if (start == count) { - xe = &xh->xh_entries[start-1]; + /* + * start == count means every entry shares one hash, so the + * first of them carries it too. An empty bucket has no + * entries[start - 1], but its entries[0] still carries the + * bucket key: either the hash ocfs2_xa_remove_entry() + * deliberately leaves behind, or the sentinel a previous + * divide wrote. + */ + xe = &xh->xh_entries[0]; /* * initialized a new empty bucket here. @@ -5825,11 +5882,20 @@ static int ocfs2_check_xattr_bucket_collision(struct ocfs2_xattr_bucket *bucket, u32 name_hash) { struct ocfs2_xattr_header *xh = bucket_xh(bucket); + u16 count = le16_to_cpu(xh->xh_count); + + /* + * An empty bucket keeps the hash of the entry that was removed in + * xh_entries[0] so that it can still be found, but it holds no + * entries to collide with. + */ + if (!count) + return 0; if (name_hash != le32_to_cpu(xh->xh_entries[0].xe_name_hash)) return 0; - if (xh->xh_entries[le16_to_cpu(xh->xh_count) - 1].xe_name_hash == + if (xh->xh_entries[count - 1].xe_name_hash == xh->xh_entries[0].xe_name_hash) { mlog(ML_ERROR, "Too much hash collision in xattr bucket %llu, " "hash = %u\n", @@ -5850,13 +5916,18 @@ static int ocfs2_xattr_set_entry_bucket(struct inode *inode, struct ocfs2_xattr_search *xs, struct ocfs2_xattr_set_ctxt *ctxt) { - int ret; + int ret, found = 0; + u16 xe_index; + u32 name_hash; struct ocfs2_xa_loc loc; trace_ocfs2_xattr_set_entry_bucket(xi->xi_name); ocfs2_init_xattr_bucket_xa_loc(&loc, xs->bucket, xs->not_found ? NULL : xs->here); + /* Defragmenting below may move this entry, so remember its hash. */ + name_hash = loc.xl_entry ? le32_to_cpu(loc.xl_entry->xe_name_hash) : 0; + ret = ocfs2_xa_set(&loc, xi, ctxt); if (!ret) { xs->here = loc.xl_entry; @@ -5875,6 +5946,29 @@ static int ocfs2_xattr_set_entry_bucket(struct inode *inode, goto out; } + /* + * ocfs2_defrag_xattr_bucket() sorts the entries by name hash and + * lib/sort.c is not stable, so entries sharing a hash may have traded + * slots. Look the entry up again before writing through it. + */ + if (loc.xl_entry) { + ret = ocfs2_find_xe_in_bucket(inode, xs->bucket, + xi->xi_name_index, xi->xi_name, + name_hash, &xe_index, &found); + if (ret) { + mlog_errno(ret); + goto out; + } + if (!found) { + mlog(ML_ERROR, "xattr entry not found in bucket %llu after defrag\n", + (unsigned long long)bucket_blkno(xs->bucket)); + ret = -EFSCORRUPTED; + goto out; + } + loc.xl_entry = &bucket_xh(xs->bucket)->xh_entries[xe_index]; + xs->here = loc.xl_entry; + } + ret = ocfs2_xa_set(&loc, xi, ctxt); if (!ret) { xs->here = loc.xl_entry; diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c index 444aeeccb6da..576c89e440f7 100644 --- a/fs/overlayfs/export.c +++ b/fs/overlayfs/export.c @@ -720,9 +720,9 @@ static struct dentry *ovl_lower_fh_to_d(struct super_block *sb, } } - /* Then lookup indexed upper/whiteout by origin fh */ + /* Then lookup indexed upper/whiteout by origin */ if (ovl_indexdir(sb)) { - index = ovl_get_index_fh(ofs, fh); + index = ovl_get_index_origin(ofs, origin.dentry); err = PTR_ERR(index); if (IS_ERR(index)) { index = NULL; diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c index 8e8c416a06fd..9a44af6bf911 100644 --- a/fs/overlayfs/namei.c +++ b/fs/overlayfs/namei.c @@ -748,14 +748,21 @@ int ovl_get_index_name(struct ovl_fs *ofs, struct dentry *origin, return err; } -/* Lookup index by file handle for NFS export */ -struct dentry *ovl_get_index_fh(struct ovl_fs *ofs, struct ovl_fh *fh) +/* + * Lookup index by decoded origin for NFS export. + * + * The name of an index is the hex of the file handle that copy up encoded. + * The bytes of the handle to decode cannot be used as the name, because a + * lower fs may decode many handles to the same inode. For example, it may + * ignore the fid type, extra fid bytes or a zero generation. + */ +struct dentry *ovl_get_index_origin(struct ovl_fs *ofs, struct dentry *origin) { struct dentry *index; struct qstr name; int err; - err = ovl_get_index_name_fh(fh, &name); + err = ovl_get_index_name(ofs, origin, &name); if (err) return ERR_PTR(err); diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h index 75eb9d379549..53d1f0e0c924 100644 --- a/fs/overlayfs/overlayfs.h +++ b/fs/overlayfs/overlayfs.h @@ -715,7 +715,7 @@ int ovl_verify_index(struct ovl_fs *ofs, struct dentry *index); int ovl_get_index_name_fh(const struct ovl_fh *fh, struct qstr *name); int ovl_get_index_name(struct ovl_fs *ofs, struct dentry *origin, struct qstr *name); -struct dentry *ovl_get_index_fh(struct ovl_fs *ofs, struct ovl_fh *fh); +struct dentry *ovl_get_index_origin(struct ovl_fs *ofs, struct dentry *origin); struct dentry *ovl_lookup_index(struct ovl_fs *ofs, struct dentry *upper, struct dentry *origin, bool verify); int ovl_path_next(int idx, struct dentry *dentry, struct path *path, diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 0f45320caf72..bc38ba6aa37e 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -2943,7 +2943,7 @@ static long do_pagemap_cmd(struct file *file, unsigned int cmd, return do_pagemap_scan(mm, arg); default: - return -EINVAL; + return -ENOIOCTLCMD; } } diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h index f5ca2cd653e9..8ffc8b2ab6db 100644 --- a/fs/smb/client/cifsglob.h +++ b/fs/smb/client/cifsglob.h @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include @@ -647,6 +648,7 @@ struct TCP_Server_Info { spinlock_t srv_lock; /* protect anything here that is not protected */ __u64 conn_id; /* connection identifier (useful for debugging) */ int srv_count; /* reference counter */ + refcount_t obj_refs; /* struct lifetime: srv_count users + cifsd */ /* 15 character server name + 0x20 16th byte indicating type = srv */ char server_RFC1001_name[RFC1001_NAME_LEN_WITH_NULL]; struct smb_version_operations *ops; diff --git a/fs/smb/client/cifssmb.c b/fs/smb/client/cifssmb.c index 45c6eb11356b..c893b9fed8a0 100644 --- a/fs/smb/client/cifssmb.c +++ b/fs/smb/client/cifssmb.c @@ -2092,6 +2092,7 @@ CIFSSMBPosixLock(const unsigned int xid, struct cifs_tcon *tcon, cifs_dbg(FYI, "Send error in Posix Lock = %d\n", rc); } else if (pLockData) { /* lock structure can be returned on get */ + unsigned int rsp_len; __u16 data_offset; __u16 data_count; rc = validate_t2((struct smb_t2_rsp *)pSMBr); @@ -2102,10 +2103,16 @@ CIFSSMBPosixLock(const unsigned int xid, struct cifs_tcon *tcon, } data_offset = le16_to_cpu(pSMBr->t2.DataOffset); data_count = le16_to_cpu(pSMBr->t2.DataCount); + rsp_len = smbCalcSize(&pSMBr->hdr) - 4; if (data_count < sizeof(struct cifs_posix_lock)) { rc = -EIO; goto plk_err_exit; } + if (data_offset > rsp_len || + data_count > rsp_len - data_offset) { + rc = -EIO; + goto plk_err_exit; + } parm_data = (struct cifs_posix_lock *) ((char *)&pSMBr->hdr.Protocol + data_offset); if (parm_data->lock_type == cpu_to_le16(CIFS_UNLCK)) @@ -3998,8 +4005,9 @@ CIFSFindFirst(const unsigned int xid, struct cifs_tcon *tcon, /* level 257 SMB_ */ TRANSACTION2_FFIRST_REQ *pSMB = NULL; TRANSACTION2_FFIRST_RSP *pSMBr = NULL; - T2_FFIRST_RSP_PARMS *parms; + unsigned int data_off, data_count; struct nls_table *nls_codepage; + T2_FFIRST_RSP_PARMS *parms; unsigned int lnoff; __u16 params, byte_count; int bytes_returned = 0; @@ -4118,11 +4126,19 @@ CIFSFindFirst(const unsigned int xid, struct cifs_tcon *tcon, return rc; } + data_off = le16_to_cpu(pSMBr->t2.DataOffset); + data_count = le16_to_cpu(pSMBr->t2.DataCount); + if (data_off + data_count > (unsigned int)bytes_returned) { + cifs_dbg(VFS, "%s: data area offset %u count %u past response length %d\n", + __func__, data_off, data_count, bytes_returned); + cifs_buf_release(pSMB); + return -EINVAL; + } + psrch_inf->unicode = !!(pSMBr->hdr.Flags2 & SMBFLG2_UNICODE); psrch_inf->ntwrk_buf_start = (char *)pSMBr; psrch_inf->smallBuf = false; - psrch_inf->srch_entries_start = (char *)&pSMBr->hdr.Protocol + - le16_to_cpu(pSMBr->t2.DataOffset); + psrch_inf->srch_entries_start = (char *)&pSMBr->hdr.Protocol + data_off; parms = (T2_FFIRST_RSP_PARMS *)((char *)&pSMBr->hdr.Protocol + le16_to_cpu(pSMBr->t2.ParameterOffset)); @@ -4155,8 +4171,9 @@ int CIFSFindNext(const unsigned int xid, struct cifs_tcon *tcon, { TRANSACTION2_FNEXT_REQ *pSMB = NULL; TRANSACTION2_FNEXT_RSP *pSMBr = NULL; - T2_FNEXT_RSP_PARMS *parms; + unsigned int data_off, data_count; unsigned int name_len; + T2_FNEXT_RSP_PARMS *parms; unsigned int lnoff; __u16 params, byte_count; char *response_data; @@ -4239,13 +4256,22 @@ int CIFSFindNext(const unsigned int xid, struct cifs_tcon *tcon, cifs_buf_release(pSMB); return rc; } + + data_off = le16_to_cpu(pSMBr->t2.DataOffset); + data_count = le16_to_cpu(pSMBr->t2.DataCount); + if (data_off + data_count > (unsigned int)bytes_returned) { + cifs_dbg(VFS, "%s: data area offset %u count %u past response length %d\n", + __func__, data_off, data_count, bytes_returned); + cifs_buf_release(pSMB); + return -EINVAL; + } + /* BB fixme add lock for file (srch_info) struct here */ psrch_inf->unicode = !!(pSMBr->hdr.Flags2 & SMBFLG2_UNICODE); response_data = (char *)&pSMBr->hdr.Protocol + le16_to_cpu(pSMBr->t2.ParameterOffset); parms = (T2_FNEXT_RSP_PARMS *)response_data; - response_data = (char *)&pSMBr->hdr.Protocol + - le16_to_cpu(pSMBr->t2.DataOffset); + response_data = (char *)&pSMBr->hdr.Protocol + data_off; if (psrch_inf->smallBuf) cifs_small_buf_release(psrch_inf->ntwrk_buf_start); @@ -5763,9 +5789,11 @@ CIFSSMBQAllEAs(const unsigned int xid, struct cifs_tcon *tcon, int rc = 0; int bytes_returned; int list_len; + u32 ea_list_len; struct fealist *ea_response_data; struct fea *temp_fea; char *temp_ptr; + char *byte_area; char *end_of_smb; __u16 params, byte_count, data_offset; unsigned int ea_name_len = ea_name ? strlen(ea_name) : 0; @@ -5842,9 +5870,18 @@ CIFSSMBQAllEAs(const unsigned int xid, struct cifs_tcon *tcon, ea_response_data = (struct fealist *) (((char *) &pSMBr->hdr.Protocol) + data_offset); - list_len = le32_to_cpu(ea_response_data->list_len); - cifs_dbg(FYI, "ea length %d\n", list_len); - if (list_len <= 8) { + byte_area = pByteArea(&pSMBr->hdr); + end_of_smb = byte_area + get_bcc(&pSMBr->hdr); + if ((char *)ea_response_data < byte_area || + (char *)ea_response_data > + end_of_smb - sizeof(ea_response_data->list_len)) { + rc = -EIO; + goto QAllEAsOut; + } + + ea_list_len = le32_to_cpu(ea_response_data->list_len); + cifs_dbg(FYI, "%s: ea length: %u\n", __func__, ea_list_len); + if (ea_list_len <= 8) { cifs_dbg(FYI, "empty EA list returned from server\n"); /* didn't find the named attribute */ if (ea_name) @@ -5853,12 +5890,12 @@ CIFSSMBQAllEAs(const unsigned int xid, struct cifs_tcon *tcon, } /* make sure list_len doesn't go past end of SMB */ - end_of_smb = (char *)pByteArea(&pSMBr->hdr) + get_bcc(&pSMBr->hdr); - if ((char *)ea_response_data + list_len > end_of_smb) { + if (ea_list_len > end_of_smb - (char *)ea_response_data) { cifs_dbg(FYI, "EA list appears to go beyond SMB\n"); rc = -EIO; goto QAllEAsOut; } + list_len = ea_list_len; /* account for ea list len */ list_len -= 4; diff --git a/fs/smb/client/connect.c b/fs/smb/client/connect.c index eace4d1a3dce..82a70370a60f 100644 --- a/fs/smb/client/connect.c +++ b/fs/smb/client/connect.c @@ -977,6 +977,19 @@ cifs_enable_signing(struct TCP_Server_Info *server, bool mnt_sign_required) return 0; } +static void cifs_tcp_server_put(struct TCP_Server_Info *server) +{ + if (!refcount_dec_and_test(&server->obj_refs)) + return; + + cifs_crypto_secmech_release(server); + kfree_sensitive(server->session_key.response); + put_net(cifs_net_ns(server)); + kfree(server->leaf_fullpath); + kfree(server->hostname); + kfree_sensitive(server); +} + static noinline_for_stack void clean_demultiplex_info(struct TCP_Server_Info *server) { @@ -1062,14 +1075,11 @@ clean_demultiplex_info(struct TCP_Server_Info *server) */ } - put_net(cifs_net_ns(server)); - kfree(server->leaf_fullpath); - kfree(server->hostname); - kfree(server); - length = atomic_dec_return(&tcpSesAllocCount); if (length > 0) mempool_resize(cifs_req_poolp, length + cifs_min_rcv); + + cifs_tcp_server_put(server); } static int @@ -1680,7 +1690,7 @@ cifs_put_tcp_session(struct TCP_Server_Info *server, int from_reconnect) else cancel_delayed_work_sync(&server->reconnect); - /* For secondary channels, we pick up ref-count on the primary server */ + /* For secondary channels, drop the reference on the primary server */ if (SERVER_IS_CHAN(server)) cifs_put_tcp_session(server->primary_server, from_reconnect); @@ -1688,15 +1698,11 @@ cifs_put_tcp_session(struct TCP_Server_Info *server, int from_reconnect) server->tcpStatus = CifsExiting; spin_unlock(&server->srv_lock); - cifs_crypto_secmech_release(server); - - kfree_sensitive(server->session_key.response); - server->session_key.response = NULL; - server->session_key.len = 0; - task = xchg(&server->tsk, NULL); if (task) send_sig(SIGKILL, task, 1); + + cifs_tcp_server_put(server); } struct TCP_Server_Info * @@ -1718,6 +1724,7 @@ cifs_get_tcp_session(struct smb3_fs_context *ctx, rc = -ENOMEM; goto out_err; } + refcount_set(&tcp_ses->obj_refs, 1); tcp_ses->hostname = kstrdup(ctx->server_hostname, GFP_KERNEL); if (!tcp_ses->hostname) { @@ -1858,6 +1865,8 @@ cifs_get_tcp_session(struct smb3_fs_context *ctx, module_put(THIS_MODULE); goto out_err_crypto_release; } + /* Keep the server alive until the demultiplex thread has exited */ + refcount_inc(&tcp_ses->obj_refs); /* thread created, put it on the list */ spin_lock(&cifs_tcp_ses_lock); list_add(&tcp_ses->tcp_ses_list, &cifs_tcp_ses_list); diff --git a/fs/smb/client/inode.c b/fs/smb/client/inode.c index 9307a2ad9e44..5db59b429fc3 100644 --- a/fs/smb/client/inode.c +++ b/fs/smb/client/inode.c @@ -87,7 +87,9 @@ static void cifs_set_ops(struct inode *inode) else inode->i_data.a_ops = &cifs_addr_ops; mapping_set_large_folios(inode->i_mapping); - if (cifs_sb_master_tcon(cifs_sb)->ses->server->sign) + if (cifs_sb_master_tcon(cifs_sb)->ses->server->sign || + (cifs_sb_master_tcon(cifs_sb)->ses->server->compression.enabled && + (cifs_sb_master_tcon(cifs_sb)->share_flags & SMB2_SHAREFLAG_COMPRESS_DATA))) mapping_set_stable_writes(inode->i_mapping); break; case S_IFDIR: diff --git a/fs/smb/client/readdir.c b/fs/smb/client/readdir.c index 9412c69e5aea..55b4171cec09 100644 --- a/fs/smb/client/readdir.c +++ b/fs/smb/client/readdir.c @@ -454,17 +454,20 @@ initiate_cifs_search(const unsigned int xid, struct file *file, } /* return length of unicode string in bytes */ -static int cifs_unicode_bytelen(const char *str) +static size_t cifs_unicode_bytelen(const char *str, size_t maxlen) { - int len; const __le16 *ustr = (const __le16 *)str; + size_t len; - for (len = 0; len <= PATH_MAX; len++) { + maxlen = umin(PATH_MAX + 1, maxlen / sizeof(__le16)); + for (len = 0; len < maxlen; len++) { if (ustr[len] == 0) - return len << 1; + return len * sizeof(__le16); } - cifs_dbg(FYI, "Unicode string longer than PATH_MAX found\n"); - return len << 1; + len *= sizeof(__le16); + cifs_dbg(FYI, "%s: Unicode string not terminated within %zu bytes\n", + __func__, len); + return len; } static char *nxt_dir_entry(char *old_entry, char *end_of_smb, int level) @@ -530,13 +533,13 @@ static void cifs_fill_dirent_posix(struct cifs_dirent *de, } static void cifs_fill_dirent_unix(struct cifs_dirent *de, - const FILE_UNIX_INFO *info, bool is_unicode) + const FILE_UNIX_INFO *info, bool is_unicode, size_t maxlen) { de->name = &info->FileName[0]; if (is_unicode) - de->namelen = cifs_unicode_bytelen(de->name); + de->namelen = cifs_unicode_bytelen(de->name, maxlen); else - de->namelen = strnlen(de->name, PATH_MAX); + de->namelen = strnlen(de->name, min_t(size_t, maxlen, PATH_MAX)); de->resume_key = info->ResumeKey; de->ino = le64_to_cpu(info->basic.UniqueId); } @@ -584,30 +587,49 @@ static void cifs_fill_dirent_std(struct cifs_dirent *de, } static int cifs_fill_dirent(struct cifs_dirent *de, const void *info, - u16 level, bool is_unicode) + const char *end, u16 level, bool is_unicode) { + size_t len = end > (const char *)info ? end - (const char *)info : 0; + size_t nul_len = 0; + memset(de, 0, sizeof(*de)); switch (level) { case SMB_FIND_FILE_POSIX_INFO: + if (len < sizeof(struct smb2_posix_info)) + goto too_short; cifs_fill_dirent_posix(de, info); break; case SMB_FIND_FILE_UNIX: - cifs_fill_dirent_unix(de, info, is_unicode); + if (len < offsetof(FILE_UNIX_INFO, FileName)) + goto too_short; + cifs_fill_dirent_unix(de, info, is_unicode, + len - offsetof(FILE_UNIX_INFO, FileName)); + nul_len = is_unicode ? sizeof(__le16) : sizeof(__u8); break; case SMB_FIND_FILE_DIRECTORY_INFO: + if (len < offsetof(FILE_DIRECTORY_INFO, FileName)) + goto too_short; cifs_fill_dirent_dir(de, info); break; case SMB_FIND_FILE_FULL_DIRECTORY_INFO: + if (len < offsetof(FILE_FULL_DIRECTORY_INFO, FileName)) + goto too_short; cifs_fill_dirent_full(de, info); break; case SMB_FIND_FILE_ID_FULL_DIR_INFO: + if (len < offsetof(SEARCH_ID_FULL_DIR_INFO, FileName)) + goto too_short; cifs_fill_dirent_search(de, info); break; case SMB_FIND_FILE_BOTH_DIRECTORY_INFO: + if (len < offsetof(FILE_BOTH_DIRECTORY_INFO, FileName)) + goto too_short; cifs_fill_dirent_both(de, info); break; case SMB_FIND_FILE_INFO_STANDARD: + if (len < offsetof(FIND_FILE_STANDARD_INFO, FileName)) + goto too_short; cifs_fill_dirent_std(de, info); break; default: @@ -615,7 +637,20 @@ static int cifs_fill_dirent(struct cifs_dirent *de, const void *info, return -EINVAL; } + if (de->name && + (end < de->name || + (size_t)(end - de->name) < de->namelen + nul_len)) { + cifs_dbg(VFS, "%s: search entry name extends past end of SMB\n", + __func__); + return -EINVAL; + } + return 0; + +too_short: + cifs_dbg(VFS, "%s: search entry extends past end of SMB\n", + __func__); + return -EINVAL; } #define UNICODE_DOT cpu_to_le16(0x2e) @@ -670,10 +705,14 @@ static int is_dir_changed(struct file *file) static int cifs_save_resume_key(const char *current_entry, struct cifsFileInfo *file_info) { + struct TCP_Server_Info *server = tlink_tcon(file_info->tlink)->ses->server; + char *buf = file_info->srch_inf.ntwrk_buf_start; + const char *end = buf + server->ops->calc_smb_size(buf); struct cifs_dirent de; int rc; - rc = cifs_fill_dirent(&de, current_entry, file_info->srch_inf.info_level, + rc = cifs_fill_dirent(&de, current_entry, end, + file_info->srch_inf.info_level, file_info->srch_inf.unicode); if (!rc) { file_info->srch_inf.presume_name = de.name; @@ -958,7 +997,8 @@ static int cifs_filldir(char *find_entry, struct file *file, struct qstr name; int rc = 0; - rc = cifs_fill_dirent(&de, find_entry, file_info->srch_inf.info_level, + rc = cifs_fill_dirent(&de, find_entry, end_of_smb, + file_info->srch_inf.info_level, file_info->srch_inf.unicode); if (rc) return rc; diff --git a/fs/smb/client/smb2misc.c b/fs/smb/client/smb2misc.c index 23320b794b17..7be338ad755b 100644 --- a/fs/smb/client/smb2misc.c +++ b/fs/smb/client/smb2misc.c @@ -193,6 +193,21 @@ smb2_check_message(char *buf, unsigned int len, struct TCP_Server_Info *server) return 1; } + /* + * A SESSION_SETUP response that asks for more processing carries a + * security blob and is parsed as a normal reply. Do not let the + * short-error path below accept a frame that cannot hold the fixed + * response fields. + */ + if (command == SMB2_SESSION_SETUP_HE && + shdr->Status == STATUS_MORE_PROCESSING_REQUIRED && + len < sizeof(struct smb2_sess_setup_rsp)) { + cifs_server_dbg(VFS, + "%s: session setup response too short: %u < %zu\n", + __func__, len, sizeof(struct smb2_sess_setup_rsp)); + return 1; + } + if (len < pdu_size) { if ((len >= hdr_size) && (shdr->Status != 0)) { diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c index cb164f87f908..8e72506a2409 100644 --- a/fs/smb/client/smb2pdu.c +++ b/fs/smb/client/smb2pdu.c @@ -1773,15 +1773,21 @@ SMB2_sess_auth_rawntlmssp_negotiate(struct SMB2_sess_data *sess_data) if (rc) goto out; - if (offsetof(struct smb2_sess_setup_rsp, Buffer) != - le16_to_cpu(rsp->SecurityBufferOffset)) { - cifs_dbg(VFS, "Invalid security buffer offset %d\n", - le16_to_cpu(rsp->SecurityBufferOffset)); + u16 boff = le16_to_cpu(rsp->SecurityBufferOffset); + u16 blen = le16_to_cpu(rsp->SecurityBufferLength); + + if (offsetof(struct smb2_sess_setup_rsp, Buffer) != boff) { + cifs_dbg(VFS, "Invalid security buffer offset %d\n", boff); rc = -EIO; goto out; } - rc = decode_ntlmssp_challenge(rsp->Buffer, - le16_to_cpu(rsp->SecurityBufferLength), ses); + if ((u32)boff + blen > sess_data->iov[0].iov_len) { + cifs_dbg(VFS, "%s: invalid security buffer length: %u\n", + __func__, blen); + rc = -EINVAL; + goto out; + } + rc = decode_ntlmssp_challenge(rsp->Buffer, blen, ses); if (rc) goto out; @@ -4792,6 +4798,7 @@ SMB2_read(const unsigned int xid, struct cifs_io_parms *io_parms, struct kvec iov[1]; struct kvec rsp_iov; unsigned int total_len; + u16 data_offset; int flags = CIFS_LOG_ERROR; struct cifs_ses *ses = io_parms->tcon->ses; @@ -4850,8 +4857,28 @@ SMB2_read(const unsigned int xid, struct cifs_io_parms *io_parms, *nbytes = 0; } + data_offset = rsp->DataOffset; + if (data_offset < io_parms->server->vals->read_rsp_size) { + /* + * win2k8 sometimes sends an offset of 0 when the read is beyond + * the EOF. Treat it as if the data starts just after the header, + * as handle_read_data() does. + */ + cifs_dbg(FYI, "data offset %u inside read response header\n", + data_offset); + data_offset = io_parms->server->vals->read_rsp_size; + } + + /* The data area must lie within the response actually received */ + if (*nbytes && (size_t)data_offset + *nbytes > rsp_iov.iov_len) { + cifs_dbg(FYI, "bad data offset %u length %u for response of %zu\n", + data_offset, *nbytes, rsp_iov.iov_len); + rc = -EIO; + *nbytes = 0; + } + if (*buf) { - memcpy(*buf, (char *)rsp + rsp->DataOffset, *nbytes); + memcpy(*buf, (char *)rsp + data_offset, *nbytes); free_rsp_buf(resp_buftype, rsp_iov.iov_base); } else if (resp_buftype != CIFS_NO_BUFFER) { *buf = rsp_iov.iov_base; diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c index 4e0f43051d9e..f7e3d7986c4c 100644 --- a/fs/smb/server/smb2pdu.c +++ b/fs/smb/server/smb2pdu.c @@ -3180,6 +3180,14 @@ int smb2_open(struct ksmbd_work *work) req->CreateOptions = ~(FILE_NO_COMPRESSION_LE); } } + + /* MS-FSA 2.1.5.1: delete-on-close requires DELETE access */ + if (req->CreateOptions & FILE_DELETE_ON_CLOSE_LE && + !(smb_map_generic_desired_access(req->DesiredAccess) & + FILE_DELETE_LE)) { + rc = -EINVAL; + goto err_out2; + } } if (le32_to_cpu(req->CreateDisposition) > @@ -3359,6 +3367,13 @@ int smb2_open(struct ksmbd_work *work) maximal_access = daccess; } + /* MAXIMUM_ALLOWED may not have granted the requested DELETE */ + if (req->CreateOptions & FILE_DELETE_ON_CLOSE_LE && + !(daccess & FILE_DELETE_LE)) { + rc = -EACCES; + goto err_out; + } + open_flags = smb2_create_open_flags(file_present, daccess, req->CreateDisposition, &may_flags, @@ -5275,17 +5290,8 @@ static int get_file_stream_info(struct ksmbd_work *work, streamlen *= 2; kfree(stream_buf); file_info->StreamNameLength = cpu_to_le32(streamlen); - /* - * stream_name_len is the byte length of the xattr's *name*, - * not its value -- same class of bug ksmbd_stream_eof() - * (smb2pdu.c) already fixes for EndOfFile/AllocationSize on - * a stream handle; this enumeration path needs the same - * real xattr value length, not the name length reused as a - * size. - */ - slen = ksmbd_vfs_casexattr_len(file_mnt_idmap(fp->filp), - path->dentry, stream_name, - strlen(stream_name) + 1, NULL); + slen = ksmbd_vfs_xattr_len(file_mnt_idmap(fp->filp), + path->dentry, stream_name); ssize = slen < 0 ? 0 : (loff_t)slen; file_info->StreamSize = cpu_to_le64(ssize); file_info->StreamAllocationSize = cpu_to_le64(ssize); diff --git a/fs/smb/server/smbacl.c b/fs/smb/server/smbacl.c index 33a607e709f1..63f7bccb4d9a 100644 --- a/fs/smb/server/smbacl.c +++ b/fs/smb/server/smbacl.c @@ -1606,7 +1606,10 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path, else continue; - if (id == uid) { + /* A group entry grants only to the group's members */ + if (id == uid && + (pa_entry->e_tag == ACL_USER || + in_group_p(make_kgid(&init_user_ns, id)))) { mode_to_access_flags(pa_entry->e_perm, 0777, &access_bits); diff --git a/fs/smb/server/vfs.c b/fs/smb/server/vfs.c index dafb92aa39a6..9aee0214e8fd 100644 --- a/fs/smb/server/vfs.c +++ b/fs/smb/server/vfs.c @@ -943,8 +943,8 @@ ssize_t ksmbd_vfs_listxattr(struct dentry *dentry, char **list) return size; } -static ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap, - struct dentry *dentry, char *xattr_name) +ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap, + struct dentry *dentry, char *xattr_name) { return vfs_getxattr(idmap, dentry, xattr_name, NULL, 0); } diff --git a/fs/smb/server/vfs.h b/fs/smb/server/vfs.h index 397dd2dd0988..1ca170dfb349 100644 --- a/fs/smb/server/vfs.h +++ b/fs/smb/server/vfs.h @@ -106,6 +106,8 @@ ssize_t ksmbd_vfs_getxattr(struct mnt_idmap *idmap, struct dentry *dentry, char *xattr_name, char **xattr_buf); +ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap, + struct dentry *dentry, char *xattr_name); ssize_t ksmbd_vfs_casexattr_len(struct mnt_idmap *idmap, struct dentry *dentry, char *attr_name, int attr_name_len, char *actual_name); diff --git a/fs/udf/balloc.c b/fs/udf/balloc.c index 30cec5600149..75124188e745 100644 --- a/fs/udf/balloc.c +++ b/fs/udf/balloc.c @@ -457,6 +457,7 @@ static void udf_table_free_blocks(struct super_block *sb, int adsize; + eloc.partitionReferenceNum = bloc->partitionReferenceNum; eloc.logicalBlockNum = start; elen = EXT_RECORDED_ALLOCATED | (count << sb->s_blocksize_bits); @@ -532,7 +533,7 @@ static int udf_table_prealloc_blocks(struct super_block *sb, eloc.logicalBlockNum, elen, first_block); } - if (first_block == eloc.logicalBlockNum) { + if (ret > 0 && first_block == eloc.logicalBlockNum) { epos.offset -= adsize; alloc_count = (elen >> sb->s_blocksize_bits); @@ -572,7 +573,7 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb, uint32_t elen, goal_elen = 0; struct kernel_lb_addr eloc, goal_eloc; struct extent_position epos, goal_epos; - int8_t etype; + int8_t etype, goal_etype = 0; struct udf_inode_info *iinfo = UDF_I(table); int ret = 0; @@ -623,7 +624,8 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb, goal_epos.block = epos.block; goal_epos.offset = epos.offset - adsize; goal_eloc = eloc; - goal_elen = (etype << 30) | elen; + goal_elen = elen; + goal_etype = etype; } } @@ -647,7 +649,8 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb, goal_elen -= sb->s_blocksize; if (goal_elen) - udf_write_aext(table, &goal_epos, &goal_eloc, goal_elen, 1); + udf_write_aext(table, &goal_epos, &goal_eloc, + (goal_etype << 30) | goal_elen, 1); else udf_delete_aext(table, goal_epos, &freed); brelse(goal_epos.bh); diff --git a/fs/udf/inode.c b/fs/udf/inode.c index 96934d42d949..49d8a2b19641 100644 --- a/fs/udf/inode.c +++ b/fs/udf/inode.c @@ -1339,6 +1339,26 @@ int udf_setsize(struct inode *inode, loff_t newsize) return err; } +/* + * Verify validity of struct deviceSpec on disk. udf_get_extendedattr() has + * already verified the generic header and made sure attribute fits in the + * inode so we just have to make sure attribute space is large enough for + * deviceSpec struct and required impUse information. + */ +static bool udf_device_spec_valid(struct deviceSpec *dsea) +{ + u32 attr_length, imp_use_length; + + attr_length = le32_to_cpu(dsea->attrLength); + imp_use_length = le32_to_cpu(dsea->impUseLength); + if (attr_length < sizeof(struct deviceSpec) || + imp_use_length < sizeof(struct regid) || + imp_use_length > attr_length - sizeof(struct deviceSpec)) + return false; + + return true; +} + /* * Maximum length of linked list formed by ICB hierarchy. The chosen number is * arbitrary - just that we hopefully don't limit any real use of rewritten @@ -1657,13 +1677,12 @@ static int udf_read_inode(struct inode *inode, bool hidden_inode) if (S_ISCHR(inode->i_mode) || S_ISBLK(inode->i_mode)) { struct deviceSpec *dsea = (struct deviceSpec *)udf_get_extendedattr(inode, 12, 1); - if (dsea) { - init_special_inode(inode, inode->i_mode, + + if (!dsea || !udf_device_spec_valid(dsea)) + goto out; + init_special_inode(inode, inode->i_mode, MKDEV(le32_to_cpu(dsea->majorDeviceIdent), le32_to_cpu(dsea->minorDeviceIdent))); - /* Developer ID ??? */ - } else - goto out; } ret = 0; out: @@ -1811,6 +1830,10 @@ static int udf_update_inode(struct inode *inode, int do_sync) udf_add_extendedattr(inode, sizeof(struct deviceSpec) + sizeof(struct regid), 12, 0x3); + if (!dsea) { + err = -ENOSPC; + goto out_unlock; + } dsea->attrType = cpu_to_le32(12); dsea->attrSubtype = 1; dsea->attrLength = cpu_to_le32( @@ -1818,6 +1841,10 @@ static int udf_update_inode(struct inode *inode, int do_sync) sizeof(struct regid)); dsea->impUseLength = cpu_to_le32(sizeof(struct regid)); } + if (!udf_device_spec_valid(dsea)) { + err = -EFSCORRUPTED; + goto out_unlock; + } eid = (struct regid *)dsea->impUse; memset(eid, 0, sizeof(*eid)); strcpy(eid->ident, UDF_ID_DEVELOPER); @@ -1959,6 +1986,11 @@ static int udf_update_inode(struct inode *inode, int do_sync) brelse(bh); return err; + +out_unlock: + unlock_buffer(bh); + brelse(bh); + return err; } struct inode *__udf_iget(struct super_block *sb, struct kernel_lb_addr *ino, diff --git a/fs/udf/super.c b/fs/udf/super.c index 556540f2797a..8d3d758714da 100644 --- a/fs/udf/super.c +++ b/fs/udf/super.c @@ -1105,6 +1105,73 @@ static int check_partition_desc(struct super_block *sb, return 0; } +/* + * Check the Unallocated Space Table once when it is loaded: the allocator + * hands out blocks from the start of each extent and trusts that every + * extent covers at least one whole block inside the partition. + */ +static int udf_check_unalloc_table(struct super_block *sb, + struct inode *table, u32 partition_len) +{ + struct extent_position epos = { + .block = UDF_I(table)->i_location, + .offset = sizeof(struct unallocSpaceEntry), + }; + struct kernel_lb_addr eloc; + uint32_t elen, blocks; + struct kernel_lb_addr seen_block = {}; + uint32_t seen_offset = 0; + u64 steps = 0, period = 1; + int8_t etype; + int ret; + + /* The walk below assumes the layout of an Unallocated Space Entry */ + if (!UDF_I(table)->i_use) { + udf_err(sb, "unallocated space table is not an unallocated space entry\n"); + return -EFSCORRUPTED; + } + + while ((ret = udf_next_aext(table, &epos, &eloc, &elen, &etype, 1)) > 0) { + blocks = elen >> sb->s_blocksize_bits; + if (!blocks || (elen & (sb->s_blocksize - 1)) || + eloc.logicalBlockNum >= partition_len || + blocks > partition_len - eloc.logicalBlockNum) { + udf_err(sb, "invalid unallocated space table extent (block %u, length %u)\n", + eloc.logicalBlockNum, elen); + ret = -EFSCORRUPTED; + break; + } + /* + * A chain of allocation extents can lead back to itself, and + * then the walk returns the same extents forever. Remember a + * position at doubling intervals; a walk that comes back to it + * is in a loop. + */ + if (epos.block.logicalBlockNum == seen_block.logicalBlockNum && + epos.block.partitionReferenceNum == + seen_block.partitionReferenceNum && + epos.offset == seen_offset) { + udf_err(sb, "unallocated space table loops back on itself\n"); + ret = -EFSCORRUPTED; + break; + } + if (++steps == period) { + seen_block = epos.block; + seen_offset = epos.offset; + steps = 0; + period <<= 1; + } + if (fatal_signal_pending(current)) { + udf_err(sb, "interrupted while checking the unallocated space table\n"); + ret = -EINTR; + break; + } + cond_resched(); + } + brelse(epos.bh); + return ret; +} + static int udf_fill_partdesc_info(struct super_block *sb, struct partitionDesc *p, int p_index) { @@ -1165,6 +1232,16 @@ static int udf_fill_partdesc_info(struct super_block *sb, p_index); return PTR_ERR(inode); } + err = udf_check_unalloc_table(sb, inode, map->s_partition_len); + if (err) { + iput(inode); + if (err == -EINTR) + return err; + if (!sb_rdonly(sb)) + return -EACCES; + UDF_SET_FLAG(sb, UDF_FLAG_RW_INCOMPAT); + return 0; + } map->s_uspace.s_table = inode; map->s_partition_flags |= UDF_PART_FLAG_UNALLOC_TABLE; udf_debug("unallocSpaceTable (part %d) @ %lu\n", @@ -2228,8 +2305,9 @@ static int udf_fill_super(struct super_block *sb, struct fs_context *fc) /* * EACCES is special - we want to propagate to * upper layers that we cannot handle RW mount. + * EINTR means that a fatal signal is pending. */ - if (ret == -EACCES) + if (ret == -EACCES || ret == -EINTR) break; } else break; diff --git a/fs/udf/truncate.c b/fs/udf/truncate.c index da5a61306390..85bbd6fa2a30 100644 --- a/fs/udf/truncate.c +++ b/fs/udf/truncate.c @@ -124,8 +124,8 @@ void udf_discard_prealloc(struct inode *inode) { struct extent_position epos = {}; struct extent_position prev_epos = {}; - struct kernel_lb_addr eloc; - uint32_t elen; + struct kernel_lb_addr eloc, tmpeloc; + uint32_t elen, tmpelen; uint64_t lbcount = 0; int8_t etype = -1; struct udf_inode_info *iinfo = UDF_I(inode); @@ -141,7 +141,8 @@ void udf_discard_prealloc(struct inode *inode) /* Find the last extent in the file */ while (1) { - ret = udf_next_aext(inode, &epos, &eloc, &elen, &tmpetype, 0); + ret = udf_next_aext(inode, &epos, &tmpeloc, &tmpelen, + &tmpetype, 0); if (ret < 0) goto out; if (ret == 0) diff --git a/fs/verity/enable.c b/fs/verity/enable.c index c284f46d1b53..197564ecd3ef 100644 --- a/fs/verity/enable.c +++ b/fs/verity/enable.c @@ -292,6 +292,14 @@ static int enable_verity(struct file *filp, * after the filesystem has successfully enabled verity. */ fsverity_set_info(inode, vi); + + /* + * Now that verity is enabled, report the signature (if any) to + * LSMs. We tolerate that a process racing with this ioctl can + * briefly see verity enabled with no signature reported yet. + */ + fsverity_report_valid_sig(inode, desc->signature, + le32_to_cpu(desc->sig_size)); } out: kfree(params.hashstate); diff --git a/fs/verity/fsverity_private.h b/fs/verity/fsverity_private.h index b3506f56e180..760f2674c816 100644 --- a/fs/verity/fsverity_private.h +++ b/fs/verity/fsverity_private.h @@ -135,6 +135,8 @@ void __init fsverity_init_info_cache(void); extern int fsverity_require_signatures; int fsverity_verify_signature(const struct fsverity_info *vi, const u8 *signature, size_t sig_size); +void fsverity_report_valid_sig(const struct inode *inode, const u8 *signature, + size_t sig_size); void __init fsverity_init_signature(void); #else /* !CONFIG_FS_VERITY_BUILTIN_SIGNATURES */ @@ -145,6 +147,12 @@ fsverity_verify_signature(const struct fsverity_info *vi, return 0; } +static inline void fsverity_report_valid_sig(const struct inode *inode, + const u8 *signature, + size_t sig_size) +{ +} + static inline void fsverity_init_signature(void) { } diff --git a/fs/verity/measure.c b/fs/verity/measure.c index 91ce39ced77b..b5af400c725e 100644 --- a/fs/verity/measure.c +++ b/fs/verity/measure.c @@ -30,6 +30,16 @@ int fsverity_ioctl_measure(struct file *filp, void __user *_uarg) const struct fsverity_hash_alg *hash_alg; struct fsverity_digest arg; + /* + * This ioctl returns a hash of the file contents, so it should require + * read permission. Disallow it on ioctl-only (i.e., nonstandard access + * mode 3) fds, which can in some cases be acquired without passing all + * LSM read-permission checks. Also disallow write-only fds, which can + * refer only to non-verity files which would fail with ENODATA anyway. + */ + if (!(filp->f_mode & FMODE_READ)) + return -EBADF; + vi = fsverity_get_info(inode); if (!vi) return -ENODATA; /* not a verity file */ diff --git a/fs/verity/open.c b/fs/verity/open.c index 9c1b37a3b96a..dba70c0ab5b6 100644 --- a/fs/verity/open.c +++ b/fs/verity/open.c @@ -377,6 +377,15 @@ static int ensure_verity_info(struct inode *inode) goto out_free_desc; } + /* + * Now that the signature (if any) has been verified, report it to LSMs. + * Do this before setting the fsverity_info, so that anyone who + * concurrently retrieves the fsverity_info for the same inode is + * guaranteed to see the LSM's signature status populated. + */ + fsverity_report_valid_sig(inode, desc->signature, + le32_to_cpu(desc->sig_size)); + fsverity_set_info(inode, vi); err = 0; out_free_desc: diff --git a/fs/verity/read_metadata.c b/fs/verity/read_metadata.c index f58432772d9e..680792701ba2 100644 --- a/fs/verity/read_metadata.c +++ b/fs/verity/read_metadata.c @@ -156,13 +156,19 @@ int fsverity_ioctl_read_metadata(struct file *filp, const void __user *uarg) int length; void __user *buf; + /* + * This ioctl returns file contents block hashes, so it should require + * read permission. Disallow it on ioctl-only (i.e., nonstandard access + * mode 3) fds, which can in some cases be acquired without passing all + * LSM read-permission checks. Also disallow write-only fds, which can + * refer only to non-verity files which would fail with ENODATA anyway. + */ + if (!(filp->f_mode & FMODE_READ)) + return -EBADF; + vi = fsverity_get_info(inode); if (!vi) return -ENODATA; /* not a verity file */ - /* - * Note that we don't have to explicitly check that the file is open for - * reading, since verity files can only be opened for reading. - */ if (copy_from_user(&arg, uarg, sizeof(arg))) return -EFAULT; diff --git a/fs/verity/signature.c b/fs/verity/signature.c index 0302a4e506ec..212e873c8499 100644 --- a/fs/verity/signature.c +++ b/fs/verity/signature.c @@ -42,13 +42,16 @@ static struct key *fsverity_keyring; * @sig_size: size of signature in bytes, or 0 if no signature * * If the file includes a signature of its fs-verity file digest, verify it - * against the certificates in the fs-verity keyring. Note that signatures - * are verified regardless of the state of the 'fsverity_require_signatures' - * variable and the LSM subsystem relies on this behavior to help enforce - * file integrity policies. Please discuss changes with the LSM list - * (thank you!). + * against the certificates in the fs-verity keyring. * - * Return: 0 on success (signature valid or not required); -errno on failure + * Note that signatures are verified regardless of the state of the + * 'fsverity_require_signatures' variable, and the LSM subsystem relies on this + * behavior to help enforce file integrity policies. (See + * fsverity_report_valid_sig(), which assumes that the signature, if present, + * was verified here.) Please discuss changes with the LSM list (thank you!). + * + * Return: 0 if the signature is present and valid, or absent and not required; + * -errno on failure */ int fsverity_verify_signature(const struct fsverity_info *vi, const u8 *signature, size_t sig_size) @@ -111,18 +114,40 @@ int fsverity_verify_signature(const struct fsverity_info *vi, return err; } - err = security_inode_setintegrity(inode, - LSM_INT_FSVERITY_BUILTINSIG_VALID, - signature, - sig_size); + return 0; +} - if (err) { +/* + * If the signature exists, this tells the LSM that it was verified. + * + * fsverity_verify_signature() must have already been called with the same + * signature, *and* the inode must have verity enabled with the signature + * matching it. If called from FS_IOC_ENABLE_VERITY, the latter means that we + * must be past the point of no return, i.e. verity won't fail to be enabled at + * a later point. This is critical, since LSMs must be told about the signature + * only when the verity status is permanent in the in-core inode. + */ +void fsverity_report_valid_sig(const struct inode *inode, const u8 *signature, + size_t sig_size) +{ + int err; + + if (WARN_ON_ONCE(!IS_VERITY(inode))) + return; + + if (sig_size == 0) + return; + + err = security_inode_setintegrity( + inode, LSM_INT_FSVERITY_BUILTINSIG_VALID, signature, sig_size); + if (err) fsverity_err(inode, "Error %d exposing file signature to LSMs", err); - return err; - } - - return 0; + /* + * Besides the logging above, ignore errors here. The LSM hook just + * records the signature and doesn't perform access control. And in the + * case of FS_IOC_ENABLE_VERITY verity was already enabled. + */ } void __init fsverity_init_signature(void) diff --git a/fs/xfs/libxfs/xfs_attr_leaf.c b/fs/xfs/libxfs/xfs_attr_leaf.c index aef8d66a016f..b3f1eaa1ec45 100644 --- a/fs/xfs/libxfs/xfs_attr_leaf.c +++ b/fs/xfs/libxfs/xfs_attr_leaf.c @@ -2961,7 +2961,14 @@ xfs_attr3_leaf_setflag( #endif entry = &xfs_attr3_leaf_entryp(leaf)[args->index]; - ASSERT((entry->flags & XFS_ATTR_INCOMPLETE) == 0); + if (entry->flags & XFS_ATTR_INCOMPLETE) { + if (!(args->op_flags & XFS_DA_OP_RECOVERY)) { + ASSERT(args->op_flags & XFS_DA_OP_RECOVERY); + return -EFSCORRUPTED; + } + return 0; + } + entry->flags |= XFS_ATTR_INCOMPLETE; xfs_trans_log_buf(args->trans, bp, XFS_DA_LOGRANGE(leaf, entry, sizeof(*entry))); diff --git a/fs/xfs/libxfs/xfs_btree.c b/fs/xfs/libxfs/xfs_btree.c index f0ddaa136a0b..de09db78df1a 100644 --- a/fs/xfs/libxfs/xfs_btree.c +++ b/fs/xfs/libxfs/xfs_btree.c @@ -2568,7 +2568,7 @@ xfs_btree_lshift( if (XFS_IS_CORRUPT(tcur->bc_mp, i != 1)) { xfs_btree_mark_sick(cur); error = -EFSCORRUPTED; - goto error0; + goto error1; } error = xfs_btree_decrement(tcur, level, &i); @@ -2738,7 +2738,7 @@ xfs_btree_rshift( if (XFS_IS_CORRUPT(tcur->bc_mp, i != 1)) { xfs_btree_mark_sick(cur); error = -EFSCORRUPTED; - goto error0; + goto error1; } error = xfs_btree_increment(tcur, level, &i); diff --git a/fs/xfs/scrub/attr.c b/fs/xfs/scrub/attr.c index 7315d0cb1e2e..68d9666ab201 100644 --- a/fs/xfs/scrub/attr.c +++ b/fs/xfs/scrub/attr.c @@ -293,6 +293,78 @@ xchk_xattr_set_map( return ret; } +static unsigned int +xchk_xattr_local_entry( + struct xchk_da_btree *ds, + int level, + char *buf_end, + struct xfs_attr_leafblock *leaf, + int idx) +{ + struct xfs_attr_leaf_name_local *lentry = + xfs_attr3_leaf_name_local(leaf, idx); + char *name_end; + unsigned int namesize; + + name_end = (char *)lentry + offsetof(typeof(*lentry), nameval[0]); + if (name_end > buf_end) { + xchk_da_set_corrupt(ds, level); + return 0; + } + + namesize = xfs_attr_leaf_entsize_local(lentry->namelen, + be16_to_cpu(lentry->valuelen)); + name_end = (char *)lentry + namesize; + if (lentry->namelen == 0) { + xchk_da_set_corrupt(ds, level); + return 0; + } + if (name_end > buf_end) { + xchk_da_set_corrupt(ds, level); + return 0; + } + + return namesize; +} + +static unsigned int +xchk_xattr_remote_entry( + struct xchk_da_btree *ds, + int level, + char *buf_end, + struct xfs_attr_leafblock *leaf, + int idx, + struct xfs_attr_leaf_entry *ent) +{ + struct xfs_attr_leaf_name_remote *rentry = + xfs_attr3_leaf_name_remote(leaf, idx); + char *name_end; + unsigned int namesize; + + name_end = (char *)rentry + offsetof(typeof(*rentry), name[0]); + if (name_end > buf_end) { + xchk_da_set_corrupt(ds, level); + return 0; + } + + namesize = xfs_attr_leaf_entsize_remote(rentry->namelen); + name_end = (char *)rentry + namesize; + if (rentry->namelen == 0) { + xchk_da_set_corrupt(ds, level); + return 0; + } + if (rentry->valueblk == 0 && !(ent->flags & XFS_ATTR_INCOMPLETE)) { + xchk_da_set_corrupt(ds, level); + return 0; + } + if (name_end > buf_end) { + xchk_da_set_corrupt(ds, level); + return 0; + } + + return namesize; +} + /* * Check this leaf entry's relations to everything else. * Returns the number of bytes used for the name/value data. @@ -311,9 +383,6 @@ xchk_xattr_entry( { struct xfs_mount *mp = ds->state->mp; struct xchk_xattr_buf *ab = ds->sc->buf; - char *name_end; - struct xfs_attr_leaf_name_local *lentry; - struct xfs_attr_leaf_name_remote *rentry; unsigned int nameidx; unsigned int namesize; @@ -333,25 +402,14 @@ xchk_xattr_entry( } /* Check the name information. */ - if (ent->flags & XFS_ATTR_LOCAL) { - lentry = xfs_attr3_leaf_name_local(leaf, idx); - namesize = xfs_attr_leaf_entsize_local(lentry->namelen, - be16_to_cpu(lentry->valuelen)); - name_end = (char *)lentry + namesize; - if (lentry->namelen == 0) - xchk_da_set_corrupt(ds, level); - } else { - rentry = xfs_attr3_leaf_name_remote(leaf, idx); - namesize = xfs_attr_leaf_entsize_remote(rentry->namelen); - name_end = (char *)rentry + namesize; - if (rentry->namelen == 0) - xchk_da_set_corrupt(ds, level); - if (rentry->valueblk == 0 && - !(ent->flags & XFS_ATTR_INCOMPLETE)) - xchk_da_set_corrupt(ds, level); - } - if (name_end > buf_end) - xchk_da_set_corrupt(ds, level); + if (ent->flags & XFS_ATTR_LOCAL) + namesize = xchk_xattr_local_entry(ds, level, buf_end, leaf, + idx); + else + namesize = xchk_xattr_remote_entry(ds, level, buf_end, leaf, + idx, ent); + if (!namesize) + return; if (!xchk_xattr_set_map(ds->sc, ab->usedmap, nameidx, namesize)) xchk_da_set_corrupt(ds, level); diff --git a/fs/xfs/scrub/attr_repair.c b/fs/xfs/scrub/attr_repair.c index ab5bd7a4d4ab..2c573897c0c6 100644 --- a/fs/xfs/scrub/attr_repair.c +++ b/fs/xfs/scrub/attr_repair.c @@ -285,6 +285,7 @@ xrep_xattr_salvage_local_attr( struct xfs_attr_leaf_name_local *lentry) { struct xchk_xattr_buf *ab = rx->sc->buf; + char *name_end; unsigned char *value; unsigned int valuelen; unsigned int namesize; @@ -293,10 +294,14 @@ xrep_xattr_salvage_local_attr( * Decode the leaf local entry format. If something seems wrong, we * junk the attribute. */ + name_end = (char *)lentry + offsetof(typeof(*lentry), nameval[0]); + if (name_end > buf_end) + return 0; value = &lentry->nameval[lentry->namelen]; valuelen = be16_to_cpu(lentry->valuelen); namesize = xfs_attr_leaf_entsize_local(lentry->namelen, valuelen); - if ((char *)lentry + namesize > buf_end) + name_end = (char *)lentry + namesize; + if (name_end > buf_end) return 0; if (!xrep_xattr_want_salvage(rx, ent->flags, lentry->nameval, lentry->namelen, value, valuelen)) @@ -333,8 +338,8 @@ xrep_xattr_salvage_remote_attr( .attr_filter = ent->flags & XFS_ATTR_NSP_ONDISK_MASK, .namelen = rentry->namelen, .name = rentry->name, - .valuelen = be32_to_cpu(rentry->valuelen), }; + char *name_end; unsigned int namesize; int error; @@ -343,9 +348,14 @@ xrep_xattr_salvage_remote_attr( * junk the attribute. Note that we should never find a zero-length * remote attribute value. */ + name_end = (char *)rentry + offsetof(typeof(*rentry), name[0]); + if (name_end > buf_end) + return 0; namesize = xfs_attr_leaf_entsize_remote(rentry->namelen); - if ((char *)rentry + namesize > buf_end) + name_end = (char *)rentry + namesize; + if (name_end > buf_end) return 0; + args.valuelen = be32_to_cpu(rentry->valuelen); if (args.valuelen == 0 || !xrep_xattr_want_salvage(rx, ent->flags, rentry->name, rentry->namelen, NULL, args.valuelen)) diff --git a/fs/xfs/scrub/cow_repair.c b/fs/xfs/scrub/cow_repair.c index e8f7cfd4dd22..853744729577 100644 --- a/fs/xfs/scrub/cow_repair.c +++ b/fs/xfs/scrub/cow_repair.c @@ -194,30 +194,37 @@ xrep_cow_mark_missing_staging_rmap( void *priv) { struct xrep_cow *xc = priv; - xfs_fsblock_t fsbno; - xfs_agblock_t rec_bno; - xfs_extlen_t rec_len; - unsigned int adj; + const xfs_agblock_t rec_end = + rec->rm_startblock + rec->rm_blockcount; + int error = 0; - if (rec->rm_owner == XFS_RMAP_OWN_COW) - return 0; - - rec_bno = rec->rm_startblock; - rec_len = rec->rm_blockcount; - if (rec_bno < xc->irec_startbno) { - adj = xc->irec_startbno - rec_bno; - rec_len -= adj; - rec_bno += adj; + /* Was there a gap since the last rmap? */ + if (xc->next_bno < rec->rm_startblock) { + error = xrep_cow_mark_file_range(xc, + XFS_AGB_TO_FSB(xc->sc->mp, cur->bc_ag.pag->pag_agno, + xc->next_bno), + rec->rm_startblock - xc->next_bno); + if (error) + return error; } - if (rec_bno + rec_len > xc->irec_startbno + xc->irec.br_blockcount) { - adj = (rec_bno + rec_len) - - (xc->irec_startbno + xc->irec.br_blockcount); - rec_len -= adj; + if (rec->rm_owner != XFS_RMAP_OWN_COW) { + const xfs_agblock_t bad_bno = + max(xc->irec_startbno, rec->rm_startblock); + const xfs_agblock_t irec_end = + xc->irec_startbno + xc->irec.br_blockcount; + const xfs_extlen_t bad_end = min(irec_end, rec_end); + + error = xrep_cow_mark_file_range(xc, + XFS_AGB_TO_FSB(xc->sc->mp, cur->bc_ag.pag->pag_agno, + bad_bno), + bad_end - bad_bno); + if (error) + return error; } - fsbno = XFS_AGB_TO_FSB(xc->sc->mp, cur->bc_ag.pag->pag_agno, rec_bno); - return xrep_cow_mark_file_range(xc, fsbno, rec_len); + xc->next_bno = max(xc->next_bno, rec_end); + return error; } /* @@ -235,10 +242,12 @@ xrep_cow_find_bad( struct xfs_perag *pag; struct xfs_scrub *sc = xc->sc; xfs_agnumber_t agno; + xfs_agblock_t irec_end; int error; agno = XFS_FSB_TO_AGNO(sc->mp, xc->irec.br_startblock); xc->irec_startbno = XFS_FSB_TO_AGBNO(sc->mp, xc->irec.br_startblock); + irec_end = xc->irec_startbno + xc->irec.br_blockcount; pag = xfs_perag_get(sc->mp, agno); if (!pag) @@ -250,7 +259,7 @@ xrep_cow_find_bad( /* Mark any CoW fork extents that are shared. */ rc_low.rc_startblock = xc->irec_startbno; - rc_high.rc_startblock = xc->irec_startbno + xc->irec.br_blockcount - 1; + rc_high.rc_startblock = irec_end - 1; rc_low.rc_domain = rc_high.rc_domain = XFS_REFC_DOMAIN_SHARED; error = xfs_refcount_query_range(sc->sa.refc_cur, &rc_low, &rc_high, xrep_cow_mark_shared_staging, xc); @@ -259,7 +268,7 @@ xrep_cow_find_bad( /* Make sure there are CoW staging extents for the whole mapping. */ rc_low.rc_startblock = xc->irec_startbno; - rc_high.rc_startblock = xc->irec_startbno + xc->irec.br_blockcount - 1; + rc_high.rc_startblock = irec_end - 1; rc_low.rc_domain = rc_high.rc_domain = XFS_REFC_DOMAIN_COW; xc->next_bno = xc->irec_startbno; error = xfs_refcount_query_range(sc->sa.refc_cur, &rc_low, &rc_high, @@ -267,12 +276,11 @@ xrep_cow_find_bad( if (error) goto out_sa; - if (xc->next_bno < xc->irec_startbno + xc->irec.br_blockcount) { + if (xc->next_bno < irec_end) { error = xrep_cow_mark_file_range(xc, XFS_AGB_TO_FSB(sc->mp, pag->pag_agno, xc->next_bno), - xc->irec_startbno + xc->irec.br_blockcount - - xc->next_bno); + irec_end - xc->next_bno); if (error) goto out_sa; } @@ -281,11 +289,21 @@ xrep_cow_find_bad( rm_low.rm_startblock = xc->irec_startbno; memset(&rm_high, 0xFF, sizeof(rm_high)); rm_high.rm_startblock = xc->irec_startbno + xc->irec.br_blockcount - 1; + xc->next_bno = xc->irec_startbno; error = xfs_rmap_query_range(sc->sa.rmap_cur, &rm_low, &rm_high, xrep_cow_mark_missing_staging_rmap, xc); if (error) goto out_sa; + if (xc->next_bno < irec_end) { + error = xrep_cow_mark_file_range(xc, + XFS_AGB_TO_FSB(sc->mp, pag->pag_agno, + xc->next_bno), + irec_end - xc->next_bno); + if (error) + goto out_sa; + } + /* * If userspace is forcing us to rebuild the CoW fork or someone turned * on the debugging knob, replace everything in the CoW fork. diff --git a/fs/xfs/scrub/dir.c b/fs/xfs/scrub/dir.c index 284f72ef2b58..2bb5ead7cffd 100644 --- a/fs/xfs/scrub/dir.c +++ b/fs/xfs/scrub/dir.c @@ -305,6 +305,74 @@ xchk_dir_actor( return error; } +/* + * Deal with an unused directory entry. Returns a positive number of bytes to + * advance in the directory block, zero to stop processing, or -1 if corruption + * was found. + */ +static int +xchk_dir_check_unused( + struct xchk_da_btree *ds, + struct xfs_buf *bp, + unsigned int iter_off, + unsigned int end, + xfs_dablk_t rec_bno) +{ + struct xfs_dir2_data_unused *dup = bp->b_addr + iter_off; + unsigned int advance; + + if (iter_off + sizeof(*dup) > end) { + xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); + return -1; + } + + advance = xfs_dir2_data_unusedsize(be16_to_cpu(dup->length)); + if (!advance || iter_off + advance > end) { + xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); + return -1; + } + + return advance; +} + +/* + * Check a directory entry. Returns a positive number of bytes to advance + * in the directory block, zero to stop processing, or -1 if corruption was + * found. + */ +static int +xchk_dir_check_entry( + struct xchk_da_btree *ds, + struct xfs_buf *bp, + unsigned int iter_off, + unsigned int end, + struct xfs_dir2_data_entry *dent, + xfs_dablk_t rec_bno) +{ + struct xfs_dir2_data_entry *dep = bp->b_addr + iter_off; + struct xfs_mount *mp = ds->state->mp; + unsigned int advance; + + /* must have namelen */ + advance = offsetof(typeof(*dep), name); + if (iter_off + advance >= end) { + xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); + return -1; + } + + /* Don't walk off the end of the block. */ + advance = xfs_dir2_data_entsize(mp, dep->namelen); + if (!advance || iter_off + advance > end) { + xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); + return -1; + } + + if (dep == dent) + return 0; + + return advance; +} + /* Scrub a directory btree record. */ STATIC int xchk_dir_rec( @@ -330,6 +398,7 @@ xchk_dir_rec( xfs_dahash_t hash; struct xfs_dir3_icleaf_hdr hdr; unsigned int tag; + bool foundit = false; int error; ASSERT(blk->magic == XFS_DIR2_LEAF1_MAGIC || @@ -374,28 +443,42 @@ xchk_dir_rec( dent = bp->b_addr + off; /* Make sure we got a real directory entry. */ - iter_off = geo->data_entry_offset; end = xfs_dir3_data_end_offset(geo, bp->b_addr); if (!end) { xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); goto out_relse; } - for (;;) { - struct xfs_dir2_data_entry *dep = bp->b_addr + iter_off; + iter_off = geo->data_entry_offset; + while (iter_off < end) { struct xfs_dir2_data_unused *dup = bp->b_addr + iter_off; + int advance; - if (iter_off >= end) { + /* must have freetag */ + advance = offsetof(typeof(*dup), length); + if (iter_off + advance >= end) { xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); goto out_relse; } - if (be16_to_cpu(dup->freetag) == XFS_DIR2_DATA_FREE_TAG) { - iter_off += be16_to_cpu(dup->length); - continue; - } - if (dep == dent) + if (dup->freetag == cpu_to_be16(XFS_DIR2_DATA_FREE_TAG)) + advance = xchk_dir_check_unused(ds, bp, iter_off, end, + rec_bno); + else + advance = xchk_dir_check_entry(ds, bp, iter_off, end, + dent, rec_bno); + if (advance < 0) + goto out_relse; + if (!advance) { + foundit = true; break; - iter_off += xfs_dir2_data_entsize(mp, dep->namelen); + } + iter_off += advance; + } + + /* Hash tree must point to the exact dirent */ + if (!foundit) { + xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); + goto out_relse; } /* Retrieve the entry, sanity check it, and compare hashes. */ @@ -409,6 +492,13 @@ xchk_dir_rec( goto out_relse; } + /* Name must not overflow end of block */ + if ((char *)dent->name + dent->namelen > + (char *)bp->b_addr + BBTOB(bp->b_length)) { + xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno); + goto out_relse; + } + /* Does the directory hash match? */ dname.name = dent->name; dname.len = dent->namelen; diff --git a/fs/xfs/scrub/dir_repair.c b/fs/xfs/scrub/dir_repair.c index 2bc2354cd85d..72fe0ae3e18d 100644 --- a/fs/xfs/scrub/dir_repair.c +++ b/fs/xfs/scrub/dir_repair.c @@ -456,6 +456,63 @@ xrep_dir_salvage_data_entry( return xrep_dir_salvage_entry(rd, dep->name, dep->namelen, ino); } +/* + * Deal with an unused directory entry. Returns a positive number of bytes to + * advance in the directory block, or zero to stop processing. + */ +static int +xrep_dir_recover_unused( + struct xfs_buf *bp, + unsigned int offset, + unsigned int end) +{ + struct xfs_dir2_data_unused *dup = bp->b_addr + offset; + unsigned int advance; + + if (offset + sizeof(*dup) > end) + return 0; + + advance = xfs_dir2_data_unusedsize(be16_to_cpu(dup->length)); + if (!advance || offset + advance > end) + return 0; + + return advance; +} + +/* + * Salvage a directory entry. Returns a positive number of bytes to advance + * in the directory block, zero to stop processing, or a negative errno. + */ +static int +xrep_dir_recover_entry( + struct xrep_dir *rd, + struct xfs_buf *bp, + unsigned int offset, + unsigned int end) +{ + struct xfs_dir2_data_entry *dep = bp->b_addr + offset; + unsigned int advance; + int error; + + /* must have namelen */ + advance = offsetof(typeof(*dep), name); + if (offset + advance >= end) + return 0; + + /* Don't walk off the end of the block. */ + advance = xfs_dir2_data_entsize(rd->sc->mp, dep->namelen); + if (!advance || offset + advance > end) + return 0; + + /* Ok, let's save this entry. */ + error = xrep_dir_salvage_data_entry(rd, dep); + if (error < 0) + return error; + + return advance; +} + + /* Try to recover block/data format directory entries. */ STATIC int xrep_dir_recover_data( @@ -477,33 +534,25 @@ xrep_dir_recover_data( while (offset < end) { struct xfs_dir2_data_unused *dup = bp->b_addr + offset; - struct xfs_dir2_data_entry *dep = bp->b_addr + offset; - unsigned int advance; + int advance; if (xchk_should_terminate(rd->sc, &error)) return error; - /* Skip unused entries. */ - if (be16_to_cpu(dup->freetag) == XFS_DIR2_DATA_FREE_TAG) { - if (!dup->length) - break; - offset += be16_to_cpu(dup->length); - continue; - } + /* must have freetag */ + advance = offsetof(typeof(*dup), length); + if (offset + advance >= end) + break; - /* Don't walk off the end of the block. */ - advance = xfs_dir2_data_entsize(rd->sc->mp, dep->namelen); + if (dup->freetag == cpu_to_be16(XFS_DIR2_DATA_FREE_TAG)) + advance = xrep_dir_recover_unused(bp, offset, end); + else + advance = xrep_dir_recover_entry(rd, bp, offset, end); + if (advance < 0) + return advance; if (!advance) break; offset += advance; - if (offset > end) - break; - - /* Ok, let's save this entry. */ - error = xrep_dir_salvage_data_entry(rd, dep); - if (error) - return error; - } return 0; @@ -526,6 +575,19 @@ xrep_dir_recover_sf( hdr = ifp->if_data; end = (unsigned char *)ifp->if_data + ifp->if_bytes; + /* sf header must be big enough for count/i8count */ + if (!hdr || ifp->if_bytes < 2) + return 0; + + /* and large enough for the parent inumber */ + if (hdr->i8count) { + if (ifp->if_bytes < offsetof(struct xfs_dir2_sf_hdr, parent[8])) + return 0; + } else { + if (ifp->if_bytes < offsetof(struct xfs_dir2_sf_hdr, parent[4])) + return 0; + } + ino = xfs_dir2_sf_get_parent_ino(hdr); trace_xrep_dir_salvaged_parent(rd->sc->ip, ino); diff --git a/fs/xfs/scrub/dirtree.c b/fs/xfs/scrub/dirtree.c index 8933506c4830..35848cbfae92 100644 --- a/fs/xfs/scrub/dirtree.c +++ b/fs/xfs/scrub/dirtree.c @@ -849,7 +849,8 @@ xchk_dirtree_find_paths_to_root( * A parent pointer of @sc->ip is bad, don't * bother continuing. */ - break; + xchk_set_incomplete(sc); + return error; } if (error == -ESTALE) { /* This had better be an invalidation. */ @@ -970,6 +971,7 @@ xchk_dirtree( * parent pointers are corrupt; this scan cannot be completed * without full information. */ + xchk_set_incomplete(sc); xchk_ino_xref_set_corrupt(sc, sc->ip->i_ino); error = 0; goto out_scanlock; diff --git a/fs/xfs/scrub/fscounters.c b/fs/xfs/scrub/fscounters.c index 98634c668b56..ad23343c52f7 100644 --- a/fs/xfs/scrub/fscounters.c +++ b/fs/xfs/scrub/fscounters.c @@ -296,7 +296,7 @@ xchk_fscount_aggregate_agcounts( { struct xfs_mount *mp = sc->mp; struct xfs_perag *pag; - uint64_t delayed; + int64_t delayed; xfs_agnumber_t agno; int tries = 8; int error = 0; @@ -359,6 +359,11 @@ xchk_fscount_aggregate_agcounts( * of the computation. */ delayed = percpu_counter_sum(&mp->m_delalloc_blks); + if (delayed < 0) { + ASSERT(delayed >= 0); + xchk_set_incomplete(sc); + return -EFSCORRUPTED; + } fsc->fdblocks -= delayed; trace_xchk_fscounters_calc(mp, fsc->icount, fsc->ifree, fsc->fdblocks, @@ -416,6 +421,7 @@ xchk_fscount_count_frextents( struct xchk_fscounters *fsc) { struct xfs_mount *mp = sc->mp; + int64_t delayed; int error; fsc->frextents = 0; @@ -431,7 +437,14 @@ xchk_fscount_count_frextents( goto out_unlock; } - fsc->frextents_delayed = percpu_counter_sum(&mp->m_delalloc_rtextents); + delayed = percpu_counter_sum(&mp->m_delalloc_rtextents); + if (delayed < 0) { + ASSERT(delayed >= 0); + xchk_set_incomplete(sc); + error = -EFSCORRUPTED; + goto out_unlock; + } + fsc->frextents_delayed = delayed; out_unlock: xfs_rtbitmap_unlock_shared(sc->mp, XFS_RBMLOCK_BITMAP); @@ -606,6 +619,13 @@ xchk_fscounters( try_again = true; } + if (fsc->frextents < fsc->frextents_delayed) { + if (fsc->frozen) + xchk_set_incomplete(sc); + else + try_again = true; + } + if (!xchk_fscount_within_range(sc, frextents, &mp->m_frextents, fsc->frextents - fsc->frextents_delayed)) { if (fsc->frozen) diff --git a/fs/xfs/scrub/inode_repair.c b/fs/xfs/scrub/inode_repair.c index 6be4551bffbd..fa84a2efee43 100644 --- a/fs/xfs/scrub/inode_repair.c +++ b/fs/xfs/scrub/inode_repair.c @@ -38,6 +38,8 @@ #include "xfs_log_priv.h" #include "xfs_health.h" #include "xfs_symlink_remote.h" +#include "xfs_dquot_item.h" +#include "xfs_dquot.h" #include "scrub/xfs_scrub.h" #include "scrub/scrub.h" #include "scrub/common.h" @@ -1570,6 +1572,20 @@ xrep_inode_ids( trace_xrep_inode_ids(sc); + /* + * We changed one of the file IDs, so we must detach the dquots to + * prevent further changes from being accounted to them, and try to + * attach the dquot for the new id to absorb any other block usage + * changes. We can't do that right now because the existing dquots + * might be attached to the scrub transaction and we're in the middle + * of fixing the inode, so we'll switch the incore dquot pointers + * later. + * + * Schedule a quotacheck so the accounting discrepancies will get fixed + * eventually. We'll have to repeat this later when we do the actual + * switch. + */ + if (!uid_valid(VFS_I(sc->ip)->i_uid)) { i_uid_write(VFS_I(sc->ip), 0); dirty = true; @@ -1876,6 +1892,112 @@ xrep_inode_unlinked( return 0; } +#ifdef CONFIG_XFS_QUOTA +static inline bool +xrep_inode_need_reset_dquots( + struct xfs_scrub *sc) +{ + struct xfs_mount *mp = sc->mp; + struct xfs_inode *ip = sc->ip; + + if (!XFS_IS_QUOTA_ON(mp)) + return false; + if (xfs_is_quota_inode(&mp->m_sb, I_INO(ip))) + return false; + + if (XFS_IS_UQUOTA_ON(sc->mp) && + (!ip->i_udquot || ip->i_udquot->q_id != i_uid_read(VFS_I(ip)))) + return true; + if (XFS_IS_GQUOTA_ON(sc->mp) && + (!ip->i_gdquot || ip->i_gdquot->q_id != i_gid_read(VFS_I(ip)))) + return true; + if (XFS_IS_PQUOTA_ON(sc->mp) && + (!ip->i_pdquot || ip->i_pdquot->q_id != ip->i_projid)) + return true; + + return false; +} + +/* Switch dquot, return dqtype if we need quotacheck */ +static xfs_dqtype_t +xrep_inode_switch_dquot( + struct xfs_scrub *sc, + xfs_dqtype_t type, + struct xfs_dquot **dqp) +{ + xfs_dqtype_t ret = 0; + xfs_dqid_t id = xfs_qm_id_for_quotatype(sc->ip, type); + + if (*dqp != NULL && (*dqp)->q_id != id) { + xfs_qm_dqrele(*dqp); + *dqp = NULL; + } + + if (!*dqp) { + if (!xfs_qm_dqget_inode(sc->ip, type, true, dqp)) + xfs_dqunlock(*dqp); + ret = type; + } + return ret; +} + +static int +xrep_inode_reset_dquots( + struct xfs_scrub *sc) +{ + xfs_dqtype_t need = 0; + int error; + + /* + * If we changed the file IDs, then we need to drop the transaction so + * that we can detach the dquots and attach new ones. + * + * Commit the scrub transaction so that there are no dangling + * references to any dquots that might get detached, and retain the + * ILOCK, as required by the dqattach code. + * + * Note that dqattach can cycle the ILOCK when creating a new dquot, so + * this must be the very last thing that the repair function does. + */ + error = xrep_trans_commit(sc); + if (error) + return error; + + if (XFS_IS_UQUOTA_ON(sc->mp)) + need |= xrep_inode_switch_dquot(sc, XFS_DQTYPE_USER, + &sc->ip->i_udquot); + if (XFS_IS_GQUOTA_ON(sc->mp)) + need |= xrep_inode_switch_dquot(sc, XFS_DQTYPE_GROUP, + &sc->ip->i_gdquot); + if (XFS_IS_PQUOTA_ON(sc->mp)) + need |= xrep_inode_switch_dquot(sc, XFS_DQTYPE_PROJ, + &sc->ip->i_pdquot); + + xchk_iunlock(sc, XFS_ILOCK_EXCL); + + /* + * Now that we've updated the dquots, force quotacheck once again. + * In theory we could be racing against a concurrent quotacheck. + * Note that we can't hold the ILOCK when allocating a transaction. + */ + error = xchk_trans_alloc(sc, 0); + if (error) + return error; + + if (need & XFS_DQTYPE_USER) + xrep_force_quotacheck(sc, XFS_DQTYPE_USER); + if (need & XFS_DQTYPE_GROUP) + xrep_force_quotacheck(sc, XFS_DQTYPE_GROUP); + if (need & XFS_DQTYPE_PROJ) + xrep_force_quotacheck(sc, XFS_DQTYPE_PROJ); + + return 0; +} +#else +# define xrep_inode_need_reset_dquots(sc) (false) +# define xrep_inode_reset_dquots(sc) (0) +#endif + /* Repair an inode's fields. */ int xrep_inode( @@ -1930,5 +2052,13 @@ xrep_inode( if (error) return error; - return xrep_defer_finish(sc); + error = xrep_defer_finish(sc); + if (error) + return error; + + /* Must come last */ + if (xrep_inode_need_reset_dquots(sc)) + return xrep_inode_reset_dquots(sc); + + return 0; } diff --git a/fs/xfs/scrub/newbt.c b/fs/xfs/scrub/newbt.c index b818d9b21578..80f9bf7e399b 100644 --- a/fs/xfs/scrub/newbt.c +++ b/fs/xfs/scrub/newbt.c @@ -60,7 +60,7 @@ xrep_newbt_estimate_slack( free = sc->sa.pag->pagf_freeblks; sz = xfs_ag_block_count(sc->mp, sc->sa.pag->pag_agno); } else { - free = percpu_counter_sum(&sc->mp->m_fdblocks); + free = percpu_counter_sum_positive(&sc->mp->m_fdblocks); sz = sc->mp->m_sb.sb_dblocks; } diff --git a/fs/xfs/scrub/quota_repair.c b/fs/xfs/scrub/quota_repair.c index 95d5bb8c1003..0def7a1b366a 100644 --- a/fs/xfs/scrub/quota_repair.c +++ b/fs/xfs/scrub/quota_repair.c @@ -299,7 +299,6 @@ xrep_quota_block( xfs_dqid_t id) { struct xfs_dqblk *dqblk; - struct xfs_disk_dquot *ddq; struct xfs_quotainfo *qi = sc->mp->m_quotainfo; struct xfs_def_quota *defq = xfs_get_defquota(qi, dqtype); struct xfs_buf *bp = NULL; @@ -321,14 +320,21 @@ xrep_quota_block( break; case 0: dqblk = bp->b_addr; - ddq = &dqblk[0].dd_diskdq; + error = 0; /* * If there's nothing that would impede a dqiterate, we're * done. */ - if ((ddq->d_type & XFS_DQTYPE_REC_MASK) == dqtype && - id == be32_to_cpu(ddq->d_id)) { + for (i = 0; i < qi->qi_dqperchunk; i++, dqblk++) { + struct xfs_disk_dquot *ddq = &dqblk->dd_diskdq; + + if ((ddq->d_type & XFS_DQTYPE_REC_MASK) != dqtype || + id + i != be32_to_cpu(ddq->d_id)) + error++; + } + + if (!error) { xfs_trans_brelse(sc->tp, bp); return 0; } @@ -341,7 +347,9 @@ xrep_quota_block( dqblk = bp->b_addr; bp->b_ops = &xfs_dquot_buf_ops; for (i = 0; i < qi->qi_dqperchunk; i++, dqblk++) { - ddq = &dqblk->dd_diskdq; + struct xfs_disk_dquot *ddq = &dqblk->dd_diskdq; + bool was_bigtime = + !!(ddq->d_type & XFS_DQTYPE_BIGTIME); trace_xrep_disk_dquot(sc->mp, dqtype, id + i); @@ -350,8 +358,21 @@ xrep_quota_block( ddq->d_type = dqtype; ddq->d_id = cpu_to_be32(id + i); - if (xfs_has_bigtime(sc->mp) && ddq->d_id) + if (xfs_has_bigtime(sc->mp) && ddq->d_id) { + /* + * If something was broken with this dquot and it was a + * non-bigtime dquot, we'll grant everyone a fresh + * grace period by zeroing the timer field rather than + * try to interpret what might be garbage. + */ + if (!was_bigtime) { + ddq->d_btimer = 0; + ddq->d_itimer = 0; + ddq->d_rtbtimer = 0; + } + ddq->d_type |= XFS_DQTYPE_BIGTIME; + } xrep_quota_fix_timer(sc->mp, ddq, ddq->d_blk_softlimit, ddq->d_bcount, &ddq->d_btimer, diff --git a/fs/xfs/scrub/reap.c b/fs/xfs/scrub/reap.c index 4ce345ae4565..0c827798ade6 100644 --- a/fs/xfs/scrub/reap.c +++ b/fs/xfs/scrub/reap.c @@ -905,12 +905,12 @@ xrep_reap_bmapi_iter( /* * Schedule removal of the mapping from the fork. We use * deferred log intents in this function to control the exact - * sequence of metadata updates. + * sequence of metadata updates. The BUI queues a RUI to + * remove the reverse mapping. */ xfs_bmap_unmap_extent(sc->tp, ip, whichfork, imap); xfs_trans_mod_dquot_byino(sc->tp, ip, XFS_TRANS_DQ_BCOUNT, -(int64_t)imap->br_blockcount); - xfs_rmap_unmap_extent(sc->tp, ip, whichfork, imap); return 0; } diff --git a/fs/xfs/scrub/rtsummary.c b/fs/xfs/scrub/rtsummary.c index 746990ae0ec1..942f93b1e22e 100644 --- a/fs/xfs/scrub/rtsummary.c +++ b/fs/xfs/scrub/rtsummary.c @@ -201,15 +201,22 @@ xchk_rtsum_compute( struct xfs_scrub *sc) { struct xfs_mount *mp = sc->mp; + struct xchk_rtsummary *rts = sc->buf; unsigned long long rtbmp_blocks; + int error; /* If the bitmap size doesn't match the computed size, bail. */ rtbmp_blocks = xfs_rtbitmap_blockcount(mp, mp->m_sb.sb_rextents); if (XFS_FSB_TO_B(mp, rtbmp_blocks) != mp->m_rbmip->i_disk_size) return -EFSCORRUPTED; - return xfs_rtalloc_query_all(sc->mp, sc->tp, xchk_rtsum_record_free, + error = xfs_rtalloc_query_all(sc->mp, sc->tp, xchk_rtsum_record_free, sc); + if (error) + return error; + + rts->summary_computed = true; + return 0; } /* Compare the rtsummary file against the one we computed. */ diff --git a/fs/xfs/scrub/rtsummary.h b/fs/xfs/scrub/rtsummary.h index e44b04cb6e2d..af1ab64c3f21 100644 --- a/fs/xfs/scrub/rtsummary.h +++ b/fs/xfs/scrub/rtsummary.h @@ -17,6 +17,7 @@ struct xchk_rtsummary { xfs_filblks_t rsumblocks; unsigned int rsumlevels; unsigned int resblks; + bool summary_computed; /* suminfo position of xfile as we write buffers to disk. */ xfs_rtsumoff_t prep_wordoff; diff --git a/fs/xfs/scrub/rtsummary_repair.c b/fs/xfs/scrub/rtsummary_repair.c index 7deeb948cb70..536f0b8e3bfd 100644 --- a/fs/xfs/scrub/rtsummary_repair.c +++ b/fs/xfs/scrub/rtsummary_repair.c @@ -110,7 +110,7 @@ xrep_rtsummary( return -EOPNOTSUPP; /* Walk away if we disagree on the size of the rt bitmap. */ - if (rts->rbmblocks != mp->m_sb.sb_rbmblocks) + if (rts->rbmblocks != mp->m_sb.sb_rbmblocks || !rts->summary_computed) return 0; /* Make sure any problems with the fork are fixed. */ diff --git a/fs/xfs/scrub/xfarray.c b/fs/xfs/scrub/xfarray.c index ccf7cfb63bbc..3cf1ab618f2e 100644 --- a/fs/xfs/scrub/xfarray.c +++ b/fs/xfs/scrub/xfarray.c @@ -819,7 +819,7 @@ xfarray_sort_scan( (idx < si->first_folio_idx || idx > si->last_folio_idx)) xfarray_sort_scan_done(si); - /* Grab the first folio that backs this array element. */ + /* Grab the folio that backs this array element. */ if (!si->folio) { struct folio *folio; loff_t next_pos; @@ -830,6 +830,10 @@ xfarray_sort_scan( return PTR_ERR(folio); si->folio = folio; + /* No folio? Read into the scratchpad. */ + if (!si->folio) + goto read_scratchpad; + si->first_folio_idx = xfarray_idx(si->array, folio_pos(si->folio) + si->array->obj_size - 1); @@ -838,14 +842,24 @@ xfarray_sort_scan( if (xfarray_pos(si->array, si->last_folio_idx + 1) > next_pos) si->last_folio_idx--; + /* + * If this folio still doesn't cover the desired element, it must + * cross a folio boundary. Read into the scratchpad. + */ + if (idx < si->first_folio_idx || idx > si->last_folio_idx) { + xfarray_sort_scan_done(si); + goto read_scratchpad; + } + trace_xfarray_sort_scan(si, idx); } +read_scratchpad: /* - * If this folio still doesn't cover the desired element, it must cross - * a folio boundary. Read into the scratchpad and we're done. + * If we don't have a folio mapping the entire array element, read into + * the scratchpad and we're done. */ - if (idx < si->first_folio_idx || idx > si->last_folio_idx) { + if (!si->folio) { void *temp = xfarray_scratch(si->array); error = xfile_load(si->array->xfile, temp, si->array->obj_size, diff --git a/fs/xfs/xfs_discard.c b/fs/xfs/xfs_discard.c index 0b343776da8c..66c2f2fcec7b 100644 --- a/fs/xfs/xfs_discard.c +++ b/fs/xfs/xfs_discard.c @@ -160,6 +160,8 @@ struct xfs_trim_cur { xfs_agblock_t end; xfs_extlen_t minlen; bool by_bno; + /* restart point (with count) for the by-length walk */ + xfs_agblock_t restart_bno; }; static int @@ -205,14 +207,15 @@ xfs_trim_gather_extents( error = xfs_alloc_lookup_le(cur, tcur->start, 0, &i); if (!error && !i) error = xfs_alloc_lookup_ge(cur, tcur->start, 0, &i); - } else if (tcur->start == 0) { - /* first time through a by-len starts with max length */ - cur = xfs_cntbt_init_cursor(mp, tp, agbp, pag); - error = xfs_alloc_lookup_ge(cur, 0, tcur->count, &i); } else { - /* nth time through a by-len starts where we left off */ + /* + * A by-len walk starts at the last record of the longest + * length (restart_bno is NULLAGBLOCK until the first batch + * ends) and then where the previous batch stopped. + */ cur = xfs_cntbt_init_cursor(mp, tp, agbp, pag); - error = xfs_alloc_lookup_le(cur, tcur->start, tcur->count, &i); + error = xfs_alloc_lookup_le(cur, tcur->restart_bno, + tcur->count, &i); } if (error) goto out_del_cursor; @@ -242,9 +245,15 @@ xfs_trim_gather_extents( if (--batch <= 0) { /* * Update the cursor to point at this extent so we - * restart the next batch from this extent. + * restart the next batch from this extent. The + * by-length walk does not visit extents in block + * order, so it must keep the start of the range and + * restart from restart_bno instead. */ - tcur->start = fbno; + if (tcur->by_bno) + tcur->start = fbno; + else + tcur->restart_bno = fbno; tcur->count = flen; break; } @@ -347,6 +356,7 @@ xfs_trim_perag_extents( .start = start, .end = end, .minlen = minlen, + .restart_bno = NULLAGBLOCK, }; int error = 0; diff --git a/fs/xfs/xfs_ioctl32.c b/fs/xfs/xfs_ioctl32.c index b64785dc4354..c2e01fcb6015 100644 --- a/fs/xfs/xfs_ioctl32.c +++ b/fs/xfs/xfs_ioctl32.c @@ -123,8 +123,8 @@ xfs_ioctl32_bstat_copyin( xfs_ioctl32_bstime_copyin(&bstat->bs_atime, &bstat32->bs_atime) || xfs_ioctl32_bstime_copyin(&bstat->bs_mtime, &bstat32->bs_mtime) || xfs_ioctl32_bstime_copyin(&bstat->bs_ctime, &bstat32->bs_ctime) || - get_user(bstat->bs_blocks, &bstat32->bs_size) || - get_user(bstat->bs_xflags, &bstat32->bs_size) || + get_user(bstat->bs_blocks, &bstat32->bs_blocks) || + get_user(bstat->bs_xflags, &bstat32->bs_xflags) || get_user(bstat->bs_extsize, &bstat32->bs_extsize) || get_user(bstat->bs_extents, &bstat32->bs_extents) || get_user(bstat->bs_gen, &bstat32->bs_gen) || diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c index 3483b3728b7b..afd655d8f2ae 100644 --- a/fs/xfs/xfs_iomap.c +++ b/fs/xfs/xfs_iomap.c @@ -258,6 +258,14 @@ xfs_iomap_write_direct( ASSERT(count_fsb > 0); + /* + * xfs_bmapi_write() maps at most XFS_MAX_BMBT_EXTLEN blocks in one + * mapping, so don't reserve blocks for more. A pNFS layout range + * comes from a client, and its block count may not even fit the + * 32-bit reservation. The caller maps the rest with a later call. + */ + count_fsb = min_t(xfs_filblks_t, count_fsb, XFS_MAX_BMBT_EXTLEN); + resaligned = xfs_aligned_fsb_count(offset_fsb, count_fsb, xfs_get_extsz_hint(ip)); if (unlikely(XFS_IS_REALTIME_INODE(ip))) { diff --git a/fs/xfs/xfs_pnfs.c b/fs/xfs/xfs_pnfs.c index 23d16186e1a3..fa027574d6de 100644 --- a/fs/xfs/xfs_pnfs.c +++ b/fs/xfs/xfs_pnfs.c @@ -101,7 +101,15 @@ xfs_fs_map_update_inode( ip->i_diflags |= XFS_DIFLAG_PREALLOC; xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); - return xfs_trans_commit(tp); + error = xfs_trans_commit(tp); + if (error) + return error; + + /* + * Force the log so that the blocks handed out to the client are + * guaranteed to be present even after a server crash. + */ + return xfs_log_force_inode(ip); } /* @@ -126,6 +134,7 @@ xfs_fs_map_blocks( uint lock_flags; int error = 0; u64 seq; + bool update_inode = false; if (xfs_is_shutdown(mp)) return -EIO; @@ -159,10 +168,9 @@ xfs_fs_map_blocks( if (!write) limit = max(limit, round_up(i_size_read(inode), inode->i_sb->s_blocksize)); - if (offset > limit) + if (offset < 0 || offset >= limit) goto out_unlock; - if (offset > limit - length) - length = limit - offset; + length = min_t(u64, length, limit - offset); error = filemap_write_and_wait(inode->i_mapping); if (error) @@ -194,20 +202,27 @@ xfs_fs_map_blocks( end_fsb - offset_fsb, 0, &imap, &seq); if (error) goto out_unlock; + update_inode = true; + } else { + xfs_iunlock(ip, lock_flags); /* - * Ensure the next transaction is committed synchronously so - * that the blocks allocated and handed out to the client are - * guaranteed to be present even after a server crash. + * Blocks past EOF that were allocated for buffered writes, such + * as speculative preallocation, can be freed by + * xfs_free_eofblocks() without breaking the layout unless the + * inode is marked as preallocated. Mark it before handing them + * out for writing, as for blocks allocated above. */ + if (write && !(ip->i_diflags & XFS_DIFLAG_PREALLOC) && + imap.br_startoff + imap.br_blockcount > + XFS_B_TO_FSB(mp, XFS_ISIZE(ip))) + update_inode = true; + } + + if (update_inode) { error = xfs_fs_map_update_inode(ip); - if (!error) - error = xfs_log_force_inode(ip); if (error) goto out_unlock; - - } else { - xfs_iunlock(ip, lock_flags); } xfs_iunlock(ip, XFS_IOLOCK_EXCL); @@ -286,9 +301,7 @@ xfs_fs_commit_blocks( if (start > size) continue; - end = start + maps[i].length; - if (end > size) - end = size; + end = start + min_t(u64, maps[i].length, size - start); length = end - start; if (!length) diff --git a/include/acpi/cppc_acpi.h b/include/acpi/cppc_acpi.h index 31767c65be20..5fe0aa7a6c5a 100644 --- a/include/acpi/cppc_acpi.h +++ b/include/acpi/cppc_acpi.h @@ -160,6 +160,7 @@ extern int cpc_write_ffh(int cpunum, struct cpc_reg *reg, u64 val); extern int cppc_get_epp_perf(int cpunum, u64 *epp_perf); extern int cppc_set_epp_perf(int cpu, struct cppc_perf_ctrls *perf_ctrls, bool enable); extern int cppc_get_auto_sel(int cpu, bool *enable); +bool cppc_auto_sel_is_immutable(int cpu); extern int cppc_set_auto_sel(int cpu, bool enable); extern int amd_get_highest_perf(unsigned int cpu, u32 *highest_perf); extern int amd_get_boost_ratio_numerator(unsigned int cpu, u64 *numerator); @@ -233,6 +234,11 @@ static inline int cppc_get_auto_sel(int cpu, bool *enable) { return -EOPNOTSUPP; } +static inline bool cppc_auto_sel_is_immutable(int cpu) +{ + return false; +} + static inline int cppc_set_auto_sel(int cpu, bool enable) { return -EOPNOTSUPP; diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h index 253dee8077b1..c046a09b0b48 100644 --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -375,8 +375,7 @@ struct mmu_gather { void tlb_flush_mmu(struct mmu_gather *tlb); static inline void __tlb_adjust_range(struct mmu_gather *tlb, - unsigned long address, - unsigned int range_size) + unsigned long address, unsigned long range_size) { tlb->start = min(tlb->start, address); tlb->end = max(tlb->end, address + range_size); diff --git a/include/drm/drm_panel.h b/include/drm/drm_panel.h index 10015891b056..daaa259ef898 100644 --- a/include/drm/drm_panel.h +++ b/include/drm/drm_panel.h @@ -269,6 +269,28 @@ struct drm_panel { bool enabled; }; +void *__devm_drm_panel_alloc(struct device *dev, size_t size, size_t offset, + const struct drm_panel_funcs *funcs, + int connector_type); + +/** + * devm_drm_panel_alloc - Allocate and initialize a refcounted panel. + * + * @dev: struct device of the panel device + * @type: the type of the struct which contains struct &drm_panel + * @member: the name of the &drm_panel within @type + * @funcs: callbacks for this panel + * @connector_type: the connector type (DRM_MODE_CONNECTOR_*) corresponding to + * the panel interface + * + * Returns: + * Pointer to container structure embedding the panel, ERR_PTR on failure. + */ +#define devm_drm_panel_alloc(dev, type, member, funcs, connector_type) \ + ((type *)__devm_drm_panel_alloc(dev, sizeof(type), \ + offsetof(type, member), funcs, \ + connector_type)) + void drm_panel_init(struct drm_panel *panel, struct device *dev, const struct drm_panel_funcs *funcs, int connector_type); diff --git a/include/dt-bindings/clock/nuvoton,ma35d1-clk.h b/include/dt-bindings/clock/nuvoton,ma35d1-clk.h index ba2d70f776a6..764d837d6162 100644 --- a/include/dt-bindings/clock/nuvoton,ma35d1-clk.h +++ b/include/dt-bindings/clock/nuvoton,ma35d1-clk.h @@ -248,6 +248,7 @@ #define ADC_GATE 233 #define EADC_DIV 234 #define EADC_GATE 235 -#define CLK_MAX_IDX 236 +#define PCLK3_DIV4096 236 +#define PCLK4_DIV4096 237 #endif /* __DT_BINDINGS_CLOCK_NUVOTON_MA35D1_CLK_H */ diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h index c819c5d16613..c468e3d177d7 100644 --- a/include/kvm/arm_arch_timer.h +++ b/include/kvm/arm_arch_timer.h @@ -152,4 +152,19 @@ static inline bool has_cntpoff(void) return (has_vhe() && cpus_have_final_cap(ARM64_HAS_ECV_CNTPOFF)); } +/* + * CVAL to program when the hardware won't apply the timer's offset, so that + * its compare against the raw counter matches the guest's at 'now'. An + * expired timer stays expired; one past the wrap never fires. + */ +static inline u64 timer_apply_offset(u64 cval, u64 offset, u64 now) +{ + u64 hw = cval + offset; + + if (now - offset >= cval) + return 0; + + return hw > now ? hw : U64_MAX; +} + #endif diff --git a/include/linux/assoc_array_priv.h b/include/linux/assoc_array_priv.h index dca733ef6750..37a9f76e1505 100644 --- a/include/linux/assoc_array_priv.h +++ b/include/linux/assoc_array_priv.h @@ -80,7 +80,7 @@ struct assoc_array_edit { struct assoc_array_ptr **leaf_p; struct assoc_array_ptr *dead_leaf; struct assoc_array_ptr *new_meta[3]; - struct assoc_array_ptr *excised_meta[1]; + struct assoc_array_ptr *excised_meta[2]; struct assoc_array_ptr *excised_subtree; struct assoc_array_ptr **set_backpointers[ASSOC_ARRAY_FAN_OUT]; struct assoc_array_ptr *set_backpointers_to; diff --git a/include/linux/bio.h b/include/linux/bio.h index d4f58a2d15b6..104a68749ab1 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -377,6 +377,21 @@ static inline struct bio *bio_alloc(struct block_device *bdev, void submit_bio(struct bio *bio); +/** + * bio_in_atomic - check if the current context is unsafe for bio completion + * + * Return: %true in atomic contexts (e.g. hard/soft IRQ, preempt-disabled); + * %false when a bio can be safely completed in the current context. + */ +static inline bool bio_in_atomic(void) +{ + if (IS_ENABLED(CONFIG_PREEMPTION) && rcu_preempt_depth()) + return true; + if (!IS_ENABLED(CONFIG_PREEMPT_COUNT)) + return true; + return !preemptible(); +} + extern void bio_endio(struct bio *); static inline void bio_io_error(struct bio *bio) diff --git a/include/linux/bitfield.h b/include/linux/bitfield.h index 20978c60b565..c60554acc16d 100644 --- a/include/linux/bitfield.h +++ b/include/linux/bitfield.h @@ -8,6 +8,7 @@ #define _LINUX_BITFIELD_H #include +#include #include /* @@ -38,8 +39,7 @@ * FIELD_PREP(REG_FIELD_D, 0x40); * * Modify: - * reg &= ~REG_FIELD_C; - * reg |= FIELD_PREP(REG_FIELD_C, c); + * FIELD_MODIFY(REG_FIELD_C, ®, c); */ #define __bf_shf __builtin_ctzll @@ -156,6 +156,23 @@ (typeof(_mask))(((_reg) & (_mask)) >> __bf_shf(_mask)); \ }) +/** + * FIELD_MODIFY() - modify a bitfield element + * @_mask: shifted mask defining the field's length and position + * @_reg_p: pointer to the memory that should be updated + * @_val: value to store in the bitfield + * + * FIELD_MODIFY() modifies the set of bits in @_reg_p specified by @_mask, + * by replacing them with the bitfield value passed in as @_val. + */ +#define FIELD_MODIFY(_mask, _reg_p, _val) \ + ({ \ + typecheck_pointer(_reg_p); \ + __BF_FIELD_CHECK(_mask, *(_reg_p), _val, "FIELD_MODIFY: "); \ + *(_reg_p) &= ~(_mask); \ + *(_reg_p) |= (((typeof(_mask))(_val) << __bf_shf(_mask)) & (_mask)); \ + }) + extern void __compiletime_error("value doesn't fit into mask") __field_overflow(void); extern void __compiletime_error("bad bitfield mask") diff --git a/include/linux/can/can-ml.h b/include/linux/can/can-ml.h index 8afa92d15a66..f10444c6115f 100644 --- a/include/linux/can/can-ml.h +++ b/include/linux/can/can-ml.h @@ -45,6 +45,7 @@ #include #include #include +#include #define CAN_SFF_RCV_ARRAY_SZ (1 << CAN_SFF_ID_BITS) #define CAN_EFF_RCV_HASH_BITS 10 @@ -53,6 +54,7 @@ enum { RX_ERR, RX_ALL, RX_FIL, RX_INV, RX_MAX }; struct can_dev_rcv_lists { + spinlock_t lock; /* protects the receive lists and entries */ struct hlist_head rx[RX_MAX]; struct hlist_head rx_sff[CAN_SFF_RCV_ARRAY_SZ]; struct hlist_head rx_eff[CAN_EFF_RCV_ARRAY_SZ]; @@ -74,6 +76,7 @@ static inline struct can_ml_priv *can_get_ml_priv(struct net_device *dev) static inline void can_set_ml_priv(struct net_device *dev, struct can_ml_priv *ml_priv) { + spin_lock_init(&ml_priv->dev_rcv_lists.lock); netdev_set_ml_priv(dev, ml_priv, ML_PRIV_CAN); } diff --git a/include/linux/damon.h b/include/linux/damon.h index 4688a10eca89..b05c626f6479 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -641,6 +641,12 @@ struct damon_ctx { struct list_head adaptive_targets; struct list_head schemes; +/* private: */ + /* + * number of regions after last merge. Read kdamond_split_regions() + * for more details. + */ + unsigned long last_nr_regions; }; static inline struct damon_region *damon_next_region(struct damon_region *r) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 32cbaaaccfe5..96a201151a80 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -121,10 +121,9 @@ void hugepage_put_subpool(struct hugepage_subpool *spool); void hugetlb_dup_vma_private(struct vm_area_struct *vma); void clear_vma_resv_huge_pages(struct vm_area_struct *vma); -int move_hugetlb_page_tables(struct vm_area_struct *vma, - struct vm_area_struct *new_vma, - unsigned long old_addr, unsigned long new_addr, - unsigned long len); +unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma, + struct vm_area_struct *new_vma, unsigned long old_addr, + unsigned long new_addr, unsigned long len); int copy_hugetlb_page_range(struct mm_struct *, struct mm_struct *, struct vm_area_struct *, struct vm_area_struct *); void unmap_hugepage_range(struct vm_area_struct *, @@ -334,11 +333,9 @@ static inline int copy_hugetlb_page_range(struct mm_struct *dst, return 0; } -static inline int move_hugetlb_page_tables(struct vm_area_struct *vma, - struct vm_area_struct *new_vma, - unsigned long old_addr, - unsigned long new_addr, - unsigned long len) +static inline unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma, + struct vm_area_struct *new_vma, unsigned long old_addr, + unsigned long new_addr, unsigned long len) { BUG(); return 0; @@ -690,6 +687,7 @@ struct huge_bootmem_page { int isolate_or_dissolve_huge_page(struct page *page, struct list_head *list); void wait_for_freed_hugetlb_folios(void); +int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long end_pfn); struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, unsigned long addr, bool cow_from_owner); struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid, @@ -1081,6 +1079,12 @@ static inline void wait_for_freed_hugetlb_folios(void) { } +static inline int replace_free_hugepage_folios(unsigned long start_pfn, + unsigned long end_pfn) +{ + return 0; +} + static inline struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, unsigned long addr, bool cow_from_owner) diff --git a/include/linux/libata.h b/include/linux/libata.h index e5e098279803..c5ae2d98eb22 100644 --- a/include/linux/libata.h +++ b/include/linux/libata.h @@ -947,6 +947,13 @@ struct ata_port { */ #define ATA_OP_NULL (void *)(unsigned long)(-ENOENT) +struct ata_reset_operations { + ata_prereset_fn_t prereset; + ata_reset_fn_t softreset; + ata_reset_fn_t hardreset; + ata_postreset_fn_t postreset; +}; + struct ata_port_operations { /* * Command execution @@ -973,14 +980,8 @@ struct ata_port_operations { void (*freeze)(struct ata_port *ap); void (*thaw)(struct ata_port *ap); - ata_prereset_fn_t prereset; - ata_reset_fn_t softreset; - ata_reset_fn_t hardreset; - ata_postreset_fn_t postreset; - ata_prereset_fn_t pmp_prereset; - ata_reset_fn_t pmp_softreset; - ata_reset_fn_t pmp_hardreset; - ata_postreset_fn_t pmp_postreset; + struct ata_reset_operations reset; + struct ata_reset_operations pmp_reset; void (*error_handler)(struct ata_port *ap); void (*lost_interrupt)(struct ata_port *ap); void (*post_internal_cmd)(struct ata_queued_cmd *qc); @@ -1418,9 +1419,6 @@ extern void ata_eh_thaw_port(struct ata_port *ap); extern void ata_eh_qc_complete(struct ata_queued_cmd *qc); extern void ata_eh_qc_retry(struct ata_queued_cmd *qc); -extern void ata_do_eh(struct ata_port *ap, ata_prereset_fn_t prereset, - ata_reset_fn_t softreset, ata_reset_fn_t hardreset, - ata_postreset_fn_t postreset); extern void ata_std_error_handler(struct ata_port *ap); extern void ata_std_sched_eh(struct ata_port *ap); extern void ata_std_end_eh(struct ata_port *ap); @@ -2153,6 +2151,12 @@ static inline u8 ata_wait_idle(struct ata_port *ap) return status; } +#else /* CONFIG_ATA_SFF */ +static inline int sata_sff_hardreset(struct ata_link *link, unsigned int *class, + unsigned long deadline) +{ + return -EOPNOTSUPP; +} #endif /* CONFIG_ATA_SFF */ #endif /* __LINUX_LIBATA_H__ */ diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 8dab6c755e10..2aba72d1de3a 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1884,6 +1884,16 @@ static inline void count_objcg_events(struct obj_cgroup *objcg, #endif /* CONFIG_MEMCG */ +#if defined(CONFIG_MEMCG) && defined(CONFIG_LRU_GEN) +void mem_cgroup_calculate_protection_path(struct mem_cgroup *root, + struct mem_cgroup *memcg); +#else +static inline void mem_cgroup_calculate_protection_path(struct mem_cgroup *root, + struct mem_cgroup *memcg) +{ +} +#endif + #if defined(CONFIG_MEMCG) && defined(CONFIG_ZSWAP) bool obj_cgroup_may_zswap(struct obj_cgroup *objcg); void obj_cgroup_charge_zswap(struct obj_cgroup *objcg, size_t size); diff --git a/include/linux/memfd.h b/include/linux/memfd.h index 246daadbfde8..bb07ba299290 100644 --- a/include/linux/memfd.h +++ b/include/linux/memfd.h @@ -2,8 +2,17 @@ #ifndef __LINUX_MEMFD_H #define __LINUX_MEMFD_H +#include #include +static inline int memfd_check_seals_mode(unsigned int seals, umode_t old_mode, + umode_t new_mode) +{ + if ((seals & F_SEAL_EXEC) && ((old_mode ^ new_mode) & 0111)) + return -EPERM; + return 0; +} + #ifdef CONFIG_MEMFD_CREATE extern long memfd_fcntl(struct file *file, unsigned int cmd, unsigned int arg); struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx); diff --git a/include/linux/mm.h b/include/linux/mm.h index 1728dc335608..f05962963e28 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2058,11 +2058,13 @@ static inline long folio_nr_pages(const struct folio *folio) /* Only hugetlbfs can allocate folios larger than MAX_ORDER */ #ifdef CONFIG_ARCH_HAS_GIGANTIC_PAGE -#define MAX_FOLIO_NR_PAGES (1UL << PUD_ORDER) +#define MAX_FOLIO_ORDER PUD_ORDER #else -#define MAX_FOLIO_NR_PAGES MAX_ORDER_NR_PAGES +#define MAX_FOLIO_ORDER MAX_PAGE_ORDER #endif +#define MAX_FOLIO_NR_PAGES (1UL << MAX_FOLIO_ORDER) + /* * compound_nr() returns the number of pages in this potentially compound * page. compound_nr() can be called on a tail page, and is defined to diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index d39ebb10caeb..fe486d52d48e 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -444,7 +444,8 @@ mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) /* * This version of mmu_notifier_invalidate_range_start() avoids blocking, but it * can return an error if a notifier can't proceed without blocking, in which - * case you're not allowed to modify PTEs in the specified range. + * case you're not allowed to modify PTEs in the specified range or call + * mmu_notifier_invalidate_range_end(). * * This is mainly intended for OOM handling. */ diff --git a/include/linux/panic.h b/include/linux/panic.h index f0b779d2915e..99c53d2792b1 100644 --- a/include/linux/panic.h +++ b/include/linux/panic.h @@ -3,6 +3,7 @@ #define _LINUX_PANIC_H #include +#include #include struct pt_regs; @@ -10,7 +11,10 @@ struct pt_regs; extern long (*panic_blink)(int state); __printf(1, 2) void panic(const char *fmt, ...) __noreturn __cold; -void nmi_panic(struct pt_regs *regs, const char *msg); +__printf(1, 0) +void vpanic(const char *fmt, va_list args) __noreturn __cold; +__printf(2, 3) +void nmi_panic(struct pt_regs *regs, const char *fmt, ...); void check_panic_on_warn(const char *origin); extern void oops_enter(void); extern void oops_exit(void); diff --git a/include/linux/pci.h b/include/linux/pci.h index c65de9e8aa8a..63c24b5bdce5 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -602,6 +602,8 @@ struct pci_host_bridge { u8 (*swizzle_irq)(struct pci_dev *, u8 *); /* Platform IRQ swizzler */ int (*map_irq)(const struct pci_dev *, u8, u8); void (*release_fn)(struct pci_host_bridge *); + int (*enable_device)(struct pci_host_bridge *bridge, struct pci_dev *dev); + void (*disable_device)(struct pci_host_bridge *bridge, struct pci_dev *dev); void *release_data; unsigned int ignore_reset_delay:1; /* For entire hierarchy */ unsigned int no_ext_tags:1; /* No Extended Tags */ diff --git a/include/linux/power_supply.h b/include/linux/power_supply.h index b03991a29a65..1e059ff7a971 100644 --- a/include/linux/power_supply.h +++ b/include/linux/power_supply.h @@ -236,6 +236,8 @@ struct power_supply_config { char **supplied_to; size_t num_supplicants; + + bool no_wakeup_source; }; /* Description of power supply */ @@ -287,7 +289,6 @@ struct power_supply { char **supplied_from; size_t num_supplies; - struct device_node *of_node; /* Driver private data */ void *drv_data; @@ -890,6 +891,7 @@ extern int power_supply_powers(struct power_supply *psy, struct device *dev); extern void *power_supply_get_drvdata(struct power_supply *psy); extern int power_supply_for_each_device(void *data, int (*fn)(struct device *dev, void *data)); +extern int power_supply_for_each_psy(void *data, int (*fn)(struct power_supply *psy, void *data)); static inline bool power_supply_is_amp_property(enum power_supply_property psp) { diff --git a/include/linux/srcu.h b/include/linux/srcu.h index 835bbb2d1f88..4958b16d187e 100644 --- a/include/linux/srcu.h +++ b/include/linux/srcu.h @@ -18,6 +18,7 @@ #include #include +#include #include #include @@ -58,6 +59,41 @@ int __srcu_read_lock(struct srcu_struct *ssp) __acquires(ssp); void __srcu_read_unlock(struct srcu_struct *ssp, int idx) __releases(ssp); void synchronize_srcu(struct srcu_struct *ssp); +/** + * list_splice_init_srcu - splice an SRCU-protected list into an existing list + * @list: the SRCU-protected list to splice + * @head: the place in the existing list to splice the first list into + * @ssp: the srcu_struct that readers of @list use + * + * Like list_splice_init_rcu(), but wait for the SRCU readers of @list + * instead of RCU ones, so that none of them can follow a next pointer + * onto @head. + * + * Note that this function blocks, and that the caller must prevent any + * other updates to @list. + */ +static inline void list_splice_init_srcu(struct list_head *list, + struct list_head *head, + struct srcu_struct *ssp) +{ + struct list_head *first = list->next; + struct list_head *last = list->prev; + struct list_head *next = head->next; + + if (list_empty(list)) + return; + + INIT_LIST_HEAD_RCU(list); + synchronize_srcu(ssp); + ASSERT_EXCLUSIVE_ACCESS(*first); + ASSERT_EXCLUSIVE_ACCESS(*last); + + last->next = next; + rcu_assign_pointer(list_next_rcu(head), first); + first->prev = head; + next->prev = last; +} + #define SRCU_GET_STATE_COMPLETED 0x1 /** diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h index bd6ac060450c..acdb31039c35 100644 --- a/include/linux/sunrpc/clnt.h +++ b/include/linux/sunrpc/clnt.h @@ -195,7 +195,8 @@ struct rpc_xprt *rpc_task_get_xprt(struct rpc_clnt *clnt, int rpcb_create_local(struct net *); void rpcb_put_local(struct net *); -int rpcb_register(struct net *, u32, u32, int, unsigned short); +int rpcb_register(struct net *net, u32 prog, u32 vers, int prot, + unsigned short port); int rpcb_v4_register(struct net *net, const u32 program, const u32 version, const struct sockaddr *address, diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h index 79f3c5c1d0e9..b7a61aafc2dd 100644 --- a/include/linux/sunrpc/svc.h +++ b/include/linux/sunrpc/svc.h @@ -79,6 +79,7 @@ struct svc_serv { unsigned int sv_max_payload; /* datagram payload size */ unsigned int sv_max_mesg; /* max_payload + 1 page for overheads */ unsigned int sv_xdrsize; /* XDR buffer size */ + atomic_t sv_rpcb_failures; /* unanswered rpcbind calls */ struct list_head sv_permsocks; /* all permanent sockets */ struct list_head sv_tempsocks; /* all temporary sockets */ int sv_tmpcnt; /* count of temporary "valid" sockets */ @@ -451,6 +452,7 @@ int sunrpc_set_pool_mode(const char *val); int sunrpc_get_pool_mode(char *val, size_t size); void svc_rpcb_cleanup(struct svc_serv *serv, struct net *net); int svc_bind(struct svc_serv *serv, struct net *net); +unsigned int svc_rpcb_failure_count(struct svc_serv *serv); struct svc_serv *svc_create(struct svc_program *, unsigned int, int (*threadfn)(void *data)); bool svc_rqst_replace_page(struct svc_rqst *rqstp, @@ -466,8 +468,9 @@ int svc_set_num_threads(struct svc_serv *, struct svc_pool *, int); int svc_pool_stats_open(struct svc_info *si, struct file *file); void svc_process(struct svc_rqst *rqstp); void svc_process_bc(struct rpc_rqst *req, struct svc_rqst *rqstp); -int svc_register(const struct svc_serv *, struct net *, const int, - const unsigned short, const unsigned short); +int svc_register(struct svc_serv *serv, struct net *net, + const int family, const unsigned short proto, + const unsigned short port); void svc_wake_up(struct svc_serv *); void svc_reserve(struct svc_rqst *rqstp, int space); diff --git a/include/linux/usb/quirks.h b/include/linux/usb/quirks.h index b3cc7beab4a3..a4043b33c2c2 100644 --- a/include/linux/usb/quirks.h +++ b/include/linux/usb/quirks.h @@ -81,4 +81,7 @@ /* Device claims zero configurations, forcing to 1 */ #define USB_QUIRK_FORCE_ONE_CONFIG BIT(18) +/* Use a 255 bytes config descriptor request mirroring windows behavior */ +#define USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE BIT(19) + #endif /* __LINUX_USB_QUIRKS_H */ diff --git a/include/net/espintcp.h b/include/net/espintcp.h index c70efd704b6d..7cad3c6822df 100644 --- a/include/net/espintcp.h +++ b/include/net/espintcp.h @@ -34,7 +34,8 @@ static inline struct espintcp_ctx *espintcp_getctx(const struct sock *sk) { const struct inet_connection_sock *icsk = inet_csk(sk); - /* RCU is only needed for diag */ - return (__force void *)icsk->icsk_ulp_data; + /* pairs with smp_wmb() in espintcp_init_sk() */ + smp_rmb(); + return (__force void *)READ_ONCE(icsk->icsk_ulp_data); } #endif diff --git a/include/net/ip.h b/include/net/ip.h index 97c60b300270..f00719e1863f 100644 --- a/include/net/ip.h +++ b/include/net/ip.h @@ -515,31 +515,6 @@ static inline unsigned int ip_dst_mtu_maybe_forward(const struct dst_entry *dst, return res; } -/* Configured/administrative MTU of a route, for advertising the TCP MSS. - * - * Unlike ip_dst_mtu_maybe_forward(), this deliberately ignores the - * ICMP-learned path MTU (rt->rt_pmtu). The advertised MSS bounds what the - * peer may send to us and must reflect our receive capability (the device or - * route-configured MTU), not a path MTU learned on the reverse (send) - * direction, which may not apply to the peer->us path and outlives the fnhe - * for the whole connection. See RFC 2923 section 2.3 and the comment above - * tcp_advertise_mss(). - */ -static inline unsigned int ip_dst_mtu_configured(const struct dst_entry *dst) -{ - unsigned int mtu, res; - - rcu_read_lock(); - mtu = dst_metric_raw(dst, RTAX_MTU); - if (!mtu) - mtu = READ_ONCE(dst_dev_rcu(dst)->mtu); - mtu = min_t(unsigned int, mtu, IP_MAX_MTU); - res = mtu - lwtunnel_headroom(dst->lwtstate, mtu); - rcu_read_unlock(); - - return res; -} - static inline unsigned int ip_skb_dst_mtu(struct sock *sk, const struct sk_buff *skb) { diff --git a/include/net/ip6_route.h b/include/net/ip6_route.h index 8be44dc5b5b7..ad35c7512f12 100644 --- a/include/net/ip6_route.h +++ b/include/net/ip6_route.h @@ -354,43 +354,6 @@ static inline unsigned int ip6_dst_mtu_maybe_forward(const struct dst_entry *dst return mtu - lwtunnel_headroom(dst->lwtstate, mtu); } -/* Configured/administrative MTU of a route, for advertising the TCP MSS. - * - * Unlike ip6_dst_mtu_maybe_forward(), this ignores any ICMPv6-learned path - * MTU (which is kept on the RTF_CACHE exception route) and returns the MTU of - * the underlying route (fib6_pmtu) or the egress device. The advertised MSS - * bounds what the peer may send to us and must reflect our receive - * capability, not a path MTU learned on the reverse (send) direction. See - * RFC 2923 section 2.3 and the comment above tcp_advertise_mss(). - */ -static inline unsigned int ip6_dst_mtu_configured(const struct dst_entry *dst) -{ - const struct rt6_info *rt = dst_rt6_info(dst); - const struct fib6_info *from; - struct inet6_dev *idev; - unsigned int mtu = 0; - - rcu_read_lock(); - /* IPv6 keeps the learned PMTU and the configured MTU in the same - * RTAX_MTU slot: the learned value sits on this (possibly RTF_CACHE) - * dst, the configured one on the underlying route. Reach the latter - * via ->from (fib6_pmtu), populated by ip6_route_info_create(). - */ - from = rcu_dereference(rt->from); - if (from) - mtu = from->fib6_pmtu; - if (!mtu) { - mtu = IPV6_MIN_MTU; - idev = __in6_dev_get(dst_dev_rcu(dst)); - if (idev) - mtu = max_t(unsigned int, mtu, READ_ONCE(idev->cnf.mtu6)); - } - rcu_read_unlock(); - - mtu = min_t(unsigned int, mtu, IP6_MAX_MTU); - return mtu - lwtunnel_headroom(dst->lwtstate, mtu); -} - u32 ip6_mtu_from_fib6(const struct fib6_result *res, const struct in6_addr *daddr, const struct in6_addr *saddr); diff --git a/include/net/netns/can.h b/include/net/netns/can.h index 48b79f7e6236..54f23681e0d4 100644 --- a/include/net/netns/can.h +++ b/include/net/netns/can.h @@ -29,7 +29,7 @@ struct netns_can { /* receive filters subscribed for 'all' CAN devices */ struct can_dev_rcv_lists *rx_alldev_list; - spinlock_t rcvlists_lock; + spinlock_t rcvlists_lock; /* protects the rcv_entries(_max) stats */ struct timer_list stattimer; /* timer for statistics update */ struct can_pkg_stats *pkg_stats; struct can_rcv_lists_stats *rcv_lists_stats; diff --git a/include/net/netrom.h b/include/net/netrom.h index 2b4685d52bd8..3f6fd2f89434 100644 --- a/include/net/netrom.h +++ b/include/net/netrom.h @@ -87,6 +87,19 @@ struct nr_sock { #define nr_sk(sk) ((struct nr_sock *)(sk)) +/* + * Window negotiation, for a CONNREQ received or a CONNACK: use the smaller + * of our window and the peer's. A window of 0 would never let a frame + * out: take it as 1. + */ +static inline void nr_negotiate_window(struct nr_sock *nr, unsigned char window) +{ + if (window == 0) + window = 1; + if (window < nr->window) + nr->window = window; +} + struct nr_neigh { struct hlist_node neigh_node; ax25_address callsign; @@ -189,7 +202,7 @@ void nr_loopback_clear(void); int nr_loopback_queue(struct sk_buff *); /* nr_out.c */ -void nr_output(struct sock *, struct sk_buff *); +int nr_output(struct sock *sk, struct sk_buff *skb); void nr_send_nak_frame(struct sock *); void nr_kick(struct sock *); void nr_transmit_buffer(struct sock *, struct sk_buff *); diff --git a/include/net/rose.h b/include/net/rose.h index 2fa0331e36f0..30b0809dd0a6 100644 --- a/include/net/rose.h +++ b/include/net/rose.h @@ -231,6 +231,8 @@ int rose_rcv_frame(struct sock *sk, struct sk_buff *skb); int rose_backlog_rcv(struct sock *sk, struct sk_buff *skb); /* rose_link.c */ +void rose_neigh_init_timers(struct rose_neigh *neigh); +bool rose_neigh_clear_ax25(struct rose_neigh *neigh, ax25_cb *ax25); void rose_start_ftimer(struct rose_neigh *); void rose_stop_ftimer(struct rose_neigh *); void rose_stop_t0timer(struct rose_neigh *); @@ -266,6 +268,7 @@ void rose_link_device_down(struct net_device *); struct net_device *rose_dev_first(void); struct net_device *rose_dev_get(rose_address *); struct rose_route *rose_route_free_lci(unsigned int, struct rose_neigh *); +unsigned int rose_assign_lci(struct sock *sk); struct rose_neigh *rose_get_neigh(rose_address *, unsigned char *, unsigned char *, int); int rose_rt_ioctl(unsigned int, void __user *); diff --git a/include/net/sctp/sm.h b/include/net/sctp/sm.h index 64c42bd56bb2..84e71381e19b 100644 --- a/include/net/sctp/sm.h +++ b/include/net/sctp/sm.h @@ -253,6 +253,7 @@ struct sctp_chunk *sctp_make_fwdtsn(const struct sctp_association *asoc, struct sctp_fwdtsn_skip *skiplist); struct sctp_chunk *sctp_make_auth(const struct sctp_association *asoc, __u16 key_id); +void sctp_control_set_owner_w(struct sctp_chunk *chunk); struct sctp_chunk *sctp_make_strreset_req(const struct sctp_association *asoc, __u16 stream_num, __be16 *stream_list, bool out, bool in); diff --git a/include/sound/hwdep.h b/include/sound/hwdep.h index b0da633184cd..7357727f0660 100644 --- a/include/sound/hwdep.h +++ b/include/sound/hwdep.h @@ -57,6 +57,7 @@ struct snd_hwdep { struct mutex open_mutex; int used; /* reference counter */ + struct mutex dsp_mutex; /* serializes DSP load requests */ unsigned int dsp_loaded; /* bit fields of loaded dsp indices */ unsigned int exclusive:1; /* exclusive access mode */ }; diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h index 8c4d791507f1..5e29744e4503 100644 --- a/include/trace/events/rxrpc.h +++ b/include/trace/events/rxrpc.h @@ -20,10 +20,10 @@ /* AFS errors */ \ EM(afs_abort_general_error, "afs-error") \ EM(afs_abort_interrupted, "afs-intr") \ - EM(afs_abort_oom, "afs-oom") \ EM(afs_abort_op_not_supported, "afs-op-notsupp") \ EM(afs_abort_probeuuid_negative, "afs-probeuuid-neg") \ EM(afs_abort_send_data_error, "afs-send-data") \ + EM(afs_abort_send_error, "afs-send-error") \ EM(afs_abort_unmarshal_error, "afs-unmarshal") \ /* rxperf errors */ \ EM(rxperf_abort_general_error, "rxperf-error") \ diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 41048271a066..e245fa295662 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -621,6 +621,12 @@ enum iommufd_hwpt_get_dirty_bitmap_flags { * Walk the IOMMU pagetables for a given IOVA range to return a bitmap * with the dirty IOVAs. In doing so it will also by default clear any * dirty bit metadata set in the IOPTE. + * + * The IOVA range given must fully enclose any IOPTE. If the range starts or + * ends inside a dirty IOPTE then the ioctl will fail. This avoids partially + * reporting a dirty range. + * + * On failure the dirty state becomes unknown and the userspace must restart. */ struct iommu_hwpt_get_dirty_bitmap { __u32 size; diff --git a/io_uring/cancel.c b/io_uring/cancel.c index 13d5d4f2924f..727166a2ba3e 100644 --- a/io_uring/cancel.c +++ b/io_uring/cancel.c @@ -189,7 +189,8 @@ static int __io_async_cancel(struct io_cancel_data *cd, break; if (!all) return ret; - nr++; + /* poll, futex and waitid return how many they cancelled */ + nr += max(ret, 1); } while (1); /* slow path, try all io-wq's */ diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 52759ee3bd6c..09551e3907f9 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -3282,7 +3282,16 @@ __cold void io_uring_cancel_generic(bool cancel_all, struct io_sq_data *sqd) } if (loop) { + /* + * A request that runs in io-wq and doesn't end makes + * each round come here. Don't hold sqd->lock for that + * long either. + */ + if (sqd) + mutex_unlock(&sqd->lock); cond_resched(); + if (sqd) + mutex_lock(&sqd->lock); continue; } @@ -3300,8 +3309,20 @@ __cold void io_uring_cancel_generic(bool cancel_all, struct io_sq_data *sqd) * avoids a race where a completion comes in before we did * prepare_to_wait(). */ - if (inflight == tctx_inflight(tctx, !cancel_all)) + if (inflight == tctx_inflight(tctx, !cancel_all)) { + /* + * A zero-copy notification can't be canceled, and a + * peer that doesn't read keeps it pending. Don't hold + * sqd->lock while waiting for it, or whoever wants the + * lock sleeps as long. No ring is used across this, + * the next round walks ->ctx_list anew. + */ + if (sqd) + mutex_unlock(&sqd->lock); schedule(); + if (sqd) + mutex_lock(&sqd->lock); + } end_wait: finish_wait(&tctx->wait, &wait); } while (1); diff --git a/io_uring/poll.c b/io_uring/poll.c index 31454c27072b..c1a9f697a023 100644 --- a/io_uring/poll.c +++ b/io_uring/poll.c @@ -169,12 +169,24 @@ static void io_poll_tw_hash_eject(struct io_kiocb *req, struct io_tw_state *ts) } } +#define IO_POLL_UNMASK (EPOLLERR|EPOLLHUP|EPOLLNVAL) + +static __poll_t io_poll_unmask(__poll_t events) +{ + /* + * A peer that stopped sending is an event for a reader, but a write + * only poll must stay armed until the file is writable again. + */ + if (events & (EPOLLIN | EPOLLRDNORM | EPOLLRDBAND)) + return IO_POLL_UNMASK | EPOLLRDHUP; + return IO_POLL_UNMASK; +} + static void io_init_poll_iocb(struct io_poll *poll, __poll_t events) { poll->head = NULL; -#define IO_POLL_UNMASK (EPOLLERR|EPOLLHUP|EPOLLNVAL|EPOLLRDHUP) /* mask in events that we always want/need */ - poll->events = events | IO_POLL_UNMASK; + poll->events = events | io_poll_unmask(events); INIT_LIST_HEAD(&poll->wait.entry); init_waitqueue_func_entry(&poll->wait, io_poll_wake); } @@ -745,7 +757,7 @@ int io_arm_poll_handler(struct io_kiocb *req, unsigned issue_flags) const struct io_issue_def *def = &io_issue_defs[req->opcode]; struct async_poll *apoll; struct io_poll_table ipt; - __poll_t mask = POLLPRI | POLLERR | EPOLLET; + __poll_t mask = POLLERR | EPOLLET; int ret; /* @@ -762,7 +774,8 @@ int io_arm_poll_handler(struct io_kiocb *req, unsigned issue_flags) mask |= EPOLLONESHOT; if (def->pollin) { - mask |= EPOLLIN | EPOLLRDNORM; + /* urgent data is an event for a reader only */ + mask |= EPOLLIN | EPOLLRDNORM | EPOLLPRI; /* If reading from MSG_ERRQUEUE using recvmsg, ignore POLLIN */ if (req->flags & REQ_F_CLEAR_POLLIN) @@ -892,14 +905,52 @@ static int io_poll_disarm(struct io_kiocb *req) return 0; } +/* + * Cancel every request matching @cd in one pass. Cancelled requests stay + * hashed until their task work runs, so finding them one at a time would + * walk past all the earlier ones on each lookup. + */ +static int io_poll_cancel_all(struct io_ring_ctx *ctx, struct io_cancel_data *cd, + struct io_hash_table *table) +{ + unsigned int nr_buckets = 1U << table->hash_bits; + unsigned int i = 0; + struct hlist_node *tmp; + struct io_kiocb *req; + int nr = 0; + + if (!(cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP | + IORING_ASYNC_CANCEL_ANY))) { + /* keyed by user_data, so only one bucket can match */ + i = hash_long(cd->data, table->hash_bits); + nr_buckets = i + 1; + } + + for (; i < nr_buckets; i++) { + struct io_hash_bucket *hb = &table->hbs[i]; + + spin_lock(&hb->lock); + hlist_for_each_entry_safe(req, tmp, &hb->list, hash_node) { + if (!io_cancel_req_match(req, cd)) + continue; + io_poll_cancel_req(req); + nr++; + } + spin_unlock(&hb->lock); + } + return nr; +} + static int __io_poll_cancel(struct io_ring_ctx *ctx, struct io_cancel_data *cd, struct io_hash_table *table) { struct io_hash_bucket *bucket; struct io_kiocb *req; - if (cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP | - IORING_ASYNC_CANCEL_ANY)) + if (cd->flags & (IORING_ASYNC_CANCEL_ALL | IORING_ASYNC_CANCEL_ANY)) + return io_poll_cancel_all(ctx, cd, table) ?: -ENOENT; + + if (cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP)) req = io_poll_file_find(ctx, cd, table, &bucket); else req = io_poll_find(ctx, false, cd, table, &bucket); @@ -914,16 +965,22 @@ static int __io_poll_cancel(struct io_ring_ctx *ctx, struct io_cancel_data *cd, int io_poll_cancel(struct io_ring_ctx *ctx, struct io_cancel_data *cd, unsigned issue_flags) { - int ret; + bool all = cd->flags & (IORING_ASYNC_CANCEL_ALL | IORING_ASYNC_CANCEL_ANY); + int ret, nr; ret = __io_poll_cancel(ctx, cd, &ctx->cancel_table); - if (ret != -ENOENT) + if (ret != -ENOENT && !all) return ret; + nr = max(ret, 0); io_ring_submit_lock(ctx, issue_flags); ret = __io_poll_cancel(ctx, cd, &ctx->cancel_table_locked); io_ring_submit_unlock(ctx, issue_flags); - return ret; + if (!all) + return ret; + /* with ALL or ANY, both tables return how many they cancelled */ + nr += max(ret, 0); + return nr ?: -ENOENT; } static __poll_t io_poll_parse_events(const struct io_uring_sqe *sqe, @@ -1068,7 +1125,7 @@ int io_poll_remove(struct io_kiocb *req, unsigned int issue_flags) poll->events &= ~0xffff; poll->events |= poll_update->events & 0xffff; - poll->events |= IO_POLL_UNMASK; + poll->events |= io_poll_unmask(poll->events); } if (poll_update->update_user_data) preq->cqe.user_data = poll_update->new_user_data; diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index ad7ad35545d6..168c6987435e 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -562,11 +562,12 @@ static int io_files_update_with_index_alloc(struct io_kiocb *req, { struct io_rsrc_update *up = io_kiocb_to_cmd(req, struct io_rsrc_update); __s32 __user *fds = u64_to_user_ptr(up->arg); + struct io_ring_ctx *ctx = req->ctx; unsigned int done; struct file *file; int ret, fd; - if (!req->ctx->file_data) + if (!ctx->file_data) return -ENXIO; for (done = 0; done < up->nr_args; done++) { @@ -580,15 +581,25 @@ static int io_files_update_with_index_alloc(struct io_kiocb *req, ret = -EBADF; break; } - ret = io_fixed_fd_install(req, issue_flags, file, - IORING_FILE_INDEX_ALLOC); - if (ret < 0) + /* + * Hold the lock across the copy_to_user(), if it's dropped the + * slot can be closed and reused before the error path runs. + */ + io_ring_submit_lock(ctx, issue_flags); + ret = __io_fixed_fd_install(ctx, file, + IORING_FILE_INDEX_ALLOC); + if (ret < 0) { + io_ring_submit_unlock(ctx, issue_flags); + fput(file); break; + } if (copy_to_user(&fds[done], &ret, sizeof(ret))) { - __io_close_fixed(req->ctx, issue_flags, ret); + io_fixed_fd_remove(ctx, ret); + io_ring_submit_unlock(ctx, issue_flags); ret = -EFAULT; break; } + io_ring_submit_unlock(ctx, issue_flags); } if (done) diff --git a/io_uring/sqpoll.c b/io_uring/sqpoll.c index 696aba6bbf81..130da8f8f67b 100644 --- a/io_uring/sqpoll.c +++ b/io_uring/sqpoll.c @@ -26,6 +26,7 @@ enum { IO_SQ_THREAD_SHOULD_STOP = 0, IO_SQ_THREAD_SHOULD_PARK, + IO_SQ_THREAD_EXITING, }; void io_sq_thread_unpark(struct io_sq_data *sqd) @@ -354,6 +355,18 @@ static int io_sq_thread(void *data) timeout = jiffies + sqd->sq_thread_idle; } + /* + * The SQPOLL thread never returns to userspace, so do the + * return-to-userspace work, like memory.high reclaim, here. + * It can sleep for a long time, so drop sqd->lock. + */ + if (test_thread_flag(TIF_NOTIFY_RESUME)) { + mutex_unlock(&sqd->lock); + resume_user_mode_work(NULL); + mutex_lock(&sqd->lock); + sqd->sq_cpu = raw_smp_processor_id(); + } + cap_entries = !list_is_singular(&sqd->ctx_list); list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) { int ret = __io_sq_thread(ctx, sqd, cap_entries, &ist); @@ -435,6 +448,13 @@ static int io_sq_thread(void *data) if (retry_list) io_sq_tw(&retry_list, UINT_MAX); + /* + * Cancelation drops sqd->lock while it waits, and a ring could attach + * to a thread that is on its way out. A fatal signal doesn't set the + * stop bit, so mark the thread as exiting here for + * io_sq_offload_create() to see. + */ + set_bit(IO_SQ_THREAD_EXITING, &sqd->state); io_uring_cancel_generic(true, sqd); rcu_assign_pointer(sqd->thread, NULL); put_task_struct(current); @@ -503,7 +523,11 @@ __cold int io_sq_offload_create(struct io_ring_ctx *ctx, list_add(&ctx->sqd_list, &sqd->ctx_list); io_sqd_update_thread_idle(sqd); /* don't attach to a dying SQPOLL thread, would be racy */ - ret = (attached && !sqd->thread) ? -ENXIO : 0; + ret = 0; + if (attached && (!sqd->thread || + test_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state) || + test_bit(IO_SQ_THREAD_EXITING, &sqd->state))) + ret = -ENXIO; io_sq_thread_unpark(sqd); if (ret < 0) diff --git a/kernel/events/core.c b/kernel/events/core.c index d14cf23a16a6..21a54cedd854 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -6916,6 +6916,15 @@ static void perf_sigtrap(struct perf_event *event) if (event->ctx->task == TASK_TOMBSTONE) return; + /* + * The event was removed after this signal was queued, e.g. by exec() + * (remove_on_exec), which may also have closed its fd (close-on-exec). + * The new program has default signal handlers, so a SIGTRAP would + * kill it. + */ + if (READ_ONCE(event->state) <= PERF_EVENT_STATE_EXIT) + return; + /* * We'd expect this to only occur if the irq_work is delayed and either * ctx->task or current has changed in the meantime. This can be the diff --git a/kernel/irq/internals.h b/kernel/irq/internals.h index 7062a99a1ddd..0b60f8aa6286 100644 --- a/kernel/irq/internals.h +++ b/kernel/irq/internals.h @@ -77,6 +77,7 @@ enum { #include "settings.h" extern int __irq_set_trigger(struct irq_desc *desc, unsigned long flags); +extern int __irq_set_irqchip_state(struct irq_desc *desc, enum irqchip_irq_state which, bool val); extern void __disable_irq(struct irq_desc *desc); extern void __enable_irq(struct irq_desc *desc); diff --git a/kernel/irq/kexec.c b/kernel/irq/kexec.c index 0f9548c1708d..58d92a7f2af1 100644 --- a/kernel/irq/kexec.c +++ b/kernel/irq/kexec.c @@ -25,7 +25,7 @@ void machine_kexec_mask_interrupts(void) * First try to remove the active state from an interrupt which is forwarded * to a VM. If the interrupt is not forwarded, try to EOI the interrupt. */ - check_eoi = irq_set_irqchip_state(i, IRQCHIP_STATE_ACTIVE, false); + check_eoi = __irq_set_irqchip_state(desc, IRQCHIP_STATE_ACTIVE, false); } if (check_eoi && chip->irq_eoi && irqd_irq_inprogress(&desc->irq_data)) diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c index 67b1c5a35201..927a6e52a58c 100644 --- a/kernel/irq/manage.c +++ b/kernel/irq/manage.c @@ -2867,6 +2867,28 @@ int irq_get_irqchip_state(unsigned int irq, enum irqchip_irq_state which, } EXPORT_SYMBOL_GPL(irq_get_irqchip_state); +int __irq_set_irqchip_state(struct irq_desc *desc, enum irqchip_irq_state which, bool val) +{ + struct irq_data *data = irq_desc_get_irq_data(desc); + struct irq_chip *chip; + + do { + chip = irq_data_get_irq_chip(data); + + if (WARN_ON_ONCE(!chip)) + return -ENODEV; + + if (chip->irq_set_irqchip_state) + break; + + data = irqd_get_parent_data(data); + } while (data); + + if (data) + return chip->irq_set_irqchip_state(data, which, val); + return -EINVAL; +} + /** * irq_set_irqchip_state - set the state of a forwarded interrupt. * @irq: Interrupt line that is forwarded to a VM @@ -2883,36 +2905,15 @@ int irq_set_irqchip_state(unsigned int irq, enum irqchip_irq_state which, bool val) { struct irq_desc *desc; - struct irq_data *data; - struct irq_chip *chip; unsigned long flags; - int err = -EINVAL; + int err; desc = irq_get_desc_buslock(irq, &flags, 0); if (!desc) - return err; - - data = irq_desc_get_irq_data(desc); - - do { - chip = irq_data_get_irq_chip(data); - if (WARN_ON_ONCE(!chip)) { - err = -ENODEV; - goto out_unlock; - } - if (chip->irq_set_irqchip_state) - break; -#ifdef CONFIG_IRQ_DOMAIN_HIERARCHY - data = data->parent_data; -#else - data = NULL; -#endif - } while (data); + return -EINVAL; - if (data) - err = chip->irq_set_irqchip_state(data, which, val); + err = __irq_set_irqchip_state(desc, which, val); -out_unlock: irq_put_desc_busunlock(desc, flags); return err; } diff --git a/kernel/panic.c b/kernel/panic.c index 97c472164393..0bc445155035 100644 --- a/kernel/panic.c +++ b/kernel/panic.c @@ -246,18 +246,19 @@ EXPORT_SYMBOL(panic_on_other_cpu); * nmi_panic_self_stop() which can provide architecture dependent code such * as saving register state for crash dump. */ -void nmi_panic(struct pt_regs *regs, const char *msg) +void nmi_panic(struct pt_regs *regs, const char *fmt, ...) { - int old_cpu, this_cpu; + va_list args; - old_cpu = PANIC_CPU_INVALID; - this_cpu = raw_smp_processor_id(); + va_start(args, fmt); - /* atomic_try_cmpxchg updates old_cpu on failure */ - if (atomic_try_cmpxchg(&panic_cpu, &old_cpu, this_cpu)) - panic("%s", msg); - else if (old_cpu != this_cpu) + if (panic_try_start()) + vpanic(fmt, args); + + if (panic_on_other_cpu()) nmi_panic_self_stop(regs); + + va_end(args); } EXPORT_SYMBOL(nmi_panic); @@ -330,17 +331,15 @@ static void panic_other_cpus_shutdown(bool crash_kexec) } /** - * panic - halt the system - * @fmt: The text string to print + * panic - halt the system + * @fmt: The text string to print + * @args: Arguments for the format string * - * Display a message, then perform cleanups. - * - * This function never returns. + * Display a message, then perform cleanups. This function never returns. */ -void panic(const char *fmt, ...) +void vpanic(const char *fmt, va_list args) { static char buf[1024]; - va_list args; long i, i_next = 0, len; int state = 0; int old_cpu, this_cpu; @@ -391,9 +390,7 @@ void panic(const char *fmt, ...) console_verbose(); bust_spinlocks(1); - va_start(args, fmt); len = vscnprintf(buf, sizeof(buf), fmt, args); - va_end(args); if (len && buf[len - 1] == '\n') buf[len - 1] = '\0'; @@ -530,7 +527,17 @@ void panic(const char *fmt, ...) mdelay(PANIC_TIMER_STEP); } } +EXPORT_SYMBOL(vpanic); + +/* Identical to vpanic(), except it takes variadic arguments instead of va_list */ +void panic(const char *fmt, ...) +{ + va_list args; + va_start(args, fmt); + vpanic(fmt, args); + va_end(args); +} EXPORT_SYMBOL(panic); #define TAINT_FLAG(taint, _c_true, _c_false, _module) \ diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index b18c3b4763b7..7a4b23e48cc6 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -2173,7 +2173,11 @@ static void ops_dequeue(struct task_struct *p, u64 deq_flags) { unsigned long opss; - /* dequeue is always temporary, don't reset runnable_at */ + /* + * Don't reset runnable_at. If @p is waiting, the wait continues across + * temporary dequeues, e.g. for a migration or an attribute change. + * scx_ops_disable_task() ends it if @p leaves its scheduler. + */ clr_task_runnable(p, false); /* acquire ensures that we see the preceding updates on QUEUED */ @@ -3794,6 +3798,14 @@ static void scx_ops_disable_task(struct task_struct *p) if (SCX_HAS_OP(disable)) SCX_CALL_OP_TASK(SCX_KF_REST, disable, p); scx_set_task_state(p, SCX_TASK_READY); + + /* + * If @p is runnable but not running, its wait started at runnable_at, + * which ops_dequeue() didn't reset. Leaving the BPF scheduler ends the + * wait. Reset runnable_at the next time @p is enqueued on SCX, so that + * the watchdog doesn't count the time @p spends elsewhere. + */ + p->scx.flags |= SCX_TASK_RESET_RUNNABLE_AT; } static void scx_ops_exit_task(struct task_struct *p) diff --git a/kernel/time/posix-stubs.c b/kernel/time/posix-stubs.c index 9b6fcb8d85e7..5a27335a917e 100644 --- a/kernel/time/posix-stubs.c +++ b/kernel/time/posix-stubs.c @@ -83,7 +83,7 @@ SYSCALL_DEFINE2(clock_getres, const clockid_t, which_clock, struct __kernel_time case CLOCK_REALTIME: case CLOCK_MONOTONIC: case CLOCK_BOOTTIME: - if (put_timespec64(&rtn_tp, tp)) + if (tp && put_timespec64(&rtn_tp, tp)) return -EFAULT; return 0; default: @@ -166,7 +166,7 @@ SYSCALL_DEFINE2(clock_getres_time32, clockid_t, which_clock, case CLOCK_REALTIME: case CLOCK_MONOTONIC: case CLOCK_BOOTTIME: - if (put_old_timespec32(&rtn_tp, tp)) + if (tp && put_old_timespec32(&rtn_tp, tp)) return -EFAULT; return 0; default: diff --git a/kernel/time/posix-timers.c b/kernel/time/posix-timers.c index c5d9bfbb082b..1e7f01c3daaa 100644 --- a/kernel/time/posix-timers.c +++ b/kernel/time/posix-timers.c @@ -656,16 +656,8 @@ void common_timer_get(struct k_itimer *timr, struct itimerspec64 *cur_setting) if (iv) { cur_setting->it_interval = ktime_to_timespec64(iv); } else if (!timr->it_active) { - /* - * SIGEV_NONE oneshot timers are never queued and therefore - * timr->it_active is always false. The check below - * vs. remaining time will handle this case. - * - * For all other timers there is nothing to update here, so - * return. - */ - if (!sig_none) - return; + /* Nothing to update for a disarmed timer */ + return; } now = kc->clock_get_ktime(timr->it_clock); @@ -916,7 +908,15 @@ int common_timer_set(struct k_itimer *timr, int flags, sigev_none = timr->it_sigev_notify == SIGEV_NONE; kc->timer_arm(timr, expires, flags & TIMER_ABSTIME, sigev_none); - timr->it_active = !sigev_none; + /* + * SIGEV_NONE timers are not queued, but mark them armed + * so that common_timer_get() can tell them apart from + * disarmed timers. An expired SIGEV_NONE timer stays + * armed, which is harmless: common_timer_get() reports + * zero remaining time for it and it never reaches the + * signal delivery code. + */ + timr->it_active = 1; return 0; } diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 88fcd439e93d..2d4872092561 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -385,9 +385,21 @@ static bool can_stop_full_tick(int cpu, struct tick_sched *ts) return true; } +static void tick_nohz_full_update_tick(struct tick_sched *ts); + static void nohz_full_kick_func(struct irq_work *work) { - /* Empty, the tick restart happens on tick_nohz_irq_exit() */ + struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); + + if (WARN_ON_ONCE(!tick_nohz_full_cpu(smp_processor_id()))) + return; + + /* Task-context flushes may not be followed by an IRQ exit. */ + if (!in_hardirq()) { + if (WARN_ON_ONCE(tick_sched_flag_test(ts, TS_FLAG_INIDLE))) + return; + tick_nohz_full_update_tick(ts); + } } static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) = diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c index 171f3a13cd6b..aa9aca76e76e 100644 --- a/kernel/trace/trace_events_hist.c +++ b/kernel/trace/trace_events_hist.c @@ -1747,6 +1747,9 @@ static bool expr_field_str(struct hist_field *field, struct seq_buf *s) seq_buf_printf(s, ".%s", flags_str); } + if (field->buckets) + seq_buf_printf(s, "=%lu", field->buckets); + return !seq_buf_has_overflowed(s); } diff --git a/lib/assoc_array.c b/lib/assoc_array.c index 01619e88a52c..d3ee08fa2ab5 100644 --- a/lib/assoc_array.c +++ b/lib/assoc_array.c @@ -470,33 +470,32 @@ static bool assoc_array_insert_in_empty_tree(struct assoc_array_edit *edit) } /* - * Handle insertion into a terminal node. + * The pointer that leads to a node or shortcut with the given back pointer + * and parent slot: the root, a slot in a node, or a shortcut's next node. */ -static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, - const struct assoc_array_ops *ops, - const void *index_key, - struct assoc_array_walk_result *result) +static struct assoc_array_ptr **assoc_array_parent_link(struct assoc_array *array, + struct assoc_array_ptr *back_pointer, + int parent_slot) { - struct assoc_array_shortcut *shortcut, *new_s0; - struct assoc_array_node *node, *new_n0, *new_n1, *side; - struct assoc_array_ptr *ptr; - unsigned long dissimilarity, base_seg, blank; - size_t keylen; - bool have_meta; - int level, diff; - int slot, next_slot, free_slot, i, j; - - node = result->terminal_node.node; - level = result->terminal_node.level; - edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = result->terminal_node.slot; + if (!back_pointer) + return &array->root; + if (assoc_array_ptr_is_node(back_pointer)) + return &assoc_array_ptr_to_node(back_pointer)->slots[parent_slot]; + return &assoc_array_ptr_to_shortcut(back_pointer)->next_node; +} - pr_devel("-->%s()\n", __func__); +/* + * Insert into a terminal node without changing its shape, if that can be + * done: in place of a leaf with the same index key, or in a free slot. + */ +static bool assoc_array_insert_in_node(struct assoc_array_edit *edit, + const struct assoc_array_ops *ops, + const void *index_key, + struct assoc_array_node *node) +{ + struct assoc_array_ptr *ptr; + int free_slot, i; - /* We arrived at a node which doesn't have an onward node or shortcut - * pointer that we have to follow. This means that (a) the leaf we - * want must go here (either by insertion or replacement) or (b) we - * need to split this node and insert in one of the fragments. - */ free_slot = -1; /* Firstly, we have to check the leaves in this node to see if there's @@ -514,7 +513,6 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, pr_devel("replace in slot %d\n", i); edit->leaf_p = &node->slots[i]; edit->dead_leaf = node->slots[i]; - pr_devel("<--%s() = ok [replace]\n", __func__); return true; } } @@ -526,28 +524,28 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, pr_devel("insert in free slot %d\n", free_slot); edit->leaf_p = &node->slots[free_slot]; edit->adjust_count_on = node; - pr_devel("<--%s() = ok [insert]\n", __func__); return true; } - /* The node has no spare slots - so we're either going to have to split - * it or insert another node before it. - * - * Whatever, we're going to need at least two new nodes - so allocate - * those now. We may also need a new shortcut, but we deal with that - * when we need it. - */ - new_n0 = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL); - if (!new_n0) - return false; - edit->new_meta[0] = assoc_array_node_to_ptr(new_n0); - new_n1 = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL); - if (!new_n1) - return false; - edit->new_meta[1] = assoc_array_node_to_ptr(new_n1); + return false; +} + +/* + * Fill in the edit script's segment cache for a full node: the segment at + * this level of each leaf's index key, or 0xff for a metadata pointer. + * The new key's segment, in the last entry, is left to the caller. Returns + * true if there are any metadata pointers. + */ +static bool assoc_array_fill_segment_cache(struct assoc_array_edit *edit, + const struct assoc_array_ops *ops, + const struct assoc_array_node *node, + int level) +{ + struct assoc_array_ptr *ptr; + unsigned long base_seg; + bool have_meta; + int i; - /* We need to find out how similar the leaves are. */ - pr_devel("no spare slots\n"); have_meta = false; for (i = 0; i < ASSOC_ARRAY_FAN_OUT; i++) { ptr = node->slots[i]; @@ -561,65 +559,42 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK; edit->segment_cache[i] = base_seg & ASSOC_ARRAY_FAN_MASK; } + return have_meta; +} - if (have_meta) { - pr_devel("have meta\n"); - goto split_node; - } - - /* The node contains only leaves */ - dissimilarity = 0; - base_seg = edit->segment_cache[0]; - for (i = 1; i < ASSOC_ARRAY_FAN_OUT; i++) - dissimilarity |= edit->segment_cache[i] ^ base_seg; - - pr_devel("only leaves; dissimilarity=%lx\n", dissimilarity); +/* + * Whether the leaves of a full node and the new key all share one segment, + * so that splitting the node cannot make room. + */ +static bool assoc_array_all_one_segment(const struct assoc_array_edit *edit) +{ + int i; - if ((dissimilarity & ASSOC_ARRAY_FAN_MASK) == 0) { - /* The old leaves all cluster in the same slot. We will need - * to insert a shortcut if the new node wants to cluster with them. - */ - if ((edit->segment_cache[ASSOC_ARRAY_FAN_OUT] ^ base_seg) == 0) - goto all_leaves_cluster_together; + for (i = 1; i < ASSOC_ARRAY_FAN_OUT + 1; i++) + if (edit->segment_cache[i] != edit->segment_cache[0]) + return false; + return true; +} - /* Otherwise all the old leaves cluster in the same slot, but - * the new leaf wants to go into a different slot - so we - * create a new node (n0) to hold the new leaf and a pointer to - * a new node (n1) holding all the old leaves. - * - * This can be done by falling through to the node splitting - * path. - */ - pr_devel("present leaves cluster but not new leaf\n"); - } +/* + * Split a full node into new_n0, which takes its place, and new_n1, below + * new_n0 in the slot of a segment that at least two of the leaves share, + * the new one included. The caller has filled in the segment cache and + * set new_n0's back pointer and parent slot. + */ +static void assoc_array_split_node(struct assoc_array_edit *edit, + struct assoc_array_node *node, + struct assoc_array_node *new_n0, + struct assoc_array_node *new_n1) +{ + struct assoc_array_shortcut *shortcut; + struct assoc_array_node *side; + struct assoc_array_ptr *ptr; + int slot, next_slot, free_slot, i, j; -split_node: pr_devel("split node\n"); - /* We need to split the current node. The node must contain anything - * from a single leaf (in the one leaf case, this leaf will cluster - * with the new leaf) and the rest meta-pointers, to all leaves, some - * of which may cluster. - * - * It won't contain the case in which all the current leaves plus the - * new leaves want to cluster in the same slot. - * - * We need to expel at least two leaves out of a set consisting of the - * leaves in the node and the new leaf. The current meta pointers can - * just be copied as they shouldn't cluster with any of the leaves. - * - * We need a new node (n0) to replace the current one and a new node to - * take the expelled nodes (n1). - */ - edit->set[0].to = assoc_array_node_to_ptr(new_n0); - new_n0->back_pointer = node->back_pointer; - new_n0->parent_slot = node->parent_slot; new_n1->back_pointer = assoc_array_node_to_ptr(new_n0); - new_n1->parent_slot = -1; /* Need to calculate this */ - -do_split_node: - pr_devel("do_split_node\n"); - new_n0->nr_leaves_on_branch = node->nr_leaves_on_branch; new_n1->nr_leaves_on_branch = 0; @@ -699,26 +674,29 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, } } } +} - ptr = node->back_pointer; - if (!ptr) - edit->set[0].ptr = &edit->array->root; - else if (assoc_array_ptr_is_node(ptr)) - edit->set[0].ptr = &assoc_array_ptr_to_node(ptr)->slots[node->parent_slot]; - else - edit->set[0].ptr = &assoc_array_ptr_to_shortcut(ptr)->next_node; - edit->excised_meta[0] = assoc_array_node_to_ptr(node); - pr_devel("<--%s() = ok [split node]\n", __func__); - return true; +/* + * All the leaves, new and old, want to cluster together in this node in the + * same slot, so we have to replace this node with a shortcut to skip over + * the identical parts of the key, leading to new_n0. If the node is the + * target of a shortcut, the new one replaces that shortcut as well. The + * caller then splits the leaves between new_n0 and new_n1 at the shortcut's + * far end, for which this fills in the segment cache. + */ +static struct assoc_array_shortcut * +assoc_array_insert_shortcut(struct assoc_array_edit *edit, + const struct assoc_array_ops *ops, + const void *index_key, + struct assoc_array_node *node, int level, + struct assoc_array_node *new_n0) +{ + struct assoc_array_shortcut *new_s0; + unsigned long base_seg, blank; + size_t keylen; + int diff, i; -all_leaves_cluster_together: - /* All the leaves, new and old, want to cluster together in this node - * in the same slot, so we have to replace this node with a shortcut to - * skip over the identical parts of the key and then place a pair of - * nodes, one inside the other, at the end of the shortcut and - * distribute the keys between them. - * - * Firstly we need to work out where the leaves start diverging as a + /* Firstly we need to work out where the leaves start diverging as a * bit position into their keys so that we know how big the shortcut * needs to be. * @@ -744,17 +722,28 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, new_s0 = kzalloc(struct_size(new_s0, index_key, keylen), GFP_KERNEL); if (!new_s0) - return false; + return NULL; edit->new_meta[2] = assoc_array_shortcut_to_ptr(new_s0); - edit->set[0].to = assoc_array_shortcut_to_ptr(new_s0); new_s0->back_pointer = node->back_pointer; new_s0->parent_slot = node->parent_slot; + + /* A shortcut has to lead to a node, so a node at the end of a shortcut + * is replaced together with that shortcut. The new one holds the index + * key from the root, so it covers what the old one did. + */ + if (node->back_pointer && + assoc_array_ptr_is_shortcut(node->back_pointer)) { + struct assoc_array_shortcut *above = + assoc_array_ptr_to_shortcut(node->back_pointer); + + new_s0->back_pointer = above->back_pointer; + new_s0->parent_slot = above->parent_slot; + edit->excised_meta[1] = node->back_pointer; + } new_s0->next_node = assoc_array_node_to_ptr(new_n0); new_n0->back_pointer = assoc_array_shortcut_to_ptr(new_s0); new_n0->parent_slot = 0; - new_n1->back_pointer = assoc_array_node_to_ptr(new_n0); - new_n1->parent_slot = -1; /* Need to calculate this */ new_s0->skip_to_level = level = diff & ~ASSOC_ARRAY_LEVEL_STEP_MASK; pr_devel("skip_to_level = %d [diff %d]\n", level, diff); @@ -773,18 +762,104 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, /* This now reduces to a node splitting exercise for which we'll need * to regenerate the disparity table. */ - for (i = 0; i < ASSOC_ARRAY_FAN_OUT; i++) { - ptr = node->slots[i]; - base_seg = ops->get_object_key_chunk(assoc_array_ptr_to_leaf(ptr), - level); - base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK; - edit->segment_cache[i] = base_seg & ASSOC_ARRAY_FAN_MASK; - } - + assoc_array_fill_segment_cache(edit, ops, node, level); base_seg = ops->get_key_chunk(index_key, level); base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK; edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = base_seg & ASSOC_ARRAY_FAN_MASK; - goto do_split_node; + return new_s0; +} + +/* + * Handle insertion into a terminal node. + */ +static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit, + const struct assoc_array_ops *ops, + const void *index_key, + struct assoc_array_walk_result *result) +{ + struct assoc_array_shortcut *new_s0; + struct assoc_array_node *node, *new_n0, *new_n1; + bool have_meta; + int level; + + node = result->terminal_node.node; + level = result->terminal_node.level; + + pr_devel("-->%s()\n", __func__); + + /* We arrived at a node which doesn't have an onward node or shortcut + * pointer that we have to follow. This means that (a) the leaf we + * want must go here (either by insertion or replacement) or (b) we + * need to split this node and insert in one of the fragments. + */ + if (assoc_array_insert_in_node(edit, ops, index_key, node)) { + pr_devel("<--%s() = ok [in node]\n", __func__); + return true; + } + + /* The node has no spare slots - so we're either going to have to split + * it or insert another node before it. + * + * Whatever, we're going to need at least two new nodes - so allocate + * those now. We may also need a new shortcut, but we deal with that + * when we need it. + */ + new_n0 = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL); + if (!new_n0) + return false; + edit->new_meta[0] = assoc_array_node_to_ptr(new_n0); + new_n1 = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL); + if (!new_n1) + return false; + edit->new_meta[1] = assoc_array_node_to_ptr(new_n1); + + /* We need to find out how similar the leaves are. */ + pr_devel("no spare slots\n"); + have_meta = assoc_array_fill_segment_cache(edit, ops, node, level); + edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = result->terminal_node.slot; + + if (!have_meta && assoc_array_all_one_segment(edit)) { + /* The node only contains leaves and all the leaves cluster in + * the same slot, so we need to insert a shortcut to skip the + * common portion of the key and split the node at the + * shortcut's far end. The shortcut goes wherever its back + * pointer says, which may be further up than the node. + */ + new_s0 = assoc_array_insert_shortcut(edit, ops, index_key, + node, level, new_n0); + if (!new_s0) + return false; + edit->set[0].ptr = assoc_array_parent_link(edit->array, + new_s0->back_pointer, + new_s0->parent_slot); + edit->set[0].to = assoc_array_shortcut_to_ptr(new_s0); + } else { + /* We need to split the current node. The node must contain + * anything from a single leaf (in the one leaf case, this leaf + * will cluster with the new leaf) and the rest meta-pointers, + * to all leaves, some of which may cluster. + * + * We need to expel at least two leaves out of a set consisting + * of the leaves in the node and the new leaf. The current meta + * pointers can just be copied as they shouldn't cluster with + * any of the leaves. If all the old leaves cluster in one slot + * and the new leaf does not, they all go and the new leaf stays. + * + * We need a new node (n0) to replace the current one and a new + * node to take the expelled nodes (n1). + */ + new_n0->back_pointer = node->back_pointer; + new_n0->parent_slot = node->parent_slot; + edit->set[0].ptr = assoc_array_parent_link(edit->array, + new_n0->back_pointer, + new_n0->parent_slot); + edit->set[0].to = assoc_array_node_to_ptr(new_n0); + } + + assoc_array_split_node(edit, node, new_n0, new_n1); + edit->excised_meta[0] = assoc_array_node_to_ptr(node); + pr_devel("<--%s() = ok [split node]\n", __func__); + return true; } /* @@ -843,7 +918,8 @@ static bool assoc_array_insert_mid_shortcut(struct assoc_array_edit *edit, * zero length - otherwise we just connect the new node directly to the * parent. */ - level += ASSOC_ARRAY_LEVEL_STEP; + if (shortcut->back_pointer) + level += ASSOC_ARRAY_LEVEL_STEP; if (diff > level) { pr_devel("pre-shortcut %d...%d\n", level, diff); keylen = round_up(diff, ASSOC_ARRAY_KEY_CHUNK_SIZE); @@ -866,9 +942,11 @@ static bool assoc_array_insert_mid_shortcut(struct assoc_array_edit *edit, memcpy(new_s0->index_key, shortcut->index_key, flex_array_size(new_s0, index_key, keylen)); - blank = ULONG_MAX << (diff & ASSOC_ARRAY_KEY_CHUNK_MASK); - pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, diff, blank); - new_s0->index_key[keylen - 1] &= ~blank; + if (diff & ASSOC_ARRAY_KEY_CHUNK_MASK) { + blank = ULONG_MAX << (diff & ASSOC_ARRAY_KEY_CHUNK_MASK); + pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, diff, blank); + new_s0->index_key[keylen - 1] &= ~blank; + } } else { pr_devel("no pre-shortcut\n"); edit->set[0].to = assoc_array_node_to_ptr(new_n0); @@ -1484,17 +1562,26 @@ int assoc_array_gc(struct assoc_array *array, cursor = array->root; descend: - /* If this point is a shortcut, then we need to duplicate it and - * advance the target cursor. + /* The node at this position is duplicated, and so is the shortcut in + * front of it if there is one. The node is allocated first: the copy + * of the shortcut starts out pointing at the old node, and must not be + * in the new tree if the node cannot be allocated, or the cleanup + * would follow it into the old tree. */ + new_n = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL); + if (!new_n) + goto enomem; + if (assoc_array_ptr_is_shortcut(cursor)) { shortcut = assoc_array_ptr_to_shortcut(cursor); keylen = round_up(shortcut->skip_to_level, ASSOC_ARRAY_KEY_CHUNK_SIZE); keylen >>= ASSOC_ARRAY_KEY_CHUNK_SHIFT; new_s = kmalloc(struct_size(new_s, index_key, keylen), GFP_KERNEL); - if (!new_s) + if (!new_s) { + kfree(new_n); goto enomem; + } pr_devel("dup shortcut %p -> %p\n", shortcut, new_s); memcpy(new_s, shortcut, struct_size(new_s, index_key, keylen)); new_s->back_pointer = new_parent; @@ -1504,11 +1591,7 @@ int assoc_array_gc(struct assoc_array *array, cursor = shortcut->next_node; } - /* Duplicate the node at this position */ node = assoc_array_ptr_to_node(cursor); - new_n = kzalloc(sizeof(struct assoc_array_node), GFP_KERNEL); - if (!new_n) - goto enomem; pr_devel("dup node %p -> %p\n", node, new_n); new_n->back_pointer = new_parent; new_n->parent_slot = node->parent_slot; diff --git a/lib/cmdline.c b/lib/cmdline.c index 90ed997d9570..f5dc59b3548e 100644 --- a/lib/cmdline.c +++ b/lib/cmdline.c @@ -11,6 +11,7 @@ #include #include +#include #include #include @@ -26,7 +27,9 @@ static int get_range(char **str, int *pint, int n) (*str)++; upper_range = simple_strtol((*str), NULL, 0); - inc_counter = upper_range - *pint; + /* Keep the sign of the result when the difference doesn't fit */ + if (check_sub_overflow(upper_range, *pint, &inc_counter)) + inc_counter = upper_range < *pint ? -1 : INT_MAX; for (x = *pint; n && x < upper_range; x++, n--) *pint++ = x; return inc_counter; @@ -122,6 +125,14 @@ char *get_options(const char *str, int nints, int *ints) range_nums = get_range((char **)&str, pint, n); if (range_nums < 0) break; + /* The range didn't fit, so the array is full */ + if (!validate && range_nums > n) { + i = nints; + break; + } + /* Leave room for the upper number of the range */ + if (range_nums >= INT_MAX - i) + break; /* * Decrement the result by one to leave out the * last number in the range. The next iteration diff --git a/lib/cmdline_kunit.c b/lib/cmdline_kunit.c index c1602f797637..74c4b5ea3225 100644 --- a/lib/cmdline_kunit.c +++ b/lib/cmdline_kunit.c @@ -139,11 +139,112 @@ static void cmdline_test_range(struct kunit *test) } while (++i < ARRAY_SIZE(cmdline_test_range_strings)); } +static const struct { + const char *in; + int parsed[4]; + int validated; +} cmdline_test_range_overflow_cases[] = { + { "1-100", { 3, 1, 2, 3, }, 100, }, + { "1,5-100,7", { 3, 1, 5, 6, }, 98, }, + { "1-2147483646", { 3, 1, 2, 3, }, 2147483646, }, + { "0-2147483647", { 3, 0, 1, 2, }, 0, }, + { "-5-2147483647", { 3, -5, -4, -3, }, 0, }, + { "2147483647--5", { 0, 2147483647, }, 0, }, + { "1,0-2147483647", { 3, 1, 0, 1, }, 1, }, +}; + +static void cmdline_test_range_overflow(struct kunit *test) +{ + unsigned int i, j; + + for (i = 0; i < ARRAY_SIZE(cmdline_test_range_overflow_cases); i++) { + const char *in = cmdline_test_range_overflow_cases[i].in; + const int *e = cmdline_test_range_overflow_cases[i].parsed; + /* Two guard elements past the array handed to get_options() */ + int r[ARRAY_SIZE(cmdline_test_range_overflow_cases[0].parsed) + 2]; + int n; + + memset(r, 0, sizeof(r)); + get_options(in, ARRAY_SIZE(r) - 2, r); + for (j = 0; j < ARRAY_SIZE(r) - 2; j++) + KUNIT_EXPECT_EQ_MSG(test, r[j], e[j], "Pattern: %s at %u", in, j); + for (; j < ARRAY_SIZE(r); j++) + KUNIT_EXPECT_EQ_MSG(test, r[j], 0, "Pattern: %s out of bound at %u", in, j); + + get_options(in, 0, &n); + KUNIT_EXPECT_EQ_MSG(test, n, cmdline_test_range_overflow_cases[i].validated, + "Pattern: %s (validated)", in); + } +} + +static void cmdline_test_next_arg_quoted_value(struct kunit *test) +{ + char in[] = "foo=\"bar baz\" qux=1"; + char *next, *param, *val; + + next = next_arg(in, ¶m, &val); + KUNIT_EXPECT_STREQ(test, param, "foo"); + KUNIT_ASSERT_NOT_NULL(test, val); + KUNIT_EXPECT_STREQ(test, val, "bar baz"); + KUNIT_EXPECT_STREQ(test, next, "qux=1"); + + next = next_arg(next, ¶m, &val); + KUNIT_EXPECT_STREQ(test, param, "qux"); + KUNIT_ASSERT_NOT_NULL(test, val); + KUNIT_EXPECT_STREQ(test, val, "1"); + KUNIT_EXPECT_STREQ(test, next, ""); +} + +static void cmdline_test_next_arg_bare_quote_regression(struct kunit *test) +{ + char in[] = "foo=bar \""; + char *next, *param, *val; + + next = next_arg(in, ¶m, &val); + KUNIT_EXPECT_STREQ(test, param, "foo"); + KUNIT_ASSERT_NOT_NULL(test, val); + KUNIT_EXPECT_STREQ(test, val, "bar"); + KUNIT_EXPECT_STREQ(test, next, "\""); + + /* This hits the i == 0 quoted-token case fixed by 9847f21225c4. */ + next = next_arg(next, ¶m, &val); + KUNIT_EXPECT_STREQ(test, param, ""); + KUNIT_EXPECT_PTR_EQ(test, val, NULL); + KUNIT_EXPECT_STREQ(test, next, ""); +} + +static void cmdline_test_next_arg_mixed_tokens(struct kunit *test) +{ + char in[] = "bbb= jjj kkk=\"a=b\""; + char *next, *param, *val; + + next = next_arg(in, ¶m, &val); + KUNIT_EXPECT_STREQ(test, param, "bbb"); + KUNIT_ASSERT_NOT_NULL(test, val); + KUNIT_EXPECT_STREQ(test, val, ""); + KUNIT_EXPECT_STREQ(test, next, "jjj kkk=\"a=b\""); + + next = next_arg(next, ¶m, &val); + KUNIT_EXPECT_STREQ(test, param, "jjj"); + KUNIT_EXPECT_NULL(test, val); + KUNIT_EXPECT_STREQ(test, next, "kkk=\"a=b\""); + + next = next_arg(next, ¶m, &val); + KUNIT_EXPECT_STREQ(test, param, "kkk"); + KUNIT_ASSERT_NOT_NULL(test, val); + KUNIT_EXPECT_STREQ(test, val, "a=b"); + KUNIT_EXPECT_STREQ(test, next, ""); +} + static struct kunit_case cmdline_test_cases[] = { KUNIT_CASE(cmdline_test_noint), KUNIT_CASE(cmdline_test_lead_int), KUNIT_CASE(cmdline_test_tail_int), KUNIT_CASE(cmdline_test_range), + KUNIT_CASE(cmdline_test_range_overflow), + KUNIT_CASE(cmdline_test_next_arg_quoted_value), + KUNIT_CASE(cmdline_test_next_arg_bare_quote_regression), + KUNIT_CASE(cmdline_test_next_arg_mixed_tokens), {} }; diff --git a/lib/raid6/avx2.c b/lib/raid6/avx2.c index 059024234dce..2d03ce97f655 100644 --- a/lib/raid6/avx2.c +++ b/lib/raid6/avx2.c @@ -76,6 +76,7 @@ static void raid6_avx21_gen_syndrome(int disks, size_t bytes, void **ptrs) } asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -124,6 +125,7 @@ static void raid6_avx21_xor_syndrome(int disks, int start, int stop, } asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -186,6 +188,7 @@ static void raid6_avx22_gen_syndrome(int disks, size_t bytes, void **ptrs) } asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -254,6 +257,7 @@ static void raid6_avx22_xor_syndrome(int disks, int start, int stop, } asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -347,6 +351,7 @@ static void raid6_avx24_gen_syndrome(int disks, size_t bytes, void **ptrs) } asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -457,6 +462,7 @@ static void raid6_avx24_xor_syndrome(int disks, int start, int stop, asm volatile("vmovntdq %%ymm14,%0" : "=m" (q[d+96])); } asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } diff --git a/lib/raid6/avx512.c b/lib/raid6/avx512.c index 9c3e822e1adf..30956989e7d7 100644 --- a/lib/raid6/avx512.c +++ b/lib/raid6/avx512.c @@ -95,6 +95,7 @@ static void raid6_avx5121_gen_syndrome(int disks, size_t bytes, void **ptrs) } asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -154,6 +155,7 @@ static void raid6_avx5121_xor_syndrome(int disks, int start, int stop, } asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -227,6 +229,7 @@ static void raid6_avx5122_gen_syndrome(int disks, size_t bytes, void **ptrs) } asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -311,6 +314,7 @@ static void raid6_avx5122_xor_syndrome(int disks, int start, int stop, } asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -417,6 +421,7 @@ static void raid6_avx5124_gen_syndrome(int disks, size_t bytes, void **ptrs) } asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -550,6 +555,7 @@ static void raid6_avx5124_xor_syndrome(int disks, int start, int stop, "m" (q[d+128]), "m" (q[d+192])); } asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } const struct raid6_calls raid6_avx512x4 = { diff --git a/lib/raid6/recov_avx2.c b/lib/raid6/recov_avx2.c index 4e8095403ee2..a7fd1ba177f1 100644 --- a/lib/raid6/recov_avx2.c +++ b/lib/raid6/recov_avx2.c @@ -180,6 +180,7 @@ static void raid6_2data_recov_avx2(int disks, size_t bytes, int faila, #endif } + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -297,6 +298,7 @@ static void raid6_datap_recov_avx2(int disks, size_t bytes, int faila, #endif } + asm volatile("vzeroupper"); kernel_fpu_end(); } diff --git a/lib/raid6/recov_avx512.c b/lib/raid6/recov_avx512.c index fd9e15bf3f30..c8a152d9ab6d 100644 --- a/lib/raid6/recov_avx512.c +++ b/lib/raid6/recov_avx512.c @@ -221,6 +221,7 @@ static void raid6_2data_recov_avx512(int disks, size_t bytes, int faila, #endif } + asm volatile("vzeroupper"); kernel_fpu_end(); } @@ -363,6 +364,7 @@ static void raid6_datap_recov_avx512(int disks, size_t bytes, int faila, #endif } + asm volatile("vzeroupper"); kernel_fpu_end(); } diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 245d5b416699..1f1d63d21f26 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -273,7 +273,7 @@ static bool depot_init_pool(void **prealloc) * NULL; do not reset to NULL if we have reached the maximum number of * pools. */ - if (pools_num < DEPOT_MAX_POOLS) + if (pools_num + 1 < DEPOT_MAX_POOLS) WRITE_ONCE(new_pool, NULL); else WRITE_ONCE(new_pool, STACK_DEPOT_POISON); diff --git a/mm/damon/core.c b/mm/damon/core.c index 3142776e2e0e..e41d201d4b8f 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -324,6 +324,8 @@ struct damos_quota_goal *damos_new_quota_goal( return NULL; goal->metric = metric; goal->target_value = target_value; + if (metric == DAMOS_QUOTA_SOME_MEM_PSI_US) + goal->last_psi_total = U64_MAX; INIT_LIST_HEAD(&goal->list); return goal; } @@ -707,7 +709,8 @@ static int damos_commit_quota_goal( dst->target_value = src->target_value; if (dst->metric == DAMOS_QUOTA_USER_INPUT) dst->current_value = src->current_value; - /* keep last_psi_total as is, since it will be updated in next cycle */ + else if (dst->metric == DAMOS_QUOTA_SOME_MEM_PSI_US) + dst->last_psi_total = U64_MAX; return 0; } @@ -1274,7 +1277,7 @@ static void kdamond_reset_aggregated(struct damon_ctx *c) } } -static void damon_split_region_at(struct damon_target *t, +static int damon_split_region_at(struct damon_target *t, struct damon_region *r, unsigned long sz_r); static bool __damos_valid_target(struct damon_region *r, struct damos *s) @@ -1377,13 +1380,14 @@ static void damos_update_stat(struct damos *s, } static bool __damos_filter_out(struct damon_ctx *ctx, struct damon_target *t, - struct damon_region *r, struct damos_filter *filter) + struct damon_region *r, struct damos_filter *filter, int *err) { bool matched = false; struct damon_target *ti; int target_idx = 0; unsigned long start, end; + *err = 0; switch (filter->type) { case DAMOS_FILTER_TYPE_TARGET: damon_for_each_target(ti, ctx) { @@ -1409,12 +1413,13 @@ static bool __damos_filter_out(struct damon_ctx *ctx, struct damon_target *t, } /* start before the range and overlap */ if (r->ar.start < start) { - damon_split_region_at(t, r, start - r->ar.start); + *err = damon_split_region_at(t, r, + start - r->ar.start); matched = false; break; } /* start inside the range */ - damon_split_region_at(t, r, end - r->ar.start); + *err = damon_split_region_at(t, r, end - r->ar.start); matched = true; break; default: @@ -1428,9 +1433,12 @@ static bool damos_filter_out(struct damon_ctx *ctx, struct damon_target *t, struct damon_region *r, struct damos *s) { struct damos_filter *filter; + bool matching; + int err; damos_for_each_filter(filter, s) { - if (__damos_filter_out(ctx, t, r, filter)) + matching = __damos_filter_out(ctx, t, r, filter, &err); + if (matching || err) return true; } return false; @@ -1478,7 +1486,8 @@ static void damos_apply_scheme(struct damon_ctx *c, struct damon_target *t, DAMON_MIN_REGION); if (!sz) goto update_stat; - damon_split_region_at(t, r, sz); + if (damon_split_region_at(t, r, sz)) + goto update_stat; } if (damos_filter_out(c, t, r, s)) return; @@ -1616,7 +1625,12 @@ static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal) break; case DAMOS_QUOTA_SOME_MEM_PSI_US: now_psi_total = damos_get_some_mem_psi_total(); - goal->current_value = now_psi_total - goal->last_psi_total; + /* uninitialized last_psi_total; make no effect this round */ + if (goal->last_psi_total == U64_MAX) + goal->current_value = goal->target_value; + else + goal->current_value = now_psi_total - + goal->last_psi_total; goal->last_psi_total = now_psi_total; break; default: @@ -1681,6 +1695,26 @@ static void damos_set_effective_quota(struct damos_quota *quota) quota->esz = esz; } +static void damos_fix_stale_charge_pointers(struct damon_ctx *c, + struct damos *s) +{ + struct damos_quota *quota; + struct damon_target *t; + + quota = &s->quota; + if (!quota->charge_target_from) + return; + damon_for_each_target(t, c) { + if (t == quota->charge_target_from) { + if (damon_nr_regions(t) > 0) + return; + break; + } + } + quota->charge_target_from = NULL; + quota->charge_addr_from = 0; +} + static void damos_adjust_quota(struct damon_ctx *c, struct damos *s) { struct damos_quota *quota = &s->quota; @@ -1709,6 +1743,8 @@ static void damos_adjust_quota(struct damon_ctx *c, struct damos *s) damos_set_effective_quota(quota); } + damos_fix_stale_charge_pointers(c, s); + if (!c->ops.get_scheme_score) return; @@ -1870,15 +1906,17 @@ static void kdamond_merge_regions(struct damon_ctx *c, unsigned int threshold, * * r the region to be split * sz_r size of the first sub-region that will be made + * + * Return: 0 on success, negative error code otherwise. */ -static void damon_split_region_at(struct damon_target *t, +static int damon_split_region_at(struct damon_target *t, struct damon_region *r, unsigned long sz_r) { struct damon_region *new; new = damon_new_region(r->ar.start + sz_r, r->ar.end); if (!new) - return; + return -ENOMEM; r->ar.end = new->ar.start; @@ -1888,6 +1926,7 @@ static void damon_split_region_at(struct damon_target *t, new->nr_accesses = r->nr_accesses; damon_insert_region(new, r, damon_next_region(r), t); + return 0; } /* Split every region in the given target into 'nr_subs' regions */ @@ -1932,7 +1971,6 @@ static void kdamond_split_regions(struct damon_ctx *ctx) { struct damon_target *t; unsigned int nr_regions = 0; - static unsigned int last_nr_regions; int nr_subregions = 2; damon_for_each_target(t, ctx) @@ -1942,14 +1980,14 @@ static void kdamond_split_regions(struct damon_ctx *ctx) return; /* Maybe the middle of the region has different access frequency */ - if (last_nr_regions == nr_regions && + if (ctx->last_nr_regions == nr_regions && nr_regions < ctx->attrs.max_nr_regions / 3) nr_subregions = 3; damon_for_each_target(t, ctx) damon_split_regions_of(t, nr_subregions); - last_nr_regions = nr_regions; + ctx->last_nr_regions = nr_regions; } /* diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index a6b1de97b1e8..6bac269cd679 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -273,7 +273,7 @@ static unsigned long damon_pa_pageout(struct damon_region *r, struct damos *s) else list_add(&folio->lru, &folio_list); put_folio: - addr += folio_size(folio); + addr = PFN_PHYS(folio_pfn(folio)) + folio_size(folio); folio_put(folio); } if (install_young_filter) @@ -306,7 +306,7 @@ static inline unsigned long damon_pa_mark_accessed_or_deactivate( folio_deactivate(folio); applied += folio_nr_pages(folio); put_folio: - addr += folio_size(folio); + addr = PFN_PHYS(folio_pfn(folio)) + folio_size(folio); folio_put(folio); } return applied * PAGE_SIZE; @@ -483,7 +483,7 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s) folio_is_file_lru(folio)); list_add(&folio->lru, &folio_list); put_folio: - addr += folio_size(folio); + addr = PFN_PHYS(folio_pfn(folio)) + folio_size(folio); folio_put(folio); } applied = damon_pa_migrate_pages(&folio_list, s->target_nid); diff --git a/mm/damon/tests/core-kunit.h b/mm/damon/tests/core-kunit.h index ff9f73579f7b..6b7cd8149a0e 100644 --- a/mm/damon/tests/core-kunit.h +++ b/mm/damon/tests/core-kunit.h @@ -527,6 +527,7 @@ static void damos_test_filter_out(struct kunit *test) struct damon_target *t; struct damon_region *r, *r2; struct damos_filter *f; + int err; f = damos_new_filter(DAMOS_FILTER_TYPE_ADDR, true); f->addr_range = (struct damon_addr_range){ @@ -537,25 +538,25 @@ static void damos_test_filter_out(struct kunit *test) damon_add_region(r, t); /* region in the range */ - KUNIT_EXPECT_TRUE(test, __damos_filter_out(NULL, t, r, f)); + KUNIT_EXPECT_TRUE(test, __damos_filter_out(NULL, t, r, f, &err)); KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 1); /* region before the range */ r->ar.start = DAMON_MIN_REGION * 1; r->ar.end = DAMON_MIN_REGION * 2; - KUNIT_EXPECT_FALSE(test, __damos_filter_out(NULL, t, r, f)); + KUNIT_EXPECT_FALSE(test, __damos_filter_out(NULL, t, r, f, &err)); KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 1); /* region after the range */ r->ar.start = DAMON_MIN_REGION * 6; r->ar.end = DAMON_MIN_REGION * 8; - KUNIT_EXPECT_FALSE(test, __damos_filter_out(NULL, t, r, f)); + KUNIT_EXPECT_FALSE(test, __damos_filter_out(NULL, t, r, f, &err)); KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 1); /* region started before the range */ r->ar.start = DAMON_MIN_REGION * 1; r->ar.end = DAMON_MIN_REGION * 4; - KUNIT_EXPECT_FALSE(test, __damos_filter_out(NULL, t, r, f)); + KUNIT_EXPECT_FALSE(test, __damos_filter_out(NULL, t, r, f, &err)); /* filter should have split the region */ KUNIT_EXPECT_EQ(test, r->ar.start, DAMON_MIN_REGION * 1); KUNIT_EXPECT_EQ(test, r->ar.end, DAMON_MIN_REGION * 2); @@ -570,7 +571,7 @@ static void damos_test_filter_out(struct kunit *test) /* region started in the range */ r->ar.start = DAMON_MIN_REGION * 2; r->ar.end = DAMON_MIN_REGION * 8; - KUNIT_EXPECT_TRUE(test, __damos_filter_out(NULL, t, r, f)); + KUNIT_EXPECT_TRUE(test, __damos_filter_out(NULL, t, r, f, &err)); /* filter should have split the region */ KUNIT_EXPECT_EQ(test, r->ar.start, DAMON_MIN_REGION * 2); KUNIT_EXPECT_EQ(test, r->ar.end, DAMON_MIN_REGION * 6); diff --git a/mm/filemap.c b/mm/filemap.c index 64c2325b9238..602106b329bc 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1346,10 +1346,22 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, * return value based on that state without races. * * Also note that WQ_FLAG_WOKEN is sufficient for a non-exclusive - * waiter, but an exclusive one requires WQ_FLAG_DONE. + * waiter, but an exclusive one must also acquire the bit. */ - if (behavior == EXCLUSIVE) - return wait->flags & WQ_FLAG_DONE ? 0 : -EINTR; + if (behavior == EXCLUSIVE) { + if (wait->flags & WQ_FLAG_DONE) + return 0; + /* + * We might have just been woken up and the waker will not + * wake up anybody else. If we ignore that now, the remaining + * waiters may not get woken up: take the bit now if it is + * still free. + */ + if ((wait->flags & WQ_FLAG_WOKEN) && + !test_and_set_bit(bit_nr, folio_flags(folio, 0))) + return 0; + return -EINTR; + } return wait->flags & WQ_FLAG_WOKEN ? 0 : -EINTR; } @@ -3096,11 +3108,11 @@ loff_t mapping_seek_hole_data(struct address_space *mapping, loff_t start, * @folio - the folio to lock. * @fpin - the pointer to the file we may pin (or is already pinned). * - * This works similar to lock_folio_or_retry in that it can drop the - * mmap_lock. It differs in that it actually returns the folio locked - * if it returns 1 and 0 if it couldn't lock the folio. If we did have - * to drop the mmap_lock then fpin will point to the pinned file and - * needs to be fput()'ed at a later point. + * Returns 1 with the folio locked, or 0 if locking failed. + * + * On both entry and return, a non-NULL *fpin means the fault lock has + * already been dropped and the caller must eventually fput() the file. + * Otherwise the fault lock is still held, including on failure. */ static int lock_folio_maybe_drop_mmap(struct vm_fault *vmf, struct folio *folio, struct file **fpin) @@ -3117,21 +3129,9 @@ static int lock_folio_maybe_drop_mmap(struct vm_fault *vmf, struct folio *folio, return 0; *fpin = maybe_unlock_mmap_for_io(vmf, *fpin); - if (vmf->flags & FAULT_FLAG_KILLABLE) { - if (__folio_lock_killable(folio)) { - /* - * We didn't have the right flags to drop the - * fault lock, but all fault_handlers only check - * for fatal signals if we return VM_FAULT_RETRY, - * so we need to drop the fault lock here and - * return 0 if we don't have a fpin. - */ - if (*fpin == NULL) - release_fault_lock(vmf); - return 0; - } - } else - __folio_lock(folio); + if (vmf->flags & FAULT_FLAG_KILLABLE) + return !__folio_lock_killable(folio); + __folio_lock(folio); return 1; } @@ -3293,7 +3293,7 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf) * vma->vm_mm->mmap_lock must be held on entry. * * If our return value has VM_FAULT_RETRY set, it's because the mmap_lock - * may be dropped before doing I/O or by lock_folio_maybe_drop_mmap(). + * may be dropped before doing I/O or when aborting an interrupted wait. * * If our return value does not have VM_FAULT_RETRY set, the mmap_lock * has not been released. @@ -3449,9 +3449,8 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) out_retry: /* - * We dropped the mmap_lock, we need to return to the fault handler to - * re-find the vma and come back and find our hopefully still populated - * page. + * With fpin set, the fault lock was already dropped. Otherwise it is + * still held. Release invalidate_lock before dropping either protection. */ if (!IS_ERR(folio)) folio_put(folio); @@ -3459,6 +3458,8 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) filemap_invalidate_unlock_shared(mapping); if (fpin) fput(fpin); + else if (!(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) + release_fault_lock(vmf); return ret | VM_FAULT_RETRY; } EXPORT_SYMBOL(filemap_fault); diff --git a/mm/gup.c b/mm/gup.c index 810a4c32ee87..d9107eb461a0 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -3448,6 +3448,9 @@ static int gup_fast_fallback(unsigned long start, unsigned long nr_pages, FOLL_PCI_P2PDMA | FOLL_HONOR_NUMA_FAULT))) return -EINVAL; + if (unlikely(!nr_pages)) + return 0; + if (gup_flags & FOLL_PIN) mm_set_has_pinned_flag(¤t->mm->flags); @@ -3455,8 +3458,8 @@ static int gup_fast_fallback(unsigned long start, unsigned long nr_pages, might_lock_read(¤t->mm->mmap_lock); start = untagged_addr(start) & PAGE_MASK; - len = nr_pages << PAGE_SHIFT; - if (check_add_overflow(start, len, &end)) + if (check_shl_overflow(nr_pages, PAGE_SHIFT, &len) || + check_add_overflow(start, len, &end)) return -EOVERFLOW; if (end > TASK_SIZE_MAX) return -EFAULT; @@ -3703,7 +3706,7 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end, { unsigned int flags, nr_folios, nr_found; unsigned int i, pgshift = PAGE_SHIFT; - pgoff_t start_idx, end_idx, next_idx; + pgoff_t start_idx, end_idx, next_idx, retry_idx; struct folio *folio = NULL; struct folio_batch fbatch; struct hstate *h; @@ -3744,6 +3747,7 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end, * reason, we try to allocate them and add them to the * page cache. */ + retry_idx = start_idx; nr_found = filemap_get_folios_contig(memfd->f_mapping, &start_idx, end_idx, @@ -3768,12 +3772,38 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end, folio = page_folio(&fbatch.folios[i]->page); + /* + * The folio might have been removed from the + * page cache (e.g., truncated or collapsed) + * after the lookup. Recheck under the folio + * lock. + */ + folio_lock(folio); + if (unlikely(folio->mapping != memfd->f_mapping)) { + folio_unlock(folio); + start_idx = retry_idx; + break; + } + if (try_grab_folio(folio, 1, FOLL_PIN)) { + folio_unlock(folio); folio_batch_release(&fbatch); ret = -EINVAL; goto err; } + /* + * fallocate() can leave folios in the + * page cache that were never cleared. + */ + if (!folio_test_uptodate(folio)) { + folio_zero_range(folio, 0, folio_size(folio)); + flush_dcache_folio(folio); + folio_mark_uptodate(folio); + } + retry_idx = folio_next_index(folio); + folio_unlock(folio); + if (nr_folios == 0) *offset = offset_in_folio(folio, start); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d175e93988a9..ce4d17871ff4 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2988,16 +2988,66 @@ void split_huge_pmd_address(struct vm_area_struct *vma, unsigned long address, __split_huge_pmd(vma, pmd, address, freeze, folio); } -static inline void split_huge_pmd_if_needed(struct vm_area_struct *vma, unsigned long address) +static bool pud_split_required(struct vm_area_struct *vma, + unsigned long address) { + if (!IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD)) + return false; + if (IS_ALIGNED(address, HPAGE_PUD_SIZE)) + return false; /* - * If the new address isn't hpage aligned and it could previously - * contain an hugepage: check if we need to split an huge pmd. + * Only selected VMAs support mapping PUDs for now. Lift the + * restriction once this changes. */ - if (!IS_ALIGNED(address, HPAGE_PMD_SIZE) && - range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PMD_SIZE), - ALIGN(address, HPAGE_PMD_SIZE))) - split_huge_pmd_address(vma, address, false, NULL); + if (!vma_is_dax(vma) && !(vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP))) + return false; + return range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PUD_SIZE), + ALIGN(address, HPAGE_PUD_SIZE)); +} + +static bool pmd_split_required(struct vm_area_struct *vma, + unsigned long address) +{ + if (IS_ALIGNED(address, HPAGE_PMD_SIZE)) + return false; + + return range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PMD_SIZE), + ALIGN(address, HPAGE_PMD_SIZE)); +} + +static void split_huge_leaf(struct vm_area_struct *vma, + unsigned long address) +{ + const bool split_pud = pud_split_required(vma, address); + const bool split_pmd = pmd_split_required(vma, address); + pgd_t *pgd; + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + + /* + * mmap, VMA and rmap write locks protect the walk; the split helpers + * take the page table locks. + */ + if (!split_pud && !split_pmd) + return; + + pgd = pgd_offset(vma->vm_mm, address); + if (!pgd_present(*pgd)) + return; + + p4d = p4d_offset(pgd, address); + if (!p4d_present(*p4d)) + return; + + pud = pud_offset(p4d, address); + if (pud_leaf(*pud)) + __split_huge_pud(vma, pud, address); + if (!split_pmd || !pud_present(*pud)) + return; + + pmd = pmd_offset(pud, address); + __split_huge_pmd(vma, pmd, address, false, NULL); } void vma_adjust_trans_huge(struct vm_area_struct *vma, @@ -3006,10 +3056,10 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma, long adjust_next) { /* Check if we need to split start first. */ - split_huge_pmd_if_needed(vma, start); + split_huge_leaf(vma, start); /* Check if we need to split end next. */ - split_huge_pmd_if_needed(vma, end); + split_huge_leaf(vma, end); /* * If we're also updating the next vma vm_start, @@ -3019,7 +3069,7 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma, struct vm_area_struct *next = find_vma(vma->vm_mm, vma->vm_end); unsigned long nstart = next->vm_start; nstart += adjust_next; - split_huge_pmd_if_needed(next, nstart); + split_huge_leaf(next, nstart); } } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 049726560c1a..1dd80a5b72b0 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -48,6 +48,7 @@ #include #include "internal.h" #include "hugetlb_vmemmap.h" +#include int hugetlb_max_hstate __read_mostly; unsigned int default_hstate_idx; @@ -1350,6 +1351,9 @@ static struct folio *dequeue_hugetlb_folio_node_exact(struct hstate *h, if (folio_test_hwpoison(folio)) continue; + if (is_migrate_isolate_page(&folio->page)) + continue; + list_move(&folio->lru, &h->hugepage_activelist); folio_ref_unfreeze(folio, 1); folio_clear_hugetlb_freed(folio); @@ -2862,20 +2866,24 @@ void restore_reserve_on_error(struct hstate *h, struct vm_area_struct *vma, /* * alloc_and_dissolve_hugetlb_folio - Allocate a new folio and dissolve * the old one - * @h: struct hstate old page belongs to * @old_folio: Old folio to dissolve * @list: List to isolate the page in case we need to * Returns 0 on success, otherwise negated error. */ -static int alloc_and_dissolve_hugetlb_folio(struct hstate *h, - struct folio *old_folio, struct list_head *list) +static int alloc_and_dissolve_hugetlb_folio(struct folio *old_folio, + struct list_head *list) { - gfp_t gfp_mask = htlb_alloc_mask(h) | __GFP_THISNODE; + gfp_t gfp_mask; + struct hstate *h; int nid = folio_nid(old_folio); struct folio *new_folio = NULL; int ret = 0; retry: + /* + * The old_folio might have been dissolved from under our feet, so make sure + * to carefully check the state under the lock. + */ spin_lock_irq(&hugetlb_lock); if (!folio_test_hugetlb(old_folio)) { /* @@ -2904,8 +2912,10 @@ static int alloc_and_dissolve_hugetlb_folio(struct hstate *h, cond_resched(); goto retry; } else { + h = folio_hstate(old_folio); if (!new_folio) { spin_unlock_irq(&hugetlb_lock); + gfp_mask = htlb_alloc_mask(h) | __GFP_THISNODE; new_folio = alloc_buddy_hugetlb_folio(h, gfp_mask, nid, NULL, NULL); if (!new_folio) @@ -2949,36 +2959,25 @@ static int alloc_and_dissolve_hugetlb_folio(struct hstate *h, int isolate_or_dissolve_huge_page(struct page *page, struct list_head *list) { - struct hstate *h; struct folio *folio = page_folio(page); int ret = -EBUSY; - /* - * The page might have been dissolved from under our feet, so make sure - * to carefully check the state under the lock. - * Return success when racing as if we dissolved the page ourselves. - */ - spin_lock_irq(&hugetlb_lock); - if (folio_test_hugetlb(folio)) { - h = folio_hstate(folio); - } else { - spin_unlock_irq(&hugetlb_lock); + /* Not to disrupt normal path by vainly holding hugetlb_lock */ + if (!folio_test_hugetlb(folio)) return 0; - } - spin_unlock_irq(&hugetlb_lock); /* * Fence off gigantic pages as there is a cyclic dependency between * alloc_contig_range and them. Return -ENOMEM as this has the effect * of bailing out right away without further retrying. */ - if (hstate_is_gigantic(h)) + if (folio_order(folio) > MAX_PAGE_ORDER) return -ENOMEM; if (folio_ref_count(folio) && isolate_hugetlb(folio, list)) ret = 0; else if (!folio_ref_count(folio)) - ret = alloc_and_dissolve_hugetlb_folio(h, folio, list); + ret = alloc_and_dissolve_hugetlb_folio(folio, list); return ret; } @@ -2991,6 +2990,37 @@ void wait_for_freed_hugetlb_folios(void) flush_work(&free_hpage_work); } +/* + * replace_free_hugepage_folios - Replace free hugepage folios in a given pfn + * range with new folios. + * @start_pfn: start pfn of the given pfn range + * @end_pfn: end pfn of the given pfn range + * Returns 0 on success, otherwise negated error. + */ +int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long end_pfn) +{ + struct folio *folio; + int ret = 0; + + LIST_HEAD(isolate_list); + + while (start_pfn < end_pfn) { + folio = pfn_folio(start_pfn); + + /* Not to disrupt normal path by vainly holding hugetlb_lock */ + if (folio_test_hugetlb(folio) && !folio_ref_count(folio)) { + ret = alloc_and_dissolve_hugetlb_folio(folio, &isolate_list); + if (ret) + break; + + putback_movable_pages(&isolate_list); + } + start_pfn++; + } + + return ret; +} + /* * NOTE! "cow_from_owner" represents a very hacky usage only used in CoW * faults of hugetlb private mappings on top of a non-page-cache folio (in @@ -3013,7 +3043,7 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, bool deferred_reserve; gfp_t gfp = htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; - memcg = get_mem_cgroup_from_current(); + memcg = get_mem_cgroup_from_mm(vma->vm_mm); memcg_charge_ret = mem_cgroup_hugetlb_try_charge(memcg, gfp, nr_pages); if (memcg_charge_ret == -ENOMEM) { mem_cgroup_put(memcg); @@ -5275,10 +5305,11 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, raw_write_seqcount_begin(&src->write_protect_seq); } else { /* - * For shared mappings the vma lock must be held before - * calling hugetlb_walk() in the src vma. Otherwise, the - * returned ptep could go away if part of a shared pmd and - * another thread calls huge_pmd_unshare. + * Shared mappings only need copying when we could have + * information stored in page tables (e.g., userfaultfd PTE + * bits or markers), which is incompatible with PMD sharing. + * Still take the hugetlb VMA lock for now to satisfy + * hugetlb_walk()'s locking requirements. */ hugetlb_vma_lock_read(src_vma); } @@ -5286,6 +5317,13 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, last_addr_mask = hugetlb_mask_last_page(h); for (addr = src_vma->vm_start; addr < src_vma->vm_end; addr += sz) { spinlock_t *src_ptl, *dst_ptl; + + if (WARN_ON_ONCE(want_pmd_share(src_vma, addr) || + want_pmd_share(dst_vma, addr))) { + ret = -EINVAL; + break; + } + src_pte = hugetlb_walk(src_vma, addr, sz); if (!src_pte) { addr |= last_addr_mask; @@ -5297,14 +5335,6 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, break; } -#ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING - /* If the pagetables are shared, there is nothing to do */ - if (ptdesc_pmd_is_shared(virt_to_ptdesc(dst_pte))) { - addr |= last_addr_mask; - continue; - } -#endif - dst_ptl = huge_pte_lock(h, dst, dst_pte); src_ptl = huge_pte_lockptr(h, src, src_pte); spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING); @@ -5478,10 +5508,9 @@ static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr, spin_unlock(dst_ptl); } -int move_hugetlb_page_tables(struct vm_area_struct *vma, - struct vm_area_struct *new_vma, - unsigned long old_addr, unsigned long new_addr, - unsigned long len) +unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma, + struct vm_area_struct *new_vma, unsigned long old_addr, + unsigned long new_addr, unsigned long len) { struct hstate *h = hstate_vma(vma); struct address_space *mapping = vma->vm_file->f_mapping; @@ -5776,6 +5805,8 @@ static void unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma, struct address_space *mapping; pgoff_t pgoff; + VM_WARN_ON_ONCE(!folio_test_anon(page_folio(page))); + /* * vm_pgoff is in PAGE_SIZE units, hence the different calculation * from page cache lookup which is in HPAGE_SIZE units. @@ -5811,9 +5842,17 @@ static void unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma, * could insert a zeroed page instead of the data existing * from the time of fork. This would look like data corruption */ - if (!is_vma_resv_set(iter_vma, HPAGE_RESV_OWNER)) - unmap_hugepage_range(iter_vma, address, - address + huge_page_size(h), page, 0); + if (is_vma_resv_set(iter_vma, HPAGE_RESV_OWNER)) + continue; + + /* + * The address in other VMAs can differ. Even though our old + * folio is anon, we always operate on the underlying hugetlb + * file. + */ + address = vma_address(iter_vma, pgoff, 1); + unmap_hugepage_range(iter_vma, address, + address + huge_page_size(h), page, 0); } i_mmap_unlock_write(mapping); } diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 7a54ce0f126b..1250d878d724 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2018,6 +2018,11 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, index += folio_nr_pages(folio); continue; out_unlock: + /* + * The folio may have been unmapped with TTU_BATCH_FLUSH. + * Flush before releasing the lock and our last reference. + */ + try_to_unmap_flush(); folio_unlock(folio); folio_put(folio); goto xa_unlocked; @@ -2028,9 +2033,8 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, xa_unlocked: /* - * If collapse is successful, flush must be done now before copying. - * If collapse is unsuccessful, does flush actually need to be done? - * Do it anyway, to clear the state. + * Flush before copying the folios, or releasing them in rollback. + * This is a no-op if out_unlock already flushed the batch. */ try_to_unmap_flush(); diff --git a/mm/ksm.c b/mm/ksm.c index 15f558f69093..8ce4cd5a382c 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -470,6 +470,7 @@ static void advisor_stop_scan(void) #ifdef CONFIG_NUMA /* Zeroed when merging across nodes is not allowed */ static unsigned int ksm_merge_across_nodes = 1; +/* Number of allocated stable and unstable tree roots. */ static int ksm_nr_node_ids = 1; #else #define ksm_merge_across_nodes 1U @@ -1407,6 +1408,10 @@ static int replace_page(struct vm_area_struct *vma, struct page *page, dec_mm_counter(mm, MM_ANONPAGES); } + /* Preserve soft-dirty when replacing the page. */ + if (pte_soft_dirty(orig_pte)) + newpte = pte_mksoft_dirty(newpte); + flush_cache_page(vma, addr, pte_pfn(ptep_get(ptep))); /* * No need to notify as we are replacing a read only page with another @@ -3528,14 +3533,13 @@ static ssize_t merge_across_nodes_store(struct kobject *kobj, else { root_stable_tree = buf; root_unstable_tree = buf + nr_node_ids; + ksm_nr_node_ids = nr_node_ids; /* Stable tree is empty but not the unstable */ root_unstable_tree[0] = one_unstable_tree[0]; } } - if (!err) { + if (!err) ksm_merge_across_nodes = knob; - ksm_nr_node_ids = knob ? 1 : nr_node_ids; - } } mutex_unlock(&ksm_thread_mutex); diff --git a/mm/madvise.c b/mm/madvise.c index ea966d402f07..6ce97106d6d4 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -752,6 +752,15 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, if (folio_test_swapcache(folio) || folio_test_dirty(folio)) { if (!folio_trylock(folio)) continue; + /* + * KSM folios can gain new mappings at any time, so we must + * not clear their dirty flag. + */ + if (folio_test_ksm(folio)) { + folio_unlock(folio); + continue; + } + /* * If we have a large folio at this point, we know it is * fully mapped so if its mapcount is the same as its diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f7dd9bea5196..2aae088c8608 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -591,7 +591,7 @@ static inline void memcg_rstat_updated(struct mem_cgroup *memcg, int val) static void do_flush_stats(struct mem_cgroup *memcg) { if (mem_cgroup_is_root(memcg)) - WRITE_ONCE(flush_last_time, jiffies_64); + WRITE_ONCE(flush_last_time, get_jiffies_64()); cgroup_rstat_flush(memcg->css.cgroup); } @@ -620,7 +620,7 @@ void mem_cgroup_flush_stats(struct mem_cgroup *memcg) void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg) { /* Only flush if the periodic flusher is one full cycle late */ - if (time_after64(jiffies_64, READ_ONCE(flush_last_time) + 2*FLUSH_TIME)) + if (time_after64(get_jiffies_64(), READ_ONCE(flush_last_time) + 2 * FLUSH_TIME)) mem_cgroup_flush_stats(memcg); } @@ -1928,7 +1928,10 @@ static void high_work_func(struct work_struct *work) struct mem_cgroup *memcg; memcg = container_of(work, struct mem_cgroup, high_work); + if (!css_tryget_online(&memcg->css)) + return; reclaim_high(memcg, MEMCG_CHARGE_BATCH, GFP_KERNEL); + css_put(&memcg->css); } /* @@ -2179,10 +2182,11 @@ int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, bool drained = false; bool raised_max_event = false; unsigned long pflags; + int ret = 0; retry: if (consume_stock(memcg, nr_pages)) - return 0; + return ret; if (!do_memsw_account() || page_counter_try_charge(&memcg->memsw, batch, &counter)) { @@ -2294,16 +2298,11 @@ int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, * put the burden of reclaim on regular allocation requests * and let these go through as privileged allocations. */ - if (!(gfp_mask & (__GFP_NOFAIL | __GFP_HIGH))) - return -ENOMEM; + if (!(gfp_mask & (__GFP_NOFAIL | __GFP_HIGH))) { + ret = -ENOMEM; + goto out; + } force: - /* - * If the allocation has to be enforced, don't forget to raise - * a MEMCG_MAX event. - */ - if (!raised_max_event) - memcg_memory_event(mem_over_limit, MEMCG_MAX); - /* * The allocation either can't fail or will lead to more memory * being freed very soon. Allow memory usage go over the limit @@ -2313,7 +2312,15 @@ int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, if (do_memsw_account()) page_counter_charge(&memcg->memsw, nr_pages); - return 0; +out: + /* + * Don't forget to raise a MEMCG_MAX event for forced or rejected + * requests. + */ + if (!raised_max_event) + memcg_memory_event(mem_over_limit, MEMCG_MAX); + + return ret; done_restock: if (batch > nr_pages) @@ -2372,7 +2379,7 @@ int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, !(current->flags & PF_MEMALLOC) && gfpflags_allow_blocking(gfp_mask)) mem_cgroup_handle_over_high(gfp_mask); - return 0; + return ret; } /** @@ -3311,7 +3318,7 @@ void mem_cgroup_flush_foreign(struct bdi_writeback *wb) { struct mem_cgroup *memcg = mem_cgroup_from_css(wb->memcg_css); unsigned long intv = msecs_to_jiffies(dirty_expire_interval * 10); - u64 now = jiffies_64; + u64 now = get_jiffies_64(); int i; for (i = 0; i < MEMCG_CGWB_FRN_CNT; i++) { @@ -3684,6 +3691,8 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css) page_counter_set_min(&memcg->memory, 0); page_counter_set_low(&memcg->memory, 0); + page_counter_set_high(&memcg->memory, PAGE_COUNTER_MAX); + page_counter_set_high(&memcg->swap, PAGE_COUNTER_MAX); zswap_memcg_offline_cleanup(memcg); @@ -4498,6 +4507,51 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root, page_counter_calculate_protection(&root->memory, &memcg->memory, recursive_protection); } +#ifdef CONFIG_LRU_GEN +/** + * mem_cgroup_calculate_protection_path - compute protection along a path + * @root: the top ancestor of the sub-tree being checked (NULL for root_mem_cgroup) + * @memcg: the target memory cgroup + * + * Walk the ancestor path from @root down to @memcg and compute the effective + * protection at each level. This is safe for isolated queries because it + * ensures parents are computed before children. + */ +void mem_cgroup_calculate_protection_path(struct mem_cgroup *root, + struct mem_cgroup *memcg) +{ + bool recursive_protection = + cgrp_dfl_root.flags & CGRP_ROOT_MEMORY_RECURSIVE_PROT; + struct cgroup *cg; + int root_level, i; + + if (mem_cgroup_disabled()) + return; + + if (!root) + root = root_mem_cgroup; + + if (memcg == root) + return; + + root_level = root->css.cgroup->level; + cg = memcg->css.cgroup; + + rcu_read_lock(); + for (i = root_level + 1; i <= cg->level; i++) { + struct mem_cgroup *cur; + + cur = mem_cgroup_from_css(rcu_dereference( + cg->ancestors[i]->subsys[memory_cgrp_id])); + if (cur) + page_counter_calculate_protection(&root->memory, + &cur->memory, + recursive_protection); + } + rcu_read_unlock(); +} +#endif /* CONFIG_LRU_GEN */ + static int charge_memcg(struct folio *folio, struct mem_cgroup *memcg, gfp_t gfp) { diff --git a/mm/memfd.c b/mm/memfd.c index 3e5a014fdacc..e41dd806ac50 100644 --- a/mm/memfd.c +++ b/mm/memfd.c @@ -465,6 +465,8 @@ SYSCALL_DEFINE2(memfd_create, goto err_fd; } file->f_mode |= FMODE_LSEEK | FMODE_PREAD | FMODE_PWRITE; + /* a regular file like any other: POSIX.1-2008/SUSv4 Section XSI 2.9.7 */ + file->f_mode |= FMODE_ATOMIC_POS; file->f_flags |= O_LARGEFILE; if (flags & MFD_NOEXEC_SEAL) { diff --git a/mm/memory.c b/mm/memory.c index c493f18cfb3a..38019497d8d4 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3046,6 +3046,13 @@ static inline int __wp_page_copy_user(struct page *dst, struct page *src, return 0; } + /* + * __copy_from_user_inatomic() only works as expected for local faults. + * Reject remote faults. + */ + if (unlikely(mm != current->mm)) + return -EFAULT; + /* * If the source page was a PFN mapping, we don't have * a "struct page" for it. We do a best-effort copy by @@ -3371,6 +3378,8 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) folio_put(old_folio); delayacct_wpcopy_end(); + if (err == -EFAULT) + return VM_FAULT_SIGBUS; return err == -EHWPOISON ? VM_FAULT_HWPOISON : 0; } kmsan_copy_page_meta(&new_folio->page, vmf->page); @@ -3858,8 +3867,8 @@ EXPORT_SYMBOL_GPL(unmap_mapping_pages); void unmap_mapping_range(struct address_space *mapping, loff_t const holebegin, loff_t const holelen, int even_cows) { - pgoff_t hba = (pgoff_t)(holebegin) >> PAGE_SHIFT; - pgoff_t hlen = ((pgoff_t)(holelen) + PAGE_SIZE - 1) >> PAGE_SHIFT; + pgoff_t hba = (u64)holebegin >> PAGE_SHIFT; + pgoff_t hlen = ((u64)holelen + PAGE_SIZE - 1) >> PAGE_SHIFT; /* Check for overflow. */ if (sizeof(holelen) > sizeof(hlen)) { @@ -6112,8 +6121,12 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, lru_gen_exit_fault(); - /* If the mapping is droppable, then errors due to OOM aren't fatal. */ - if (is_droppable) + /* + * If the mapping is droppable, then errors due to OOM aren't fatal: the + * task returns to user mode, handles its signals and tries again. A + * fault from kernel mode has no such return, so it must see the error. + */ + if (is_droppable && (flags & FAULT_FLAG_USER)) ret &= ~VM_FAULT_OOM; if (flags & FAULT_FLAG_USER) { @@ -6554,14 +6567,34 @@ int generic_access_phys(struct vm_area_struct *vma, unsigned long addr, resource_size_t phys_addr; unsigned long prot = 0; void __iomem *maddr; + struct page *page; int offset = offset_in_page(addr); int ret = -EINVAL; bool writable; struct follow_pfnmap_args args = { .vma = vma, .address = addr }; + /* + * Limit access to one page at a time, as that's what follow_pfnmap_start() + * guarantees; expect the caller to retry to read larger ranges. + */ + len = min_t(int, len, PAGE_SIZE - offset); + retry: if (follow_pfnmap_start(&args)) return -EINVAL; + + /* + * Reject anything that looks like ordinary RAM, including anonymous + * folios in CoW mappings or allocated kernel memory. Note that we + * cannot distinguish memblock allocations from non-RAM reliably, so + * we cannot easily reject them here. + */ + page = pfn_to_online_page(args.pfn); + if (page && !PageReserved(page)) { + follow_pfnmap_end(&args); + return -EINVAL; + } + prot = pgprot_val(args.pgprot); phys_addr = (resource_size_t)args.pfn << PAGE_SHIFT; writable = args.writable; @@ -6570,7 +6603,7 @@ int generic_access_phys(struct vm_area_struct *vma, unsigned long addr, if ((write & FOLL_WRITE) && !writable) return -EINVAL; - maddr = ioremap_prot(phys_addr, PAGE_ALIGN(len + offset), prot); + maddr = ioremap_prot(phys_addr, PAGE_SIZE, prot); if (!maddr) return -ENOMEM; @@ -6578,7 +6611,7 @@ int generic_access_phys(struct vm_area_struct *vma, unsigned long addr, goto out_unmap; if ((prot != pgprot_val(args.pgprot)) || - (phys_addr != (args.pfn << PAGE_SHIFT)) || + (phys_addr != ((resource_size_t)args.pfn << PAGE_SHIFT)) || (writable != args.writable)) { follow_pfnmap_end(&args); iounmap(maddr); diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 3853b6ff7117..193ebf34bf08 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -500,7 +500,8 @@ static inline bool queue_folio_required(struct folio *folio, return node_isset(nid, *qp->nmask) == !(flags & MPOL_MF_INVERT); } -static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk) +static void queue_folios_pmd(pmd_t *pmd, unsigned long addr, + struct mm_walk *walk) { struct folio *folio; struct queue_pages *qp = walk->private; @@ -509,11 +510,9 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk) qp->nr_failed++; return; } - folio = pmd_folio(*pmd); - if (is_huge_zero_folio(folio)) { - walk->action = ACTION_CONTINUE; + folio = vm_normal_folio_pmd(walk->vma, addr, *pmd); + if (!folio || folio_is_zone_device(folio)) return; - } if (!queue_folio_required(folio, qp)) return; if (!(qp->flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || @@ -545,7 +544,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr, ptl = pmd_trans_huge_lock(pmd, vma); if (ptl) { - queue_folios_pmd(pmd, walk); + queue_folios_pmd(pmd, addr, walk); spin_unlock(ptl); goto out; } @@ -3099,11 +3098,11 @@ void __init numa_policy_init(void) policy_cache = kmem_cache_create("numa_policy", sizeof(struct mempolicy), - 0, SLAB_PANIC, NULL); + 0, SLAB_PANIC | SLAB_ACCOUNT, NULL); sn_cache = kmem_cache_create("shared_policy_node", sizeof(struct sp_node), - 0, SLAB_PANIC, NULL); + 0, SLAB_PANIC | SLAB_ACCOUNT, NULL); for_each_node(nid) { preferred_node_policy[nid] = (struct mempolicy) { diff --git a/mm/migrate.c b/mm/migrate.c index e07a098671ce..66af1dd6a941 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2411,7 +2411,7 @@ static void do_pages_stat_array(struct mm_struct *mm, unsigned long nr_pages, mmap_read_lock(mm); for (i = 0; i < nr_pages; i++) { - unsigned long addr = (unsigned long)(*pages); + unsigned long addr = (unsigned long)untagged_addr_remote(mm, *pages); struct vm_area_struct *vma; struct folio_walk fw; struct folio *folio; @@ -2700,6 +2700,7 @@ int migrate_misplaced_folio(struct folio *folio, struct vm_area_struct *vma, pg_data_t *pgdat = NODE_DATA(node); int nr_remaining; unsigned int nr_succeeded; + int src_node = folio_nid(folio); LIST_HEAD(migratepages); struct mem_cgroup *memcg = get_mem_cgroup_from_folio(folio); struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); @@ -2714,7 +2715,7 @@ int migrate_misplaced_folio(struct folio *folio, struct vm_area_struct *vma, count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded); count_memcg_events(memcg, NUMA_PAGE_MIGRATE, nr_succeeded); if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) - && !node_is_toptier(folio_nid(folio)) + && !node_is_toptier(src_node) && node_is_toptier(node)) mod_lruvec_state(lruvec, PGPROMOTE_SUCCESS, nr_succeeded); } diff --git a/mm/mm_init.c b/mm/mm_init.c index 77f5f62628b7..f024ab7ae363 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -2481,8 +2481,6 @@ void __init memblock_free_pages(struct page *page, unsigned long pfn, return; } - /* pages were reserved and not allocated */ - clear_page_tag_ref(page); __free_pages_core(page, order, MEMINIT_EARLY); } diff --git a/mm/mmap.c b/mm/mmap.c index 2d69552e6be3..c930e03bd799 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1415,14 +1415,17 @@ static unsigned long __mmap_region(struct file *file, unsigned long addr, * Private writable mapping: check memory availability */ if (accountable_mapping(file, vm_flags)) { - charged = pglen; - charged -= vms.nr_accounted; - if (charged) { - error = security_vm_enough_memory_mm(mm, charged); + const unsigned long uncharged = pglen - vms.nr_accounted; + + /* Charge only what's not already charged by mappings we replace. */ + if (uncharged) { + error = security_vm_enough_memory_mm(mm, uncharged); if (error) goto abort_munmap; } + /* Our new mapping owns the entire charge now. */ + charged = pglen; vms.nr_accounted = 0; vm_flags |= VM_ACCOUNT; vmg.flags = vm_flags; diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index fc18fe274505..1c2707a8ed76 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -522,16 +522,25 @@ int __mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) { struct mmu_notifier_subscriptions *subscriptions = range->mm->notifier_subscriptions; - int ret; + const bool has_itree = subscriptions->has_itree; + int ret = 0; - if (subscriptions->has_itree) { + if (has_itree) { ret = mn_itree_invalidate(subscriptions, range); if (ret) return ret; } if (!hlist_empty(&subscriptions->list)) - return mn_hlist_invalidate_range_start(subscriptions, range); - return 0; + ret = mn_hlist_invalidate_range_start(subscriptions, range); + + /* + * On error, the non-blocking caller is not allowed to call + * invalidate_range_end(), as in mn_itree_invalidate(). + */ + if (ret && has_itree && !mmu_notifier_range_blockable(range)) + mn_itree_inv_end(subscriptions); + + return ret; } static void @@ -892,7 +901,6 @@ static int __mmu_interval_notifier_insert( struct mmu_notifier_subscriptions *subscriptions, unsigned long start, unsigned long length, const struct mmu_interval_notifier_ops *ops) { - interval_sub->mm = mm; interval_sub->ops = ops; RB_CLEAR_NODE(&interval_sub->interval_tree.rb); interval_sub->interval_tree.start = start; @@ -909,8 +917,12 @@ static int __mmu_interval_notifier_insert( if (WARN_ON(atomic_read(&mm->mm_users) <= 0)) return -EINVAL; - /* pairs with mmdrop in mmu_interval_notifier_remove() */ + /* + * Pairs with mmdrop in mmu_interval_notifier_remove(). Drivers take a + * non-NULL ->mm for "registered", so set it only with the reference. + */ mmgrab(mm); + interval_sub->mm = mm; /* * If some invalidate_range_start/end region is going on in parallel diff --git a/mm/mremap.c b/mm/mremap.c index 8b507e228a27..d4666f1ab593 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -782,6 +782,21 @@ static unsigned long move_vma(struct vm_area_struct *vma, */ move_page_tables(new_vma, new_addr, vma, old_addr, moved_len, true, false); + + /* + * When the error path completes, the new VMA will be + * unmapped rather than the old one. + * + * When an accountable move is being performed, this unmapping + * will not uncharge any expansion, so do that now. + * + * A move is accountable if the VMA is accountable and + * MREMAP_DONTUNMAP is not specified, so only do so if the + * latter is true. + */ + if (vm_flags & VM_ACCOUNT && !(flags & MREMAP_DONTUNMAP)) + vm_unacct_memory(to_account >> PAGE_SHIFT); + vma = new_vma; old_len = new_len; old_addr = new_addr; diff --git a/mm/page-writeback.c b/mm/page-writeback.c index bfb3f903bb6d..89b2e4a290b5 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -917,7 +917,9 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc, unsigned long thresh) { struct wb_domain *dom = dtc_dom(dtc); + u64 threshold = thresh; u64 wb_thresh; + u64 wb_max_thresh; unsigned long numerator, denominator; unsigned long wb_min_ratio, wb_max_ratio; @@ -927,15 +929,17 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc, fprop_fraction_percpu(&dom->completions, dtc->wb_completions, &numerator, &denominator); - wb_thresh = (thresh * (100 * BDI_RATIO_SCALE - bdi_min_ratio)) / (100 * BDI_RATIO_SCALE); + wb_thresh = div_u64(threshold * (100 * BDI_RATIO_SCALE - bdi_min_ratio), + 100 * BDI_RATIO_SCALE); wb_thresh *= numerator; wb_thresh = div64_ul(wb_thresh, denominator); wb_min_max_ratio(dtc->wb, &wb_min_ratio, &wb_max_ratio); - wb_thresh += (thresh * wb_min_ratio) / (100 * BDI_RATIO_SCALE); - if (wb_thresh > (thresh * wb_max_ratio) / (100 * BDI_RATIO_SCALE)) - wb_thresh = thresh * wb_max_ratio / (100 * BDI_RATIO_SCALE); + wb_thresh += div_u64(threshold * wb_min_ratio, 100 * BDI_RATIO_SCALE); + wb_max_thresh = div_u64(threshold * wb_max_ratio, 100 * BDI_RATIO_SCALE); + if (wb_thresh > wb_max_thresh) + wb_thresh = wb_max_thresh; return wb_thresh; } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index a649f0245e3b..869296e43349 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1355,6 +1355,9 @@ void __meminit __free_pages_core(struct page *page, unsigned int order, atomic_long_add(nr_pages, &page_zone(page)->managed_pages); } + /* pages were never allocated */ + clear_page_tag_ref(page); + if (page_contains_unaccepted(page, order)) { if (order == MAX_PAGE_ORDER && __free_unaccepted(page)) return; @@ -6682,6 +6685,7 @@ static int __alloc_contig_verify_gfp_mask(gfp_t gfp_mask, gfp_t *gfp_cc_mask) int alloc_contig_range_noprof(unsigned long start, unsigned long end, unsigned migratetype, gfp_t gfp_mask) { + const unsigned int order = ilog2(end - start); unsigned long outer_start, outer_end; int ret = 0; @@ -6696,6 +6700,14 @@ int alloc_contig_range_noprof(unsigned long start, unsigned long end, }; INIT_LIST_HEAD(&cc.migratepages); + /* + * In contrast to the buddy, we allow for orders here that exceed + * MAX_PAGE_ORDER, so we must manually make sure that we are not + * exceeding the maximum folio order. + */ + if (WARN_ON_ONCE((gfp_mask & __GFP_COMP) && order > MAX_FOLIO_ORDER)) + return -EINVAL; + gfp_mask = current_gfp_context(gfp_mask); if (__alloc_contig_verify_gfp_mask(gfp_mask, (gfp_t *)&cc.gfp_mask)) return -EINVAL; @@ -6740,7 +6752,17 @@ int alloc_contig_range_noprof(unsigned long start, unsigned long end, ret = __alloc_contig_migrate_range(&cc, start, end, migratetype); if (ret && ret != -EBUSY) goto done; - ret = 0; + + /* + * When in-use hugetlb pages are migrated, they may simply be released + * back into the free hugepage pool instead of being returned to the + * buddy system. After the migration of in-use huge pages is completed, + * we will invoke replace_free_hugepage_folios() to ensure that these + * hugepages are properly released to the buddy system. + */ + ret = replace_free_hugepage_folios(start, end); + if (ret) + goto done; /* * Pages from [start, end) are within a pageblock_nr_pages @@ -6783,7 +6805,6 @@ int alloc_contig_range_noprof(unsigned long start, unsigned long end, free_contig_range(end, outer_end - end); } else if (start == outer_start && end == outer_end && is_power_of_2(end - start)) { struct page *head = pfn_to_page(start); - int order = ilog2(end - start); check_new_pages(head, order); prep_new_page(head, order, gfp_mask, 0); diff --git a/mm/page_counter.c b/mm/page_counter.c index b249d15af9dd..9203b448e4f5 100644 --- a/mm/page_counter.c +++ b/mm/page_counter.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -354,7 +355,8 @@ static unsigned long effective_protection(unsigned long usage, * otherwise get a smaller chunk than what they claimed. */ if (siblings_protected > parent_effective) - return protected * parent_effective / siblings_protected; + return mul_u64_u64_div_u64(protected, parent_effective, + siblings_protected); /* * Ok, utilized protection of all children is within what the @@ -395,13 +397,20 @@ static unsigned long effective_protection(unsigned long usage, if (parent_effective > siblings_protected && parent_usage > siblings_protected && usage > protected) { - unsigned long unclaimed; + unsigned long parent_unclaimed, parent_unprotected, unprotected; - unclaimed = parent_effective - siblings_protected; - unclaimed *= usage - protected; - unclaimed /= parent_usage - siblings_protected; + parent_unclaimed = parent_effective - siblings_protected; + parent_unprotected = parent_usage - siblings_protected; - ep += unclaimed; + /* + * The usages aren't read atomically, so a child can transiently + * appear to use more than its parent, making the ratio exceed 1 + * and the quotient overflow 64 bits (#DE on x86). Cap it. + */ + unprotected = min(usage - protected, parent_unprotected); + + ep += mul_u64_u64_div_u64(parent_unclaimed, unprotected, + parent_unprotected); } return ep; diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 6989c5ffd474..b5ceebc8e744 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -376,13 +376,19 @@ static int isolate_single_pageblock(unsigned long boundary_pfn, int flags, } if (PageBuddy(page)) { - int order = buddy_order(page); + unsigned int order = buddy_order_unsafe(page); - /* move_freepages_block_isolate() handled this */ - VM_WARN_ON_ONCE(pfn + (1 << order) > boundary_pfn); - - pfn += 1UL << order; - continue; + /* buddy_order_unsafe() is racy. Validate the order before shifting. */ + if (order <= MAX_PAGE_ORDER && + /* + * move_freepages_block_isolate() splits + * cross-boundary PageBuddy, verify it. + */ + pfn + (1UL << order) <= boundary_pfn) { + pfn += 1UL << order; + continue; + } + goto failed; } /* @@ -401,10 +407,28 @@ static int isolate_single_pageblock(unsigned long boundary_pfn, int flags, if (PageCompound(page)) { struct page *head = compound_head(page); unsigned long head_pfn = page_to_pfn(head); - unsigned long nr_pages = compound_nr(head); + unsigned int order = compound_order(head); + unsigned long nr_pages; + + /* compound_order() is racy. Cap it at MAX_FOLIO_ORDER. */ + if (order > MAX_FOLIO_ORDER) + goto failed; + + nr_pages = 1UL << order; + + /* + * compound_head() is also racy, so the derived head_pfn + * needs additional checks to make sure it is valid. + * Otherwise, just fail the check. pfn comes from + * __first_valid_page() as a legitimate PFN, so use it to + * check head_pfn. + */ + if (head_pfn > pfn || !IS_ALIGNED(head_pfn, nr_pages) || + pfn - head_pfn >= nr_pages) + goto failed; if (head_pfn + nr_pages <= boundary_pfn || - PageHuge(page)) { + PageHuge(head)) { pfn = head_pfn + nr_pages; continue; } diff --git a/mm/page_owner.c b/mm/page_owner.c index 5d1c946c0008..d2783b7c0661 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -937,7 +937,7 @@ static const struct file_operations page_owner_stack_operations = { .open = page_owner_stack_open, .read = seq_read, .llseek = seq_lseek, - .release = seq_release_private, + .release = seq_release, }; static int page_owner_threshold_get(void *data, u64 *val) diff --git a/mm/page_table_check.c b/mm/page_table_check.c index 2efa910eef23..1076f8e055f3 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -67,7 +67,8 @@ static void page_table_check_clear(unsigned long pfn, unsigned long pgcnt) unsigned long i; bool anon; - if (!pfn_valid(pfn)) + if (!pfn_valid(pfn) || is_zero_pfn(pfn) || + is_huge_zero_folio(pfn_folio(pfn))) return; page = pfn_to_page(pfn); @@ -107,7 +108,8 @@ static void page_table_check_set(unsigned long pfn, unsigned long pgcnt, unsigned long i; bool anon; - if (!pfn_valid(pfn)) + if (!pfn_valid(pfn) || is_zero_pfn(pfn) || + is_huge_zero_folio(pfn_folio(pfn))) return; page = pfn_to_page(pfn); diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index 1cc8fd382cd1..67e6a32644ba 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -255,18 +255,18 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) if (!thp_migration_supported() || !(pvmw->flags & PVMW_MIGRATION)) - return not_found(pvmw); + goto next_pmd; entry = pmd_to_swp_entry(pmde); if (!is_migration_entry(entry) || !check_pmd(swp_offset_pfn(entry), pvmw)) - return not_found(pvmw); + goto next_pmd; return true; } if (likely(pmd_trans_huge(pmde) || pmd_devmap(pmde))) { if (pvmw->flags & PVMW_MIGRATION) - return not_found(pvmw); + goto next_pmd; if (!check_pmd(pmd_pfn(pmde), pvmw)) - return not_found(pvmw); + goto next_pmd; return true; } /* THP pmd was split under us: handle on pte level */ @@ -286,8 +286,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) spin_unlock(ptl); } - step_forward(pvmw, PMD_SIZE); - continue; + goto next_pmd; } if (!map_pte(pvmw, &ptl)) { if (!pvmw->pte) @@ -320,6 +319,17 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) spin_lock(pvmw->ptl); } goto this_pte; +next_pmd: + /* + * A large folio can be PTE-mapped across PMDs, so a + * nonmatching PMD does not end the walk. + */ + if (pvmw->ptl) { + spin_unlock(pvmw->ptl); + pvmw->ptl = NULL; + } + pvmw->pmd = NULL; + step_forward(pvmw, PMD_SIZE); } while (pvmw->address < end); return false; diff --git a/mm/shmem.c b/mm/shmem.c index 4d01fc877e76..8de38d7a98f9 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -75,6 +75,7 @@ static struct vfsmount *shm_mnt __ro_after_init; #include #include #include +#include #include #include #include @@ -985,12 +986,10 @@ void shmem_unlock_mapping(struct address_space *mapping) static struct folio *shmem_get_partial_folio(struct inode *inode, pgoff_t index) { + const gfp_t gfp = mapping_gfp_mask(inode->i_mapping); struct folio *folio; - /* - * At first avoid shmem_get_folio(,,,SGP_READ): that fails - * beyond i_size, and reports fallocated folios as holes. - */ +retry: folio = filemap_get_entry(inode->i_mapping, index); if (!folio) return folio; @@ -1001,13 +1000,14 @@ static struct folio *shmem_get_partial_folio(struct inode *inode, pgoff_t index) /* The folio has been swapped out */ folio_unlock(folio); folio_put(folio); + goto retry; } /* - * But read a folio back from swap if any of it is within i_size - * (although in some cases this is just a waste of time). + * Truncation can look up a large swap entry at or beyond i_size. + * Swap it in directly to bypass the size check in shmem_get_folio(). */ - folio = NULL; - shmem_get_folio(inode, index, 0, &folio, SGP_READ); + if (shmem_swapin_folio(inode, index, &folio, SGP_READ, gfp, NULL, NULL) == -EEXIST) + goto retry; return folio; } @@ -1126,14 +1126,16 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, order = shmem_confirm_swap(mapping, indices[i], radix_to_swp_entry(folio)); /* - * If found a large swap entry cross the end or start - * border, skip it as the truncate_inode_partial_folio - * above should have at least zerod its content once. + * Skip boundary entries when restarting + * the scan to avoid endless loops after + * swapin failure. */ if (order > 0) { base = round_down(base, 1 << order); - if (base < start || base + (1 << order) > end) + if (base < start || base + (1 << order) > end) { + start = max(start, base + (1 << order)); continue; + } } /* Swap was replaced by page or extended, retry */ index = base; @@ -1155,7 +1157,16 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, VM_BUG_ON_FOLIO(folio_test_writeback(folio), folio); - if (!folio_test_large(folio)) { + /* + * A shared mapping can have dirty PTEs while + * the folio is still clean. Mark the folio + * dirty so a failed split cannot discard data + * outside the range. + */ + if (!unfalloc && folio_test_large(folio)) + folio_mark_dirty(folio); + + if (unfalloc || !folio_test_large(folio)) { truncate_inode_folio(mapping, folio); } else if (truncate_inode_partial_folio(folio, lstart, lend)) { /* @@ -1171,6 +1182,15 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, index = start; break; } + } else if (folio->mapping) { + /* + * Splitting failed, but the part in the + * range is already zeroed. The folio + * was not dropped, so skip it when + * restarting the scan to avoid retrying + * it forever. + */ + start = max(start, folio_next_index(folio)); } } folio_unlock(folio); @@ -1236,10 +1256,11 @@ static int shmem_setattr(struct mnt_idmap *idmap, if (error) return error; - if ((info->seals & F_SEAL_EXEC) && (attr->ia_valid & ATTR_MODE)) { - if ((inode->i_mode ^ attr->ia_mode) & 0111) { - return -EPERM; - } + if (attr->ia_valid & ATTR_MODE) { + error = memfd_check_seals_mode(info->seals, inode->i_mode, + attr->ia_mode); + if (error) + return error; } if (S_ISREG(inode->i_mode) && (attr->ia_valid & ATTR_SIZE)) { @@ -1266,7 +1287,8 @@ static int shmem_setattr(struct mnt_idmap *idmap, if (oldsize > holebegin) unmap_mapping_range(inode->i_mapping, holebegin, 0, 1); - if (info->alloced) + /* Swapin-error markers are not counted in alloced. */ + if (info->alloced || !mapping_empty(inode->i_mapping)) shmem_truncate_range(inode, newsize, (loff_t)-1); /* unmap again to remove racily COWed private pages */ @@ -1435,6 +1457,10 @@ static int shmem_unuse_inode(struct inode *inode, unsigned int type) if (ret < 0) break; + if (signal_pending(current)) { + ret = -ERESTARTSYS; + break; + } start = indices[folio_batch_count(&fbatch) - 1]; } while (true); @@ -4915,12 +4941,35 @@ static const struct file_operations shmem_file_operations = { #endif }; +#ifdef CONFIG_TMPFS_POSIX_ACL +static int shmem_set_acl(struct mnt_idmap *idmap, struct dentry *dentry, + struct posix_acl *acl, int type) +{ + struct inode *inode = d_inode(dentry); + umode_t mode = inode->i_mode; + int error; + + if (type == ACL_TYPE_ACCESS) { + error = posix_acl_equiv_mode(acl, &mode); + if (error < 0) + return error; + error = memfd_check_seals_mode(SHMEM_I(inode)->seals, + inode->i_mode, mode); + if (error) + return error; + } + return simple_set_acl(idmap, dentry, acl, type); +} +#else +#define shmem_set_acl simple_set_acl +#endif + static const struct inode_operations shmem_inode_operations = { .getattr = shmem_getattr, .setattr = shmem_setattr, #ifdef CONFIG_TMPFS_XATTR .listxattr = shmem_listxattr, - .set_acl = simple_set_acl, + .set_acl = shmem_set_acl, .fileattr_get = shmem_fileattr_get, .fileattr_set = shmem_fileattr_set, #endif diff --git a/mm/shmem_quota.c b/mm/shmem_quota.c index d1e32ac01407..868b3e05f1aa 100644 --- a/mm/shmem_quota.c +++ b/mm/shmem_quota.c @@ -234,7 +234,7 @@ static int shmem_acquire_dquot(struct dquot *dquot) return ret; } -static bool shmem_is_empty_dquot(struct dquot *dquot) +static bool shmem_is_default_dquot(struct dquot *dquot) { struct shmem_sb_info *sbinfo = dquot->dq_sb->s_fs_info; qsize_t bhardlimit; @@ -248,19 +248,21 @@ static bool shmem_is_empty_dquot(struct dquot *dquot) ihardlimit = sbinfo->qlimits.grpquota_ihardlimit; } - if (test_bit(DQ_FAKE_B, &dquot->dq_flags) || - (dquot->dq_dqb.dqb_curspace == 0 && - dquot->dq_dqb.dqb_curinodes == 0 && - dquot->dq_dqb.dqb_bhardlimit == bhardlimit && - dquot->dq_dqb.dqb_ihardlimit == ihardlimit)) - return true; - - return false; + /* + * Defaults are zero usage and soft limits, and the mount's hard limits. + * DQ_FAKE_B only means that *all* limits are zero, which need not match + * the mount defaults. + */ + return dquot->dq_dqb.dqb_curspace == 0 && + dquot->dq_dqb.dqb_curinodes == 0 && + dquot->dq_dqb.dqb_bsoftlimit == 0 && + dquot->dq_dqb.dqb_isoftlimit == 0 && + dquot->dq_dqb.dqb_bhardlimit == bhardlimit && + dquot->dq_dqb.dqb_ihardlimit == ihardlimit; } /* - * Store limits from dquot in the tree unless it's fake. If it is fake - * remove the id from the tree since there is no useful information in - * there. + * Remove the ID from the tree if it has no usage and its limits match + * the mount defaults. Otherwise, save its limits for the next acquire. */ static int shmem_release_dquot(struct dquot *dquot) { @@ -295,7 +297,7 @@ static int shmem_release_dquot(struct dquot *dquot) return -ENOENT; found: - if (shmem_is_empty_dquot(dquot)) { + if (shmem_is_default_dquot(dquot)) { /* Remove entry from the tree */ rb_erase(&entry->node, info->dqi_priv); kfree(entry); diff --git a/mm/swapfile.c b/mm/swapfile.c index 4ea0ebefe92d..99eb5878f5ac 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -2262,6 +2262,8 @@ static inline int unuse_pmd_range(struct vm_area_struct *vma, pud_t *pud, pmd = pmd_offset(pud, addr); do { cond_resched(); + if (signal_pending(current)) + return -ERESTARTSYS; next = pmd_addr_end(addr, end); ret = unuse_pte_range(vma, pmd, addr, next, type); if (ret) @@ -2475,7 +2477,7 @@ static int try_to_unuse(unsigned int type) if (READ_ONCE(si->inuse_pages)) { if (!signal_pending(current)) goto retry; - return -EINTR; + return -ERESTARTSYS; } success: diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 9951b4f42c65..29d2366d88fa 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -48,8 +48,7 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm, vma = vma_lookup(mm, addr); if (!vma) vma = ERR_PTR(-ENOENT); - else if (!(vma->vm_flags & VM_SHARED) && - unlikely(anon_vma_prepare(vma))) + else if (is_cow_mapping(vma->vm_flags) && unlikely(anon_vma_prepare(vma))) vma = ERR_PTR(-ENOMEM); return vma; @@ -77,7 +76,7 @@ static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm, * We know we're going to need to use anon_vma, so check * that early. */ - if (!(vma->vm_flags & VM_SHARED) && unlikely(!vma->anon_vma)) + if (is_cow_mapping(vma->vm_flags) && unlikely(!vma->anon_vma)) vma_end_read(vma); else return vma; @@ -1323,8 +1322,12 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pte_unmap(dst_pte); src_pte = dst_pte = NULL; err = split_folio(src_folio); - if (err) + if (err) { + /* do not retry, extra refs may be held forever */ + if (err == -EAGAIN) + err = -EBUSY; goto out; + } /* have to reacquire the folio after it got split */ folio_unlock(src_folio); folio_put(src_folio); diff --git a/mm/vma.c b/mm/vma.c index 66b0472b7d8a..85660d67b794 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -1694,7 +1694,10 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap, faulted_in_anon_vma = true; } - new_vma = vma_merge_new_range(&vmg); + /* If not registered for remap, uffd context will be cleared. */ + new_vma = NULL; + if (!vma_has_uffd_without_event_remap(vma)) + new_vma = vma_merge_new_range(&vmg); if (new_vma) { /* diff --git a/mm/vmscan.c b/mm/vmscan.c index 512a31d7ae9e..b248dcb88369 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3600,7 +3600,7 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end, walk->mm_stats[MM_NONLEAF_FOUND]++; - if (!walk_pte_range(&val, addr, next, args)) + if (!walk_pte_range(pmd + i, addr, next, args)) continue; walk->mm_stats[MM_NONLEAF_ADDED]++; @@ -3636,7 +3636,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long start, unsigned long end, if (!pud_present(val) || WARN_ON_ONCE(pud_leaf(val))) continue; - walk_pmd_range(&val, addr, next, args); + walk_pmd_range(pud + i, addr, next, args); if (need_resched() || walk->batched >= MAX_LRU_BATCH) { end = (addr | ~PUD_MASK) + 1; @@ -4752,10 +4752,6 @@ static bool should_abort_scan(struct lruvec *lruvec, struct scan_control *sc) int i; enum zone_watermarks mark; - /* don't abort memcg reclaim to ensure fairness */ - if (!root_reclaim(sc)) - return false; - if (sc->nr_reclaimed >= max(sc->nr_to_reclaim, compact_gap(sc->order))) return true; @@ -4839,7 +4835,13 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc) struct mem_cgroup *memcg = lruvec_memcg(lruvec); struct pglist_data *pgdat = lruvec_pgdat(lruvec); - /* lru_gen_age_node() called mem_cgroup_calculate_protection() */ + /* + * For kswapd, mem_cgroup_calculate_protection() has already + * been called during the top-down cgroup traversal. + */ + if (!current_is_kswapd()) + mem_cgroup_calculate_protection_path(NULL, memcg); + if (mem_cgroup_below_min(NULL, memcg)) return MEMCG_LRU_YOUNG; diff --git a/mm/zswap.c b/mm/zswap.c index 6c50249d5398..61311c92bd6f 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1778,7 +1778,7 @@ static int zswap_setup(void) if (pool) { pr_info("loaded using pool %s/%s\n", pool->tfm_name, zpool_get_type(pool->zpool)); - list_add(&pool->list, &zswap_pools); + list_add_rcu(&pool->list, &zswap_pools); zswap_has_pool = true; } else { pr_err("pool creation failed\n"); diff --git a/net/9p/trans_usbg.c b/net/9p/trans_usbg.c index 468f7e8f0277..938e3130ab81 100644 --- a/net/9p/trans_usbg.c +++ b/net/9p/trans_usbg.c @@ -148,7 +148,8 @@ static void usb9pfs_tx_complete(struct usb_ep *ep, struct usb_request *req) { struct f_usb9pfs *usb9pfs = ep->driver_data; struct usb_composite_dev *cdev = usb9pfs->function.config->cdev; - struct p9_req_t *p9_tx_req = req->context; + struct p9_client *client; + struct p9_req_t *p9_tx_req; unsigned long flags; /* reset zero packages */ @@ -164,18 +165,25 @@ static void usb9pfs_tx_complete(struct usb_ep *ep, struct usb_request *req) ep->name, req->status, req->actual, req->length); spin_lock_irqsave(&usb9pfs->lock, flags); - WRITE_ONCE(p9_tx_req->status, REQ_STATUS_SENT); + client = usb9pfs->client; + p9_tx_req = req->context; + req->context = NULL; - p9_req_put(usb9pfs->client, p9_tx_req); + if (!client || !p9_tx_req) + goto unlock_complete; - req->context = NULL; + WRITE_ONCE(p9_tx_req->status, REQ_STATUS_SENT); + + p9_req_put(client, p9_tx_req); +unlock_complete: spin_unlock_irqrestore(&usb9pfs->lock, flags); complete(&usb9pfs->send); } -static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf) +static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, + struct p9_client *client, void *buf) { struct p9_req_t *p9_rx_req; struct p9_fcall rc; @@ -201,7 +209,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf) "mux %p pkt: size: %d bytes tag: %d\n", usb9pfs, rc.size, rc.tag); - p9_rx_req = p9_tag_lookup(usb9pfs->client, rc.tag); + p9_rx_req = p9_tag_lookup(client, rc.tag); if (!p9_rx_req || p9_rx_req->status != REQ_STATUS_SENT) { p9_debug(P9_DEBUG_ERROR, "Unexpected packet tag %d\n", rc.tag); return NULL; @@ -211,7 +219,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf) p9_debug(P9_DEBUG_ERROR, "requested packet size too big: %d for tag %d with capacity %zd\n", rc.size, rc.tag, p9_rx_req->rc.capacity); - p9_req_put(usb9pfs->client, p9_rx_req); + p9_req_put(client, p9_rx_req); return NULL; } @@ -219,7 +227,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf) p9_debug(P9_DEBUG_ERROR, "No recv fcall for tag %d (req %p), disconnecting!\n", rc.tag, p9_rx_req); - p9_req_put(usb9pfs->client, p9_rx_req); + p9_req_put(client, p9_rx_req); return NULL; } @@ -230,8 +238,10 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req) { struct f_usb9pfs *usb9pfs = ep->driver_data; struct usb_composite_dev *cdev = usb9pfs->function.config->cdev; + struct p9_client *client; struct p9_req_t *p9_rx_req; unsigned int req_size = req->actual; + unsigned long flags; int status = REQ_STATUS_RCVD; if (req->status) { @@ -240,9 +250,16 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req) return; } - p9_rx_req = usb9pfs_rx_header(usb9pfs, req->buf); - if (!p9_rx_req) + spin_lock_irqsave(&usb9pfs->lock, flags); + client = usb9pfs->client; + if (!client) { + spin_unlock_irqrestore(&usb9pfs->lock, flags); return; + } + + p9_rx_req = usb9pfs_rx_header(usb9pfs, client, req->buf); + if (!p9_rx_req) + goto out_unlock; if (req_size > p9_rx_req->rc.capacity) { dev_err(&cdev->gadget->dev, @@ -256,8 +273,11 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req) p9_rx_req->rc.size = req_size; - p9_client_cb(usb9pfs->client, p9_rx_req, status); - p9_req_put(usb9pfs->client, p9_rx_req); + p9_client_cb(client, p9_rx_req, status); + p9_req_put(client, p9_rx_req); + +out_unlock: + spin_unlock_irqrestore(&usb9pfs->lock, flags); complete(&usb9pfs->received); } @@ -277,18 +297,18 @@ static void disable_usb9pfs(struct f_usb9pfs *usb9pfs) struct usb_composite_dev *cdev = usb9pfs->function.config->cdev; + disable_ep(cdev, usb9pfs->in_ep); + disable_ep(cdev, usb9pfs->out_ep); + if (usb9pfs->in_req) { usb_ep_free_request(usb9pfs->in_ep, usb9pfs->in_req); usb9pfs->in_req = NULL; } if (usb9pfs->out_req) { - usb_ep_free_request(usb9pfs->out_ep, usb9pfs->out_req); + free_ep_req(usb9pfs->out_ep, usb9pfs->out_req); usb9pfs->out_req = NULL; } - - disable_ep(cdev, usb9pfs->in_ep); - disable_ep(cdev, usb9pfs->out_ep); dev_dbg(&cdev->gadget->dev, "%s disabled\n", usb9pfs->function.name); } @@ -313,10 +333,6 @@ static int alloc_requests(struct usb_composite_dev *cdev, usb9pfs->in_req->complete = usb9pfs_tx_complete; usb9pfs->out_req->complete = usb9pfs_rx_complete; - /* length will be set in complete routine */ - usb9pfs->in_req->context = usb9pfs; - usb9pfs->out_req->context = usb9pfs; - return 0; fail_in: @@ -389,7 +405,7 @@ static int p9_usbg_create(struct p9_client *client, const char *devname, char *a guard(mutex)(&usb9pfs_lock); list_for_each_entry(dev, &usbg_instance_list, usb9pfs_instance) { - if (!strncmp(devname, dev->tag, strlen(devname))) { + if (!strcmp(devname, dev->tag)) { if (!dev->inuse) { dev->inuse = true; found = 1; @@ -414,7 +430,9 @@ static int p9_usbg_create(struct p9_client *client, const char *devname, char *a client->status = Disconnected; else client->status = Connected; + spin_lock_irq(&usb9pfs->lock); usb9pfs->client = client; + spin_unlock_irq(&usb9pfs->lock); client->trans_mod->maxsize = usb9pfs->buflen; @@ -425,18 +443,26 @@ static int p9_usbg_create(struct p9_client *client, const char *devname, char *a static void usb9pfs_clear_tx(struct f_usb9pfs *usb9pfs) { + struct p9_client *client; struct p9_req_t *req; + unsigned long flags; - guard(spinlock_irqsave)(&usb9pfs->lock); + spin_lock_irqsave(&usb9pfs->lock, flags); + client = usb9pfs->client; + usb9pfs->client = NULL; + req = usb9pfs->in_req ? usb9pfs->in_req->context : NULL; + if (usb9pfs->in_req) + usb9pfs->in_req->context = NULL; + spin_unlock_irqrestore(&usb9pfs->lock, flags); - req = usb9pfs->in_req->context; - if (!req) + if (!req || !client) return; if (!req->t_err) req->t_err = -ECONNRESET; - p9_client_cb(usb9pfs->client, req, REQ_STATUS_ERROR); + p9_client_cb(client, req, REQ_STATUS_ERROR); + usb9pfs->in_req->context = NULL; } static void p9_usbg_close(struct p9_client *client) @@ -464,6 +490,9 @@ static void p9_usbg_close(struct p9_client *client) mutex_lock(&usb9pfs_lock); dev->inuse = false; mutex_unlock(&usb9pfs_lock); + + disable_usb9pfs(usb9pfs); + reinit_completion(&usb9pfs->send); } static int p9_usbg_request(struct p9_client *client, struct p9_req_t *p9_req) @@ -745,8 +774,18 @@ static int usb9pfs_set_alt(struct usb_function *f, static void usb9pfs_disable(struct usb_function *f) { struct f_usb9pfs *usb9pfs = func_to_usb9pfs(f); + unsigned long flags; + spin_lock_irqsave(&usb9pfs->lock, flags); + if (usb9pfs->client) + usb9pfs->client->status = Disconnected; + spin_unlock_irqrestore(&usb9pfs->lock, flags); usb9pfs_clear_tx(usb9pfs); + + if (usb9pfs->in_ep->enabled) + disable_usb9pfs(usb9pfs); + + reinit_completion(&usb9pfs->send); } static struct usb_function *usb9pfs_alloc(struct usb_function_instance *fi) diff --git a/net/9p/trans_virtio.c b/net/9p/trans_virtio.c index 0b8086f58ad5..d1a680f69911 100644 --- a/net/9p/trans_virtio.c +++ b/net/9p/trans_virtio.c @@ -629,6 +629,11 @@ static int p9_virtio_probe(struct virtio_device *vdev) err = -EINVAL; goto out_free_vq; } + if (tag_len > NAME_MAX) { + dev_err(&vdev->dev, "mount tag too long (%u bytes)\n", tag_len); + err = -EINVAL; + goto out_free_vq; + } tag = kzalloc(tag_len + 1, GFP_KERNEL); if (!tag) { err = -ENOMEM; diff --git a/net/9p/trans_xen.c b/net/9p/trans_xen.c index 068d57515dd5..67f464acf617 100644 --- a/net/9p/trans_xen.c +++ b/net/9p/trans_xen.c @@ -198,9 +198,19 @@ static void p9_xen_response(struct work_struct *work) masked_prod, &masked_cons, XEN_9PFS_RING_SIZE(ring)); + if (h.size < sizeof(h) || + h.size > xen_9pfs_queued(prod, cons, + XEN_9PFS_RING_SIZE(ring))) { + dev_warn_ratelimited(&priv->dev->dev, + "bad response size %u from backend\n", h.size); + break; + } + req = p9_tag_lookup(priv->client, h.tag); if (!req || req->status != REQ_STATUS_SENT) { dev_warn(&priv->dev->dev, "Wrong req tag=%x\n", h.tag); + if (req) + p9_req_put(priv->client, req); cons += h.size; virt_mb(); ring->intf->in_cons = cons; diff --git a/net/ax25/af_ax25.c b/net/ax25/af_ax25.c index 95224600b5a1..d464f8ea4e55 100644 --- a/net/ax25/af_ax25.c +++ b/net/ax25/af_ax25.c @@ -78,6 +78,7 @@ static void ax25_kill_by_device(struct net_device *dev) ax25_dev *ax25_dev; ax25_cb *s; struct sock *sk; + bool orphan; if ((ax25_dev = ax25_dev_ax25dev(dev)) == NULL) return; @@ -109,6 +110,16 @@ static void ax25_kill_by_device(struct net_device *dev) sock_hold(sk); spin_unlock_bh(&ax25_list_lock); lock_sock(sk); + /* + * A socket closed by its owner, or a connection left + * unaccepted by a closed listener, is freed by its + * heartbeat once it is down. ax25_disconnect() below + * stops that heartbeat for good, so free the socket + * here, or it is never freed and pins the module. + */ + orphan = sock_flag(sk, SOCK_DESTROY) || + (sk->sk_state == TCP_LISTEN && + sock_flag(sk, SOCK_DEAD)); ax25_disconnect(s, ENETUNREACH); WRITE_ONCE(s->ax25_dev, NULL); if (sk->sk_socket) { @@ -116,7 +127,10 @@ static void ax25_kill_by_device(struct net_device *dev) &s->dev_tracker); ax25_dev_put(ax25_dev); } - ax25_cb_del(s); + if (orphan) + ax25_destroy_socket(s); + else + ax25_cb_del(s); release_sock(sk); spin_lock_bh(&ax25_list_lock); sock_put(sk); @@ -694,6 +708,12 @@ static int ax25_setsockopt(struct socket *sock, int level, int optname, case AX25_EXTSEQ: ax25->modulus = opt ? AX25_EMODULUS : AX25_MODULUS; + /* + * A window set for modulo 128 may not fit modulo 8: + * ax25_kick() would then find it full and send nothing. + */ + if (ax25->modulus == AX25_MODULUS && ax25->window > 7) + ax25->window = 7; break; case AX25_PIDINCL: @@ -1233,7 +1253,7 @@ static int __must_check ax25_connect(struct socket *sock, struct sock *sk = sock->sk; ax25_cb *ax25 = sk_to_ax25(sk), *ax25t; struct full_sockaddr_ax25 *fsa = (struct full_sockaddr_ax25 *)uaddr; - ax25_digi *digi = NULL; + ax25_digi *digi = NULL, *old_digi; int ct = 0, err = 0; /* @@ -1282,11 +1302,28 @@ static int __must_check ax25_connect(struct socket *sock, goto out_release; } + /* + * A connection that failed or ended through ax25_disconnect() left + * the socket SOCK_DEAD and shut down for sending. The socket is still + * its owner's: start afresh, or the state changes of the new attempt + * would wake no one and every send() would fail with -EPIPE. + */ + sock_reset_flag(sk, SOCK_DEAD); + sk->sk_shutdown = 0; + sk->sk_state = TCP_CLOSE; sock->state = SS_UNCONNECTED; - kfree(ax25->digipeat); + /* + * A bound socket is on ax25_list, where the receive path and + * /proc/net/ax25 read its digipeater path under ax25_list_lock: + * take the old path off under that lock before freeing it. + */ + spin_lock_bh(&ax25_list_lock); + old_digi = ax25->digipeat; ax25->digipeat = NULL; + spin_unlock_bh(&ax25_list_lock); + kfree(old_digi); /* * Handle digi-peaters to be used. @@ -1337,6 +1374,15 @@ static int __must_check ax25_connect(struct socket *sock, goto out_release; } + /* + * The socket is on ax25_list with the destination of its previous + * attempt, if any: forget it, or the lookup below finds the socket + * itself when the same station is called again. + */ + spin_lock_bh(&ax25_list_lock); + ax25->dest_addr = null_ax25_address; + spin_unlock_bh(&ax25_list_lock); + if (sk->sk_type == SOCK_SEQPACKET && (ax25t=ax25_find_cb(&ax25->source_addr, &fsa->fsa_ax25.sax25_call, digi, ax25->ax25_dev->dev))) { @@ -1346,8 +1392,10 @@ static int __must_check ax25_connect(struct socket *sock, goto out_release; } + spin_lock_bh(&ax25_list_lock); ax25->dest_addr = fsa->fsa_ax25.sax25_call; ax25->digipeat = digi; + spin_unlock_bh(&ax25_list_lock); /* First the easy one */ if (sk->sk_type != SOCK_SEQPACKET) { @@ -1484,16 +1532,29 @@ static int ax25_accept(struct socket *sock, struct socket *newsock, goto out; newsk = skb->sk; + + /* + * ax25_kill_by_device() detaches a connection from a device going + * down under the lock of its socket, clears its ax25_dev and drops + * the device references if the socket has been grafted by then. + * Graft the socket and take those references under the same lock, + * and take none if the device has already gone: the connection is + * closed then, and the socket is returned as such. + */ + lock_sock_nested(newsk, SINGLE_DEPTH_NESTING); sock_graft(newsk, newsock); + ax25 = sk_to_ax25(newsk); + ax25_dev = ax25->ax25_dev; + if (ax25_dev) { + netdev_hold(ax25_dev->dev, &ax25->dev_tracker, GFP_ATOMIC); + ax25_dev_hold(ax25_dev); + } + release_sock(newsk); /* Now attach up the new socket */ kfree_skb(skb); sk_acceptq_removed(sk); newsock->state = SS_CONNECTED; - ax25 = sk_to_ax25(newsk); - ax25_dev = ax25->ax25_dev; - netdev_hold(ax25_dev->dev, &ax25->dev_tracker, GFP_ATOMIC); - ax25_dev_hold(ax25_dev); out: release_sock(sk); @@ -1555,7 +1616,7 @@ static int ax25_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) struct sock *sk = sock->sk; struct sockaddr_ax25 sax; struct sk_buff *skb; - ax25_digi dtmp, *dp; + ax25_digi dtmp, *dp = NULL; ax25_cb *ax25; size_t size; int lv, err, addr_len = msg->msg_namelen; @@ -1630,6 +1691,7 @@ static int ax25_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) } dtmp.lastrepeat = 0; + dp = &dtmp; } sax = *usax; @@ -1638,10 +1700,6 @@ static int ax25_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) err = -EISCONN; goto out; } - if (usax->sax25_ndigis == 0) - dp = NULL; - else - dp = &dtmp; } else { /* * FIXME: 1003.1g - if the socket is like this because @@ -1779,25 +1837,36 @@ static int ax25_recvmsg(struct socket *sock, struct msghdr *msg, size_t size, if (msg->msg_name) { ax25_digi digi; ax25_address src; - const unsigned char *mac = skb_mac_header(skb); + const unsigned char *mac; DECLARE_SOCKADDR(struct sockaddr_ax25 *, sax, msg->msg_name); memset(sax, 0, sizeof(struct full_sockaddr_ax25)); - ax25_addr_parse(mac + 1, skb->data - mac - 1, &src, NULL, - &digi, NULL, NULL); sax->sax25_family = AF_AX25; - /* We set this correctly, even though we may not let the - application know the digi calls further down (because it - did NOT ask to know them). This could get political... **/ - sax->sax25_ndigis = digi.ndigi; - sax->sax25_call = src; - - if (sax->sax25_ndigis != 0) { - int ct; - struct full_sockaddr_ax25 *fsa = (struct full_sockaddr_ax25 *)sax; - - for (ct = 0; ct < digi.ndigi; ct++) - fsa->fsa_digipeater[ct] = digi.calls[ct]; + + /* + * A frame put together from fragments by ax25_rx_fragment() + * has no address header left, and no mac header: give no + * address then. ax25_addr_parse() fills neither src nor digi + * when it fails. + */ + mac = skb_mac_header_was_set(skb) ? skb_mac_header(skb) : NULL; + if (mac && ax25_addr_parse(mac + 1, skb->data - mac - 1, &src, + NULL, &digi, NULL, NULL)) { + /* + * We set this correctly, even though we may not let the + * application know the digi calls further down (because + * it did NOT ask to know them). This could get political... + */ + sax->sax25_ndigis = digi.ndigi; + sax->sax25_call = src; + + if (sax->sax25_ndigis != 0) { + int ct; + struct full_sockaddr_ax25 *fsa = (struct full_sockaddr_ax25 *)sax; + + for (ct = 0; ct < digi.ndigi; ct++) + fsa->fsa_digipeater[ct] = digi.calls[ct]; + } } msg->msg_namelen = sizeof(struct full_sockaddr_ax25); } diff --git a/net/ax25/ax25_ds_timer.c b/net/ax25/ax25_ds_timer.c index c4f8adbf8144..5eb54dd1120a 100644 --- a/net/ax25/ax25_ds_timer.c +++ b/net/ax25/ax25_ds_timer.c @@ -211,7 +211,7 @@ void ax25_ds_t1_timeout(ax25_cb *ax25) case AX25_STATE_2: if (ax25->n2count == ax25->n2) { ax25_send_control(ax25, AX25_DISC, AX25_POLLON, AX25_COMMAND); - if (!sock_flag(ax25->sk, SOCK_DESTROY)) + if (!ax25->sk || !sock_flag(ax25->sk, SOCK_DESTROY)) ax25_disconnect(ax25, ETIMEDOUT); return; } else { diff --git a/net/ax25/ax25_in.c b/net/ax25/ax25_in.c index 1bcb5e321467..fd8669e5b731 100644 --- a/net/ax25/ax25_in.c +++ b/net/ax25/ax25_in.c @@ -201,7 +201,7 @@ static int ax25_rcv(struct sk_buff *skb, struct net_device *dev, int type = 0, mine = 0, dama; struct sock *make, *sk; ax25_digi dp, reverse_dp; - ax25_cb *ax25; + ax25_cb *ax25, *existing; ax25_dev *ax25_dev; /* @@ -401,11 +401,12 @@ static int ax25_rcv(struct sk_buff *skb, struct net_device *dev, } } - skb_set_owner_r(skb, make); - skb_queue_head(&sk->sk_receive_queue, skb); - - make->sk_state = TCP_ESTABLISHED; - + /* + * Keep a place in the backlog, but put the skb on the + * listener's queue only once the connection is set up + * below: accept() takes it from there, and close() on the + * new socket frees the connection. + */ sk_acceptq_added(sk); bh_unlock_sock(sk); } else { @@ -450,6 +451,24 @@ static int ax25_rcv(struct sk_buff *skb, struct net_device *dev, ax25->window = ax25_dev->values[AX25_VALUES_WINDOW]; } + if (!sk) { + /* + * ax25_send_frame() may have created a connection for this + * link since the lookup above: add ours only if there is + * none, otherwise give the SABM to the existing one. Frames + * queued on ours before it reaches state 3 are sent by the + * ax25_kick() below. + */ + existing = ax25_cb_add_unique(ax25); + if (existing) { + ax25_cb_put(ax25); + if (ax25_process_rx_frame(existing, skb, type, dama) == 0) + kfree_skb(skb); + ax25_cb_put(existing); + return 0; + } + } + ax25_send_control(ax25, AX25_UA, AX25_POLLON, AX25_RESPONSE); #ifdef CONFIG_AX25_DAMA_SLAVE @@ -459,15 +478,36 @@ static int ax25_rcv(struct sk_buff *skb, struct net_device *dev, ax25->state = AX25_STATE_3; - ax25_cb_add(ax25); + if (sk) + ax25_cb_add(ax25); ax25_start_heartbeat(ax25); ax25_start_t3timer(ax25); ax25_start_idletimer(ax25); + if (!sk) + ax25_kick(ax25); + if (sk) { - if (!sock_flag(sk, SOCK_DEAD)) + bh_lock_sock(sk); + skb_set_owner_r(skb, make); + if (sk->sk_state == TCP_LISTEN && !sock_flag(sk, SOCK_DEAD)) { + skb_queue_head(&sk->sk_receive_queue, skb); sk->sk_data_ready(sk); + } else { + /* + * The listener was closed meanwhile: leave the + * connection to its heartbeat, as + * ax25_destroy_socket() does with the connections + * that were still waiting on its queue. + */ + sk_acceptq_removed(sk); + kfree_skb(skb); + sock_orphan(make); + make->sk_state = TCP_LISTEN; + ax25->state = AX25_STATE_0; + } + bh_unlock_sock(sk); sock_put(sk); } else { free: diff --git a/net/ax25/ax25_ip.c b/net/ax25/ax25_ip.c index 215d4ccf12b9..b24cb88d75de 100644 --- a/net/ax25/ax25_ip.c +++ b/net/ax25/ax25_ip.c @@ -176,9 +176,14 @@ netdev_tx_t ax25_ip_xmit(struct sk_buff *skb) ax25_dev->values[AX25_VALUES_PACLEN], &src_c, &dst_c, digipeat, dev); - if (ax25) { + /* + * ax25_send_frame() leaves the skb to its caller + * when it gives up, as ROSE and NET/ROM expect. + */ + if (ax25) ax25_cb_put(ax25); - } + else + kfree_skb(ourskb); goto put; } } diff --git a/net/ax25/ax25_out.c b/net/ax25/ax25_out.c index 7c4ebef5ae6c..50b014f7b0df 100644 --- a/net/ax25/ax25_out.c +++ b/net/ax25/ax25_out.c @@ -137,7 +137,9 @@ EXPORT_SYMBOL(ax25_send_frame); */ void ax25_output(ax25_cb *ax25, int paclen, struct sk_buff *skb) { + struct sk_buff_head frags; struct sk_buff *skbn; + unsigned long flags; unsigned char *p; int frontlen, len, fragno, ka9qfrag, first = 1; @@ -161,11 +163,20 @@ void ax25_output(ax25_cb *ax25, int paclen, struct sk_buff *skb) frontlen = skb_headroom(skb); /* Address space + CTRL */ + /* + * Build the fragments on a list of our own: a message that + * cannot be fragmented whole is not sent at all, and its + * fragments go on the write queue one after the other. + */ + __skb_queue_head_init(&frags); + while (skb->len > 0) { spin_lock_bh(&ax25_frag_lock); if ((skbn = alloc_skb(paclen + 2 + frontlen, GFP_ATOMIC)) == NULL) { spin_unlock_bh(&ax25_frag_lock); printk(KERN_CRIT "AX.25: ax25_output - out of memory\n"); + __skb_queue_purge(&frags); + kfree_skb(skb); return; } @@ -200,10 +211,14 @@ void ax25_output(ax25_cb *ax25, int paclen, struct sk_buff *skb) } skb_pull(skb, len); - skb_queue_tail(&ax25->write_queue, skbn); /* Throw it on the queue */ + __skb_queue_tail(&frags, skbn); } kfree_skb(skb); + + spin_lock_irqsave(&ax25->write_queue.lock, flags); + skb_queue_splice_tail_init(&frags, &ax25->write_queue); + spin_unlock_irqrestore(&ax25->write_queue.lock, flags); } else { skb_queue_tail(&ax25->write_queue, skb); /* Throw it on the queue */ } diff --git a/net/ax25/ax25_std_timer.c b/net/ax25/ax25_std_timer.c index b17da41210cb..05ff1032748f 100644 --- a/net/ax25/ax25_std_timer.c +++ b/net/ax25/ax25_std_timer.c @@ -143,7 +143,7 @@ void ax25_std_t1timer_expiry(ax25_cb *ax25) case AX25_STATE_2: if (ax25->n2count == ax25->n2) { ax25_send_control(ax25, AX25_DISC, AX25_POLLON, AX25_COMMAND); - if (!sock_flag(ax25->sk, SOCK_DESTROY)) + if (!ax25->sk || !sock_flag(ax25->sk, SOCK_DESTROY)) ax25_disconnect(ax25, ETIMEDOUT); return; } else { diff --git a/net/ax25/sysctl_net_ax25.c b/net/ax25/sysctl_net_ax25.c index 68753aa30334..41a83e3a7e55 100644 --- a/net/ax25/sysctl_net_ax25.c +++ b/net/ax25/sysctl_net_ax25.c @@ -20,7 +20,7 @@ static int min_t2[] = {1}, max_t2[] = {20000}; static int min_t3[1], max_t3[] = {3600000}; static int min_idle[1], max_idle[] = {65535000}; static int min_n2[] = {1}, max_n2[] = {31}; -static int min_paclen[] = {1}, max_paclen[] = {512}; +static int min_paclen[] = {16}, max_paclen[] = {512}; static int min_proto[1], max_proto[] = { AX25_PROTO_MAX }; #ifdef CONFIG_AX25_DAMA_SLAVE static int min_ds_timeout[1], max_ds_timeout[] = {65535000}; diff --git a/net/batman-adv/bat_iv_ogm.c b/net/batman-adv/bat_iv_ogm.c index 5dd3e1f281ba..dd1f88af5b96 100644 --- a/net/batman-adv/bat_iv_ogm.c +++ b/net/batman-adv/bat_iv_ogm.c @@ -449,6 +449,7 @@ batadv_iv_ogm_can_aggregate(const struct batadv_ogm_packet *new_bat_ogm_packet, struct batadv_ogm_packet *batadv_ogm_packet; int aggregated_bytes = forw_packet->packet_len + packet_len; struct batadv_hard_iface *primary_if = NULL; + u8 packet_num = forw_packet->num_packets + 1; bool res = false; unsigned long aggregation_end_time; @@ -474,6 +475,9 @@ batadv_iv_ogm_can_aggregate(const struct batadv_ogm_packet *new_bat_ogm_packet, if (aggregated_bytes > BATADV_MAX_AGGREGATION_BYTES) return false; + if (packet_num >= BITS_PER_TYPE(forw_packet->direct_link_flags)) + return false; + /* packet is not leaving on the same interface. */ if (forw_packet->if_outgoing != if_outgoing) return false; @@ -1789,6 +1793,11 @@ static void batadv_iv_send_outstanding_bat_ogm_packet(struct work_struct *work) goto out; } + /* Mark aggregate as full before forcing emit. */ + spin_lock_bh(&bat_priv->forw_bat_list_lock); + forw_packet->num_packets = BITS_PER_TYPE(forw_packet->direct_link_flags); + spin_unlock_bh(&bat_priv->forw_bat_list_lock); + batadv_iv_ogm_emit(forw_packet); /* we have to have at least one packet in the queue to determine the diff --git a/net/batman-adv/distributed-arp-table.c b/net/batman-adv/distributed-arp-table.c index 45e58b348bd7..39d74b908d64 100644 --- a/net/batman-adv/distributed-arp-table.c +++ b/net/batman-adv/distributed-arp-table.c @@ -583,6 +583,7 @@ static void batadv_choose_next_candidate(struct batadv_priv *bat_priv, * NOT_FOUND */ cands[select].type = BATADV_DAT_CANDIDATE_NOT_FOUND; + cands[select].orig_node = NULL; /* iterate over the originator list and find the node with the closest * dat_address which has not been selected yet diff --git a/net/batman-adv/main.c b/net/batman-adv/main.c index 8091619981c0..ccfe1c7f33ed 100644 --- a/net/batman-adv/main.c +++ b/net/batman-adv/main.c @@ -263,6 +263,11 @@ void batadv_mesh_free(struct net_device *soft_iface) atomic_set(&bat_priv->mesh_state, BATADV_MESH_DEACTIVATING); + /* the originator purge worker accesses (among others) the TT tables + * and must therefore be stopped before any of them are freed + */ + disable_delayed_work_sync(&bat_priv->orig_work); + batadv_purge_outstanding_packets(bat_priv, NULL); batadv_tp_stop_all(bat_priv); @@ -449,8 +454,10 @@ int batadv_batman_skb_recv(struct sk_buff *skb, struct net_device *dev, * shut down otherwise the packet may trigger de-reference errors * further down in the receive path. */ - if (!kref_get_unless_zero(&hard_iface->refcount)) - goto err_out; + if (!kref_get_unless_zero(&hard_iface->refcount)) { + kfree_skb(skb); + return NET_RX_DROP; + } skb = skb_share_check(skb, GFP_ATOMIC); @@ -509,7 +516,6 @@ int batadv_batman_skb_recv(struct sk_buff *skb, struct net_device *dev, kfree_skb(skb); err_put: batadv_hardif_put(hard_iface); -err_out: return NET_RX_DROP; } diff --git a/net/batman-adv/main.h b/net/batman-adv/main.h index 3d4c36ae2e1a..9f3c1134e006 100644 --- a/net/batman-adv/main.h +++ b/net/batman-adv/main.h @@ -102,9 +102,6 @@ */ #define BATADV_TQ_SIMILARITY_THRESHOLD 50 -/* should not be bigger than 512 bytes or change the size of - * forw_packet->direct_link_flags - */ #define BATADV_MAX_AGGREGATION_BYTES 512 #define BATADV_MAX_AGGREGATION_MS 100 diff --git a/net/batman-adv/multicast_forw.c b/net/batman-adv/multicast_forw.c index 893cb9ce9971..9752e3dd79ef 100644 --- a/net/batman-adv/multicast_forw.c +++ b/net/batman-adv/multicast_forw.c @@ -950,8 +950,11 @@ static int batadv_mcast_forw_packet(struct batadv_priv *bat_priv, dest = (u8 *)mcast_tracker + sizeof(*mcast_tracker); - /* check if full tracker tvlv is within skb length */ + /* check if full tracker tvlv, including padding, is within skb length */ tvlv_len = sizeof(*mcast_tracker) + ETH_ALEN * num_dests; + if (!(num_dests % 2)) + tvlv_len += 2; + if (tvlv_len > skb_network_header_len(skb)) return -EINVAL; diff --git a/net/batman-adv/tp_meter.c b/net/batman-adv/tp_meter.c index 50b83e0438a0..e28c4923f52f 100644 --- a/net/batman-adv/tp_meter.c +++ b/net/batman-adv/tp_meter.c @@ -226,9 +226,9 @@ static void batadv_tp_batctl_notify(enum batadv_tp_meter_reason reason, unsigned long start_time, u64 total_sent, u32 cookie) { + u64 total_bytes; u32 test_time; u8 result; - u32 total_bytes; if (!batadv_tp_is_error(reason)) { result = BATADV_TP_REASON_COMPLETE; diff --git a/net/batman-adv/translation-table.c b/net/batman-adv/translation-table.c index 7df77cca52a0..c3331304cc39 100644 --- a/net/batman-adv/translation-table.c +++ b/net/batman-adv/translation-table.c @@ -455,6 +455,9 @@ batadv_tt_orig_list_entry_put(struct batadv_tt_orig_list_entry *orig_entry) * @bat_priv: the bat priv with all the soft interface information * @tt_local_entry: the TT entry involved in the event * @event_flags: flags to store in the event structure + * + * Only a single change per client is queued. It describes the latest state of + * the client and a new event therefore replaces an already queued one. */ static void batadv_tt_local_event(struct batadv_priv *bat_priv, struct batadv_tt_local_entry *tt_local_entry, @@ -463,14 +466,12 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv, struct batadv_tt_change_node *tt_change_node, *entry, *safe; struct batadv_tt_common_entry *common = &tt_local_entry->common; u8 flags = common->flags | event_flags; - bool event_removed = false; - bool del_op_requested, del_op_entry; + bool del_op_requested; tt_change_node = kmem_cache_alloc(batadv_tt_change_cache, GFP_ATOMIC); if (!tt_change_node) return; - tt_change_node->change.flags = flags; memset(tt_change_node->change.reserved, 0, sizeof(tt_change_node->change.reserved)); ether_addr_copy(tt_change_node->change.addr, common->addr); @@ -478,6 +479,15 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv, del_op_requested = flags & BATADV_TT_CLIENT_DEL; + /* the ROAM flag is only meaningful for the receiver in combination + * with a DEL. An ADD with ROAM would be interpreted as a roaming + * advertisement + */ + if (!del_op_requested) + flags &= ~BATADV_TT_CLIENT_ROAM; + + tt_change_node->change.flags = flags; + /* check for ADD+DEL or DEL+ADD events */ spin_lock_bh(&bat_priv->tt.changes_list_lock); list_for_each_entry_safe(entry, safe, &bat_priv->tt.changes_list, @@ -488,44 +498,23 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv, if (entry->change.vid != tt_change_node->change.vid) continue; - /* DEL+ADD in the same orig interval have no effect and can be - * removed to avoid silly behaviour on the receiver side. The - * other way around (ADD+DEL) can happen in case of roaming of - * a client still in the NEW state. Roaming of NEW clients is - * now possible due to automatically recognition of "temporary" - * clients - */ - del_op_entry = entry->change.flags & BATADV_TT_CLIENT_DEL; - if (!del_op_requested && del_op_entry) - goto del; - if (del_op_requested && !del_op_entry) - goto del; - - /* this is a second add in the same originator interval. It - * means that flags have been changed: update them! + /* the other nodes may know the client with different + * flags. A DEL+ADD must therefore still announce the + * current flags and an ADD+DEL must still remove the + * client. ADD+ADD and DEL+DEL only update the flags */ - if (!del_op_requested && !del_op_entry) - entry->change.flags = flags; - - continue; -del: - list_del(&entry->list); - kmem_cache_free(batadv_tt_change_cache, entry); + entry->change.flags = flags; kmem_cache_free(batadv_tt_change_cache, tt_change_node); - event_removed = true; + goto unlock; } /* track the change in the OGMinterval list */ list_add_tail(&tt_change_node->list, &bat_priv->tt.changes_list); + atomic_inc(&bat_priv->tt.local_changes); unlock: spin_unlock_bh(&bat_priv->tt.changes_list_lock); - - if (event_removed) - atomic_dec(&bat_priv->tt.local_changes); - else - atomic_inc(&bat_priv->tt.local_changes); } /** @@ -650,6 +639,7 @@ bool batadv_tt_local_add(struct net_device *soft_iface, const u8 *addr, int hash_added, table_size, packet_size_max; bool ret = false; bool roamed_back = false; + bool added = false; u8 remote_flags; u32 match_mark; @@ -676,7 +666,7 @@ bool batadv_tt_local_add(struct net_device *soft_iface, const u8 *addr, * flag can be reset like it was never enqueued */ tt_local->common.flags &= ~BATADV_TT_CLIENT_PENDING; - goto add_event; + added = true; } if (tt_local->common.flags & BATADV_TT_CLIENT_ROAM) { @@ -757,14 +747,17 @@ bool batadv_tt_local_add(struct net_device *soft_iface, const u8 *addr, goto out; } -add_event: - batadv_tt_local_event(bat_priv, tt_local, BATADV_NO_FLAGS); + added = true; check_roaming: + if (added) + batadv_tt_local_event(bat_priv, tt_local, BATADV_NO_FLAGS); + /* Check whether it is a roaming, but don't do anything if the roaming * process has already been handled */ - if (tt_global && !(tt_global->common.flags & BATADV_TT_CLIENT_ROAM)) { + if (tt_global && + (roamed_back || !(tt_global->common.flags & BATADV_TT_CLIENT_ROAM))) { /* These node are probably going to update their tt table */ head = &tt_global->orig_list; rcu_read_lock(); @@ -1026,20 +1019,22 @@ batadv_tt_prepare_tvlv_local_data(struct batadv_priv *bat_priv, * batadv_tt_tvlv_container_update() - update the translation table tvlv * container after local tt changes have been committed * @bat_priv: the bat priv with all the soft interface information + * @changes_list: list of committed changes which will be consumed + * @tt_diff_entries_num: number of entries in @changes_list */ -static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv) +static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv, + struct list_head *changes_list, + int tt_diff_entries_num) { struct batadv_tt_change_node *entry, *safe; struct batadv_tvlv_tt_data *tt_data; struct batadv_tvlv_tt_change *tt_change; int tt_diff_len, tt_change_len = 0; - int tt_diff_entries_num = 0; int tt_diff_entries_count = 0; bool drop_changes = false; size_t tt_extra_len = 0; u16 tvlv_len; - tt_diff_entries_num = atomic_read(&bat_priv->tt.local_changes); tt_diff_len = batadv_tt_len(tt_diff_entries_num); /* if we have too many changes for one packet don't send any @@ -1057,19 +1052,39 @@ static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv) tvlv_len = batadv_tt_prepare_tvlv_local_data(bat_priv, &tt_data, &tt_change, &tt_diff_len); - if (!tvlv_len) + if (!tvlv_len) { + /* the changes are lost and the other nodes have to request the + * full table. Don't answer TT requests for the new TTVN with + * the stale changeset of the previous TTVN. Without changes, the + * TTVN was not incremented and the last changeset is still valid + */ + if (drop_changes || tt_diff_entries_num > 0) { + list_for_each_entry_safe(entry, safe, changes_list, + list) { + list_del(&entry->list); + kmem_cache_free(batadv_tt_change_cache, entry); + } + + spin_lock_bh(&bat_priv->tt.last_changeset_lock); + kfree(bat_priv->tt.last_changeset); + bat_priv->tt.last_changeset = NULL; + bat_priv->tt.last_changeset_len = 0; + spin_unlock_bh(&bat_priv->tt.last_changeset_lock); + } + + /* announce the already committed TTVN in the next OGM interval + * instead of waiting for BATADV_TT_OGM_APPEND_MAX idle rounds + */ + atomic_set(&bat_priv->tt.ogm_append_cnt, 0); return; + } tt_data->flags = BATADV_TT_OGM_DIFF; if (!drop_changes && tt_diff_len == 0) goto container_register; - spin_lock_bh(&bat_priv->tt.changes_list_lock); - atomic_set(&bat_priv->tt.local_changes, 0); - - list_for_each_entry_safe(entry, safe, &bat_priv->tt.changes_list, - list) { + list_for_each_entry_safe(entry, safe, changes_list, list) { if (tt_diff_entries_count < tt_diff_entries_num) { memcpy(tt_change + tt_diff_entries_count, &entry->change, @@ -1079,7 +1094,6 @@ static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv) list_del(&entry->list); kmem_cache_free(batadv_tt_change_cache, entry); } - spin_unlock_bh(&bat_priv->tt.changes_list_lock); tt_extra_len = batadv_tt_len(tt_diff_entries_num - tt_diff_entries_count); @@ -2757,8 +2771,12 @@ static bool batadv_tt_local_valid(const void *entry_ptr, if (tt_common_entry->flags & BATADV_TT_CLIENT_NEW) return false; + /* the entry is (still) announced. A ROAM flag of an entry which is + * pending to be removed must not be transmitted. Otherwise, the + * receiver would handle it like a roaming advertisement + */ if (flags) - *flags = tt_common_entry->flags; + *flags = tt_common_entry->flags & ~BATADV_TT_CLIENT_ROAM; return true; } @@ -3323,20 +3341,29 @@ static void _batadv_tt_update_changes(struct batadv_priv *bat_priv, { int i; int roams; + u8 flags; for (i = 0; i < tt_num_changes; i++) { - if ((tt_change + i)->flags & BATADV_TT_CLIENT_DEL) { - roams = (tt_change + i)->flags & BATADV_TT_CLIENT_ROAM; + flags = (tt_change + i)->flags; + + if (flags & BATADV_TT_CLIENT_DEL) { + roams = flags & BATADV_TT_CLIENT_ROAM; batadv_tt_global_del(bat_priv, orig_node, (tt_change + i)->addr, ntohs((tt_change + i)->vid), "tt removed by changes", roams); } else { + /* an announced client cannot have roamed away from + * orig_node. Only a ROAMING_ADV is allowed to add a + * global entry with the ROAM flag + */ + flags &= ~BATADV_TT_CLIENT_ROAM; + if (!batadv_tt_global_add(bat_priv, orig_node, (tt_change + i)->addr, ntohs((tt_change + i)->vid), - (tt_change + i)->flags, ttvn)) + flags, ttvn)) /* In case of problem while storing a * global_entry, we stop the updating * procedure without committing the @@ -3755,11 +3782,25 @@ static void batadv_tt_local_purge_pending_clients(struct batadv_priv *bat_priv) */ static void batadv_tt_local_commit_changes_nolock(struct batadv_priv *bat_priv) { + LIST_HEAD(changes_list); + int changes_num; + lockdep_assert_held(&bat_priv->tt.commit_lock); - if (atomic_read(&bat_priv->tt.local_changes) < 1) { + /* atomically get the list of changes. The TTVN decision and the + * announced changeset must cover the same events. Events queued + * afterwards stay in the tt.changes_list for the next TT round + */ + spin_lock_bh(&bat_priv->tt.changes_list_lock); + list_splice_init(&bat_priv->tt.changes_list, &changes_list); + changes_num = atomic_read(&bat_priv->tt.local_changes); + atomic_set(&bat_priv->tt.local_changes, 0); + spin_unlock_bh(&bat_priv->tt.changes_list_lock); + + if (changes_num == 0) { if (!batadv_atomic_dec_not_zero(&bat_priv->tt.ogm_append_cnt)) - batadv_tt_tvlv_container_update(bat_priv); + batadv_tt_tvlv_container_update(bat_priv, &changes_list, + changes_num); return; } @@ -3776,7 +3817,7 @@ static void batadv_tt_local_commit_changes_nolock(struct batadv_priv *bat_priv) /* reset the sending counter */ atomic_set(&bat_priv->tt.ogm_append_cnt, BATADV_TT_OGM_APPEND_MAX); - batadv_tt_tvlv_container_update(bat_priv); + batadv_tt_tvlv_container_update(bat_priv, &changes_list, changes_num); } /** diff --git a/net/batman-adv/types.h b/net/batman-adv/types.h index 1b8ab18dcf2c..62fe0e29f669 100644 --- a/net/batman-adv/types.h +++ b/net/batman-adv/types.h @@ -2208,7 +2208,13 @@ struct batadv_forw_packet { /** @direct_link_flags: direct link flags for aggregated OGM packets */ u32 direct_link_flags; - /** @num_packets: counter for aggregated OGMv1 packets */ + /** + * @num_packets: counter for aggregated OGMv1 packets. Set to + * BITS_PER_TYPE(forw_packet->direct_link_flags) (independent of the + * real number of aggregated packets) by the send worker to prevent + * further aggregation while the packet is emitted. It must therefore + * not be used to iterate over the aggregated packets + */ u8 num_packets; /** @delayed_work: work queue callback item for packet sending */ diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c index 654bb2454e20..e0bcb000107c 100644 --- a/net/bluetooth/hci_sync.c +++ b/net/bluetooth/hci_sync.c @@ -2732,7 +2732,7 @@ static int hci_pause_advertising_sync(struct hci_dev *hdev) static int hci_resume_advertising_sync(struct hci_dev *hdev) { struct adv_info *adv, *tmp; - int err; + int err = 0; /* If advertising has not been paused there is nothing to do. */ if (!hdev->advertising_paused) @@ -2765,13 +2765,19 @@ static int hci_resume_advertising_sync(struct hci_dev *hdev) */ if (hci_dev_test_and_clear_flag(hdev, HCI_LE_ADV_0)) err = hci_enable_ext_advertising_sync(hdev, 0x00); - } else { + } else if (hdev->cur_adv_instance) { /* Schedule for most recent instance to be restarted and begin * the software rotation loop */ err = hci_schedule_adv_instance_sync(hdev, hdev->cur_adv_instance, true); + } else if (hci_dev_test_flag(hdev, HCI_ADVERTISING) && + !hci_dev_test_flag(hdev, HCI_LE_ADV)) { + /* hci_schedule_adv_instance_sync() rejects instance 0x00 while + * HCI_ADVERTISING is set, so enable it directly. + */ + err = hci_start_adv_sync(hdev, 0x00); } hdev->advertising_paused = false; @@ -6148,6 +6154,7 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval) u8 filter_policy = 0x00; /* Default is to enable duplicates filter */ u8 filter_dup = LE_SCAN_FILTER_DUP_ENABLE; + bool paused; int err; bt_dev_dbg(hdev, ""); @@ -6171,6 +6178,12 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval) if (err) goto failed; + /* LE Set Random Address is disallowed while advertising is enabled. */ + paused = !hdev->advertising_paused; + err = hci_pause_advertising_sync(hdev); + if (err) + goto failed; + /* All active scans will be done with either a resolvable private * address (when privacy feature has been enabled) or non-resolvable * private address. @@ -6180,6 +6193,9 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval) if (err < 0) own_addr_type = ADDR_LE_DEV_PUBLIC; + if (paused) + hci_resume_advertising_sync(hdev); + if (hci_is_adv_monitoring(hdev) || (test_bit(HCI_QUIRK_STRICT_DUPLICATE_FILTER, &hdev->quirks) && hdev->discovery.result_filtering)) { @@ -6203,9 +6219,8 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval) return err; failed: - /* Resume advertising if it was paused */ - if (use_ll_privacy(hdev)) - hci_resume_advertising_sync(hdev); + /* No-op when advertising was not paused. */ + hci_resume_advertising_sync(hdev); /* Resume passive scanning */ hci_update_passive_scan_sync(hdev); diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c index d8c39a430686..514abca6ed88 100644 --- a/net/bluetooth/l2cap_core.c +++ b/net/bluetooth/l2cap_core.c @@ -161,6 +161,28 @@ static struct l2cap_chan *__l2cap_get_chan_by_ident(struct l2cap_conn *conn, return NULL; } +/* Our connection request has been sent and is still outstanding */ +static bool l2cap_chan_connect_pending(struct l2cap_chan *chan) +{ + return chan->state == BT_CONNECT && + test_bit(CONF_CONNECT_PEND, &chan->conf_state); +} + +/* Other channels may hold the same ident for a response they still have + * to send, so only look at channels waiting for a connection response. + */ +static struct l2cap_chan *__l2cap_get_connect_chan_by_ident(struct l2cap_conn *conn, + u8 ident) +{ + struct l2cap_chan *c; + + list_for_each_entry(c, &conn->chan_l, list) { + if (c->ident == ident && l2cap_chan_connect_pending(c)) + return c; + } + return NULL; +} + static struct l2cap_chan *__l2cap_global_chan_by_addr(__le16 psm, bdaddr_t *src, u8 src_type) { @@ -4265,7 +4287,7 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn, if (!chan) return -EBADSLT; } else { - chan = __l2cap_get_chan_by_ident(conn, cmd->ident); + chan = __l2cap_get_connect_chan_by_ident(conn, cmd->ident); if (!chan) return -EBADSLT; } @@ -4278,6 +4300,13 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn, l2cap_chan_lock(chan); + /* Only accept a response matching a connection request we sent and + * that is still outstanding. A channel waiting in BT_CONNECT for + * security or the feature mask has not sent one yet. + */ + if (!l2cap_chan_connect_pending(chan) || chan->ident != cmd->ident) + goto done; + switch (result) { case L2CAP_CR_SUCCESS: if (__l2cap_get_chan_by_dcid(conn, dcid)) { @@ -4307,6 +4336,7 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn, break; } +done: l2cap_chan_unlock(chan); l2cap_chan_put(chan); @@ -4439,7 +4469,13 @@ static inline int l2cap_config_req(struct l2cap_conn *conn, goto unlock; } - if (!test_and_set_bit(CONF_REQ_SENT, &chan->conf_state)) { + /* A channel in BT_CONNECT2 has not passed security or userspace + * accept yet. Answer the peer's request, but hold our own until + * the channel reaches BT_CONFIG: the CONNECT2 -> CONFIG sites send + * it. Connected channels may also need to reconfigure. + */ + if ((chan->state == BT_CONFIG || chan->state == BT_CONNECTED) && + !test_and_set_bit(CONF_REQ_SENT, &chan->conf_state)) { u8 buf[64]; l2cap_send_cmd(conn, l2cap_get_ident(conn), L2CAP_CONF_REQ, l2cap_build_conf_req(chan, buf, sizeof(buf)), buf); @@ -4487,6 +4523,10 @@ static inline int l2cap_config_rsp(struct l2cap_conn *conn, if (!chan) return 0; + /* Connected channels may also receive reconfiguration responses. */ + if (chan->state != BT_CONFIG && chan->state != BT_CONNECTED) + goto done; + switch (result) { case L2CAP_CONF_SUCCESS: l2cap_conf_rfc_get(chan, rsp->data, len); @@ -4554,6 +4594,9 @@ static inline int l2cap_config_rsp(struct l2cap_conn *conn, if (test_bit(CONF_OUTPUT_DONE, &chan->conf_state)) { set_default_fcs(chan); + if (chan->state != BT_CONFIG) + goto done; + if (chan->mode == L2CAP_MODE_ERTM || chan->mode == L2CAP_MODE_STREAMING) err = l2cap_ertm_init(chan); diff --git a/net/bluetooth/selftest.c b/net/bluetooth/selftest.c index f49604d44b87..6abf70c14f59 100644 --- a/net/bluetooth/selftest.c +++ b/net/bluetooth/selftest.c @@ -191,6 +191,7 @@ static ssize_t test_ecdh_read(struct file *file, char __user *user_buf, } static const struct file_operations test_ecdh_fops = { + .owner = THIS_MODULE, .open = simple_open, .read = test_ecdh_read, .llseek = default_llseek, diff --git a/net/bluetooth/smp.c b/net/bluetooth/smp.c index c09379796226..be1bde5b8ed6 100644 --- a/net/bluetooth/smp.c +++ b/net/bluetooth/smp.c @@ -3735,6 +3735,7 @@ static ssize_t test_smp_read(struct file *file, char __user *user_buf, } static const struct file_operations test_smp_fops = { + .owner = THIS_MODULE, .open = simple_open, .read = test_smp_read, .llseek = default_llseek, diff --git a/net/can/af_can.c b/net/can/af_can.c index 1172073499d2..694a7741bb54 100644 --- a/net/can/af_can.c +++ b/net/can/af_can.c @@ -224,7 +224,7 @@ int can_send(struct sk_buff *skb, int loop) goto inval_skb; } - if (unlikely(skb->dev->type != ARPHRD_CAN)) { + if (unlikely(!can_get_ml_priv(skb->dev))) { err = -EPERM; goto inval_skb; } @@ -450,7 +450,7 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id, /* insert new receiver (dev,canid,mask) -> (func,data) */ - if (dev && (dev->type != ARPHRD_CAN || !can_get_ml_priv(dev))) + if (dev && !can_get_ml_priv(dev)) return -ENODEV; if (dev && !net_eq(net, dev_net(dev))) @@ -460,9 +460,10 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id, if (!rcv) return -ENOMEM; - spin_lock_bh(&net->can.rcvlists_lock); - dev_rcv_lists = can_dev_rcv_lists_find(net, dev); + + spin_lock_bh(&dev_rcv_lists->lock); + rcv_list = can_rcv_list_find(&can_id, &mask, dev_rcv_lists); rcv->can_id = can_id; @@ -476,6 +477,9 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id, hlist_add_head_rcu(&rcv->list, rcv_list); dev_rcv_lists->entries++; + spin_unlock_bh(&dev_rcv_lists->lock); + + spin_lock_bh(&net->can.rcvlists_lock); rcv_lists_stats->rcv_entries++; rcv_lists_stats->rcv_entries_max = max(rcv_lists_stats->rcv_entries_max, rcv_lists_stats->rcv_entries); @@ -507,6 +511,14 @@ static void can_rx_delete_receiver(struct rcu_head *rp) * * Description: * Removes subscription entry depending on given (subscription) values. + * The receive lists of a CAN device are protected by a per-device lock, + * so the subscription is found even when the device has been moved to + * another network namespace after can_rx_register(). The caller has to + * ensure that @dev stays valid, e.g. by holding a reference to @dev or + * RTNL. @net should be the namespace that has been used for + * can_rx_register(). It selects the 'any' CAN device lists for + * @dev == NULL and is otherwise only used for the receive list + * statistics, which become imprecise when another namespace is passed. */ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id, canid_t mask, void (*func)(struct sk_buff *, void *), @@ -517,15 +529,13 @@ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id, struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats; struct can_dev_rcv_lists *dev_rcv_lists; - if (dev && dev->type != ARPHRD_CAN) + if (dev && !can_get_ml_priv(dev)) return; - if (dev && !net_eq(net, dev_net(dev))) - return; + dev_rcv_lists = can_dev_rcv_lists_find(net, dev); - spin_lock_bh(&net->can.rcvlists_lock); + spin_lock_bh(&dev_rcv_lists->lock); - dev_rcv_lists = can_dev_rcv_lists_find(net, dev); rcv_list = can_rcv_list_find(&can_id, &mask, dev_rcv_lists); /* Search the receiver list for the item to delete. This should @@ -545,26 +555,26 @@ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id, * a warning here. */ if (!rcv) { + spin_unlock_bh(&dev_rcv_lists->lock); pr_warn("can: receive list entry not found for dev %s, id %03X, mask %03X\n", DNAME(dev), can_id, mask); - goto out; + return; } hlist_del_rcu(&rcv->list); dev_rcv_lists->entries--; + spin_unlock_bh(&dev_rcv_lists->lock); + + spin_lock_bh(&net->can.rcvlists_lock); if (rcv_lists_stats->rcv_entries > 0) rcv_lists_stats->rcv_entries--; - - out: spin_unlock_bh(&net->can.rcvlists_lock); /* schedule the receiver item for deletion */ - if (rcv) { - if (rcv->sk) - sock_hold(rcv->sk); - call_rcu(&rcv->rcu, can_rx_delete_receiver); - } + if (rcv->sk) + sock_hold(rcv->sk); + call_rcu(&rcv->rcu, can_rx_delete_receiver); } EXPORT_SYMBOL(can_rx_unregister); @@ -683,7 +693,7 @@ static void can_receive(struct sk_buff *skb, struct net_device *dev) static int can_rcv(struct sk_buff *skb, struct net_device *dev, struct packet_type *pt, struct net_device *orig_dev) { - if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || !can_is_can_skb(skb))) { + if (unlikely(!can_get_ml_priv(dev) || !can_is_can_skb(skb))) { pr_warn_once("PF_CAN: dropped non conform CAN skbuff: dev type %d, len %d\n", dev->type, skb->len); @@ -698,7 +708,7 @@ static int can_rcv(struct sk_buff *skb, struct net_device *dev, static int canfd_rcv(struct sk_buff *skb, struct net_device *dev, struct packet_type *pt, struct net_device *orig_dev) { - if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || !can_is_canfd_skb(skb))) { + if (unlikely(!can_get_ml_priv(dev) || !can_is_canfd_skb(skb))) { pr_warn_once("PF_CAN: dropped non conform CAN FD skbuff: dev type %d, len %d\n", dev->type, skb->len); @@ -713,7 +723,7 @@ static int canfd_rcv(struct sk_buff *skb, struct net_device *dev, static int canxl_rcv(struct sk_buff *skb, struct net_device *dev, struct packet_type *pt, struct net_device *orig_dev) { - if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || !can_is_canxl_skb(skb))) { + if (unlikely(!can_get_ml_priv(dev) || !can_is_canxl_skb(skb))) { pr_warn_once("PF_CAN: dropped non conform CAN XL skbuff: dev type %d, len %d\n", dev->type, skb->len); @@ -795,6 +805,7 @@ static int can_pernet_init(struct net *net) kzalloc(sizeof(*net->can.rx_alldev_list), GFP_KERNEL); if (!net->can.rx_alldev_list) goto out; + spin_lock_init(&net->can.rx_alldev_list->lock); net->can.pkg_stats = kzalloc(sizeof(*net->can.pkg_stats), GFP_KERNEL); if (!net->can.pkg_stats) goto out_free_rx_alldev_list; diff --git a/net/can/bcm.c b/net/can/bcm.c index 0c7e6203eb44..e8d3cb5ce0f1 100644 --- a/net/can/bcm.c +++ b/net/can/bcm.c @@ -54,6 +54,7 @@ #include #include #include +#include #include #include #include @@ -956,7 +957,7 @@ static void bcm_remove_op(struct bcm_op *op) static void bcm_rx_unreg(struct net_device *dev, struct bcm_op *op) { if (op->rx_reg_dev == dev) { - can_rx_unregister(dev_net(dev), dev, op->can_id, + can_rx_unregister(sock_net(op->sk), dev, op->can_id, REGMASK(op->can_id), bcm_rx_handler, op); /* mark as removed subscription */ @@ -982,21 +983,10 @@ static int bcm_delete_rx_op(struct list_head *ops, struct bcm_msg_head *mh, /* disable automatic timer on frame reception */ op->flags |= RX_NO_AUTOTIMER; - /* - * Don't care if we're bound or not (due to netdev - * problems) can_rx_unregister() is always a save - * thing to do here. - */ if (op->ifindex) { - /* - * Only remove subscriptions that had not - * been removed due to NETDEV_UNREGISTER - * in bcm_notifier() - * - * op->rx_reg_dev is a tracked reference taken - * when the subscription was registered, so it - * stays valid here even if a concurrent - * NETDEV_UNREGISTER already unlisted the dev. + /* Only remove subscriptions that had not been + * removed in bcm_notify(). op->rx_reg_dev keeps + * the device's receive lists accessible. */ if (op->rx_reg_dev) bcm_rx_unreg(op->rx_reg_dev, op); @@ -1710,7 +1700,7 @@ static int bcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) goto out_release; } - if (dev->type != ARPHRD_CAN) { + if (!can_get_ml_priv(dev)) { dev_put(dev); ret = -ENODEV; goto out_release; @@ -1858,7 +1848,7 @@ static int bcm_notifier(struct notifier_block *nb, unsigned long msg, { struct net_device *dev = netdev_notifier_info_to_dev(ptr); - if (dev->type != ARPHRD_CAN) + if (!can_get_ml_priv(dev)) return NOTIFY_DONE; if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN) return NOTIFY_DONE; @@ -1926,6 +1916,7 @@ static int bcm_release(struct socket *sock) list_del(&bo->notifier); spin_unlock(&bcm_notifier_lock); + /* bcm_notify() can not access this socket anymore */ lock_sock(sk); #if IS_ENABLED(CONFIG_PROC_FS) @@ -1939,29 +1930,20 @@ static int bcm_release(struct socket *sock) bcm_remove_op(op); } - list_for_each_entry_safe(op, next, &bo->rx_ops, list) { - /* - * Don't care if we're bound or not (due to netdev problems) - * can_rx_unregister() is always a save thing to do here. - */ + list_for_each_entry(op, &bo->rx_ops, list) { if (op->ifindex) { - /* - * Only remove subscriptions that had not - * been removed due to NETDEV_UNREGISTER - * in bcm_notifier() - * - * op->rx_reg_dev is a tracked reference taken - * when the subscription was registered, so it - * stays valid here even if a concurrent - * NETDEV_UNREGISTER already unlisted the device. + /* Only remove subscriptions that had not been removed + * in bcm_notify(). op->rx_reg_dev keeps the device's + * receive lists accessible. */ if (op->rx_reg_dev) bcm_rx_unreg(op->rx_reg_dev, op); - } else + } else { can_rx_unregister(net, NULL, op->can_id, REGMASK(op->can_id), bcm_rx_handler, op); - + } + cond_resched(); } synchronize_rcu(); @@ -2014,7 +1996,7 @@ static int bcm_connect(struct socket *sock, struct sockaddr *uaddr, int len, ret = -ENODEV; goto fail; } - if (dev->type != ARPHRD_CAN) { + if (!can_get_ml_priv(dev)) { dev_put(dev); ret = -ENODEV; goto fail; diff --git a/net/can/gw.c b/net/can/gw.c index 676af9a0b659..1fff84d8e4e1 100644 --- a/net/can/gw.c +++ b/net/can/gw.c @@ -52,6 +52,7 @@ #include #include #include +#include #include #include #include @@ -603,7 +604,7 @@ static int cgw_notifier(struct notifier_block *nb, struct net_device *dev = netdev_notifier_info_to_dev(ptr); struct net *net = dev_net(dev); - if (dev->type != ARPHRD_CAN) + if (!can_get_ml_priv(dev)) return NOTIFY_DONE; if (msg == NETDEV_UNREGISTER) { @@ -1154,7 +1155,7 @@ static int cgw_create_job(struct sk_buff *skb, struct nlmsghdr *nlh, if (!gwj->src.dev) goto out; - if (gwj->src.dev->type != ARPHRD_CAN) + if (!can_get_ml_priv(gwj->src.dev)) goto out; gwj->dst.dev = __dev_get_by_index(net, gwj->ccgw.dst_idx); @@ -1162,7 +1163,7 @@ static int cgw_create_job(struct sk_buff *skb, struct nlmsghdr *nlh, if (!gwj->dst.dev) goto out; - if (gwj->dst.dev->type != ARPHRD_CAN) + if (!can_get_ml_priv(gwj->dst.dev)) goto out; /* is sending the skb back to the incoming interface intended? */ diff --git a/net/can/isotp.c b/net/can/isotp.c index 4c65fa4057a0..c3155d7473ab 100644 --- a/net/can/isotp.c +++ b/net/can/isotp.c @@ -65,6 +65,7 @@ #include #include #include +#include #include #include #include @@ -747,6 +748,14 @@ static void isotp_rcv(struct sk_buff *skb, void *data) if (skb->len != so->ll.mtu) return; + /* check for correct CAN CC/FD frame content */ + if (so->ll.mtu == CAN_MTU) { + if (!can_is_can_skb(skb)) + return; + } else if (!can_is_canfd_skb(skb)) { + return; + } + cf = (struct canfd_frame *)skb->data; /* if enabled: check reception of my configured extended address */ @@ -1104,7 +1113,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) int off; int err; - if (!so->bound || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN) + if (!READ_ONCE(so->bound) || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN) return -EADDRNOTAVAIL; /* claim the socket under so->rx_lock: this serializes the claim @@ -1149,7 +1158,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) /* so->bound is only checked once above - a wakeup may have * unbound/rebound the socket meanwhile => recheck */ - if (!so->bound) { + if (!READ_ONCE(so->bound)) { err = -EADDRNOTAVAIL; goto err_out_drop; } @@ -1389,7 +1398,7 @@ static int isotp_recvmsg(struct socket *sock, struct msghdr *msg, size_t size, if (flags & ~(MSG_DONTWAIT | MSG_TRUNC | MSG_PEEK | MSG_CMSG_COMPAT)) return -EINVAL; - if (!so->bound) + if (!READ_ONCE(so->bound)) return -EADDRNOTAVAIL; skb = skb_recv_datagram(sk, flags, &ret); @@ -1463,11 +1472,11 @@ static int isotp_release(struct socket *sock) list_del(&so->notifier); spin_unlock(&isotp_notifier_lock); - rtnl_lock(); lock_sock(sk); /* remove current filters & unregister - * tracked reference so->dev is taken at bind() time with rtnl_lock + * the tracked reference so->dev keeps the device's receive lists + * accessible, even after a namespace change */ if (so->bound && so->dev) { if (isotp_register_rxid(so)) @@ -1482,11 +1491,9 @@ static int isotp_release(struct socket *sock) } so->ifindex = 0; - so->bound = 0; + WRITE_ONCE(so->bound, 0); so->dev = NULL; - rtnl_unlock(); - /* Always wait for a grace period before touching the timers below. * A concurrent NETDEV_UNREGISTER may have already unregistered our * filters and cleared so->bound in isotp_notify() without waiting @@ -1515,7 +1522,6 @@ static int isotp_bind(struct socket *sock, struct sockaddr *uaddr, int len) struct sock *sk = sock->sk; struct isotp_sock *so = isotp_sk(sk); struct net *net = sock_net(sk); - int ifindex; struct net_device *dev; canid_t tx_id = addr->can_addr.tp.tx_id; canid_t rx_id = addr->can_addr.tp.rx_id; @@ -1553,7 +1559,7 @@ static int isotp_bind(struct socket *sock, struct sockaddr *uaddr, int len) if (!addr->can_ifindex) return -ENODEV; - rtnl_lock(); + /* lock_sock() serializes the binding with isotp_notify() */ lock_sock(sk); if (so->bound) { @@ -1579,61 +1585,83 @@ static int isotp_bind(struct socket *sock, struct sockaddr *uaddr, int len) goto out; } - dev = dev_get_by_index(net, addr->can_ifindex); - if (!dev) { - err = -ENODEV; - goto out; - } - if (dev->type != ARPHRD_CAN) { + /* Publish the device inside the RCU section of the lookup: When it + * is unlisted concurrently, either isotp_notify() sees so->dev after + * synchronize_net() or the lookup fails. Check the device before + * publishing it. + */ + rcu_read_lock(); + dev = dev_get_by_index_rcu(net, addr->can_ifindex); + if (!dev || !can_get_ml_priv(dev)) err = -ENODEV; - goto out_put_dev; - } - if (READ_ONCE(dev->mtu) < so->ll.mtu) { + else if (READ_ONCE(dev->mtu) < so->ll.mtu) err = -EINVAL; - goto out_put_dev; + if (!err) { + dev_hold(dev); + WRITE_ONCE(so->dev, dev); } - if (!(dev->flags & IFF_UP)) - notify_enetdown = 1; - - ifindex = dev->ifindex; + rcu_read_unlock(); + if (err) + goto out; - if (isotp_register_rxid(so)) - can_rx_register(net, dev, rx_id, SINGLE_MASK(rx_id), - isotp_rcv, sk, "isotp", sk); + /* switch to new settings */ + so->ifindex = dev->ifindex; + so->rxid = rx_id; + so->txid = tx_id; /* no consecutive frame echo skb in flight */ WRITE_ONCE(so->cfecho, 0); - /* register for echo skb's */ - can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id), - isotp_rcv_echo, sk, "isotpe", sk); + /* register for echo skb's first (no echo traffic at this point) */ + err = can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id), + isotp_rcv_echo, sk, "isotpe", sk); + if (err) + goto out_put_dev; - /* switch to new settings */ - so->ifindex = ifindex; - so->rxid = rx_id; - so->txid = tx_id; - so->bound = 1; + if (isotp_register_rxid(so)) { + err = can_rx_register(net, dev, rx_id, SINGLE_MASK(rx_id), + isotp_rcv, sk, "isotp", sk); + if (err) { + /* unregister echo skb subscription */ + can_rx_unregister(net, dev, tx_id, SINGLE_MASK(tx_id), + isotp_rcv_echo, sk); + goto out_put_dev; + } + } + + WRITE_ONCE(so->bound, 1); /* bind() ok -> hold a reference for so->dev so that isotp_release() * can safely reach the device later, even if a concurrent * NETDEV_UNREGISTER has already unlisted it by ifindex. */ - so->dev = dev; dev_hold(so->dev); + /* Either isotp_notify() sees so->dev on NETDEV_DOWN or we see the + * cleared IFF_UP, pairs with isotp_notify() + */ + smp_mb(); + if (!(READ_ONCE(dev->flags) & IFF_UP)) + notify_enetdown = 1; + out_put_dev: - /* remove potential reference from dev_get_by_index() */ + /* revert so->ifindex and so->dev on failed bind() */ + if (err) { + so->ifindex = 0; + WRITE_ONCE(so->dev, NULL); + } + + /* remove reference from the device lookup */ dev_put(dev); out: - release_sock(sk); - rtnl_unlock(); - if (notify_enetdown) { sk->sk_err = ENETDOWN; if (!sock_flag(sk, SOCK_DEAD)) sk_error_report(sk); } + release_sock(sk); + return err; } @@ -1655,8 +1683,8 @@ static int isotp_getname(struct socket *sock, struct sockaddr *uaddr, int peer) return ISOTP_MIN_NAMELEN; } -static int isotp_setsockopt_locked(struct socket *sock, int level, int optname, - sockptr_t optval, unsigned int optlen) +static int isotp_setsockopt_locked(struct socket *sock, int optname, + const void *val) { struct sock *sk = sock->sk; struct isotp_sock *so = isotp_sk(sk); @@ -1667,11 +1695,7 @@ static int isotp_setsockopt_locked(struct socket *sock, int level, int optname, switch (optname) { case CAN_ISOTP_OPTS: - if (optlen != sizeof(struct can_isotp_options)) - return -EINVAL; - - if (copy_from_sockptr(&so->opt, optval, optlen)) - return -EFAULT; + memcpy(&so->opt, val, sizeof(so->opt)); /* no separate rx_ext_address is given => use ext_address */ if (!(so->opt.flags & CAN_ISOTP_RX_EXT_ADDR)) @@ -1696,55 +1720,37 @@ static int isotp_setsockopt_locked(struct socket *sock, int level, int optname, break; case CAN_ISOTP_RECV_FC: - if (optlen != sizeof(struct can_isotp_fc_options)) - return -EINVAL; - - if (copy_from_sockptr(&so->rxfc, optval, optlen)) - return -EFAULT; + memcpy(&so->rxfc, val, sizeof(so->rxfc)); break; case CAN_ISOTP_TX_STMIN: - if (optlen != sizeof(u32)) - return -EINVAL; - - if (copy_from_sockptr(&so->force_tx_stmin, optval, optlen)) - return -EFAULT; + so->force_tx_stmin = *(const u32 *)val; break; case CAN_ISOTP_RX_STMIN: - if (optlen != sizeof(u32)) - return -EINVAL; - - if (copy_from_sockptr(&so->force_rx_stmin, optval, optlen)) - return -EFAULT; + so->force_rx_stmin = *(const u32 *)val; break; - case CAN_ISOTP_LL_OPTS: - if (optlen == sizeof(struct can_isotp_ll_options)) { - struct can_isotp_ll_options ll; - - if (copy_from_sockptr(&ll, optval, optlen)) - return -EFAULT; + case CAN_ISOTP_LL_OPTS: { + const struct can_isotp_ll_options *ll = val; - /* check for correct ISO 11898-1 DLC data length */ - if (ll.tx_dl != padlen(ll.tx_dl)) - return -EINVAL; + /* check for correct ISO 11898-1 DLC data length */ + if (ll->tx_dl != padlen(ll->tx_dl)) + return -EINVAL; - if (ll.mtu != CAN_MTU && ll.mtu != CANFD_MTU) - return -EINVAL; + if (ll->mtu != CAN_MTU && ll->mtu != CANFD_MTU) + return -EINVAL; - if (ll.mtu == CAN_MTU && - (ll.tx_dl > CAN_MAX_DLEN || ll.tx_flags != 0)) - return -EINVAL; + if (ll->mtu == CAN_MTU && + (ll->tx_dl > CAN_MAX_DLEN || ll->tx_flags != 0)) + return -EINVAL; - memcpy(&so->ll, &ll, sizeof(ll)); + memcpy(&so->ll, ll, sizeof(*ll)); - /* set ll_dl for tx path to similar place as for rx */ - so->tx.ll_dl = ll.tx_dl; - } else { - return -EINVAL; - } + /* set ll_dl for tx path to similar place as for rx */ + so->tx.ll_dl = ll->tx_dl; break; + } default: ret = -ENOPROTOOPT; @@ -1758,13 +1764,49 @@ static int isotp_setsockopt(struct socket *sock, int level, int optname, { struct sock *sk = sock->sk; + union { + struct can_isotp_options opt; + struct can_isotp_fc_options rxfc; + struct can_isotp_ll_options ll; + u32 stmin; + } val; + unsigned int size; int ret; if (level != SOL_CAN_ISOTP) return -EINVAL; + /* copy the option values from user space before lock_sock() */ + switch (optname) { + case CAN_ISOTP_OPTS: + size = sizeof(val.opt); + break; + + case CAN_ISOTP_RECV_FC: + size = sizeof(val.rxfc); + break; + + case CAN_ISOTP_TX_STMIN: + case CAN_ISOTP_RX_STMIN: + size = sizeof(val.stmin); + break; + + case CAN_ISOTP_LL_OPTS: + size = sizeof(val.ll); + break; + + default: + return -ENOPROTOOPT; + } + + if (optlen != size) + return -EINVAL; + + if (copy_from_sockptr(&val, optval, size)) + return -EFAULT; + lock_sock(sk); - ret = isotp_setsockopt_locked(sock, level, optname, optval, optlen); + ret = isotp_setsockopt_locked(sock, optname, &val); release_sock(sk); return ret; } @@ -1829,12 +1871,21 @@ static void isotp_notify(struct isotp_sock *so, unsigned long msg, if (!net_eq(dev_net(dev), sock_net(sk))) return; - if (so->dev != dev) + if (msg == NETDEV_DOWN) + smp_mb(); /* pairs with isotp_bind(), see raw_notify() */ + + /* lockless fast path, see raw_notify() */ + if (READ_ONCE(so->dev) != dev) return; switch (msg) { case NETDEV_UNREGISTER: lock_sock(sk); + if (so->dev != dev) { + release_sock(sk); + break; + } + /* remove current filters & unregister */ if (so->bound) { if (isotp_register_rxid(so)) @@ -1849,16 +1900,19 @@ static void isotp_notify(struct isotp_sock *so, unsigned long msg, } so->ifindex = 0; - so->bound = 0; - so->dev = NULL; - release_sock(sk); + WRITE_ONCE(so->bound, 0); + WRITE_ONCE(so->dev, NULL); sk->sk_err = ENODEV; if (!sock_flag(sk, SOCK_DEAD)) sk_error_report(sk); + release_sock(sk); break; case NETDEV_DOWN: + if (!READ_ONCE(so->bound)) + break; + sk->sk_err = ENETDOWN; if (!sock_flag(sk, SOCK_DEAD)) sk_error_report(sk); @@ -1871,7 +1925,7 @@ static int isotp_notifier(struct notifier_block *nb, unsigned long msg, { struct net_device *dev = netdev_notifier_info_to_dev(ptr); - if (dev->type != ARPHRD_CAN) + if (!can_get_ml_priv(dev)) return NOTIFY_DONE; if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN) return NOTIFY_DONE; diff --git a/net/can/proc.c b/net/can/proc.c index 2f78ea8ac30b..42145a157335 100644 --- a/net/can/proc.c +++ b/net/can/proc.c @@ -76,16 +76,39 @@ static const char rx_list_name[][8] = { * af_can statistics stuff */ +static void can_reset_pkg_stats(struct can_pkg_stats *pkg_stats) +{ + atomic_long_set(&pkg_stats->rx_frames, 0); + atomic_long_set(&pkg_stats->tx_frames, 0); + atomic_long_set(&pkg_stats->matches, 0); + + pkg_stats->total_rx_rate = 0; + pkg_stats->total_tx_rate = 0; + pkg_stats->total_rx_match_ratio = 0; + + pkg_stats->current_rx_rate = 0; + pkg_stats->current_tx_rate = 0; + pkg_stats->current_rx_match_ratio = 0; + + pkg_stats->max_rx_rate = 0; + pkg_stats->max_tx_rate = 0; + pkg_stats->max_rx_match_ratio = 0; + + atomic_long_set(&pkg_stats->rx_frames_delta, 0); + atomic_long_set(&pkg_stats->tx_frames_delta, 0); + atomic_long_set(&pkg_stats->matches_delta, 0); +} + static void can_init_stats(struct net *net) { struct can_pkg_stats *pkg_stats = net->can.pkg_stats; struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats; /* - * This memset function is called from a timer context (when + * This stats reset is called from a timer context (when * can_stattimer is active which is the default) OR in a process * context (reading the proc_fs when can_stattimer is disabled). */ - memset(pkg_stats, 0, sizeof(struct can_pkg_stats)); + can_reset_pkg_stats(pkg_stats); pkg_stats->jiffies_init = jiffies; rcv_lists_stats->stats_reset++; @@ -121,29 +144,23 @@ void can_stat_update(struct timer_list *t) long rx_frames = atomic_long_read(&pkg_stats->rx_frames); long tx_frames = atomic_long_read(&pkg_stats->tx_frames); long matches = atomic_long_read(&pkg_stats->matches); - long rx_frames_delta = atomic_long_read(&pkg_stats->rx_frames_delta); - long tx_frames_delta = atomic_long_read(&pkg_stats->tx_frames_delta); - long matches_delta = atomic_long_read(&pkg_stats->matches_delta); + /* fetch and clear values for 'current rate' calculation */ + long rx_frames_delta = atomic_long_xchg(&pkg_stats->rx_frames_delta, 0); + long tx_frames_delta = atomic_long_xchg(&pkg_stats->tx_frames_delta, 0); + long matches_delta = atomic_long_xchg(&pkg_stats->matches_delta, 0); - /* restart counting in timer context on user request */ - if (user_reset) - can_init_stats(net); - - /* restart counting on jiffies overflow */ - if (j < pkg_stats->jiffies_init) - can_init_stats(net); - - /* prevent overflow in calc_rate() */ - if (rx_frames > (LONG_MAX / HZ)) - can_init_stats(net); - - /* prevent overflow in calc_rate() */ - if (tx_frames > (LONG_MAX / HZ)) - can_init_stats(net); - - /* matches overflow - very improbable */ - if (matches > (LONG_MAX / 100)) + /* + * Restart counting in timer context on user request, on jiffies + * overflow or to prevent overflows in calc_rate() and in the + * match ratio calculation. The local snapshots are stale after + * the reset, so skip the calculation for this interval. + */ + if (user_reset || j < pkg_stats->jiffies_init || + rx_frames > (LONG_MAX / HZ) || tx_frames > (LONG_MAX / HZ) || + matches > (LONG_MAX / 100)) { can_init_stats(net); + goto out; + } /* calc total values */ if (rx_frames) @@ -172,11 +189,7 @@ void can_stat_update(struct timer_list *t) if (pkg_stats->max_rx_match_ratio < pkg_stats->current_rx_match_ratio) pkg_stats->max_rx_match_ratio = pkg_stats->current_rx_match_ratio; - /* clear values for 'current rate' calculation */ - atomic_long_set(&pkg_stats->tx_frames_delta, 0); - atomic_long_set(&pkg_stats->rx_frames_delta, 0); - atomic_long_set(&pkg_stats->matches_delta, 0); - +out: /* restart timer (one second) */ mod_timer(&net->can.stattimer, round_jiffies(jiffies + HZ)); } diff --git a/net/can/raw.c b/net/can/raw.c index a5a0f69cd7ae..fd9f71ee32e9 100644 --- a/net/can/raw.c +++ b/net/can/raw.c @@ -49,6 +49,7 @@ #include #include #include +#include #include #include /* for can_is_canxl_dev_mtu() */ #include @@ -86,6 +87,7 @@ struct raw_sock { int ifindex; struct net_device *dev; netdevice_tracker dev_tracker; + struct net_device *bind_dev; /* hint for raw_notify(), no reference */ struct list_head notifier; int loopback; int recv_own_msgs; @@ -217,6 +219,7 @@ static int raw_enable_filters(struct net *net, struct net_device *dev, int i; for (i = 0; i < count; i++) { + cond_resched(); err = can_rx_register(net, dev, filter[i].can_id, filter[i].can_mask, raw_rcv, sk, "raw", sk); @@ -251,9 +254,11 @@ static void raw_disable_filters(struct net *net, struct net_device *dev, { int i; - for (i = 0; i < count; i++) + for (i = 0; i < count; i++) { can_rx_unregister(net, dev, filter[i].can_id, filter[i].can_mask, raw_rcv, sk); + cond_resched(); + } } static inline void raw_disable_errfilter(struct net *net, @@ -298,16 +303,31 @@ static void raw_notify(struct raw_sock *ro, unsigned long msg, struct net_device *dev) { struct sock *sk = &ro->sk; + struct net_device *bind_dev; if (!net_eq(dev_net(dev), sock_net(sk))) return; - if (ro->dev != dev) + if (msg == NETDEV_DOWN) + smp_mb(); /* IFF_UP is cleared before, pairs with raw_bind() */ + + /* Lockless fast path, ro->bind_dev covers a concurrent bind(). + * Read it before ro->dev, pairs with smp_store_release() in + * raw_bind(). A stale ro->bind_dev only costs a lock_sock(). + */ + bind_dev = smp_load_acquire(&ro->bind_dev); + if (READ_ONCE(ro->dev) != dev && + (msg != NETDEV_UNREGISTER || bind_dev != dev)) return; switch (msg) { case NETDEV_UNREGISTER: lock_sock(sk); + if (ro->dev != dev) { + release_sock(sk); + break; + } + /* remove current filters & unregister */ if (ro->bound) { raw_disable_allfilters(dev_net(dev), dev, sk); @@ -319,13 +339,13 @@ static void raw_notify(struct raw_sock *ro, unsigned long msg, ro->ifindex = 0; ro->bound = 0; - ro->dev = NULL; + WRITE_ONCE(ro->dev, NULL); ro->count = 0; - release_sock(sk); sk->sk_err = ENODEV; if (!sock_flag(sk, SOCK_DEAD)) sk_error_report(sk); + release_sock(sk); break; case NETDEV_DOWN: @@ -341,7 +361,7 @@ static int raw_notifier(struct notifier_block *nb, unsigned long msg, { struct net_device *dev = netdev_notifier_info_to_dev(ptr); - if (dev->type != ARPHRD_CAN) + if (!can_get_ml_priv(dev)) return NOTIFY_DONE; if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN) return NOTIFY_DONE; @@ -374,6 +394,7 @@ static int raw_init(struct sock *sk) ro->bound = 0; ro->ifindex = 0; ro->dev = NULL; + ro->bind_dev = NULL; /* set default filter to single entry dfilter */ ro->dfilter.can_id = 0; @@ -422,13 +443,15 @@ static int raw_release(struct socket *sock) list_del(&ro->notifier); spin_unlock(&raw_notifier_lock); - rtnl_lock(); lock_sock(sk); - /* remove current filters & unregister */ + /* remove current filters & unregister + * the tracked reference ro->dev keeps the device's receive lists + * accessible, even after a namespace change + */ if (ro->bound) { if (ro->dev) { - raw_disable_allfilters(dev_net(ro->dev), ro->dev, sk); + raw_disable_allfilters(sock_net(sk), ro->dev, sk); netdev_put(ro->dev, &ro->dev_tracker); } else { raw_disable_allfilters(sock_net(sk), NULL, sk); @@ -447,7 +470,6 @@ static int raw_release(struct socket *sock) sock->sk = NULL; release_sock(sk); - rtnl_unlock(); sock_put(sk); @@ -469,26 +491,34 @@ static int raw_bind(struct socket *sock, struct sockaddr *uaddr, int len) if (addr->can_family != AF_CAN) return -EINVAL; - rtnl_lock(); + /* lock_sock() serializes the binding with raw_notify() */ lock_sock(sk); if (ro->bound && addr->can_ifindex == ro->ifindex) goto out; if (addr->can_ifindex) { - dev = dev_get_by_index(sock_net(sk), addr->can_ifindex); + /* Publish the device inside the RCU section of the lookup: + * When it is unlisted concurrently, either raw_notify() sees + * ro->bind_dev after synchronize_net() or the lookup fails. + */ + rcu_read_lock(); + dev = dev_get_by_index_rcu(sock_net(sk), addr->can_ifindex); + if (dev) { + dev_hold(dev); + /* pairs with smp_load_acquire() in raw_notify() */ + smp_store_release(&ro->bind_dev, dev); + } + rcu_read_unlock(); if (!dev) { err = -ENODEV; goto out; } - if (dev->type != ARPHRD_CAN) { + if (!can_get_ml_priv(dev)) { err = -ENODEV; goto out_put_dev; } - if (!(dev->flags & IFF_UP)) - notify_enetdown = 1; - ifindex = dev->ifindex; /* filters set by default/setsockopt */ @@ -507,7 +537,7 @@ static int raw_bind(struct socket *sock, struct sockaddr *uaddr, int len) if (ro->bound) { /* unregister old filters */ if (ro->dev) { - raw_disable_allfilters(dev_net(ro->dev), + raw_disable_allfilters(sock_net(sk), ro->dev, sk); /* drop reference to old ro->dev */ netdev_put(ro->dev, &ro->dev_tracker); @@ -518,24 +548,32 @@ static int raw_bind(struct socket *sock, struct sockaddr *uaddr, int len) ro->ifindex = ifindex; ro->bound = 1; /* bind() ok -> hold a reference for new ro->dev */ - ro->dev = dev; + WRITE_ONCE(ro->dev, dev); if (ro->dev) netdev_hold(ro->dev, &ro->dev_tracker, GFP_KERNEL); + + if (dev) { + /* Either raw_notify() sees ro->dev on NETDEV_DOWN or + * we see the cleared IFF_UP, pairs with raw_notify() + */ + smp_mb(); + if (!(READ_ONCE(dev->flags) & IFF_UP)) + notify_enetdown = 1; + } } out_put_dev: - /* remove potential reference from dev_get_by_index() */ + /* remove reference from the device lookup */ dev_put(dev); out: - release_sock(sk); - rtnl_unlock(); - if (notify_enetdown) { sk->sk_err = ENETDOWN; if (!sock_flag(sk, SOCK_DEAD)) sk_error_report(sk); } + release_sock(sk); + return err; } @@ -592,17 +630,19 @@ static int raw_setsockopt(struct socket *sock, int level, int optname, return -EFAULT; } - rtnl_lock(); lock_sock(sk); + /* The reg_state check is only a shortcut: On a concurrent + * unregistration or namespace change raw_notify() waits for + * lock_sock() and removes the filters set here. + */ dev = ro->dev; - if (ro->bound && dev) { - if (dev->reg_state != NETREG_REGISTERED) { - if (count > 1) - kfree(filter); - err = -ENODEV; - goto out_fil; - } + if (ro->bound && dev && + READ_ONCE(dev->reg_state) != NETREG_REGISTERED) { + if (count > 1) + kfree(filter); + err = -ENODEV; + goto out_fil; } if (ro->bound) { @@ -639,7 +679,6 @@ static int raw_setsockopt(struct socket *sock, int level, int optname, out_fil: release_sock(sk); - rtnl_unlock(); break; @@ -652,15 +691,14 @@ static int raw_setsockopt(struct socket *sock, int level, int optname, err_mask &= CAN_ERR_MASK; - rtnl_lock(); lock_sock(sk); + /* lockless shortcut, see CAN_RAW_FILTER */ dev = ro->dev; - if (ro->bound && dev) { - if (dev->reg_state != NETREG_REGISTERED) { - err = -ENODEV; - goto out_err; - } + if (ro->bound && dev && + READ_ONCE(dev->reg_state) != NETREG_REGISTERED) { + err = -ENODEV; + goto out_err; } /* remove current error mask */ @@ -682,7 +720,6 @@ static int raw_setsockopt(struct socket *sock, int level, int optname, out_err: release_sock(sk); - rtnl_unlock(); break; @@ -775,29 +812,40 @@ static int raw_getsockopt(struct socket *sock, int level, int optname, switch (optname) { case CAN_RAW_FILTER: { + struct can_filter *filter = NULL; + int fsize = 0; int err = 0; lock_sock(sk); if (ro->count > 0) { - int fsize = ro->count * sizeof(struct can_filter); + fsize = ro->count * sizeof(struct can_filter); /* user space buffer to small for filter list? */ if (len < fsize) { /* return -ERANGE and needed space in optlen */ err = -ERANGE; - if (put_user(fsize, optlen)) - err = -EFAULT; } else { if (len > fsize) len = fsize; - if (copy_to_user(optval, ro->filter, len)) - err = -EFAULT; + /* copy to user space after release_sock() */ + filter = kmemdup(ro->filter, len, GFP_KERNEL); + if (!filter) + err = -ENOMEM; } } else { len = 0; } release_sock(sk); + if (err == -ERANGE) { + if (put_user(fsize, optlen)) + err = -EFAULT; + } else if (filter) { + if (copy_to_user(optval, filter, len)) + err = -EFAULT; + kfree(filter); + } + if (!err) err = put_user(len, optlen); return err; diff --git a/net/core/dev.c b/net/core/dev.c index 397fd0f9cc7a..f42ec2837143 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -3890,7 +3890,9 @@ static enum skb_drop_reason qdisc_pkt_len_init(struct sk_buff *skb) if (likely(shinfo->gso_type & (SKB_GSO_TCPV4 | SKB_GSO_TCPV6))) { const struct tcphdr *th; - if (!pskb_may_pull(skb, hdr_len + sizeof(struct tcphdr))) + /* A header in front of skb->data gives a negative offset. */ + if ((int)hdr_len < 0 || + !pskb_may_pull(skb, hdr_len + sizeof(struct tcphdr))) return SKB_DROP_REASON_SKB_BAD_GSO; th = (const struct tcphdr *)(skb->data + hdr_len); @@ -5760,6 +5762,8 @@ static int __netif_receive_skb_core(struct sk_buff **pskb, bool pfmemalloc, } if (unlikely(skb_vlan_tag_present(skb)) && !netdev_uses_dsa(skb->dev)) { + unsigned int priority_tag_depth = 1; + check_vlan_id: if (skb_vlan_tag_get_id(skb)) { /* Vlan id is non 0 and vlan_do_receive() above couldn't @@ -5771,6 +5775,10 @@ static int __netif_receive_skb_core(struct sk_buff **pskb, bool pfmemalloc, * 802.1Q or 802.1AD and vlan_do_receive() above could * not find vlan dev for vlan id 0. */ + if (unlikely(priority_tag_depth >= VLAN_MAX_DEPTH)) + goto drop; + + priority_tag_depth++; __vlan_hwaccel_clear_tag(skb); skb = skb_vlan_untag(skb); if (unlikely(!skb)) diff --git a/net/core/filter.c b/net/core/filter.c index ce290de4b165..2d6e2f772a60 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -2278,9 +2278,13 @@ static int bpf_out_neigh_v6(struct net *net, struct sk_buff *skb, static int __bpf_redirect_neigh_v6(struct sk_buff *skb, struct net_device *dev, struct bpf_nh_params *nh) { - const struct ipv6hdr *ip6h = ipv6_hdr(skb); struct net *net = dev_net(dev); int err, ret = NET_XMIT_DROP; + const struct ipv6hdr *ip6h; + + if (unlikely(!pskb_may_pull(skb, sizeof(*ip6h)))) + goto out_drop; + ip6h = ipv6_hdr(skb); if (!nh) { struct dst_entry *dst; @@ -2383,9 +2387,13 @@ static int bpf_out_neigh_v4(struct net *net, struct sk_buff *skb, static int __bpf_redirect_neigh_v4(struct sk_buff *skb, struct net_device *dev, struct bpf_nh_params *nh) { - const struct iphdr *ip4h = ip_hdr(skb); struct net *net = dev_net(dev); int err, ret = NET_XMIT_DROP; + const struct iphdr *ip4h; + + if (unlikely(!pskb_may_pull(skb, sizeof(*ip4h)))) + goto out_drop; + ip4h = ip_hdr(skb); if (!nh) { struct flowi4 fl4 = { @@ -2435,15 +2443,14 @@ static int __bpf_redirect_neigh_v4(struct sk_buff *skb, struct net_device *dev, static int __bpf_redirect_neigh(struct sk_buff *skb, struct net_device *dev, struct bpf_nh_params *nh) { - struct ethhdr *ethh = eth_hdr(skb); - - if (unlikely(skb->mac_header >= skb->network_header)) + if (unlikely(skb_network_offset(skb) < ETH_HLEN || + !pskb_may_pull(skb, ETH_HLEN))) goto out; bpf_push_mac_rcsum(skb); - if (is_multicast_ether_addr(ethh->h_dest)) + if (is_multicast_ether_addr(eth_hdr(skb)->h_dest)) goto out; - skb_pull(skb, sizeof(*ethh)); + skb_pull(skb, ETH_HLEN); skb_unset_mac_header(skb); skb_reset_network_header(skb); @@ -7618,7 +7625,7 @@ BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags) if (sk_unhashed(sk)) return -EOPNOTSUPP; if (sk_is_refcounted(sk) && - unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt))) return -ENOENT; skb_orphan(skb); diff --git a/net/core/sock.c b/net/core/sock.c index 181a66488ec2..6a718b8e1dde 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -3131,7 +3131,7 @@ void __release_sock(struct sock *sk) spin_unlock_bh(&sk->sk_lock.slock); - while (1) { + do { next = skb->next; prefetch(next); DEBUG_NET_WARN_ON_ONCE(skb_dst_is_noref(skb)); @@ -3139,12 +3139,10 @@ void __release_sock(struct sock *sk) sk_backlog_rcv(sk, skb); skb = next; - if (!skb) - break; if (!(++nb & 15)) cond_resched(); - } + } while (skb); spin_lock_bh(&sk->sk_lock.slock); } diff --git a/net/ipv4/cipso_ipv4.c b/net/ipv4/cipso_ipv4.c index 53fcb2089cde..019c16d6d695 100644 --- a/net/ipv4/cipso_ipv4.c +++ b/net/ipv4/cipso_ipv4.c @@ -2015,6 +2015,10 @@ static int cipso_v4_delopt(struct ip_options_rcu __rcu **opt_ptr) cipso_ptr = &opt->opt.__data[cipso_off]; cipso_len = cipso_ptr[1]; + optlen_new = cipso_v4_get_actual_opt_len(opt->opt.__data, + opt->opt.optlen); + optlen_new -= cipso_len; + if (opt->opt.srr > opt->opt.cipso) opt->opt.srr -= cipso_len; if (opt->opt.rr > opt->opt.cipso) @@ -2028,11 +2032,12 @@ static int cipso_v4_delopt(struct ip_options_rcu __rcu **opt_ptr) memmove(cipso_ptr, cipso_ptr + cipso_len, opt->opt.optlen - cipso_off - cipso_len); - optlen_new = cipso_v4_get_actual_opt_len(opt->opt.__data, - opt->opt.optlen); hdr_delta = opt->opt.optlen; opt->opt.optlen = (optlen_new + 3) & ~3; hdr_delta -= opt->opt.optlen; + + memset(&opt->opt.__data[optlen_new], IPOPT_END, + opt->opt.optlen - optlen_new); } else { /* only the cipso option was present on the socket so we can * remove the entire option struct */ diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c index 2cb125bbfb9d..61fbcaa913b3 100644 --- a/net/ipv4/inet_connection_sock.c +++ b/net/ipv4/inet_connection_sock.c @@ -1105,8 +1105,8 @@ static void reqsk_timer_handler(struct timer_list *t) * hold another count to prevent use-after-free and * call reqsk_put() just before return. */ - refcount_set(&nreq->rsk_refcnt, 2 + 1); timer_setup(&nreq->rsk_timer, reqsk_timer_handler, TIMER_PINNED); + refcount_set_release(&nreq->rsk_refcnt, 2 + 1); reqsk_queue_migrated(&inet_csk(nsk)->icsk_accept_queue, req); req = nreq; @@ -1465,7 +1465,7 @@ struct sock *inet_csk_complete_hashdance(struct sock *sk, struct sock *child, goto child_put; } - refcount_set(&nreq->rsk_refcnt, 1); + refcount_set_release(&nreq->rsk_refcnt, 1); if (inet_csk_reqsk_queue_add(sk, nreq, child)) { __NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMIGRATEREQSUCCESS); reqsk_migrate_reset(req); @@ -1519,7 +1519,7 @@ void inet_csk_listen_stop(struct sock *sk) if (nsk) { nreq = inet_reqsk_clone(req, nsk); if (nreq) { - refcount_set(&nreq->rsk_refcnt, 1); + refcount_set_release(&nreq->rsk_refcnt, 1); rcu_read_lock(); if (inet_csk_reqsk_queue_add(nsk, nreq, child)) { diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c index e8adf8530596..e7b312f7cd98 100644 --- a/net/ipv4/inet_hashtables.c +++ b/net/ipv4/inet_hashtables.c @@ -510,7 +510,7 @@ struct sock *__inet_lookup_established(const struct net *net, if (sk->sk_hash != hash) continue; if (likely(inet_match(net, sk, acookie, ports, dif, sdif))) { - if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt))) goto out; if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) || !inet_match(net, sk, acookie, ports, dif, diff --git a/net/ipv4/inet_timewait_sock.c b/net/ipv4/inet_timewait_sock.c index 74b84ac418e9..e34deccdcb30 100644 --- a/net/ipv4/inet_timewait_sock.c +++ b/net/ipv4/inet_timewait_sock.c @@ -139,7 +139,7 @@ void inet_twsk_hashdance_schedule(struct inet_timewait_sock *tw, * Also note that after this point, we lost our implicit reference * so we are not allowed to use tw anymore. */ - refcount_set(&tw->tw_refcnt, 3); + refcount_set_release(&tw->tw_refcnt, 3); /* Ensure tw_refcnt has been set before tw is published. * smp_wmb() provides the necessary memory barrier to enforce this @@ -324,7 +324,7 @@ void inet_twsk_purge(struct inet_hashinfo *hashinfo) if (refcount_read(&sock_net(sk)->ns.count)) continue; - if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt))) continue; if (refcount_read(&sock_net(sk)->ns.count)) { diff --git a/net/ipv4/ip_gre.c b/net/ipv4/ip_gre.c index 5bf6c6777c34..f4a269938340 100644 --- a/net/ipv4/ip_gre.c +++ b/net/ipv4/ip_gre.c @@ -1362,6 +1362,7 @@ static int erspan_tunnel_init(struct net_device *dev) dev->features |= GRE_FEATURES; dev->hw_features |= GRE_FEATURES; dev->priv_flags |= IFF_LIVE_ADDR_CHANGE; + dev->lltx = true; netif_keep_dst(dev); return ip_tunnel_init(dev); diff --git a/net/ipv4/ipmr.c b/net/ipv4/ipmr.c index abd234f0861f..f9cec6b2bc1a 100644 --- a/net/ipv4/ipmr.c +++ b/net/ipv4/ipmr.c @@ -513,15 +513,22 @@ static netdev_tx_t reg_vif_xmit(struct sk_buff *skb, struct net_device *dev) }; int err; + if (skb->protocol != htons(ETH_P_IP) || !pskb_inet_may_pull(skb)) { + DEV_STATS_INC(dev, tx_errors); + kfree_skb(skb); + return NETDEV_TX_OK; + } + + rcu_read_lock(); err = ipmr_fib_lookup(net, &fl4, &mrt); if (err < 0) { + rcu_read_unlock(); kfree_skb(skb); return err; } DEV_STATS_ADD(dev, tx_bytes, skb->len); DEV_STATS_INC(dev, tx_packets); - rcu_read_lock(); /* Pairs with WRITE_ONCE() in vif_add() and vif_delete() */ ipmr_cache_report(mrt, skb, READ_ONCE(mrt->mroute_reg_vif_num), @@ -1031,6 +1038,7 @@ static void ipmr_cache_resolve(struct net *net, struct mr_table *mrt, static int ipmr_cache_report(const struct mr_table *mrt, struct sk_buff *pkt, vifi_t vifi, int assert) { + unsigned char pkt_cb[sizeof(pkt->cb)]; const int ihl = ip_hdrlen(pkt); struct sock *mroute_sk; struct igmphdr *igmp; @@ -1086,17 +1094,25 @@ static int ipmr_cache_report(const struct mr_table *mrt, msg = (struct igmpmsg *)skb_network_header(skb); msg->im_vif = vifi; msg->im_vif_hi = vifi >> 8; - ipv4_pktinfo_prepare(mroute_sk, pkt, false); - memcpy(skb->cb, pkt->cb, sizeof(skb->cb)); - /* Add our header */ - igmp = skb_put(skb, sizeof(struct igmphdr)); + /* Add our header. + * Note that code, csum and group fields are cleared. + */ + igmp = skb_put_zero(skb, sizeof(struct igmphdr)); igmp->type = assert; msg->im_msgtype = assert; - igmp->code = 0; ip_hdr(skb)->tot_len = htons(skb->len); /* Fix the length */ skb->transport_header = skb->network_header; } + memset(skb->cb, 0, sizeof(skb->cb)); + /* Route queries from ipmr_get_route() carry NETLINK_CB, not IPCB. */ + if (ip_hdr(pkt)->version) { + memcpy(pkt_cb, pkt->cb, sizeof(pkt_cb)); + ipv4_pktinfo_prepare(mroute_sk, pkt, false); + *PKTINFO_SKB_CB(skb) = *PKTINFO_SKB_CB(pkt); + memcpy(pkt->cb, pkt_cb, sizeof(pkt_cb)); + } + igmpmsg_netlink_event(mrt, skb); /* Deliver to mrouted */ @@ -1991,7 +2007,8 @@ static void ip_mr_forward(struct net *net, struct mr_table *mrt, } /* Wrong interface: drop packet and (maybe) send PIM assert. */ - if (rcu_access_pointer(mrt->vif_table[vif].dev) != dev) { + if (true_vifi < 0 || + rcu_access_pointer(mrt->vif_table[vif].dev) != dev) { if (rt_is_output_route(skb_rtable(skb))) { /* It is our own packet, looped back. * Very complicated situation... diff --git a/net/ipv4/route.c b/net/ipv4/route.c index 596c84fd5ab4..f1b0ab77b443 100644 --- a/net/ipv4/route.c +++ b/net/ipv4/route.c @@ -1358,8 +1358,8 @@ static unsigned int ipv4_default_advmss(const struct dst_entry *dst) rcu_read_lock(); net = dst_dev_net_rcu(dst); - advmss = max_t(unsigned int, ip_dst_mtu_configured(dst) - header_size, - net->ipv4.ip_rt_min_advmss); + advmss = max_t(unsigned int, ipv4_mtu(dst) - header_size, + net->ipv4.ip_rt_min_advmss); rcu_read_unlock(); return min(advmss, IPV4_MAX_PMTU - header_size); diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c index 16361ce3e806..43cf33f282c1 100644 --- a/net/ipv4/syncookies.c +++ b/net/ipv4/syncookies.c @@ -204,7 +204,7 @@ struct sock *tcp_get_cookie_sock(struct sock *sk, struct sk_buff *skb, child = icsk->icsk_af_ops->syn_recv_sock(sk, skb, req, dst, NULL, &own_req, NULL); if (child) { - refcount_set(&req->rsk_refcnt, 1); + refcount_set_release(&req->rsk_refcnt, 1); sock_rps_save_rxhash(child, skb); if (rsk_drop_req(req)) { diff --git a/net/ipv4/tcp_fastopen.c b/net/ipv4/tcp_fastopen.c index 5378ef3b6ab5..fbcd7d8c8d99 100644 --- a/net/ipv4/tcp_fastopen.c +++ b/net/ipv4/tcp_fastopen.c @@ -278,7 +278,7 @@ static struct sock *tcp_fastopen_create_child(struct sock *sk, inet_csk_reset_xmit_timer(child, ICSK_TIME_RETRANS, req->timeout, TCP_RTO_MAX); - refcount_set(&req->rsk_refcnt, 2); + refcount_set_release(&req->rsk_refcnt, 2); sk_mark_napi_id_set(child, skb); diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c index eb32a0097844..60b2ed8b8fe7 100644 --- a/net/ipv6/addrconf.c +++ b/net/ipv6/addrconf.c @@ -1477,7 +1477,8 @@ static int ipv6_create_tempaddr(struct inet6_ifaddr *ifp, bool block) age = (now - tmp_tstamp + ADDRCONF_TIMER_FUZZ_MINUS) / HZ; if (cfg.preferred_lft <= regen_advance + age) { cfg.preferred_lft = regen_advance + age + 1; - if (cfg.preferred_lft > cfg.valid_lft || + if (cfg.preferred_lft <= regen_advance + age || + cfg.preferred_lft > cfg.valid_lft || cfg.preferred_lft > if_public_preferred_lft) { in6_ifa_put(ifp); in6_dev_put(idev); diff --git a/net/ipv6/datagram.c b/net/ipv6/datagram.c index 9bcec0828fe8..92b772ea224e 100644 --- a/net/ipv6/datagram.c +++ b/net/ipv6/datagram.c @@ -148,6 +148,7 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr *uaddr, __be32 fl6_flowlabel = 0; __be32 old_fl6_flowlabel; __be16 old_dport; + int old_bound_dev_if; int addr_type; int err; @@ -218,6 +219,8 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr *uaddr, goto out; } + old_bound_dev_if = sk->sk_bound_dev_if; + if (__ipv6_addr_needs_scope_id(addr_type)) { if (addr_len >= sizeof(struct sockaddr_in6) && usin->sin6_scope_id) { @@ -260,6 +263,7 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr *uaddr, sk->sk_v6_daddr = old_daddr; np->flow_label = old_fl6_flowlabel; inet->inet_dport = old_dport; + WRITE_ONCE(sk->sk_bound_dev_if, old_bound_dev_if); goto out; } diff --git a/net/ipv6/inet6_hashtables.c b/net/ipv6/inet6_hashtables.c index 70633fed6e6a..dee00ba290f6 100644 --- a/net/ipv6/inet6_hashtables.c +++ b/net/ipv6/inet6_hashtables.c @@ -71,7 +71,7 @@ struct sock *__inet6_lookup_established(const struct net *net, continue; if (!inet6_match(net, sk, saddr, daddr, ports, dif, sdif)) continue; - if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt))) goto out; if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) || diff --git a/net/ipv6/ip6_gre.c b/net/ipv6/ip6_gre.c index 7cf429f11a1d..9cb3c1fea4c9 100644 --- a/net/ipv6/ip6_gre.c +++ b/net/ipv6/ip6_gre.c @@ -1902,6 +1902,7 @@ static int ip6erspan_tap_init(struct net_device *dev) dev->mtu -= 8; dev->priv_flags |= IFF_LIVE_ADDR_CHANGE; + dev->lltx = true; ip6erspan_tnl_link_config(tunnel, 1); netdev_hold(dev, &tunnel->dev_tracker, GFP_KERNEL); diff --git a/net/ipv6/ip6mr.c b/net/ipv6/ip6mr.c index 47afc8cef688..0ecb6af344fb 100644 --- a/net/ipv6/ip6mr.c +++ b/net/ipv6/ip6mr.c @@ -616,7 +616,7 @@ static netdev_tx_t reg_vif_xmit(struct sk_buff *skb, .flowi6_mark = skb->mark, }; - if (!pskb_inet_may_pull(skb)) + if (skb->protocol != htons(ETH_P_IPV6) || !pskb_inet_may_pull(skb)) goto tx_err; if (ip6mr_fib_lookup(net, &fl6, &mrt) < 0) @@ -2146,7 +2146,8 @@ static void ip6_mr_forward(struct net *net, struct mr_table *mrt, /* * Wrong interface: drop packet and (maybe) send PIM assert. */ - if (rcu_access_pointer(mrt->vif_table[vif].dev) != dev) { + if (true_vifi < 0 || + rcu_access_pointer(mrt->vif_table[vif].dev) != dev) { atomic_long_inc(&c->_c.mfc_un.res.wrong_if); if (true_vifi >= 0 && mrt->mroute_do_assert && diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c index deaaae63bda0..27bb9443c884 100644 --- a/net/ipv6/raw.c +++ b/net/ipv6/raw.c @@ -652,6 +652,9 @@ static int rawv6_send_hdrinc(struct sock *sk, struct msghdr *msg, int length, if (unlikely(!skb)) return 0; + /* l3mdev_ip6_out() may have replaced the skb dst. */ + rt = dst_rt6_info(skb_dst(skb)); + /* Acquire rcu_read_lock() in case we need to use rt->rt6i_idev * in the error path. Since skb has been freed, the dst could * have been queued for deletion. diff --git a/net/ipv6/route.c b/net/ipv6/route.c index 94fa1b34be5c..e6c2db89860c 100644 --- a/net/ipv6/route.c +++ b/net/ipv6/route.c @@ -3252,7 +3252,7 @@ EXPORT_SYMBOL_GPL(ip6_sk_redirect); static unsigned int ip6_default_advmss(const struct dst_entry *dst) { struct net_device *dev = dst_dev(dst); - unsigned int mtu = ip6_dst_mtu_configured(dst); + unsigned int mtu = dst6_mtu(dst); struct net *net; mtu -= sizeof(struct ipv6hdr) + sizeof(struct tcphdr); diff --git a/net/l2tp/l2tp_core.c b/net/l2tp/l2tp_core.c index 7cdfab3a7809..6c83cb02d434 100644 --- a/net/l2tp/l2tp_core.c +++ b/net/l2tp/l2tp_core.c @@ -315,10 +315,19 @@ EXPORT_SYMBOL_GPL(l2tp_v2_session_get); struct l2tp_session *l2tp_session_get(const struct net *net, struct sock *sk, int pver, u32 tunnel_id, u32 session_id) { + struct l2tp_session *session; + if (pver == L2TP_HDR_VER_2) return l2tp_v2_session_get(net, tunnel_id, session_id); - else - return l2tp_v3_session_get(net, sk, session_id); + + /* l2tp_v3_session_get() doesn't check sk if the ID hasn't collided */ + session = l2tp_v3_session_get(net, sk, session_id); + if (session && session->tunnel->sock != sk) { + l2tp_session_put(session); + return NULL; + } + + return session; } EXPORT_SYMBOL_GPL(l2tp_session_get); diff --git a/net/mac80211/tdls.c b/net/mac80211/tdls.c index b4bedf0b2b55..33657ca92ae5 100644 --- a/net/mac80211/tdls.c +++ b/net/mac80211/tdls.c @@ -2016,7 +2016,7 @@ void ieee80211_process_tdls_channel_switch(struct ieee80211_sub_if_data *sdata, struct sk_buff *skb) { - struct ieee80211_tdls_data *tf = (void *)skb->data; + struct ieee80211_tdls_data *tf; struct wiphy *wiphy = sdata->local->hw.wiphy; lockdep_assert_wiphy(wiphy); @@ -2032,6 +2032,7 @@ ieee80211_process_tdls_channel_switch(struct ieee80211_sub_if_data *sdata, * The packet/size was already validated by mac80211 Rx path, only look * at the action type. */ + tf = (void *)skb->data; switch (tf->action_code) { case WLAN_TDLS_CHANNEL_SWITCH_REQUEST: ieee80211_process_tdls_channel_switch_req(sdata, skb); diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c index 0fad278d8f24..fff9ea0dba07 100644 --- a/net/mac80211/tx.c +++ b/net/mac80211/tx.c @@ -276,7 +276,6 @@ ieee80211_tx_h_dynamic_ps(struct ieee80211_tx_data *tx) static ieee80211_tx_result debug_noinline ieee80211_tx_h_check_assoc(struct ieee80211_tx_data *tx) { - struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)tx->skb->data; struct ieee80211_tx_info *info = IEEE80211_SKB_CB(tx->skb); bool assoc = false; @@ -284,23 +283,6 @@ ieee80211_tx_h_check_assoc(struct ieee80211_tx_data *tx) if (unlikely(info->flags & IEEE80211_TX_CTL_INJECTED)) return TX_CONTINUE; - if (unlikely(test_bit(SCAN_SW_SCANNING, &tx->local->scanning)) && - test_bit(SDATA_STATE_OFFCHANNEL, &tx->sdata->state) && - !ieee80211_is_probe_req(hdr->frame_control) && - !ieee80211_is_any_nullfunc(hdr->frame_control)) - /* - * When software scanning only nullfunc frames (to notify - * the sleep state to the AP) and probe requests (for the - * active scan) are allowed, all other frames should not be - * sent and we should not get here, but if we do - * nonetheless, drop them to avoid sending them - * off-channel. See the link below and - * ieee80211_start_scan() for more. - * - * http://article.gmane.org/gmane.linux.kernel.wireless.general/30089 - */ - return TX_DROP; - if (tx->sdata->vif.type == NL80211_IFTYPE_OCB) return TX_CONTINUE; @@ -810,6 +792,33 @@ ieee80211_tx_h_rate_ctrl(struct ieee80211_tx_data *tx) return TX_CONTINUE; } +static ieee80211_tx_result debug_noinline +ieee80211_tx_h_check_offchannel(struct ieee80211_tx_data *tx) +{ + struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)tx->skb->data; + struct ieee80211_tx_info *info = IEEE80211_SKB_CB(tx->skb); + + if (unlikely(info->flags & IEEE80211_TX_CTL_INJECTED)) + return TX_CONTINUE; + + if (unlikely(test_bit(SCAN_SW_SCANNING, &tx->local->scanning)) && + test_bit(SDATA_STATE_OFFCHANNEL, &tx->sdata->state) && + !ieee80211_is_probe_req(hdr->frame_control) && + !ieee80211_is_any_nullfunc(hdr->frame_control)) { + /* + * When software scanning only nullfunc frames (to notify + * the sleep state to the AP) and probe requests (for the + * active scan) are allowed, all other frames should not be + * sent and we should not get here, but if we do + * nonetheless, drop them to avoid sending them + * off-channel. See __ieee80211_start_scan() for more. + */ + return TX_DROP; + } + + return TX_CONTINUE; +} + static __le16 ieee80211_tx_next_seq(struct sta_info *sta, int tid) { u16 *seq = &sta->tid_seq[tid]; @@ -1119,6 +1128,7 @@ static bool ieee80211_tx_prep_agg(struct ieee80211_tx_data *tx, struct sk_buff *purge_skb = NULL; if (test_bit(HT_AGG_STATE_OPERATIONAL, &tid_tx->state)) { + info->flags |= IEEE80211_TX_CTL_AMPDU; reset_agg_timer = true; } else if (test_bit(HT_AGG_STATE_WANT_START, &tid_tx->state)) { /* @@ -1150,6 +1160,7 @@ static bool ieee80211_tx_prep_agg(struct ieee80211_tx_data *tx, if (!tid_tx) { /* do nothing, let packet pass through */ } else if (test_bit(HT_AGG_STATE_OPERATIONAL, &tid_tx->state)) { + info->flags |= IEEE80211_TX_CTL_AMPDU; reset_agg_timer = true; } else { queued = true; @@ -1853,6 +1864,7 @@ static int invoke_tx_handlers_late(struct ieee80211_tx_data *tx) goto txh_done; } + CALL_TXH(ieee80211_tx_h_check_offchannel); CALL_TXH(ieee80211_tx_h_michael_mic_add); CALL_TXH(ieee80211_tx_h_sequence); CALL_TXH(ieee80211_tx_h_fragment); @@ -4667,10 +4679,6 @@ static void ieee80211_8023_xmit(struct ieee80211_sub_if_data *sdata, queue = ieee80211_select_queue(sdata, sta, skb); skb_set_queue_mapping(skb, queue); - if (unlikely(test_bit(SCAN_SW_SCANNING, &local->scanning)) && - test_bit(SDATA_STATE_OFFCHANNEL, &sdata->state)) - goto out_free; - skb = skb_share_check(skb, GFP_ATOMIC); if (unlikely(!skb)) return; @@ -4748,11 +4756,6 @@ static void ieee80211_8023_xmit(struct ieee80211_sub_if_data *sdata, ieee80211_tpt_led_trig_tx(local, len); ieee80211_tx_8023(sdata, skb, sta, false); - - return; - -out_free: - kfree_skb(skb); } static bool ieee80211_check_mcast_offload(struct ieee80211_sub_if_data *sdata, diff --git a/net/mptcp/pm_kernel.c b/net/mptcp/pm_kernel.c index 4de1375bf2a5..6e51b16f10d9 100644 --- a/net/mptcp/pm_kernel.c +++ b/net/mptcp/pm_kernel.c @@ -1270,6 +1270,7 @@ static int mptcp_pm_nl_create_listen_socket(struct sock *sk, * old status is known (TCP_CLOSE). */ inet_sk_state_store(newsk, TCP_LISTEN); + sock_set_flag(newsk, SOCK_RCU_FREE); lock_sock(ssk); WRITE_ONCE(mptcp_subflow_ctx(ssk)->pm_listener, true); err = __inet_listen_sk(ssk, backlog); diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index ed41368392c9..c32ce7da5fd6 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -3485,6 +3485,24 @@ static void mptcp_copy_ip_options(struct sock *newsk, const struct sock *sk) rcu_read_unlock(); } +static void mptcp_sk_clone_destroy(struct sock *nsk) +{ + struct mptcp_sock *msk = mptcp_sk(nsk); + + mptcp_release_sched(msk); + mptcp_set_state(nsk, TCP_CLOSE); + /* inet_csk_prepare_forced_close() clears TCP sock_ops state via + * tcp_sk(), but nsk is an MPTCP master socket; keep the inet-level + * destroy preparation here. + */ + bh_unlock_sock(nsk); + sock_put(nsk); + sock_set_flag(nsk, SOCK_DEAD); + tcp_orphan_count_inc(); + inet_sk(nsk)->inet_num = 0; + inet_csk_destroy_sock(nsk); +} + struct sock *mptcp_sk_clone_init(const struct sock *sk, const struct mptcp_options_received *mp_opt, struct sock *ssk, @@ -3544,11 +3562,6 @@ struct sock *mptcp_sk_clone_init(const struct sock *sk, list_add(&subflow->node, &msk->conn_list); sock_hold(ssk); - /* new mpc subflow takes ownership of the newly - * created mptcp socket - */ - mptcp_token_accept(subflow_req, msk); - /* set msk addresses early to ensure mptcp_pm_get_local_id() * uses the correct data */ @@ -3557,6 +3570,14 @@ struct sock *mptcp_sk_clone_init(const struct sock *sk, mptcp_rcv_space_init(msk, ssk); msk->rcvq_space.time = mptcp_stamp(); + if (!mptcp_token_accept(subflow_req, msk)) { + list_del_init(&subflow->node); + WRITE_ONCE(msk->first, NULL); + sock_put(ssk); + mptcp_sk_clone_destroy(nsk); + return NULL; + } + if (mp_opt->suboptions & OPTION_MPTCP_MPC_ACK) __mptcp_subflow_fully_established(msk, subflow, mp_opt); bh_unlock_sock(nsk); diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h index dbfe959d3a18..d9c20de69dc5 100644 --- a/net/mptcp/protocol.h +++ b/net/mptcp/protocol.h @@ -544,14 +544,16 @@ struct mptcp_subflow_context { stale : 1, /* unable to snd/rcv data, do not use for xmit */ valid_csum_seen : 1, /* at least one csum validated */ is_mptfo : 1, /* subflow is doing TFO */ - close_event_done : 1, /* has done the post-closed part */ mpc_drop : 1, /* the MPC option has been dropped in a rtx */ resetting : 1, /* subflow is resetting */ - __unused : 7; + __unused : 8; bool data_avail; bool scheduled; bool pm_listener; /* a listener managed by the kernel PM? */ bool fully_established; /* path validated */ + bool close_event_done; /* has done the post-closed part, + * protected by msk socket lock + */ u32 remote_nonce; u64 thmac; u32 local_nonce; @@ -992,9 +994,11 @@ static inline void mptcp_token_init_request(struct request_sock *req) } int mptcp_token_new_request(struct request_sock *req); +void mptcp_token_move_request(struct request_sock *req, + struct request_sock *new_req); void mptcp_token_destroy_request(struct request_sock *req); int mptcp_token_new_connect(struct sock *ssk); -void mptcp_token_accept(struct mptcp_subflow_request_sock *r, +bool mptcp_token_accept(struct mptcp_subflow_request_sock *r, struct mptcp_sock *msk); bool mptcp_token_exists(u32 token); struct mptcp_sock *mptcp_token_get_sock(struct net *net, u32 token); diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c index 17d392b042ba..8f8ffc88214b 100644 --- a/net/mptcp/subflow.c +++ b/net/mptcp/subflow.c @@ -78,6 +78,8 @@ void mptcp_subflow_reqsk_clone(struct request_sock *req, } new_subflow_req->msk = msk; + + mptcp_token_move_request(req, new_req); } static void subflow_generate_hmac(u64 key1, u64 key2, u32 nonce1, u32 nonce2, @@ -379,6 +381,10 @@ static void subflow_prep_synack(const struct sock *sk, struct request_sock *req, struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk); struct inet_request_sock *ireq = inet_rsk(req); + /* MP_JOIN cannot use TFO, do not send a cookie in the SYN/ACK */ + if (foc && mptcp_subflow_rsk(req)->mp_join) + foc->len = -1; + /* clear tstamp_ok, as needed depending on cookie */ if (foc && foc->len > -1) ireq->tstamp_ok = 0; @@ -893,6 +899,13 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk, fallback = true; } else if (subflow_req->mp_join) { + /* a SYN skb here comes from TFO, which MP_JOIN cannot use: just + * fall back to the regular path. + */ + if ((TCP_SKB_CB(skb)->tcp_flags & (TCPHDR_SYN | TCPHDR_ACK)) == + TCPHDR_SYN) + return NULL; + mptcp_get_options(skb, &mp_opt); if (!(mp_opt.suboptions & OPTION_MPTCP_MPJ_ACK)) fallback = true; @@ -924,8 +937,10 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk, if (ctx->mp_capable) { ctx->conn = mptcp_sk_clone_init(listener->conn, &mp_opt, child, req); - if (!ctx->conn) + if (!ctx->conn) { + fallback = true; goto fallback; + } ctx->subflow_id = 1; owner = mptcp_sk(ctx->conn); diff --git a/net/mptcp/token.c b/net/mptcp/token.c index 5bb924534387..d5735d3ac793 100644 --- a/net/mptcp/token.c +++ b/net/mptcp/token.c @@ -180,6 +180,43 @@ int mptcp_token_new_connect(struct sock *ssk) return 0; } +/** + * mptcp_token_move_request - move request token ownership to a clone + * @req: original request socket + * @new_req: cloned request socket + * + * Move the token hash entry from the original request to its clone. + */ +void mptcp_token_move_request(struct request_sock *req, + struct request_sock *new_req) +{ + struct mptcp_subflow_request_sock *subflow_req = mptcp_subflow_rsk(req); + struct mptcp_subflow_request_sock *new_subflow_req; + struct mptcp_subflow_request_sock *pos; + struct token_bucket *bucket; + + new_subflow_req = mptcp_subflow_rsk(new_req); + + if (hlist_nulls_unhashed_lockless(&subflow_req->token_node)) { + mptcp_token_init_request(new_req); + return; + } + + bucket = token_bucket(subflow_req->token); + spin_lock_bh(&bucket->lock); + if (hlist_nulls_unhashed(&subflow_req->token_node)) { + mptcp_token_init_request(new_req); + } else { + pos = __token_lookup_req(bucket, subflow_req->token); + if (pos == subflow_req) + hlist_nulls_replace_init_rcu(&subflow_req->token_node, + &new_subflow_req->token_node); + else + mptcp_token_init_request(new_req); + } + spin_unlock_bh(&bucket->lock); +} + /** * mptcp_token_accept - replace a req sk with full sock in token hash * @req: the request socket to be removed @@ -187,24 +224,36 @@ int mptcp_token_new_connect(struct sock *ssk) * * Called when a SYN packet creates a new logical connection, i.e. * is not a join request. + * + * Return: true on success. */ -void mptcp_token_accept(struct mptcp_subflow_request_sock *req, +bool mptcp_token_accept(struct mptcp_subflow_request_sock *req, struct mptcp_sock *msk) { struct mptcp_subflow_request_sock *pos; struct sock *sk = (struct sock *)msk; struct token_bucket *bucket; + bool ret = false; - sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1); bucket = token_bucket(req->token); spin_lock_bh(&bucket->lock); + if (hlist_nulls_unhashed(&req->token_node)) + goto unlock; - /* pedantic lookup check for the moved token */ pos = __token_lookup_req(bucket, req->token); - if (!WARN_ON_ONCE(pos != req)) - hlist_nulls_del_init_rcu(&req->token_node); + if (pos != req) + goto unlock; + + hlist_nulls_del_init_rcu(&req->token_node); __sk_nulls_add_node_rcu((struct sock *)msk, &bucket->msk_chain); + ret = true; + +unlock: spin_unlock_bh(&bucket->lock); + if (ret) + sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1); + + return ret; } bool mptcp_token_exists(u32 token) @@ -353,16 +402,21 @@ void mptcp_token_destroy_request(struct request_sock *req) struct mptcp_subflow_request_sock *pos; struct token_bucket *bucket; - if (hlist_nulls_unhashed(&subflow_req->token_node)) + if (hlist_nulls_unhashed_lockless(&subflow_req->token_node)) return; bucket = token_bucket(subflow_req->token); spin_lock_bh(&bucket->lock); + if (hlist_nulls_unhashed(&subflow_req->token_node)) + goto unlock; + pos = __token_lookup_req(bucket, subflow_req->token); - if (!WARN_ON_ONCE(pos != subflow_req)) { + if (pos == subflow_req) { hlist_nulls_del_init_rcu(&pos->token_node); bucket->chain_len--; } + +unlock: spin_unlock_bh(&bucket->lock); } diff --git a/net/mptcp/token_test.c b/net/mptcp/token_test.c index 4fc39fa2e262..be9acce8a567 100644 --- a/net/mptcp/token_test.c +++ b/net/mptcp/token_test.c @@ -99,7 +99,7 @@ static void mptcp_token_test_accept(struct kunit *test) KUNIT_ASSERT_EQ(test, 0, mptcp_token_new_request((struct request_sock *)req)); msk->token = req->token; - mptcp_token_accept(req, msk); + KUNIT_EXPECT_TRUE(test, mptcp_token_accept(req, msk)); KUNIT_EXPECT_PTR_EQ(test, msk, mptcp_token_get_sock(&init_net, msk->token)); /* this is now a no-op */ @@ -122,7 +122,7 @@ static void mptcp_token_test_destroyed(struct kunit *test) KUNIT_ASSERT_EQ(test, 0, mptcp_token_new_request((struct request_sock *)req)); msk->token = req->token; - mptcp_token_accept(req, msk); + KUNIT_EXPECT_TRUE(test, mptcp_token_accept(req, msk)); /* simulate race on removal */ refcount_set(&sk->sk_refcnt, 0); diff --git a/net/netrom/af_netrom.c b/net/netrom/af_netrom.c index fc7bd19ad0e3..99d9fe19d27f 100644 --- a/net/netrom/af_netrom.c +++ b/net/netrom/af_netrom.c @@ -96,16 +96,22 @@ static void nr_remove_socket(struct sock *sk) } /* - * Kill all bound sockets on a dropped device. + * Kill all bound sockets on a dropped device. Each socket is disconnected + * under its lock, which its timers and nr_rx_frame() hold while they work + * on it; nr_info_show() takes the two locks in the same order. */ static void nr_kill_by_device(struct net_device *dev) { struct sock *s; spin_lock_bh(&nr_list_lock); - sk_for_each(s, &nr_list) - if (nr_sk(s)->device == dev) - nr_disconnect(s, ENETUNREACH); + sk_for_each(s, &nr_list) { + if (nr_sk(s)->device != dev) + continue; + bh_lock_sock(s); + nr_disconnect(s, ENETUNREACH); + bh_unlock_sock(s); + } spin_unlock_bh(&nr_list_lock); } @@ -208,29 +214,40 @@ static struct sock *nr_find_peer(unsigned char index, unsigned char id, } /* - * Find next free circuit ID. + * Give a socket the next free circuit ID. The search and the assignment + * are done under nr_list_lock, which the caller holds, with the socket + * already on nr_list: two sockets cannot pick the same ID. */ -static unsigned short nr_find_next_circuit(void) +static void nr_assign_circuit(struct nr_sock *nr) { unsigned short id = circuit; unsigned char i, j; - struct sock *sk; + struct sock *s; + bool used; + + lockdep_assert_held(&nr_list_lock); for (;;) { i = id / 256; j = id % 256; if (i != 0 && j != 0) { - sk = nr_find_socket(i, j, NULL); - if (!sk) + used = false; + sk_for_each(s, &nr_list) + if (nr_sk(s)->my_index == i && nr_sk(s)->my_id == j) { + used = true; + break; + } + if (!used) break; - sock_put(sk); } id++; } - return id; + nr->my_index = i; + nr->my_id = j; + circuit = id + 1; } /* @@ -459,8 +476,7 @@ static int nr_create(struct net *net, struct socket *sock, int protocol, msecs_to_jiffies(READ_ONCE(sysctl_netrom_transport_timeout)); nr->t2 = msecs_to_jiffies(READ_ONCE(sysctl_netrom_transport_acknowledge_delay)); - nr->n2 = - msecs_to_jiffies(READ_ONCE(sysctl_netrom_transport_maximum_tries)); + nr->n2 = READ_ONCE(sysctl_netrom_transport_maximum_tries); nr->t4 = msecs_to_jiffies(READ_ONCE(sysctl_netrom_transport_busy_delay)); nr->idle = @@ -710,14 +726,10 @@ static int nr_connect(struct socket *sock, struct sockaddr *uaddr, nr->dest_addr = addr->sax25_call; - release_sock(sk); - circuit = nr_find_next_circuit(); - lock_sock(sk); - - nr->my_index = circuit / 256; - nr->my_id = circuit % 256; - - circuit++; + /* The socket is on nr_list now, so others see the ID it gets */ + spin_lock_bh(&nr_list_lock); + nr_assign_circuit(nr); + spin_unlock_bh(&nr_list_lock); /* Move to connecting socket, start sending Connect Requests */ sock->state = SS_CONNECTING; @@ -949,10 +961,16 @@ int nr_rx_frame(struct sk_buff *skb, struct net_device *dev) bh_lock_sock(sk); skb_reset_transport_header(skb); - if (frametype == NR_CONNACK && skb->len == 22) - nr_sk(sk)->bpqext = 1; - else - nr_sk(sk)->bpqext = 0; + /* + * Only the circuit set-up frames tell whether the peer uses + * the BPQ extension: a CONNACK of 22 bytes, or a CONNREQ of + * 37 bytes (repeated, for a circuit already accepted). The + * other frames say nothing about it. + */ + if (frametype == NR_CONNACK) + nr_sk(sk)->bpqext = (skb->len == 22); + else if (frametype == NR_CONNREQ) + nr_sk(sk)->bpqext = (skb->len == 37); ret = nr_process_rx_frame(sk, skb); bh_unlock_sock(sk); @@ -1014,23 +1032,29 @@ int nr_rx_frame(struct sk_buff *skb, struct net_device *dev) nr_make->your_index = circuit_index; nr_make->your_id = circuit_id; + /* + * Put the new socket on nr_list as it gets its circuit ID, under the + * same lock, so that no other socket can be given the same ID. + * nr_list_lock is not taken under a socket lock (nr_info_show() takes + * them the other way round). + */ bh_unlock_sock(sk); - circuit = nr_find_next_circuit(); + spin_lock_bh(&nr_list_lock); + nr_assign_circuit(nr_make); + sk_add_node(make, &nr_list); + spin_unlock_bh(&nr_list_lock); bh_lock_sock(sk); - nr_make->my_index = circuit / 256; - nr_make->my_id = circuit % 256; - - circuit++; - /* Window negotiation */ - if (window < nr_make->window) - nr_make->window = window; + nr_negotiate_window(nr_make, window); - /* L4 timeout negotiation */ + /* + * L4 timeout negotiation. Ignore a proposal shorter than the + * smallest transport timeout the sysctl accepts (5 s), such as 0. + */ if (skb->len == 37) { timeout = skb->data[36] * 256 + skb->data[35]; - if (timeout * HZ < nr_make->t1) + if (timeout >= 5 && timeout * HZ < nr_make->t1) nr_make->t1 = timeout * HZ; nr_make->bpqext = 1; } else { @@ -1054,8 +1078,6 @@ int nr_rx_frame(struct sk_buff *skb, struct net_device *dev) bh_unlock_sock(sk); sock_put(sk); - nr_insert_socket(make); - nr_start_heartbeat(make); nr_start_idletimer(make); @@ -1163,9 +1185,9 @@ static int nr_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) goto out; } - nr_output(sk, skb); /* Shove it onto the queue */ - - err = len; + err = nr_output(sk, skb); /* Shove it onto the queue */ + if (!err) + err = len; out: release_sock(sk); return err; @@ -1214,10 +1236,13 @@ static int nr_recvmsg(struct socket *sock, struct msghdr *msg, size_t size, } if (sax != NULL) { + /* + * The NET/ROM headers were pulled when the frame was queued: + * give the node at the other end of the circuit. + */ memset(sax, 0, sizeof(*sax)); sax->sax25_family = AF_NETROM; - skb_copy_from_linear_data_offset(skb, 7, sax->sax25_call.ax25_call, - AX25_ADDR_LEN); + sax->sax25_call = nr_sk(sk)->dest_addr; msg->msg_namelen = sizeof(*sax); } diff --git a/net/netrom/nr_in.c b/net/netrom/nr_in.c index b90a47dcf165..805881e5d6c0 100644 --- a/net/netrom/nr_in.c +++ b/net/netrom/nr_in.c @@ -67,26 +67,44 @@ static int nr_queue_rx_frame(struct sock *sk, struct sk_buff *skb, int more) return 0; } - if (!more && nr->fraglen > 0) { /* End of fragment */ - nr->fraglen += skb->len; - skb_queue_tail(&nr->frag_queue, skb); - - if ((skbn = alloc_skb(nr->fraglen, GFP_ATOMIC)) == NULL) - return 1; + if (nr->fraglen > 0) { /* End of fragment */ + skbn = alloc_skb(nr->fraglen + skb->len, GFP_ATOMIC); + if (!skbn) + goto busy; skb_reset_transport_header(skbn); - while ((skbo = skb_dequeue(&nr->frag_queue)) != NULL) { + skb_queue_walk(&nr->frag_queue, skbo) skb_copy_from_linear_data(skbo, skb_put(skbn, skbo->len), skbo->len); - kfree_skb(skbo); - } + skb_copy_from_linear_data(skb, skb_put(skbn, skb->len), + skb->len); + } + + if (sock_queue_rcv_skb(sk, skbn) != 0) { + if (skbn != skb) + kfree_skb(skbn); + goto busy; + } + if (skbn != skb) { + /* The whole message is delivered: drop its pieces */ + skb_queue_purge(&nr->frag_queue); nr->fraglen = 0; + kfree_skb(skb); } - return sock_queue_rcv_skb(sk, skbn); + return 0; + +busy: + /* + * Not delivered: give the frame back as it came, headers included, + * and leave the earlier fragments where they are, so that the + * caller can keep it on reseq_queue and deliver it later. + */ + skb_push(skb, NR_NETWORK_LEN + NR_TRANSPORT_LEN); + return 1; } /* @@ -111,7 +129,7 @@ static int nr_state1_machine(struct sock *sk, struct sk_buff *skb, nr->vl = 0; nr->state = NR_STATE_3; nr->n2count = 0; - nr->window = skb->data[20]; + nr_negotiate_window(nr, skb->data[20]); sk->sk_state = TCP_ESTABLISHED; if (!sock_flag(sk, SOCK_DEAD)) sk->sk_state_change(sk); @@ -176,7 +194,7 @@ static int nr_state3_machine(struct sock *sk, struct sk_buff *skb, int frametype struct sk_buff *skbn; unsigned short save_vr; unsigned short nr, ns; - int queued = 0; + int queued = 0, more; nr = skb->data[18]; @@ -258,7 +276,9 @@ static int nr_state3_machine(struct sock *sk, struct sk_buff *skb, int frametype while ((skbn = skb_dequeue(&nrom->reseq_queue)) != NULL) { ns = skbn->data[17]; if (ns == nrom->vr) { - if (nr_queue_rx_frame(sk, skbn, frametype & NR_MORE_FLAG) == 0) { + /* The flags of this frame, not of the one just received */ + more = skbn->data[19] & NR_MORE_FLAG; + if (nr_queue_rx_frame(sk, skbn, more) == 0) { nrom->vr = (nrom->vr + 1) % NR_MODULUS; } else { nrom->condition |= NR_COND_OWN_RX_BUSY; diff --git a/net/netrom/nr_out.c b/net/netrom/nr_out.c index 2b3cbceb0b52..150cb4263d31 100644 --- a/net/netrom/nr_out.c +++ b/net/netrom/nr_out.c @@ -28,11 +28,19 @@ /* * This is where all NET/ROM frames pass, except for IP-over-NET/ROM which * cannot be fragmented in this manner. + * + * A message too long for one frame is cut into fragments built on a list + * of our own, and only put on the write queue once all of them exist: a + * message that cannot be fragmented whole (no memory, or a signal while + * waiting for send buffer space) is not sent at all, and the error is + * returned. Consumes skb in all cases. */ -void nr_output(struct sock *sk, struct sk_buff *skb) +int nr_output(struct sock *sk, struct sk_buff *skb) { + struct sk_buff_head frags; struct sk_buff *skbn; unsigned char transport[NR_TRANSPORT_LEN]; + unsigned long flags; int err, frontlen, len; if (skb->len - NR_TRANSPORT_LEN > NR_MAX_PACKET_SIZE) { @@ -42,10 +50,13 @@ void nr_output(struct sock *sk, struct sk_buff *skb) frontlen = skb_headroom(skb); + __skb_queue_head_init(&frags); + while (skb->len > 0) { if ((skbn = sock_alloc_send_skb(sk, frontlen + NR_MAX_PACKET_SIZE, 0, &err)) == NULL) { + __skb_queue_purge(&frags); kfree_skb(skb); - return; + return err; } skb_reserve(skbn, frontlen); @@ -63,15 +74,21 @@ void nr_output(struct sock *sk, struct sk_buff *skb) if (skb->len > 0) skbn->data[4] |= NR_MORE_FLAG; - skb_queue_tail(&sk->sk_write_queue, skbn); /* Throw it on the queue */ + __skb_queue_tail(&frags, skbn); } kfree_skb(skb); + + spin_lock_irqsave(&sk->sk_write_queue.lock, flags); + skb_queue_splice_tail_init(&frags, &sk->sk_write_queue); + spin_unlock_irqrestore(&sk->sk_write_queue.lock, flags); } else { skb_queue_tail(&sk->sk_write_queue, skb); /* Throw it on the queue */ } nr_kick(sk); + + return 0; } /* diff --git a/net/netrom/nr_route.c b/net/netrom/nr_route.c index 288c4a91ea9b..7fbacc5fee84 100644 --- a/net/netrom/nr_route.c +++ b/net/netrom/nr_route.c @@ -344,6 +344,24 @@ static void nr_remove_neigh_locked(struct nr_neigh *nr_neigh) nr_neigh_put(nr_neigh); } +/* + * Remove route i of a node: move the routes after it up, and keep which on + * the route it selected. If that route is the one removed, which now + * selects the next one, as after a link failure. The caller drops the + * route's use of its neighbour. Called with nr_route_lock held. + */ +static void nr_node_remove_route_locked(struct nr_node *nr_node, int i) +{ + lockdep_assert_held(&nr_route_lock); + + if (i < nr_node->which) + nr_node->which--; + + nr_node->count--; + for (; i < nr_node->count; i++) + nr_node->routes[i] = nr_node->routes[i + 1]; +} + /* * "Delete" a node. Strictly speaking remove a route to a node. The node * is only deleted if no routes are left to it. @@ -373,22 +391,10 @@ static int nr_del_node(ax25_address *callsign, ax25_address *neighbour, struct n nr_neigh_route_put_locked(nr_neigh); nr_neigh_put(nr_neigh); - nr_node->count--; + nr_node_remove_route_locked(nr_node, i); - if (nr_node->count == 0) { + if (nr_node->count == 0) nr_remove_node_locked(nr_node); - } else { - switch (i) { - case 0: - nr_node->routes[0] = nr_node->routes[1]; - fallthrough; - case 1: - nr_node->routes[1] = nr_node->routes[2]; - fallthrough; - case 2: - break; - } - } /* Drop the reference taken by __nr_node_get() */ nr_node_put(nr_node); ret = 0; @@ -472,34 +478,24 @@ static int nr_dec_obs(void) spin_lock_bh(&nr_route_lock); nr_node_for_each_safe(s, nodet, &nr_node_list) { - for (i = 0; i < s->count; i++) { + /* i only moves on when route i stays: the next one moves up */ + for (i = 0; i < s->count; ) { switch (s->routes[i].obs_count) { case 0: /* A locked entry */ + i++; break; case 1: /* From 1 -> 0 */ nr_neigh = s->routes[i].neighbour; nr_neigh_route_put_locked(nr_neigh); - - s->count--; - - switch (i) { - case 0: - s->routes[0] = s->routes[1]; - fallthrough; - case 1: - s->routes[1] = s->routes[2]; - break; - case 2: - break; - } + nr_node_remove_route_locked(s, i); break; default: s->routes[i].obs_count--; + i++; break; - } } @@ -525,23 +521,14 @@ void nr_rt_device_down(struct net_device *dev) nr_neigh_for_each_safe(s, nodet, &nr_neigh_list) { if (s->dev == dev) { nr_node_for_each_safe(t, node2t, &nr_node_list) { - for (i = 0; i < t->count; i++) { + for (i = 0; i < t->count; ) { if (t->routes[i].neighbour == s) { s->count--; nr_neigh_put(s); - t->count--; - - switch (i) { - case 0: - t->routes[0] = t->routes[1]; - fallthrough; - case 1: - t->routes[1] = t->routes[2]; - break; - case 2: - break; - } + nr_node_remove_route_locked(t, i); + } else { + i++; } } @@ -774,11 +761,16 @@ int nr_route_frame(struct sk_buff *skb, ax25_cb *ax25) nr_dest = (ax25_address *)(skb->data + 7); if (ax25 != NULL) { - ret = nr_add_node(nr_src, "", &ax25->dest_addr, ax25->digipeat, - ax25->ax25_dev->dev, 0, - READ_ONCE(sysctl_netrom_obsolescence_count_initialiser)); - if (ret) - return ret; + /* + * A frame that cannot be noted here (it claims to come from + * one of our own callsigns, or memory is short) is dropped. + * Return 0, not the error: AX.25 takes any other value as + * "frame consumed" and would never free it. + */ + if (nr_add_node(nr_src, "", &ax25->dest_addr, ax25->digipeat, + ax25->ax25_dev->dev, 0, + READ_ONCE(sysctl_netrom_obsolescence_count_initialiser))) + return 0; } if ((dev = nr_dev_get(nr_dest)) != NULL) { /* Its for me */ @@ -793,10 +785,12 @@ int nr_route_frame(struct sk_buff *skb, ax25_cb *ax25) if (!READ_ONCE(sysctl_netrom_routing_control) && ax25 != NULL) return 0; - /* Its Time-To-Live has expired */ - if (skb->data[14] == 1) { + /* + * Its Time-To-Live has expired. A frame received with a TTL of 0 + * has expired too: decremented below, it would leave with 255. + */ + if (skb->data[14] <= 1) return 0; - } nr_node = nr_node_get(nr_dest); if (nr_node == NULL) @@ -848,8 +842,14 @@ int nr_route_frame(struct sk_buff *skb, ax25_cb *ax25) spin_unlock_bh(&nr_route_lock); nr_node_put(nr_node); + /* + * ax25_send_frame() only returns NULL before taking the frame: + * free our copy then; the caller frees the original. + */ if (ret) kfree_skb(oskb); + else + kfree_skb(skb); return ret; } diff --git a/net/rds/recv.c b/net/rds/recv.c index bd9d00326e94..110fb22c3e2e 100644 --- a/net/rds/recv.c +++ b/net/rds/recv.c @@ -343,7 +343,8 @@ void rds_recv_incoming(struct rds_connection *conn, struct in6_addr *saddr, rds_stats_inc(s_recv_ping); rds_send_pong(cp, inc->i_hdr.h_sport); /* if this is a handshake ping, start multipath if necessary */ - if (RDS_HS_PROBE(be16_to_cpu(inc->i_hdr.h_sport), + if (conn->c_trans->t_mp_capable && + RDS_HS_PROBE(be16_to_cpu(inc->i_hdr.h_sport), be16_to_cpu(inc->i_hdr.h_dport))) { rds_recv_hs_exthdrs(&inc->i_hdr, cp->cp_conn); rds_start_mprds(cp->cp_conn); @@ -351,7 +352,8 @@ void rds_recv_incoming(struct rds_connection *conn, struct in6_addr *saddr, goto out; } - if (be16_to_cpu(inc->i_hdr.h_dport) == RDS_FLAG_PROBE_PORT && + if (conn->c_trans->t_mp_capable && + be16_to_cpu(inc->i_hdr.h_dport) == RDS_FLAG_PROBE_PORT && inc->i_hdr.h_sport == 0) { rds_recv_hs_exthdrs(&inc->i_hdr, cp->cp_conn); /* if this is a handshake pong, start multipath if necessary */ diff --git a/net/rose/af_rose.c b/net/rose/af_rose.c index d8a86006c744..332d26771681 100644 --- a/net/rose/af_rose.c +++ b/net/rose/af_rose.c @@ -278,7 +278,7 @@ static void rose_insert_socket(struct sock *sk) /* * Find a socket that wants to accept the Call Request we just - * received. + * received. A reference is taken on the socket found. */ static struct sock *rose_find_listener(rose_address *addr, ax25_address *call) { @@ -304,48 +304,73 @@ static struct sock *rose_find_listener(rose_address *addr, ax25_address *call) } s = NULL; found: + if (s) + sock_hold(s); spin_unlock_bh(&rose_list_lock); return s; } /* * Find a connected ROSE socket given my LCI and device. + * The caller holds rose_list_lock. */ -struct sock *rose_find_socket(unsigned int lci, struct rose_neigh *neigh) +static struct sock *__rose_find_socket(unsigned int lci, struct rose_neigh *neigh) { struct sock *s; - spin_lock_bh(&rose_list_lock); sk_for_each(s, &rose_list) { struct rose_sock *rose = rose_sk(s); if (rose->lci == lci && rose->neighbour == neigh) - goto found; + return s; } - s = NULL; -found: + + return NULL; +} + +/* + * Find a connected ROSE socket given my LCI and device. A reference is + * taken on the socket found: it can be released and destroyed as soon + * as rose_list_lock is dropped. + */ +struct sock *rose_find_socket(unsigned int lci, struct rose_neigh *neigh) +{ + struct sock *s; + + spin_lock_bh(&rose_list_lock); + s = __rose_find_socket(lci, neigh); + if (s) + sock_hold(s); spin_unlock_bh(&rose_list_lock); return s; } /* * Find a unique LCI for a given device. + * The caller holds rose_route_list_lock. */ unsigned int rose_new_lci(struct rose_neigh *neigh) { + unsigned int res = 0; int lci; + spin_lock_bh(&rose_list_lock); if (neigh->dce_mode) { for (lci = 1; lci <= sysctl_rose_maximum_vcs; lci++) - if (rose_find_socket(lci, neigh) == NULL && rose_route_free_lci(lci, neigh) == NULL) - return lci; + if (!__rose_find_socket(lci, neigh) && !rose_route_free_lci(lci, neigh)) { + res = lci; + break; + } } else { for (lci = sysctl_rose_maximum_vcs; lci > 0; lci--) - if (rose_find_socket(lci, neigh) == NULL && rose_route_free_lci(lci, neigh) == NULL) - return lci; + if (!__rose_find_socket(lci, neigh) && !rose_route_free_lci(lci, neigh)) { + res = lci; + break; + } } + spin_unlock_bh(&rose_list_lock); - return 0; + return res; } /* @@ -673,6 +698,7 @@ static struct sock *rose_make_new(struct sock *osk) static int rose_release(struct socket *sock) { struct sock *sk = sock->sk; + struct rose_neigh *neigh; struct rose_sock *rose; if (sk == NULL) return 0; @@ -691,10 +717,21 @@ static int rose_release(struct socket *sock) break; case ROSE_STATE_2: - rose_neigh_put(rose->neighbour); - release_sock(sk); + /* + * Disconnect before the socket lock is dropped: frames put on + * the backlog meanwhile, a Clear Confirmation for one, then + * find it in state 0 instead of dropping the neighbour + * reference a second time. rose_kill_by_neigh() does not take + * the socket lock: take the neighbour under rose_list_lock, as + * it does, so that only one of them drops the reference. + */ rose_disconnect(sk, 0, -1, -1); - lock_sock(sk); + spin_lock_bh(&rose_list_lock); + neigh = rose->neighbour; + rose->neighbour = NULL; + spin_unlock_bh(&rose_list_lock); + if (neigh) + rose_neigh_put(neigh); rose_destroy_socket(sk); break; @@ -854,20 +891,6 @@ static int rose_connect(struct socket *sock, struct sockaddr *uaddr, int addr_le sk->sk_state = TCP_CLOSE; sock->state = SS_UNCONNECTED; - rose->neighbour = rose_get_neigh(&addr->srose_addr, &cause, - &diagnostic, 0); - if (!rose->neighbour) { - err = -ENETUNREACH; - goto out_release; - } - - rose->lci = rose_new_lci(rose->neighbour); - if (!rose->lci) { - err = -ENETUNREACH; - rose_neigh_put(rose->neighbour); - goto out_release; - } - if (sock_flag(sk, SOCK_ZAPPED)) { /* Must bind first - autobinding in this may or may not work */ struct net_device *dev; @@ -876,14 +899,12 @@ static int rose_connect(struct socket *sock, struct sockaddr *uaddr, int addr_le dev = rose_dev_first(); if (!dev) { err = -ENETUNREACH; - rose_neigh_put(rose->neighbour); goto out_release; } user = ax25_findbyuid(current_euid()); if (!user) { err = -EINVAL; - rose_neigh_put(rose->neighbour); dev_put(dev); goto out_release; } @@ -897,6 +918,21 @@ static int rose_connect(struct socket *sock, struct sockaddr *uaddr, int addr_le rose_insert_socket(sk); /* Finish the bind */ } + + rose->neighbour = rose_get_neigh(&addr->srose_addr, &cause, + &diagnostic, 0); + if (!rose->neighbour) { + err = -ENETUNREACH; + goto out_release; + } + + /* The socket is on the list now, so others see the LCI it gets */ + if (!rose_assign_lci(sk)) { + err = -ENETUNREACH; + rose_neigh_put(rose->neighbour); + goto out_release; + } + rose->dest_addr = addr->srose_addr; rose->dest_call = addr->srose_call; rose->rand = ((long)rose & 0xFFFF) + rose->lci; @@ -1103,12 +1139,31 @@ int rose_rx_call_request(struct sk_buff *skb, struct net_device *dev, struct ros } sk = rose_find_listener(&facilities.source_addr, &facilities.source_call); + if (!sk) { + rose_transmit_clear_request(neigh, lci, ROSE_NETWORK_CONGESTION, 120); + return 0; + } + + /* + * The listener may be closed by rose_release() at any time. It is + * marked dead before rose_destroy_socket() empties its queue, so + * once it is locked a listener that is not dead will see the new + * socket on its queue, either in rose_accept() or when it is + * destroyed. + */ + bh_lock_sock(sk); /* * We can't accept the Call Request. */ - if (sk == NULL || sk_acceptq_is_full(sk) || - (make = rose_make_new(sk)) == NULL) { + if (sock_flag(sk, SOCK_DEAD) || sk->sk_state != TCP_LISTEN || + sk_acceptq_is_full(sk)) + make = NULL; + else + make = rose_make_new(sk); + if (!make) { + bh_unlock_sock(sk); + sock_put(sk); rose_transmit_clear_request(neigh, lci, ROSE_NETWORK_CONGESTION, 120); return 0; } @@ -1164,8 +1219,9 @@ int rose_rx_call_request(struct sk_buff *skb, struct net_device *dev, struct ros rose_start_heartbeat(make); - if (!sock_flag(sk, SOCK_DEAD)) - sk->sk_data_ready(sk); + sk->sk_data_ready(sk); + bh_unlock_sock(sk); + sock_put(sk); return 1; } diff --git a/net/rose/rose_in.c b/net/rose/rose_in.c index 7a556254ee79..84025b4b498f 100644 --- a/net/rose/rose_in.c +++ b/net/rose/rose_in.c @@ -75,17 +75,24 @@ static int rose_state1_machine(struct sock *sk, struct sk_buff *skb, int framety static int rose_state2_machine(struct sock *sk, struct sk_buff *skb, int frametype) { struct rose_sock *rose = rose_sk(sk); + /* + * A call whose Call Request timed out (T1) is cleared from state 2 + * while connect() is still waiting for it: report the timeout, or + * connect() and SO_ERROR would find no error on a call that never + * got through. + */ + int reason = sk->sk_state == TCP_SYN_SENT ? ETIMEDOUT : 0; switch (frametype) { case ROSE_CLEAR_REQUEST: rose_write_internal(sk, ROSE_CLEAR_CONFIRMATION); - rose_disconnect(sk, 0, skb->data[3], skb->data[4]); + rose_disconnect(sk, reason, skb->data[3], skb->data[4]); rose_neigh_put(rose->neighbour); rose->neighbour = NULL; break; case ROSE_CLEAR_CONFIRMATION: - rose_disconnect(sk, 0, -1, -1); + rose_disconnect(sk, reason, -1, -1); rose_neigh_put(rose->neighbour); rose->neighbour = NULL; break; diff --git a/net/rose/rose_link.c b/net/rose/rose_link.c index 9cfd4aa174f8..2e2837e63242 100644 --- a/net/rose/rose_link.c +++ b/net/rose/rose_link.c @@ -27,29 +27,66 @@ static void rose_ftimer_expiry(struct timer_list *); static void rose_t0timer_expiry(struct timer_list *); -static void rose_transmit_restart_confirmation(struct rose_neigh *neigh); -static void rose_transmit_restart_request(struct rose_neigh *neigh); +/* + * neigh->ax25 is replaced when a frame is sent or the link is looked up, + * from process context, softirq and the neighbour timers, and cleared by + * rose_link_failed(). Each holder of the pointer owns a reference on the + * ax25_cb: swap the pointer under this lock, then drop the reference that + * was taken off, so that it is dropped once and none is lost. + */ +static DEFINE_SPINLOCK(rose_neigh_ax25_lock); -void rose_start_ftimer(struct rose_neigh *neigh) +static void rose_neigh_set_ax25(struct rose_neigh *neigh, ax25_cb *ax25) { - del_timer(&neigh->ftimer); + ax25_cb *old; - neigh->ftimer.function = rose_ftimer_expiry; - neigh->ftimer.expires = - jiffies + msecs_to_jiffies(sysctl_rose_link_fail_timeout); + spin_lock_bh(&rose_neigh_ax25_lock); + old = neigh->ax25; + neigh->ax25 = ax25; + spin_unlock_bh(&rose_neigh_ax25_lock); - add_timer(&neigh->ftimer); + if (old) + ax25_cb_put(old); } -static void rose_start_t0timer(struct rose_neigh *neigh) +/* + * Clear neigh->ax25 if it still is @ax25, and drop its reference. + */ +bool rose_neigh_clear_ax25(struct rose_neigh *neigh, ax25_cb *ax25) { - del_timer(&neigh->t0timer); + bool found; - neigh->t0timer.function = rose_t0timer_expiry; - neigh->t0timer.expires = - jiffies + msecs_to_jiffies(sysctl_rose_restart_request_timeout); + spin_lock_bh(&rose_neigh_ax25_lock); + found = neigh->ax25 == ax25; + if (found) + neigh->ax25 = NULL; + spin_unlock_bh(&rose_neigh_ax25_lock); + + if (found) + ax25_cb_put(ax25); + + return found; +} - add_timer(&neigh->t0timer); +static void rose_transmit_restart_confirmation(struct rose_neigh *neigh); +static void rose_transmit_restart_request(struct rose_neigh *neigh); + +void rose_neigh_init_timers(struct rose_neigh *neigh) +{ + timer_setup(&neigh->ftimer, rose_ftimer_expiry, 0); + timer_setup(&neigh->t0timer, rose_t0timer_expiry, 0); +} + +void rose_start_ftimer(struct rose_neigh *neigh) +{ + mod_timer(&neigh->ftimer, + jiffies + msecs_to_jiffies(sysctl_rose_link_fail_timeout)); +} + +static void rose_start_t0timer(struct rose_neigh *neigh) +{ + mod_timer(&neigh->t0timer, + jiffies + msecs_to_jiffies(sysctl_rose_restart_request_timeout)); } void rose_stop_ftimer(struct rose_neigh *neigh) @@ -95,19 +132,17 @@ static void rose_t0timer_expiry(struct timer_list *t) static int rose_send_frame(struct sk_buff *skb, struct rose_neigh *neigh) { const ax25_address *rose_call; - ax25_cb *ax25s; + ax25_cb *ax25; if (ax25cmp(&rose_callsign, &null_ax25_address) == 0) rose_call = (const ax25_address *)neigh->dev->dev_addr; else rose_call = &rose_callsign; - ax25s = neigh->ax25; - neigh->ax25 = ax25_send_frame(skb, 260, rose_call, &neigh->callsign, neigh->digipeat, neigh->dev); - if (ax25s) - ax25_cb_put(ax25s); + ax25 = ax25_send_frame(skb, 260, rose_call, &neigh->callsign, neigh->digipeat, neigh->dev); + rose_neigh_set_ax25(neigh, ax25); - return neigh->ax25 != NULL; + return !!ax25; } /* @@ -118,19 +153,17 @@ static int rose_send_frame(struct sk_buff *skb, struct rose_neigh *neigh) static int rose_link_up(struct rose_neigh *neigh) { const ax25_address *rose_call; - ax25_cb *ax25s; + ax25_cb *ax25; if (ax25cmp(&rose_callsign, &null_ax25_address) == 0) rose_call = (const ax25_address *)neigh->dev->dev_addr; else rose_call = &rose_callsign; - ax25s = neigh->ax25; - neigh->ax25 = ax25_find_cb(rose_call, &neigh->callsign, neigh->digipeat, neigh->dev); - if (ax25s) - ax25_cb_put(ax25s); + ax25 = ax25_find_cb(rose_call, &neigh->callsign, neigh->digipeat, neigh->dev); + rose_neigh_set_ax25(neigh, ax25); - return neigh->ax25 != NULL; + return !!ax25; } /* diff --git a/net/rose/rose_loopback.c b/net/rose/rose_loopback.c index 4e4a473910cc..6c5c35030e67 100644 --- a/net/rose/rose_loopback.c +++ b/net/rose/rose_loopback.c @@ -109,6 +109,7 @@ static void rose_loopback_timer(struct timer_list *unused) if (sk) { if (rose_rcv_frame(sk, skb) == 0) kfree_skb(skb); + sock_put(sk); continue; } diff --git a/net/rose/rose_route.c b/net/rose/rose_route.c index 6a1de0055e28..00fbe47fffe0 100644 --- a/net/rose/rose_route.c +++ b/net/rose/rose_route.c @@ -102,8 +102,7 @@ static int __must_check rose_add_node(struct rose_route_struct *rose_route, skb_queue_head_init(&rose_neigh->queue); - timer_setup(&rose_neigh->ftimer, NULL, 0); - timer_setup(&rose_neigh->t0timer, NULL, 0); + rose_neigh_init_timers(rose_neigh); if (rose_route->ndigis != 0) { rose_neigh->digipeat = @@ -446,8 +445,7 @@ void rose_add_loopback_neigh(void) skb_queue_head_init(&sn->queue); - timer_setup(&sn->ftimer, NULL, 0); - timer_setup(&sn->t0timer, NULL, 0); + rose_neigh_init_timers(sn); spin_lock_bh(&rose_neigh_list_lock); sn->next = rose_neigh_list; @@ -726,6 +724,8 @@ struct rose_route *rose_route_free_lci(unsigned int lci, struct rose_neigh *neig { struct rose_route *rose_route; + lockdep_assert_held(&rose_route_list_lock); + for (rose_route = rose_route_list; rose_route != NULL; rose_route = rose_route->next) if ((rose_route->neigh1 == neigh && rose_route->lci1 == lci) || (rose_route->neigh2 == neigh && rose_route->lci2 == lci)) @@ -734,6 +734,23 @@ struct rose_route *rose_route_free_lci(unsigned int lci, struct rose_neigh *neig return NULL; } +/* + * Give a socket placing a call a free LCI on its neighbour. As for a + * through route in rose_route_frame(), the search and the assignment + * are done under rose_route_list_lock, so two calls cannot pick the + * same LCI. The socket must already be on the socket list. + */ +unsigned int rose_assign_lci(struct sock *sk) +{ + struct rose_sock *rose = rose_sk(sk); + + spin_lock_bh(&rose_route_list_lock); + rose->lci = rose_new_lci(rose->neighbour); + spin_unlock_bh(&rose_route_list_lock); + + return rose->lci; +} + /* * Find a neighbour or a route given a ROSE address. */ @@ -919,14 +936,13 @@ void rose_link_failed(ax25_cb *ax25, int reason) spin_lock_bh(&rose_neigh_list_lock); rose_neigh = rose_neigh_list; while (rose_neigh != NULL) { - if (rose_neigh->ax25 == ax25) + if (READ_ONCE(rose_neigh->ax25) == ax25) break; rose_neigh = rose_neigh->next; } if (rose_neigh != NULL) { - rose_neigh->ax25 = NULL; - ax25_cb_put(ax25); + rose_neigh_clear_ax25(rose_neigh, ax25); rose_del_route_by_neigh(rose_neigh); rose_kill_by_neigh(rose_neigh); @@ -1037,25 +1053,34 @@ int rose_route_frame(struct sk_buff *skb, ax25_cb *ax25) if (frametype == ROSE_CALL_REQUEST) { struct rose_sock *rose = rose_sk(sk); - /* Remove an existing unused socket */ - rose_clear_queues(sk); - rose->cause = ROSE_NETWORK_CONGESTION; - rose->diagnostic = 0; - rose_neigh_put(rose->neighbour); - rose->neighbour = NULL; - rose->lci = 0; - rose->state = ROSE_STATE_0; - sk->sk_state = TCP_CLOSE; - sk->sk_err = 0; - sk->sk_shutdown |= SEND_SHUTDOWN; - if (!sock_flag(sk, SOCK_DEAD)) { - sk->sk_state_change(sk); - sock_set_flag(sk, SOCK_DEAD); + /* + * Remove an existing unused socket, unless it was + * disconnected or reaped since it was found. + */ + bh_lock_sock(sk); + if (rose->lci == lci && rose->neighbour == rose_neigh) { + rose_clear_queues(sk); + rose->cause = ROSE_NETWORK_CONGESTION; + rose->diagnostic = 0; + rose_neigh_put(rose->neighbour); + rose->neighbour = NULL; + rose->lci = 0; + rose->state = ROSE_STATE_0; + sk->sk_state = TCP_CLOSE; + sk->sk_err = 0; + sk->sk_shutdown |= SEND_SHUTDOWN; + if (!sock_flag(sk, SOCK_DEAD)) { + sk->sk_state_change(sk); + sock_set_flag(sk, SOCK_DEAD); + } } + bh_unlock_sock(sk); + sock_put(sk); } else { skb_reset_transport_header(skb); res = rose_rcv_frame(sk, skb); + sock_put(sk); goto out; } } diff --git a/net/rose/sysctl_net_rose.c b/net/rose/sysctl_net_rose.c index d801315b7083..a82735c34871 100644 --- a/net/rose/sysctl_net_rose.c +++ b/net/rose/sysctl_net_rose.c @@ -9,13 +9,14 @@ #include #include -static int min_timer[] = {1 * HZ}; -static int max_timer[] = {300 * HZ}; -static int min_idle[] = {0 * HZ}; -static int max_idle[] = {65535 * HZ}; +/* The timers are given in milliseconds */ +static int min_timer[] = {1 * MSEC_PER_SEC}; +static int max_timer[] = {300 * MSEC_PER_SEC}; +static int min_idle[] = {0 * MSEC_PER_SEC}; +static int max_idle[] = {65535 * MSEC_PER_SEC}; static int min_route[1], max_route[] = {1}; -static int min_ftimer[] = {60 * HZ}; -static int max_ftimer[] = {600 * HZ}; +static int min_ftimer[] = {60 * MSEC_PER_SEC}; +static int max_ftimer[] = {600 * MSEC_PER_SEC}; static int min_maxvcs[] = {1}, max_maxvcs[] = {254}; static int min_window[] = {1}, max_window[] = {7}; diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c index c9b85848e825..9d68e58dace9 100644 --- a/net/rxrpc/sendmsg.c +++ b/net/rxrpc/sendmsg.c @@ -279,10 +279,9 @@ static void rxrpc_queue_packet(struct rxrpc_sock *rx, struct rxrpc_call *call, static int rxrpc_send_data(struct rxrpc_sock *rx, struct rxrpc_call *call, struct msghdr *msg, size_t len, - rxrpc_notify_end_tx_t notify_end_tx, - bool *_dropped_lock) + rxrpc_notify_end_tx_t notify_end_tx) + __releases(&call->user_mutex) { - struct rxrpc_txbuf *txb; struct sock *sk = &rx->sk; enum rxrpc_call_state state; long timeo; @@ -293,23 +292,18 @@ static int rxrpc_send_data(struct rxrpc_sock *rx, ret = rxrpc_wait_to_be_connected(call, &timeo); if (ret < 0) - return ret; + goto out_unlock; if (call->conn->state == RXRPC_CONN_CLIENT_UNSECURED) { ret = rxrpc_init_client_conn_security(call->conn); if (ret < 0) - return ret; + goto out_unlock; } /* this should be in poll */ sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk); reload: - txb = call->tx_pending; - call->tx_pending = NULL; - if (txb) - rxrpc_see_txbuf(txb, rxrpc_txbuf_see_send_more); - ret = -EPIPE; if (sk->sk_shutdown & SEND_SHUTDOWN) goto maybe_error; @@ -337,6 +331,8 @@ static int rxrpc_send_data(struct rxrpc_sock *rx, } do { + struct rxrpc_txbuf *txb = call->tx_pending; + if (!txb) { size_t remain; @@ -355,6 +351,9 @@ static int rxrpc_send_data(struct rxrpc_sock *rx, ret = -ENOMEM; goto maybe_error; } + call->tx_pending = txb; + } else { + rxrpc_see_txbuf(txb, rxrpc_txbuf_see_send_more); } _debug("append"); @@ -392,12 +391,12 @@ static int rxrpc_send_data(struct rxrpc_sock *rx, ret = call->security->secure_packet(call, txb); if (ret < 0) - goto out; + goto out_unlock; txb->kvec[0].iov_len += txb->len; txb->len = txb->kvec[0].iov_len; rxrpc_queue_packet(rx, call, txb, notify_end_tx); - txb = NULL; + call->tx_pending = NULL; } } while (msg_data_left(msg) > 0); @@ -406,45 +405,46 @@ static int rxrpc_send_data(struct rxrpc_sock *rx, if (rxrpc_call_is_complete(call) && call->error < 0) ret = call->error; -out: - call->tx_pending = txb; +out_unlock: + mutex_unlock(&call->user_mutex); _leave(" = %d", ret); return ret; call_terminated: - rxrpc_put_txbuf(txb, rxrpc_txbuf_put_send_aborted); - _leave(" = %d", call->error); - return call->error; + ret = call->error; + goto out_unlock; maybe_error: if (copied) goto success; - goto out; + goto out_unlock; efault: ret = -EFAULT; - goto out; + goto out_unlock; wait_for_space: ret = -EAGAIN; if (msg->msg_flags & MSG_DONTWAIT) goto maybe_error; mutex_unlock(&call->user_mutex); - *_dropped_lock = true; + ret = rxrpc_wait_for_tx_window(rx, call, &timeo, msg->msg_flags & MSG_WAITALL); if (ret < 0) - goto maybe_error; + goto out_nolock; if (call->interruptibility == RXRPC_INTERRUPTIBLE) { if (mutex_lock_interruptible(&call->user_mutex) < 0) { ret = sock_intr_errno(timeo); - goto maybe_error; + goto out_nolock; } } else { mutex_lock(&call->user_mutex); } - *_dropped_lock = false; goto reload; +out_nolock: + _leave(" = %d [intr]", ret); + return copied ?: ret; } /* @@ -611,7 +611,6 @@ int rxrpc_do_sendmsg(struct rxrpc_sock *rx, struct msghdr *msg, size_t len) __releases(&rx->sk.sk_lock.slock) { struct rxrpc_call *call; - bool dropped_lock = false; int ret; struct rxrpc_send_params p = { @@ -717,12 +716,12 @@ int rxrpc_do_sendmsg(struct rxrpc_sock *rx, struct msghdr *msg, size_t len) } else if (p.command != RXRPC_CMD_SEND_DATA) { ret = -EINVAL; } else { - ret = rxrpc_send_data(rx, call, msg, len, NULL, &dropped_lock); + ret = rxrpc_send_data(rx, call, msg, len, NULL); + goto error_put; } out_put_unlock: - if (!dropped_lock) - mutex_unlock(&call->user_mutex); + mutex_unlock(&call->user_mutex); error_put: rxrpc_put_call(call, rxrpc_call_put_sendmsg); _leave(" = %d", ret); @@ -752,7 +751,6 @@ int rxrpc_kernel_send_data(struct socket *sock, struct rxrpc_call *call, struct msghdr *msg, size_t len, rxrpc_notify_end_tx_t notify_end_tx) { - bool dropped_lock = false; int ret; _enter("{%d},", call->debug_id); @@ -763,12 +761,9 @@ int rxrpc_kernel_send_data(struct socket *sock, struct rxrpc_call *call, mutex_lock(&call->user_mutex); ret = rxrpc_send_data(rxrpc_sk(sock->sk), call, msg, len, - notify_end_tx, &dropped_lock); + notify_end_tx); if (ret == -ESHUTDOWN) ret = call->error; - - if (!dropped_lock) - mutex_unlock(&call->user_mutex); _leave(" = %d", ret); return ret; } diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c index b8619b306fca..e5b65ea313c8 100644 --- a/net/sched/sch_generic.c +++ b/net/sched/sch_generic.c @@ -70,9 +70,34 @@ static void qdisc_maybe_clear_missed(struct Qdisc *q, #define SKB_XOFF_MAGIC ((struct sk_buff *)1UL) +/* Re-cap a stale queue_mapping; an ONETXQUEUE qdisc pins q->dev_queue. */ +static u16 qdisc_cap_skb_tx_queue(struct Qdisc *q, struct sk_buff *skb) +{ + u16 queue = skb_get_queue_mapping(skb); + struct net_device *dev = qdisc_dev(q); + u16 capped; + + if (q->flags & TCQ_F_ONETXQUEUE) + capped = q->dev_queue - netdev_get_tx_queue(dev, 0); + else + capped = netdev_cap_txqueue(dev, queue); + + if (unlikely(capped != queue)) + skb_set_queue_mapping(skb, capped); + + return capped; +} + +static struct netdev_queue *skb_cap_tx_queue(struct Qdisc *q, + struct sk_buff *skb) +{ + return netdev_get_tx_queue(qdisc_dev(q), + qdisc_cap_skb_tx_queue(q, skb)); +} + static inline struct sk_buff *__skb_dequeue_bad_txq(struct Qdisc *q) { - const struct netdev_queue *txq = q->dev_queue; + const struct netdev_queue *txq; spinlock_t *lock = NULL; struct sk_buff *skb; @@ -84,7 +109,8 @@ static inline struct sk_buff *__skb_dequeue_bad_txq(struct Qdisc *q) skb = skb_peek(&q->skb_bad_txq); if (skb) { /* check the reason of requeuing without tx lock first */ - txq = skb_get_tx_queue(txq->dev, skb); + /* re-cap: real_num_tx_queues may have shrunk */ + txq = skb_cap_tx_queue(q, skb); if (!netif_xmit_frozen_or_stopped(txq)) { skb = __skb_dequeue(&q->skb_bad_txq); if (qdisc_is_percpu_stats(q)) { @@ -198,6 +224,8 @@ static void try_bulk_dequeue_skb(struct Qdisc *q, if (!nskb) break; + /* Every element of an ONETXQUEUE qdisc is pinned to q->dev_queue. */ + qdisc_cap_skb_tx_queue(q, nskb); bytelimit -= nskb->len; /* covers GSO len */ skb->next = nskb; skb = nskb; @@ -215,7 +243,6 @@ static void try_bulk_dequeue_skb_slow(struct Qdisc *q, struct sk_buff *skb, int *packets) { - int mapping = skb_get_queue_mapping(skb); struct sk_buff *nskb; int cnt = 0; @@ -223,7 +250,9 @@ static void try_bulk_dequeue_skb_slow(struct Qdisc *q, nskb = q->dequeue(q); if (!nskb) break; - if (unlikely(skb_get_queue_mapping(nskb) != mapping)) { + /* A tc BPF store may have changed the mapping; cap it. */ + if (unlikely(qdisc_cap_skb_tx_queue(q, nskb) != + skb_get_queue_mapping(skb))) { qdisc_enqueue_skb_bad_txq(q, nskb); break; } @@ -268,7 +297,7 @@ static struct sk_buff *dequeue_skb(struct Qdisc *q, bool *validate, if (xfrm_offload(skb)) *validate = true; /* check the reason of requeuing without tx lock first */ - txq = skb_get_tx_queue(txq->dev, skb); + txq = skb_cap_tx_queue(q, skb); if (!netif_xmit_frozen_or_stopped(txq)) { skb = __skb_dequeue(&q->gso_skb); if (qdisc_is_percpu_stats(q)) { @@ -304,6 +333,8 @@ static struct sk_buff *dequeue_skb(struct Qdisc *q, bool *validate, skb = q->dequeue(q); if (skb) { bulk: + /* cap the head; the bulk helpers cap every appended element */ + qdisc_cap_skb_tx_queue(q, skb); if (qdisc_may_bulk(q)) try_bulk_dequeue_skb(q, skb, txq, packets, budget); else diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c index 70a08f195010..b0a9966cc6ab 100644 --- a/net/sched/sch_qfq.c +++ b/net/sched/sch_qfq.c @@ -520,11 +520,14 @@ static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid, sch_tree_lock(sch); qfq_init_agg(q, new_agg, lmax, weight); } + if (existing && new_agg == cl->agg) + goto unlock; if (existing) qfq_deact_rm_from_agg(q, cl); else qdisc_class_hash_insert(&q->clhash, &cl->common); qfq_add_to_agg(q, new_agg, cl); +unlock: sch_tree_unlock(sch); qdisc_class_hash_grow(sch, &q->clhash); diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c index 5091ffc3f475..54d0136dfe6c 100644 --- a/net/sched/sch_taprio.c +++ b/net/sched/sch_taprio.c @@ -631,6 +631,11 @@ static int taprio_enqueue(struct sk_buff *skb, struct Qdisc *sch, int queue; queue = skb_get_queue_mapping(skb); + if (unlikely(queue >= qdisc_dev(sch)->real_num_tx_queues)) { + /* A tc BPF program may have stored an out-of-range mapping. */ + queue = netdev_cap_txqueue(qdisc_dev(sch), queue); + skb_set_queue_mapping(skb, queue); + } child = q->qdiscs[queue]; if (unlikely(child == &noop_qdisc)) diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c index 1bbc041c0644..6dae9831613a 100644 --- a/net/sctp/sm_make_chunk.c +++ b/net/sctp/sm_make_chunk.c @@ -95,7 +95,7 @@ static void sctp_control_release_owner(struct sk_buff *skb) } } -static void sctp_control_set_owner_w(struct sctp_chunk *chunk) +void sctp_control_set_owner_w(struct sctp_chunk *chunk) { struct sctp_association *asoc = chunk->asoc; struct sk_buff *skb = chunk->skb; @@ -108,7 +108,8 @@ static void sctp_control_set_owner_w(struct sctp_chunk *chunk) * For now don't do anything for now. */ if (chunk->auth) { - chunk->shkey = asoc->shkey; + if (!chunk->shkey) + chunk->shkey = asoc->shkey; sctp_auth_shkey_hold(chunk->shkey); } skb->sk = asoc ? asoc->base.sk : NULL; @@ -145,6 +146,7 @@ int sctp_init_cause(struct sctp_chunk *chunk, __be16 cause_code, size_t paylen) { struct sctp_errhdr err; + int chunklen, padlen; __u16 len; /* Cause code constants are now defined in network order. */ @@ -152,7 +154,13 @@ int sctp_init_cause(struct sctp_chunk *chunk, __be16 cause_code, len = sizeof(err) + paylen; err.length = htons(len); - if (skb_tailroom(chunk->skb) < len) + /* sctp_addto_chunk() pads the chunk up to a 4-byte boundary before + * appending the cause, so account for that padding here as well. + */ + chunklen = ntohs(chunk->chunk_hdr->length); + padlen = SCTP_PAD4(chunklen) - chunklen; + + if (skb_tailroom(chunk->skb) < padlen + len) return -ENOSPC; chunk->subh.err_hdr = sctp_addto_chunk(chunk, sizeof(err), &err); diff --git a/net/sctp/socket.c b/net/sctp/socket.c index e2885a994314..7734ca92335a 100644 --- a/net/sctp/socket.c +++ b/net/sctp/socket.c @@ -149,25 +149,47 @@ static inline void sctp_set_owner_w(struct sctp_chunk *chunk) static void sctp_clear_owner_w(struct sctp_chunk *chunk) { + /* Keep the shkey alive until the new owner takes its reference. */ + if (chunk->shkey) + sctp_auth_shkey_hold(chunk->shkey); skb_orphan(chunk->skb); } +static void sctp_set_owner_w_migrate(struct sctp_chunk *chunk) +{ + struct sctp_shared_key *shkey = chunk->shkey; + + if (chunk->msg) + sctp_set_owner_w(chunk); + else + sctp_control_set_owner_w(chunk); + + if (shkey) + sctp_auth_shkey_release(shkey); +} + +static void sctp_process_tx_chunk(struct sctp_association *asoc, + struct sctp_chunk *chunk, bool clear, + void (*cb)(struct sctp_chunk *)) +{ + if ((clear && asoc->base.sk == chunk->skb->sk) || + (!clear && asoc->base.sk != chunk->skb->sk)) + cb(chunk); +} + #define traverse_and_process() \ do { \ msg = chunk->msg; \ if (msg == prev_msg) \ continue; \ - list_for_each_entry(c, &msg->chunks, frag_list) { \ - if ((clear && asoc->base.sk == c->skb->sk) || \ - (!clear && asoc->base.sk != c->skb->sk)) \ - cb(c); \ - } \ + list_for_each_entry(c, &msg->chunks, frag_list) \ + sctp_process_tx_chunk(asoc, c, clear, cb); \ prev_msg = msg; \ } while (0) -static void sctp_for_each_tx_datachunk(struct sctp_association *asoc, - bool clear, - void (*cb)(struct sctp_chunk *)) +static void sctp_for_each_tx_chunk(struct sctp_association *asoc, + bool clear, + void (*cb)(struct sctp_chunk *)) { struct sctp_datamsg *msg, *prev_msg = NULL; @@ -190,6 +212,21 @@ static void sctp_for_each_tx_datachunk(struct sctp_association *asoc, list_for_each_entry(chunk, &q->out_chunk_list, list) traverse_and_process(); + + list_for_each_entry(chunk, &q->control_chunk_list, list) + sctp_process_tx_chunk(asoc, chunk, clear, cb); + + list_for_each_entry(chunk, &asoc->asconf_ack_list, transmitted_list) + sctp_process_tx_chunk(asoc, chunk, clear, cb); + + list_for_each_entry(chunk, &asoc->addip_chunk_list, list) + sctp_process_tx_chunk(asoc, chunk, clear, cb); + + if (asoc->strreset_chunk) + sctp_process_tx_chunk(asoc, asoc->strreset_chunk, clear, cb); + + if (asoc->addip_last_asconf) + sctp_process_tx_chunk(asoc, asoc->addip_last_asconf, clear, cb); } static void sctp_for_each_rx_skb(struct sctp_association *asoc, struct sock *sk, @@ -9705,9 +9742,9 @@ static int sctp_sock_migrate(struct sock *oldsk, struct sock *newsk, * paths won't try to lock it and then oldsk. */ lock_sock_nested(newsk, SINGLE_DEPTH_NESTING); - sctp_for_each_tx_datachunk(assoc, true, sctp_clear_owner_w); + sctp_for_each_tx_chunk(assoc, true, sctp_clear_owner_w); sctp_assoc_migrate(assoc, newsk); - sctp_for_each_tx_datachunk(assoc, false, sctp_set_owner_w); + sctp_for_each_tx_chunk(assoc, false, sctp_set_owner_w_migrate); /* If the association on the newsk is already closed before accept() * is called, set RCV_SHUTDOWN flag. diff --git a/net/smc/smc_rx.c b/net/smc/smc_rx.c index 56dc3175f047..947ac2e1b16b 100644 --- a/net/smc/smc_rx.c +++ b/net/smc/smc_rx.c @@ -286,6 +286,27 @@ int smc_rx_wait(struct smc_sock *smc, long *timeo, size_t peeked, return rc; } +/* Wait for the pipe to release the spliced data. Unlike smc_rx_wait() + * it does not end on shutdown or sk_err, the data is already in the + * RMB; smc_conn_kill() wakes it and the caller checks conn->killed. + */ +static void smc_rx_wait_splice(struct smc_sock *smc, long *timeo) +{ + DEFINE_WAIT_FUNC(wait, woken_wake_function); + struct smc_connection *conn = &smc->conn; + struct sock *sk = &smc->sk; + + sk_set_bit(SOCKWQ_ASYNC_WAITDATA, sk); + add_wait_queue(sk_sleep(sk), &wait); + /* pairs with the smp_mb() in wq_has_sleeper(): the release drops + * splice_pending, then looks for a sleeper + */ + smp_mb(); + sk_wait_event(sk, timeo, !atomic_read(&conn->splice_pending), &wait); + remove_wait_queue(sk_sleep(sk), &wait); + sk_clear_bit(SOCKWQ_ASYNC_WAITDATA, sk); +} + static int smc_rx_recv_urg(struct smc_sock *smc, struct msghdr *msg, int len, int flags) { @@ -443,7 +464,23 @@ int smc_rx_recvmsg(struct smc_sock *smc, struct msghdr *msg, /* could be just 1 byte, even after waiting on data above */ readable = smc_rx_data_available(conn, peeked_bytes); splbytes = atomic_read(&conn->splice_pending); - if (!readable || (msg && splbytes)) { + if (msg && splbytes) { + /* a pipe holds bytes ahead of these; the copy path + * skips the timeout and signal checks + */ + if (!read_remaining) + break; + if (!timeo || signal_pending(current)) { + if (!read_done) + read_done = timeo ? + sock_intr_errno(timeo) : + -EAGAIN; + break; + } + smc_rx_wait_splice(smc, &timeo); + continue; + } + if (!readable) { if (splbytes) func = smc_rx_data_available_and_no_splice_pend; else diff --git a/net/sunrpc/rpcb_clnt.c b/net/sunrpc/rpcb_clnt.c index 2d74c660de66..9b90350820be 100644 --- a/net/sunrpc/rpcb_clnt.c +++ b/net/sunrpc/rpcb_clnt.c @@ -400,7 +400,8 @@ static struct rpc_clnt *rpcb_create(struct net *net, const char *nodename, return rpc_create(&args); } -static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, struct rpc_message *msg, bool is_set) +static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, + struct rpc_message *msg, bool is_set) { int flags = RPC_TASK_NOCONNECT; int error, result = 0; @@ -410,8 +411,23 @@ static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, stru msg->rpc_resp = &result; error = rpc_call_sync(clnt, msg, flags); - if (error < 0) - return error; + if (error < 0) { + switch (error) { + /* rpcbind answered; the reply itself carries the error */ + case -EPROTONOSUPPORT: + case -EPFNOSUPPORT: + case -EOPNOTSUPP: + case -EACCES: + /* the call never made it onto the wire */ + case -ENOMEM: + case -EMSGSIZE: + /* the caller is going away; this says nothing about rpcbind */ + case -ERESTARTSYS: + return error; + } + /* anything else, we assume that rpcbind isn't functional */ + return -EIO; + } if (!result) return -EACCES; diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index 4b4a2890a874..6d43f479342d 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -1204,10 +1204,40 @@ int svc_generic_rpcbind_set(struct net *net, error = svc_rpcbind_set_version(net, progp, version, family, proto, port); + /* -EIO means no answer, not a refusal, so vs_rpcb_optnl must keep it. */ + if (error == -EIO) + return error; + return (vers->vs_rpcb_optnl) ? 0 : error; } EXPORT_SYMBOL_GPL(svc_generic_rpcbind_set); +/** + * svc_rpcb_failure_count - local rpcbind calls for @serv that got no answer + * @serv: RPC service to query + * + * svc_register() adds one for each of its calls that got no answer. A reply + * that refuses one entry does not count, because rpcbind answered and the + * next entry may still succeed. + * + * The count is kept per serv rather than per net. The local rpcbind client + * is per-net and lockd shares it, but a count that another service can move + * says nothing about this serv's own calls. + * + * This is for callers that cannot see the svc_register() return, because a + * transport class sits in between. Such a caller reads the count before it + * starts and compares as it goes, so there is no state to reset between + * operations. The count never resets, and callers must not attach meaning + * to the value itself. + * + * Return: the number of unanswered calls since this serv was created. + */ +unsigned int svc_rpcb_failure_count(struct svc_serv *serv) +{ + return atomic_read(&serv->sv_rpcb_failures); +} +EXPORT_SYMBOL_GPL(svc_rpcb_failure_count); + /** * svc_register - register an RPC service with the local portmapper * @serv: svc_serv struct for the service to register @@ -1218,10 +1248,11 @@ EXPORT_SYMBOL_GPL(svc_generic_rpcbind_set); * * Service is registered for any address in the passed-in protocol family */ -int svc_register(const struct svc_serv *serv, struct net *net, +int svc_register(struct svc_serv *serv, struct net *net, const int family, const unsigned short proto, const unsigned short port) { + bool noanswer = false; unsigned int p, i; int error = 0; @@ -1233,10 +1264,16 @@ int svc_register(const struct svc_serv *serv, struct net *net, struct svc_program *progp = &serv->sv_programs[p]; for (i = 0; i < progp->pg_nvers; i++) { + const struct svc_version *vers = progp->pg_vers[i]; int ret; ret = progp->pg_rpcbind_set(net, progp, i, family, proto, port); + if (ret == -EIO) { + noanswer = true; + if (vers && vers->vs_rpcb_optnl) + ret = 0; + } if (ret < 0) { printk(KERN_WARNING "svc: failed to register " "%sv%u RPC service (errno %d).\n", @@ -1248,6 +1285,9 @@ int svc_register(const struct svc_serv *serv, struct net *net, } } + if (noanswer) + atomic_inc(&serv->sv_rpcb_failures); + return error; } diff --git a/net/wireless/reg.c b/net/wireless/reg.c index daf51fb56648..04bb6d7f955c 100644 --- a/net/wireless/reg.c +++ b/net/wireless/reg.c @@ -691,6 +691,9 @@ static bool valid_rule(const u8 *data, unsigned int size, u16 rule_ptr) /* mandatory fields */ if (rule->len < offsetofend(struct fwdb_rule, max_bw)) return false; + /* the rule must lie entirely within the database */ + if ((u8 *)rule + rule->len > data + size) + return false; if (rule->len >= offsetofend(struct fwdb_rule, wmm_ptr)) { u32 wmm_ptr = be16_to_cpu(rule->wmm_ptr) << 2; struct fwdb_wmm_rule *wmm; diff --git a/net/xfrm/espintcp.c b/net/xfrm/espintcp.c index c4291ea86d1d..67254678180e 100644 --- a/net/xfrm/espintcp.c +++ b/net/xfrm/espintcp.c @@ -437,7 +437,9 @@ static void espintcp_destruct(struct sock *sk) bool tcp_is_ulp_esp(struct sock *sk) { - return sk->sk_prot == &espintcp_prot || sk->sk_prot == &espintcp6_prot; + const struct proto *prot = READ_ONCE(sk->sk_prot); + + return prot == &espintcp_prot || prot == &espintcp6_prot; } EXPORT_SYMBOL_GPL(tcp_is_ulp_esp); @@ -469,34 +471,41 @@ static int espintcp_init_sk(struct sock *sk) __sk_dst_reset(sk); - strp_check_rcv(&ctx->strp); skb_queue_head_init(&ctx->ike_queue); skb_queue_head_init(&ctx->out_queue); + INIT_WORK(&ctx->work, espintcp_tx_work); + + ctx->saved_data_ready = sk->sk_data_ready; + ctx->saved_write_space = sk->sk_write_space; + ctx->saved_destruct = sk->sk_destruct; + + /* avoid using task_frag */ + sk->sk_allocation = GFP_ATOMIC; + sk->sk_use_task_frag = false; + + rcu_assign_pointer(icsk->icsk_ulp_data, ctx); + + /* pairs with smp_rmb() in espintcp_getctx() */ + smp_wmb(); + + WRITE_ONCE(sk->sk_data_ready, espintcp_data_ready); + WRITE_ONCE(sk->sk_write_space, espintcp_write_space); + sk->sk_destruct = espintcp_destruct; if (sk->sk_family == AF_INET) { - sk->sk_prot = &espintcp_prot; - sk->sk_socket->ops = &espintcp_ops; + WRITE_ONCE(sk->sk_prot, &espintcp_prot); + WRITE_ONCE(sk->sk_socket->ops, &espintcp_ops); } else { mutex_lock(&tcpv6_prot_mutex); if (!espintcp6_prot.recvmsg) build_protos(&espintcp6_prot, &espintcp6_ops, sk->sk_prot, sk->sk_socket->ops); mutex_unlock(&tcpv6_prot_mutex); - sk->sk_prot = &espintcp6_prot; - sk->sk_socket->ops = &espintcp6_ops; + WRITE_ONCE(sk->sk_prot, &espintcp6_prot); + WRITE_ONCE(sk->sk_socket->ops, &espintcp6_ops); } - ctx->saved_data_ready = sk->sk_data_ready; - ctx->saved_write_space = sk->sk_write_space; - ctx->saved_destruct = sk->sk_destruct; - sk->sk_data_ready = espintcp_data_ready; - sk->sk_write_space = espintcp_write_space; - sk->sk_destruct = espintcp_destruct; - rcu_assign_pointer(icsk->icsk_ulp_data, ctx); - INIT_WORK(&ctx->work, espintcp_tx_work); - /* avoid using task_frag */ - sk->sk_allocation = GFP_ATOMIC; - sk->sk_use_task_frag = false; + strp_check_rcv(&ctx->strp); return 0; diff --git a/net/xfrm/xfrm_interface_core.c b/net/xfrm/xfrm_interface_core.c index be0000448af2..2ff96577ff72 100644 --- a/net/xfrm/xfrm_interface_core.c +++ b/net/xfrm/xfrm_interface_core.c @@ -532,8 +532,13 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev) memset(&fl, 0, sizeof(fl)); + if (!pskb_inet_may_pull(skb)) + goto tx_err; + switch (skb->protocol) { case htons(ETH_P_IPV6): + if (ipv6_hdr(skb)->version != 6) + goto tx_err; memset(IP6CB(skb), 0, sizeof(*IP6CB(skb))); xfrm_decode_session(dev_net(dev), skb, &fl, AF_INET6); if (!dst) { @@ -549,6 +554,9 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev) } break; case htons(ETH_P_IP): + if (ip_hdr(skb)->version != 4 || ip_hdr(skb)->ihl < 5 || + !pskb_network_may_pull(skb, ip_hdr(skb)->ihl * 4)) + goto tx_err; memset(IPCB(skb), 0, sizeof(*IPCB(skb))); xfrm_decode_session(dev_net(dev), skb, &fl, AF_INET); if (!dst) { diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c index e033390e7785..106e2f5cc586 100644 --- a/net/xfrm/xfrm_policy.c +++ b/net/xfrm/xfrm_policy.c @@ -2157,6 +2157,7 @@ static struct xfrm_policy *xfrm_policy_lookup_bytype(struct net *net, u8 type, struct xfrm_pol_inexact_bin *bin; struct xfrm_policy *pol, *ret; struct hlist_head *chain; + unsigned int inexact_sequence; unsigned int sequence; int err; @@ -2191,12 +2192,18 @@ static struct xfrm_policy *xfrm_policy_lookup_bytype(struct net *net, u8 type, goto skip_inexact; bin = xfrm_policy_inexact_lookup_rcu(net, type, family, dir, if_id); - if (!bin || !xfrm_policy_find_inexact_candidates(&cand, bin, saddr, - daddr)) + if (!bin) + goto skip_inexact; + + inexact_sequence = read_seqcount_begin(&bin->count); + if (!xfrm_policy_find_inexact_candidates(&cand, bin, saddr, daddr)) goto skip_inexact; pol = xfrm_policy_eval_candidates(&cand, ret, fl, type, family, if_id); + if (read_seqcount_retry(&bin->count, inexact_sequence)) + goto retry; + if (pol) { ret = pol; if (IS_ERR(pol)) diff --git a/security/apparmor/policy_unpack.c b/security/apparmor/policy_unpack.c index 46420587bcd5..61d7ef232f32 100644 --- a/security/apparmor/policy_unpack.c +++ b/security/apparmor/policy_unpack.c @@ -1202,13 +1202,22 @@ static int verify_header(struct aa_ext *e, int required, const char **ns) * verify_dfa_accept_index - verify accept indexes are in range of perms table * @dfa: the dfa to check accept indexes are in range * @table_size: the permission table size the indexes should be within + * @pair: the accept indexes reference two slots, index and index + 1 */ -static bool verify_dfa_accept_index(struct aa_dfa *dfa, int table_size) +static bool verify_dfa_accept_index(struct aa_dfa *dfa, int table_size, + bool pair) { int i; for (i = 0; i < dfa->tables[YYTD_ID_ACCEPT]->td_lolen; i++) { if (ACCEPT_TABLE(dfa)[i] >= table_size) return false; + /* + * Accept indexes for owner-conditional permissions come + * in pairs, so the non-owner entry at index + 1 must also + * be in bounds. + */ + if (pair && ACCEPT_TABLE(dfa)[i] + 1 >= table_size) + return false; } return true; } @@ -1272,15 +1281,17 @@ static int verify_profile(struct aa_profile *profile) if (!rules) return 0; - if (rules->file->dfa && !verify_dfa_accept_index(rules->file->dfa, - rules->file->size)) { + if (rules->file->dfa && + !verify_dfa_accept_index(rules->file->dfa, rules->file->size, + rules->file != rules->policy)) { audit_iface(profile, NULL, NULL, "Unpack: file Invalid named transition", NULL, -EPROTO); return -EPROTO; } if (rules->policy->dfa && - !verify_dfa_accept_index(rules->policy->dfa, rules->policy->size)) { + !verify_dfa_accept_index(rules->policy->dfa, rules->policy->size, + false)) { audit_iface(profile, NULL, NULL, "Unpack: policy Invalid named transition", NULL, -EPROTO); diff --git a/sound/core/compress_offload.c b/sound/core/compress_offload.c index ae8ab99f53f8..ec0d4f34404c 100644 --- a/sound/core/compress_offload.c +++ b/sound/core/compress_offload.c @@ -1014,17 +1014,17 @@ static long snd_compr_ioctl_compat(struct file *file, unsigned int cmd, #endif static const struct file_operations snd_compr_file_ops = { - .owner = THIS_MODULE, - .open = snd_compr_open, - .release = snd_compr_free, - .write = snd_compr_write, - .read = snd_compr_read, - .unlocked_ioctl = snd_compr_ioctl, + .owner = THIS_MODULE, + .open = snd_compr_open, + .release = snd_compr_free, + .write = snd_compr_write, + .read = snd_compr_read, + .unlocked_ioctl = snd_compr_ioctl, #ifdef CONFIG_COMPAT - .compat_ioctl = snd_compr_ioctl_compat, + .compat_ioctl = snd_compr_ioctl_compat, #endif - .mmap = snd_compr_mmap, - .poll = snd_compr_poll, + .mmap = snd_compr_mmap, + .poll = snd_compr_poll, }; static int snd_compress_dev_register(struct snd_device *device) diff --git a/sound/core/control.c b/sound/core/control.c index 460d8dd8303f..d8f071218c0e 100644 --- a/sound/core/control.c +++ b/sound/core/control.c @@ -2269,16 +2269,15 @@ EXPORT_SYMBOL_GPL(snd_ctl_disconnect_layer); * INIT PART */ -static const struct file_operations snd_ctl_f_ops = -{ - .owner = THIS_MODULE, - .read = snd_ctl_read, - .open = snd_ctl_open, - .release = snd_ctl_release, - .poll = snd_ctl_poll, - .unlocked_ioctl = snd_ctl_ioctl, - .compat_ioctl = snd_ctl_ioctl_compat, - .fasync = snd_ctl_fasync, +static const struct file_operations snd_ctl_f_ops = { + .owner = THIS_MODULE, + .read = snd_ctl_read, + .open = snd_ctl_open, + .release = snd_ctl_release, + .poll = snd_ctl_poll, + .unlocked_ioctl = snd_ctl_ioctl, + .compat_ioctl = snd_ctl_ioctl_compat, + .fasync = snd_ctl_fasync, }; /* call lops under rwsems; called from snd_ctl_dev_*() below() */ diff --git a/sound/core/hwdep.c b/sound/core/hwdep.c index 09200df2932c..4334162bc0b2 100644 --- a/sound/core/hwdep.c +++ b/sound/core/hwdep.c @@ -212,6 +212,7 @@ static int snd_hwdep_dsp_load(struct snd_hwdep *hw, return -ENXIO; if (info->index >= 32) return -EINVAL; + guard(mutex)(&hw->dsp_mutex); /* check whether the dsp was already loaded */ if (hw->dsp_loaded & (1u << info->index)) return -EBUSY; @@ -323,18 +324,17 @@ static int snd_hwdep_control_ioctl(struct snd_card *card, */ -static const struct file_operations snd_hwdep_f_ops = -{ - .owner = THIS_MODULE, - .llseek = snd_hwdep_llseek, - .read = snd_hwdep_read, - .write = snd_hwdep_write, - .open = snd_hwdep_open, - .release = snd_hwdep_release, - .poll = snd_hwdep_poll, - .unlocked_ioctl = snd_hwdep_ioctl, - .compat_ioctl = snd_hwdep_ioctl_compat, - .mmap = snd_hwdep_mmap, +static const struct file_operations snd_hwdep_f_ops = { + .owner = THIS_MODULE, + .llseek = snd_hwdep_llseek, + .read = snd_hwdep_read, + .write = snd_hwdep_write, + .open = snd_hwdep_open, + .release = snd_hwdep_release, + .poll = snd_hwdep_poll, + .unlocked_ioctl = snd_hwdep_ioctl, + .compat_ioctl = snd_hwdep_ioctl_compat, + .mmap = snd_hwdep_mmap, }; static void snd_hwdep_free(struct snd_hwdep *hwdep) @@ -381,6 +381,7 @@ int snd_hwdep_new(struct snd_card *card, char *id, int device, init_waitqueue_head(&hwdep->open_wait); mutex_init(&hwdep->open_mutex); + mutex_init(&hwdep->dsp_mutex); hwdep->card = card; hwdep->device = device; if (id) diff --git a/sound/core/init.c b/sound/core/init.c index 92dff1736cfe..b37269b86be8 100644 --- a/sound/core/init.c +++ b/sound/core/init.c @@ -461,20 +461,19 @@ static int snd_disconnect_fasync(int fd, struct file *file, int on) return -ENODEV; } -static const struct file_operations snd_shutdown_f_ops = -{ - .owner = THIS_MODULE, - .llseek = snd_disconnect_llseek, - .read = snd_disconnect_read, - .write = snd_disconnect_write, - .release = snd_disconnect_release, - .poll = snd_disconnect_poll, - .unlocked_ioctl = snd_disconnect_ioctl, +static const struct file_operations snd_shutdown_f_ops = { + .owner = THIS_MODULE, + .llseek = snd_disconnect_llseek, + .read = snd_disconnect_read, + .write = snd_disconnect_write, + .release = snd_disconnect_release, + .poll = snd_disconnect_poll, + .unlocked_ioctl = snd_disconnect_ioctl, #ifdef CONFIG_COMPAT - .compat_ioctl = snd_disconnect_ioctl, + .compat_ioctl = snd_disconnect_ioctl, #endif - .mmap = snd_disconnect_mmap, - .fasync = snd_disconnect_fasync + .mmap = snd_disconnect_mmap, + .fasync = snd_disconnect_fasync }; /** diff --git a/sound/core/jack.c b/sound/core/jack.c index e4bcecdf89b7..c21f041bf457 100644 --- a/sound/core/jack.c +++ b/sound/core/jack.c @@ -304,41 +304,41 @@ static ssize_t jack_type_read(struct file *file, } static const struct file_operations jack_type_fops = { - .open = simple_open, - .read = jack_type_read, - .llseek = default_llseek, + .open = simple_open, + .read = jack_type_read, + .llseek = default_llseek, }; #endif static const struct file_operations sw_inject_enable_fops = { - .open = simple_open, - .read = sw_inject_enable_read, - .write = sw_inject_enable_write, - .llseek = default_llseek, + .open = simple_open, + .read = sw_inject_enable_read, + .write = sw_inject_enable_write, + .llseek = default_llseek, }; static const struct file_operations jackin_inject_fops = { - .open = simple_open, - .write = jackin_inject_write, - .llseek = default_llseek, + .open = simple_open, + .write = jackin_inject_write, + .llseek = default_llseek, }; static const struct file_operations jack_kctl_id_fops = { - .open = simple_open, - .read = jack_kctl_id_read, - .llseek = default_llseek, + .open = simple_open, + .read = jack_kctl_id_read, + .llseek = default_llseek, }; static const struct file_operations jack_kctl_mask_bits_fops = { - .open = simple_open, - .read = jack_kctl_mask_bits_read, - .llseek = default_llseek, + .open = simple_open, + .read = jack_kctl_mask_bits_read, + .llseek = default_llseek, }; static const struct file_operations jack_kctl_status_fops = { - .open = simple_open, - .read = jack_kctl_status_read, - .llseek = default_llseek, + .open = simple_open, + .read = jack_kctl_status_read, + .llseek = default_llseek, }; static int snd_jack_debugfs_add_inject_node(struct snd_jack *jack, diff --git a/sound/core/oss/mixer_oss.c b/sound/core/oss/mixer_oss.c index a6cc74bb25fd..34e4dc791119 100644 --- a/sound/core/oss/mixer_oss.c +++ b/sound/core/oss/mixer_oss.c @@ -407,13 +407,12 @@ static long snd_mixer_oss_ioctl_compat(struct file *file, unsigned int cmd, * REGISTRATION PART */ -static const struct file_operations snd_mixer_oss_f_ops = -{ - .owner = THIS_MODULE, - .open = snd_mixer_oss_open, - .release = snd_mixer_oss_release, - .unlocked_ioctl = snd_mixer_oss_ioctl, - .compat_ioctl = snd_mixer_oss_ioctl_compat, +static const struct file_operations snd_mixer_oss_f_ops = { + .owner = THIS_MODULE, + .open = snd_mixer_oss_open, + .release = snd_mixer_oss_release, + .unlocked_ioctl = snd_mixer_oss_ioctl, + .compat_ioctl = snd_mixer_oss_ioctl_compat, }; /* diff --git a/sound/core/oss/pcm_oss.c b/sound/core/oss/pcm_oss.c index a65a3b8d04b8..c49d71dd4280 100644 --- a/sound/core/oss/pcm_oss.c +++ b/sound/core/oss/pcm_oss.c @@ -3121,17 +3121,16 @@ static inline void snd_pcm_oss_proc_done(struct snd_pcm *pcm) * ENTRY functions */ -static const struct file_operations snd_pcm_oss_f_reg = -{ - .owner = THIS_MODULE, - .read = snd_pcm_oss_read, - .write = snd_pcm_oss_write, - .open = snd_pcm_oss_open, - .release = snd_pcm_oss_release, - .poll = snd_pcm_oss_poll, - .unlocked_ioctl = snd_pcm_oss_ioctl, - .compat_ioctl = snd_pcm_oss_ioctl_compat, - .mmap = snd_pcm_oss_mmap, +static const struct file_operations snd_pcm_oss_f_reg = { + .owner = THIS_MODULE, + .read = snd_pcm_oss_read, + .write = snd_pcm_oss_write, + .open = snd_pcm_oss_open, + .release = snd_pcm_oss_release, + .poll = snd_pcm_oss_poll, + .unlocked_ioctl = snd_pcm_oss_ioctl, + .compat_ioctl = snd_pcm_oss_ioctl_compat, + .mmap = snd_pcm_oss_mmap, }; static void register_oss_dsp(struct snd_pcm *pcm, int index) diff --git a/sound/core/pcm_native.c b/sound/core/pcm_native.c index 45b34e34412c..dd135ed91c99 100644 --- a/sound/core/pcm_native.c +++ b/sound/core/pcm_native.c @@ -4182,29 +4182,29 @@ static unsigned long snd_pcm_get_unmapped_area(struct file *file, const struct file_operations snd_pcm_f_ops[2] = { { - .owner = THIS_MODULE, - .write = snd_pcm_write, - .write_iter = snd_pcm_writev, - .open = snd_pcm_playback_open, - .release = snd_pcm_release, - .poll = snd_pcm_poll, - .unlocked_ioctl = snd_pcm_ioctl, - .compat_ioctl = snd_pcm_ioctl_compat, - .mmap = snd_pcm_mmap, - .fasync = snd_pcm_fasync, - .get_unmapped_area = snd_pcm_get_unmapped_area, + .owner = THIS_MODULE, + .write = snd_pcm_write, + .write_iter = snd_pcm_writev, + .open = snd_pcm_playback_open, + .release = snd_pcm_release, + .poll = snd_pcm_poll, + .unlocked_ioctl = snd_pcm_ioctl, + .compat_ioctl = snd_pcm_ioctl_compat, + .mmap = snd_pcm_mmap, + .fasync = snd_pcm_fasync, + .get_unmapped_area = snd_pcm_get_unmapped_area, }, { - .owner = THIS_MODULE, - .read = snd_pcm_read, - .read_iter = snd_pcm_readv, - .open = snd_pcm_capture_open, - .release = snd_pcm_release, - .poll = snd_pcm_poll, - .unlocked_ioctl = snd_pcm_ioctl, - .compat_ioctl = snd_pcm_ioctl_compat, - .mmap = snd_pcm_mmap, - .fasync = snd_pcm_fasync, - .get_unmapped_area = snd_pcm_get_unmapped_area, + .owner = THIS_MODULE, + .read = snd_pcm_read, + .read_iter = snd_pcm_readv, + .open = snd_pcm_capture_open, + .release = snd_pcm_release, + .poll = snd_pcm_poll, + .unlocked_ioctl = snd_pcm_ioctl, + .compat_ioctl = snd_pcm_ioctl_compat, + .mmap = snd_pcm_mmap, + .fasync = snd_pcm_fasync, + .get_unmapped_area = snd_pcm_get_unmapped_area, } }; diff --git a/sound/core/rawmidi.c b/sound/core/rawmidi.c index c7b39e07e7c3..1eb234250c58 100644 --- a/sound/core/rawmidi.c +++ b/sound/core/rawmidi.c @@ -1784,14 +1784,14 @@ static void snd_rawmidi_proc_info_read(struct snd_info_entry *entry, */ static const struct file_operations snd_rawmidi_f_ops = { - .owner = THIS_MODULE, - .read = snd_rawmidi_read, - .write = snd_rawmidi_write, - .open = snd_rawmidi_open, - .release = snd_rawmidi_release, - .poll = snd_rawmidi_poll, - .unlocked_ioctl = snd_rawmidi_ioctl, - .compat_ioctl = snd_rawmidi_ioctl_compat, + .owner = THIS_MODULE, + .read = snd_rawmidi_read, + .write = snd_rawmidi_write, + .open = snd_rawmidi_open, + .release = snd_rawmidi_release, + .poll = snd_rawmidi_poll, + .unlocked_ioctl = snd_rawmidi_ioctl, + .compat_ioctl = snd_rawmidi_ioctl_compat, }; static int snd_rawmidi_alloc_substreams(struct snd_rawmidi *rmidi, diff --git a/sound/core/seq/oss/seq_oss.c b/sound/core/seq/oss/seq_oss.c index 77c1214acd90..9de191f59c67 100644 --- a/sound/core/seq/oss/seq_oss.c +++ b/sound/core/seq/oss/seq_oss.c @@ -211,17 +211,16 @@ odev_poll(struct file *file, poll_table * wait) * registration of sequencer minor device */ -static const struct file_operations seq_oss_f_ops = -{ - .owner = THIS_MODULE, - .read = odev_read, - .write = odev_write, - .open = odev_open, - .release = odev_release, - .poll = odev_poll, - .unlocked_ioctl = odev_ioctl, - .compat_ioctl = odev_ioctl_compat, - .llseek = noop_llseek, +static const struct file_operations seq_oss_f_ops = { + .owner = THIS_MODULE, + .read = odev_read, + .write = odev_write, + .open = odev_open, + .release = odev_release, + .poll = odev_poll, + .unlocked_ioctl = odev_ioctl, + .compat_ioctl = odev_ioctl_compat, + .llseek = noop_llseek, }; static int __init diff --git a/sound/core/seq/seq_clientmgr.c b/sound/core/seq/seq_clientmgr.c index 86316cf027ae..4dc5b15029e4 100644 --- a/sound/core/seq/seq_clientmgr.c +++ b/sound/core/seq/seq_clientmgr.c @@ -2785,16 +2785,15 @@ void snd_seq_info_clients_read(struct snd_info_entry *entry, * REGISTRATION PART */ -static const struct file_operations snd_seq_f_ops = -{ - .owner = THIS_MODULE, - .read = snd_seq_read, - .write = snd_seq_write, - .open = snd_seq_open, - .release = snd_seq_release, - .poll = snd_seq_poll, - .unlocked_ioctl = snd_seq_ioctl, - .compat_ioctl = snd_seq_ioctl_compat, +static const struct file_operations snd_seq_f_ops = { + .owner = THIS_MODULE, + .read = snd_seq_read, + .write = snd_seq_write, + .open = snd_seq_open, + .release = snd_seq_release, + .poll = snd_seq_poll, + .unlocked_ioctl = snd_seq_ioctl, + .compat_ioctl = snd_seq_ioctl_compat, }; static struct device *seq_dev; diff --git a/sound/core/sound.c b/sound/core/sound.c index 7980b60f4ba0..5ce042db4755 100644 --- a/sound/core/sound.c +++ b/sound/core/sound.c @@ -167,11 +167,10 @@ static int snd_open(struct inode *inode, struct file *file) return err; } -static const struct file_operations snd_fops = -{ - .owner = THIS_MODULE, - .open = snd_open, - .llseek = noop_llseek, +static const struct file_operations snd_fops = { + .owner = THIS_MODULE, + .open = snd_open, + .llseek = noop_llseek, }; #ifdef CONFIG_SND_DYNAMIC_MINORS diff --git a/sound/core/timer.c b/sound/core/timer.c index a548885e8178..8f84055d3e53 100644 --- a/sound/core/timer.c +++ b/sound/core/timer.c @@ -2101,9 +2101,10 @@ static long snd_utimer_ioctl(struct file *file, unsigned int ioctl, unsigned lon } static const struct file_operations snd_utimer_fops = { - .llseek = noop_llseek, - .release = snd_utimer_release, - .unlocked_ioctl = snd_utimer_ioctl, + .llseek = noop_llseek, + .release = snd_utimer_release, + .unlocked_ioctl = snd_utimer_ioctl, + .compat_ioctl = snd_utimer_ioctl, }; static int snd_utimer_start(struct snd_timer *t) @@ -2455,16 +2456,15 @@ static __poll_t snd_timer_user_poll(struct file *file, poll_table * wait) #define snd_timer_user_ioctl_compat NULL #endif -static const struct file_operations snd_timer_f_ops = -{ - .owner = THIS_MODULE, - .read = snd_timer_user_read, - .open = snd_timer_user_open, - .release = snd_timer_user_release, - .poll = snd_timer_user_poll, - .unlocked_ioctl = snd_timer_user_ioctl, - .compat_ioctl = snd_timer_user_ioctl_compat, - .fasync = snd_timer_user_fasync, +static const struct file_operations snd_timer_f_ops = { + .owner = THIS_MODULE, + .read = snd_timer_user_read, + .open = snd_timer_user_open, + .release = snd_timer_user_release, + .poll = snd_timer_user_poll, + .unlocked_ioctl = snd_timer_user_ioctl, + .compat_ioctl = snd_timer_user_ioctl_compat, + .fasync = snd_timer_user_fasync, }; /* unregister the system timer */ diff --git a/sound/core/timer_compat.c b/sound/core/timer_compat.c index 4ae9eaeb5afb..7b648edcb9cb 100644 --- a/sound/core/timer_compat.c +++ b/sound/core/timer_compat.c @@ -98,6 +98,7 @@ static long __snd_timer_user_ioctl_compat(struct file *file, unsigned int cmd, case SNDRV_TIMER_IOCTL_PAUSE: case SNDRV_TIMER_IOCTL_PAUSE_OLD: case SNDRV_TIMER_IOCTL_NEXT_DEVICE: + case SNDRV_TIMER_IOCTL_CREATE: return __snd_timer_user_ioctl(file, cmd, (unsigned long)argp, true); case SNDRV_TIMER_IOCTL_GPARAMS32: return snd_timer_user_gparams_compat(file, argp); diff --git a/sound/core/ump_convert.c b/sound/core/ump_convert.c index 0a7972d91ec5..ecbd894f4c83 100644 --- a/sound/core/ump_convert.c +++ b/sound/core/ump_convert.c @@ -69,7 +69,7 @@ static u32 upscale_14_to_32bit(u16 src) static int cvt_ump_system_to_legacy(u32 data, unsigned char *buf) { buf[0] = ump_message_status_channel(data); - switch (ump_message_status_code(data)) { + switch (buf[0]) { case UMP_SYSTEM_STATUS_MIDI_TIME_CODE: case UMP_SYSTEM_STATUS_SONG_SELECT: buf[1] = (data >> 8) & 0x7f; diff --git a/sound/drivers/aloop.c b/sound/drivers/aloop.c index 50b8e7b3d817..94ea73b3869c 100644 --- a/sound/drivers/aloop.c +++ b/sound/drivers/aloop.c @@ -532,13 +532,12 @@ static int loopback_prepare(struct snd_pcm_substream *substream) dpcm->pcm_salign = salign; dpcm->pcm_period_size = frames_to_bytes(runtime, runtime->period_size); - mutex_lock(&dpcm->loopback->cable_lock); + guard(mutex)(&dpcm->loopback->cable_lock); if (!(cable->valid & ~(1 << substream->stream)) || (get_setup(dpcm)->notify && substream->stream == SNDRV_PCM_STREAM_PLAYBACK)) params_change(substream); cable->valid |= 1 << substream->stream; - mutex_unlock(&dpcm->loopback->cable_lock); return 0; } @@ -979,9 +978,8 @@ static int loopback_hw_free(struct snd_pcm_substream *substream) struct loopback_pcm *dpcm = runtime->private_data; struct loopback_cable *cable = dpcm->cable; - mutex_lock(&dpcm->loopback->cable_lock); + guard(mutex)(&dpcm->loopback->cable_lock); cable->valid &= ~(1 << substream->stream); - mutex_unlock(&dpcm->loopback->cable_lock); return 0; } @@ -993,6 +991,18 @@ static unsigned int get_cable_index(struct snd_pcm_substream *substream) return !substream->stream; } +/* + * In notify mode the playback side may change format, rate, channels and, + * with a sound timer, the period size. A running capture that no longer + * matches is stopped by loopback_check_format(). + * call in loopback->cable_lock + */ +static bool loopback_playback_unconstrained(struct loopback_pcm *dpcm) +{ + return dpcm->substream->stream == SNDRV_PCM_STREAM_PLAYBACK && + get_notify(dpcm); +} + static int rule_format(struct snd_pcm_hw_params *params, struct snd_pcm_hw_rule *rule) { @@ -1001,10 +1011,12 @@ static int rule_format(struct snd_pcm_hw_params *params, struct snd_mask m; snd_mask_none(&m); - mutex_lock(&dpcm->loopback->cable_lock); - m.bits[0] = (u_int32_t)cable->hw.formats; - m.bits[1] = (u_int32_t)(cable->hw.formats >> 32); - mutex_unlock(&dpcm->loopback->cable_lock); + scoped_guard(mutex, &dpcm->loopback->cable_lock) { + if (loopback_playback_unconstrained(dpcm)) + return 0; + m.bits[0] = (u_int32_t)cable->hw.formats; + m.bits[1] = (u_int32_t)(cable->hw.formats >> 32); + } return snd_mask_refine(hw_param_mask(params, rule->var), &m); } @@ -1015,10 +1027,12 @@ static int rule_rate(struct snd_pcm_hw_params *params, struct loopback_cable *cable = dpcm->cable; struct snd_interval t; - mutex_lock(&dpcm->loopback->cable_lock); - t.min = cable->hw.rate_min; - t.max = cable->hw.rate_max; - mutex_unlock(&dpcm->loopback->cable_lock); + scoped_guard(mutex, &dpcm->loopback->cable_lock) { + if (loopback_playback_unconstrained(dpcm)) + return 0; + t.min = cable->hw.rate_min; + t.max = cable->hw.rate_max; + } t.openmin = t.openmax = 0; t.integer = 0; return snd_interval_refine(hw_param_interval(params, rule->var), &t); @@ -1031,10 +1045,12 @@ static int rule_channels(struct snd_pcm_hw_params *params, struct loopback_cable *cable = dpcm->cable; struct snd_interval t; - mutex_lock(&dpcm->loopback->cable_lock); - t.min = cable->hw.channels_min; - t.max = cable->hw.channels_max; - mutex_unlock(&dpcm->loopback->cable_lock); + scoped_guard(mutex, &dpcm->loopback->cable_lock) { + if (loopback_playback_unconstrained(dpcm)) + return 0; + t.min = cable->hw.channels_min; + t.max = cable->hw.channels_max; + } t.openmin = t.openmax = 0; t.integer = 0; return snd_interval_refine(hw_param_interval(params, rule->var), &t); @@ -1047,10 +1063,12 @@ static int rule_period_bytes(struct snd_pcm_hw_params *params, struct loopback_cable *cable = dpcm->cable; struct snd_interval t; - mutex_lock(&dpcm->loopback->cable_lock); - t.min = cable->hw.period_bytes_min; - t.max = cable->hw.period_bytes_max; - mutex_unlock(&dpcm->loopback->cable_lock); + scoped_guard(mutex, &dpcm->loopback->cable_lock) { + if (loopback_playback_unconstrained(dpcm)) + return 0; + t.min = cable->hw.period_bytes_min; + t.max = cable->hw.period_bytes_max; + } t.openmin = 0; t.openmax = 0; t.integer = 0; @@ -1264,12 +1282,10 @@ static int loopback_open(struct snd_pcm_substream *substream) int err = 0; int dev = get_cable_index(substream); - mutex_lock(&loopback->cable_lock); + guard(mutex)(&loopback->cable_lock); dpcm = kzalloc(sizeof(*dpcm), GFP_KERNEL); - if (!dpcm) { - err = -ENOMEM; - goto unlock; - } + if (!dpcm) + return -ENOMEM; dpcm->loopback = loopback; dpcm->substream = substream; @@ -1354,7 +1370,6 @@ static int loopback_open(struct snd_pcm_substream *substream) free_cable(substream); kfree(dpcm); } - mutex_unlock(&loopback->cable_lock); return err; } @@ -1366,9 +1381,8 @@ static int loopback_close(struct snd_pcm_substream *substream) if (dpcm->cable->ops->close_substream) err = dpcm->cable->ops->close_substream(dpcm); - mutex_lock(&loopback->cable_lock); + guard(mutex)(&loopback->cable_lock); free_cable(substream); - mutex_unlock(&loopback->cable_lock); return err; } @@ -1419,11 +1433,10 @@ static int loopback_rate_shift_get(struct snd_kcontrol *kcontrol, { struct loopback *loopback = snd_kcontrol_chip(kcontrol); - mutex_lock(&loopback->cable_lock); + guard(mutex)(&loopback->cable_lock); ucontrol->value.integer.value[0] = loopback->setup[kcontrol->id.subdevice] [kcontrol->id.device].rate_shift; - mutex_unlock(&loopback->cable_lock); return 0; } @@ -1439,14 +1452,13 @@ static int loopback_rate_shift_put(struct snd_kcontrol *kcontrol, val = 80000; if (val > 120000) val = 120000; - mutex_lock(&loopback->cable_lock); + guard(mutex)(&loopback->cable_lock); if (val != loopback->setup[kcontrol->id.subdevice] [kcontrol->id.device].rate_shift) { loopback->setup[kcontrol->id.subdevice] [kcontrol->id.device].rate_shift = val; change = 1; } - mutex_unlock(&loopback->cable_lock); return change; } @@ -1455,11 +1467,10 @@ static int loopback_notify_get(struct snd_kcontrol *kcontrol, { struct loopback *loopback = snd_kcontrol_chip(kcontrol); - mutex_lock(&loopback->cable_lock); + guard(mutex)(&loopback->cable_lock); ucontrol->value.integer.value[0] = loopback->setup[kcontrol->id.subdevice] [kcontrol->id.device].notify; - mutex_unlock(&loopback->cable_lock); return 0; } @@ -1471,14 +1482,13 @@ static int loopback_notify_put(struct snd_kcontrol *kcontrol, int change = 0; val = ucontrol->value.integer.value[0] ? 1 : 0; - mutex_lock(&loopback->cable_lock); + guard(mutex)(&loopback->cable_lock); if (val != loopback->setup[kcontrol->id.subdevice] [kcontrol->id.device].notify) { loopback->setup[kcontrol->id.subdevice] [kcontrol->id.device].notify = val; change = 1; } - mutex_unlock(&loopback->cable_lock); return change; } @@ -1490,14 +1500,13 @@ static int loopback_active_get(struct snd_kcontrol *kcontrol, unsigned int val = 0; - mutex_lock(&loopback->cable_lock); + guard(mutex)(&loopback->cable_lock); cable = loopback->cables[kcontrol->id.subdevice][kcontrol->id.device ^ 1]; if (cable != NULL) { unsigned int running = cable->running ^ cable->pause; val = (running & (1 << SNDRV_PCM_STREAM_PLAYBACK)) ? 1 : 0; } - mutex_unlock(&loopback->cable_lock); ucontrol->value.integer.value[0] = val; return 0; } @@ -1540,11 +1549,10 @@ static int loopback_rate_get(struct snd_kcontrol *kcontrol, { struct loopback *loopback = snd_kcontrol_chip(kcontrol); - mutex_lock(&loopback->cable_lock); + guard(mutex)(&loopback->cable_lock); ucontrol->value.integer.value[0] = loopback->setup[kcontrol->id.subdevice] [kcontrol->id.device].rate; - mutex_unlock(&loopback->cable_lock); return 0; } @@ -1564,11 +1572,10 @@ static int loopback_channels_get(struct snd_kcontrol *kcontrol, { struct loopback *loopback = snd_kcontrol_chip(kcontrol); - mutex_lock(&loopback->cable_lock); + guard(mutex)(&loopback->cable_lock); ucontrol->value.integer.value[0] = loopback->setup[kcontrol->id.subdevice] [kcontrol->id.device].channels; - mutex_unlock(&loopback->cable_lock); return 0; } @@ -1586,12 +1593,11 @@ static int loopback_access_get(struct snd_kcontrol *kcontrol, struct loopback *loopback = snd_kcontrol_chip(kcontrol); snd_pcm_access_t access; - mutex_lock(&loopback->cable_lock); + guard(mutex)(&loopback->cable_lock); access = loopback->setup[kcontrol->id.subdevice][kcontrol->id.device].access; ucontrol->value.enumerated.item[0] = !is_access_interleaved(access); - mutex_unlock(&loopback->cable_lock); return 0; } @@ -1759,12 +1765,11 @@ static void print_cable_info(struct snd_info_entry *entry, struct loopback *loopback = entry->private_data; int sub, num; - mutex_lock(&loopback->cable_lock); + guard(mutex)(&loopback->cable_lock); num = entry->name[strlen(entry->name)-1]; num = num == '0' ? 0 : 1; for (sub = 0; sub < MAX_PCM_SUBSTREAMS; sub++) print_substream_info(buffer, loopback, sub, num); - mutex_unlock(&loopback->cable_lock); } static int loopback_cable_proc_new(struct loopback *loopback, int cidx) @@ -1793,10 +1798,9 @@ static void print_timer_source_info(struct snd_info_entry *entry, { struct loopback *loopback = entry->private_data; - mutex_lock(&loopback->cable_lock); + guard(mutex)(&loopback->cable_lock); snd_iprintf(buffer, "%s\n", loopback->timer_source ? loopback->timer_source : ""); - mutex_unlock(&loopback->cable_lock); } static void change_timer_source_info(struct snd_info_entry *entry, @@ -1805,10 +1809,9 @@ static void change_timer_source_info(struct snd_info_entry *entry, struct loopback *loopback = entry->private_data; char line[64]; - mutex_lock(&loopback->cable_lock); + guard(mutex)(&loopback->cable_lock); if (!snd_info_get_line(buffer, line, sizeof(line))) loopback_set_timer_source(loopback, strim(line)); - mutex_unlock(&loopback->cable_lock); } static int loopback_timer_source_proc_new(struct loopback *loopback) diff --git a/sound/drivers/mpu401/mpu401_uart.c b/sound/drivers/mpu401/mpu401_uart.c index 8e3318e17717..86235aefab02 100644 --- a/sound/drivers/mpu401/mpu401_uart.c +++ b/sound/drivers/mpu401/mpu401_uart.c @@ -162,14 +162,15 @@ static void snd_mpu401_uart_timer(struct timer_list *t) spin_lock_irqsave(&mpu->timer_lock, flags); /*mpu->mode |= MPU401_MODE_TIMER;*/ - mod_timer(&mpu->timer, 1 + jiffies); + if (mpu->timer_invoked) + mod_timer(&mpu->timer, 1 + jiffies); spin_unlock_irqrestore(&mpu->timer_lock, flags); if (mpu->rmidi) _snd_mpu401_uart_interrupt(mpu); } /* - * initialize the timer callback if not programmed yet + * arm the timer if it is not already active */ static void snd_mpu401_uart_add_timer (struct snd_mpu401 *mpu, int input) { @@ -177,7 +178,6 @@ static void snd_mpu401_uart_add_timer (struct snd_mpu401 *mpu, int input) spin_lock_irqsave (&mpu->timer_lock, flags); if (mpu->timer_invoked == 0) { - timer_setup(&mpu->timer, snd_mpu401_uart_timer, 0); mod_timer(&mpu->timer, 1 + jiffies); } mpu->timer_invoked |= input ? MPU401_MODE_INPUT_TIMER : @@ -493,6 +493,7 @@ static void snd_mpu401_uart_free(struct snd_rawmidi *rmidi) struct snd_mpu401 *mpu = rmidi->private_data; if (mpu->irq >= 0) free_irq(mpu->irq, (void *) mpu); + timer_shutdown_sync(&mpu->timer); release_and_free_resource(mpu->res); kfree(mpu); } @@ -547,6 +548,7 @@ int snd_mpu401_uart_new(struct snd_card *card, int device, spin_lock_init(&mpu->input_lock); spin_lock_init(&mpu->output_lock); spin_lock_init(&mpu->timer_lock); + timer_setup(&mpu->timer, snd_mpu401_uart_timer, 0); mpu->hardware = hardware; mpu->irq = -1; mpu->rmidi = rmidi; diff --git a/sound/pci/cs46xx/dsp_spos.c b/sound/pci/cs46xx/dsp_spos.c index dd23b7d68c16..dd37c185bcf0 100644 --- a/sound/pci/cs46xx/dsp_spos.c +++ b/sound/pci/cs46xx/dsp_spos.c @@ -1110,6 +1110,8 @@ int cs46xx_dsp_scb_and_task_init (struct snd_cs46xx *chip) null_scb.entry_point = null_algorithm->address; ins->the_null_scb = cs46xx_dsp_create_scb(chip, "nullSCB", (u32 *)&null_scb, NULL_SCB_ADDR); + if (!ins->the_null_scb) + return -ENOMEM; ins->the_null_scb->task_entry = null_algorithm; ins->the_null_scb->sub_list_ptr = ins->the_null_scb; ins->the_null_scb->next_scb_ptr = ins->the_null_scb; diff --git a/sound/pci/cs46xx/dsp_spos_scb_lib.c b/sound/pci/cs46xx/dsp_spos_scb_lib.c index 1f90ca723f4d..af5c78e3fd20 100644 --- a/sound/pci/cs46xx/dsp_spos_scb_lib.c +++ b/sound/pci/cs46xx/dsp_spos_scb_lib.c @@ -285,7 +285,8 @@ _dsp_create_generic_scb (struct snd_cs46xx *chip, char * name, u32 * scb_data, u dev_dbg(chip->card->dev, "dsp_spos: creating SCB <%s>\n", name); scb = cs46xx_dsp_create_scb(chip,name,scb_data,dest); - + if (!scb) + return NULL; scb->sub_list_ptr = ins->the_null_scb; scb->next_scb_ptr = ins->the_null_scb; diff --git a/sound/pci/hda/cs35l41_hda.c b/sound/pci/hda/cs35l41_hda.c index 031978a62cb9..210cf48facec 100644 --- a/sound/pci/hda/cs35l41_hda.c +++ b/sound/pci/hda/cs35l41_hda.c @@ -1887,7 +1887,6 @@ int cs35l41_hda_parse_acpi(struct cs35l41_hda *cs35l41, struct device *physdev, hw_cfg->valid = false; hw_cfg->gpio1.valid = false; hw_cfg->gpio2.valid = false; - acpi_dev_put(cs35l41->dacpi); return ret; } @@ -1895,7 +1894,6 @@ int cs35l41_hda_parse_acpi(struct cs35l41_hda *cs35l41, struct device *physdev, static int cs35l41_hda_read_acpi(struct cs35l41_hda *cs35l41, const char *hid, int id) { struct acpi_device *adev; - struct device *physdev; struct spi_device *spi; const char *sub; int ret; @@ -1907,11 +1905,10 @@ static int cs35l41_hda_read_acpi(struct cs35l41_hda *cs35l41, const char *hid, i } cs35l41->dacpi = adev; - physdev = get_device(acpi_get_first_physical_node(adev)); - if (!physdev) { - acpi_dev_put(adev); + struct device *physdev __free(put_device) = + get_device(acpi_get_first_physical_node(adev)); + if (!physdev) return -ENODEV; - } sub = acpi_get_subsystem_id(ACPI_HANDLE(physdev)); if (IS_ERR(sub)) @@ -1925,13 +1922,9 @@ static int cs35l41_hda_read_acpi(struct cs35l41_hda *cs35l41, const char *hid, i } ret = cs35l41_hda_parse_acpi(cs35l41, physdev, id); - if (ret) { - put_device(physdev); + if (ret) return ret; - } out: - put_device(physdev); - cs35l41->bypass_fw = false; if (cs35l41->control_bus == SPI) { spi = to_spi_device(cs35l41->dev); @@ -1970,8 +1963,10 @@ int cs35l41_hda_probe(struct device *dev, const char *device_name, int id, int i dev_set_drvdata(dev, cs35l41); ret = cs35l41_hda_read_acpi(cs35l41, device_name, id); - if (ret) - return dev_err_probe(cs35l41->dev, ret, "Platform not supported\n"); + if (ret) { + dev_err_probe(cs35l41->dev, ret, "Platform not supported\n"); + goto err_acpi; + } if (IS_ERR(cs35l41->reset_gpio)) { ret = PTR_ERR(cs35l41->reset_gpio); @@ -2059,6 +2054,7 @@ int cs35l41_hda_probe(struct device *dev, const char *device_name, int id, int i err: if (cs35l41_safe_reset(cs35l41->regmap, cs35l41->hw_cfg.bst_type)) gpiod_set_value_cansleep(cs35l41->reset_gpio, 0); +err_acpi: gpiod_put(cs35l41->reset_gpio); gpiod_put(cs35l41->cs_gpio); acpi_dev_put(cs35l41->dacpi); diff --git a/sound/pci/hda/hda_eld.c b/sound/pci/hda/hda_eld.c index 301730432375..2980d0b949f6 100644 --- a/sound/pci/hda/hda_eld.c +++ b/sound/pci/hda/hda_eld.c @@ -525,9 +525,11 @@ void snd_hdmi_write_eld_info(struct hdmi_eld *eld, e->aud_synch_delay = val; else if (!strcmp(name, "speakers")) e->spk_alloc = val; - else if (!strcmp(name, "sad_count")) + else if (!strcmp(name, "sad_count")) { + if (val > ELD_MAX_SAD) + continue; e->sad_count = val; - else if (!strncmp(name, "sad", 3)) { + } else if (!strncmp(name, "sad", 3)) { sname = name + 4; n = name[3] - '0'; if (name[4] >= '0' && name[4] <= '9') { diff --git a/sound/pci/hda/patch_hdmi.c b/sound/pci/hda/patch_hdmi.c index 8b68386e1ac9..c097fc844b18 100644 --- a/sound/pci/hda/patch_hdmi.c +++ b/sound/pci/hda/patch_hdmi.c @@ -1355,7 +1355,8 @@ static int hdmi_read_pin_conn(struct hda_codec *codec, int pin_idx) snd_hda_set_dev_select(codec, pin_nid, dev_id); if (spec->intel_hsw_fixup) { - conns = spec->num_cvts; + /* spec->cvt_nids[] only has ARRAY_SIZE(spec->cvt_nids) slots */ + conns = min_t(int, spec->num_cvts, ARRAY_SIZE(spec->cvt_nids)); memcpy(per_pin->mux_nids, spec->cvt_nids, sizeof(hda_nid_t) * conns); } else { diff --git a/sound/pci/hda/patch_realtek.c b/sound/pci/hda/patch_realtek.c index 0b370a9eb9f2..1a68e683801b 100644 --- a/sound/pci/hda/patch_realtek.c +++ b/sound/pci/hda/patch_realtek.c @@ -8212,6 +8212,7 @@ enum { ALC287_FIXUP_LEGION_16ACHG6, ALC287_FIXUP_CS35L41_I2C_2, ALC287_FIXUP_CS35L41_I2C_2_HP_GPIO_LED, + ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS, ALC287_FIXUP_CS35L41_I2C_4, ALC245_FIXUP_CS35L41_SPI_1, ALC245_FIXUP_CS35L41_SPI_2, @@ -10414,6 +10415,12 @@ static const struct hda_fixup alc269_fixups[] = { .chained = true, .chain_id = ALC285_FIXUP_HP_MUTE_LED, }, + [ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS] = { + .type = HDA_FIXUP_FUNC, + .v.func = cs35l41_fixup_i2c_two, + .chained = true, + .chain_id = ALC245_FIXUP_HP_X360_MUTE_LEDS, + }, [ALC287_FIXUP_CS35L41_I2C_4] = { .type = HDA_FIXUP_FUNC, .v.func = cs35l41_fixup_i2c_four, @@ -10835,6 +10842,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = { SND_PCI_QUIRK(0x1028, 0x097e, "Dell Precision", ALC289_FIXUP_DUAL_SPK), SND_PCI_QUIRK(0x1028, 0x098d, "Dell Precision", ALC233_FIXUP_ASUS_MIC_NO_PRESENCE), SND_PCI_QUIRK(0x1028, 0x09bf, "Dell Precision", ALC233_FIXUP_ASUS_MIC_NO_PRESENCE), + SND_PCI_QUIRK(0x1028, 0x09d1, "Alienware m17 R3", ALC289_FIXUP_DUAL_SPK), SND_PCI_QUIRK(0x1028, 0x0a2e, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC), SND_PCI_QUIRK(0x1028, 0x0a30, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC), SND_PCI_QUIRK(0x1028, 0x0a38, "Dell Latitude 7520", ALC269_FIXUP_DELL4_MIC_NO_PRESENCE_QUIET), @@ -11218,7 +11226,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = { SND_PCI_QUIRK(0x103c, 0x8d9c, "HP 17 Turbine OmniBook 7 DIS", ALC287_FIXUP_CS35L41_I2C_2), SND_PCI_QUIRK(0x103c, 0x8d9d, "HP 17 Turbine OmniBook X UMA", ALC287_FIXUP_CS35L41_I2C_2), SND_PCI_QUIRK(0x103c, 0x8d9e, "HP 17 Turbine OmniBook X DIS", ALC287_FIXUP_CS35L41_I2C_2), - SND_PCI_QUIRK(0x103c, 0x8d9f, "HP 14 Cadet (x360)", ALC287_FIXUP_CS35L41_I2C_2), + SND_PCI_QUIRK(0x103c, 0x8d9f, "HP 14 Cadet (x360)", ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS), SND_PCI_QUIRK(0x103c, 0x8da0, "HP 16 Clipper OmniBook 7(X360)", ALC287_FIXUP_CS35L41_I2C_2), SND_PCI_QUIRK(0x103c, 0x8da1, "HP 16 Clipper OmniBook X", ALC287_FIXUP_CS35L41_I2C_2), SND_PCI_QUIRK(0x103c, 0x8da7, "HP 14 Enstrom OmniBook X", ALC287_FIXUP_CS35L41_I2C_2), diff --git a/sound/soc/amd/acp-config.c b/sound/soc/amd/acp-config.c index 653e8ce94608..7ed461c1d519 100644 --- a/sound/soc/amd/acp-config.c +++ b/sound/soc/amd/acp-config.c @@ -23,6 +23,44 @@ static int acp_quirk_data; +static const struct dmi_system_id acp70_acpi_flag_override_table[] = { + { + .matches = { + DMI_MATCH(DMI_BOARD_VENDOR, "ASUSTeK COMPUTER INC."), + DMI_MATCH(DMI_PRODUCT_NAME, "HN7306EA"), + }, + }, + { + /* ASUS Zenbook S16 UM5606GA (Strix Point, ACP 7.0) */ + .matches = { + DMI_MATCH(DMI_BOARD_VENDOR, "ASUSTeK COMPUTER INC."), + DMI_MATCH(DMI_PRODUCT_NAME, "Zenbook S16 UM5606GA"), + }, + }, + { + /* Lenovo Yoga Pro 7 15ASH11 (Strix Halo, ACP 7.0) */ + .matches = { + DMI_MATCH(DMI_BOARD_VENDOR, "LENOVO"), + DMI_MATCH(DMI_PRODUCT_NAME, "83W5"), + }, + }, + { + /* HP OmniBook X Flip 16-cc0xxx */ + .matches = { + DMI_MATCH(DMI_BOARD_VENDOR, "HP"), + DMI_MATCH(DMI_BOARD_NAME, "8EA2"), + }, + }, + { + /* Dell 14S DS14265 */ + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."), + DMI_MATCH(DMI_PRODUCT_NAME, "Dell 14S DS14265"), + }, + }, + {} +}; + static const struct config_entry config_table[] = { { .flags = FLAG_AMD_SOF, @@ -200,8 +238,11 @@ int snd_amd_acp_find_config(struct pci_dev *pci) */ if (!pci->revision) return 0; - else if (pci->revision >= ACP_7_0_REV) + else if (pci->revision >= ACP_7_0_REV) { + if (dmi_check_system(acp70_acpi_flag_override_table)) + return 0; return snd_amd_acp_acpi_find_config(pci); + } for (i = 0; i < ARRAY_SIZE(config_table); i++, table++) { if (table->device != device) diff --git a/sound/soc/amd/yc/acp6x-mach.c b/sound/soc/amd/yc/acp6x-mach.c index 1aedead0a927..4a9ff060ebbe 100644 --- a/sound/soc/amd/yc/acp6x-mach.c +++ b/sound/soc/amd/yc/acp6x-mach.c @@ -675,6 +675,13 @@ static const struct dmi_system_id yc_acp_quirk_table[] = { DMI_MATCH(DMI_BOARD_NAME, "8BD6"), } }, + { + .driver_data = &acp6x_card, + .matches = { + DMI_MATCH(DMI_BOARD_VENDOR, "HP"), + DMI_MATCH(DMI_BOARD_NAME, "8EE3"), + } + }, { .driver_data = &acp6x_card, .matches = { diff --git a/sound/soc/codecs/hdac_hda.c b/sound/soc/codecs/hdac_hda.c index 25a145cb8a51..fc5067edd530 100644 --- a/sound/soc/codecs/hdac_hda.c +++ b/sound/soc/codecs/hdac_hda.c @@ -635,12 +635,35 @@ static const struct snd_soc_component_driver hdac_hda_hdmi_codec = { .endianness = 1, }; +static void hdac_hda_restore_dev_type(void *data) +{ + struct hdac_device *hdev = data; + + hdev->type = HDA_DEV_LEGACY; +} + static int hdac_hda_dev_probe(struct hdac_device *hdev) { - struct hdac_hda_priv *hda_pvt = dev_get_drvdata(&hdev->dev); + struct hdac_hda_priv *hda_pvt; struct hdac_ext_link *hlink; int ret; + hda_pvt = devm_kzalloc(&hdev->dev, sizeof(*hda_pvt), GFP_KERNEL); + if (!hda_pvt) + return -ENOMEM; + + hda_pvt->codec = dev_to_hda_codec(&hdev->dev); + hda_pvt->dev_index = hdev->addr; + /* Intel iDisp (HDMI/DP) codec */ + if (((hdev->vendor_id >> 16) & 0xFFFF) == 0x8086) + hda_pvt->need_display_power = true; + dev_set_drvdata(&hdev->dev, hda_pvt); + + /* Allow the legacy codec driver to match again after unbind */ + ret = devm_add_action(&hdev->dev, hdac_hda_restore_dev_type, hdev); + if (ret) + return ret; + /* hold the ref while we probe */ hlink = snd_hdac_ext_bus_get_hlink_by_name(hdev->bus, dev_name(&hdev->dev)); if (!hlink) { diff --git a/sound/soc/qcom/qdsp6/q6apm-dai.c b/sound/soc/qcom/qdsp6/q6apm-dai.c index 3b00c6d5174a..fef69e431890 100644 --- a/sound/soc/qcom/qdsp6/q6apm-dai.c +++ b/sound/soc/qcom/qdsp6/q6apm-dai.c @@ -441,11 +441,15 @@ static snd_pcm_uframes_t q6apm_dai_pointer(struct snd_soc_component *component, struct q6apm_dai_rtd *prtd = runtime->private_data; snd_pcm_uframes_t ptr; - ptr = q6apm_get_hw_pointer(prtd->graph, substream->stream) * runtime->period_size; - if (ptr) - return ptr - 1; + /* + * The hw pointer counts the period buffers the DSP has returned in + * this lap, 1..periods. A full lap is the start of the ring again: + * report 0 rather than one frame less, so that the position of a + * drained buffer reaches appl_ptr and ALSA sees the underrun. + */ + ptr = q6apm_get_hw_pointer(prtd->graph, substream->stream) % runtime->periods; - return 0; + return ptr * runtime->period_size; } static int q6apm_dai_hw_params(struct snd_soc_component *component, diff --git a/sound/soc/qcom/qdsp6/q6apm.c b/sound/soc/qcom/qdsp6/q6apm.c index 426c1429d01b..1025ddde9e31 100644 --- a/sound/soc/qcom/qdsp6/q6apm.c +++ b/sound/soc/qcom/qdsp6/q6apm.c @@ -762,8 +762,6 @@ static int apm_probe(gpr_device_t *gdev) if (!apm) return -ENOMEM; - dev_set_drvdata(dev, apm); - mutex_init(&apm->lock); apm->dev = dev; apm->gdev = gdev; @@ -777,6 +775,7 @@ static int apm_probe(gpr_device_t *gdev) idr_init(&apm->modules_idr); + dev_set_drvdata(dev, apm); g_apm = apm; q6apm_get_apm_state(apm); @@ -817,6 +816,9 @@ static int apm_callback(struct gpr_resp_pkt *data, void *priv, int op) struct gpr_ibasic_rsp_result_t *result; struct gpr_hdr *hdr = &data->hdr; + if (!apm) + return 0; + result = data->payload; switch (hdr->opcode) { diff --git a/sound/soc/qcom/qdsp6/q6prm.c b/sound/soc/qcom/qdsp6/q6prm.c index 81554d202658..22e2a174a387 100644 --- a/sound/soc/qcom/qdsp6/q6prm.c +++ b/sound/soc/qcom/qdsp6/q6prm.c @@ -197,6 +197,9 @@ static int prm_callback(struct gpr_resp_pkt *data, void *priv, int op) struct gpr_ibasic_rsp_result_t *result; struct gpr_hdr *hdr = &data->hdr; + if (!prm) + return 0; + switch (hdr->opcode) { case PRM_CMD_RSP_REQUEST_HW_RSC: case PRM_CMD_RSP_RELEASE_HW_RSC: diff --git a/sound/soc/sof/intel/hda-codec.c b/sound/soc/sof/intel/hda-codec.c index c0c58b429715..57998d3d8d15 100644 --- a/sound/soc/sof/intel/hda-codec.c +++ b/sound/soc/sof/intel/hda-codec.c @@ -15,8 +15,6 @@ #include "hda.h" #if IS_ENABLED(CONFIG_SND_SOC_SOF_HDA_AUDIO_CODEC) -#include "../../codecs/hdac_hda.h" - #define CODEC_PROBE_RETRIES 3 #define IDISP_VID_INTEL 0x80860000 @@ -148,7 +146,6 @@ static struct hda_codec *hda_codec_device_init(struct hdac_bus *bus, int addr, i /* probe individual codec */ static int hda_codec_probe(struct snd_sof_dev *sdev, int address) { - struct hdac_hda_priv *hda_priv; struct hda_bus *hbus = sof_to_hbus(sdev); struct hda_codec *codec; u32 hda_cmd = (address << 28) | (AC_NODE_ROOT << 20) | @@ -168,27 +165,15 @@ static int hda_codec_probe(struct snd_sof_dev *sdev, int address) dev_dbg(sdev->dev, "HDA codec #%d probed OK: response: %x\n", address, resp); - hda_priv = devm_kzalloc(sdev->dev, sizeof(*hda_priv), GFP_KERNEL); - if (!hda_priv) - return -ENOMEM; - codec = hda_codec_device_init(&hbus->core, address, HDA_DEV_LEGACY); ret = PTR_ERR_OR_ZERO(codec); if (ret < 0) return ret; - hda_priv->codec = codec; - hda_priv->dev_index = address; - dev_set_drvdata(&codec->core.dev, hda_priv); - - if ((resp & 0xFFFF0000) == IDISP_VID_INTEL) { - if (!hbus->core.audio_component) { - dev_dbg(sdev->dev, - "iDisp hw present but no driver\n"); - ret = -ENOENT; - goto out; - } - hda_priv->need_display_power = true; + if ((resp & 0xFFFF0000) == IDISP_VID_INTEL && !hbus->core.audio_component) { + dev_dbg(sdev->dev, "iDisp hw present but no driver\n"); + ret = -ENOENT; + goto out; } if (is_generic_config(hbus)) diff --git a/sound/soc/stm/stm32_i2s.c b/sound/soc/stm/stm32_i2s.c index faa00103ee7f..d62802f4f12e 100644 --- a/sound/soc/stm/stm32_i2s.c +++ b/sound/soc/stm/stm32_i2s.c @@ -1026,7 +1026,7 @@ static int stm32_i2s_parse_dt(struct platform_device *pdev, struct device_node *np = pdev->dev.of_node; struct reset_control *rst; struct resource *res; - int irq, ret; + int ret; if (!np) return -ENODEV; @@ -1069,18 +1069,6 @@ static int stm32_i2s_parse_dt(struct platform_device *pdev, return ret; } - /* Get irqs */ - irq = platform_get_irq(pdev, 0); - if (irq < 0) - return irq; - - ret = devm_request_irq(&pdev->dev, irq, stm32_i2s_isr, 0, - dev_name(&pdev->dev), i2s); - if (ret) { - dev_err(&pdev->dev, "irq request returned %d\n", ret); - return ret; - } - /* Reset */ rst = devm_reset_control_get_optional_exclusive(&pdev->dev, NULL); if (IS_ERR(rst)) @@ -1105,7 +1093,7 @@ static int stm32_i2s_probe(struct platform_device *pdev) { struct stm32_i2s_data *i2s; u32 val; - int ret; + int irq, ret; i2s = devm_kzalloc(&pdev->dev, sizeof(*i2s), GFP_KERNEL); if (!i2s) @@ -1131,9 +1119,18 @@ static int stm32_i2s_probe(struct platform_device *pdev) return dev_err_probe(&pdev->dev, PTR_ERR(i2s->regmap), "Regmap init error\n"); + irq = platform_get_irq(pdev, 0); + if (irq < 0) + return irq; + + ret = devm_request_irq(&pdev->dev, irq, stm32_i2s_isr, 0, + dev_name(&pdev->dev), i2s); + if (ret) + return ret; + ret = snd_dmaengine_pcm_register(&pdev->dev, &stm32_i2s_pcm_config, 0); if (ret) - return dev_err_probe(&pdev->dev, ret, "PCM DMA register error\n"); + return ret; ret = snd_soc_register_component(&pdev->dev, &stm32_i2s_component, i2s->dai_drv, 1); diff --git a/sound/synth/emux/emux_effect.c b/sound/synth/emux/emux_effect.c index 3c7314f5fb19..bfe383fa90ba 100644 --- a/sound/synth/emux/emux_effect.c +++ b/sound/synth/emux/emux_effect.c @@ -168,7 +168,6 @@ snd_emux_send_effect(struct snd_emux_port *port, struct snd_midi_channel *chan, unsigned char *srcp, *origp; struct snd_emux *emu; struct snd_emux_effect_table *fx; - unsigned long flags; emu = port->emu; fx = chan->private; @@ -195,22 +194,22 @@ snd_emux_send_effect(struct snd_emux_port *port, struct snd_midi_channel *chan, offset++; #endif /* modify the register values */ - spin_lock_irqsave(&emu->voice_lock, flags); - for (i = 0; i < emu->max_voices; i++) { - struct snd_emux_voice *vp = &emu->voices[i]; - if (!STATE_IS_PLAYING(vp->state) || vp->chan != chan) - continue; - srcp = (unsigned char*)&vp->reg.parm + offset; - origp = (unsigned char*)&vp->zone->v.parm + offset; - if (parm_defs[i].type & PARM_IS_BYTE) { - *srcp = *origp; - effect_set_byte(srcp, chan, type); - } else { - *(unsigned short*)srcp = *(unsigned short*)origp; - effect_set_word((unsigned short*)srcp, chan, type); + scoped_guard(spinlock_irqsave, &emu->voice_lock) { + for (i = 0; i < emu->max_voices; i++) { + struct snd_emux_voice *vp = &emu->voices[i]; + if (!STATE_IS_PLAYING(vp->state) || vp->chan != chan) + continue; + srcp = (unsigned char *)&vp->reg.parm + offset; + origp = (unsigned char *)&vp->zone->v.parm + offset; + if (parm_defs[i].type & PARM_IS_BYTE) { + *srcp = *origp; + effect_set_byte(srcp, chan, type); + } else { + *(unsigned short *)srcp = *(unsigned short *)origp; + effect_set_word((unsigned short *)srcp, chan, type); + } } } - spin_unlock_irqrestore(&emu->voice_lock, flags); /* activate them */ snd_emux_update_channel(port, chan, parm_defs[type].update); diff --git a/sound/synth/emux/emux_synth.c b/sound/synth/emux/emux_synth.c index 02e2c69d7f18..537a7c47ef47 100644 --- a/sound/synth/emux/emux_synth.c +++ b/sound/synth/emux/emux_synth.c @@ -49,7 +49,6 @@ snd_emux_note_on(void *p, int note, int vel, struct snd_midi_channel *chan) int i, key, nvoices; struct snd_emux_voice *vp; struct snd_sf_zone *table[SNDRV_EMUX_MAX_MULTI_VOICES]; - unsigned long flags; struct snd_emux_port *port; port = p; @@ -61,6 +60,9 @@ snd_emux_note_on(void *p, int note, int vel, struct snd_midi_channel *chan) return; key = note; /* remember the original note */ + + /* hold sflist->lock for protecting against concurrent free */ + guard(spinlock_irqsave)(&emu->sflist->lock); nvoices = get_zone(emu, port, ¬e, vel, chan, table); if (! nvoices) return; @@ -77,7 +79,7 @@ snd_emux_note_on(void *p, int note, int vel, struct snd_midi_channel *chan) terminate_note1(emu, key, chan, 0); #endif - spin_lock_irqsave(&emu->voice_lock, flags); + guard(spinlock_irqsave)(&emu->voice_lock); for (i = 0; i < nvoices; i++) { /* set up each voice parameter */ @@ -124,7 +126,6 @@ snd_emux_note_on(void *p, int note, int vel, struct snd_midi_channel *chan) vp->ontime = jiffies; /* remember the trigger timing */ } } - spin_unlock_irqrestore(&emu->voice_lock, flags); #ifdef SNDRV_EMUX_USE_RAW_EFFECT if (port->port_mode == SNDRV_EMUX_PORT_MODE_OSS_SYNTH) { @@ -147,7 +148,6 @@ snd_emux_note_off(void *p, int note, int vel, struct snd_midi_channel *chan) int ch; struct snd_emux *emu; struct snd_emux_voice *vp; - unsigned long flags; struct snd_emux_port *port; port = p; @@ -158,7 +158,7 @@ snd_emux_note_off(void *p, int note, int vel, struct snd_midi_channel *chan) if (snd_BUG_ON(!emu || !emu->ops.release)) return; - spin_lock_irqsave(&emu->voice_lock, flags); + guard(spinlock_irqsave)(&emu->voice_lock); for (ch = 0; ch < emu->max_voices; ch++) { vp = &emu->voices[ch]; if (STATE_IS_PLAYING(vp->state) && @@ -180,7 +180,6 @@ snd_emux_note_off(void *p, int note, int vel, struct snd_midi_channel *chan) emu->ops.release(vp); } } - spin_unlock_irqrestore(&emu->voice_lock, flags); } /* @@ -192,10 +191,9 @@ void snd_emux_timer_callback(struct timer_list *t) { struct snd_emux *emu = from_timer(emu, t, tlist); struct snd_emux_voice *vp; - unsigned long flags; int ch, do_again = 0; - spin_lock_irqsave(&emu->voice_lock, flags); + guard(spinlock_irqsave)(&emu->voice_lock); for (ch = 0; ch < emu->max_voices; ch++) { vp = &emu->voices[ch]; if (vp->state == SNDRV_EMUX_ST_PENDING) { @@ -212,7 +210,6 @@ void snd_emux_timer_callback(struct timer_list *t) emu->timer_active = 1; } else emu->timer_active = 0; - spin_unlock_irqrestore(&emu->voice_lock, flags); } /* @@ -224,7 +221,6 @@ snd_emux_key_press(void *p, int note, int vel, struct snd_midi_channel *chan) int ch; struct snd_emux *emu; struct snd_emux_voice *vp; - unsigned long flags; struct snd_emux_port *port; port = p; @@ -235,7 +231,7 @@ snd_emux_key_press(void *p, int note, int vel, struct snd_midi_channel *chan) if (snd_BUG_ON(!emu || !emu->ops.update)) return; - spin_lock_irqsave(&emu->voice_lock, flags); + guard(spinlock_irqsave)(&emu->voice_lock); for (ch = 0; ch < emu->max_voices; ch++) { vp = &emu->voices[ch]; if (vp->state == SNDRV_EMUX_ST_ON && @@ -244,7 +240,6 @@ snd_emux_key_press(void *p, int note, int vel, struct snd_midi_channel *chan) update_voice(emu, vp, SNDRV_EMUX_UPDATE_VOLUME); } } - spin_unlock_irqrestore(&emu->voice_lock, flags); } @@ -257,7 +252,6 @@ snd_emux_update_channel(struct snd_emux_port *port, struct snd_midi_channel *cha struct snd_emux *emu; struct snd_emux_voice *vp; int i; - unsigned long flags; if (! update) return; @@ -266,13 +260,12 @@ snd_emux_update_channel(struct snd_emux_port *port, struct snd_midi_channel *cha if (snd_BUG_ON(!emu || !emu->ops.update)) return; - spin_lock_irqsave(&emu->voice_lock, flags); + guard(spinlock_irqsave)(&emu->voice_lock); for (i = 0; i < emu->max_voices; i++) { vp = &emu->voices[i]; if (vp->chan == chan) update_voice(emu, vp, update); } - spin_unlock_irqrestore(&emu->voice_lock, flags); } /* @@ -284,7 +277,6 @@ snd_emux_update_port(struct snd_emux_port *port, int update) struct snd_emux *emu; struct snd_emux_voice *vp; int i; - unsigned long flags; if (! update) return; @@ -293,13 +285,12 @@ snd_emux_update_port(struct snd_emux_port *port, int update) if (snd_BUG_ON(!emu || !emu->ops.update)) return; - spin_lock_irqsave(&emu->voice_lock, flags); + guard(spinlock_irqsave)(&emu->voice_lock); for (i = 0; i < emu->max_voices; i++) { vp = &emu->voices[i]; if (vp->port == port) update_voice(emu, vp, update); } - spin_unlock_irqrestore(&emu->voice_lock, flags); } @@ -365,16 +356,14 @@ terminate_note1(struct snd_emux *emu, int note, struct snd_midi_channel *chan, i { int i; struct snd_emux_voice *vp; - unsigned long flags; - spin_lock_irqsave(&emu->voice_lock, flags); + guard(spinlock_irqsave)(&emu->voice_lock); for (i = 0; i < emu->max_voices; i++) { vp = &emu->voices[i]; if (STATE_IS_PLAYING(vp->state) && vp->chan == chan && vp->key == note) terminate_voice(emu, vp, free); } - spin_unlock_irqrestore(&emu->voice_lock, flags); } @@ -407,9 +396,8 @@ snd_emux_terminate_all(struct snd_emux *emu) { int i; struct snd_emux_voice *vp; - unsigned long flags; - spin_lock_irqsave(&emu->voice_lock, flags); + guard(spinlock_irqsave)(&emu->voice_lock); for (i = 0; i < emu->max_voices; i++) { vp = &emu->voices[i]; if (STATE_IS_PLAYING(vp->state)) @@ -424,7 +412,6 @@ snd_emux_terminate_all(struct snd_emux *emu) } /* initialize allocation time */ emu->use_time = 0; - spin_unlock_irqrestore(&emu->voice_lock, flags); } EXPORT_SYMBOL(snd_emux_terminate_all); @@ -438,7 +425,6 @@ snd_emux_sounds_off_all(struct snd_emux_port *port) int i; struct snd_emux *emu; struct snd_emux_voice *vp; - unsigned long flags; if (snd_BUG_ON(!port)) return; @@ -446,7 +432,7 @@ snd_emux_sounds_off_all(struct snd_emux_port *port) if (snd_BUG_ON(!emu || !emu->ops.terminate)) return; - spin_lock_irqsave(&emu->voice_lock, flags); + guard(spinlock_irqsave)(&emu->voice_lock); for (i = 0; i < emu->max_voices; i++) { vp = &emu->voices[i]; if (STATE_IS_PLAYING(vp->state) && @@ -459,7 +445,6 @@ snd_emux_sounds_off_all(struct snd_emux_port *port) emu->ops.reset(emu, i); } } - spin_unlock_irqrestore(&emu->voice_lock, flags); } @@ -472,9 +457,8 @@ exclusive_note_off(struct snd_emux *emu, struct snd_emux_port *port, int exclass { struct snd_emux_voice *vp; int i; - unsigned long flags; - spin_lock_irqsave(&emu->voice_lock, flags); + guard(spinlock_irqsave)(&emu->voice_lock); for (i = 0; i < emu->max_voices; i++) { vp = &emu->voices[i]; if (STATE_IS_PLAYING(vp->state) && vp->port == port && @@ -482,7 +466,6 @@ exclusive_note_off(struct snd_emux *emu, struct snd_emux_port *port, int exclass terminate_voice(emu, vp, 0); } } - spin_unlock_irqrestore(&emu->voice_lock, flags); } /* @@ -916,9 +899,8 @@ snd_emux_init_voices(struct snd_emux *emu) { struct snd_emux_voice *vp; int i; - unsigned long flags; - spin_lock_irqsave(&emu->voice_lock, flags); + guard(spinlock_irqsave)(&emu->voice_lock); for (i = 0; i < emu->max_voices; i++) { vp = &emu->voices[i]; vp->ch = -1; /* not used */ @@ -929,23 +911,19 @@ snd_emux_init_voices(struct snd_emux *emu) vp->emu = emu; vp->hw = emu->hw; } - spin_unlock_irqrestore(&emu->voice_lock, flags); } /* */ void snd_emux_lock_voice(struct snd_emux *emu, int voice) { - unsigned long flags; - - spin_lock_irqsave(&emu->voice_lock, flags); + guard(spinlock_irqsave)(&emu->voice_lock); if (emu->voices[voice].state == SNDRV_EMUX_ST_OFF) emu->voices[voice].state = SNDRV_EMUX_ST_LOCKED; else dev_warn(emu->card->dev, "invalid voice for lock %d (state = %x)\n", voice, emu->voices[voice].state); - spin_unlock_irqrestore(&emu->voice_lock, flags); } EXPORT_SYMBOL(snd_emux_lock_voice); @@ -954,16 +932,13 @@ EXPORT_SYMBOL(snd_emux_lock_voice); */ void snd_emux_unlock_voice(struct snd_emux *emu, int voice) { - unsigned long flags; - - spin_lock_irqsave(&emu->voice_lock, flags); + guard(spinlock_irqsave)(&emu->voice_lock); if (emu->voices[voice].state == SNDRV_EMUX_ST_LOCKED) emu->voices[voice].state = SNDRV_EMUX_ST_OFF; else dev_warn(emu->card->dev, "invalid voice for unlock %d (state = %x)\n", voice, emu->voices[voice].state); - spin_unlock_irqrestore(&emu->voice_lock, flags); } EXPORT_SYMBOL(snd_emux_unlock_voice); diff --git a/sound/synth/emux/soundfont.c b/sound/synth/emux/soundfont.c index 47137cca50b8..ad4a0f262afa 100644 --- a/sound/synth/emux/soundfont.c +++ b/sound/synth/emux/soundfont.c @@ -66,11 +66,9 @@ static void snd_sf_clear(struct snd_sf_list *sflist); static void lock_preset(struct snd_sf_list *sflist) { - unsigned long flags; mutex_lock(&sflist->presets_mutex); - spin_lock_irqsave(&sflist->lock, flags); + guard(spinlock_irqsave)(&sflist->lock); sflist->presets_locked = 1; - spin_unlock_irqrestore(&sflist->lock, flags); } @@ -80,10 +78,8 @@ lock_preset(struct snd_sf_list *sflist) static void unlock_preset(struct snd_sf_list *sflist) { - unsigned long flags; - spin_lock_irqsave(&sflist->lock, flags); + guard(spinlock_irqsave)(&sflist->lock); sflist->presets_locked = 0; - spin_unlock_irqrestore(&sflist->lock, flags); mutex_unlock(&sflist->presets_mutex); } @@ -94,14 +90,11 @@ unlock_preset(struct snd_sf_list *sflist) int snd_soundfont_close_check(struct snd_sf_list *sflist, int client) { - unsigned long flags; - spin_lock_irqsave(&sflist->lock, flags); - if (sflist->open_client == client) { - spin_unlock_irqrestore(&sflist->lock, flags); - return close_patch(sflist); + scoped_guard(spinlock_irqsave, &sflist->lock) { + if (sflist->open_client != client) + return 0; } - spin_unlock_irqrestore(&sflist->lock, flags); - return 0; + return close_patch(sflist); } @@ -119,7 +112,6 @@ snd_soundfont_load(struct snd_card *card, long count, int client) { struct soundfont_patch_info patch; - unsigned long flags; int rc; if (count < (long)sizeof(patch)) { @@ -155,12 +147,10 @@ snd_soundfont_load(struct snd_card *card, } /* check if other client already opened patch */ - spin_lock_irqsave(&sflist->lock, flags); - if (sflist->open_client != client) { - spin_unlock_irqrestore(&sflist->lock, flags); - return -EBUSY; + scoped_guard(spinlock_irqsave, &sflist->lock) { + if (sflist->open_client != client) + return -EBUSY; } - spin_unlock_irqrestore(&sflist->lock, flags); lock_preset(sflist); rc = -EINVAL; @@ -223,14 +213,11 @@ open_patch(struct snd_sf_list *sflist, const char __user *data, { struct soundfont_open_parm parm; struct snd_soundfont *sf; - unsigned long flags; - spin_lock_irqsave(&sflist->lock, flags); - if (sflist->open_client >= 0 || sflist->currsf) { - spin_unlock_irqrestore(&sflist->lock, flags); - return -EBUSY; + scoped_guard(spinlock_irqsave, &sflist->lock) { + if (sflist->open_client >= 0 || sflist->currsf) + return -EBUSY; } - spin_unlock_irqrestore(&sflist->lock, flags); if (copy_from_user(&parm, data, sizeof(parm))) return -EFAULT; @@ -244,10 +231,10 @@ open_patch(struct snd_sf_list *sflist, const char __user *data, return -ENOMEM; } - spin_lock_irqsave(&sflist->lock, flags); - sflist->open_client = client; - sflist->currsf = sf; - spin_unlock_irqrestore(&sflist->lock, flags); + scoped_guard(spinlock_irqsave, &sflist->lock) { + sflist->open_client = client; + sflist->currsf = sf; + } return 0; } @@ -305,12 +292,10 @@ is_identical_font(struct snd_soundfont *sf, int type, unsigned char *name) static int close_patch(struct snd_sf_list *sflist) { - unsigned long flags; - - spin_lock_irqsave(&sflist->lock, flags); - sflist->currsf = NULL; - sflist->open_client = -1; - spin_unlock_irqrestore(&sflist->lock, flags); + scoped_guard(spinlock_irqsave, &sflist->lock) { + sflist->currsf = NULL; + sflist->open_client = -1; + } rebuild_presets(sflist); @@ -1279,6 +1264,9 @@ delete_preset(struct snd_sf_list *sflist, struct snd_sf_zone *zp) * The found zones are stored on 'table' array. max_layers defines * the maximum number of elements in this array. * This function returns the number of found zones. 0 if not found. + * + * The caller must hold sflist->lock, and keep holding it as long as + * the returned zones are accessed. */ int snd_soundfont_search_zone(struct snd_sf_list *sflist, int *notep, int vel, @@ -1287,17 +1275,14 @@ snd_soundfont_search_zone(struct snd_sf_list *sflist, int *notep, int vel, struct snd_sf_zone **table, int max_layers) { int nvoices; - unsigned long flags; - /* this function is supposed to be called atomically, - * so we check the lock. if it's busy, just returns 0 to + lockdep_assert_held(&sflist->lock); + + /* if the presets are being modified, just returns 0 to * tell the caller the busy state */ - spin_lock_irqsave(&sflist->lock, flags); - if (sflist->presets_locked) { - spin_unlock_irqrestore(&sflist->lock, flags); + if (sflist->presets_locked) return 0; - } nvoices = search_zones(sflist, notep, vel, preset, bank, table, max_layers, 0); if (! nvoices) { @@ -1306,7 +1291,6 @@ snd_soundfont_search_zone(struct snd_sf_list *sflist, int *notep, int vel, def_preset, def_bank, table, max_layers, 0); } - spin_unlock_irqrestore(&sflist->lock, flags); return nvoices; } diff --git a/sound/usb/bcd2000/bcd2000.c b/sound/usb/bcd2000/bcd2000.c index dce6d1f611f9..fc18a4348265 100644 --- a/sound/usb/bcd2000/bcd2000.c +++ b/sound/usb/bcd2000/bcd2000.c @@ -183,10 +183,24 @@ static int bcd2000_midi_output_open(struct snd_rawmidi_substream *substream) static int bcd2000_midi_output_close(struct snd_rawmidi_substream *substream) { struct bcd2000 *bcd2k = substream->rmidi->private_data; + struct urb *urb = NULL; - if (bcd2k->midi_out_active && bcd2k->midi_out_urb) { - usb_kill_urb(bcd2k->midi_out_urb); - bcd2k->midi_out_active = 0; + /* + * Reference the URB under the lock so that it stays valid until + * usb_kill_urb() returns, even if a disconnect frees it in the + * meantime. usb_kill_urb() cannot run under the lock, as the + * completion handler takes the same lock. + */ + scoped_guard(spinlock_irqsave, &bcd2k->midi_lock) { + if (bcd2k->midi_out_active && bcd2k->midi_out_urb) { + urb = usb_get_urb(bcd2k->midi_out_urb); + bcd2k->midi_out_active = 0; + } + } + + if (urb) { + usb_kill_urb(urb); + usb_put_urb(urb); } return 0; diff --git a/sound/usb/caiaq/input.c b/sound/usb/caiaq/input.c index c12eeb971000..6a5adb8d1e6a 100644 --- a/sound/usb/caiaq/input.c +++ b/sound/usb/caiaq/input.c @@ -830,6 +830,8 @@ int snd_usb_caiaq_input_init(struct snd_usb_caiaqdev *cdev) return 0; exit_free_idev: + usb_free_urb(cdev->ep4_in_urb); + cdev->ep4_in_urb = NULL; input_free_device(input); cdev->input_dev = NULL; return ret; diff --git a/sound/usb/line6/capture.c b/sound/usb/line6/capture.c index 84a9b7b76f43..f59ec9c4a1a5 100644 --- a/sound/usb/line6/capture.c +++ b/sound/usb/line6/capture.c @@ -94,6 +94,18 @@ void line6_capture_copy(struct snd_line6_pcm *line6pcm, char *fbuf, int fsize) if (runtime == NULL) return; + /* + * snd_line6_capture_open() constrains the buffer to hold at least + * two packets, so this cannot happen for a conforming device. Drop + * the packet rather than overrun the single wrap below. + */ + if (unlikely(frames > runtime->buffer_size)) { + dev_err_ratelimited(line6pcm->line6->ifcdev, + "packet of %d frames exceeds %lu-frame buffer, dropping\n", + frames, runtime->buffer_size); + return; + } + if (line6pcm->in.pos_done + frames > runtime->buffer_size) { /* The transferred area goes over buffer boundary, @@ -125,28 +137,23 @@ void line6_capture_copy(struct snd_line6_pcm *line6pcm, char *fbuf, int fsize) line6pcm->in.pos_done -= runtime->buffer_size; } -void line6_capture_check_period(struct snd_line6_pcm *line6pcm, int length) +/* update the capture position and check whether we need period_elapsed call */ +bool line6_capture_check_period(struct snd_line6_pcm *line6pcm, int length) { - struct snd_pcm_substream *substream = - get_substream(line6pcm, SNDRV_PCM_STREAM_CAPTURE); - line6pcm->in.bytes += length; if (line6pcm->in.bytes >= line6pcm->in.period) { line6pcm->in.bytes %= line6pcm->in.period; - spin_unlock(&line6pcm->in.lock); - snd_pcm_period_elapsed(substream); - spin_lock(&line6pcm->in.lock); + return true; } + return false; } /* * Callback for completed capture URB. */ -static void audio_in_callback(struct urb *urb) +static bool handle_audio_in(struct urb *urb) { int i, index, length = 0, shutdown = 0; - unsigned long flags; - struct snd_line6_pcm *line6pcm = (struct snd_line6_pcm *)urb->context; line6pcm->in.last_frame = urb->start_frame; @@ -156,7 +163,7 @@ static void audio_in_callback(struct urb *urb) if (urb == line6pcm->in.urbs[index]) break; - spin_lock_irqsave(&line6pcm->in.lock, flags); + guard(spinlock_irqsave)(&line6pcm->in.lock); for (i = 0; i < LINE6_ISO_PACKETS; ++i) { char *fbuf; @@ -209,10 +216,18 @@ static void audio_in_callback(struct urb *urb) if (!test_bit(LINE6_STREAM_IMPULSE, &line6pcm->in.running) && test_bit(LINE6_STREAM_PCM, &line6pcm->in.running)) - line6_capture_check_period(line6pcm, length); + return line6_capture_check_period(line6pcm, length); } - spin_unlock_irqrestore(&line6pcm->in.lock, flags); + return false; +} + +static void audio_in_callback(struct urb *urb) +{ + struct snd_line6_pcm *line6pcm = (struct snd_line6_pcm *)urb->context; + + if (handle_audio_in(urb)) + snd_pcm_period_elapsed(get_substream(line6pcm, SNDRV_PCM_STREAM_CAPTURE)); } /* open capture callback */ @@ -228,6 +243,23 @@ static int snd_line6_capture_open(struct snd_pcm_substream *substream) if (err < 0) return err; + /* + * line6_capture_copy() wraps the ring buffer once per packet and + * advances pos_done by the packet, which only works when the buffer + * holds more than one packet: with exactly one, pos_done gains a + * buffer and loses it again on every packet. The packet size comes + * from the device, so bound the buffer by two packets here. The + * impulse-response path copies a playback URB into this ring, so + * take the larger of the two endpoints. + */ + err = snd_pcm_hw_constraint_minmax(runtime, + SNDRV_PCM_HW_PARAM_BUFFER_BYTES, + 2 * max(line6pcm->max_packet_size_in, + line6pcm->max_packet_size_out), + UINT_MAX); + if (err < 0) + return err; + line6_pcm_acquire(line6pcm, LINE6_STREAM_CAPTURE_HELPER, false); runtime->hw = line6pcm->properties->capture_hw; diff --git a/sound/usb/line6/capture.h b/sound/usb/line6/capture.h index 90572dae134e..628315389efc 100644 --- a/sound/usb/line6/capture.h +++ b/sound/usb/line6/capture.h @@ -17,7 +17,7 @@ extern const struct snd_pcm_ops snd_line6_capture_ops; extern void line6_capture_copy(struct snd_line6_pcm *line6pcm, char *fbuf, int fsize); -extern void line6_capture_check_period(struct snd_line6_pcm *line6pcm, +extern bool line6_capture_check_period(struct snd_line6_pcm *line6pcm, int length); extern int line6_create_audio_in_urbs(struct snd_line6_pcm *line6pcm); extern int line6_submit_audio_in_all_urbs(struct snd_line6_pcm *line6pcm); diff --git a/sound/usb/line6/driver.c b/sound/usb/line6/driver.c index e9eb5c74d6c7..caf9bc661686 100644 --- a/sound/usb/line6/driver.c +++ b/sound/usb/line6/driver.c @@ -286,31 +286,30 @@ static void line6_data_received(struct urb *urb) { struct usb_line6 *line6 = (struct usb_line6 *)urb->context; struct midi_buffer *mb = &line6->line6midi->midibuf_in; - unsigned long flags; int done; if (urb->status == -ESHUTDOWN) return; if (line6->properties->capabilities & LINE6_CAP_CONTROL_MIDI) { - spin_lock_irqsave(&line6->line6midi->lock, flags); - done = - line6_midibuf_write(mb, urb->transfer_buffer, urb->actual_length); - - if (done < urb->actual_length) { - line6_midibuf_ignore(mb, done); - dev_dbg(line6->ifcdev, "%d %d buffer overflow - message skipped\n", - done, urb->actual_length); + scoped_guard(spinlock_irqsave, &line6->line6midi->lock) { + done = + line6_midibuf_write(mb, urb->transfer_buffer, urb->actual_length); + + if (done < urb->actual_length) { + line6_midibuf_ignore(mb, done); + dev_dbg(line6->ifcdev, "%d %d buffer overflow - message skipped\n", + done, urb->actual_length); + } } - spin_unlock_irqrestore(&line6->line6midi->lock, flags); for (;;) { - spin_lock_irqsave(&line6->line6midi->lock, flags); - done = - line6_midibuf_read(mb, line6->buffer_message, - LINE6_MIDI_MESSAGE_MAXLEN, - LINE6_MIDIBUF_READ_RX); - spin_unlock_irqrestore(&line6->line6midi->lock, flags); + scoped_guard(spinlock_irqsave, &line6->line6midi->lock) { + done = + line6_midibuf_read(mb, line6->buffer_message, + LINE6_MIDI_MESSAGE_MAXLEN, + LINE6_MIDIBUF_READ_RX); + } if (done <= 0) break; diff --git a/sound/usb/line6/midi.c b/sound/usb/line6/midi.c index 9b5176086280..80d9a6d90342 100644 --- a/sound/usb/line6/midi.c +++ b/sound/usb/line6/midi.c @@ -72,7 +72,6 @@ static void line6_midi_transmit(struct snd_rawmidi_substream *substream) */ static void midi_sent(struct urb *urb) { - unsigned long flags; int status; int num; struct usb_line6 *line6 = (struct usb_line6 *)urb->context; @@ -84,7 +83,7 @@ static void midi_sent(struct urb *urb) if (status == -ESHUTDOWN) return; - spin_lock_irqsave(&line6->line6midi->lock, flags); + guard(spinlock_irqsave)(&line6->line6midi->lock); num = --line6->line6midi->num_active_send_urbs; if (num == 0) { @@ -94,8 +93,6 @@ static void midi_sent(struct urb *urb) if (num == 0) wake_up(&line6->line6midi->send_wait); - - spin_unlock_irqrestore(&line6->line6midi->lock, flags); } /* @@ -158,17 +155,14 @@ static int line6_midi_output_close(struct snd_rawmidi_substream *substream) static void line6_midi_output_trigger(struct snd_rawmidi_substream *substream, int up) { - unsigned long flags; struct usb_line6 *line6 = line6_rawmidi_substream_midi(substream)->line6; line6->line6midi->substream_transmit = substream; - spin_lock_irqsave(&line6->line6midi->lock, flags); + guard(spinlock_irqsave)(&line6->line6midi->lock); if (line6->line6midi->num_active_send_urbs == 0) line6_midi_transmit(substream); - - spin_unlock_irqrestore(&line6->line6midi->lock, flags); } static void line6_midi_output_drain(struct snd_rawmidi_substream *substream) diff --git a/sound/usb/line6/pcm.c b/sound/usb/line6/pcm.c index 7f51a81d7ec9..77d5802872b1 100644 --- a/sound/usb/line6/pcm.c +++ b/sound/usb/line6/pcm.c @@ -190,11 +190,10 @@ static void line6_buffer_release(struct snd_line6_pcm *line6pcm, static int line6_stream_start(struct snd_line6_pcm *line6pcm, int direction, int type) { - unsigned long flags; struct line6_pcm_stream *pstr = get_stream(line6pcm, direction); int ret = 0; - spin_lock_irqsave(&pstr->lock, flags); + guard(spinlock_irqsave)(&pstr->lock); if (!test_and_set_bit(type, &pstr->running) && !(pstr->active_urbs || pstr->unlink_urbs)) { pstr->count = 0; @@ -207,7 +206,6 @@ static int line6_stream_start(struct snd_line6_pcm *line6pcm, int direction, if (ret < 0) clear_bit(type, &pstr->running); - spin_unlock_irqrestore(&pstr->lock, flags); return ret; } @@ -215,21 +213,20 @@ static int line6_stream_start(struct snd_line6_pcm *line6pcm, int direction, static void line6_stream_stop(struct snd_line6_pcm *line6pcm, int direction, int type) { - unsigned long flags; struct line6_pcm_stream *pstr = get_stream(line6pcm, direction); - spin_lock_irqsave(&pstr->lock, flags); - clear_bit(type, &pstr->running); - if (!pstr->running) { - spin_unlock_irqrestore(&pstr->lock, flags); - line6_unlink_audio_urbs(line6pcm, pstr); - spin_lock_irqsave(&pstr->lock, flags); - if (direction == SNDRV_PCM_STREAM_CAPTURE) { - line6pcm->prev_fbuf = NULL; - line6pcm->prev_fsize = 0; - } + scoped_guard(spinlock_irqsave, &pstr->lock) { + clear_bit(type, &pstr->running); + if (pstr->running) + return; + } + + line6_unlink_audio_urbs(line6pcm, pstr); + if (direction == SNDRV_PCM_STREAM_CAPTURE) { + guard(spinlock_irqsave)(&pstr->lock); + line6pcm->prev_fbuf = NULL; + line6pcm->prev_fsize = 0; } - spin_unlock_irqrestore(&pstr->lock, flags); } /* common PCM trigger callback */ diff --git a/sound/usb/line6/playback.c b/sound/usb/line6/playback.c index 2e70ca5456c5..495a983a0c51 100644 --- a/sound/usb/line6/playback.c +++ b/sound/usb/line6/playback.c @@ -136,6 +136,8 @@ static void add_monitor_signal(struct urb *urb_out, unsigned char *signal, /* Find a free URB, prepare audio data, and submit URB. must be called in line6pcm->out.lock context + Returns 1 if snd_pcm_period_elapsed() needs to be called for the + capture stream after releasing the lock, 0 if not, or a negative error */ static int submit_audio_out_urb(struct snd_line6_pcm *line6pcm) { @@ -151,6 +153,7 @@ static int submit_audio_out_urb(struct snd_line6_pcm *line6pcm) line6pcm->properties->rates.rats[0].den * (line6pcm->line6->intervals_per_second / LINE6_ISO_INTERVAL); struct urb *urb_out; + bool capture_period_elapsed = false; index = find_first_zero_bit(&line6pcm->out.active_urbs, line6pcm->line6->iso_buffers); @@ -260,8 +263,9 @@ static int submit_audio_out_urb(struct snd_line6_pcm *line6pcm) urb_out->transfer_buffer, urb_out-> transfer_buffer_length); - line6_capture_check_period(line6pcm, - urb_out->transfer_buffer_length); + capture_period_elapsed = + line6_capture_check_period(line6pcm, + urb_out->transfer_buffer_length); } } else { if (!(line6pcm->line6->properties->capabilities & LINE6_CAP_HWMON) @@ -283,24 +287,26 @@ static int submit_audio_out_urb(struct snd_line6_pcm *line6pcm) dev_err(line6pcm->line6->ifcdev, "URB out #%d submission failed (%d)\n", index, ret); - return 0; + return capture_period_elapsed; } /* Submit all currently available playback URBs. must be called in line6pcm->out.lock context + The capture period notification is skipped here since we can't call + snd_pcm_period_elapsed() in the lock; it'll be caught up at the next one. */ int line6_submit_audio_out_all_urbs(struct snd_line6_pcm *line6pcm) { - int ret = 0, i; + int ret, i; for (i = 0; i < line6pcm->line6->iso_buffers; ++i) { ret = submit_audio_out_urb(line6pcm); if (ret < 0) - break; + return ret; } - return ret; + return 0; } /* @@ -316,6 +322,8 @@ static void audio_out_callback(struct urb *urb) const int bytes_per_frame = line6pcm->properties->bytes_per_channel * line6pcm->properties->playback_hw.channels_max; + bool capture_period_elapsed = false; + bool playback_period_elapsed = false; #if USE_CLEAR_BUFFER_WORKAROUND memset(urb->transfer_buffer, 0, urb->transfer_buffer_length); @@ -358,19 +366,22 @@ static void audio_out_callback(struct urb *urb) shutdown = 1; if (!shutdown) { - submit_audio_out_urb(line6pcm); + capture_period_elapsed = submit_audio_out_urb(line6pcm) > 0; if (test_bit(LINE6_STREAM_PCM, &line6pcm->out.running)) { line6pcm->out.bytes += length; if (line6pcm->out.bytes >= line6pcm->out.period) { line6pcm->out.bytes %= line6pcm->out.period; - spin_unlock(&line6pcm->out.lock); - snd_pcm_period_elapsed(substream); - spin_lock(&line6pcm->out.lock); + playback_period_elapsed = true; } } } spin_unlock_irqrestore(&line6pcm->out.lock, flags); + + if (capture_period_elapsed) + snd_pcm_period_elapsed(get_substream(line6pcm, SNDRV_PCM_STREAM_CAPTURE)); + if (playback_period_elapsed) + snd_pcm_period_elapsed(substream); } /* open playback callback */ diff --git a/sound/usb/midi.c b/sound/usb/midi.c index cb9bcfa22afc..6ac38297c6c5 100644 --- a/sound/usb/midi.c +++ b/sound/usb/midi.c @@ -1152,7 +1152,9 @@ static int substream_open(struct snd_rawmidi_substream *substream, int dir, int open) { struct snd_usb_midi *umidi = substream->rmidi->private_data; - struct snd_kcontrol *ctl; + struct snd_ctl_elem_id ctl_id; + bool activate_ctl = false; + bool active; down_read(&umidi->disc_rwsem); if (umidi->disconnected) { @@ -1164,11 +1166,9 @@ static int substream_open(struct snd_rawmidi_substream *substream, int dir, if (open) { if (!umidi->opened[0] && !umidi->opened[1]) { if (umidi->roland_load_ctl) { - ctl = umidi->roland_load_ctl; - ctl->vd[0].access |= - SNDRV_CTL_ELEM_ACCESS_INACTIVE; - snd_ctl_notify(umidi->card, - SNDRV_CTL_EVENT_MASK_INFO, &ctl->id); + ctl_id = umidi->roland_load_ctl->id; + activate_ctl = true; + active = false; update_roland_altsetting(umidi); } } @@ -1181,15 +1181,15 @@ static int substream_open(struct snd_rawmidi_substream *substream, int dir, snd_usbmidi_input_stop(&umidi->list); if (!umidi->opened[0] && !umidi->opened[1]) { if (umidi->roland_load_ctl) { - ctl = umidi->roland_load_ctl; - ctl->vd[0].access &= - ~SNDRV_CTL_ELEM_ACCESS_INACTIVE; - snd_ctl_notify(umidi->card, - SNDRV_CTL_EVENT_MASK_INFO, &ctl->id); + ctl_id = umidi->roland_load_ctl->id; + activate_ctl = true; + active = true; } } } mutex_unlock(&umidi->mutex); + if (activate_ctl) + snd_ctl_activate_id(umidi->card, &ctl_id, active); up_read(&umidi->disc_rwsem); return 0; } diff --git a/sound/usb/mixer.c b/sound/usb/mixer.c index 82c859626404..45145ef7db7c 100644 --- a/sound/usb/mixer.c +++ b/sound/usb/mixer.c @@ -1156,6 +1156,14 @@ static void volume_control_quirks(struct usb_mixer_elem_info *cval, } break; + case USB_ID(0x045e, 0x070f): /* MS LifeChat LX-3000 Headset */ + if (!strcmp(kctl->id.name, "Speaker Playback Volume")) { + usb_audio_info(chip, + "set volume quirk for MS LifeChat LX-3000\n"); + cval->res = 192; + } + break; + case USB_ID(0x0471, 0x0101): case USB_ID(0x0471, 0x0104): case USB_ID(0x0471, 0x0105): diff --git a/sound/usb/mixer_quirks.c b/sound/usb/mixer_quirks.c index 48f6b53fefd8..18e41ea17956 100644 --- a/sound/usb/mixer_quirks.c +++ b/sound/usb/mixer_quirks.c @@ -4501,9 +4501,12 @@ void snd_usb_mixer_fu_apply_quirk(struct usb_mixer_interface *mixer, } /* lowest playback value is muted on some devices */ - if (mixer->chip->quirk_flags & QUIRK_FLAG_MIXER_MIN_MUTE) - if (strstr(kctl->id.name, "Playback")) + if (mixer->chip->quirk_flags & QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE) + if (strstr(kctl->id.name, "Playback")) { + usb_audio_info(mixer->chip, + "applying playback min mute quirk\n"); cval->min_mute = 1; + } /* ALSA-ify some Plantronics headset control names */ if (USB_ID_VENDOR(mixer->chip->usb_id) == 0x047f && diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c index 5aadd1221f72..dd7554cde865 100644 --- a/sound/usb/pcm.c +++ b/sound/usb/pcm.c @@ -1292,6 +1292,18 @@ static void retire_capture_urb(struct snd_usb_substream *subs, bytes -= adj; subs->stream_offset_adj -= adj; } + /* + * A packet can be as large as the endpoint's maxpacksize, which + * the device chooses, while the ring buffer can be as small as + * two 64-byte periods. The hwptr_done wrap below handles one + * wrap only, so never take more than one buffer from a packet. + */ + if (unlikely(bytes > subs->buffer_bytes)) { + dev_warn_ratelimited(&subs->dev->dev, + "%u-byte packet exceeds %u-byte buffer, truncating\n", + bytes, subs->buffer_bytes); + bytes = subs->buffer_bytes; + } frames = bytes / stride; if (!subs->txfr_quirk) bytes = frames * stride; @@ -1505,8 +1517,14 @@ static int prepare_playback_urb(struct snd_usb_substream *subs, for (i = 0; i < ctx->packets; i++) { counts = snd_usb_endpoint_next_packet_size(ep, ctx, i, avail); + /* + * Bound by the ring buffer as well as by the URB: + * copy_to_urb() wraps hwptr_done once, so one URB must + * never carry more than the whole buffer. + */ if (counts < 0 || - (frames + counts) * stride > ctx->buffer_size) + (frames + counts) * stride > ctx->buffer_size || + (frames + counts) * stride > subs->buffer_bytes) break; /* set up descriptor */ urb->iso_frame_desc[i].offset = frames * stride; diff --git a/sound/usb/quirks.c b/sound/usb/quirks.c index f224e41209e8..5a69bd87628a 100644 --- a/sound/usb/quirks.c +++ b/sound/usb/quirks.c @@ -2158,6 +2158,10 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = { DEVICE_FLG(0x045e, 0x083c, /* MS USB Link headset */ QUIRK_FLAG_GET_SAMPLE_RATE | QUIRK_FLAG_CTL_MSG_DELAY | QUIRK_FLAG_DISABLE_AUTOSUSPEND), + DEVICE_FLG(0x045e, 0x070f, /* MS LifeChat LX-3000 Headset */ + QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE), + DEVICE_FLG(0x046d, 0x0802, /* Logitech Webcam C200 */ + QUIRK_FLAG_CTL_MSG_DELAY_1M | QUIRK_FLAG_MIC_RES_384), DEVICE_FLG(0x046d, 0x0807, /* Logitech Webcam C500 */ QUIRK_FLAG_CTL_MSG_DELAY_1M | QUIRK_FLAG_MIC_RES_384), DEVICE_FLG(0x046d, 0x0808, /* Logitech Webcam C600 */ @@ -2202,9 +2206,9 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = { DEVICE_FLG(0x0556, 0x0014, /* Phoenix Audio TMX320VC */ QUIRK_FLAG_GET_SAMPLE_RATE), DEVICE_FLG(0x0572, 0x1b08, /* Conexant Systems (Rockwell), Inc. */ - QUIRK_FLAG_MIXER_MIN_MUTE), + QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE), DEVICE_FLG(0x0572, 0x1b09, /* Conexant Systems (Rockwell), Inc. */ - QUIRK_FLAG_MIXER_MIN_MUTE), + QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE), DEVICE_FLG(0x05a3, 0x9420, /* ELP HD USB Camera */ QUIRK_FLAG_GET_SAMPLE_RATE), DEVICE_FLG(0x05a7, 0x1020, /* Bose Companion 5 */ @@ -2254,15 +2258,15 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = { DEVICE_FLG(0x0b0e, 0x0349, /* Jabra 550a */ QUIRK_FLAG_CTL_MSG_DELAY_1M), DEVICE_FLG(0x0bda, 0x498a, /* Realtek Semiconductor Corp. */ - QUIRK_FLAG_MIXER_MIN_MUTE), + QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE), DEVICE_FLG(0x0c45, 0x6340, /* Sonix HD USB Camera */ QUIRK_FLAG_GET_SAMPLE_RATE), DEVICE_FLG(0x0c45, 0x636b, /* Microdia JP001 USB Camera */ QUIRK_FLAG_GET_SAMPLE_RATE), DEVICE_FLG(0x0d8c, 0x000c, /* C-Media */ - QUIRK_FLAG_MIXER_MIN_MUTE), + QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE), DEVICE_FLG(0x0d8c, 0x0014, /* C-Media */ - QUIRK_FLAG_CTL_MSG_DELAY_1M | QUIRK_FLAG_MIXER_MIN_MUTE), + QUIRK_FLAG_CTL_MSG_DELAY_1M | QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE), DEVICE_FLG(0x0ecb, 0x205c, /* JBL Quantum610 Wireless */ QUIRK_FLAG_FIXED_RATE), DEVICE_FLG(0x0ecb, 0x2125, /* JBL Quantum650 Wireless */ @@ -2274,7 +2278,7 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = { DEVICE_FLG(0x1101, 0x0003, /* Audioengine D1 */ QUIRK_FLAG_GET_SAMPLE_RATE), DEVICE_FLG(0x12d1, 0x3a07, /* Huawei Technologies Co., Ltd. */ - QUIRK_FLAG_MIXER_MIN_MUTE), + QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE), DEVICE_FLG(0x1224, 0x2a25, /* Jieli Technology USB PHY 2.0 */ QUIRK_FLAG_GET_SAMPLE_RATE | QUIRK_FLAG_MIC_RES_16), DEVICE_FLG(0x1395, 0x740a, /* Sennheiser DECT */ @@ -2320,7 +2324,7 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = { DEVICE_FLG(0x1901, 0x0191, /* GE B850V3 CP2114 audio interface */ QUIRK_FLAG_GET_SAMPLE_RATE), DEVICE_FLG(0x19f7, 0x0003, /* RODE NT-USB */ - QUIRK_FLAG_MIXER_MIN_MUTE), + QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE), DEVICE_FLG(0x19f7, 0x0035, /* RODE NT-USB+ */ QUIRK_FLAG_GET_SAMPLE_RATE), DEVICE_FLG(0x1bcf, 0x2281, /* HD Webcam */ @@ -2378,7 +2382,7 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = { DEVICE_FLG(0x2912, 0x30c8, /* Audioengine D1 */ QUIRK_FLAG_GET_SAMPLE_RATE), DEVICE_FLG(0x2a70, 0x1881, /* OnePlus Technology (Shenzhen) Co., Ltd. BE02T */ - QUIRK_FLAG_MIXER_MIN_MUTE), + QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE), DEVICE_FLG(0x2b53, 0x0023, /* Fiero SC-01 (firmware v1.0.0 @ 48 kHz) */ QUIRK_FLAG_GENERIC_IMPLICIT_FB), DEVICE_FLG(0x2b53, 0x0024, /* Fiero SC-01 (firmware v1.0.0 @ 96 kHz) */ @@ -2390,7 +2394,7 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = { DEVICE_FLG(0x2d95, 0x8021, /* VIVO USB-C-XE710 HEADSET */ QUIRK_FLAG_CTL_MSG_DELAY_1M), DEVICE_FLG(0x2d99, 0x0026, /* HECATE G2 GAMING HEADSET */ - QUIRK_FLAG_MIXER_MIN_MUTE), + QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE), DEVICE_FLG(0x2fc6, 0xf0b7, /* iBasso DC07 Pro */ QUIRK_FLAG_CTL_MSG_DELAY_1M), DEVICE_FLG(0x30be, 0x0101, /* Schiit Hel */ @@ -2398,7 +2402,7 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = { DEVICE_FLG(0x3255, 0x0000, /* Luxman D-10X */ QUIRK_FLAG_ITF_USB_DSD_DAC | QUIRK_FLAG_CTL_MSG_DELAY), DEVICE_FLG(0x339b, 0x3a07, /* Synaptics HONOR USB-C HEADSET */ - QUIRK_FLAG_MIXER_MIN_MUTE), + QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE), DEVICE_FLG(0x413c, 0xa506, /* Dell AE515 sound bar */ QUIRK_FLAG_GET_SAMPLE_RATE), DEVICE_FLG(0x534d, 0x0021, /* MacroSilicon MS2100/MS2106 */ diff --git a/sound/usb/usbaudio.h b/sound/usb/usbaudio.h index 1ef4d39978df..8a951394cabf 100644 --- a/sound/usb/usbaudio.h +++ b/sound/usb/usbaudio.h @@ -196,7 +196,7 @@ extern bool snd_usb_skip_validation; * for the given endpoint. * QUIRK_FLAG_MIC_RES_16 and QUIRK_FLAG_MIC_RES_384 * Set the fixed resolution for Mic Capture Volume (mostly for webcams) - * QUIRK_FLAG_MIXER_MIN_MUTE + * QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE * Set minimum volume control value as mute for devices where the lowest * playback value represents muted state instead of minimum audible volume */ @@ -225,6 +225,6 @@ extern bool snd_usb_skip_validation; #define QUIRK_FLAG_FIXED_RATE (1U << 21) #define QUIRK_FLAG_MIC_RES_16 (1U << 22) #define QUIRK_FLAG_MIC_RES_384 (1U << 23) -#define QUIRK_FLAG_MIXER_MIN_MUTE (1U << 24) +#define QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE (1U << 24) #endif /* __USBAUDIO_H */ diff --git a/sound/usb/usx2y/us122l.c b/sound/usb/usx2y/us122l.c index 2a107803af88..888005192704 100644 --- a/sound/usb/usx2y/us122l.c +++ b/sound/usb/usx2y/us122l.c @@ -148,19 +148,26 @@ static const struct vm_operations_struct usb_stream_hwdep_vm_ops = { static int usb_stream_hwdep_open(struct snd_hwdep *hw, struct file *file) { struct us122l *us122l = hw->private_data; - struct usb_interface *iface; + struct usb_interface *iface, *iface0 = NULL; if (hw->used >= 2) return -EBUSY; - if (!us122l->first) - us122l->first = file; - if (us122l->is_us144) { - iface = usb_ifnum_to_if(us122l->dev, 0); - usb_autopm_get_interface(iface); + iface0 = usb_ifnum_to_if(us122l->dev, 0); + if (!iface0) + return -ENODEV; } + iface = usb_ifnum_to_if(us122l->dev, 1); + if (!iface) + return -ENODEV; + + if (!us122l->first) + us122l->first = file; + + if (iface0) + usb_autopm_get_interface(iface0); usb_autopm_get_interface(iface); return 0; } @@ -172,10 +179,12 @@ static int usb_stream_hwdep_release(struct snd_hwdep *hw, struct file *file) if (us122l->is_us144) { iface = usb_ifnum_to_if(us122l->dev, 0); - usb_autopm_put_interface(iface); + if (iface) + usb_autopm_put_interface(iface); } iface = usb_ifnum_to_if(us122l->dev, 1); - usb_autopm_put_interface(iface); + if (iface) + usb_autopm_put_interface(iface); if (us122l->first == file) us122l->first = NULL; mutex_lock(&us122l->mutex); @@ -200,6 +209,8 @@ static int usb_stream_hwdep_mmap(struct snd_hwdep *hw, offset = area->vm_pgoff << PAGE_SHIFT; mutex_lock(&us122l->mutex); s = us122l->sk.s; + if (!s) + return -ENODEV; read = offset < s->read_size; if (read) { if (area->vm_flags & VM_WRITE) { diff --git a/sound/usb/usx2y/usX2Yhwdep.c b/sound/usb/usx2y/usX2Yhwdep.c index 2cc0f90a23be..be755a8afaf2 100644 --- a/sound/usb/usx2y/usX2Yhwdep.c +++ b/sound/usb/usx2y/usX2Yhwdep.c @@ -179,6 +179,9 @@ static int snd_usx2y_hwdep_dsp_load(struct snd_hwdep *hw, int lret, err; char *buf; + if (priv->chip_status & USX2Y_STAT_CHIP_HUP) + return -ENODEV; + buf = memdup_user(dsp->image, dsp->length); if (IS_ERR(buf)) return PTR_ERR(buf); @@ -201,11 +204,13 @@ static int snd_usx2y_hwdep_dsp_load(struct snd_hwdep *hw, err = usx2y_in04_init(priv); if (err) { dev_err(&dev->dev, "usx2y_in04_init error\n"); + priv->chip_status |= USX2Y_STAT_CHIP_HUP; return err; } err = usx2y_create_alsa_devices(hw->card); if (err) { dev_err(&dev->dev, "usx2y_create_alsa_devices error %i\n", err); + priv->chip_status |= USX2Y_STAT_CHIP_HUP; return err; } priv->chip_status |= USX2Y_STAT_CHIP_INIT; diff --git a/sound/usb/usx2y/usb_stream.c b/sound/usb/usx2y/usb_stream.c index 3122cf653273..7d621280fba3 100644 --- a/sound/usb/usx2y/usb_stream.c +++ b/sound/usb/usx2y/usb_stream.c @@ -682,6 +682,10 @@ int usb_stream_start(struct usb_stream_kernel *sk) s->sync_packet = -1; s->inpacket_head = -1; sk->iso_frame_balance = 0; + sk->idle_inurb = sk->inurb[USB_STREAM_NURBS - 2]; + sk->idle_outurb = sk->outurb[USB_STREAM_NURBS - 2]; + sk->completed_inurb = sk->inurb[USB_STREAM_NURBS - 1]; + sk->completed_outurb = sk->outurb[USB_STREAM_NURBS - 1]; ++try; for (u = 0; u < 2; u++) { struct urb *inurb = sk->inurb[u]; @@ -739,11 +743,6 @@ int usb_stream_start(struct usb_stream_kernel *sk) return -EFAULT; } - sk->idle_inurb = sk->inurb[USB_STREAM_NURBS - 2]; - sk->idle_outurb = sk->outurb[USB_STREAM_NURBS - 2]; - sk->completed_inurb = sk->inurb[USB_STREAM_NURBS - 1]; - sk->completed_outurb = sk->outurb[USB_STREAM_NURBS - 1]; - /* wait, check */ { int wait_ms = 3000; diff --git a/sound/usb/usx2y/usbusx2yaudio.c b/sound/usb/usx2y/usbusx2yaudio.c index 147dd0680528..bd803874412b 100644 --- a/sound/usb/usx2y/usbusx2yaudio.c +++ b/sound/usb/usx2y/usbusx2yaudio.c @@ -51,6 +51,18 @@ static int usx2y_urb_capt_retire(struct snd_usx2y_substream *subs) dev_dbg(&usx2y->dev->dev, "%s: 0 == len ERROR!\n", __func__); continue; } + /* + * snd_usx2y_pcm_open() constrains the buffer to hold at least + * two URBs, so this cannot happen for a device that keeps its + * packets at 50 frames. Drop the packet rather than exceed the + * single wrap below. + */ + if (unlikely(len > runtime->buffer_size)) { + dev_err_ratelimited(&usx2y->dev->dev, + "%s: packet of %i frames exceeds %lu-frame buffer, dropping\n", + __func__, len, runtime->buffer_size); + continue; + } /* copy a data chunk */ if ((hwptr_done + len) > runtime->buffer_size) { @@ -106,14 +118,31 @@ static int usx2y_urb_play_prepare(struct snd_usx2y_substream *subs, __func__, counts); return -EPIPE; } + if (counts * usx2y->stride > subs->maxpacksize) { + dev_err_ratelimited(&usx2y->dev->dev, + "%s: packet of %i bytes exceeds maxpacksize %u\n", + __func__, counts * usx2y->stride, + subs->maxpacksize); + return -EPIPE; + } /* set up descriptor */ urb->iso_frame_desc[pack].offset = pack ? urb->iso_frame_desc[pack - 1].offset + urb->iso_frame_desc[pack - 1].length : 0; - urb->iso_frame_desc[pack].length = cap_urb->iso_frame_desc[pack].actual_length; + urb->iso_frame_desc[pack].length = counts * usx2y->stride; } if (atomic_read(&subs->state) >= STATE_PRERUNNING) { + /* + * The copy below wraps once, which needs count <= buffer_size. + * snd_usx2y_pcm_open() guarantees it; guard anyway. + */ + if (unlikely(count > runtime->buffer_size)) { + dev_err_ratelimited(&usx2y->dev->dev, + "%s: URB of %i frames exceeds %lu-frame buffer, stopping\n", + __func__, count, runtime->buffer_size); + return -EPIPE; + } if (subs->hwptr + count > runtime->buffer_size) { /* err, the transferred area goes over buffer boundary. * copy the data to the temp buffer. @@ -897,6 +926,18 @@ static int snd_usx2y_pcm_open(struct snd_pcm_substream *substream) runtime->private_data = subs; subs->pcm_substream = substream; snd_pcm_hw_constraint_minmax(runtime, SNDRV_PCM_HW_PARAM_PERIOD_TIME, 1000, 200000); + /* + * Capture wraps the ring buffer once per packet and playback once + * per URB of nr_of_packs() packets of at most 50 frames (see + * usx2y_urb_play_prepare()). Playback needs the buffer to hold more + * than one URB and capture more than one packet; use the URB bound + * for both, and two URBs rather than one, because the position only + * advances when a URB is less than the whole buffer. The bound is in + * frames and does not depend on the altsetting, which is selected + * in prepare. + */ + snd_pcm_hw_constraint_minmax(runtime, SNDRV_PCM_HW_PARAM_BUFFER_SIZE, + 2 * nr_of_packs() * 50, UINT_MAX); return 0; } diff --git a/sound/x86/intel_hdmi_audio.c b/sound/x86/intel_hdmi_audio.c index 571a092b128a..3c8f1793522a 100644 --- a/sound/x86/intel_hdmi_audio.c +++ b/sound/x86/intel_hdmi_audio.c @@ -479,7 +479,8 @@ static void had_build_channel_allocation_map(struct snd_intelhad *intelhaddata) kfree(intelhaddata->chmap->chmap); intelhaddata->chmap->chmap = NULL; - chmap = kzalloc(sizeof(*chmap), GFP_KERNEL); + /* info->chmap must be zero-terminated; allocate a spare terminator entry */ + chmap = kzalloc_objs(*chmap, 2); if (!chmap) return; @@ -582,6 +583,12 @@ static int had_register_chmap_ctls(struct snd_intelhad *intelhaddata, intelhaddata->chmap->kctl->info = had_chmap_ctl_info; intelhaddata->chmap->kctl->get = had_chmap_ctl_get; intelhaddata->chmap->chmap = NULL; + + /* drop the inherited TLV read: it walks chmap without our mutex */ + intelhaddata->chmap->kctl->tlv.c = NULL; + intelhaddata->chmap->kctl->vd[0].access &= + ~(SNDRV_CTL_ELEM_ACCESS_TLV_READ | SNDRV_CTL_ELEM_ACCESS_TLV_CALLBACK); + return 0; } diff --git a/tools/arch/x86/include/asm/cpufeatures.h b/tools/arch/x86/include/asm/cpufeatures.h index dd4682857c12..f66ac38c9f6f 100644 --- a/tools/arch/x86/include/asm/cpufeatures.h +++ b/tools/arch/x86/include/asm/cpufeatures.h @@ -473,6 +473,11 @@ #define X86_FEATURE_CLEAR_BHB_HW (21*32+ 3) /* BHI_DIS_S HW control enabled */ #define X86_FEATURE_CLEAR_BHB_LOOP_ON_VMEXIT (21*32+ 4) /* Clear branch history at vmexit using SW loop */ #define X86_FEATURE_FAST_CPPC (21*32 + 5) /* AMD Fast CPPC */ +/* + * Upstream defines X86_FEATURE_ZEN6 as (3*32+6). That bit is still + * X86_FEATURE_P3 in this kernel, so use a free bit in this word. + */ +#define X86_FEATURE_ZEN6 (21*32+7) /* CPU based on Zen6 microarchitecture */ /* * BUG word(s) diff --git a/tools/testing/selftests/net/fib_nexthops.sh b/tools/testing/selftests/net/fib_nexthops.sh index 25a4ad02ac16..3a9d3d8419d4 100755 --- a/tools/testing/selftests/net/fib_nexthops.sh +++ b/tools/testing/selftests/net/fib_nexthops.sh @@ -421,6 +421,14 @@ stop_ip_monitor() return $rc } +fdb_entry_aged() +{ + local mac=$1 + local dev=$2 + + ! $BRIDGE fdb get to $mac dev $dev self &> /dev/null +} + check_nexthop_fdb_support() { $IP nexthop help 2>&1 | grep -q fdb @@ -505,14 +513,18 @@ ipv6_fdb_grp_fcnal() log_test $? 0 "Fdb mac add with nexthop group and static state" run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic" - log_test $? 255 "Fdb mac add with nexthop group and dynamic state" + log_test $? 0 "Fdb mac add with nexthop group and dynamic state" + + run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self dynamic extern_learn" + log_test $? 0 "Fdb mac add with nexthop group and extern_learn" - run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self" - run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic" - log_test $? 255 "Fdb mac replace with nexthop group and dynamic state" + run_cmd "$IP link set dev vx10 up type vxlan ageing 1" + run_cmd "$BRIDGE fdb add 02:02:00:00:00:18 dev vx10 dst 2001:db8:91::11 self dynamic" + slowwait 20 fdb_entry_aged 02:02:00:00:00:18 vx10 + log_test $? 0 "Fdb mac without nexthop group aged out" - run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic" - log_test $? 255 "Fdb mac append with nexthop group and dynamic state" + run_cmd "$BRIDGE fdb get to 02:02:00:00:00:16 dev vx10 self" + log_test $? 0 "Fdb mac with nexthop group not aged out" run_cmd "$IP -6 ro add 2001:db8:101::1/128 nhid 66" log_test $? 2 "Route add with fdb nexthop" @@ -599,14 +611,18 @@ ipv4_fdb_grp_fcnal() log_test $? 0 "Fdb mac add with nexthop group and static state" run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic" - log_test $? 255 "Fdb mac add with nexthop group and dynamic state" + log_test $? 0 "Fdb mac add with nexthop group and dynamic state" + + run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self dynamic extern_learn" + log_test $? 0 "Fdb mac add with nexthop group and extern_learn" - run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self" - run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic" - log_test $? 255 "Fdb mac replace with nexthop group and dynamic state" + run_cmd "$IP link set dev vx10 up type vxlan ageing 1" + run_cmd "$BRIDGE fdb add 02:02:00:00:00:18 dev vx10 dst 10.0.0.3 self dynamic" + slowwait 20 fdb_entry_aged 02:02:00:00:00:18 vx10 + log_test $? 0 "Fdb mac without nexthop group aged out" - run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic" - log_test $? 255 "Fdb mac append with nexthop group and dynamic state" + run_cmd "$BRIDGE fdb get to 02:02:00:00:00:16 dev vx10 self" + log_test $? 0 "Fdb mac with nexthop group not aged out" run_cmd "$IP ro add 172.16.0.0/22 nhid 16" log_test $? 2 "Route add with fdb nexthop" diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_internal.h index 0a95dbb0346f..213e40afa8ee 100644 --- a/tools/testing/vma/vma_internal.h +++ b/tools/testing/vma/vma_internal.h @@ -605,6 +605,8 @@ static inline bool is_mergeable_vm_userfaultfd_ctx(struct vm_area_struct *vma, return true; } +#define vma_has_uffd_without_event_remap(vma) false + static inline bool anon_vma_name_eq(struct anon_vma_name *anon_name1, struct anon_vma_name *anon_name2) { diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 2e9b5129b5a2..4f69260cafbb 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -472,15 +472,13 @@ int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args) return __kvm_gmem_create(kvm, size, flags); } -int kvm_gmem_bind(struct kvm *kvm, struct kvm_memory_slot *slot, - unsigned int fd, u64 offset) +int kvm_gmem_prepare_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot, + unsigned int fd, u64 offset) { u64 size = slot->npages << PAGE_SHIFT; - unsigned long start, end; struct kvm_gmem *gmem; struct inode *inode; struct file *file; - int r = -EINVAL; BUILD_BUG_ON(sizeof(gpa_t) != sizeof(offset)); BUILD_BUG_ON(sizeof(gfn_t) != sizeof(slot->gmem.pgoff)); @@ -501,90 +499,91 @@ int kvm_gmem_bind(struct kvm *kvm, struct kvm_memory_slot *slot, if (!PAGE_ALIGNED(offset) || offset + size > i_size_read(inode)) goto err; - filemap_invalidate_lock(inode->i_mapping); - - start = offset >> PAGE_SHIFT; - end = start + slot->npages; - - if (!xa_empty(&gmem->bindings) && - xa_find(&gmem->bindings, &start, end - 1, XA_PRESENT)) { - filemap_invalidate_unlock(inode->i_mapping); - goto err; - } - /* * memslots of flag KVM_MEM_GUEST_MEMFD are immutable to change, so * kvm_gmem_bind() must occur on a new memslot. Because the memslot * is not visible yet, kvm_gmem_get_pfn() is guaranteed to see the file. */ WRITE_ONCE(slot->gmem.file, file); - slot->gmem.pgoff = start; - - r = xa_err(xa_store_range(&gmem->bindings, start, end - 1, slot, GFP_KERNEL)); - if (r) { - xa_store_range(&gmem->bindings, start, end - 1, NULL, GFP_KERNEL); - slot->gmem.file = NULL; - slot->gmem.pgoff = 0; - } - filemap_invalidate_unlock(inode->i_mapping); + slot->gmem.pgoff = offset >> PAGE_SHIFT; /* - * Drop the reference to the file, even on success. The file pins KVM, - * not the other way 'round. Active bindings are invalidated if the - * file is closed before memslots are destroyed. + * Gift the caller a reference to the file. The reference will be + * dropped after bindings are established, or if installing the new + * memslot ultimately fails. */ + return 0; + err: fput(file); - return r; + return -EINVAL; } -static void __kvm_gmem_unbind(struct kvm_memory_slot *slot, struct kvm_gmem *gmem) +int kvm_gmem_commit_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot) { - unsigned long start = slot->gmem.pgoff; - unsigned long end = start + slot->npages; + struct kvm_gmem *gmem = slot->gmem.file->private_data; + struct inode *inode = file_inode(slot->gmem.file); + unsigned long start, end; + int r; - xa_store_range(&gmem->bindings, start, end - 1, NULL, GFP_KERNEL); + if (WARN_ON_ONCE(slot->gmem.file->f_op != &kvm_gmem_fops)) + return -EIO; - /* - * synchronize_srcu(&kvm->srcu) ensured that kvm_gmem_get_pfn() - * cannot see this memslot. - */ - WRITE_ONCE(slot->gmem.file, NULL); + filemap_invalidate_lock(inode->i_mapping); + + start = slot->gmem.pgoff; + end = start + slot->npages; + + if (!xa_empty(&gmem->bindings) && + xa_find(&gmem->bindings, &start, end - 1, XA_PRESENT)) { + filemap_invalidate_unlock(inode->i_mapping); + return -EEXIST; + } + + r = xa_err(xa_store_range(&gmem->bindings, start, end - 1, slot, GFP_KERNEL)); + if (r) + xa_store_range(&gmem->bindings, start, end - 1, NULL, GFP_KERNEL); + + filemap_invalidate_unlock(inode->i_mapping); + + return r; } void kvm_gmem_unbind(struct kvm_memory_slot *slot) { - struct file *file; + struct file *file = slot->gmem.file; + unsigned long start = slot->gmem.pgoff; + unsigned long end = start + slot->npages; + struct kvm_gmem *gmem; /* * Nothing to do if the underlying file was _already_ closed, as * kvm_gmem_release() invalidates and nullifies all bindings. */ - if (!slot->gmem.file) + if (!file) return; - file = kvm_gmem_get_file(slot); - /* * However, if the file is _being_ closed, then the bindings need to be * removed as kvm_gmem_release() might not run until after the memslot - * is freed. Note, modifying the bindings is safe even though the file - * is dying as kvm_gmem_release() nullifies slot->gmem.file under - * slots_lock, and only puts its reference to KVM after destroying all - * bindings. I.e. reaching this point means kvm_gmem_release() hasn't - * yet destroyed the bindings or freed the gmem_file, and can't do so - * until the caller drops slots_lock. + * is freed. Modifying the bindings is safe even if the file is dying + * as kvm_gmem_release() nullifies slot->gmem.file under slots_lock, + * and only puts its reference to KVM after destroying all bindings. + * I.e. reaching this point means kvm_gmem_release() hasn't destroyed + * the bindings or freed the gmem_file and can't do so until the caller + * drops slots_lock, so there's no need to verify the file is live. */ - if (!file) { - __kvm_gmem_unbind(slot, slot->gmem.file->private_data); - return; - } + gmem = file->private_data; filemap_invalidate_lock(file->f_mapping); - __kvm_gmem_unbind(slot, file->private_data); - filemap_invalidate_unlock(file->f_mapping); + xa_store_range(&gmem->bindings, start, end - 1, NULL, GFP_KERNEL); - fput(file); + /* + * synchronize_srcu(&kvm->srcu) ensured that kvm_gmem_get_pfn() + * cannot see this memslot. + */ + WRITE_ONCE(slot->gmem.file, NULL); + filemap_invalidate_unlock(file->f_mapping); } /* Returns a locked folio on success. */ diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 7debedc1ddaa..3fac4fe389ce 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1725,6 +1725,12 @@ static void kvm_commit_memory_region(struct kvm *kvm, kvm_free_memslot(kvm, old); break; case KVM_MR_MOVE: + /* + * Moving a guest_memfd memslot isn't supported, and will never + * be supported. + */ + WARN_ON_ONCE(old->flags & KVM_MEM_GUEST_MEMFD); + fallthrough; case KVM_MR_FLAGS_ONLY: /* * Free the dirty bitmap as needed; the below check encompasses @@ -1733,6 +1739,15 @@ static void kvm_commit_memory_region(struct kvm *kvm, if (old->dirty_bitmap && !new->dirty_bitmap) kvm_destroy_dirty_bitmap(old); + /* + * TODO: Drop the WARN and do the unbind() call only for MOVE + * when dirty logging guest_memfd memslots is supported. Until + * then, flags-only changes on guest_memfd slots should also be + * impossible; unbind the old memslot for defense-in-depth. + */ + if (WARN_ON_ONCE(old->flags & KVM_MEM_GUEST_MEMFD)) + kvm_gmem_unbind(old); + /* * The final quirk. Free the detached, old slot, but only its * memory, not any metadata. Metadata, including arch specific @@ -1917,6 +1932,15 @@ static int kvm_set_memslot(struct kvm *kvm, if (r) goto err; + if (change == KVM_MR_CREATE && (new->flags & KVM_MEM_GUEST_MEMFD)) { + r = kvm_gmem_commit_memory_region(kvm, new); + if (r) { + kvm_arch_free_memslot(kvm, new); + kvm_destroy_dirty_bitmap(new); + goto err; + } + } + /* * For DELETE and MOVE, the working slot is now active as the INVALID * version of the old slot. MOVE is particularly special as it reuses @@ -2087,21 +2111,30 @@ int __kvm_set_memory_region(struct kvm *kvm, new->npages = npages; new->flags = mem->flags; new->userspace_addr = mem->userspace_addr; - if (mem->flags & KVM_MEM_GUEST_MEMFD) { - r = kvm_gmem_bind(kvm, new, mem->guest_memfd, mem->guest_memfd_offset); + if (change == KVM_MR_CREATE && (mem->flags & KVM_MEM_GUEST_MEMFD)) { + r = kvm_gmem_prepare_memory_region(kvm, new, mem->guest_memfd, + mem->guest_memfd_offset); if (r) goto out; } r = kvm_set_memslot(kvm, old, new, change); + + /* + * Drop the reference to the gmem file, even on success. The file pins + * KVM, not the other way 'round. Active bindings are invalidated if + * the file is closed before memslots are destroyed. + */ +#ifdef CONFIG_KVM_PRIVATE_MEM + if (change == KVM_MR_CREATE && (mem->flags & KVM_MEM_GUEST_MEMFD)) + fput(new->gmem.file); +#endif + if (r) - goto out_unbind; + goto out; return 0; -out_unbind: - if (mem->flags & KVM_MEM_GUEST_MEMFD) - kvm_gmem_unbind(new); out: kfree(new); return r; diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index 15b2c728f7dd..92ec2bde6532 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -41,8 +41,9 @@ static inline void gfn_to_pfn_cache_invalidate_start(struct kvm *kvm, #ifdef CONFIG_KVM_PRIVATE_MEM void kvm_gmem_init(struct module *module); int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args); -int kvm_gmem_bind(struct kvm *kvm, struct kvm_memory_slot *slot, - unsigned int fd, u64 offset); +int kvm_gmem_prepare_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot, + unsigned int fd, u64 offset); +int kvm_gmem_commit_memory_region(struct kvm *kvm, struct kvm_memory_slot *slot); void kvm_gmem_unbind(struct kvm_memory_slot *slot); #else static inline void kvm_gmem_init(struct module *module) @@ -50,9 +51,16 @@ static inline void kvm_gmem_init(struct module *module) } -static inline int kvm_gmem_bind(struct kvm *kvm, - struct kvm_memory_slot *slot, - unsigned int fd, u64 offset) +static inline int kvm_gmem_prepare_memory_region(struct kvm *kvm, + struct kvm_memory_slot *slot, + unsigned int fd, u64 offset) +{ + WARN_ON_ONCE(1); + return -EIO; +} + +static inline int kvm_gmem_commit_memory_region(struct kvm *kvm, + struct kvm_memory_slot *slot) { WARN_ON_ONCE(1); return -EIO;