* Re: Linux 7.2.7
2026-09-21 13:30 Linux 7.2.7 Greg Kroah-Hartman
@ 2026-09-21 13:30 ` Greg Kroah-Hartman
0 siblings, 0 replies; 2+ messages in thread
From: Greg Kroah-Hartman @ 2026-09-21 13:30 UTC (permalink / raw)
To: linux-kernel, akpm, torvalds, stable; +Cc: lwn, jslaby, Greg Kroah-Hartman
diff --git a/Documentation/ABI/testing/sysfs-fs-erofs b/Documentation/ABI/testing/sysfs-fs-erofs
index e4cf6fc6a106..0b8b4354e40b 100644
--- a/Documentation/ABI/testing/sysfs-fs-erofs
+++ b/Documentation/ABI/testing/sysfs-fs-erofs
@@ -5,7 +5,7 @@ Description: Shows all enabled kernel features.
Supported features:
compr_cfgs, big_pcluster, chunked_file, device_table,
compr_head2, sb_chksum, ztailpacking, dedupe, fragments,
- 48bit, metabox.
+ xattr_prefixes, 48bit, metabox.
What: /sys/fs/erofs/<disk>/sync_decompress
Date: November 2021
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index e8ff36982d97..d0f174288365 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -4922,18 +4922,6 @@ Kernel parameters
Set to non-zero if a chip is present that snoops speed
changes. Disabled by default.
- pata_legacy.ht6560a= [HW,LIBATA]
- Format: <int>
- Set to 1, 2, or 3 for HT 6560A on the primary channel,
- the secondary channel, or both channels respectively.
- Disabled by default.
-
- pata_legacy.ht6560b= [HW,LIBATA]
- Format: <int>
- Set to 1, 2, or 3 for HT 6560B on the primary channel,
- the secondary channel, or both channels respectively.
- Disabled by default.
-
pata_legacy.iordy_mask= [HW,LIBATA]
Format: <int>
IORDY enable mask. Set individual bits to allow IORDY
@@ -4946,18 +4934,6 @@ Kernel parameters
with the sequence. By default IORDY is allowed across
all channels.
- pata_legacy.opti82c46x= [HW,LIBATA]
- Format: <int>
- Set to 1, 2, or 3 for Opti 82c611A on the primary
- channel, the secondary channel, or both channels
- respectively. Disabled by default.
-
- pata_legacy.opti82c611a= [HW,LIBATA]
- Format: <int>
- Set to 1, 2, or 3 for Opti 82c465MV on the primary
- channel, the secondary channel, or both channels
- respectively. Disabled by default.
-
pata_legacy.pio_mask= [HW,LIBATA]
Format: <int>
PIO mode mask for autospeed devices. Set individual
@@ -4981,19 +4957,6 @@ Kernel parameters
the first port in the list above (0x1f0), and so on.
By default all supported ports are probed.
- pata_legacy.qdi= [HW,LIBATA]
- Format: <int>
- Set to non-zero to probe QDI controllers. By default
- set to 1 if CONFIG_PATA_QDI_MODULE, 0 otherwise.
-
- pata_legacy.winbond= [HW,LIBATA]
- Format: <int>
- Set to non-zero to probe Winbond controllers. Use
- the standard I/O port (0x130) if 1, otherwise the
- value given is the I/O port to use (typically 0x1b0).
- By default set to 1 if CONFIG_PATA_WINBOND_VLB_MODULE,
- 0 otherwise.
-
pata_platform.pio_mask= [HW,LIBATA]
Format: <int>
Supported PIO mode mask. Set individual bits to allow
diff --git a/Documentation/hwmon/gpd-fan.rst b/Documentation/hwmon/gpd-fan.rst
index 29527a77fe88..b27657d33056 100644
--- a/Documentation/hwmon/gpd-fan.rst
+++ b/Documentation/hwmon/gpd-fan.rst
@@ -67,7 +67,7 @@ pwm1_enable
at full speed. Write "1" to set to manual, write "2" to let the EC control
decide fan speed. Read this attribute to see current status.
- NB：In consideration of the safety of the device, when setting to manual mode,
+ NB: In consideration of the safety of the device, when setting to manual mode,
the pwm speed will be set to the maximum value (255) by default. You can set
a different value by writing pwm1 later.
diff --git a/Documentation/hwmon/hwmon-kernel-api.rst b/Documentation/hwmon/hwmon-kernel-api.rst
index 9fcde32a140d..c3eb433a78f6 100644
--- a/Documentation/hwmon/hwmon-kernel-api.rst
+++ b/Documentation/hwmon/hwmon-kernel-api.rst
@@ -42,6 +42,9 @@ register/unregister functions::
char *devm_hwmon_sanitize_name(struct device *dev, const char *name);
+ int hwmon_notify_event(struct device *dev, enum hwmon_sensor_types type,
+ u32 attr, int channel);
+
void hwmon_lock(struct device *dev);
void hwmon_unlock(struct device *dev);
@@ -90,6 +93,18 @@ implemented in the driver, or debugfs functions, hwmon_lock() and hwmon_unlock()
can be used to ensure that calls to those functions are serialized. Those
functions also support guard() and scoped_guard() variants.
+Drivers can call hwmon_notify_event() to notify userspace and the thermal
+subsystem when a hardware monitoring event (such as an alarm or a fault
+condition) occurs or clears. The parameters are the hwmon device, the sensor
+type, the attribute identifier associated with the event (such as
+hwmon_temp_max_alarm or hwmon_fan_fault), and the sensor channel number.
+hwmon_notify_event() generates a sysfs event (calling sysfs_notify()) and a
+udev event with the attribute name passed in the NAME environment property
+(e.g., "NAME=temp1_max_alarm"). If the event is for a temperature sensor and
+the sensor is attached to a thermal zone, it also notifies the thermal
+subsystem to update the thermal zone. hwmon_notify_event() returns 0 on
+success or a negative error code on failure.
+
Using devm_hwmon_device_register_with_info()
--------------------------------------------
diff --git a/Documentation/netlink/specs/rt-link.yaml b/Documentation/netlink/specs/rt-link.yaml
index 68c26a70bb64..c75731ecbe1a 100644
--- a/Documentation/netlink/specs/rt-link.yaml
+++ b/Documentation/netlink/specs/rt-link.yaml
@@ -898,6 +898,8 @@ attribute-sets:
-
name: txqlen
type: u32
+ checks:
+ max: 32767
-
name: map
type: binary
diff --git a/Documentation/scheduler/sched-ext.rst b/Documentation/scheduler/sched-ext.rst
index 2771ea4cc14a..96ff66c91ceb 100644
--- a/Documentation/scheduler/sched-ext.rst
+++ b/Documentation/scheduler/sched-ext.rst
@@ -229,7 +229,7 @@ optional. The following modified excerpt is from
void BPF_STRUCT_OPS(simple_exit, struct scx_exit_info *ei)
{
- exit_type = ei->type;
+ exit_type = ei->kind;
}
SEC(".struct_ops")
diff --git a/MAINTAINERS b/MAINTAINERS
index 8014b9f8253e..95cd976a068c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -26241,6 +26241,7 @@ R: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
L: linux-i2c@vger.kernel.org
S: Supported
F: drivers/i2c/busses/i2c-designware-*
+F: include/linux/designware_i2c.h
SYNOPSYS DESIGNWARE I2C DRIVER - AMDISP
M: Nirujogi Pratap <pratap.nirujogi@amd.com>
diff --git a/Makefile b/Makefile
index 73e7bac7527e..c28f11c617a4 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
VERSION = 7
PATCHLEVEL = 2
-SUBLEVEL = 6
+SUBLEVEL = 7
EXTRAVERSION =
NAME = Baby Opossum Posse
diff --git a/arch/arm/mach-ux500/cpu-db8500.c b/arch/arm/mach-ux500/cpu-db8500.c
index b1a70f203372..0d7530fb6ad0 100644
--- a/arch/arm/mach-ux500/cpu-db8500.c
+++ b/arch/arm/mach-ux500/cpu-db8500.c
@@ -12,7 +12,7 @@
#include <linux/irq.h>
#include <linux/irqchip.h>
#include <linux/irqchip/arm-gic.h>
-#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/mfd/db8500-prcmu.h>
#include <linux/platform_data/arm-ux500-pm.h>
#include <linux/platform_device.h>
#include <linux/io.h>
@@ -81,7 +81,7 @@ static void __init ux500_init_irq(void)
struct resource r;
irqchip_init();
- prcmu_early_init();
+ db8500_prcmu_early_init();
np = of_find_compatible_node(NULL, NULL, "stericsson,db8500-prcmu");
of_address_to_resource(np, 0, &r);
of_node_put(np);
@@ -101,7 +101,7 @@ static void ux500_restart(enum reboot_mode mode, const char *cmd)
local_irq_disable();
local_fiq_disable();
- prcmu_system_reset(0);
+ db8500_prcmu_system_reset(0);
}
static const struct of_device_id u8500_local_bus_nodes[] = {
diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
index c5693a32e49b..8f9bc2327dc8 100644
--- a/arch/arm64/kernel/machine_kexec.c
+++ b/arch/arm64/kernel/machine_kexec.c
@@ -129,7 +129,8 @@ int machine_kexec_post_load(struct kimage *kimage)
}
/* Create a copy of the linear map */
- rc = trans_pgd_create_copy(&info, &trans_pgd, PAGE_OFFSET, PAGE_END);
+ rc = trans_pgd_create_copy(&info, &trans_pgd,
+ _PAGE_OFFSET(vabits_actual), PAGE_END);
if (rc)
return rc;
kimage->arch.ttbr1 = __pa(trans_pgd);
diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg
index 7cb61aca3797..03f5e1755a63 100644
--- a/arch/arm64/tools/sysreg
+++ b/arch/arm64/tools/sysreg
@@ -3736,7 +3736,9 @@ Sysreg ICC_CR0_EL1 3 1 12 0 1
Res0 63:39
Field 38 PID
Field 37:32 IPPT
-Res0 31:1
+Res0 31:3
+Field 2 LINK_IDLE
+Field 1 LINK
Field 0 EN
EndSysreg
diff --git a/arch/powerpc/include/asm/entry-common.h b/arch/powerpc/include/asm/entry-common.h
index c5adb5006361..8e91489fdf2b 100644
--- a/arch/powerpc/include/asm/entry-common.h
+++ b/arch/powerpc/include/asm/entry-common.h
@@ -222,8 +222,6 @@ static inline void arch_interrupt_enter_prepare(struct pt_regs *regs)
if (user_mode(regs)) {
kuap_lock();
- account_cpu_user_entry();
- account_stolen_time();
} else {
kuap_save_and_lock(regs);
/*
@@ -270,7 +268,7 @@ static inline void arch_interrupt_exit_prepare(struct pt_regs *regs)
}
/* irqentry_exit expects to be called with interrupts disabled */
- local_irq_disable();
+ hard_irq_disable();
}
static inline void arch_interrupt_async_enter_prepare(struct pt_regs *regs)
@@ -515,8 +513,14 @@ static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs,
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
local_paca->tm_scratch = regs->msr;
#endif
- /* Restore user access locks last */
- kuap_user_restore(regs);
+ /*
+ * Do not restore KUAP here. Generic entry might treat this as the last
+ * arch step before userspace but PowerPC still has kernel work after
+ * irqentry_exit()/syscall_exit_to_user_mode() i.e. in
+ * interrupt_exit_user_prepare() / syscall_exit_prepare() may enable
+ * IRQs and retry. Those functions restore KUAP immediately before rfi,
+ * which is where it should belong.
+ */
}
#define arch_exit_to_user_mode_prepare arch_exit_to_user_mode_prepare
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index 028f69158532..d64cce17a4e0 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -533,9 +533,7 @@ static void eeh_rmv_device(struct eeh_dev *edev, void *userdata)
if (rmv_data)
list_add(&edev->rmv_entry, &rmv_data->removed_vf_list);
} else {
- pci_lock_rescan_remove();
pci_stop_and_remove_bus_device(dev);
- pci_unlock_rescan_remove();
}
}
diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c
index 5b88bf72786c..55f9c0c9922a 100644
--- a/arch/powerpc/kernel/interrupt.c
+++ b/arch/powerpc/kernel/interrupt.c
@@ -175,7 +175,7 @@ notrace unsigned long syscall_exit_restart(unsigned long r3, struct pt_regs *reg
current_thread_info()->exit_flags &= ~_TIF_RESTOREALL;
regs->exit_result |= ret;
- return ret;
+ return regs->exit_result;
}
#endif
diff --git a/arch/powerpc/kexec/file_load_64.c b/arch/powerpc/kexec/file_load_64.c
index 6075b1c88511..d990880b77df 100644
--- a/arch/powerpc/kexec/file_load_64.c
+++ b/arch/powerpc/kexec/file_load_64.c
@@ -57,7 +57,7 @@ int arch_check_excluded_range(struct kimage *image, unsigned long start,
emem = image->arch.exclude_ranges;
for (i = 0; i < emem->nr_ranges; i++)
- if (start < emem->ranges[i].end && end > emem->ranges[i].start)
+ if (start <= emem->ranges[i].end && end >= emem->ranges[i].start)
return 1;
return 0;
@@ -113,7 +113,7 @@ static int add_usable_mem(struct umem_info *um_info, u64 base, u64 end)
loc_end = um_info->ranges[i].end;
if (loc_base >= base && loc_end <= end)
add = true;
- else if (base < loc_end && end > loc_base) {
+ else if (base <= loc_end && end >= loc_base) {
if (loc_base < base)
loc_base = base;
if (loc_end > end)
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index f9380ef65750..dd128b293992 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -6124,12 +6124,12 @@ static int kvmppc_irq_bypass_add_producer_hv(struct irq_bypass_consumer *cons,
struct kvm_kernel_irqfd *irqfd =
container_of(cons, struct kvm_kernel_irqfd, consumer);
- irqfd->producer = prod;
-
ret = kvmppc_set_passthru_irq(irqfd->kvm, prod->irq, irqfd->gsi);
if (ret)
pr_info("kvmppc_set_passthru_irq (irq %d, gsi %d) fails: %d\n",
prod->irq, irqfd->gsi, ret);
+ else
+ irqfd->producer = prod;
return ret;
}
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 32ecbc46e74b..728a5610d167 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -1623,15 +1623,13 @@ int64_t pnv_opal_pci_msi_eoi(struct irq_data *d)
return opal_pci_msi_eoi(phb->opal_id, d->parent_data->hwirq);
}
-static struct irq_chip pnv_pci_msi_irq_chip;
-
/*
* Returns true iff chip is something that we could call
* pnv_opal_pci_msi_eoi for.
*/
bool is_pnv_opal_msi(struct irq_chip *chip)
{
- return chip == &pnv_pci_msi_irq_chip;
+ return chip && chip->name && str_has_prefix(chip->name, "PNV-");
}
EXPORT_SYMBOL_GPL(is_pnv_opal_msi);
@@ -1728,7 +1726,7 @@ static const struct msi_parent_ops pnv_msi_parent_ops = {
.chip_flags = MSI_CHIP_FLAG_SET_EOI,
.bus_select_token = DOMAIN_BUS_NEXUS,
.bus_select_mask = MATCH_PCI_MSI,
- .prefix = "PNV-",
+ .prefix = "PNV-", /* Note: is_pnv_opal_msi() uses this */
.init_dev_msi_info = pnv_init_dev_msi_info,
};
diff --git a/arch/powerpc/platforms/ps3/repository.c b/arch/powerpc/platforms/ps3/repository.c
index b8c030eab138..0cc755ac3e7f 100644
--- a/arch/powerpc/platforms/ps3/repository.c
+++ b/arch/powerpc/platforms/ps3/repository.c
@@ -6,6 +6,8 @@
* Copyright 2006 Sony Corp.
*/
+#include <linux/minmax.h>
+
#include <asm/lv1call.h>
#include "platform.h"
@@ -74,8 +76,9 @@ static void _dump_node(unsigned int lpar_id, u64 n1, u64 n2, u64 n3, u64 n4,
static u64 make_first_field(const char *text, u64 index)
{
u64 n = 0;
+ size_t len = min(strlen(text), sizeof(n));
- memcpy((char *)&n, text, strnlen(text, sizeof(n)));
+ memcpy(&n, text, len);
return PS3_VENDOR_ID_NONE + (n >> 32) + index;
}
diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
index b2df7f72241a..7cc5a6a5c020 100644
--- a/arch/riscv/kernel/process.c
+++ b/arch/riscv/kernel/process.c
@@ -349,10 +349,8 @@ long set_tagged_addr_ctrl(struct task_struct *task, unsigned long arg)
if (arg & PR_TAGGED_ADDR_ENABLE && (tagged_addr_disabled || !pmlen))
return -EINVAL;
- if (!(arg & PR_TAGGED_ADDR_ENABLE)) {
+ if (!(arg & PR_TAGGED_ADDR_ENABLE))
pmlen = PMLEN_0;
- pmm = ENVCFG_PMM_PMLEN_0;
- }
if (mmap_write_lock_killable(mm))
return -EINTR;
diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c
index caf6762427c8..d2bb70fed8b9 100644
--- a/arch/riscv/kernel/sys_hwprobe.c
+++ b/arch/riscv/kernel/sys_hwprobe.c
@@ -295,6 +295,8 @@ static u64 hwprobe_vec_misaligned(const struct cpumask *cpus)
static void hwprobe_one_pair(struct riscv_hwprobe *pair,
const struct cpumask *cpus)
{
+ pair->value = 0;
+
switch (pair->key) {
case RISCV_HWPROBE_KEY_MVENDORID:
case RISCV_HWPROBE_KEY_MARCHID:
@@ -329,17 +331,14 @@ static void hwprobe_one_pair(struct riscv_hwprobe *pair,
break;
case RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE:
- pair->value = 0;
if (hwprobe_ext0_has(cpus, RISCV_HWPROBE_EXT_ZICBOZ))
pair->value = riscv_cboz_block_size;
break;
case RISCV_HWPROBE_KEY_ZICBOM_BLOCK_SIZE:
- pair->value = 0;
if (hwprobe_ext0_has(cpus, RISCV_HWPROBE_EXT_ZICBOM))
pair->value = riscv_cbom_block_size;
break;
case RISCV_HWPROBE_KEY_ZICBOP_BLOCK_SIZE:
- pair->value = 0;
if (hwprobe_ext0_has(cpus, RISCV_HWPROBE_EXT_ZICBOP))
pair->value = riscv_cbop_block_size;
break;
diff --git a/arch/s390/boot/ipl_parm.c b/arch/s390/boot/ipl_parm.c
index 6bc950b92be7..59eabf4a2de0 100644
--- a/arch/s390/boot/ipl_parm.c
+++ b/arch/s390/boot/ipl_parm.c
@@ -23,6 +23,7 @@ struct parmarea parmarea __section(".parmarea") = {
};
char __bootdata(early_command_line)[COMMAND_LINE_SIZE];
+static char command_line_buf[COMMAND_LINE_SIZE];
unsigned int __bootdata_preserved(zlib_dfltcc_support) = ZLIB_DFLTCC_FULL;
struct ipl_parameter_block __bootdata_preserved(ipl_block);
@@ -135,31 +136,29 @@ static size_t ipl_block_get_ascii_scpdata(char *dest, size_t size,
static void append_ipl_block_parm(void)
{
- char *parm, *delim;
- size_t len, rc = 0;
+ size_t len, extra = 0;
+ char *delim;
len = strlen(early_command_line);
-
- delim = early_command_line + len; /* '\0' character position */
- parm = early_command_line + len + 1; /* append right after '\0' */
+ delim = early_command_line + len; /* '\0' character position */
switch (ipl_block.pb0_hdr.pbt) {
case IPL_PBT_CCW:
- rc = ipl_block_get_ascii_vmparm(
- parm, COMMAND_LINE_SIZE - len - 1, &ipl_block);
+ extra = ipl_block_get_ascii_vmparm(command_line_buf, sizeof(command_line_buf), &ipl_block);
break;
case IPL_PBT_FCP:
case IPL_PBT_NVME:
case IPL_PBT_ECKD:
- rc = ipl_block_get_ascii_scpdata(
- parm, COMMAND_LINE_SIZE - len - 1, &ipl_block);
+ extra = ipl_block_get_ascii_scpdata(command_line_buf, sizeof(command_line_buf), &ipl_block);
break;
}
- if (rc) {
- if (*parm == '=')
- memmove(early_command_line, parm + 1, rc);
- else
+ if (extra) {
+ if (command_line_buf[0] == '=') {
+ memmove(early_command_line, command_line_buf + 1, extra);
+ } else if (len < COMMAND_LINE_SIZE - 2) {
*delim = ' '; /* replace '\0' with space */
+ sized_strscpy(delim + 1, command_line_buf, COMMAND_LINE_SIZE - len - 1);
+ }
}
}
@@ -245,7 +244,6 @@ static void modify_fac_list(char *str)
check_cleared_facilities();
}
-static char command_line_buf[COMMAND_LINE_SIZE];
void parse_boot_command_line(void)
{
char *param, *val;
diff --git a/arch/s390/boot/physmem_info.c b/arch/s390/boot/physmem_info.c
index 1f2ca5435838..0ebb2174713f 100644
--- a/arch/s390/boot/physmem_info.c
+++ b/arch/s390/boot/physmem_info.c
@@ -141,7 +141,7 @@ static int tprot(unsigned long addr)
static unsigned long search_mem_end(void)
{
- unsigned long range = 1 << (MAX_PHYSMEM_BITS - 20); /* in 1MB blocks */
+ unsigned long range = 1UL << (MAX_PHYSMEM_BITS - 20); /* in 1MB blocks */
unsigned long offset = 0;
unsigned long pivot;
diff --git a/arch/s390/crypto/aes_s390.c b/arch/s390/crypto/aes_s390.c
index 62edc66d5478..0be6fa779d2c 100644
--- a/arch/s390/crypto/aes_s390.c
+++ b/arch/s390/crypto/aes_s390.c
@@ -26,14 +26,14 @@
#include <linux/module.h>
#include <linux/cpufeature.h>
#include <linux/init.h>
-#include <linux/mutex.h>
#include <linux/fips.h>
+#include <linux/semaphore.h>
#include <linux/string.h>
#include <crypto/xts.h>
#include <asm/cpacf.h>
static u8 *ctrblk;
-static DEFINE_MUTEX(ctrblk_lock);
+static DEFINE_SEMAPHORE(ctrblk_sem, 1);
static cpacf_mask_t km_functions, kmc_functions, kmctr_functions,
kma_functions;
@@ -129,7 +129,7 @@ static int ecb_aes_crypt(struct skcipher_request *req, unsigned long modifier)
return fallback_skcipher_crypt(sctx, req, modifier);
ret = skcipher_walk_virt(&walk, req, false);
- while ((nbytes = walk.nbytes) != 0) {
+ while (!ret && ((nbytes = walk.nbytes) != 0)) {
/* only use complete blocks */
n = nbytes & ~(AES_BLOCK_SIZE - 1);
cpacf_km(sctx->fc | modifier, sctx->key,
@@ -233,7 +233,7 @@ static int cbc_aes_crypt(struct skcipher_request *req, unsigned long modifier)
return ret;
memcpy(param.iv, walk.iv, AES_BLOCK_SIZE);
memcpy(param.key, sctx->key, sctx->key_len);
- while ((nbytes = walk.nbytes) != 0) {
+ while (!ret && ((nbytes = walk.nbytes) != 0)) {
/* only use complete blocks */
n = nbytes & ~(AES_BLOCK_SIZE - 1);
cpacf_kmc(sctx->fc | modifier, ¶m,
@@ -359,7 +359,7 @@ static int xts_aes_crypt(struct skcipher_request *req, unsigned long modifier)
memcpy(xts_param.key + offset, xts_ctx->key, xts_ctx->key_len);
memcpy(xts_param.init, pcc_param.xts, 16);
- while ((nbytes = walk.nbytes) != 0) {
+ while (!ret && ((nbytes = walk.nbytes) != 0)) {
/* only use complete blocks */
n = nbytes & ~(AES_BLOCK_SIZE - 1);
cpacf_km(xts_ctx->fc | modifier, xts_param.key + offset,
@@ -487,7 +487,7 @@ static int fullxts_aes_crypt(struct skcipher_request *req, unsigned long modifi
memcpy(fxts_param.tweak, req->iv, AES_BLOCK_SIZE);
fxts_param.nap[0] = 0x01; /* initial alpha power (1, little-endian) */
- while ((nbytes = walk.nbytes) != 0) {
+ while (!ret && ((nbytes = walk.nbytes) != 0)) {
/* only use complete blocks */
n = nbytes & ~(AES_BLOCK_SIZE - 1);
cpacf_km(xts_ctx->fc | modifier, fxts_param.key + offset,
@@ -562,48 +562,64 @@ static unsigned int __ctrblk_init(u8 *ctrptr, u8 *iv, unsigned int nbytes)
return n;
}
+static int __ctr_aes_crypt(struct s390_aes_ctx *sctx,
+ struct skcipher_walk *walk, bool locked)
+{
+ unsigned int n, nbytes;
+ int ret = 0;
+ u8 *ctrptr;
+
+ while (!ret && ((nbytes = walk->nbytes) >= AES_BLOCK_SIZE)) {
+ n = AES_BLOCK_SIZE;
+ if (nbytes >= 2 * AES_BLOCK_SIZE && locked)
+ n = __ctrblk_init(ctrblk, walk->iv, nbytes);
+ ctrptr = (n > AES_BLOCK_SIZE) ? ctrblk : walk->iv;
+ cpacf_kmctr(sctx->fc, sctx->key, walk->dst.virt.addr,
+ walk->src.virt.addr, n, ctrptr);
+ if (ctrptr == ctrblk)
+ memcpy(walk->iv, ctrptr + n - AES_BLOCK_SIZE,
+ AES_BLOCK_SIZE);
+ crypto_inc(walk->iv, AES_BLOCK_SIZE);
+ ret = skcipher_walk_done(walk, nbytes - n);
+ }
+
+ return ret;
+}
+
static int ctr_aes_crypt(struct skcipher_request *req)
{
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
struct s390_aes_ctx *sctx = crypto_skcipher_ctx(tfm);
- u8 buf[AES_BLOCK_SIZE], *ctrptr;
struct skcipher_walk walk;
- unsigned int n, nbytes;
- int ret, locked;
+ u8 buf[AES_BLOCK_SIZE];
+ int ret;
if (unlikely(!sctx->fc))
return fallback_skcipher_crypt(sctx, req, 0);
- locked = mutex_trylock(&ctrblk_lock);
-
ret = skcipher_walk_virt(&walk, req, false);
- while ((nbytes = walk.nbytes) >= AES_BLOCK_SIZE) {
- n = AES_BLOCK_SIZE;
+ if (ret)
+ return ret;
- if (nbytes >= 2*AES_BLOCK_SIZE && locked)
- n = __ctrblk_init(ctrblk, walk.iv, nbytes);
- ctrptr = (n > AES_BLOCK_SIZE) ? ctrblk : walk.iv;
- cpacf_kmctr(sctx->fc, sctx->key, walk.dst.virt.addr,
- walk.src.virt.addr, n, ctrptr);
- if (ctrptr == ctrblk)
- memcpy(walk.iv, ctrptr + n - AES_BLOCK_SIZE,
- AES_BLOCK_SIZE);
- crypto_inc(walk.iv, AES_BLOCK_SIZE);
- ret = skcipher_walk_done(&walk, nbytes - n);
+ if (down_trylock(&ctrblk_sem) == 0) {
+ ret = __ctr_aes_crypt(sctx, &walk, true);
+ up(&ctrblk_sem);
+ } else {
+ ret = __ctr_aes_crypt(sctx, &walk, false);
}
- if (locked)
- mutex_unlock(&ctrblk_lock);
+
/*
* final block may be < AES_BLOCK_SIZE, copy only nbytes
*/
- if (nbytes) {
+ if (!ret && walk.nbytes > 0) {
memset(buf, 0, AES_BLOCK_SIZE);
- memcpy(buf, walk.src.virt.addr, nbytes);
+ memcpy(buf, walk.src.virt.addr, walk.nbytes);
cpacf_kmctr(sctx->fc, sctx->key, buf, buf,
AES_BLOCK_SIZE, walk.iv);
- memcpy(walk.dst.virt.addr, buf, nbytes);
+ memcpy(walk.dst.virt.addr, buf, walk.nbytes);
crypto_inc(walk.iv, AES_BLOCK_SIZE);
ret = skcipher_walk_done(&walk, 0);
+ memzero_explicit(buf, sizeof(buf));
}
return ret;
@@ -895,10 +911,14 @@ static int gcm_aes_crypt(struct aead_request *req, unsigned int flags)
gw_in.ptr, aad_bytes);
n = aad_bytes + pc_bytes;
- if (gcm_in_walk_done(&gw_in, n) != n)
- return -ENOMEM;
- if (gcm_out_walk_done(&gw_out, n) != n)
- return -ENOMEM;
+ if (gcm_in_walk_done(&gw_in, n) != n) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ if (gcm_out_walk_done(&gw_out, n) != n) {
+ ret = -ENOMEM;
+ goto out;
+ }
aadlen -= aad_bytes;
pclen -= pc_bytes;
} while (aadlen + pclen > 0);
@@ -910,7 +930,10 @@ static int gcm_aes_crypt(struct aead_request *req, unsigned int flags)
} else
scatterwalk_map_and_copy(param.t, req->dst, len, taglen, 1);
+out:
memzero_explicit(¶m, sizeof(param));
+ memzero_explicit(gw_in.buf, sizeof(gw_in.buf));
+ memzero_explicit(gw_out.buf, sizeof(gw_out.buf));
return ret;
}
diff --git a/arch/s390/crypto/paes_s390.c b/arch/s390/crypto/paes_s390.c
index 8cfe6166c193..240c7b87a048 100644
--- a/arch/s390/crypto/paes_s390.c
+++ b/arch/s390/crypto/paes_s390.c
@@ -220,6 +220,10 @@ static inline int convert_key(const u8 *key, unsigned int keylen,
xflags);
}
+ /* But finally map -EBUSY to -EIO to indicate an IO failure */
+ if (rc == -EBUSY)
+ rc = -EIO;
+
out:
pr_debug("rc=%d\n", rc);
return rc;
@@ -568,7 +572,7 @@ static int ecb_paes_do_one_request(struct crypto_engine *engine, void *areq)
atomic_dec(&ctx->via_engine_ctr);
crypto_finalize_skcipher_request(engine, req, rc);
local_bh_enable();
- return rc;
+ return 0;
}
static struct skcipher_engine_alg ecb_paes_alg = {
@@ -576,6 +580,7 @@ static struct skcipher_engine_alg ecb_paes_alg = {
.base.cra_name = "ecb(paes)",
.base.cra_driver_name = "ecb-paes-s390",
.base.cra_priority = 401, /* combo: aes + ecb + 1 */
+ .base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NO_FALLBACK,
.base.cra_blocksize = AES_BLOCK_SIZE,
.base.cra_ctxsize = sizeof(struct s390_paes_ctx),
.base.cra_module = THIS_MODULE,
@@ -834,7 +839,7 @@ static int cbc_paes_do_one_request(struct crypto_engine *engine, void *areq)
atomic_dec(&ctx->via_engine_ctr);
crypto_finalize_skcipher_request(engine, req, rc);
local_bh_enable();
- return rc;
+ return 0;
}
static struct skcipher_engine_alg cbc_paes_alg = {
@@ -842,6 +847,7 @@ static struct skcipher_engine_alg cbc_paes_alg = {
.base.cra_name = "cbc(paes)",
.base.cra_driver_name = "cbc-paes-s390",
.base.cra_priority = 402, /* cbc-paes-s390 + 1 */
+ .base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NO_FALLBACK,
.base.cra_blocksize = AES_BLOCK_SIZE,
.base.cra_ctxsize = sizeof(struct s390_paes_ctx),
.base.cra_module = THIS_MODULE,
@@ -1034,6 +1040,7 @@ static int ctr_paes_do_crypt(struct s390_paes_ctx *ctx,
}
out:
+ memzero_explicit(buf, sizeof(buf));
pr_debug("rc=%d\n", rc);
return rc;
}
@@ -1142,7 +1149,7 @@ static int ctr_paes_do_one_request(struct crypto_engine *engine, void *areq)
atomic_dec(&ctx->via_engine_ctr);
crypto_finalize_skcipher_request(engine, req, rc);
local_bh_enable();
- return rc;
+ return 0;
}
static struct skcipher_engine_alg ctr_paes_alg = {
@@ -1150,6 +1157,7 @@ static struct skcipher_engine_alg ctr_paes_alg = {
.base.cra_name = "ctr(paes)",
.base.cra_driver_name = "ctr-paes-s390",
.base.cra_priority = 402, /* ecb-paes-s390 + 1 */
+ .base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NO_FALLBACK,
.base.cra_blocksize = 1,
.base.cra_ctxsize = sizeof(struct s390_paes_ctx),
.base.cra_module = THIS_MODULE,
@@ -1364,7 +1372,7 @@ static inline int __xts_2keys_prep_param(struct s390_pxts_ctx *ctx,
memcpy(param->init, pcc_param.xts, 16);
}
- memzero_explicit(pcc_param.key, sizeof(pcc_param.key));
+ memzero_explicit(&pcc_param, sizeof(pcc_param));
return rc;
}
@@ -1585,7 +1593,7 @@ static int xts_paes_do_one_request(struct crypto_engine *engine, void *areq)
atomic_dec(&ctx->via_engine_ctr);
crypto_finalize_skcipher_request(engine, req, rc);
local_bh_enable();
- return rc;
+ return 0;
}
static struct skcipher_engine_alg xts_paes_alg = {
@@ -1593,6 +1601,7 @@ static struct skcipher_engine_alg xts_paes_alg = {
.base.cra_name = "xts(paes)",
.base.cra_driver_name = "xts-paes-s390",
.base.cra_priority = 402, /* ecb-paes-s390 + 1 */
+ .base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NO_FALLBACK,
.base.cra_blocksize = AES_BLOCK_SIZE,
.base.cra_ctxsize = sizeof(struct s390_pxts_ctx),
.base.cra_module = THIS_MODULE,
diff --git a/arch/s390/crypto/phmac_s390.c b/arch/s390/crypto/phmac_s390.c
index 03ca33ffe6cc..44817c3d25cf 100644
--- a/arch/s390/crypto/phmac_s390.c
+++ b/arch/s390/crypto/phmac_s390.c
@@ -62,8 +62,10 @@ static inline int hwh_prepare(struct ahash_request *req,
*/
static inline int hwh_advance(struct hash_walk_helper *hwh, int n)
{
- if (n < 0)
+ if (n < 0) {
+ hwh->walkbytes = n;
return crypto_hash_walk_done(&hwh->walk, n);
+ }
hwh->walkbytes -= n;
hwh->walkaddr += n;
@@ -339,6 +341,10 @@ static inline int convert_key(const u8 *key, unsigned int keylen,
xflags);
}
+ /* But finally map -EBUSY to -EIO to indicate an IO failure */
+ if (rc == -EBUSY)
+ rc = -EIO;
+
out:
pr_debug("rc=%d\n", rc);
return rc;
@@ -606,6 +612,7 @@ static int phmac_update(struct ahash_request *req)
struct phmac_tfm_ctx *tfm_ctx = crypto_ahash_ctx(tfm);
struct kmac_sha2_ctx *kmac_ctx = &req_ctx->kmac_ctx;
struct hash_walk_helper *hwh = &req_ctx->hwh;
+ bool cleanup = true;
int rc;
/* prep the walk in the request context */
@@ -629,12 +636,15 @@ static int phmac_update(struct ahash_request *req)
req_ctx->async_op = OP_UPDATE;
atomic_inc(&tfm_ctx->via_engine_ctr);
rc = crypto_transfer_hash_request_to_engine(phmac_crypto_engine, req);
- if (rc != -EINPROGRESS)
+ if (rc == -EINPROGRESS || rc == -EBUSY)
+ cleanup = false;
+ else
atomic_dec(&tfm_ctx->via_engine_ctr);
}
- if (rc != -EINPROGRESS) {
- hwh_advance(hwh, rc);
+ if (cleanup) {
+ if (hwh->walkbytes > 0)
+ hwh_advance(hwh, rc);
memzero_explicit(kmac_ctx, sizeof(*kmac_ctx));
}
@@ -649,6 +659,7 @@ static int phmac_final(struct ahash_request *req)
struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
struct phmac_tfm_ctx *tfm_ctx = crypto_ahash_ctx(tfm);
struct kmac_sha2_ctx *kmac_ctx = &req_ctx->kmac_ctx;
+ bool cleanup = true;
int rc = 0;
/* Try synchronous operation if no active engine usage */
@@ -667,12 +678,14 @@ static int phmac_final(struct ahash_request *req)
req_ctx->async_op = OP_FINAL;
atomic_inc(&tfm_ctx->via_engine_ctr);
rc = crypto_transfer_hash_request_to_engine(phmac_crypto_engine, req);
- if (rc != -EINPROGRESS)
+ if (rc == -EINPROGRESS || rc == -EBUSY)
+ cleanup = false;
+ else
atomic_dec(&tfm_ctx->via_engine_ctr);
}
out:
- if (rc != -EINPROGRESS)
+ if (cleanup)
memzero_explicit(kmac_ctx, sizeof(*kmac_ctx));
pr_debug("rc=%d\n", rc);
return rc;
@@ -685,6 +698,7 @@ static int phmac_finup(struct ahash_request *req)
struct phmac_tfm_ctx *tfm_ctx = crypto_ahash_ctx(tfm);
struct kmac_sha2_ctx *kmac_ctx = &req_ctx->kmac_ctx;
struct hash_walk_helper *hwh = &req_ctx->hwh;
+ bool cleanup = true;
int rc;
/* prep the walk in the request context */
@@ -716,15 +730,17 @@ static int phmac_finup(struct ahash_request *req)
/* req->async_op has been set to either OP_FINUP or OP_FINAL */
atomic_inc(&tfm_ctx->via_engine_ctr);
rc = crypto_transfer_hash_request_to_engine(phmac_crypto_engine, req);
- if (rc != -EINPROGRESS)
+ if (rc == -EINPROGRESS || rc == -EBUSY)
+ cleanup = false;
+ else
atomic_dec(&tfm_ctx->via_engine_ctr);
}
- if (rc != -EINPROGRESS)
+ if (cleanup && hwh->walkbytes > 0)
hwh_advance(hwh, rc);
out:
- if (rc != -EINPROGRESS)
+ if (cleanup)
memzero_explicit(kmac_ctx, sizeof(*kmac_ctx));
pr_debug("rc=%d\n", rc);
return rc;
@@ -933,7 +949,7 @@ static int phmac_do_one_request(struct crypto_engine *engine, void *areq)
atomic_dec(&tfm_ctx->via_engine_ctr);
crypto_finalize_hash_request(engine, req, rc);
local_bh_enable();
- return rc;
+ return 0;
}
#define S390_ASYNC_PHMAC_ALG(x) \
diff --git a/arch/s390/include/asm/pai.h b/arch/s390/include/asm/pai.h
index 534d0320e2aa..a3456a36aaa7 100644
--- a/arch/s390/include/asm/pai.h
+++ b/arch/s390/include/asm/pai.h
@@ -76,7 +76,6 @@ static __always_inline void pai_kernel_exit(struct pt_regs *regs)
}
#define PAI_SAVE_AREA(x) ((x)->hw.event_base)
-#define PAI_CPU_MASK(x) ((x)->hw.addr_filters)
#define PAI_PMU_IDX(x) ((x)->hw.last_tag)
#define PAI_SWLIST(x) (&(x)->hw.tp_list)
diff --git a/arch/s390/kernel/debug.c b/arch/s390/kernel/debug.c
index 14d2b58ad093..e06abf1dbc21 100644
--- a/arch/s390/kernel/debug.c
+++ b/arch/s390/kernel/debug.c
@@ -1074,9 +1074,6 @@ static void _debug_set_level(debug_info_t *id, int new_level)
{
unsigned long flags;
- if (!id)
- return;
-
if (new_level == DEBUG_OFF_LEVEL) {
pr_info("%s: switched off\n", id->name);
} else if ((new_level > DEBUG_MAX_LEVEL) || (new_level < 0)) {
@@ -1101,6 +1098,9 @@ static void _debug_set_level(debug_info_t *id, int new_level)
*/
void debug_set_level(debug_info_t *id, int new_level)
{
+ if (!id)
+ return;
+
/* Level specified via kernel parameter takes precedence */
debug_get_param(id->name, &new_level, NULL);
diff --git a/arch/s390/kernel/diag/diag324.c b/arch/s390/kernel/diag/diag324.c
index fe325c2a2d0d..3eec0cc8fb9e 100644
--- a/arch/s390/kernel/diag/diag324.c
+++ b/arch/s390/kernel/diag/diag324.c
@@ -182,8 +182,7 @@ long diag324_pibbuf(unsigned long arg)
goto out;
rc = copy_to_user((void __user *)address, data->pib, data->pib->len);
rc |= put_user(data->sequence, &udata->sequence);
- if (rc)
- rc = -EFAULT;
+ rc = rc ? -EFAULT : data->rc;
out:
mutex_unlock(&pibmutex);
return rc;
diff --git a/arch/s390/kernel/ipl.c b/arch/s390/kernel/ipl.c
index 3c346b02ceb9..68fdd5616dfe 100644
--- a/arch/s390/kernel/ipl.c
+++ b/arch/s390/kernel/ipl.c
@@ -1157,6 +1157,8 @@ static struct attribute_group reipl_nss_attr_group = {
void set_os_info_reipl_block(void)
{
+ if (!reipl_block_actual)
+ return;
os_info_entry_add_data(OS_INFO_REIPL_BLOCK, reipl_block_actual,
reipl_block_actual->hdr.len);
}
@@ -1927,7 +1929,8 @@ static struct shutdown_action __refdata dump_action = {
static void dump_reipl_run(struct shutdown_trigger *trigger)
{
struct lowcore *abs_lc;
- unsigned int csum;
+ unsigned long ipib = 0;
+ unsigned int csum = 0;
/*
* Set REIPL_CLEAR flag in os_info flags entry indicating
@@ -1943,9 +1946,12 @@ static void dump_reipl_run(struct shutdown_trigger *trigger)
reipl_type == IPL_TYPE_UNKNOWN)
os_info_flags |= OS_INFO_FLAG_REIPL_CLEAR;
os_info_entry_add_data(OS_INFO_FLAGS_ENTRY, &os_info_flags, sizeof(os_info_flags));
- csum = (__force unsigned int)cksm(reipl_block_actual, reipl_block_actual->hdr.len, 0);
+ if (reipl_block_actual) {
+ ipib = __pa(reipl_block_actual);
+ csum = (__force unsigned int)cksm(reipl_block_actual, reipl_block_actual->hdr.len, 0);
+ }
abs_lc = get_abs_lowcore();
- abs_lc->ipib = __pa(reipl_block_actual);
+ abs_lc->ipib = ipib;
abs_lc->ipib_checksum = csum;
put_abs_lowcore(abs_lc);
dump_run(trigger);
diff --git a/arch/s390/kernel/perf_pai.c b/arch/s390/kernel/perf_pai.c
index cdb8006220ca..25228b46cdf4 100644
--- a/arch/s390/kernel/perf_pai.c
+++ b/arch/s390/kernel/perf_pai.c
@@ -67,6 +67,7 @@ struct pai_mapptr {
static struct pai_root { /* Anchor to per CPU data */
refcount_t refcnt; /* Overall active events */
+ atomic_t tskctx; /* Overall per-task events */
struct pai_mapptr __percpu *mapptr;
} pai_root[PAI_PMU_MAX];
@@ -93,14 +94,15 @@ struct pai_pmu { /* Define PAI PMU characteristics */
static struct pai_pmu pai_pmu[]; /* Forward declaration */
/* Free per CPU data when the last event is removed. */
-static void pai_root_free(int idx)
+static void pai_root_free(int idx, int tasks)
{
- if (refcount_dec_and_test(&pai_root[idx].refcnt)) {
+ if (refcount_sub_and_test(tasks, &pai_root[idx].refcnt)) {
free_percpu(pai_root[idx].mapptr);
pai_root[idx].mapptr = NULL;
}
- debug_sprintf_event(paidbg, 5, "%s root[%d].refcount %d\n", __func__,
- idx, refcount_read(&pai_root[idx].refcnt));
+ debug_sprintf_event(paidbg, 5, "%s root[%d].refcount %d tskctx %d\n",
+ __func__, idx, refcount_read(&pai_root[idx].refcnt),
+ atomic_read(&pai_root[idx].tskctx));
}
/*
@@ -137,40 +139,54 @@ static void pai_free(struct pai_mapptr *mp)
mp->mapptr = NULL;
}
-/* Adjust usage counters and remove allocated memory when all users are
- * gone.
- */
-static void pai_event_destroy_cpu(struct perf_event *event, int cpu)
+/* Called under mutex_lock */
+static void pai_event_destroy_cpu(int idx, int cpu, bool hotplug)
{
- int idx = PAI_PMU_IDX(event);
- struct pai_mapptr *mp = per_cpu_ptr(pai_root[idx].mapptr, cpu);
- struct pai_map *cpump = mp->mapptr;
+ struct pai_mapptr *mp;
+ struct pai_map *cpump;
+ int tasks = 1;
- mutex_lock(&pai_reserve_mutex);
- debug_sprintf_event(paidbg, 5, "%s event %#llx idx %d cpu %d users %d "
- "refcnt %u\n", __func__, event->attr.config, idx,
- event->cpu, cpump->active_events,
- refcount_read(&cpump->refcnt));
- if (refcount_dec_and_test(&cpump->refcnt))
+ /* Check reference count and return when all gone.
+ * 1. An event is installed on online CPU X.
+ * 2. CPU x is offlined and the per-CPU data is removed.
+ * 3. Event is destroyed via close system call.
+ */
+ if (!refcount_read(&pai_root[idx].refcnt))
+ return; /* No events at all */
+ mp = per_cpu_ptr(pai_root[idx].mapptr, cpu);
+ if (!mp || !mp->mapptr) /* No events on that CPU */
+ return;
+
+ /* When hotplug is true, invocation is from CPU hotplug callback.
+ * Delete per-CPU resource and adjust refcnt when per-task events
+ * are currently active. This can be more than one.
+ * In this case adjust counters.
+ */
+ if (hotplug)
+ tasks = atomic_read(&pai_root[idx].tskctx);
+
+ cpump = mp->mapptr;
+ if (refcount_sub_and_test(tasks, &cpump->refcnt))
pai_free(mp);
- pai_root_free(idx);
- mutex_unlock(&pai_reserve_mutex);
+ pai_root_free(idx, tasks);
}
static void pai_event_destroy(struct perf_event *event)
{
- int cpu;
+ int cpu = 0, idx = PAI_PMU_IDX(event);
free_page(PAI_SAVE_AREA(event));
+ cpus_read_lock();
+ mutex_lock(&pai_reserve_mutex);
if (event->cpu == -1) {
- struct cpumask *mask = PAI_CPU_MASK(event);
-
- for_each_cpu(cpu, mask)
- pai_event_destroy_cpu(event, cpu);
- kfree(mask);
+ atomic_dec(&pai_root[idx].tskctx);
+ for_each_online_cpu(cpu)
+ pai_event_destroy_cpu(idx, cpu, false);
} else {
- pai_event_destroy_cpu(event, event->cpu);
+ pai_event_destroy_cpu(idx, event->cpu, false);
}
+ mutex_unlock(&pai_reserve_mutex);
+ cpus_read_unlock();
}
static void paicrypt_event_destroy(struct perf_event *event)
@@ -234,25 +250,30 @@ static u64 paicrypt_getall(struct perf_event *event)
return sum;
}
-/* Check concurrent access of counting and sampling for crypto events.
- * This function is called in process context and it is save to block.
- * When the event initialization functions fails, no other call back will
- * be invoked.
- *
- * Allocate the memory for the event.
- */
-static int pai_alloc_cpu(struct perf_event *event, int cpu)
+/* Called under mutex_lock */
+static int pai_alloc_cpu(int idx, int cpu, bool hotplug)
{
- int rc, idx = PAI_PMU_IDX(event);
struct pai_map *cpump = NULL;
bool need_paiext_cb = false;
struct pai_mapptr *mp;
+ int tasks = 1, rc = 0;
+
+ /* When hotplug is true, invocation is from CPU hotplug callback.
+ * Allocate per-CPU resource when per-task events are currently active.
+ * This can be more than one. In this case adjust all reference
+ * counters. Otherwise return, this ensures memory is only allocated
+ * when needed.
+ */
+ if (hotplug) {
+ tasks = atomic_read(&pai_root[idx].tskctx);
+ if (!tasks)
+ goto out;
+ }
- mutex_lock(&pai_reserve_mutex);
/* Allocate root node */
rc = pai_root_alloc(idx);
if (rc)
- goto unlock;
+ goto out;
/* Allocate node for this event */
mp = per_cpu_ptr(pai_root[idx].mapptr, cpu);
@@ -296,28 +317,45 @@ static int pai_alloc_cpu(struct perf_event *event, int cpu)
goto undo;
}
INIT_LIST_HEAD(&cpump->syswide_list);
- refcount_set(&cpump->refcnt, 1);
+ refcount_set(&cpump->refcnt, tasks);
rc = 0;
} else {
- refcount_inc(&cpump->refcnt);
+ refcount_add(tasks, &cpump->refcnt);
}
+ /* If tasks is greater than 1, we are called from CPU hotplug path
+ * and need to adjust the pai_root[idx].refcnt by the number of
+ * per-process events. Function pai_root_alloc(idx) already
+ * incremented by one. Adjust for the rest.
+ */
+ if (tasks > 1)
+ refcount_add(tasks - 1, &pai_root[idx].refcnt);
undo:
if (rc) {
/* Error in allocation of event, decrement anchor. Since
* the event in not created, its destroy() function is never
* invoked. Adjust the reference counter for the anchor.
+ * The failure happened in the case of variable
+ * cpump == NULL branch above. The pai_root[XXX].refcnt has
+ * been incremented by one. Then the per-CPU allocation
+ * failed, so decrement it by one, regardless of tasks.
*/
- pai_root_free(idx);
+ pai_root_free(idx, 1);
}
-unlock:
- mutex_unlock(&pai_reserve_mutex);
+out:
/* If rc is non-zero, no increment of counter/sampler was done. */
return rc;
}
+/* Check concurrent access of counting and sampling for PAI events.
+ * This function is called in process context and it is safe to block.
+ * When the event initialization functions fails, no other call back will
+ * be invoked.
+ * Called under mutex_lock.
+ */
static int pai_alloc(struct perf_event *event)
{
+ int idx = PAI_PMU_IDX(event);
struct cpumask *maskptr;
int cpu, rc = -ENOMEM;
@@ -326,24 +364,20 @@ static int pai_alloc(struct perf_event *event)
goto out;
for_each_online_cpu(cpu) {
- rc = pai_alloc_cpu(event, cpu);
+ rc = pai_alloc_cpu(idx, cpu, false);
if (rc) {
for_each_cpu(cpu, maskptr)
- pai_event_destroy_cpu(event, cpu);
- kfree(maskptr);
- goto out;
+ pai_event_destroy_cpu(idx, cpu, false);
+ goto undo;
}
cpumask_set_cpu(cpu, maskptr);
}
- /*
- * On error all cpumask are freed and all events have been destroyed.
- * Save of which CPUs data structures have been allocated for.
- * Release them in pai_event_destroy call back function
- * for this event.
- */
- PAI_CPU_MASK(event) = maskptr;
rc = 0;
+ /* Trace per-task events for CPU hotplug. */
+ atomic_inc(&pai_root[idx].tskctx);
+undo:
+ kfree(maskptr);
out:
return rc;
}
@@ -391,10 +425,14 @@ static int pai_event_init(struct perf_event *event, int idx)
}
}
+ cpus_read_lock();
+ mutex_lock(&pai_reserve_mutex);
if (event->cpu >= 0)
- rc = pai_alloc_cpu(event, event->cpu);
+ rc = pai_alloc_cpu(idx, event->cpu, false);
else
rc = pai_alloc(event);
+ mutex_unlock(&pai_reserve_mutex);
+ cpus_read_unlock();
if (rc) {
free_page(PAI_SAVE_AREA(event));
goto out;
@@ -1218,23 +1256,61 @@ static int __init paipmu_setup(void)
return install_ok;
}
+static int pai_online_cpu(unsigned int cpu)
+{
+ int rc;
+
+ mutex_lock(&pai_reserve_mutex);
+ rc = pai_alloc_cpu(PAI_PMU_CRYPTO, cpu, true);
+ if (rc)
+ goto out;
+ rc = pai_alloc_cpu(PAI_PMU_EXT, cpu, true);
+ if (rc)
+ pai_event_destroy_cpu(PAI_PMU_CRYPTO, cpu, true);
+out:
+ mutex_unlock(&pai_reserve_mutex);
+ return rc;
+}
+
+static int pai_offline_cpu(unsigned int cpu)
+{
+ mutex_lock(&pai_reserve_mutex);
+ pai_event_destroy_cpu(PAI_PMU_CRYPTO, cpu, true);
+ pai_event_destroy_cpu(PAI_PMU_EXT, cpu, true);
+ mutex_unlock(&pai_reserve_mutex);
+ return 0;
+}
+
static int __init pai_init(void)
{
+ int state, rc;
+
/* Setup s390dbf facility */
- paidbg = debug_register("pai", 32, 256, 128);
+ paidbg = debug_register("pai", 1, 1, 128);
if (!paidbg) {
pr_err("Registration of s390dbf pai failed\n");
return -ENOMEM;
}
debug_register_view(paidbg, &debug_sprintf_view);
- if (!paipmu_setup()) {
- /* No PMU registration, no need for debug buffer */
- debug_unregister_view(paidbg, &debug_sprintf_view);
- debug_unregister(paidbg);
- return -ENODEV;
- }
+ /* CPUHP_BP_PREPARE_DYN --> before CPU is brought online */
+ state = cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "perf/pai:prepare",
+ pai_online_cpu, pai_offline_cpu);
+ rc = state < 0 ? state : 0;
+ if (rc < 0)
+ goto out_debug;
+
+ rc = -ENODEV;
+ if (!paipmu_setup())
+ goto out_cpuhp;
return 0;
+
+out_cpuhp:
+ cpuhp_remove_state(state);
+out_debug:
+ debug_unregister_view(paidbg, &debug_sprintf_view);
+ debug_unregister(paidbg);
+ return rc;
}
device_initcall(pai_init);
diff --git a/arch/s390/kernel/vtime.c b/arch/s390/kernel/vtime.c
index d804e1140c2e..efcbf406f03e 100644
--- a/arch/s390/kernel/vtime.c
+++ b/arch/s390/kernel/vtime.c
@@ -32,7 +32,7 @@ static atomic64_t virt_timer_elapsed;
DEFINE_PER_CPU(u64, mt_cycles[8]);
static DEFINE_PER_CPU(u64, mt_scaling_mult) = { 1 };
static DEFINE_PER_CPU(u64, mt_scaling_div) = { 1 };
-static DEFINE_PER_CPU(u64, mt_scaling_jiffies);
+static DEFINE_PER_CPU(unsigned long, mt_scaling_jiffies);
static inline void set_vtimer(u64 expires)
{
@@ -81,7 +81,7 @@ static void update_mt_scaling(void)
memcpy(cycles_old, cycles_new,
sizeof(u64) * (smp_cpu_mtid + 1));
}
- __this_cpu_write(mt_scaling_jiffies, jiffies_64);
+ __this_cpu_write(mt_scaling_jiffies, jiffies);
}
static inline u64 update_tsk_timer(unsigned long *tsk_vtime, u64 new)
@@ -144,7 +144,7 @@ static int do_account_vtime(struct task_struct *tsk)
lc->system_timer += timer;
/* Update MT utilization calculation */
- if (smp_cpu_mtid && time_after64(jiffies_64, __this_cpu_read(mt_scaling_jiffies)))
+ if (smp_cpu_mtid && time_after(jiffies, __this_cpu_read(mt_scaling_jiffies)))
update_mt_scaling();
/* Calculate cputime delta */
diff --git a/arch/x86/crypto/aria-aesni-avx2-asm_64.S b/arch/x86/crypto/aria-aesni-avx2-asm_64.S
index ed53d4f46bd7..fda8cb8a99a8 100644
--- a/arch/x86/crypto/aria-aesni-avx2-asm_64.S
+++ b/arch/x86/crypto/aria-aesni-avx2-asm_64.S
@@ -982,6 +982,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_encrypt_32way)
%ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14,
%ymm15, %rax);
+ vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_aesni_avx2_encrypt_32way)
@@ -1007,6 +1008,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_decrypt_32way)
%ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14,
%ymm15, %rax);
+ vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_aesni_avx2_decrypt_32way)
@@ -1209,6 +1211,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_ctr_crypt_32way)
%ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14,
%ymm15, %r10);
+ vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_aesni_avx2_ctr_crypt_32way)
@@ -1359,6 +1362,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_gfni_encrypt_32way)
%ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14,
%ymm15, %rax);
+ vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_aesni_avx2_gfni_encrypt_32way)
@@ -1384,6 +1388,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_gfni_decrypt_32way)
%ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14,
%ymm15, %rax);
+ vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_aesni_avx2_gfni_decrypt_32way)
@@ -1428,6 +1433,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_gfni_ctr_crypt_32way)
%ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14,
%ymm15, %r10);
+ vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_aesni_avx2_gfni_ctr_crypt_32way)
diff --git a/arch/x86/crypto/aria-gfni-avx512-asm_64.S b/arch/x86/crypto/aria-gfni-avx512-asm_64.S
index 860887e5d02e..ca83eb126e06 100644
--- a/arch/x86/crypto/aria-gfni-avx512-asm_64.S
+++ b/arch/x86/crypto/aria-gfni-avx512-asm_64.S
@@ -800,6 +800,7 @@ SYM_TYPED_FUNC_START(aria_gfni_avx512_encrypt_64way)
%zmm9, %zmm8, %zmm11, %zmm10, %zmm12, %zmm13, %zmm14,
%zmm15, %rax);
+ vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_gfni_avx512_encrypt_64way)
@@ -825,6 +826,7 @@ SYM_TYPED_FUNC_START(aria_gfni_avx512_decrypt_64way)
%zmm9, %zmm8, %zmm11, %zmm10, %zmm12, %zmm13, %zmm14,
%zmm15, %rax);
+ vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_gfni_avx512_decrypt_64way)
@@ -966,6 +968,7 @@ SYM_TYPED_FUNC_START(aria_gfni_avx512_ctr_crypt_64way)
%zmm9, %zmm8, %zmm11, %zmm10, %zmm12, %zmm13, %zmm14,
%zmm15, %r10);
+ vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_gfni_avx512_ctr_crypt_64way)
diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 5116b15438a2..24c1f4059544 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -3125,6 +3125,27 @@ static void intel_pmu_del_event(struct perf_event *event)
this_cpu_ptr(&cpu_hw_events)->n_late_setup--;
}
+int __intel_pmu_quiesce(void)
+{
+ struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
+ int pmu_enabled = cpuc->enabled;
+
+ cpuc->enabled = 0;
+ if (pmu_enabled)
+ intel_pmu_disable_all();
+
+ return pmu_enabled;
+}
+
+void __intel_pmu_resume(int pmu_enabled)
+{
+ struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
+
+ cpuc->enabled = pmu_enabled;
+ if (pmu_enabled)
+ intel_pmu_enable_all(0);
+}
+
static int icl_set_topdown_event_period(struct perf_event *event)
{
struct hw_perf_event *hwc = &event->hw;
@@ -3316,16 +3337,13 @@ static void intel_pmu_read_event(struct perf_event *event)
if (event->hw.flags & (PERF_X86_EVENT_AUTO_RELOAD | PERF_X86_EVENT_TOPDOWN) ||
is_pebs_counter_event_group(event)) {
struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
- bool pmu_enabled = cpuc->enabled;
+ int pmu_enabled;
/* Only need to call update_topdown_event() once for group read. */
if (is_metric_event(event) && (cpuc->txn_flags & PERF_PMU_TXN_READ))
return;
- cpuc->enabled = 0;
- if (pmu_enabled)
- intel_pmu_disable_all();
-
+ pmu_enabled = __intel_pmu_quiesce();
/*
* If the PEBS counters snapshotting is enabled,
* the topdown event is available in PEBS records.
@@ -3334,10 +3352,7 @@ static void intel_pmu_read_event(struct perf_event *event)
static_call(intel_pmu_update_topdown_event)(event, NULL);
else
intel_pmu_drain_pebs_buffer();
-
- cpuc->enabled = pmu_enabled;
- if (pmu_enabled)
- intel_pmu_enable_all(0);
+ __intel_pmu_resume(pmu_enabled);
return;
}
diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c
index 91a093d8cf2e..3dcd7dfc92b1 100644
--- a/arch/x86/events/intel/ds.c
+++ b/arch/x86/events/intel/ds.c
@@ -1242,8 +1242,11 @@ int intel_pmu_drain_bts_buffer(void)
void intel_pmu_drain_pebs_buffer(void)
{
+ struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
struct perf_sample_data data;
+ WARN_ON_ONCE(cpuc->enabled);
+
static_call(x86_pmu_drain_pebs)(NULL, &data);
}
@@ -1877,8 +1880,11 @@ static void intel_pmu_pebs_via_pt_enable(struct perf_event *event)
static inline void intel_pmu_drain_large_pebs(struct cpu_hw_events *cpuc)
{
if (cpuc->n_pebs == cpuc->n_large_pebs &&
- cpuc->n_pebs != cpuc->n_pebs_via_pt)
+ cpuc->n_pebs != cpuc->n_pebs_via_pt) {
+ int enabled = __intel_pmu_quiesce();
intel_pmu_drain_pebs_buffer();
+ __intel_pmu_resume(enabled);
+ }
}
static void __intel_pmu_pebs_enable(struct perf_event *event)
@@ -2445,7 +2451,7 @@ static inline void __setup_pebs_basic_group(struct perf_event *event,
{
/* The ip in basic is EventingIP */
set_linear_ip(regs, ip);
- regs->flags = PERF_EFLAGS_EXACT;
+ regs->flags |= PERF_EFLAGS_EXACT;
setup_pebs_time(event, data, tsc);
if (sample_type & PERF_SAMPLE_WEIGHT_STRUCT)
@@ -2457,9 +2463,17 @@ static inline void __setup_pebs_gpr_group(struct perf_event *event,
struct pebs_gprs *gprs,
u64 sample_type)
{
+ /*
+ * Update flags with PEBS data. PERF_EFLAGS_EXACT must be set
+ * in previous basic group handling.
+ */
+ regs->flags = gprs->flags | PERF_EFLAGS_EXACT;
+
if (event->attr.precise_ip < 2) {
set_linear_ip(regs, gprs->ip);
regs->flags &= ~PERF_EFLAGS_EXACT;
+ } else if (regs->flags & X86_VM_MASK) {
+ regs->flags ^= (PERF_EFLAGS_VM | X86_VM_MASK);
}
if (sample_type & (PERF_SAMPLE_REGS_INTR | PERF_SAMPLE_REGS_USER))
diff --git a/arch/x86/events/perf_event.h b/arch/x86/events/perf_event.h
index a8afea8d38f0..680220d311a7 100644
--- a/arch/x86/events/perf_event.h
+++ b/arch/x86/events/perf_event.h
@@ -1644,6 +1644,9 @@ static __always_inline void __intel_pmu_lbr_disable(void)
wrmsrq(MSR_IA32_DEBUGCTLMSR, debugctl);
}
+extern int __intel_pmu_quiesce(void);
+extern void __intel_pmu_resume(int pmu_enabled);
+
int intel_pmu_save_and_restart(struct perf_event *event);
struct event_constraint *
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index ac295ca6c92f..b44d24c6eb63 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -806,7 +806,7 @@ static inline pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot)
pmdval_t val = pmd_val(pmd), oldval = val;
pmd_t pmd_result;
- val &= (_HPAGE_CHG_MASK & ~_PAGE_DIRTY);
+ val &= _HPAGE_CHG_MASK;
val |= check_pgprot(newprot) & ~_HPAGE_CHG_MASK;
val = flip_protnone_guard(oldval, val, PHYSICAL_PMD_PAGE_MASK);
diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index 62936a3bde19..b0b576ab2fa6 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -6,6 +6,9 @@
#include <linux/vmalloc.h>
#include <linux/memory.h>
#include <linux/execmem.h>
+#include <linux/cleanup.h>
+#include <linux/kgdb.h>
+#include <linux/mmap_lock.h>
#include <asm/text-patching.h>
#include <asm/insn.h>
@@ -1207,6 +1210,41 @@ static bool cfi_debug __ro_after_init;
bool cfi_bhi __ro_after_init = false;
#endif
+#ifdef CONFIG_FINEIBT
+/*
+ * <fineibt_preamble_start>:
+ * 0: f3 0f 1e fa endbr64
+ * 4: 2d 78 56 34 12 sub $0x12345678, %eax
+ * 9: 2e 0f 85 03 00 00 00 jne,pn 13 <fineibt_preamble_start+0x13>
+ * 10: 0f 1f 40 d6 nopl -0x2a(%rax)
+ *
+ * Note that the JNE target is the 0xD6 byte inside the NOPL, this decodes as
+ * UDB on x86_64 and raises #UD.
+ */
+asm( ".pushsection .rodata \n"
+ "fineibt_preamble_start: \n"
+ " endbr64 \n"
+ " subl $0x12345678, %eax \n"
+ "fineibt_preamble_bhi: \n"
+ " cs jne.d32 fineibt_preamble_start+0x13 \n"
+ "#fineibt_func: \n"
+ " nopl -42(%rax) \n"
+ "fineibt_preamble_end: \n"
+ ".popsection\n"
+);
+
+extern u8 fineibt_preamble_start[];
+extern u8 fineibt_preamble_bhi[];
+extern u8 fineibt_preamble_end[];
+
+#define fineibt_preamble_size (fineibt_preamble_end - fineibt_preamble_start)
+#define fineibt_preamble_bhi (fineibt_preamble_bhi - fineibt_preamble_start)
+#define fineibt_preamble_ud 0x13
+#define fineibt_preamble_hash 5
+
+#define fineibt_prefix_size (fineibt_preamble_size - ENDBR_INSN_SIZE)
+#endif /* CONFIG_FINEIBT */
+
#ifdef CONFIG_CFI
u32 cfi_get_func_hash(void *func)
{
@@ -1214,9 +1252,11 @@ u32 cfi_get_func_hash(void *func)
func -= cfi_get_offset();
switch (cfi_mode) {
+#ifdef CONFIG_FINEIBT
case CFI_FINEIBT:
- func += 7;
+ func += fineibt_preamble_hash;
break;
+#endif
case CFI_KCFI:
func += 1;
break;
@@ -1358,39 +1398,6 @@ early_param("cfi", cfi_parse_cmdline);
* processors. Therefore, it is good to place the most frequent branch first"
*/
-/*
- * <fineibt_preamble_start>:
- * 0: f3 0f 1e fa endbr64
- * 4: 2d 78 56 34 12 sub $0x12345678, %eax
- * 9: 2e 0f 85 03 00 00 00 jne,pn 13 <fineibt_preamble_start+0x13>
- * 10: 0f 1f 40 d6 nopl -0x2a(%rax)
- *
- * Note that the JNE target is the 0xD6 byte inside the NOPL, this decodes as
- * UDB on x86_64 and raises #UD.
- */
-asm( ".pushsection .rodata \n"
- "fineibt_preamble_start: \n"
- " endbr64 \n"
- " subl $0x12345678, %eax \n"
- "fineibt_preamble_bhi: \n"
- " cs jne.d32 fineibt_preamble_start+0x13 \n"
- "#fineibt_func: \n"
- " nopl -42(%rax) \n"
- "fineibt_preamble_end: \n"
- ".popsection\n"
-);
-
-extern u8 fineibt_preamble_start[];
-extern u8 fineibt_preamble_bhi[];
-extern u8 fineibt_preamble_end[];
-
-#define fineibt_preamble_size (fineibt_preamble_end - fineibt_preamble_start)
-#define fineibt_preamble_bhi (fineibt_preamble_bhi - fineibt_preamble_start)
-#define fineibt_preamble_ud 0x13
-#define fineibt_preamble_hash 5
-
-#define fineibt_prefix_size (fineibt_preamble_size - ENDBR_INSN_SIZE)
-
/*
* <fineibt_caller_start>:
* 0: b8 78 56 34 12 mov $0x12345678, %eax
@@ -2543,6 +2550,38 @@ static void text_poke_memset(void *dst, const void *src, size_t len)
typedef void text_poke_f(void *dst, const void *src, size_t len);
+static void __poke_vmalloc_pages(struct page **pages, void *addr,
+ bool cross_page_boundary)
+{
+ pages[0] = vmalloc_to_page(addr);
+ if (cross_page_boundary)
+ pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+}
+
+static void poke_vmalloc_pages(struct page **pages, void *addr,
+ bool cross_page_boundary)
+{
+ if (in_dbg_master()) {
+ /*
+ * If called from kgdb cannot sleep, but all other CPUs stopped
+ * anyway so safe to proceed without locks
+ */
+ __poke_vmalloc_pages(pages, addr, cross_page_boundary);
+ } else {
+ /*
+ * execmem ROX ranges are shared between modules and can be
+ * collapsed to huge PMD entries, and this collapse can happen
+ * concurrently with a racing set_memory_rox().
+ *
+ * Prevent vmalloc_to_page() from racing by acquiring an
+ * init_mm read lock which pairs with the init_mm write lock in
+ * cpa_collapse_large_pages().
+ */
+ guard(mmap_read_lock)(&init_mm);
+ __poke_vmalloc_pages(pages, addr, cross_page_boundary);
+ }
+}
+
static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t len)
{
bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE;
@@ -2560,9 +2599,7 @@ static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t l
BUG_ON(!after_bootmem);
if (!core_kernel_text((unsigned long)addr)) {
- pages[0] = vmalloc_to_page(addr);
- if (cross_page_boundary)
- pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+ poke_vmalloc_pages(pages, addr, cross_page_boundary);
} else {
pages[0] = virt_to_page(addr);
WARN_ON(!PageReserved(pages[0]));
diff --git a/arch/x86/kernel/amd_node.c b/arch/x86/kernel/amd_node.c
index 0be01725a2a4..b7926ba3610a 100644
--- a/arch/x86/kernel/amd_node.c
+++ b/arch/x86/kernel/amd_node.c
@@ -38,7 +38,6 @@ static struct pci_dev **amd_roots;
/* Protect the PCI config register pairs used for SMN. */
static DEFINE_MUTEX(smn_mutex);
-static bool smn_exclusive;
#define SMN_INDEX_OFFSET 0x60
#define SMN_DATA_OFFSET 0x64
@@ -91,11 +90,16 @@ static int __amd_smn_rw(u8 i_off, u8 d_off, u16 node, u32 address, u32 *value, b
if (node >= amd_num_nodes())
return err;
- root = amd_roots[node];
- if (!root)
+ /*
+ * Uninitialized amd_roots indicates pci_request_config_region_exclusive()
+ * didn't run or failed and thus the kernel cannot rely on having
+ * exclusive access to SMN registers so prevent that.
+ */
+ if (!amd_roots)
return err;
- if (!smn_exclusive)
+ root = amd_roots[node];
+ if (!root)
return err;
guard(mutex)(&smn_mutex);
@@ -247,7 +251,7 @@ __setup("amd_smn_debugfs_enable", amd_smn_enable_dfs);
static int __init amd_smn_init(void)
{
u16 count, num_roots, roots_per_node, node, num_nodes;
- struct pci_dev *root;
+ struct pci_dev *root __free(pci_dev_put) = NULL;
if (!cpu_feature_enabled(X86_FEATURE_ZEN))
return 0;
@@ -258,7 +262,6 @@ static int __init amd_smn_init(void)
return 0;
num_roots = 0;
- root = NULL;
while ((root = get_next_root(root))) {
pci_dbg(root, "Reserving PCI config space\n");
@@ -287,17 +290,21 @@ static int __init amd_smn_init(void)
return -ENOMEM;
roots_per_node = num_roots / num_nodes;
+ if (!roots_per_node) {
+ if (!cpu_feature_enabled(X86_FEATURE_HYPERVISOR))
+ pr_warn(FW_BUG "Error detecting roots per node.\n");
+ roots_per_node = 1;
+ }
count = 0;
node = 0;
- root = NULL;
while (node < num_nodes && (root = get_next_root(root))) {
/* Use one root for each node and skip the rest. */
if (count++ % roots_per_node)
continue;
pci_dbg(root, "is root for AMD node %u\n", node);
- amd_roots[node++] = root;
+ amd_roots[node++] = pci_dev_get(root);
}
if (enable_dfs) {
@@ -308,8 +315,6 @@ static int __init amd_smn_init(void)
debugfs_create_file("value", 0600, debugfs_dir, NULL, &smn_value_fops);
}
- smn_exclusive = true;
-
return 0;
}
diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 36e0df4d1342..fd878bc7ff08 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -864,7 +864,7 @@ static void amd_deferred_error_interrupt(void)
void mce_amd_handle_storm(unsigned int bank, bool on)
{
- threshold_restart_bank(bank, on);
+ threshold_restart_bank(bank, !on);
}
static void amd_reset_thr_limit(unsigned int bank)
diff --git a/arch/x86/kernel/itmt.c b/arch/x86/kernel/itmt.c
index 243a769fdd97..85ebde361d6a 100644
--- a/arch/x86/kernel/itmt.c
+++ b/arch/x86/kernel/itmt.c
@@ -110,18 +110,14 @@ int sched_set_itmt_support(void)
arch_debugfs_dir,
&sysctl_sched_itmt_enabled,
&dfs_sched_itmt_fops);
- if (IS_ERR_OR_NULL(dfs_sched_itmt)) {
+ if (IS_ERR(dfs_sched_itmt))
dfs_sched_itmt = NULL;
- return -ENOMEM;
- }
dfs_sched_core_prio = debugfs_create_file("sched_core_priority", 0644,
arch_debugfs_dir, NULL,
&sched_core_priority_fops);
- if (IS_ERR_OR_NULL(dfs_sched_core_prio)) {
+ if (IS_ERR(dfs_sched_core_prio))
dfs_sched_core_prio = NULL;
- return -ENOMEM;
- }
sched_itmt_capable = true;
diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index 3b7e807e803c..f8d5cdbb6871 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -22,6 +22,7 @@
#include <linux/cc_platform.h>
#include <linux/set_memory.h>
#include <linux/memregion.h>
+#include <linux/cleanup.h>
#include <asm/e820/api.h>
#include <asm/processor.h>
@@ -49,7 +50,8 @@ struct cpa_data {
unsigned int flags;
unsigned int force_split : 1,
force_static_prot : 1,
- force_flush_all : 1;
+ force_flush_all : 1,
+ init_mm_read_locked : 1;
struct page **pages;
};
@@ -62,18 +64,18 @@ enum cpa_warn {
static const int cpa_warn_level = CPA_PROTECT;
/*
- * Serialize cpa() (for !DEBUG_PAGEALLOC which uses large identity mappings)
- * using cpa_lock. So that we don't allow any other cpu, with stale large tlb
- * entries change the page attribute in parallel to some other cpu
- * splitting a large page entry along with changing the attribute.
+ * Serialize cpa() using cpa_lock so that we don't allow any other cpu, with
+ * stale large tlb entries, to change the page attribute in parallel to some
+ * other cpu splitting a large page entry along with changing the attribute.
*/
static DEFINE_SPINLOCK(cpa_lock);
-#define CPA_FLUSHTLB 1
-#define CPA_ARRAY 2
-#define CPA_PAGES_ARRAY 4
-#define CPA_NO_CHECK_ALIAS 8 /* Do not search for aliases */
-#define CPA_COLLAPSE 16 /* try to collapse large pages */
+#define CPA_FLUSHTLB 0x01
+#define CPA_ARRAY 0x02
+#define CPA_PAGES_ARRAY 0x04
+#define CPA_NO_CHECK_ALIAS 0x08 /* Do not search for aliases */
+#define CPA_COLLAPSE 0x10 /* try to collapse large pages */
+#define CPA_DEBUG_PAGEALLOC 0x20
static inline pgprot_t cachemode2pgprot(enum page_cache_mode pcm)
{
@@ -410,7 +412,7 @@ static void __cpa_flush_tlb(void *data)
static int collapse_large_pages(unsigned long addr, struct list_head *pgtables);
-static void cpa_collapse_large_pages(struct cpa_data *cpa)
+static void __cpa_collapse_large_pages(struct cpa_data *cpa)
{
unsigned long start, addr, end;
struct ptdesc *ptdesc, *tmp;
@@ -442,12 +444,32 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
list_del(&ptdesc->pt_list);
- pagetable_free(ptdesc);
+ /*
+ * Only early alloc'd direct map should not be flagged PG_table
+ * here and those shouldn't be collapsed. However be abundantly
+ * cautious and handle the !PG_table case too.
+ */
+ if (PageTable((ptdesc_page(ptdesc))))
+ pagetable_dtor_free(ptdesc);
+ else
+ pagetable_free(ptdesc);
}
spin_unlock(&cpa_lock);
}
+static void cpa_collapse_large_pages(struct cpa_data *cpa)
+{
+ /*
+ * Take the mmap write lock on init_mm to:
+ * - Avoid a use-after-free if raced by ptdump (which takes its own
+ * write lock on init_mm).
+ * - Serialise concurrent CPA walkers.
+ */
+ scoped_guard(mmap_write_lock, &init_mm)
+ __cpa_collapse_large_pages(cpa);
+}
+
static void cpa_flush(struct cpa_data *cpa, int cache)
{
unsigned int i;
@@ -1131,11 +1153,10 @@ static void split_set_pte(struct cpa_data *cpa, pte_t *pte, unsigned long pfn,
static int
__split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
- struct ptdesc *ptdesc)
+ pte_t *pbase)
{
unsigned long lpaddr, lpinc, ref_pfn, pfn, pfninc = 1;
- struct page *base = ptdesc_page(ptdesc);
- pte_t *pbase = (pte_t *)page_address(base);
+ struct page *base = virt_to_page(pbase);
unsigned int i, level;
pgprot_t ref_prot;
bool nx, rw;
@@ -1239,18 +1260,20 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
unsigned long address)
{
- struct ptdesc *ptdesc;
+ pte_t *pte;
- if (!debug_pagealloc_enabled())
- spin_unlock(&cpa_lock);
- ptdesc = pagetable_alloc(GFP_KERNEL, 0);
- if (!debug_pagealloc_enabled())
- spin_lock(&cpa_lock);
- if (!ptdesc)
+ spin_unlock(&cpa_lock);
+ if (cpa->init_mm_read_locked)
+ mmap_read_unlock(&init_mm);
+ pte = pte_alloc_one_kernel(&init_mm);
+ if (cpa->init_mm_read_locked)
+ mmap_read_lock(&init_mm);
+ spin_lock(&cpa_lock);
+ if (!pte)
return -ENOMEM;
- if (__split_large_page(cpa, kpte, address, ptdesc))
- pagetable_free(ptdesc);
+ if (__split_large_page(cpa, kpte, address, pte))
+ pte_free_kernel(&init_mm, pte);
return 0;
}
@@ -2010,6 +2033,7 @@ static int __change_page_attr_set_clr(struct cpa_data *cpa, int primary)
{
unsigned long numpages = cpa->numpages;
unsigned long rempages = numpages;
+ bool lock = true;
int ret = 0;
/*
@@ -2019,6 +2043,29 @@ static int __change_page_attr_set_clr(struct cpa_data *cpa, int primary)
!cpa->force_split)
return ret;
+ /*
+ * DEBUG_PAGEALLOC is special; it is called from any context the
+ * page-allocator is, which violates the normal cpa_lock locking
+ * rules.
+ *
+ * However, since it is part of the page-allocator, things are still
+ * properly serialized by the page-allocator locking and the fact that
+ * when a page is owned by the page-allocator, it isn't owned by
+ * anybody else. That is, you *SHOULD NOT* be calling cpa() on memory
+ * that isn't allocated.
+ *
+ * Additionally, DEBUG_PAGEALLOC ensures (per probe_page_size_mask())
+ * that the kernel mapping is 4k pages, therefore there are no large
+ * pages to split/collapse.
+ *
+ * Furthermore, the page-allocator strictly manages pages that
+ * *exist*, avoiding pgd_lock.
+ *
+ * Therefore, it is safe to not take cpa_lock.
+ */
+ if (debug_pagealloc_enabled() && (cpa->flags & CPA_DEBUG_PAGEALLOC))
+ lock = false;
+
while (rempages) {
/*
* Store the remaining nr of pages for the large page
@@ -2029,11 +2076,12 @@ static int __change_page_attr_set_clr(struct cpa_data *cpa, int primary)
if (cpa->flags & (CPA_ARRAY | CPA_PAGES_ARRAY))
cpa->numpages = 1;
- if (!debug_pagealloc_enabled())
- spin_lock(&cpa_lock);
- ret = __change_page_attr(cpa, primary);
- if (!debug_pagealloc_enabled())
- spin_unlock(&cpa_lock);
+ if (lock) {
+ guard(spinlock)(&cpa_lock);
+ ret = __change_page_attr(cpa, primary);
+ } else {
+ ret = __change_page_attr(cpa, primary);
+ }
if (ret)
goto out;
@@ -2115,7 +2163,11 @@ static int change_page_attr_set_clr(unsigned long *addr, int numpages,
cpa.curpage = 0;
cpa.force_split = force_split;
- ret = __change_page_attr_set_clr(&cpa, 1);
+ /* Avoid race with concurrent CPA collapse. */
+ cpa.init_mm_read_locked = true;
+ scoped_guard(mmap_read_lock, &init_mm)
+ ret = __change_page_attr_set_clr(&cpa, 1);
+ cpa.init_mm_read_locked = false;
/*
* Check whether we really changed something:
@@ -2612,7 +2664,7 @@ int set_pages_rw(struct page *page, int numpages)
return set_memory_rw(addr, numpages);
}
-static int __set_pages_p(struct page *page, int numpages)
+static int __set_pages_p(struct page *page, int numpages, unsigned int cpa_flags)
{
unsigned long tempaddr = (unsigned long) page_address(page);
struct cpa_data cpa = { .vaddr = &tempaddr,
@@ -2620,7 +2672,7 @@ static int __set_pages_p(struct page *page, int numpages)
.numpages = numpages,
.mask_set = __pgprot(_PAGE_PRESENT | _PAGE_RW),
.mask_clr = __pgprot(0),
- .flags = CPA_NO_CHECK_ALIAS };
+ .flags = CPA_NO_CHECK_ALIAS | cpa_flags };
/*
* No alias checking needed for setting present flag. otherwise,
@@ -2631,7 +2683,7 @@ static int __set_pages_p(struct page *page, int numpages)
return __change_page_attr_set_clr(&cpa, 1);
}
-static int __set_pages_np(struct page *page, int numpages)
+static int __set_pages_np(struct page *page, int numpages, unsigned int cpa_flags)
{
unsigned long tempaddr = (unsigned long) page_address(page);
struct cpa_data cpa = { .vaddr = &tempaddr,
@@ -2639,7 +2691,7 @@ static int __set_pages_np(struct page *page, int numpages)
.numpages = numpages,
.mask_set = __pgprot(0),
.mask_clr = __pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_DIRTY),
- .flags = CPA_NO_CHECK_ALIAS };
+ .flags = CPA_NO_CHECK_ALIAS | cpa_flags };
/*
* No alias checking needed for setting not present flag. otherwise,
@@ -2652,20 +2704,20 @@ static int __set_pages_np(struct page *page, int numpages)
int set_direct_map_invalid_noflush(struct page *page)
{
- return __set_pages_np(page, 1);
+ return __set_pages_np(page, 1, 0);
}
int set_direct_map_default_noflush(struct page *page)
{
- return __set_pages_p(page, 1);
+ return __set_pages_p(page, 1, 0);
}
int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid)
{
if (valid)
- return __set_pages_p(page, nr);
+ return __set_pages_p(page, nr, 0);
- return __set_pages_np(page, nr);
+ return __set_pages_np(page, nr, 0);
}
#ifdef CONFIG_DEBUG_PAGEALLOC
@@ -2684,15 +2736,23 @@ void __kernel_map_pages(struct page *page, int numpages, int enable)
* and hence no memory allocations during large page split.
*/
if (enable)
- __set_pages_p(page, numpages);
+ __set_pages_p(page, numpages, CPA_DEBUG_PAGEALLOC);
else
- __set_pages_np(page, numpages);
+ __set_pages_np(page, numpages, CPA_DEBUG_PAGEALLOC);
/*
- * We should perform an IPI and flush all tlbs,
- * but that can deadlock->flush only current cpu.
- * Preemption needs to be disabled around __flush_tlb_all() due to
- * CR3 reload in __native_flush_tlb().
+ * We should perform an IPI and flush all tlbs, but that can
+ * deadlock, settle for a local flush.
+ *
+ * Not doing a global TLB flush means that remote CPUs will retain
+ * stale TLB entries. In case of P->NP (on free) this means the remote
+ * CPUs will not take the faults, making the debug scheme less
+ * reliable. On the NP->P (on alloc) this means the remote CPUs can
+ * take a spurious fault. However spurious_kernel_fault() will observe
+ * *_present() and fix it up.
+ *
+ * Preemption needs to be disabled around __flush_tlb_all() due to CR3
+ * reload in __native_flush_tlb().
*/
preempt_disable();
__flush_tlb_all();
diff --git a/block/bio.c b/block/bio.c
index 5018a6fc2f36..bdcb60cb8d88 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -859,6 +859,7 @@ static int __bio_clone(struct bio *bio, struct bio *bio_src, gfp_t gfp)
bio->bi_ioprio = bio_src->bi_ioprio;
bio->bi_write_hint = bio_src->bi_write_hint;
bio->bi_write_stream = bio_src->bi_write_stream;
+ bio->bi_bvec_gap_bit = bio_src->bi_bvec_gap_bit;
bio->bi_iter = bio_src->bi_iter;
if (bio->bi_bdev) {
@@ -1913,6 +1914,14 @@ struct bio *bio_split(struct bio *bio, int sectors,
bio_advance(bio, split->bi_iter.bi_size);
+ /*
+ * The gap bit is set when splitting to limits and only applies to the
+ * front bio that was split off. The remaining bio will calcualte its
+ * gap value when it is subsequently split to limits, so it is safe to
+ * re-initialize the value back to 0.
+ */
+ bio->bi_bvec_gap_bit = 0;
+
if (bio_flagged(bio, BIO_TRACE_COMPLETION))
bio_set_flag(split, BIO_TRACE_COMPLETION);
diff --git a/drivers/accel/amdxdna/aie2_message.c b/drivers/accel/amdxdna/aie2_message.c
index dfe0fbdf066d..b4c49259a1a2 100644
--- a/drivers/accel/amdxdna/aie2_message.c
+++ b/drivers/accel/amdxdna/aie2_message.c
@@ -994,7 +994,7 @@ int aie2_cmdlist_multi_execbuf(struct amdxdna_hwctx *hwctx,
}
ccnt = payload->command_count;
- if (payload_len < struct_size(payload, data, ccnt)) {
+ if (!ccnt || payload_len < struct_size(payload, data, ccnt)) {
XDNA_DBG(xdna, "Invalid command count %d", ccnt);
return -EINVAL;
}
diff --git a/drivers/accel/amdxdna/amdxdna_ctx.c b/drivers/accel/amdxdna/amdxdna_ctx.c
index 31a414c3f0d9..888e857ec558 100644
--- a/drivers/accel/amdxdna/amdxdna_ctx.c
+++ b/drivers/accel/amdxdna/amdxdna_ctx.c
@@ -183,8 +183,10 @@ int amdxdna_cmd_set_error(struct amdxdna_gem_obj *abo,
if (!abo)
return -EINVAL;
cmd = amdxdna_gem_vmap(abo);
- if (!cmd)
+ if (!cmd) {
+ amdxdna_gem_put_obj(abo);
return -ENOMEM;
+ }
}
memset(cmd->data, 0xff, abo->mem.size - sizeof(*cmd));
diff --git a/drivers/accel/amdxdna/amdxdna_ctx.h b/drivers/accel/amdxdna/amdxdna_ctx.h
index b6bef3af7dab..6e78bab8a02c 100644
--- a/drivers/accel/amdxdna/amdxdna_ctx.h
+++ b/drivers/accel/amdxdna/amdxdna_ctx.h
@@ -55,7 +55,7 @@ struct amdxdna_cmd_chain {
u32 submit_index;
u32 error_index;
u32 reserved[3];
- u64 data[] __counted_by(command_count);
+ u64 data[];
};
/*
diff --git a/drivers/accel/amdxdna/amdxdna_gem.c b/drivers/accel/amdxdna/amdxdna_gem.c
index 2a16de96e6a4..4b0d58d0329b 100644
--- a/drivers/accel/amdxdna/amdxdna_gem.c
+++ b/drivers/accel/amdxdna/amdxdna_gem.c
@@ -758,9 +758,15 @@ static int amdxdna_gem_dev_obj_vmap(struct drm_gem_object *obj, struct iosys_map
return 0;
}
+static struct dma_buf *amdxdna_gem_dev_obj_export(struct drm_gem_object *gobj, int flags)
+{
+ return ERR_PTR(-EOPNOTSUPP);
+}
+
static const struct drm_gem_object_funcs amdxdna_gem_dev_obj_funcs = {
.free = amdxdna_gem_dev_obj_free,
.vmap = amdxdna_gem_dev_obj_vmap,
+ .export = amdxdna_gem_dev_obj_export,
};
static const struct drm_gem_object_funcs amdxdna_gem_shmem_funcs = {
@@ -1240,6 +1246,9 @@ static int amdxdna_flush_bo(struct amdxdna_gem_obj *abo, u64 offset, u64 size)
{
u64 end;
+ if (is_import_bo(abo))
+ return -EOPNOTSUPP;
+
if (offset >= abo->mem.size)
return -EINVAL;
@@ -1250,9 +1259,7 @@ static int amdxdna_flush_bo(struct amdxdna_gem_obj *abo, u64 offset, u64 size)
if (!size)
return 0;
- if (is_import_bo(abo))
- drm_clflush_sg(abo->base.sgt);
- else if (amdxdna_gem_vmap(abo))
+ if (amdxdna_gem_vmap(abo))
drm_clflush_virt_range(amdxdna_gem_vmap(abo) + offset, size);
else if (abo->base.pages)
drm_clflush_pages(abo->base.pages, abo->mem.size >> PAGE_SHIFT);
diff --git a/drivers/accel/ethosu/ethosu_device.h b/drivers/accel/ethosu/ethosu_device.h
index b189fa783d6a..cf53964a4f04 100644
--- a/drivers/accel/ethosu/ethosu_device.h
+++ b/drivers/accel/ethosu/ethosu_device.h
@@ -68,6 +68,7 @@ struct gen_pool;
#define PROT_ACTIVE_CSL BIT(1)
enum ethosu_cmds {
+ NPU_OP_STOP = 0x0,
NPU_OP_CONV = 0x2,
NPU_OP_DEPTHWISE = 0x3,
NPU_OP_POOL = 0x5,
diff --git a/drivers/accel/ethosu/ethosu_drv.c b/drivers/accel/ethosu/ethosu_drv.c
index b2901eb8a7a0..ff11d7b27953 100644
--- a/drivers/accel/ethosu/ethosu_drv.c
+++ b/drivers/accel/ethosu/ethosu_drv.c
@@ -274,8 +274,6 @@ static int ethosu_device_suspend(struct device *dev)
static int ethosu_sram_init(struct ethosu_device *ethosudev)
{
- ethosudev->npu_info.sram_size = 0;
-
ethosudev->srampool = of_gen_pool_get(ethosudev->base.dev->of_node, "sram", 0);
if (!ethosudev->srampool)
return 0;
@@ -286,6 +284,7 @@ static int ethosu_sram_init(struct ethosu_device *ethosudev)
ethosudev->npu_info.sram_size,
ðosudev->sramphys);
if (!ethosudev->sram) {
+ ethosudev->npu_info.sram_size = 0;
dev_err(ethosudev->base.dev, "failed to allocate from SRAM pool\n");
return -ENOMEM;
}
diff --git a/drivers/accel/ethosu/ethosu_gem.c b/drivers/accel/ethosu/ethosu_gem.c
index d50fed64d4d9..9afe2549ec84 100644
--- a/drivers/accel/ethosu/ethosu_gem.c
+++ b/drivers/accel/ethosu/ethosu_gem.c
@@ -204,7 +204,7 @@ static u64 feat_matrix_length(struct ethosu_device *edev,
struct feat_matrix *fm,
u32 x, u32 y, u32 c, bool ofm)
{
- u32 element_size, storage = fm->precision >> 14;
+ u32 element_size, storage = ethosu_is_u65(edev) ? 0 : fm->precision >> 14;
int tile = 0;
u64 addr;
@@ -390,6 +390,7 @@ static int ethosu_gem_cmdstream_copy_and_validate(struct drm_device *ddev,
struct ethosu_validated_cmdstream_info __free(kfree) *info = kzalloc_obj(*info);
struct ethosu_device *edev = to_ethosu_device(ddev);
u32 *bocmds = bo->base.vaddr;
+ bool ends_with_stop = false;
struct cmd_state st;
int i, ret;
@@ -426,6 +427,11 @@ static int ethosu_gem_cmdstream_copy_and_validate(struct drm_device *ddev,
}
switch (cmd) {
+ case NPU_OP_STOP:
+ if (i != size / 4 - 1)
+ return -EINVAL;
+ ends_with_stop = true;
+ break;
case NPU_OP_DMA_START:
srclen = dma_length(info, &st.dma, &st.dma.src);
dstlen = dma_length(info, &st.dma, &st.dma.dst);
@@ -688,6 +694,9 @@ static int ethosu_gem_cmdstream_copy_and_validate(struct drm_device *ddev,
}
}
+ if (!ends_with_stop)
+ return -EINVAL;
+
for (i = 0; i < NPU_BASEP_REGION_MAX; i++) {
if (!info->region_size[i])
continue;
diff --git a/drivers/accel/ethosu/ethosu_job.c b/drivers/accel/ethosu/ethosu_job.c
index 1e4b65f62933..3365d2e6d51b 100644
--- a/drivers/accel/ethosu/ethosu_job.c
+++ b/drivers/accel/ethosu/ethosu_job.c
@@ -325,7 +325,7 @@ int ethosu_job_init(struct ethosu_device *edev)
ret = devm_request_threaded_irq(dev, edev->irq,
ethosu_job_irq_handler,
ethosu_job_irq_handler_thread,
- IRQF_SHARED, KBUILD_MODNAME,
+ 0, KBUILD_MODNAME,
edev);
if (ret) {
dev_err(dev, "failed to request irq\n");
@@ -356,12 +356,10 @@ int ethosu_job_open(struct ethosu_file_priv *ethosu_priv)
{
struct ethosu_device *dev = ethosu_priv->edev;
struct drm_gpu_scheduler *sched = &dev->sched;
- int ret;
- ret = drm_sched_entity_init(ðosu_priv->sched_entity,
- DRM_SCHED_PRIORITY_NORMAL,
- &sched, 1, NULL);
- return WARN_ON(ret);
+ return drm_sched_entity_init(ðosu_priv->sched_entity,
+ DRM_SCHED_PRIORITY_NORMAL,
+ &sched, 1, NULL);
}
void ethosu_job_close(struct ethosu_file_priv *ethosu_priv)
@@ -427,13 +425,13 @@ static int ethosu_ioctl_submit_job(struct drm_device *dev, struct drm_file *file
if (!cmd_info->region_size[i])
continue;
if (i == ETHOSU_SRAM_REGION) {
- if (cmd_info->region_size[i] <= edev->npu_info.sram_size)
+ if (cmd_info->region_size[i] <= ejob->sram_size)
continue;
dev_err(dev->dev,
- "cmd stream region %d size greater than SRAM size (%llu > %u)\n",
+ "cmd stream region %d size greater than job SRAM size (%llu > %u)\n",
i, cmd_info->region_size[i],
- edev->npu_info.sram_size);
+ ejob->sram_size);
ret = -EINVAL;
goto out_cleanup_job;
}
diff --git a/drivers/accel/ivpu/ivpu_fw_log.c b/drivers/accel/ivpu/ivpu_fw_log.c
index 716467aa3156..9eafc42120b6 100644
--- a/drivers/accel/ivpu/ivpu_fw_log.c
+++ b/drivers/accel/ivpu/ivpu_fw_log.c
@@ -26,10 +26,17 @@ MODULE_PARM_DESC(fw_log_level,
" error=" __stringify(IVPU_FW_LOG_ERROR)
" fatal=" __stringify(IVPU_FW_LOG_FATAL));
+struct ivpu_fw_log_desc {
+ struct vpu_tracing_buffer_header *log;
+ u32 header_size;
+ u32 size;
+};
+
static int fw_log_from_bo(struct ivpu_device *vdev, struct ivpu_bo *bo, u32 *offset,
- struct vpu_tracing_buffer_header **out_log)
+ struct ivpu_fw_log_desc *desc)
{
struct vpu_tracing_buffer_header *log;
+ u32 header_size, size;
if ((*offset + sizeof(*log)) > ivpu_bo_size(bo))
return -EINVAL;
@@ -39,26 +46,32 @@ static int fw_log_from_bo(struct ivpu_device *vdev, struct ivpu_bo *bo, u32 *off
if (log->vpu_canary_start != VPU_TRACING_BUFFER_CANARY)
return -EINVAL;
- if (log->header_size < sizeof(*log) || log->header_size > 1024) {
- ivpu_dbg(vdev, FW_BOOT, "Invalid header size 0x%x\n", log->header_size);
+ header_size = READ_ONCE(log->header_size);
+ size = READ_ONCE(log->size);
+
+ if (header_size < sizeof(*log) || header_size > 1024) {
+ ivpu_dbg(vdev, FW_BOOT, "Invalid header size 0x%x\n", header_size);
return -EINVAL;
}
- if (log->size < log->header_size) {
- ivpu_dbg(vdev, FW_BOOT, "Invalid log size 0x%x\n", log->size);
+ if ((char *)log + size > (char *)ivpu_bo_vaddr(bo) + ivpu_bo_size(bo)) {
+ ivpu_dbg(vdev, FW_BOOT, "Invalid log size 0x%x\n", size);
return -EINVAL;
}
- if ((char *)log + log->size > (char *)ivpu_bo_vaddr(bo) + ivpu_bo_size(bo)) {
- ivpu_dbg(vdev, FW_BOOT, "Invalid log size 0x%x\n", log->size);
+ if (size < header_size) {
+ ivpu_dbg(vdev, FW_BOOT, "Invalid log size 0x%x < header size 0x%x\n",
+ size, header_size);
return -EINVAL;
}
- *out_log = log;
- *offset += log->size;
+ desc->log = log;
+ desc->header_size = header_size;
+ desc->size = size;
+ *offset += size;
ivpu_dbg(vdev, FW_BOOT,
- "FW log name \"%s\", write offset 0x%x size 0x%x, wrap count %d, hdr version %d size %d format %d, alignment %d",
- log->name, log->write_index, log->size, log->wrap_count, log->header_version,
- log->header_size, log->format, log->alignment);
+ "FW log name \"%.*s\", write offset 0x%x size 0x%x, wrap count %d, hdr version %d size %d format %d, alignment %d",
+ (int)ARRAY_SIZE(log->name), log->name, log->write_index, size, log->wrap_count,
+ log->header_version, header_size, log->format, log->alignment);
return 0;
}
@@ -94,11 +107,12 @@ static void fw_log_print_lines(char *buffer, u32 size, struct drm_printer *p)
drm_printf(p, "%s", line);
}
-static void fw_log_print_buffer(struct vpu_tracing_buffer_header *log, const char *prefix,
+static void fw_log_print_buffer(struct ivpu_fw_log_desc *desc, const char *prefix,
bool only_new_msgs, struct drm_printer *p)
{
- char *log_data = (void *)log + log->header_size;
- u32 data_size = log->size - log->header_size;
+ struct vpu_tracing_buffer_header *log = desc->log;
+ char *log_data = (void *)log + desc->header_size;
+ u32 data_size = desc->size - desc->header_size;
u32 log_start = only_new_msgs ? READ_ONCE(log->read_index) : 0;
u32 log_end = READ_ONCE(log->write_index);
@@ -109,7 +123,8 @@ static void fw_log_print_buffer(struct vpu_tracing_buffer_header *log, const cha
if (log->wrap_count == log->read_wrap_count) {
if (log_end <= log_start) {
- drm_printf(p, "==== %s \"%s\" log empty ====\n", prefix, log->name);
+ drm_printf(p, "==== %s \"%.*s\" log empty ====\n", prefix,
+ (int)ARRAY_SIZE(log->name), log->name);
return;
}
} else if (log->wrap_count == log->read_wrap_count + 1) {
@@ -119,7 +134,8 @@ static void fw_log_print_buffer(struct vpu_tracing_buffer_header *log, const cha
log_start = log_end;
}
- drm_printf(p, "==== %s \"%s\" log start ====\n", prefix, log->name);
+ drm_printf(p, "==== %s \"%.*s\" log start ====\n", prefix, (int)ARRAY_SIZE(log->name),
+ log->name);
if (log_end > log_start) {
fw_log_print_lines(log_data + log_start, log_end - log_start, p);
} else {
@@ -127,18 +143,19 @@ static void fw_log_print_buffer(struct vpu_tracing_buffer_header *log, const cha
fw_log_print_lines(log_data, log_end, p);
}
drm_printf(p, "\n\x1b[0m"); /* add new line and clear formatting */
- drm_printf(p, "==== %s \"%s\" log end ====\n", prefix, log->name);
+ drm_printf(p, "==== %s \"%.*s\" log end ====\n", prefix, (int)ARRAY_SIZE(log->name),
+ log->name);
}
static void
fw_log_print_all_in_bo(struct ivpu_device *vdev, const char *name,
struct ivpu_bo *bo, bool only_new_msgs, struct drm_printer *p)
{
- struct vpu_tracing_buffer_header *log;
+ struct ivpu_fw_log_desc desc;
u32 next = 0;
- while (fw_log_from_bo(vdev, bo, &next, &log) == 0)
- fw_log_print_buffer(log, name, only_new_msgs, p);
+ while (fw_log_from_bo(vdev, bo, &next, &desc) == 0)
+ fw_log_print_buffer(&desc, name, only_new_msgs, p);
}
void ivpu_fw_log_print(struct ivpu_device *vdev, bool only_new_msgs, struct drm_printer *p)
@@ -149,36 +166,36 @@ void ivpu_fw_log_print(struct ivpu_device *vdev, bool only_new_msgs, struct drm_
void ivpu_fw_log_mark_read(struct ivpu_device *vdev)
{
- struct vpu_tracing_buffer_header *log;
+ struct ivpu_fw_log_desc desc;
u32 next;
next = 0;
- while (fw_log_from_bo(vdev, vdev->fw->mem_log_crit, &next, &log) == 0) {
- log->read_index = READ_ONCE(log->write_index);
- log->read_wrap_count = READ_ONCE(log->wrap_count);
+ while (fw_log_from_bo(vdev, vdev->fw->mem_log_crit, &next, &desc) == 0) {
+ desc.log->read_index = READ_ONCE(desc.log->write_index);
+ desc.log->read_wrap_count = READ_ONCE(desc.log->wrap_count);
}
next = 0;
- while (fw_log_from_bo(vdev, vdev->fw->mem_log_verb, &next, &log) == 0) {
- log->read_index = READ_ONCE(log->write_index);
- log->read_wrap_count = READ_ONCE(log->wrap_count);
+ while (fw_log_from_bo(vdev, vdev->fw->mem_log_verb, &next, &desc) == 0) {
+ desc.log->read_index = READ_ONCE(desc.log->write_index);
+ desc.log->read_wrap_count = READ_ONCE(desc.log->wrap_count);
}
}
void ivpu_fw_log_reset(struct ivpu_device *vdev)
{
- struct vpu_tracing_buffer_header *log;
+ struct ivpu_fw_log_desc desc;
u32 next;
next = 0;
- while (fw_log_from_bo(vdev, vdev->fw->mem_log_crit, &next, &log) == 0) {
- log->read_index = 0;
- log->read_wrap_count = 0;
+ while (fw_log_from_bo(vdev, vdev->fw->mem_log_crit, &next, &desc) == 0) {
+ desc.log->read_index = 0;
+ desc.log->read_wrap_count = 0;
}
next = 0;
- while (fw_log_from_bo(vdev, vdev->fw->mem_log_verb, &next, &log) == 0) {
- log->read_index = 0;
- log->read_wrap_count = 0;
+ while (fw_log_from_bo(vdev, vdev->fw->mem_log_verb, &next, &desc) == 0) {
+ desc.log->read_index = 0;
+ desc.log->read_wrap_count = 0;
}
}
diff --git a/drivers/accel/ivpu/ivpu_gem.h b/drivers/accel/ivpu/ivpu_gem.h
index 0c3350f22b55..b1ae020a4fc2 100644
--- a/drivers/accel/ivpu/ivpu_gem.h
+++ b/drivers/accel/ivpu/ivpu_gem.h
@@ -87,15 +87,23 @@ static inline bool ivpu_bo_is_resident(struct ivpu_bo *bo)
return !!bo->base.pages;
}
-static inline void *ivpu_to_cpu_addr(struct ivpu_bo *bo, u32 vpu_addr)
+static inline void *ivpu_to_cpu_addr(struct ivpu_bo *bo, u64 vpu_addr, u64 size)
{
+ u64 bo_size = ivpu_bo_size(bo);
+ u64 offset;
+
if (vpu_addr < bo->vpu_addr)
return NULL;
- if (vpu_addr >= (bo->vpu_addr + ivpu_bo_size(bo)))
+ if (size > bo_size)
+ return NULL;
+
+ offset = vpu_addr - bo->vpu_addr;
+
+ if (offset > bo_size - size)
return NULL;
- return ivpu_bo_vaddr(bo) + (vpu_addr - bo->vpu_addr);
+ return ivpu_bo_vaddr(bo) + offset;
}
static inline u32 cpu_to_vpu_addr(struct ivpu_bo *bo, void *cpu_addr)
diff --git a/drivers/accel/ivpu/ivpu_ipc.c b/drivers/accel/ivpu/ivpu_ipc.c
index 978bc3d8704f..8f69fb133e2e 100644
--- a/drivers/accel/ivpu/ivpu_ipc.c
+++ b/drivers/accel/ivpu/ivpu_ipc.c
@@ -79,7 +79,7 @@ ivpu_ipc_tx_prepare(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons,
return -ENOMEM;
}
- tx_buf = ivpu_to_cpu_addr(ipc->mem_tx, tx_buf_vpu_addr);
+ tx_buf = ivpu_to_cpu_addr(ipc->mem_tx, tx_buf_vpu_addr, sizeof(*tx_buf));
if (drm_WARN_ON(&vdev->drm, !tx_buf)) {
gen_pool_free(ipc->mm_tx, tx_buf_vpu_addr, sizeof(*tx_buf));
return -EIO;
@@ -420,7 +420,7 @@ void ivpu_ipc_irq_handler(struct ivpu_device *vdev)
return;
}
- ipc_hdr = ivpu_to_cpu_addr(ipc->mem_rx, vpu_addr);
+ ipc_hdr = ivpu_to_cpu_addr(ipc->mem_rx, vpu_addr, sizeof(*ipc_hdr));
if (!ipc_hdr) {
ivpu_warn_ratelimited(vdev, "IPC msg 0x%x out of range\n", vpu_addr);
continue;
@@ -429,7 +429,8 @@ void ivpu_ipc_irq_handler(struct ivpu_device *vdev)
jsm_msg = NULL;
if (ipc_hdr->channel != IVPU_IPC_CHAN_BOOT_MSG) {
- jsm_msg = ivpu_to_cpu_addr(ipc->mem_rx, ipc_hdr->data_addr);
+ jsm_msg = ivpu_to_cpu_addr(ipc->mem_rx, ipc_hdr->data_addr,
+ sizeof(*jsm_msg));
if (!jsm_msg) {
ivpu_warn_ratelimited(vdev, "JSM msg 0x%x out of range\n",
ipc_hdr->data_addr);
diff --git a/drivers/accel/qaic/qaic_control.c b/drivers/accel/qaic/qaic_control.c
index 50bf3340e49c..2ccc55486aac 100644
--- a/drivers/accel/qaic/qaic_control.c
+++ b/drivers/accel/qaic/qaic_control.c
@@ -963,11 +963,13 @@ static int decode_status(struct qaic_device *qdev, void *trans, struct manage_ms
static int decode_message(struct qaic_device *qdev, struct manage_msg *user_msg,
struct wire_msg *msg, struct ioctl_resources *resources,
- struct qaic_user *usr)
+ struct qaic_user *usr, bool orphaned_deactivate)
{
+ u32 msg_hdr_count = le32_to_cpu(msg->hdr.count);
u32 msg_hdr_len = le32_to_cpu(msg->hdr.len);
struct wire_trans_hdr *trans_hdr;
u32 msg_len = 0;
+ int trans_type;
int ret;
int i;
@@ -975,10 +977,12 @@ static int decode_message(struct qaic_device *qdev, struct manage_msg *user_msg,
msg_hdr_len > QAIC_MANAGE_MAX_MSG_LENGTH)
return -EINVAL;
- user_msg->len = 0;
- user_msg->count = le32_to_cpu(msg->hdr.count);
+ if (user_msg) {
+ user_msg->len = 0;
+ user_msg->count = msg_hdr_count;
+ }
- for (i = 0; i < user_msg->count; ++i) {
+ for (i = 0; i < msg_hdr_count; ++i) {
u32 hdr_len;
if (msg_len > msg_hdr_len - sizeof(*trans_hdr))
@@ -990,7 +994,20 @@ static int decode_message(struct qaic_device *qdev, struct manage_msg *user_msg,
size_add(msg_len, hdr_len) > msg_hdr_len)
return -EINVAL;
- switch (le32_to_cpu(trans_hdr->type)) {
+ trans_type = le32_to_cpu(trans_hdr->type);
+ /*
+ * orphaned_deactivate is the case where a deactivate response
+ * is received from the device after the user owning the DBC,
+ * and the message requesting deactivation, has gone away.
+ * In this case, only process QAIC_TRANS_DEACTIVATE_FROM_DEV
+ * transaction and skip the others.
+ */
+ if (orphaned_deactivate && trans_type != QAIC_TRANS_DEACTIVATE_FROM_DEV) {
+ msg_len += hdr_len;
+ continue;
+ }
+
+ switch (trans_type) {
case QAIC_TRANS_PASSTHROUGH_FROM_DEV:
ret = decode_passthrough(qdev, trans_hdr, user_msg, &msg_len);
break;
@@ -1281,7 +1298,7 @@ static int qaic_manage(struct qaic_device *qdev, struct qaic_user *usr, struct m
goto dma_cont_failed;
}
- ret = decode_message(qdev, user_msg, rsp, &resources, usr);
+ ret = decode_message(qdev, user_msg, rsp, &resources, usr, false);
dma_cont_failed:
free_dbc_buf(qdev, &resources);
@@ -1446,22 +1463,7 @@ static void resp_worker(struct work_struct *work)
* response to the QAIC_TRANS_TERMINATE_TO_DEV transaction,
* otherwise, the user can issue an soc_reset to the device.
*/
- u32 msg_count = le32_to_cpu(msg->hdr.count);
- u32 msg_len = le32_to_cpu(msg->hdr.len);
- u32 len = 0;
- int j;
-
- for (j = 0; j < msg_count && len < msg_len; ++j) {
- struct wire_trans_hdr *trans_hdr;
-
- trans_hdr = (struct wire_trans_hdr *)(msg->data + len);
- if (le32_to_cpu(trans_hdr->type) == QAIC_TRANS_DEACTIVATE_FROM_DEV) {
- if (decode_deactivate(qdev, trans_hdr, &len, NULL))
- len += le32_to_cpu(trans_hdr->len);
- } else {
- len += le32_to_cpu(trans_hdr->len);
- }
- }
+ decode_message(qdev, NULL, msg, NULL, NULL, true);
/* request must have timed out, drop packet */
kfree(msg);
}
diff --git a/drivers/block/sunvdc.c b/drivers/block/sunvdc.c
index 020bd9f1a7b6..24ad56536ed6 100644
--- a/drivers/block/sunvdc.c
+++ b/drivers/block/sunvdc.c
@@ -525,6 +525,23 @@ static int __send_request(struct request *req)
err = __vdc_tx_trigger(port);
if (err < 0) {
printk(KERN_ERR PFX "vdc_tx_trigger() failure, err=%d\n", err);
+ /*
+ * If the port was reset (-ENOTCONN), the dring and the
+ * LDC channel including all of its mappings are already
+ * torn down and reallocated - there is nothing to undo
+ * and @desc must not be touched.
+ *
+ * For any other failure the descriptor was never handed
+ * to the peer: unmap the cookies and free the descriptor
+ * again, so that a later retry of the request does not
+ * leak LDC map table entries.
+ */
+ if (err != -ENOTCONN) {
+ ldc_unmap(port->vio.lp, desc->cookies,
+ desc->ncookies);
+ desc->hdr.state = VIO_DESC_FREE;
+ rqe->req = NULL;
+ }
} else {
port->req_id++;
dr->prod = vio_dring_next(dr, dr->prod);
diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
index b71283588ea7..d879e8492d59 100644
--- a/drivers/block/ublk_drv.c
+++ b/drivers/block/ublk_drv.c
@@ -3040,6 +3040,7 @@ static void ublk_queue_reset_io_flags(struct ublk_queue *ubq)
ubq->canceling = false;
spin_unlock(&ubq->cancel_lock);
ubq->fail_io = false;
+ ubq->force_abort = false;
}
/* device can only be started after all IOs are ready */
diff --git a/drivers/bluetooth/btintel.c b/drivers/bluetooth/btintel.c
index cc2234470960..96a55d5badda 100644
--- a/drivers/bluetooth/btintel.c
+++ b/drivers/bluetooth/btintel.c
@@ -570,12 +570,44 @@ int btintel_version_info_tlv(struct hci_dev *hdev,
}
EXPORT_SYMBOL_GPL(btintel_version_info_tlv);
+static u8 btintel_version_tlv_min_len(u8 type)
+{
+ switch (type) {
+ case INTEL_TLV_CNVI_TOP:
+ case INTEL_TLV_CNVR_TOP:
+ case INTEL_TLV_CNVI_BT:
+ case INTEL_TLV_CNVR_BT:
+ case INTEL_TLV_BUILD_NUM:
+ case INTEL_TLV_GIT_SHA1:
+ return sizeof(u32);
+ case INTEL_TLV_DEV_REV_ID:
+ case INTEL_TLV_TIME_STAMP:
+ return sizeof(u16);
+ case INTEL_TLV_IMAGE_TYPE:
+ case INTEL_TLV_BUILD_TYPE:
+ case INTEL_TLV_SECURE_BOOT:
+ case INTEL_TLV_OTP_LOCK:
+ case INTEL_TLV_API_LOCK:
+ case INTEL_TLV_DEBUG_LOCK:
+ case INTEL_TLV_LIMITED_CCE:
+ case INTEL_TLV_SBE_TYPE:
+ return sizeof(u8);
+ case INTEL_TLV_MIN_FW:
+ return 3;
+ case INTEL_TLV_OTP_BDADDR:
+ return sizeof(bdaddr_t);
+ default:
+ return 0;
+ }
+}
+
int btintel_parse_version_tlv(struct hci_dev *hdev,
struct intel_version_tlv *version,
struct sk_buff *skb)
{
/* Consume Command Complete Status field */
- skb_pull(skb, 1);
+ if (!skb_pull(skb, 1))
+ return -EINVAL;
/* Event parameters contain multiple TLVs. Read each of them
* and only keep the required data. Also, it use existing legacy
@@ -595,6 +627,9 @@ int btintel_parse_version_tlv(struct hci_dev *hdev,
if (skb->len < tlv->len + sizeof(*tlv))
return -EINVAL;
+ if (tlv->len < btintel_version_tlv_min_len(tlv->type))
+ return -EINVAL;
+
switch (tlv->type) {
case INTEL_TLV_CNVI_TOP:
version->cnvi_top = get_unaligned_le32(tlv->val);
@@ -666,7 +701,7 @@ int btintel_parse_version_tlv(struct hci_dev *hdev,
break;
case INTEL_TLV_FW_ID:
snprintf(version->fw_id, sizeof(version->fw_id),
- "%s", tlv->val);
+ "%.*s", tlv->len, tlv->val);
break;
default:
/* Ignore rest of information */
diff --git a/drivers/bluetooth/btintel_pcie.c b/drivers/bluetooth/btintel_pcie.c
index 2b7231be5973..c64e96fe8897 100644
--- a/drivers/bluetooth/btintel_pcie.c
+++ b/drivers/bluetooth/btintel_pcie.c
@@ -1099,7 +1099,7 @@ static void btintel_pcie_msix_tx_handle(struct btintel_pcie_data *data)
urbd0 = &txq->urbd0s[cr_tia];
- if (urbd0->tfd_index > txq->count)
+ if (urbd0->tfd_index >= txq->count)
return;
cr_tia = (cr_tia + 1) % txq->count;
@@ -1537,7 +1537,9 @@ static int btintel_pcie_submit_rx_work(struct btintel_pcie_data *data, u8 status
rfh_hdr = buf;
len = rfh_hdr->packet_len;
- if (len <= 0) {
+ if (len == 0 || len > BTINTEL_PCIE_BUFFER_SIZE - sizeof(*rfh_hdr)) {
+ bt_dev_err(data->hdev, "Invalid packet_len %d (max %zu)", len,
+ BTINTEL_PCIE_BUFFER_SIZE - sizeof(*rfh_hdr));
ret = -EINVAL;
goto resubmit;
}
@@ -1634,6 +1636,9 @@ static irqreturn_t btintel_pcie_irq_msix_handler(int irq, void *dev_id)
if (unlikely(!(intr_fh | intr_hw))) {
/* Ignore interrupt, inta == 0 */
+ bt_warn_ratelimited("Bluetooth: btintel_pcie: Received spurious interrupt\n");
+ btintel_pcie_wr_reg32(data, BTINTEL_PCIE_CSR_MSIX_AUTOMASK_ST,
+ BIT(entry->entry));
return IRQ_NONE;
}
diff --git a/drivers/bluetooth/btmtk.c b/drivers/bluetooth/btmtk.c
index c0ed51567ed4..26d525acd659 100644
--- a/drivers/bluetooth/btmtk.c
+++ b/drivers/bluetooth/btmtk.c
@@ -1374,16 +1374,6 @@ int btmtk_usb_setup(struct hci_dev *hdev)
break;
case 0x7922:
case 0x7925:
- /*
- * A remote wakeup could cause the device completely unresponsive, and
- * recovering from such a state needs a power cycle.
- *
- * Since the remote wakeup capability is super broken, just disable it
- * to get rid of the troubles. The device can still be autosuspended
- * when the bluetooth interface is closed.
- */
- device_set_wakeup_capable(&btmtk_data->udev->dev, false);
- fallthrough;
case 0x7961:
case 0x7902:
case 0x6639:
@@ -1587,5 +1577,6 @@ MODULE_FIRMWARE(FIRMWARE_MT7663);
MODULE_FIRMWARE(FIRMWARE_MT7668);
MODULE_FIRMWARE(FIRMWARE_MT7922);
MODULE_FIRMWARE(FIRMWARE_MT7961);
+MODULE_FIRMWARE(FIRMWARE_MT7920);
MODULE_FIRMWARE(FIRMWARE_MT7925);
MODULE_FIRMWARE(FIRMWARE_MT7927);
diff --git a/drivers/bluetooth/btmtk.h b/drivers/bluetooth/btmtk.h
index c83c24897c95..bc26148ec544 100644
--- a/drivers/bluetooth/btmtk.h
+++ b/drivers/bluetooth/btmtk.h
@@ -7,6 +7,7 @@
#define FIRMWARE_MT7922 "mediatek/BT_RAM_CODE_MT7922_1_1_hdr.bin"
#define FIRMWARE_MT7902 "mediatek/BT_RAM_CODE_MT7902_1_1_hdr.bin"
#define FIRMWARE_MT7961 "mediatek/BT_RAM_CODE_MT7961_1_2_hdr.bin"
+#define FIRMWARE_MT7920 "mediatek/BT_RAM_CODE_MT7961_1a_2_hdr.bin"
#define FIRMWARE_MT7925 "mediatek/mt7925/BT_RAM_CODE_MT7925_1_1_hdr.bin"
#define FIRMWARE_MT7927 "mediatek/mt7927/BT_RAM_CODE_MT6639_2_1_hdr.bin"
diff --git a/drivers/bluetooth/btqcomsmd.c b/drivers/bluetooth/btqcomsmd.c
index d2e13fcb6bab..d669ea4eb3eb 100644
--- a/drivers/bluetooth/btqcomsmd.c
+++ b/drivers/bluetooth/btqcomsmd.c
@@ -188,7 +188,10 @@ static int btqcomsmd_probe(struct platform_device *pdev)
return 0;
hci_free_dev:
+ rpmsg_destroy_ept(btq->cmd_channel);
+ rpmsg_destroy_ept(btq->acl_channel);
hci_free_dev(hdev);
+ return ret;
destroy_cmd_channel:
rpmsg_destroy_ept(btq->cmd_channel);
destroy_acl_channel:
@@ -202,10 +205,11 @@ static void btqcomsmd_remove(struct platform_device *pdev)
struct btqcomsmd *btq = platform_get_drvdata(pdev);
hci_unregister_dev(btq->hdev);
- hci_free_dev(btq->hdev);
rpmsg_destroy_ept(btq->cmd_channel);
rpmsg_destroy_ept(btq->acl_channel);
+
+ hci_free_dev(btq->hdev);
}
static const struct of_device_id btqcomsmd_of_match[] = {
diff --git a/drivers/bluetooth/btrtl.c b/drivers/bluetooth/btrtl.c
index 03fa9409e3ee..d29813331603 100644
--- a/drivers/bluetooth/btrtl.c
+++ b/drivers/bluetooth/btrtl.c
@@ -591,7 +591,7 @@ static int rtlbt_parse_firmware_v2(struct hci_dev *hdev,
* headers.
*/
if (!key_id)
- break;
+ continue;
rc = btrtl_parse_section(hdev, btrtl_dev, opcode,
ptr, section_len);
break;
@@ -600,8 +600,7 @@ static int rtlbt_parse_firmware_v2(struct hci_dev *hdev,
ptr, section_len);
break;
default:
- rc = 0;
- break;
+ continue;
}
if (rc < 0) {
rtl_dev_err(hdev, "RTL: Parse section (%u) err %d",
diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c
index a3d6d3194b15..84f55f67523e 100644
--- a/drivers/bluetooth/btusb.c
+++ b/drivers/bluetooth/btusb.c
@@ -6,6 +6,7 @@
* Copyright (C) 2005-2008 Marcel Holtmann <marcel@holtmann.org>
*/
+#include <linux/cpufeature.h>
#include <linux/dmi.h>
#include <linux/module.h>
#include <linux/usb.h>
@@ -968,6 +969,8 @@ struct qca_dump_info {
#define BTUSB_USE_ALT3_FOR_WBS 15
#define BTUSB_ALT6_CONTINUOUS_TX 16
#define BTUSB_HW_SSR_ACTIVE 17
+#define BTUSB_WAKEUP_BROKEN 18
+#define BTUSB_RESET 19
struct btusb_data {
struct hci_dev *hdev;
@@ -1044,13 +1047,15 @@ static void btusb_reset(struct hci_dev *hdev)
int err;
data = hci_get_drvdata(hdev);
- /* This is not an unbalanced PM reference since the device will reset */
err = usb_autopm_get_interface(data->intf);
if (err) {
bt_dev_err(hdev, "Failed usb_autopm_get_interface: %d", err);
return;
}
+ if (test_and_set_bit(BTUSB_RESET, &data->flags))
+ usb_autopm_put_interface_no_suspend(data->intf);
+
bt_dev_err(hdev, "Resetting usb device.");
usb_queue_reset_device(data->intf);
}
@@ -2072,18 +2077,24 @@ static int btusb_close(struct hci_dev *hdev)
BT_DBG("%s", hdev->name);
- cancel_delayed_work(&data->rx_work);
cancel_work_sync(&data->work);
cancel_work_sync(&data->waker);
- skb_queue_purge(&data->acl_q);
-
clear_bit(BTUSB_ISOC_RUNNING, &data->flags);
clear_bit(BTUSB_BULK_RUNNING, &data->flags);
clear_bit(BTUSB_INTR_RUNNING, &data->flags);
clear_bit(BTUSB_DIAG_RUNNING, &data->flags);
btusb_stop_traffic(data);
+
+ /* rx_work must only be canceled once the URBs that can rearm it are
+ * gone, and it must be canceled synchronously since btusb_disconnect()
+ * frees the btusb_data it dereferences right after hci_unregister_dev().
+ */
+ cancel_delayed_work_sync(&data->rx_work);
+
+ skb_queue_purge(&data->acl_q);
+
btusb_free_frags(data);
err = usb_autopm_get_interface(data->intf);
@@ -2109,7 +2120,7 @@ static int btusb_flush(struct hci_dev *hdev)
BT_DBG("%s", hdev->name);
- cancel_delayed_work(&data->rx_work);
+ cancel_delayed_work_sync(&data->rx_work);
skb_queue_purge(&data->acl_q);
@@ -2903,8 +2914,11 @@ static int btusb_mtk_reset(struct hci_dev *hdev, void *rst_data)
}
err = usb_autopm_get_interface(data->intf);
- if (err < 0)
+ if (err < 0) {
+ bt_dev_err(hdev, "Failed usb_autopm_get_interface: %d", err);
+ clear_bit(BTMTK_HW_RESET_ACTIVE, &btmtk_data->flags);
return err;
+ }
/* Release MediaTek ISO data interface */
btusb_mtk_release_iso_intf(hdev);
@@ -2927,6 +2941,11 @@ static int btusb_mtk_reset(struct hci_dev *hdev, void *rst_data)
err = btmtk_usb_subsys_reset(hdev, btmtk_data->dev_id);
+ if (test_and_set_bit(BTUSB_RESET, &data->flags)) {
+ bt_dev_err(hdev, "last usb reset failed? Resetting again");
+ usb_autopm_put_interface_no_suspend(data->intf);
+ }
+
usb_queue_reset_device(data->intf);
clear_bit(BTMTK_HW_RESET_ACTIVE, &btmtk_data->flags);
@@ -2950,10 +2969,25 @@ static int btusb_send_frame_mtk(struct hci_dev *hdev, struct sk_buff *skb)
}
}
+static inline bool platform_is_ryzen(void)
+{
+#ifdef CONFIG_X86
+ return boot_cpu_has(X86_FEATURE_ZEN);
+#else
+ return false;
+#endif
+}
+
+static inline bool is_direct_child_of_root_hub(struct usb_device *udev)
+{
+ return udev->parent == udev->bus->root_hub;
+}
+
static int btusb_mtk_setup(struct hci_dev *hdev)
{
struct btusb_data *data = hci_get_drvdata(hdev);
struct btmtk_data *btmtk_data = hci_get_priv(hdev);
+ int err;
/* MediaTek WMT vendor cmd requiring below USB resources to
* complete the handshake.
@@ -2970,7 +3004,40 @@ static int btusb_mtk_setup(struct hci_dev *hdev)
btusb_mtk_claim_iso_intf(data);
}
- return btmtk_usb_setup(hdev);
+ err = btmtk_usb_setup(hdev);
+ if (err)
+ return err;
+
+ switch (btmtk_data->dev_id) {
+ case 0x7922:
+ case 0x7925:
+ /*
+ * All reports seen to be relevant to Ryzen-based laptops. These
+ * NICs are usually used as OEM components thanks to some sort
+ * of reference designs.
+ *
+ * Their popularity on other platforms is unclear. While there
+ * is still a chance that the quirk may exist on other
+ * platforms, be cautious and only apply the quirk to direct
+ * children of Ryzen platforms's root hubs for the time being.
+ *
+ * In most cases the root hub is on the SoC or PCH, which needs
+ * the quirk. Unfortunately, this can't distinguish root hubs on
+ * PCIe add-in cards. Such roughness should be acceptable, as
+ * PCIe USB controller add-in cards are less commonly used
+ * nowadays. On the other hand, applying the quirk doesn't hurt
+ * any functionalities either, as the device can still be used
+ * as a wakeup source if desired.
+ *
+ * Theoretically, we could retrieve the root hub's PCI vendor ID
+ * with some hierarchy magic, but that's too intrusive...
+ */
+ if (platform_is_ryzen() && is_direct_child_of_root_hub(data->udev))
+ set_bit(BTUSB_WAKEUP_BROKEN, &data->flags);
+ break;
+ }
+
+ return 0;
}
static int btusb_mtk_shutdown(struct hci_dev *hdev)
@@ -4516,6 +4583,9 @@ static void btusb_disconnect(struct usb_interface *intf)
if (data->reset_gpio)
gpiod_put(data->reset_gpio);
+ if (test_and_clear_bit(BTUSB_RESET, &data->flags))
+ usb_autopm_put_interface_no_suspend(data->intf);
+
if (intf == data->intf) {
if (data->isoc)
usb_driver_release_interface(&btusb_driver, data->isoc);
@@ -4541,11 +4611,26 @@ static int btusb_suspend(struct usb_interface *intf, pm_message_t message)
BT_DBG("intf %p", intf);
- /* Don't auto-suspend if there are connections or discovery in
- * progress; external suspend calls shall never fail.
+ /*
+ * It is reported that remote wakeup events could sometimes cause some
+ * adapters completely unresponsive. Resetting the xHCI root hub doesn't
+ * help at all, and recovering from such a state needs a power cycle.
+ * Since disabling remote wakeup simply causes the USB core to gate
+ * runtime autosuspend as well due to needs_remote_wakeup == 1, let's do
+ * this ourselves to make our life easier. The interface can be safely
+ * autosuspended as long as remote wakeup is disabled, i.e., after
+ * closing the HCI device.
+ *
+ * Don't auto-suspend if there are connections or discovery in progress.
+ *
+ * External suspend calls shall never fail. Specifically, a device with
+ * broken remote wakeup may still take the advantage of remote wakeup in
+ * order to wake up the system from sleep if userspace has enabled it as
+ * a wakeup source.
*/
if (PMSG_IS_AUTO(message) &&
- (hci_conn_count(data->hdev) || hci_discovery_active(data->hdev)))
+ ((test_bit(BTUSB_WAKEUP_BROKEN, &data->flags) && data->intf->needs_remote_wakeup) ||
+ hci_conn_count(data->hdev) || hci_discovery_active(data->hdev)))
return -EBUSY;
if (data->suspend_count++)
diff --git a/drivers/bluetooth/hci_mrvl.c b/drivers/bluetooth/hci_mrvl.c
index 516b8f74c434..5798a8db016e 100644
--- a/drivers/bluetooth/hci_mrvl.c
+++ b/drivers/bluetooth/hci_mrvl.c
@@ -307,9 +307,8 @@ static int mrvl_load_firmware(struct hci_dev *hdev, const char *name)
err = wait_on_bit_timeout(&mrvl->flags, STATE_FW_REQ_PENDING,
TASK_INTERRUPTIBLE,
msecs_to_jiffies(2000));
- if (err == 1) {
+ if (err == -EINTR) {
bt_dev_err(hdev, "Firmware load interrupted");
- err = -EINTR;
break;
} else if (err) {
bt_dev_err(hdev, "Firmware request timeout");
diff --git a/drivers/char/virtio_console.c b/drivers/char/virtio_console.c
index 198b97314168..1ec5b25d1a9a 100644
--- a/drivers/char/virtio_console.c
+++ b/drivers/char/virtio_console.c
@@ -1891,13 +1891,28 @@ static const struct file_operations portdev_fops = {
static void remove_vqs(struct ports_device *portdev)
{
struct virtqueue *vq;
+ bool multiport = use_multiport(portdev);
virtio_device_for_each_vq(portdev->vdev, vq) {
struct port_buffer *buf;
+ unsigned int len;
- flush_bufs(vq, true);
- while ((buf = virtqueue_detach_unused_buf(vq)))
- free_buf(buf, true);
+ /*
+ * c_ovq cookies are &portdev->cpkt, not port_buffer.
+ * Detach them but do not free_buf().
+ */
+ if (multiport && vq == portdev->c_ovq) {
+ spin_lock(&portdev->c_ovq_lock);
+ while (virtqueue_get_buf(vq, &len))
+ ;
+ while (virtqueue_detach_unused_buf(vq))
+ ;
+ spin_unlock(&portdev->c_ovq_lock);
+ } else {
+ flush_bufs(vq, true);
+ while ((buf = virtqueue_detach_unused_buf(vq)))
+ free_buf(buf, true);
+ }
cond_resched();
}
portdev->vdev->config->del_vqs(portdev->vdev);
diff --git a/drivers/clk/ux500/clk-prcmu.c b/drivers/clk/ux500/clk-prcmu.c
index ddc86551bf57..ac96c46bd1bb 100644
--- a/drivers/clk/ux500/clk-prcmu.c
+++ b/drivers/clk/ux500/clk-prcmu.c
@@ -7,7 +7,7 @@
*/
#include <linux/clk-provider.h>
-#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/mfd/db8500-prcmu.h>
#include <linux/slab.h>
#include <linux/io.h>
#include <linux/err.h>
@@ -35,13 +35,13 @@ static int clk_prcmu_prepare(struct clk_hw *hw)
{
struct clk_prcmu *clk = to_clk_prcmu(hw);
- return prcmu_request_clock(clk->cg_sel, true);
+ return db8500_prcmu_request_clock(clk->cg_sel, true);
}
static void clk_prcmu_unprepare(struct clk_hw *hw)
{
struct clk_prcmu *clk = to_clk_prcmu(hw);
- if (prcmu_request_clock(clk->cg_sel, false))
+ if (db8500_prcmu_request_clock(clk->cg_sel, false))
pr_err("clk_prcmu: %s failed to disable %s.\n", __func__,
clk_hw_get_name(hw));
}
@@ -86,7 +86,7 @@ static int clk_prcmu_opp_prepare(struct clk_hw *hw)
clk->opp_requested = 1;
}
- err = prcmu_request_clock(clk->cg_sel, true);
+ err = db8500_prcmu_request_clock(clk->cg_sel, true);
if (err) {
prcmu_qos_remove_requirement(PRCMU_QOS_APE_OPP,
(char *)clk_hw_get_name(hw));
@@ -101,7 +101,7 @@ static void clk_prcmu_opp_unprepare(struct clk_hw *hw)
{
struct clk_prcmu *clk = to_clk_prcmu(hw);
- if (prcmu_request_clock(clk->cg_sel, false)) {
+ if (db8500_prcmu_request_clock(clk->cg_sel, false)) {
pr_err("clk_prcmu: %s failed to disable %s.\n", __func__,
clk_hw_get_name(hw));
return;
@@ -120,7 +120,7 @@ static int clk_prcmu_opp_volt_prepare(struct clk_hw *hw)
struct clk_prcmu *clk = to_clk_prcmu(hw);
if (!clk->opp_requested) {
- err = prcmu_request_ape_opp_100_voltage(true);
+ err = db8500_prcmu_request_ape_opp_100_voltage(true);
if (err) {
pr_err("clk_prcmu: %s fail req APE OPP VOLT for %s.\n",
__func__, clk_hw_get_name(hw));
@@ -129,9 +129,9 @@ static int clk_prcmu_opp_volt_prepare(struct clk_hw *hw)
clk->opp_requested = 1;
}
- err = prcmu_request_clock(clk->cg_sel, true);
+ err = db8500_prcmu_request_clock(clk->cg_sel, true);
if (err) {
- prcmu_request_ape_opp_100_voltage(false);
+ db8500_prcmu_request_ape_opp_100_voltage(false);
clk->opp_requested = 0;
return err;
}
@@ -143,14 +143,14 @@ static void clk_prcmu_opp_volt_unprepare(struct clk_hw *hw)
{
struct clk_prcmu *clk = to_clk_prcmu(hw);
- if (prcmu_request_clock(clk->cg_sel, false)) {
+ if (db8500_prcmu_request_clock(clk->cg_sel, false)) {
pr_err("clk_prcmu: %s failed to disable %s.\n", __func__,
clk_hw_get_name(hw));
return;
}
if (clk->opp_requested) {
- prcmu_request_ape_opp_100_voltage(false);
+ db8500_prcmu_request_ape_opp_100_voltage(false);
clk->opp_requested = 0;
}
}
diff --git a/drivers/clk/ux500/u8500_of_clk.c b/drivers/clk/ux500/u8500_of_clk.c
index 6f78808387b1..d2499815226f 100644
--- a/drivers/clk/ux500/u8500_of_clk.c
+++ b/drivers/clk/ux500/u8500_of_clk.c
@@ -9,7 +9,7 @@
#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/clk-provider.h>
-#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/mfd/db8500-prcmu.h>
#include "clk.h"
#include "prcc.h"
diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
index b898b6544069..196c50f26fe0 100644
--- a/drivers/cpufreq/cpufreq.c
+++ b/drivers/cpufreq/cpufreq.c
@@ -1249,7 +1249,7 @@ static struct cpufreq_policy *cpufreq_policy_alloc(unsigned int cpu)
if (!policy)
return NULL;
- if (!alloc_cpumask_var(&policy->cpus, GFP_KERNEL))
+ if (!zalloc_cpumask_var(&policy->cpus, GFP_KERNEL))
goto err_free_policy;
if (!zalloc_cpumask_var(&policy->related_cpus, GFP_KERNEL))
@@ -1258,6 +1258,8 @@ static struct cpufreq_policy *cpufreq_policy_alloc(unsigned int cpu)
if (!zalloc_cpumask_var(&policy->real_cpus, GFP_KERNEL))
goto err_free_rcpumask;
+ init_rwsem(&policy->rwsem);
+
init_completion(&policy->kobj_unregister);
ret = kobject_init_and_add(&policy->kobj, &ktype_cpufreq,
cpufreq_global_kobject, "policy%u", cpu);
@@ -1272,8 +1274,6 @@ static struct cpufreq_policy *cpufreq_policy_alloc(unsigned int cpu)
goto err_free_real_cpus;
}
- init_rwsem(&policy->rwsem);
-
freq_constraints_init(&policy->constraints);
policy->nb_min.notifier_call = cpufreq_notifier_min;
diff --git a/drivers/cpuidle/cpuidle-ux500.c b/drivers/cpuidle/cpuidle-ux500.c
index f7d778580e9b..6d6c52c0bcc2 100644
--- a/drivers/cpuidle/cpuidle-ux500.c
+++ b/drivers/cpuidle/cpuidle-ux500.c
@@ -11,7 +11,7 @@
#include <linux/spinlock.h>
#include <linux/atomic.h>
#include <linux/smp.h>
-#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/mfd/db8500-prcmu.h>
#include <linux/platform_data/arm-ux500-pm.h>
#include <linux/platform_device.h>
@@ -66,7 +66,7 @@ static inline int ux500_enter_idle(struct cpuidle_device *dev,
/* Go to the retention state, the prcmu will wait for the
* cpu to go WFI and this is what happens after exiting this
* 'master' critical section */
- if (prcmu_set_power_state(PRCMU_AP_IDLE, true, true))
+ if (db8500_prcmu_set_power_state(PRCMU_AP_IDLE, true, true))
goto out;
/* When we switch to retention, the prcmu is in charge
@@ -109,7 +109,7 @@ static struct cpuidle_driver ux500_idle_driver = {
static int dbx500_cpuidle_probe(struct platform_device *pdev)
{
/* Configure wake up reasons */
- prcmu_enable_wakeups(PRCMU_WAKEUP(ARM) | PRCMU_WAKEUP(RTC) |
+ db8500_prcmu_enable_wakeups(PRCMU_WAKEUP(ARM) | PRCMU_WAKEUP(RTC) |
PRCMU_WAKEUP(ABB));
return cpuidle_register(&ux500_idle_driver, NULL);
diff --git a/drivers/dibs/dibs_main.c b/drivers/dibs/dibs_main.c
index 2b53a9d277dc..20c50997a7cf 100644
--- a/drivers/dibs/dibs_main.c
+++ b/drivers/dibs/dibs_main.c
@@ -251,13 +251,19 @@ static int __init dibs_init(void)
rc = class_register(&dibs_class);
if (rc)
- return rc;
+ goto err;
rc = dibs_loopback_init();
if (rc)
- pr_err("%s fails with %d\n", __func__, rc);
+ goto err_unregister;
return rc;
+
+err_unregister:
+ class_unregister(&dibs_class);
+err:
+ pr_err("%s fails with %d\n", __func__, rc);
+ return rc;
}
static void __exit dibs_exit(void)
diff --git a/drivers/edac/altera_edac.c b/drivers/edac/altera_edac.c
index 4edd2088c2db..5914b2fd94d9 100644
--- a/drivers/edac/altera_edac.c
+++ b/drivers/edac/altera_edac.c
@@ -1533,7 +1533,7 @@ static int altr_portb_setup(struct altr_edac_device_dev *device)
altdev = dci->pvt_info;
*altdev = *device;
- if (!devres_open_group(&altdev->ddev, altr_portb_setup, GFP_KERNEL))
+ if (!devres_open_group(device->edac->dev, altr_portb_setup, GFP_KERNEL))
return -ENOMEM;
/* Update PortB specific values */
@@ -1562,7 +1562,7 @@ static int altr_portb_setup(struct altr_edac_device_dev *device)
rc = -ENODEV;
goto err_release_group_1;
}
- rc = devm_request_irq(&altdev->ddev, altdev->sb_irq,
+ rc = devm_request_irq(device->edac->dev, altdev->sb_irq,
prv->ecc_irq_handler, IRQF_TRIGGER_HIGH,
ecc_name, altdev);
if (rc) {
@@ -1585,7 +1585,7 @@ static int altr_portb_setup(struct altr_edac_device_dev *device)
rc = -ENODEV;
goto err_release_group_1;
}
- rc = devm_request_irq(&altdev->ddev, altdev->db_irq,
+ rc = devm_request_irq(device->edac->dev, altdev->db_irq,
prv->ecc_irq_handler, IRQF_TRIGGER_HIGH,
ecc_name, altdev);
if (rc) {
@@ -1605,13 +1605,13 @@ static int altr_portb_setup(struct altr_edac_device_dev *device)
list_add(&altdev->next, &altdev->edac->a10_ecc_devices);
- devres_remove_group(&altdev->ddev, altr_portb_setup);
+ devres_remove_group(device->edac->dev, altr_portb_setup);
return 0;
err_release_group_1:
edac_device_free_ctl_info(dci);
- devres_release_group(&altdev->ddev, altr_portb_setup);
+ devres_release_group(device->edac->dev, altr_portb_setup);
edac_printk(KERN_ERR, EDAC_DEVICE,
"%s:Error setting up EDAC device: %d\n", ecc_name, rc);
return rc;
diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index c6aa69dbd9fb..475235c402e8 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -4173,6 +4173,8 @@ static int __init amd64_edac_init(void)
goto err_pci;
}
+ request_module_nowait("amd_atl");
+
/* register stuff with EDAC MCE */
if (boot_cpu_data.x86 >= 0x17) {
amd_register_ecc_decoder(decode_umc_error);
diff --git a/drivers/edac/edac_device_sysfs.c b/drivers/edac/edac_device_sysfs.c
index b1c2717cd023..6995ce039db9 100644
--- a/drivers/edac/edac_device_sysfs.c
+++ b/drivers/edac/edac_device_sysfs.c
@@ -88,14 +88,21 @@ static ssize_t edac_device_ctl_poll_msec_store(struct edac_device_ctl_info
*ctl_info, const char *data,
size_t count)
{
- unsigned long value;
+ unsigned int value;
+ int ret;
/* get the value and enforce that it is non-zero, must be at least
* one millisecond for the delay period, between scans
* Then cancel last outstanding delay for the work request
* and set a new one.
*/
- value = simple_strtoul(data, NULL, 0);
+ ret = kstrtouint(data, 0, &value);
+ if (ret < 0)
+ return ret;
+
+ if (value < 1)
+ return -EINVAL;
+
edac_device_reset_delay_period(ctl_info, value);
return count;
diff --git a/drivers/edac/igen6_edac.c b/drivers/edac/igen6_edac.c
index f1fc20d4ebf6..12d718a50e1c 100644
--- a/drivers/edac/igen6_edac.c
+++ b/drivers/edac/igen6_edac.c
@@ -175,8 +175,6 @@ static struct res_config {
/* Set imc->dimm_{l_size,s_size,l_map}[chan]. */
void (*set_dimm_params)(struct igen6_imc *imc, int chan);
bool (*ibecc_available)(struct pci_dev *pdev);
- /* Extract error address logged in IBECC */
- u64 (*err_addr)(u64 ecclog);
/* Convert error address logged in IBECC to system physical address */
u64 (*err_addr_to_sys_addr)(u64 eaddr, int mc);
/* Convert error address logged in IBECC to integrated memory controller address */
@@ -522,11 +520,6 @@ static u64 adl_err_addr_to_imc_addr(u64 eaddr, int mc)
return imc_addr;
}
-static u64 rpl_p_err_addr(u64 ecclog)
-{
- return field_get(res_cfg->reg_eccerrlog_addr_mask, ecclog);
-}
-
static enum mem_type ptl_h_get_mem_type(struct igen6_imc *imc)
{
u32 mtype, val;
@@ -716,22 +709,6 @@ static struct res_config adl_n_cfg = {
.err_addr_to_imc_addr = adl_err_addr_to_imc_addr,
};
-static struct res_config rpl_p_cfg = {
- .machine_check = true,
- .num_imc = 2,
- .reg_mchbar_mask = GENMASK_ULL(41, 17),
- .reg_tom_mask = GENMASK_ULL(41, 20),
- .reg_touud_mask = GENMASK_ULL(41, 20),
- .reg_eccerrlog_addr_mask = GENMASK_ULL(45, 5),
- .imc_base = 0xd800,
- .ibecc_base = 0xd400,
- .ibecc_error_log_offset = 0x68,
- .ibecc_available = tgl_ibecc_available,
- .err_addr = rpl_p_err_addr,
- .err_addr_to_sys_addr = adl_err_addr_to_sys_addr,
- .err_addr_to_imc_addr = adl_err_addr_to_imc_addr,
-};
-
static struct res_config mtl_ps_cfg = {
.machine_check = true,
.num_imc = 2,
@@ -877,11 +854,11 @@ static struct pci_device_id igen6_pci_tbl[] = {
{ PCI_VDEVICE(INTEL, DID_ASL_SKU1), .driver_data = (kernel_ulong_t)&adl_n_cfg },
{ PCI_VDEVICE(INTEL, DID_ASL_SKU2), .driver_data = (kernel_ulong_t)&adl_n_cfg },
{ PCI_VDEVICE(INTEL, DID_ASL_SKU3), .driver_data = (kernel_ulong_t)&adl_n_cfg },
- { PCI_VDEVICE(INTEL, DID_RPL_P_SKU1), .driver_data = (kernel_ulong_t)&rpl_p_cfg },
- { PCI_VDEVICE(INTEL, DID_RPL_P_SKU2), .driver_data = (kernel_ulong_t)&rpl_p_cfg },
- { PCI_VDEVICE(INTEL, DID_RPL_P_SKU3), .driver_data = (kernel_ulong_t)&rpl_p_cfg },
- { PCI_VDEVICE(INTEL, DID_RPL_P_SKU4), .driver_data = (kernel_ulong_t)&rpl_p_cfg },
- { PCI_VDEVICE(INTEL, DID_RPL_P_SKU5), .driver_data = (kernel_ulong_t)&rpl_p_cfg },
+ { PCI_VDEVICE(INTEL, DID_RPL_P_SKU1), .driver_data = (kernel_ulong_t)&adl_cfg },
+ { PCI_VDEVICE(INTEL, DID_RPL_P_SKU2), .driver_data = (kernel_ulong_t)&adl_cfg },
+ { PCI_VDEVICE(INTEL, DID_RPL_P_SKU3), .driver_data = (kernel_ulong_t)&adl_cfg },
+ { PCI_VDEVICE(INTEL, DID_RPL_P_SKU4), .driver_data = (kernel_ulong_t)&adl_cfg },
+ { PCI_VDEVICE(INTEL, DID_RPL_P_SKU5), .driver_data = (kernel_ulong_t)&adl_cfg },
{ PCI_VDEVICE(INTEL, DID_MTL_PS_SKU1), .driver_data = (kernel_ulong_t)&mtl_ps_cfg },
{ PCI_VDEVICE(INTEL, DID_MTL_PS_SKU2), .driver_data = (kernel_ulong_t)&mtl_ps_cfg },
{ PCI_VDEVICE(INTEL, DID_MTL_PS_SKU3), .driver_data = (kernel_ulong_t)&mtl_ps_cfg },
@@ -1009,14 +986,22 @@ static void set_dimm_params(struct igen6_imc *imc, int chan)
static int decode_chan_idx(u64 addr, u64 mask, int intlv_bit)
{
- u64 hash_addr = addr & mask, hash = 0;
- u64 intlv = (addr >> intlv_bit) & 1;
+ u64 hash_addr, hash = 0;
int i;
+ /*
+ * In hash mode, the @intlv_bit is the lowest selected bit of @addr
+ * to be XORed. While @mask may or may not include this @intlv_bit,
+ * we enforce that @mask includes @intlv_bit to ensure @intlv_bit is
+ * XORed exactly once.
+ */
+ mask |= 1 << intlv_bit;
+ hash_addr = addr & mask;
+
for (i = 6; i < 20; i++)
hash ^= (hash_addr >> i) & 1;
- return (int)hash ^ intlv;
+ return (int)hash;
}
static u64 decode_channel_addr(u64 addr, int intlv_bit)
@@ -1035,19 +1020,18 @@ static void decode_addr(u64 addr, u32 hash, u64 s_size, int l_map,
{
int intlv_bit = CHANNEL_HASH_LSB_MASK_BIT(hash) + 6;
- if (addr > 2 * s_size) {
+ if (addr >= 2 * s_size) {
*sub_addr = addr - s_size;
*idx = l_map;
return;
}
- if (CHANNEL_HASH_MODE(hash)) {
- *sub_addr = decode_channel_addr(addr, intlv_bit);
+ *sub_addr = decode_channel_addr(addr, intlv_bit);
+
+ if (CHANNEL_HASH_MODE(hash))
*idx = decode_chan_idx(addr, CHANNEL_HASH_MASK(hash), intlv_bit);
- } else {
- *sub_addr = decode_channel_addr(addr, 6);
- *idx = GET_BITFIELD(addr, 6, 6);
- }
+ else
+ *idx = GET_BITFIELD(addr, intlv_bit, intlv_bit);
}
static int igen6_decode(struct decoded_addr *res)
@@ -1230,11 +1214,7 @@ static void ecclog_work_cb(struct work_struct *work)
llist_for_each_entry_safe(node, tmp, head, llnode) {
memset(&res, 0, sizeof(res));
- if (res_cfg->err_addr)
- eaddr = res_cfg->err_addr(node->ecclog);
- else
- eaddr = node->ecclog & res_cfg->reg_eccerrlog_addr_mask;
-
+ eaddr = node->ecclog & res_cfg->reg_eccerrlog_addr_mask;
res.mc = node->mc;
res.sys_addr = res_cfg->err_addr_to_sys_addr(eaddr, res.mc);
res.imc_addr = res_cfg->err_addr_to_imc_addr(eaddr, res.mc);
diff --git a/drivers/gpu/drm/adp/Kconfig b/drivers/gpu/drm/adp/Kconfig
index 9fcc27eb200d..acfa21ee06d2 100644
--- a/drivers/gpu/drm/adp/Kconfig
+++ b/drivers/gpu/drm/adp/Kconfig
@@ -6,7 +6,6 @@ config DRM_ADP
select DRM_KMS_HELPER
select DRM_BRIDGE_CONNECTOR
select DRM_DISPLAY_HELPER
- select DRM_KMS_DMA_HELPER
select DRM_GEM_DMA_HELPER
select DRM_PANEL_BRIDGE
select VIDEOMODE_HELPERS
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index eaa86e32912e..ea416749138d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -190,6 +190,8 @@ amdgpu_ttm_job_submit(struct amdgpu_device *adev, struct amdgpu_ttm_buffer_entit
* @tmz: if we should setup a TMZ enabled mapping
* @size: in number of bytes to map, out number of bytes mapped
* @addr: resulting address inside the MC address space
+ * @vm_needs_flush: out, set true if a GART window was programmed (VMID 0 flush
+ * needed) or false for a direct address
*
* Setup one of the GART windows to access a specific piece of memory or return
* the physical address for local memory.
@@ -199,7 +201,8 @@ static int amdgpu_ttm_map_buffer(struct amdgpu_ttm_buffer_entity *entity,
struct ttm_resource *mem,
struct amdgpu_res_cursor *mm_cur,
unsigned int window,
- bool tmz, uint64_t *size, uint64_t *addr)
+ bool tmz, uint64_t *size, uint64_t *addr,
+ bool *vm_needs_flush)
{
struct amdgpu_device *adev = amdgpu_ttm_adev(bo->bdev);
unsigned int offset, num_pages, num_dw, num_bytes;
@@ -220,9 +223,12 @@ static int amdgpu_ttm_map_buffer(struct amdgpu_ttm_buffer_entity *entity,
if (!tmz && mem->start != AMDGPU_BO_INVALID_OFFSET) {
*addr = amdgpu_ttm_domain_start(adev, mem->mem_type) +
mm_cur->start;
+ *vm_needs_flush = false;
return 0;
}
+ /* A GART window is programmed below, so its VMID 0 TLB needs a flush */
+ *vm_needs_flush = true;
/*
* If start begins at an offset inside the page, then adjust the size
@@ -322,6 +328,7 @@ static int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
while (src_mm.remaining) {
uint64_t from, to, cur_size, tiling_flags;
uint32_t num_type, data_format, max_com, write_compress_disable;
+ bool src_vm_flush, dst_vm_flush;
struct dma_fence *next;
/* Never copy more than 256MiB at once to avoid a timeout */
@@ -329,12 +336,12 @@ static int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
/* Map src to window 0 and dst to window 1. */
r = amdgpu_ttm_map_buffer(entity, src->bo, src->mem, &src_mm,
- 0, tmz, &cur_size, &from);
+ 0, tmz, &cur_size, &from, &src_vm_flush);
if (r)
goto error;
r = amdgpu_ttm_map_buffer(entity, dst->bo, dst->mem, &dst_mm,
- 1, tmz, &cur_size, &to);
+ 1, tmz, &cur_size, &to, &dst_vm_flush);
if (r)
goto error;
@@ -362,7 +369,7 @@ static int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
}
r = amdgpu_copy_buffer(adev, entity, from, to, cur_size, resv,
- &next, true, copy_flags);
+ &next, src_vm_flush || dst_vm_flush, copy_flags);
if (r)
goto error;
@@ -2580,6 +2587,7 @@ int amdgpu_ttm_clear_buffer(struct amdgpu_ttm_buffer_entity *entity,
struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
struct dma_fence *fence = NULL;
struct amdgpu_res_cursor dst;
+ bool vm_needs_flush = false;
int r;
if (!entity)
@@ -2601,13 +2609,13 @@ int amdgpu_ttm_clear_buffer(struct amdgpu_ttm_buffer_entity *entity,
cur_size = min(dst.size, 256ULL << 20);
r = amdgpu_ttm_map_buffer(entity, &bo->tbo, bo->tbo.resource, &dst,
- 0, false, &cur_size, &to);
+ 0, false, &cur_size, &to, &vm_needs_flush);
if (r)
goto error;
r = amdgpu_ttm_fill_mem(adev, entity,
0, to, cur_size, resv,
- &next, true, k_job_id);
+ &next, vm_needs_flush, k_job_id);
if (r)
goto error;
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index 170c6b8d0a5f..5ba196dd9d7a 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -3925,6 +3925,10 @@ static int dm_resume(struct amdgpu_ip_block *ip_block)
/* On resume we need to rewrite the MSTM control bits to enable MST*/
s3_handle_mst(ddev, false);
+ /* Exit IPS before the detection loop's first AUX/DDC access. */
+ scoped_guard(mutex, &dm->dc_lock)
+ dc_exit_ips_for_hw_access(dm->dc);
+
/* Do detection*/
drm_connector_list_iter_begin(ddev, &iter);
drm_for_each_connector_iter(connector, &iter) {
@@ -6891,10 +6895,14 @@ get_output_color_space(const struct dc_crtc_timing *dc_crtc_timing,
break;
case DRM_MODE_COLORIMETRY_BT2020_RGB:
case DRM_MODE_COLORIMETRY_BT2020_YCC:
- if (dc_crtc_timing->pixel_encoding == PIXEL_ENCODING_RGB)
- color_space = COLOR_SPACE_2020_RGB_FULLRANGE;
- else
+ if (dc_crtc_timing->pixel_encoding == PIXEL_ENCODING_RGB) {
+ if (connector_state->hdmi.broadcast_rgb == DRM_HDMI_BROADCAST_RGB_LIMITED)
+ color_space = COLOR_SPACE_2020_RGB_LIMITEDRANGE;
+ else
+ color_space = COLOR_SPACE_2020_RGB_FULLRANGE;
+ } else {
color_space = COLOR_SPACE_2020_YCBCR_LIMITED;
+ }
break;
case DRM_MODE_COLORIMETRY_DEFAULT: // ITU601
default:
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
index 7db38ad3f848..8b0fe5c3a3e8 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
@@ -196,7 +196,6 @@ static ssize_t dp_link_settings_read(struct file *f, char __user *buf,
char *rd_buf_ptr = NULL;
const uint32_t rd_buf_size = 100;
uint32_t result = 0;
- uint8_t str_len = 0;
int r;
if (*pos & 3 || size & 3)
@@ -208,29 +207,26 @@ static ssize_t dp_link_settings_read(struct file *f, char __user *buf,
rd_buf_ptr = rd_buf;
- str_len = strlen("Current: %d 0x%x %d ");
- snprintf(rd_buf_ptr, str_len, "Current: %d 0x%x %d ",
+ rd_buf_ptr += scnprintf(rd_buf_ptr, rd_buf_size - (rd_buf_ptr - rd_buf),
+ "Current: %d 0x%x %d ",
link->cur_link_settings.lane_count,
link->cur_link_settings.link_rate,
link->cur_link_settings.link_spread);
- rd_buf_ptr += str_len;
- str_len = strlen("Verified: %d 0x%x %d ");
- snprintf(rd_buf_ptr, str_len, "Verified: %d 0x%x %d ",
+ rd_buf_ptr += scnprintf(rd_buf_ptr, rd_buf_size - (rd_buf_ptr - rd_buf),
+ "Verified: %d 0x%x %d ",
link->verified_link_cap.lane_count,
link->verified_link_cap.link_rate,
link->verified_link_cap.link_spread);
- rd_buf_ptr += str_len;
- str_len = strlen("Reported: %d 0x%x %d ");
- snprintf(rd_buf_ptr, str_len, "Reported: %d 0x%x %d ",
+ rd_buf_ptr += scnprintf(rd_buf_ptr, rd_buf_size - (rd_buf_ptr - rd_buf),
+ "Reported: %d 0x%x %d ",
link->reported_link_cap.lane_count,
link->reported_link_cap.link_rate,
link->reported_link_cap.link_spread);
- rd_buf_ptr += str_len;
- str_len = strlen("Preferred: %d 0x%x %d ");
- snprintf(rd_buf_ptr, str_len, "Preferred: %d 0x%x %d\n",
+ rd_buf_ptr += scnprintf(rd_buf_ptr, rd_buf_size - (rd_buf_ptr - rd_buf),
+ "Preferred: %d 0x%x %d\n",
link->preferred_link_setting.lane_count,
link->preferred_link_setting.link_rate,
link->preferred_link_setting.link_spread);
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
index 41aa8f9bb152..fe5b97e6f960 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
@@ -197,6 +197,7 @@ enum dc_edid_status dm_helpers_parse_edid_caps(
edid_caps->edid_hdmi = connector->display_info.is_hdmi;
if (edid_caps->edid_hdmi) {
+ edid_caps->qs_bit = connector->display_info.rgb_quant_range_selectable;
populate_hdmi_info_from_connector(link->dc->config.enable_frl, &connector->display_info.hdmi, edid_caps);
drm_dbg_driver(connector->dev, "%s: HDMI_FRL [%s] max_frl_rate %d\n", __func__, connector->name, edid_caps->max_frl_rate);
if (edid_caps->frl_dsc_support)
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc.c b/drivers/gpu/drm/amd/display/dc/core/dc.c
index e25b94b65dac..ebfed18e780c 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc.c
@@ -3122,6 +3122,7 @@ static struct surface_update_descriptor check_update_surfaces_for_stream(
}
if ((stream_update->hdr_static_metadata && !stream_update->stream->use_dynamic_meta) ||
+ stream_update->output_color_space ||
stream_update->vrr_infopacket ||
stream_update->vsc_infopacket ||
stream_update->vsp_infopacket ||
@@ -4090,6 +4091,7 @@ static void commit_planes_do_stream_update_sequence(struct dc *dc,
hwss_add_setup_periodic_interrupt(&seq_state, dc, pipe_ctx);
if ((stream_update->hdr_static_metadata && !stream->use_dynamic_meta) ||
+ stream_update->output_color_space ||
stream_update->vrr_infopacket ||
stream_update->vsc_infopacket ||
stream_update->vsp_infopacket ||
@@ -4272,6 +4274,7 @@ static void commit_planes_do_stream_update(struct dc *dc,
dc->hwss.setup_periodic_interrupt(dc, pipe_ctx);
if ((stream_update->hdr_static_metadata && !stream->use_dynamic_meta) ||
+ stream_update->output_color_space ||
stream_update->vrr_infopacket ||
stream_update->vsc_infopacket ||
stream_update->vsp_infopacket ||
diff --git a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
index e5e89294958a..68a1e78cd4b8 100644
--- a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
+++ b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
@@ -348,7 +348,8 @@ int amdgpu_dpm_switch_power_profile(struct amdgpu_device *adev,
const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
int ret = 0;
- if (amdgpu_sriov_vf(adev))
+ if (amdgpu_sriov_vf(adev) ||
+ amdgpu_in_reset(adev))
return 0;
if (pp_funcs && pp_funcs->switch_power_profile) {
@@ -367,7 +368,8 @@ int amdgpu_dpm_pause_power_profile(struct amdgpu_device *adev,
const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
int ret = 0;
- if (amdgpu_sriov_vf(adev))
+ if (amdgpu_sriov_vf(adev) ||
+ amdgpu_in_reset(adev))
return 0;
if (pp_funcs && pp_funcs->pause_power_profile) {
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
index 0be34fbdac26..4177211f32dc 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
@@ -2094,8 +2094,6 @@ static ssize_t smu_v13_0_0_get_gpu_metrics(struct smu_context *smu,
if ((mp1_ver == IP_VERSION(13, 0, 0) && smu->smc_fw_version <= 0x004e1e00) ||
(mp1_ver == IP_VERSION(13, 0, 10) && smu->smc_fw_version <= 0x00500800))
gpu_metrics->energy_accumulator = metrics->EnergyAccumulator;
- else
- gpu_metrics->energy_accumulator = UINT_MAX;
if (metrics->AverageGfxActivity <= SMU_13_0_0_BUSY_THRESHOLD)
gpu_metrics->average_gfxclk_frequency = metrics->AverageGfxclkFrequencyPostDs;
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
index 5d11ed8be4c3..845bd00c9df8 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
@@ -2096,8 +2096,8 @@ static ssize_t smu_v13_0_7_get_gpu_metrics(struct smu_context *smu,
metrics->Vcn1ActivityPercentage);
gpu_metrics->average_socket_power = metrics->AverageSocketPower;
- gpu_metrics->energy_accumulator = smu->smc_fw_version <= 0x00521400 ?
- metrics->EnergyAccumulator : UINT_MAX;
+ if (smu->smc_fw_version <= 0x00521400)
+ gpu_metrics->energy_accumulator = metrics->EnergyAccumulator;
if (metrics->AverageGfxActivity <= SMU_13_0_7_BUSY_THRESHOLD)
gpu_metrics->average_gfxclk_frequency = metrics->AverageGfxclkFrequencyPostDs;
diff --git a/drivers/gpu/drm/bridge/tc358768.c b/drivers/gpu/drm/bridge/tc358768.c
index 0f2820b37302..d18136a4b5c6 100644
--- a/drivers/gpu/drm/bridge/tc358768.c
+++ b/drivers/gpu/drm/bridge/tc358768.c
@@ -1262,10 +1262,13 @@ tc358768_atomic_get_input_bus_fmts(struct drm_bridge *bridge,
return input_fmts;
}
-static bool tc358768_mode_fixup(struct drm_bridge *bridge,
- const struct drm_display_mode *mode,
- struct drm_display_mode *adjusted_mode)
+static int tc358768_bridge_atomic_check(struct drm_bridge *bridge,
+ struct drm_bridge_state *bridge_state,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
{
+ struct drm_display_mode *adjusted_mode = &crtc_state->adjusted_mode;
+
/* Default to positive sync */
if (!(adjusted_mode->flags &
@@ -1276,13 +1279,15 @@ static bool tc358768_mode_fixup(struct drm_bridge *bridge,
(DRM_MODE_FLAG_PVSYNC | DRM_MODE_FLAG_NVSYNC)))
adjusted_mode->flags |= DRM_MODE_FLAG_PVSYNC;
- return true;
+ bridge_state->input_bus_cfg.flags = bridge->timings->input_bus_flags;
+
+ return 0;
}
static const struct drm_bridge_funcs tc358768_bridge_funcs = {
.attach = tc358768_bridge_attach,
.mode_valid = tc358768_bridge_mode_valid,
- .mode_fixup = tc358768_mode_fixup,
+ .atomic_check = tc358768_bridge_atomic_check,
.atomic_pre_enable = tc358768_bridge_atomic_pre_enable,
.atomic_enable = tc358768_bridge_atomic_enable,
.atomic_disable = tc358768_bridge_atomic_disable,
diff --git a/drivers/gpu/drm/bridge/ti-sn65dsi83.c b/drivers/gpu/drm/bridge/ti-sn65dsi83.c
index 42b451432bbb..9f530cd05d2f 100644
--- a/drivers/gpu/drm/bridge/ti-sn65dsi83.c
+++ b/drivers/gpu/drm/bridge/ti-sn65dsi83.c
@@ -403,7 +403,7 @@ static int sn65dsi83_reset_pipe(struct sn65dsi83 *sn65dsi83)
drm_modeset_drop_locks(&ctx);
drm_modeset_acquire_fini(&ctx);
- return 0;
+ return err;
}
static void sn65dsi83_reset_work(struct work_struct *ws)
@@ -419,11 +419,13 @@ static void sn65dsi83_reset_work(struct work_struct *ws)
ret = sn65dsi83_reset_pipe(ctx);
if (ret) {
dev_err(ctx->dev, "reset pipe failed %pe\n", ERR_PTR(ret));
- return;
+ goto bridge_exit;
}
+
if (ctx->irq)
enable_irq(ctx->irq);
+bridge_exit:
drm_bridge_exit(idx);
}
diff --git a/drivers/gpu/drm/drm_exec.c b/drivers/gpu/drm/drm_exec.c
index 7988f5e7d56a..2abc5c623fff 100644
--- a/drivers/gpu/drm/drm_exec.c
+++ b/drivers/gpu/drm/drm_exec.c
@@ -322,6 +322,19 @@ int drm_exec_prepare_array(struct drm_exec *exec,
{
int ret;
+ /*
+ * Make sure to lock a contended object even when no objects are
+ * given, otherwise drm_exec_retry_on_contention() would loop
+ * forever on patterns like:
+ *
+ * ret = drm_exec_prepare_array(exec, objs, num_objects, ...);
+ * drm_exec_retry_on_contention(exec);
+ *
+ * with num_objects == 0.
+ */
+ if (!num_objects)
+ return drm_exec_lock_contended(exec);
+
for (unsigned int i = 0; i < num_objects; ++i) {
ret = drm_exec_prepare_obj(exec, objects[i], num_fences);
if (unlikely(ret))
diff --git a/drivers/gpu/drm/drm_pagemap.c b/drivers/gpu/drm/drm_pagemap.c
index 05eb7254028f..89e1ddff84dd 100644
--- a/drivers/gpu/drm/drm_pagemap.c
+++ b/drivers/gpu/drm/drm_pagemap.c
@@ -1195,12 +1195,117 @@ void drm_pagemap_put(struct drm_pagemap *dpagemap)
}
EXPORT_SYMBOL(drm_pagemap_put);
+/**
+ * drm_pagemap_page_get_flags() - Read flags from a device-private folio
+ * @page: Pointer to a page of the device-private folio
+ *
+ * Return: The DRM_PAGEMAP_ZDD_FLAG_* bits encoded in zone_device_data.
+ */
+static unsigned long drm_pagemap_page_get_flags(struct page *page)
+{
+ struct folio *folio = page_folio(page);
+
+ return (unsigned long)folio_zone_device_data(folio) &
+ DRM_PAGEMAP_ZDD_FLAG_MASK;
+}
+
+/**
+ * drm_pagemap_page_set_flags() - Set flags on a device-private folio
+ * @page: Pointer to a page of the device-private folio
+ * @flags: DRM_PAGEMAP_ZDD_FLAG_* bits to set
+ *
+ * Preserve any flags already encoded alongside the ZDD pointer.
+ */
+static void drm_pagemap_page_set_flags(struct page *page,
+ unsigned long flags)
+{
+ struct folio *folio = page_folio(page);
+ unsigned long old;
+
+ if (WARN_ON_ONCE(flags & ~DRM_PAGEMAP_ZDD_FLAG_MASK))
+ return;
+
+ old = (unsigned long)folio_zone_device_data(folio);
+ folio_set_zone_device_data(folio, (void *)(old | flags));
+}
+
+/**
+ * drm_pagemap_retire_migrated_pages() - Record migrated device-private folios
+ * @src_pfns: source array after migrate_vma_pages() or migrate_device_pages()
+ * @npages: number of entries in @src_pfns
+ *
+ * Flag device-private folios successfully migrated to RAM before finalize
+ * unlocks the sources. The migrated state is stored in the physical folio, so
+ * it survives later folio splits and subsequent migrations can skip it.
+ */
+static void drm_pagemap_retire_migrated_pages(unsigned long *src_pfns,
+ unsigned long npages)
+{
+ unsigned long i = 0;
+
+ while (i < npages) {
+ struct page *page = migrate_pfn_to_page(src_pfns[i]);
+ unsigned long nr = 1;
+
+ if (!page) {
+ i++;
+ continue;
+ }
+
+ if (src_pfns[i] & MIGRATE_PFN_COMPOUND)
+ nr = folio_nr_pages(page_folio(page));
+
+ if ((src_pfns[i] & MIGRATE_PFN_MIGRATE) &&
+ is_device_private_page(page))
+ drm_pagemap_page_set_flags(page,
+ DRM_PAGEMAP_ZDD_FLAG_MIGRATED);
+
+ i += nr;
+ }
+}
+
+/**
+ * drm_pagemap_skip_retired_pages() - Skip retired device-private folios
+ * @src_pfns: MIGRATE_PFN-encoded source array
+ * @npages: number of entries in @src_pfns
+ *
+ * Skip source folios already migrated to RAM, identified by the migrated flag
+ * stored in the physical folio's zone_device_data.
+ */
+static void drm_pagemap_skip_retired_pages(unsigned long *src_pfns,
+ unsigned long npages)
+{
+ unsigned long i = 0;
+
+ while (i < npages) {
+ struct page *page = migrate_pfn_to_page(src_pfns[i]);
+ unsigned long nr = 1;
+
+ if (!page) {
+ i++;
+ continue;
+ }
+
+ if (src_pfns[i] & MIGRATE_PFN_COMPOUND)
+ nr = folio_nr_pages(page_folio(page));
+
+ if ((src_pfns[i] & MIGRATE_PFN_MIGRATE) &&
+ is_device_private_page(page) &&
+ (drm_pagemap_page_get_flags(page) &
+ DRM_PAGEMAP_ZDD_FLAG_MIGRATED))
+ src_pfns[i] &= ~MIGRATE_PFN_MIGRATE;
+
+ i += nr;
+ }
+}
+
/**
* drm_pagemap_evict_to_ram() - Evict GPU SVM range to RAM
* @devmem_allocation: Pointer to the device memory allocation
*
- * Similar to __drm_pagemap_migrate_to_ram but does not require mmap lock and
- * migration done via migrate_device_* functions.
+ * Similar to __drm_pagemap_migrate_to_ram(), but uses the
+ * migrate_device_* helpers and does not require the mmap lock.
+ * Device-private PFNs already migrated to RAM by either path are skipped.
*
* Return: 0 on success, negative error code on failure.
*/
@@ -1208,7 +1313,7 @@ int drm_pagemap_evict_to_ram(struct drm_pagemap_devmem *devmem_allocation)
{
const struct drm_pagemap_devmem_ops *ops = devmem_allocation->ops;
struct drm_pagemap_iova_state state = {};
- unsigned long npages, mpages = 0;
+ unsigned long npages, mpages;
struct page **pages;
unsigned long *src, *dst;
struct drm_pagemap_addr *pagemap_addr;
@@ -1219,6 +1324,7 @@ int drm_pagemap_evict_to_ram(struct drm_pagemap_devmem *devmem_allocation)
npages = devmem_allocation->size >> PAGE_SHIFT;
retry:
+ mpages = 0;
if (!mmget_not_zero(devmem_allocation->mm))
return -EFAULT;
@@ -1241,6 +1347,8 @@ int drm_pagemap_evict_to_ram(struct drm_pagemap_devmem *devmem_allocation)
if (err)
goto err_free;
+ drm_pagemap_skip_retired_pages(src, npages);
+
err = drm_pagemap_migrate_populate_ram_pfn(NULL, NULL, npages, &mpages,
src, dst, 0);
if (err || !mpages)
@@ -1263,6 +1371,7 @@ int drm_pagemap_evict_to_ram(struct drm_pagemap_devmem *devmem_allocation)
if (err)
drm_pagemap_migration_unlock_put_pages(npages, dst);
migrate_device_pages(src, dst, npages);
+ drm_pagemap_retire_migrated_pages(src, npages);
migrate_device_finalize(src, dst, npages);
drm_pagemap_migrate_unmap_pages(devmem_allocation->dev, pagemap_addr, dst, npages,
DMA_FROM_DEVICE, &state);
@@ -1360,13 +1469,15 @@ static int __drm_pagemap_migrate_to_ram(struct vm_area_struct *vas,
if (!migrate.cpages)
goto err_free;
+ drm_pagemap_skip_retired_pages(migrate.src, npages);
+
ops = zdd->devmem_allocation->ops;
dev = zdd->devmem_allocation->dev;
err = drm_pagemap_migrate_populate_ram_pfn(vas, page, npages, &mpages,
migrate.src, migrate.dst,
start);
- if (err)
+ if (err || !mpages)
goto err_finalize;
err = drm_pagemap_migrate_map_system_pages(dev, pagemap_addr,
@@ -1386,6 +1497,7 @@ static int __drm_pagemap_migrate_to_ram(struct vm_area_struct *vas,
if (err)
drm_pagemap_migration_unlock_put_pages(npages, migrate.dst);
migrate_vma_pages(&migrate);
+ drm_pagemap_retire_migrated_pages(migrate.src, npages);
migrate_vma_finalize(&migrate);
if (dev)
drm_pagemap_migrate_unmap_pages(dev, pagemap_addr, migrate.dst,
@@ -1438,13 +1550,19 @@ static vm_fault_t drm_pagemap_migrate_to_ram(struct vm_fault *vmf)
static void drm_pagemap_folio_split(struct folio *orig_folio, struct folio *new_folio)
{
struct drm_pagemap_zdd *zdd;
+ unsigned long orig_data, new_data;
if (!new_folio)
return;
new_folio->pgmap = orig_folio->pgmap;
- zdd = folio_zone_device_data(orig_folio);
- folio_set_zone_device_data(new_folio, drm_pagemap_zdd_get(zdd));
+
+ orig_data = (unsigned long)folio_zone_device_data(orig_folio);
+ zdd = (struct drm_pagemap_zdd *)(orig_data & ~DRM_PAGEMAP_ZDD_FLAG_MASK);
+
+ new_data = (unsigned long)drm_pagemap_zdd_get(zdd);
+ new_data |= orig_data & DRM_PAGEMAP_ZDD_FLAG_MASK;
+ folio_set_zone_device_data(new_folio, (void *)new_data);
}
static const struct dev_pagemap_ops drm_pagemap_pagemap_ops = {
diff --git a/drivers/gpu/drm/drm_panic_qr.rs b/drivers/gpu/drm/drm_panic_qr.rs
index ac27e86c601c..4d7eb75a3afc 100644
--- a/drivers/gpu/drm/drm_panic_qr.rs
+++ b/drivers/gpu/drm/drm_panic_qr.rs
@@ -407,8 +407,8 @@ fn push(&mut self, data: u64, len: usize) {
for i in (0..self.len).rev() {
self.decimals[i + len] = self.decimals[i];
}
- for i in 0..len {
- self.decimals[i] = (chunk % 10) as u8;
+ for decimal in &mut self.decimals[..len] {
+ *decimal = (chunk % 10) as u8;
chunk = div10(chunk);
}
self.len += len;
diff --git a/drivers/gpu/drm/gud/gud_drv.c b/drivers/gpu/drm/gud/gud_drv.c
index 89bd6ca36003..3a1b9e2a2eaa 100644
--- a/drivers/gpu/drm/gud/gud_drv.c
+++ b/drivers/gpu/drm/gud/gud_drv.c
@@ -289,6 +289,8 @@ static int gud_plane_add_properties(struct gud_device *gdrm)
* but mask out any additions on future devices.
*/
val &= GUD_ROTATION_MASK;
+ if (!(val & GUD_ROTATION_0))
+ continue;
ret = drm_plane_create_rotation_property(&gdrm->plane,
DRM_MODE_ROTATE_0, val);
break;
diff --git a/drivers/gpu/drm/i915/display/intel_cdclk.c b/drivers/gpu/drm/i915/display/intel_cdclk.c
index 7bc9b956554b..40ecad20d32b 100644
--- a/drivers/gpu/drm/i915/display/intel_cdclk.c
+++ b/drivers/gpu/drm/i915/display/intel_cdclk.c
@@ -2366,8 +2366,10 @@ static void bxt_sanitize_cdclk(struct intel_display *display)
* dividers both syncing to an active pipe, or asynchronously
* (PIPE_NONE).
*/
- cdctl &= ~bxt_cdclk_cd2x_pipe(display, INVALID_PIPE);
- cdctl |= bxt_cdclk_cd2x_pipe(display, INVALID_PIPE);
+ if (DISPLAY_VER(display) < 30) {
+ cdctl &= ~bxt_cdclk_cd2x_pipe(display, INVALID_PIPE);
+ cdctl |= bxt_cdclk_cd2x_pipe(display, INVALID_PIPE);
+ }
if (cdctl != expected) {
if (DISPLAY_VER(display) < 20) {
@@ -2697,8 +2699,8 @@ static void intel_set_cdclk(struct intel_display *display,
}
}
-static bool dg2_power_well_count(struct intel_display *display,
- const struct intel_cdclk_state *cdclk_state)
+static int dg2_power_well_count(struct intel_display *display,
+ const struct intel_cdclk_state *cdclk_state)
{
return display->platform.dg2 ? hweight8(cdclk_state->active_pipes) : 0;
}
diff --git a/drivers/gpu/drm/i915/display/intel_display_types.h b/drivers/gpu/drm/i915/display/intel_display_types.h
index c21e0c0ef0b1..96422641ae44 100644
--- a/drivers/gpu/drm/i915/display/intel_display_types.h
+++ b/drivers/gpu/drm/i915/display/intel_display_types.h
@@ -1947,6 +1947,8 @@ struct intel_dp {
bool colorimetry_support;
+ bool sst_split_sdp_support;
+
struct {
enum transcoder transcoder;
struct mutex lock;
diff --git a/drivers/gpu/drm/i915/display/intel_dp.c b/drivers/gpu/drm/i915/display/intel_dp.c
index 5733d2e7ac7f..ba0b19d1163c 100644
--- a/drivers/gpu/drm/i915/display/intel_dp.c
+++ b/drivers/gpu/drm/i915/display/intel_dp.c
@@ -3518,12 +3518,22 @@ intel_dp_audio_compute_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config,
struct drm_connector_state *conn_state)
{
+ struct intel_dp *intel_dp = enc_to_intel_dp(encoder);
+
pipe_config->has_audio =
intel_dp_has_audio(encoder, conn_state) &&
intel_audio_compute_config(encoder, pipe_config, conn_state);
pipe_config->sdp_split_enable = pipe_config->has_audio &&
intel_dp_is_uhbr(pipe_config);
+
+ /*
+ * SDP splitting for UHBR audio requires explicit sink capability in
+ * SST mode, whereas in MST mode it is inherently supported.
+ */
+ if (pipe_config->sdp_split_enable &&
+ !intel_crtc_has_type(pipe_config, INTEL_OUTPUT_DP_MST))
+ pipe_config->sdp_split_enable = intel_dp->sst_split_sdp_support;
}
void
@@ -4566,16 +4576,25 @@ void intel_dp_configure_protocol_converter(struct intel_dp *intel_dp,
str_enable_disable(tmp));
}
-static bool intel_dp_get_colorimetry_status(struct intel_dp *intel_dp)
+static u8 intel_dp_read_dprx_feature_enum(struct intel_dp *intel_dp)
{
u8 dprx = 0;
- if (drm_dp_dpcd_readb(&intel_dp->aux, DP_DPRX_FEATURE_ENUMERATION_LIST,
- &dprx) != 1)
- return false;
+ drm_dp_dpcd_read_data(&intel_dp->aux, DP_DPRX_FEATURE_ENUMERATION_LIST,
+ &dprx, sizeof(dprx));
+ return dprx;
+}
+
+static bool intel_dp_get_colorimetry_status(u8 dprx)
+{
return dprx & DP_VSC_SDP_EXT_FOR_COLORIMETRY_SUPPORTED;
}
+static bool intel_dp_get_sst_split_sdp_status(u8 dprx)
+{
+ return dprx & DP_SST_SPLIT_SDP_CAP;
+}
+
static int intel_dp_read_dsc_dpcd(struct drm_dp_aux *aux,
u8 dsc_dpcd[DP_DSC_RECEIVER_CAP_SIZE])
{
@@ -4875,6 +4894,7 @@ intel_edp_init_dpcd(struct intel_dp *intel_dp, struct intel_connector *connector
{
struct intel_display *display = to_intel_display(intel_dp);
int ret;
+ u8 dprx;
/* this function is meant to be called only once */
drm_WARN_ON(display->drm, intel_dp->dpcd[DP_DPCD_REV] != 0);
@@ -4886,8 +4906,13 @@ intel_edp_init_dpcd(struct intel_dp *intel_dp, struct intel_connector *connector
drm_dp_is_branch(intel_dp->dpcd));
intel_init_dpcd_quirks(intel_dp, &intel_dp->desc.ident);
+ dprx = intel_dp_read_dprx_feature_enum(intel_dp);
+
intel_dp->colorimetry_support =
- intel_dp_get_colorimetry_status(intel_dp);
+ intel_dp_get_colorimetry_status(dprx);
+
+ intel_dp->sst_split_sdp_support =
+ intel_dp_get_sst_split_sdp_status(dprx);
/*
* Read the eDP display control registers.
@@ -4978,13 +5003,20 @@ intel_dp_get_dpcd(struct intel_dp *intel_dp)
* the OUI/ID since we know it won't change.
*/
if (!intel_dp_is_edp(intel_dp)) {
+ u8 dprx;
+
drm_dp_read_desc(&intel_dp->aux, &intel_dp->desc,
drm_dp_is_branch(intel_dp->dpcd));
intel_init_dpcd_quirks(intel_dp, &intel_dp->desc.ident);
+ dprx = intel_dp_read_dprx_feature_enum(intel_dp);
+
intel_dp->colorimetry_support =
- intel_dp_get_colorimetry_status(intel_dp);
+ intel_dp_get_colorimetry_status(dprx);
+
+ intel_dp->sst_split_sdp_support =
+ intel_dp_get_sst_split_sdp_status(dprx);
intel_dp_update_sink_caps(intel_dp);
}
diff --git a/drivers/gpu/drm/i915/display/intel_dp_mst.c b/drivers/gpu/drm/i915/display/intel_dp_mst.c
index 0aa3e6b4c781..bcdc50491347 100644
--- a/drivers/gpu/drm/i915/display/intel_dp_mst.c
+++ b/drivers/gpu/drm/i915/display/intel_dp_mst.c
@@ -722,10 +722,6 @@ static int mst_stream_compute_config(struct intel_encoder *encoder,
pipe_config->sink_format = INTEL_OUTPUT_FORMAT_RGB;
pipe_config->output_format = INTEL_OUTPUT_FORMAT_RGB;
- ret = intel_pfit_compute_config(pipe_config, conn_state);
- if (ret)
- return ret;
-
ret = intel_pfit_compute_config(pipe_config, conn_state);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/i915_query.c b/drivers/gpu/drm/i915/i915_query.c
index 0c55fb6e9727..11157fb14db3 100644
--- a/drivers/gpu/drm/i915/i915_query.c
+++ b/drivers/gpu/drm/i915/i915_query.c
@@ -403,8 +403,10 @@ static int query_perf_config_list(struct drm_i915_private *i915,
ids = krealloc(oa_config_ids,
n_configs * sizeof(*oa_config_ids),
GFP_KERNEL);
- if (!ids)
+ if (!ids) {
+ kfree(oa_config_ids);
return -ENOMEM;
+ }
alloc = fetch_and_zero(&n_configs);
diff --git a/drivers/gpu/drm/logicvc/Kconfig b/drivers/gpu/drm/logicvc/Kconfig
index 579a358ed5cf..11aae1626199 100644
--- a/drivers/gpu/drm/logicvc/Kconfig
+++ b/drivers/gpu/drm/logicvc/Kconfig
@@ -4,7 +4,6 @@ config DRM_LOGICVC
depends on OF || COMPILE_TEST
select DRM_CLIENT_SELECTION
select DRM_KMS_HELPER
- select DRM_KMS_DMA_HELPER
select DRM_GEM_DMA_HELPER
select REGMAP
select REGMAP_MMIO
diff --git a/drivers/gpu/drm/rockchip/Kconfig b/drivers/gpu/drm/rockchip/Kconfig
index e7f49fe845ea..4e58685f58ff 100644
--- a/drivers/gpu/drm/rockchip/Kconfig
+++ b/drivers/gpu/drm/rockchip/Kconfig
@@ -68,6 +68,7 @@ config ROCKCHIP_CDN_DP
config ROCKCHIP_DW_DP
bool "Rockchip specific extensions for Synopsys DW DP"
+ select DRM_BRIDGE_CONNECTOR
help
This selects support for Rockchip SoC specific extensions
to enable Synopsys DesignWare Cores based DisplayPort transmit
@@ -145,6 +146,8 @@ config ROCKCHIP_RGB
config ROCKCHIP_RK3066_HDMI
bool "Rockchip specific extensions for RK3066 HDMI"
depends on DRM_ROCKCHIP
+ select DRM_DISPLAY_HELPER
+ select DRM_BRIDGE_CONNECTOR
help
This selects support for Rockchip SoC specific extensions
for the RK3066 HDMI driver. If you want to enable
diff --git a/drivers/gpu/drm/rockchip/analogix_dp-rockchip.c b/drivers/gpu/drm/rockchip/analogix_dp-rockchip.c
index b1ed25cefe5e..d638785ee5ab 100644
--- a/drivers/gpu/drm/rockchip/analogix_dp-rockchip.c
+++ b/drivers/gpu/drm/rockchip/analogix_dp-rockchip.c
@@ -241,10 +241,11 @@ static void rockchip_dp_drm_encoder_enable(struct drm_encoder *encoder,
of_graph_get_remote_port(endpoint.local_node);
of_property_read_u32(remote_port, "reg", &port_id);
- sprintf(name, "%s vp%d", remote_port_parent->full_name, port_id);
+ snprintf(name, sizeof(name), "%s vp%d",
+ remote_port_parent->full_name, port_id);
} else {
- sprintf(name, "%s %s",
- remote_port_parent->full_name, endpoint.id ? "vopl" : "vopb");
+ snprintf(name, sizeof(name), "%s %s",
+ remote_port_parent->full_name, endpoint.id ? "vopl" : "vopb");
}
DRM_DEV_DEBUG(dp->dev, "vop %s output to dp\n", (ret) ? "LIT" : "BIG");
diff --git a/drivers/gpu/drm/scheduler/tests/mock_scheduler.c b/drivers/gpu/drm/scheduler/tests/mock_scheduler.c
index 8e9ae7d980eb..2dfa3efef210 100644
--- a/drivers/gpu/drm/scheduler/tests/mock_scheduler.c
+++ b/drivers/gpu/drm/scheduler/tests/mock_scheduler.c
@@ -1,6 +1,8 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2025 Valve Corporation */
+#include <kunit/device.h>
+
#include "sched_tests.h"
/*
@@ -288,6 +290,7 @@ static const struct drm_sched_backend_ops drm_mock_scheduler_ops = {
*/
struct drm_mock_scheduler *drm_mock_sched_new(struct kunit *test, long timeout)
{
+ static unsigned int instance;
struct drm_sched_init_args args = {
.ops = &drm_mock_scheduler_ops,
.num_rqs = DRM_SCHED_PRIORITY_COUNT,
@@ -297,11 +300,19 @@ struct drm_mock_scheduler *drm_mock_sched_new(struct kunit *test, long timeout)
.name = "drm-mock-scheduler",
};
struct drm_mock_scheduler *sched;
+ struct device *dev;
+ char name[64];
int ret;
sched = kunit_kzalloc(test, sizeof(*sched), GFP_KERNEL);
KUNIT_ASSERT_NOT_NULL(test, sched);
+ snprintf(name, sizeof(name), "%s-%u", args.name, instance++);
+ dev = kunit_device_register(test, name);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, dev);
+
+ args.dev = dev;
+
ret = drm_sched_init(&sched->base, &args);
KUNIT_ASSERT_EQ(test, ret, 0);
diff --git a/drivers/gpu/drm/tiny/cirrus-qemu.c b/drivers/gpu/drm/tiny/cirrus-qemu.c
index 075221b431d3..3bf23fcf6574 100644
--- a/drivers/gpu/drm/tiny/cirrus-qemu.c
+++ b/drivers/gpu/drm/tiny/cirrus-qemu.c
@@ -582,6 +582,9 @@ static int cirrus_pci_probe(struct pci_dev *pdev,
struct cirrus_device *cirrus;
int ret;
+ if (pci_resource_len(pdev, 0) < CIRRUS_VRAM_SIZE)
+ return -ENODEV;
+
ret = aperture_remove_conflicting_pci_devices(pdev, cirrus_driver.name);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/virtio/virtgpu_display.c b/drivers/gpu/drm/virtio/virtgpu_display.c
index 44ffffec550f..85ea252c658e 100644
--- a/drivers/gpu/drm/virtio/virtgpu_display.c
+++ b/drivers/gpu/drm/virtio/virtgpu_display.c
@@ -344,7 +344,7 @@ virtio_gpu_user_framebuffer_create(struct drm_device *dev,
if (ret) {
kfree(virtio_gpu_fb);
drm_gem_object_put(obj);
- return NULL;
+ return ERR_PTR(ret);
}
return &virtio_gpu_fb->base;
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 7449907754a4..88fb4be92cf3 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -43,6 +43,8 @@
#include <drm/drm_probe_helper.h>
#include <drm/virtgpu_drm.h>
+#include <xen/xen.h>
+
#define DRIVER_NAME "virtio_gpu"
#define DRIVER_DESC "virtio GPU"
@@ -60,6 +62,24 @@
/* See virtio_gpu_ctx_create. One additional character for NULL terminator. */
#define DEBUG_NAME_MAX_LEN 65
+/*
+ * Whether the host must be told about resource backing pages by DMA address
+ * rather than guest-physical address.
+ *
+ * This mirrors vring_use_map_api() in drivers/virtio/virtio_ring.c, including
+ * its xen_domain() case.
+ */
+static inline bool virtio_gpu_use_dma_api(const struct virtio_device *vdev)
+{
+ if (!virtio_has_dma_quirk(vdev))
+ return true;
+
+ if (xen_domain())
+ return true;
+
+ return false;
+}
+
struct virtio_gpu_object_params {
unsigned long size;
bool dumb;
diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index ec9efacc6919..1527c62be88b 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -163,7 +163,7 @@ static int virtio_gpu_object_shmem_init(struct virtio_gpu_device *vgdev,
struct virtio_gpu_mem_entry **ents,
unsigned int *nents)
{
- bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
+ bool use_dma_api = virtio_gpu_use_dma_api(vgdev->vdev);
struct scatterlist *sg;
struct sg_table *pages;
int si;
diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c
index e5e1af8b8e8a..2b7af8e4e9e6 100644
--- a/drivers/gpu/drm/virtio/virtgpu_vq.c
+++ b/drivers/gpu/drm/virtio/virtgpu_vq.c
@@ -724,7 +724,7 @@ int virtio_gpu_panic_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(objs->objs[0]);
struct virtio_gpu_transfer_to_host_2d *cmd_p;
struct virtio_gpu_vbuffer *vbuf;
- bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
+ bool use_dma_api = virtio_gpu_use_dma_api(vgdev->vdev);
if (virtio_gpu_is_shmem(bo) && use_dma_api)
dma_sync_sgtable_for_device(vgdev->vdev->dev.parent,
@@ -755,7 +755,7 @@ void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(objs->objs[0]);
struct virtio_gpu_transfer_to_host_2d *cmd_p;
struct virtio_gpu_vbuffer *vbuf;
- bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
+ bool use_dma_api = virtio_gpu_use_dma_api(vgdev->vdev);
if (virtio_gpu_is_shmem(bo) && use_dma_api)
dma_sync_sgtable_for_device(vgdev->vdev->dev.parent,
@@ -1188,7 +1188,7 @@ void virtio_gpu_cmd_transfer_to_host_3d(struct virtio_gpu_device *vgdev,
struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(objs->objs[0]);
struct virtio_gpu_transfer_host_3d *cmd_p;
struct virtio_gpu_vbuffer *vbuf;
- bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
+ bool use_dma_api = virtio_gpu_use_dma_api(vgdev->vdev);
if (virtio_gpu_is_shmem(bo) && use_dma_api)
dma_sync_sgtable_for_device(vgdev->vdev->dev.parent,
diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
index 8e7b146880f4..a11c61908065 100644
--- a/drivers/gpu/drm/xe/Makefile
+++ b/drivers/gpu/drm/xe/Makefile
@@ -152,7 +152,9 @@ xe-y += xe_bb.o \
xe_wait_user_fence.o \
xe_wopcm.o
-xe-$(CONFIG_I2C) += xe_i2c.o
+xe-$(CONFIG_I2C) += xe_i2c.o \
+ xe_amc.o
+
xe-$(CONFIG_DRM_XE_GPUSVM) += xe_svm.o
xe-$(CONFIG_DRM_GPUSVM) += xe_userptr.o
diff --git a/drivers/gpu/drm/xe/instructions/xe_gpu_commands.h b/drivers/gpu/drm/xe/instructions/xe_gpu_commands.h
index 18d0fde8c98f..faf8d7e2c5c1 100644
--- a/drivers/gpu/drm/xe/instructions/xe_gpu_commands.h
+++ b/drivers/gpu/drm/xe/instructions/xe_gpu_commands.h
@@ -46,6 +46,7 @@
#define GFX_OP_PIPE_CONTROL(len) ((0x3<<29)|(0x3<<27)|(0x2<<24)|((len)-2))
#define PIPE_CONTROL0_QUEUE_DRAIN_MODE BIT(12)
+#define PIPE_CONTROL0_UNTYPED_DATAPORT_CACHE_FLUSH BIT(11) /* gen12 */
#define PIPE_CONTROL0_L3_READ_ONLY_CACHE_INVALIDATE BIT(10) /* gen12 */
#define PIPE_CONTROL0_HDC_PIPELINE_FLUSH BIT(9) /* gen12 */
diff --git a/drivers/gpu/drm/xe/regs/xe_i2c_regs.h b/drivers/gpu/drm/xe/regs/xe_i2c_regs.h
index f2e455e2bfe4..37550e4a20f8 100644
--- a/drivers/gpu/drm/xe/regs/xe_i2c_regs.h
+++ b/drivers/gpu/drm/xe/regs/xe_i2c_regs.h
@@ -20,4 +20,6 @@
#define I2C_CONFIG_CMD XE_REG(I2C_CONFIG_SPACE_OFFSET + PCI_COMMAND)
#define I2C_CONFIG_PMCSR XE_REG(I2C_CONFIG_SPACE_OFFSET + 0x84)
+#define I2C_REG(reg) XE_REG((reg) + I2C_MEM_SPACE_OFFSET)
+
#endif /* _XE_I2C_REGS_H_ */
diff --git a/drivers/gpu/drm/xe/xe_amc.c b/drivers/gpu/drm/xe/xe_amc.c
new file mode 100644
index 000000000000..8ecadee6eea3
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_amc.c
@@ -0,0 +1,197 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2026 Intel Corporation.
+ */
+
+#include <linux/delay.h>
+#include <linux/dev_printk.h>
+#include <linux/err.h>
+#include <linux/i2c.h>
+#include <linux/pci_ids.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+#include <linux/workqueue.h>
+
+#include "regs/xe_i2c_regs.h"
+
+#include "xe_amc.h"
+#include "xe_device.h"
+#include "xe_i2c.h"
+#include "xe_mmio.h"
+
+/**
+ * DOC: Add-In Management Controller (AMC)
+ *
+ * Handler for the SMBus Alerts from the AMC. All the alerts from AMC will cause
+ * the device to be declared wedged.
+ */
+
+#define AMC_COMMAND 0x0f
+#define AMC_GPU_I2C_ADDR 0x8f
+#define AMC_VERSION_V1 0x01
+#define AMC_DESTINATION_ID 12
+#define AMC_SOURCE_ID 8
+#define AMC_FLAGS 0xc8
+
+#define AMC_MSG_TYPE 0x7e
+#define AMC_GET_ALERT_REASON 0x01
+
+enum xe_amc_alert {
+ AMC_ALERT_UNKNOWN,
+ AMC_ALERT_FW_DOWNLOAD,
+ AMC_ALERT_THERMAL_TRIP,
+ AMC_ALERT_OOB_REQUEST,
+ AMC_ALERT_OOB_RESET,
+ AMC_ALERT_CATERR,
+};
+
+static const char * const amc_alert[] = {
+ [AMC_ALERT_FW_DOWNLOAD] = "Firmware Download",
+ [AMC_ALERT_THERMAL_TRIP] = "Thermal Trip",
+ [AMC_ALERT_OOB_REQUEST] = "OOB Request",
+ [AMC_ALERT_OOB_RESET] = "OOB Reset",
+ [AMC_ALERT_CATERR] = "Catastrophic",
+};
+
+struct xe_amc {
+ struct xe_i2c *i2c;
+ struct work_struct work;
+};
+
+struct amc_header {
+ u8 command;
+ u8 len;
+ u8 address;
+ u8 version;
+ u8 destination;
+ u8 source;
+ u8 flags;
+} __packed;
+
+struct amc_message {
+ u8 type;
+ u16 vendor;
+ u8 command;
+} __packed;
+
+struct amc_request {
+ struct amc_header header;
+ struct amc_message message;
+ u32 reserved;
+} __packed;
+
+struct amc_response {
+ struct amc_header header;
+ struct amc_message message;
+ u8 error;
+ u8 value;
+} __packed;
+
+static const struct amc_request amc_get_alert_reason = {
+ .header = {
+ .command = AMC_COMMAND,
+ .len = sizeof(struct amc_request) - 2,
+ .address = AMC_GPU_I2C_ADDR,
+ .version = AMC_VERSION_V1,
+ .destination = AMC_DESTINATION_ID,
+ .source = AMC_SOURCE_ID,
+ .flags = AMC_FLAGS,
+ },
+ .message = {
+ .type = AMC_MSG_TYPE,
+ .vendor = htons(PCI_VENDOR_ID_INTEL),
+ .command = AMC_GET_ALERT_REASON,
+ },
+};
+
+static void xe_amc_work(struct work_struct *work)
+{
+ const struct amc_request *request = &amc_get_alert_reason;
+ struct xe_amc *amc = from_work(amc, work, work);
+ u8 alert_reason = AMC_ALERT_UNKNOWN;
+ struct amc_response response;
+ struct i2c_client *client;
+ int ret;
+
+ client = amc->i2c->client[XE_I2C_CLIENT_AMC];
+ if (IS_ERR_OR_NULL(client))
+ goto out_reassert_interrupt;
+
+ ret = i2c_master_send(client, (u8 *)request, sizeof(*request));
+ if (ret < 0) {
+ dev_err(&client->dev, "failed to send request (%d)\n", ret);
+ goto out_reassert_interrupt;
+ }
+
+ /* AMC needs 20ms to generate the response. */
+ fsleep(20 * USEC_PER_MSEC);
+
+ ret = i2c_master_recv(client, (u8 *)&response, sizeof(response));
+ if (ret < 0) {
+ dev_err(&client->dev, "failed to read response (%d)\n", ret);
+ goto out_reassert_interrupt;
+ }
+
+ if (!response.header.len) {
+ dev_err(&client->dev, "empty response from AMC\n");
+ goto out_reassert_interrupt;
+ }
+
+ if (memcmp(&response.message, &request->message, sizeof(struct amc_message))) {
+ dev_err(&client->dev, "response does not match the request\n");
+ goto out_reassert_interrupt;
+ }
+
+ if (response.error) {
+ dev_err(&client->dev, "AMC error 0x%02x\n", response.error);
+ goto out_reassert_interrupt;
+ }
+
+ alert_reason = response.value;
+ dev_dbg(&client->dev, "Alert reason: %d\n", alert_reason);
+
+out_reassert_interrupt:
+ xe_mmio_rmw32(amc->i2c->mmio, I2C_CONFIG_CMD, PCI_COMMAND_INTX_DISABLE, 0);
+
+ switch (alert_reason) {
+ case AMC_ALERT_FW_DOWNLOAD:
+ case AMC_ALERT_THERMAL_TRIP:
+ case AMC_ALERT_OOB_REQUEST:
+ case AMC_ALERT_OOB_RESET:
+ case AMC_ALERT_CATERR:
+ dev_warn(amc->i2c->drm_dev, "AMC Alert: %s\n", amc_alert[alert_reason]);
+ xe_device_declare_wedged(i2c_client_to_xe_device(client));
+ break;
+ default:
+ dev_warn(amc->i2c->drm_dev, "unknown AMC alert: %d\n", alert_reason);
+ break;
+ }
+}
+
+void xe_amc_handle_alert(struct xe_i2c *i2c)
+{
+ queue_work(system_long_wq, &i2c->amc->work);
+}
+
+int xe_amc_init(struct xe_i2c *i2c)
+{
+ struct xe_amc *amc;
+
+ amc = kzalloc(sizeof(*amc), GFP_KERNEL);
+ if (!amc)
+ return -ENOMEM;
+
+ INIT_WORK(&amc->work, xe_amc_work);
+ i2c->amc = amc;
+ amc->i2c = i2c;
+
+ return 0;
+}
+
+void xe_amc_exit(struct xe_i2c *i2c)
+{
+ if (i2c->amc) {
+ cancel_work_sync(&i2c->amc->work);
+ kfree(i2c->amc);
+ }
+}
diff --git a/drivers/gpu/drm/xe/xe_amc.h b/drivers/gpu/drm/xe/xe_amc.h
new file mode 100644
index 000000000000..b1d5311fee53
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_amc.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _XE_AMC_H_
+#define _XE_AMC_H_
+
+#include <linux/i2c.h>
+
+#include "xe_device.h"
+
+struct xe_i2c;
+
+static inline struct xe_device *i2c_adapter_to_xe_device(struct i2c_adapter *adapter)
+{
+ return kdev_to_xe_device(adapter->dev.parent->parent);
+}
+
+static inline struct xe_device *i2c_client_to_xe_device(struct i2c_client *client)
+{
+ return i2c_adapter_to_xe_device(client->adapter);
+}
+
+int xe_amc_init(struct xe_i2c *i2c);
+void xe_amc_exit(struct xe_i2c *i2c);
+void xe_amc_handle_alert(struct xe_i2c *i2c);
+
+#endif /* _XE_AMC_H_ */
diff --git a/drivers/gpu/drm/xe/xe_i2c.c b/drivers/gpu/drm/xe/xe_i2c.c
index f05f23221c1b..5504cd9dd359 100644
--- a/drivers/gpu/drm/xe/xe_i2c.c
+++ b/drivers/gpu/drm/xe/xe_i2c.c
@@ -12,8 +12,6 @@
#include <linux/err.h>
#include <linux/i2c.h>
#include <linux/ioport.h>
-#include <linux/irq.h>
-#include <linux/irqdomain.h>
#include <linux/notifier.h>
#include <linux/pci.h>
#include <linux/platform_device.h>
@@ -24,9 +22,12 @@
#include <linux/types.h>
#include <linux/workqueue.h>
+#include <linux/designware_i2c.h>
+
#include "regs/xe_i2c_regs.h"
#include "regs/xe_irq_regs.h"
+#include "xe_amc.h"
#include "xe_device.h"
#include "xe_i2c.h"
#include "xe_mmio.h"
@@ -61,16 +62,32 @@ static inline void xe_i2c_read_endpoint(struct xe_mmio *mmio, void *ep)
val[1] = xe_mmio_read32(mmio, REG_SG_REMAP_ADDR_POSTFIX);
}
+static void xe_i2c_handle_smbus_alert(struct xe_i2c *i2c)
+{
+ u32 stat;
+
+ stat = xe_mmio_read32(i2c->mmio, I2C_REG(DW_IC_SMBUS_INTR_STAT));
+ if (!stat)
+ return;
+
+ xe_mmio_write32(i2c->mmio, I2C_REG(DW_IC_CLR_SMBUS_INTR), stat);
+
+ if (stat & DW_IC_SMBUS_INTR_ALERT && i2c->amc)
+ xe_amc_handle_alert(i2c);
+ else
+ xe_mmio_rmw32(i2c->mmio, I2C_CONFIG_CMD, PCI_COMMAND_INTX_DISABLE, 0);
+}
+
static void xe_i2c_client_work(struct work_struct *work)
{
struct xe_i2c *i2c = container_of(work, struct xe_i2c, work);
struct i2c_board_info info = {
.type = "amc",
.flags = I2C_CLIENT_HOST_NOTIFY,
- .addr = i2c->ep.addr[1],
+ .addr = i2c->ep.addr[XE_I2C_CLIENT_AMC],
};
- i2c->client[0] = i2c_new_client_device(i2c->adapter, &info);
+ i2c->client[XE_I2C_CLIENT_AMC] = i2c_new_client_device(i2c->adapter, &info);
}
static int xe_i2c_notifier(struct notifier_block *nb, unsigned long action, void *data)
@@ -115,16 +132,6 @@ static int xe_i2c_register_adapter(struct xe_i2c *i2c)
goto err_fwnode_remove;
}
- if (i2c->adapter_irq) {
- struct resource res;
-
- res = DEFINE_RES_IRQ_NAMED(i2c->adapter_irq, "xe_i2c");
-
- ret = platform_device_add_resources(pdev, &res, 1);
- if (ret)
- goto err_pdev_put;
- }
-
pdev->dev.parent = i2c->drm_dev;
pdev->dev.fwnode = fwnode;
i2c->adapter_node = fwnode;
@@ -166,7 +173,8 @@ bool xe_i2c_present(struct xe_device *xe)
static bool xe_i2c_irq_present(struct xe_device *xe)
{
- return xe->i2c && xe->i2c->adapter_irq;
+ return xe->i2c && xe->i2c->ep.capabilities & XE_I2C_EP_CAP_IRQ &&
+ !xe_survivability_mode_is_boot_enabled(xe);
}
/**
@@ -179,18 +187,10 @@ static bool xe_i2c_irq_present(struct xe_device *xe)
*/
void xe_i2c_irq_handler(struct xe_device *xe, u32 master_ctl)
{
- struct xe_mmio *mmio = xe_root_tile_mmio(xe);
-
if (!(master_ctl & I2C_IRQ) || !xe_i2c_irq_present(xe))
return;
- /* Forward interrupt to I2C adapter */
- generic_handle_irq_safe(xe->i2c->adapter_irq);
-
- /* Deassert after I2C adapter clears the interrupt */
- xe_mmio_rmw32(mmio, I2C_CONFIG_CMD, 0, PCI_COMMAND_INTX_DISABLE);
- /* Reassert to allow subsequent interrupt generation */
- xe_mmio_rmw32(mmio, I2C_CONFIG_CMD, PCI_COMMAND_INTX_DISABLE, 0);
+ xe_i2c_handle_smbus_alert(xe->i2c);
}
void xe_i2c_irq_reset(struct xe_device *xe)
@@ -215,45 +215,6 @@ void xe_i2c_irq_postinstall(struct xe_device *xe)
xe_mmio_rmw32(mmio, I2C_CONFIG_CMD, PCI_COMMAND_INTX_DISABLE, 0);
}
-static int xe_i2c_irq_map(struct irq_domain *h, unsigned int virq,
- irq_hw_number_t hw_irq_num)
-{
- irq_set_chip_and_handler(virq, &dummy_irq_chip, handle_simple_irq);
- return 0;
-}
-
-static const struct irq_domain_ops xe_i2c_irq_ops = {
- .map = xe_i2c_irq_map,
-};
-
-static int xe_i2c_create_irq(struct xe_device *xe)
-{
- struct xe_i2c *i2c = xe->i2c;
- struct irq_domain *domain;
-
- if (!(i2c->ep.capabilities & XE_I2C_EP_CAP_IRQ) ||
- xe_survivability_mode_is_boot_enabled(xe))
- return 0;
-
- domain = irq_domain_create_linear(dev_fwnode(i2c->drm_dev), 1, &xe_i2c_irq_ops, NULL);
- if (!domain)
- return -ENOMEM;
-
- i2c->adapter_irq = irq_create_mapping(domain, 0);
- i2c->irqdomain = domain;
-
- return 0;
-}
-
-static void xe_i2c_remove_irq(struct xe_i2c *i2c)
-{
- if (!i2c->irqdomain)
- return;
-
- irq_dispose_mapping(i2c->adapter_irq);
- irq_domain_remove(i2c->irqdomain);
-}
-
static int xe_i2c_read(void *context, unsigned int reg, unsigned int *val)
{
struct xe_i2c *i2c = context;
@@ -267,8 +228,16 @@ static int xe_i2c_write(void *context, unsigned int reg, unsigned int val)
{
struct xe_i2c *i2c = context;
- xe_mmio_write32(i2c->mmio, XE_REG(reg + I2C_MEM_SPACE_OFFSET), val);
+ switch (reg) {
+ case DW_IC_SMBUS_INTR_MASK:
+ /* Make sure the Alert is never masked. */
+ val |= DW_IC_SMBUS_INTR_ALERT;
+ break;
+ default:
+ break;
+ }
+ xe_mmio_write32(i2c->mmio, I2C_REG(reg), val);
return 0;
}
@@ -310,12 +279,15 @@ static void xe_i2c_remove(void *data)
struct xe_i2c *i2c = data;
unsigned int i;
- for (i = 0; i < XE_I2C_MAX_CLIENTS; i++)
+ xe_amc_exit(i2c);
+
+ for (i = 0; i < XE_I2C_MAX_CLIENTS; i++) {
i2c_unregister_device(i2c->client[i]);
+ i2c->client[i] = NULL;
+ }
bus_unregister_notifier(&i2c_bus_type, &i2c->bus_notifier);
xe_i2c_unregister_adapter(i2c);
- xe_i2c_remove_irq(i2c);
}
/**
@@ -366,22 +338,18 @@ int xe_i2c_probe(struct xe_device *xe)
if (ret)
return ret;
- ret = xe_i2c_create_irq(xe);
- if (ret)
- goto err_unregister_notifier;
-
ret = xe_i2c_register_adapter(i2c);
- if (ret)
- goto err_remove_irq;
+ if (ret) {
+ bus_unregister_notifier(&i2c_bus_type, &i2c->bus_notifier);
+ return ret;
+ }
+
+ ret = xe_amc_init(i2c);
+ if (ret) {
+ xe_i2c_remove(i2c);
+ return ret;
+ }
xe_i2c_irq_postinstall(xe);
return devm_add_action_or_reset(drm_dev, xe_i2c_remove, i2c);
-
-err_remove_irq:
- xe_i2c_remove_irq(i2c);
-
-err_unregister_notifier:
- bus_unregister_notifier(&i2c_bus_type, &i2c->bus_notifier);
-
- return ret;
}
diff --git a/drivers/gpu/drm/xe/xe_i2c.h b/drivers/gpu/drm/xe/xe_i2c.h
index 425d8160835f..b200966b0048 100644
--- a/drivers/gpu/drm/xe/xe_i2c.h
+++ b/drivers/gpu/drm/xe/xe_i2c.h
@@ -11,18 +11,21 @@ struct device;
struct fwnode_handle;
struct i2c_adapter;
struct i2c_client;
-struct irq_domain;
struct platform_device;
+struct xe_amc;
struct xe_device;
struct xe_mmio;
-#define XE_I2C_MAX_CLIENTS 3
-
#define XE_I2C_EP_COOKIE_DEVICE 0xde
/* Endpoint Capabilities */
#define XE_I2C_EP_CAP_IRQ BIT(0)
+enum XE_I2C_CLIENT {
+ XE_I2C_CLIENT_AMC = 1,
+ XE_I2C_MAX_CLIENTS = 3,
+};
+
struct xe_i2c_endpoint {
u8 cookie;
u8 capabilities;
@@ -38,13 +41,11 @@ struct xe_i2c {
struct notifier_block bus_notifier;
struct work_struct work;
- struct irq_domain *irqdomain;
- int adapter_irq;
-
struct xe_i2c_endpoint ep;
struct device *drm_dev;
struct xe_mmio *mmio;
+ struct xe_amc *amc;
};
#if IS_ENABLED(CONFIG_I2C)
diff --git a/drivers/gpu/drm/xe/xe_oa.c b/drivers/gpu/drm/xe/xe_oa.c
index 18d990c5d4ec..e5b0f3e2f389 100644
--- a/drivers/gpu/drm/xe/xe_oa.c
+++ b/drivers/gpu/drm/xe/xe_oa.c
@@ -2431,9 +2431,9 @@ int xe_oa_add_config_ioctl(struct drm_device *dev, u64 data, struct drm_file *fi
oa_config->id = idr_alloc(&oa->metrics_idr, oa_config, 1, 0, GFP_KERNEL);
if (oa_config->id < 0) {
- drm_dbg(&oa->xe->drm, "Failed to create sysfs entry for OA config\n");
+ drm_dbg(&oa->xe->drm, "Failed to allocate id for OA config\n");
err = oa_config->id;
- goto sysfs_err;
+ goto id_alloc_err;
}
id = oa_config->id;
@@ -2444,6 +2444,8 @@ int xe_oa_add_config_ioctl(struct drm_device *dev, u64 data, struct drm_file *fi
return id;
+id_alloc_err:
+ sysfs_remove_group(oa->metrics_kobj, &oa_config->sysfs_metric);
sysfs_err:
mutex_unlock(&oa->metrics_lock);
reg_err:
diff --git a/drivers/gpu/drm/xe/xe_pagefault.c b/drivers/gpu/drm/xe/xe_pagefault.c
index dbf8f71d3328..a4986df8328d 100644
--- a/drivers/gpu/drm/xe/xe_pagefault.c
+++ b/drivers/gpu/drm/xe/xe_pagefault.c
@@ -16,6 +16,7 @@
#include "xe_hw_engine.h"
#include "xe_pagefault.h"
#include "xe_pagefault_types.h"
+#include "xe_pm.h"
#include "xe_svm.h"
#include "xe_trace_bo.h"
#include "xe_vm.h"
@@ -292,9 +293,17 @@ static void xe_pagefault_queue_work(struct work_struct *w)
{
struct xe_pagefault_queue *pf_queue =
container_of(w, typeof(*pf_queue), worker);
+ struct xe_device *xe = pf_queue->xe;
struct xe_pagefault pf;
unsigned long threshold;
+ /*
+ * A live VM holds a PM reference, but a torn-down VM does not.
+ * Guard the entire worker loop to safely drain stale faults and
+ * prevent autosuspends from desyncing batched CT flushes.
+ */
+ guard(xe_pm_runtime)(xe);
+
#define USM_QUEUE_MAX_RUNTIME_MS 20
threshold = jiffies + msecs_to_jiffies(USM_QUEUE_MAX_RUNTIME_MS);
@@ -365,6 +374,7 @@ static int xe_pagefault_queue_init(struct xe_device *xe,
drm_dbg(&xe->drm, "xe_pagefault_entry_size=%d, total_num_eus=%d, pf_queue->size=%u",
xe_pagefault_entry_size(), total_num_eus, pf_queue->size);
+ pf_queue->xe = xe;
spin_lock_init(&pf_queue->lock);
INIT_WORK(&pf_queue->worker, xe_pagefault_queue_work);
diff --git a/drivers/gpu/drm/xe/xe_pagefault_types.h b/drivers/gpu/drm/xe/xe_pagefault_types.h
index c4ee625b93dd..f63a12aa0d4f 100644
--- a/drivers/gpu/drm/xe/xe_pagefault_types.h
+++ b/drivers/gpu/drm/xe/xe_pagefault_types.h
@@ -8,6 +8,7 @@
#include <linux/workqueue.h>
+struct xe_device;
struct xe_gt;
struct xe_pagefault;
@@ -118,6 +119,8 @@ struct xe_pagefault {
* queue to absorb the device’s worst-case number of outstanding faults.
*/
struct xe_pagefault_queue {
+ /** @xe: Back-pointer to the Xe device */
+ struct xe_device *xe;
/**
* @data: Data in queue containing struct xe_pagefault, protected by
* @lock
diff --git a/drivers/gpu/drm/xe/xe_ring_ops.c b/drivers/gpu/drm/xe/xe_ring_ops.c
index 39a670e91ba7..08b4a4283e96 100644
--- a/drivers/gpu/drm/xe/xe_ring_ops.c
+++ b/drivers/gpu/drm/xe/xe_ring_ops.c
@@ -212,6 +212,7 @@ static int emit_render_cache_flush(struct xe_sched_job *job, u32 *dw, int i)
{
struct xe_exec_queue *q = job->q;
struct xe_gt *gt = q->gt;
+ struct xe_device *xe = gt_to_xe(gt);
bool lacks_render = !(gt->info.engine_mask & XE_HW_ENGINE_RCS_MASK);
u32 flags0, flags1;
@@ -220,6 +221,16 @@ static int emit_render_cache_flush(struct xe_sched_job *job, u32 *dw, int i)
LRC_PPHWSP_FLUSH_INVAL_SCRATCH_ADDR, 0);
flags0 = PIPE_CONTROL0_HDC_PIPELINE_FLUSH;
+ /*
+ * Prior to MTL, HDC Pipeline Flush reliably also flushes the LSC
+ * untyped L1 dataport cache, provided HDC_CHICKEN0 is programmed
+ * correctly. Starting with MTL that coupling no longer holds
+ * regardless of how HDC_CHICKEN0 is programmed, but explicitly
+ * requesting the flush via PIPE_CONTROL is itself only reliable
+ * from Xe2 onward, so only gate it in on Xe2+.
+ */
+ if (GRAPHICS_VERx100(xe) >= 2000)
+ flags0 |= PIPE_CONTROL0_UNTYPED_DATAPORT_CACHE_FLUSH;
flags1 = (PIPE_CONTROL_TILE_CACHE_FLUSH |
PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH |
PIPE_CONTROL_DEPTH_CACHE_FLUSH |
diff --git a/drivers/hwmon/applesmc.c b/drivers/hwmon/applesmc.c
index 90a14a7f2c4c..bfe9c5b0c4f1 100644
--- a/drivers/hwmon/applesmc.c
+++ b/drivers/hwmon/applesmc.c
@@ -1249,12 +1249,17 @@ static void applesmc_release_light_sensor(void)
static int applesmc_create_key_backlight(void)
{
+ int ret;
+
if (!smcreg.has_key_backlight)
return 0;
applesmc_led_wq = create_singlethread_workqueue("applesmc-led");
if (!applesmc_led_wq)
return -ENOMEM;
- return led_classdev_register(&pdev->dev, &applesmc_backlight);
+ ret = led_classdev_register(&pdev->dev, &applesmc_backlight);
+ if (ret)
+ destroy_workqueue(applesmc_led_wq);
+ return ret;
}
static void applesmc_release_key_backlight(void)
diff --git a/drivers/hwmon/aspeed-pwm-tacho.c b/drivers/hwmon/aspeed-pwm-tacho.c
index 1c5945d4ba37..bfce589c3fb1 100644
--- a/drivers/hwmon/aspeed-pwm-tacho.c
+++ b/drivers/hwmon/aspeed-pwm-tacho.c
@@ -934,7 +934,9 @@ static int aspeed_pwm_tacho_probe(struct platform_device *pdev)
"missing or invalid reset controller device tree entry");
return PTR_ERR(priv->rst);
}
- reset_control_deassert(priv->rst);
+ ret = reset_control_deassert(priv->rst);
+ if (ret)
+ return ret;
ret = devm_add_action_or_reset(dev, aspeed_pwm_tacho_remove, priv);
if (ret)
diff --git a/drivers/hwmon/asus_rog_ryujin.c b/drivers/hwmon/asus_rog_ryujin.c
index 10a1f5aca988..f352036f0e52 100644
--- a/drivers/hwmon/asus_rog_ryujin.c
+++ b/drivers/hwmon/asus_rog_ryujin.c
@@ -18,15 +18,25 @@
#define USB_VENDOR_ID_ASUS_ROG 0x0b05
#define USB_PRODUCT_ID_RYUJIN_AIO 0x1988 /* ASUS ROG RYUJIN II 360 */
+struct rog_ryujin_device_info {
+ u8 temp_offset;
+ u8 pump_speed_offset;
+ u8 fan_speed_offset;
+ u8 duty_channel;
+ bool has_controller;
+};
+
+static const struct rog_ryujin_device_info rog_ryujin_ii_360_info = {
+ .temp_offset = 3,
+ .pump_speed_offset = 5,
+ .fan_speed_offset = 7,
+ .duty_channel = 0,
+ .has_controller = true,
+};
+
#define STATUS_VALIDITY 1500 /* ms */
#define MAX_REPORT_LENGTH 65
-/* Cooler status report offsets */
-#define RYUJIN_TEMP_SENSOR_1 3
-#define RYUJIN_TEMP_SENSOR_2 4
-#define RYUJIN_PUMP_SPEED 5
-#define RYUJIN_INTERNAL_FAN_SPEED 7
-
/* Cooler duty report offsets */
#define RYUJIN_PUMP_DUTY 4
#define RYUJIN_INTERNAL_FAN_DUTY 5
@@ -81,6 +91,7 @@ static const char *const rog_ryujin_speed_label[] = {
struct rog_ryujin_data {
struct hid_device *hdev;
struct device *hwmon_dev;
+ const struct rog_ryujin_device_info *info;
/* For reinitializing the completions below */
spinlock_t status_report_request_lock;
struct completion cooler_status_received;
@@ -112,6 +123,8 @@ static int rog_ryujin_pwm_to_percent(long val)
static umode_t rog_ryujin_is_visible(const void *data,
enum hwmon_sensor_types type, u32 attr, int channel)
{
+ const struct rog_ryujin_data *priv = data;
+
switch (type) {
case hwmon_temp:
switch (attr) {
@@ -123,6 +136,8 @@ static umode_t rog_ryujin_is_visible(const void *data,
}
break;
case hwmon_fan:
+ if (channel >= 2 && !priv->info->has_controller)
+ return 0;
switch (attr) {
case hwmon_fan_label:
case hwmon_fan_input:
@@ -132,6 +147,8 @@ static umode_t rog_ryujin_is_visible(const void *data,
}
break;
case hwmon_pwm:
+ if (channel >= 2 && !priv->info->has_controller)
+ return 0;
switch (attr) {
case hwmon_pwm_input:
return 0644;
@@ -156,6 +173,7 @@ static int rog_ryujin_write_expanded(struct rog_ryujin_data *priv, const u8 *cmd
static int rog_ryujin_execute_cmd(struct rog_ryujin_data *priv, const u8 *cmd, int cmd_length,
struct completion *status_completion)
{
+ unsigned long flags;
int ret;
/*
@@ -163,9 +181,9 @@ static int rog_ryujin_execute_cmd(struct rog_ryujin_data *priv, const u8 *cmd, i
* completion. Reinit is done because hidraw could have triggered
* the raw event parsing and marked the passed in completion as done.
*/
- spin_lock_bh(&priv->status_report_request_lock);
+ spin_lock_irqsave(&priv->status_report_request_lock, flags);
reinit_completion(status_completion);
- spin_unlock_bh(&priv->status_report_request_lock);
+ spin_unlock_irqrestore(&priv->status_report_request_lock, flags);
/* Send command for getting data */
ret = rog_ryujin_write_expanded(priv, cmd, cmd_length);
@@ -198,12 +216,14 @@ static int rog_ryujin_get_status(struct rog_ryujin_data *priv)
if (ret < 0)
return ret;
- /* Retrieve controller status (speeds) */
- ret =
- rog_ryujin_execute_cmd(priv, get_controller_speed_cmd, GET_CMD_LENGTH,
- &priv->controller_status_received);
- if (ret < 0)
- return ret;
+ if (priv->info->has_controller) {
+ /* Retrieve controller status (speeds) */
+ ret = rog_ryujin_execute_cmd(priv, get_controller_speed_cmd,
+ GET_CMD_LENGTH,
+ &priv->controller_status_received);
+ if (ret < 0)
+ return ret;
+ }
/* Retrieve cooler duty */
ret =
@@ -212,12 +232,14 @@ static int rog_ryujin_get_status(struct rog_ryujin_data *priv)
if (ret < 0)
return ret;
- /* Retrieve controller duty */
- ret =
- rog_ryujin_execute_cmd(priv, get_controller_duty_cmd, GET_CMD_LENGTH,
- &priv->controller_duty_received);
- if (ret < 0)
- return ret;
+ if (priv->info->has_controller) {
+ /* Retrieve controller duty */
+ ret = rog_ryujin_execute_cmd(priv, get_controller_duty_cmd,
+ GET_CMD_LENGTH,
+ &priv->controller_duty_received);
+ if (ret < 0)
+ return ret;
+ }
priv->updated = jiffies;
return 0;
@@ -289,6 +311,7 @@ static int rog_ryujin_write_fixed_duty(struct rog_ryujin_data *priv, int channel
return ret;
memcpy(set_cmd, set_cooler_duty_cmd, SET_CMD_LENGTH);
+ set_cmd[2] = priv->info->duty_channel;
/* Cooler duties are set as 0-100% */
val = rog_ryujin_pwm_to_percent(val);
@@ -388,20 +411,33 @@ static int rog_ryujin_raw_event(struct hid_device *hdev, struct hid_report *repo
int size)
{
struct rog_ryujin_data *priv = hid_get_drvdata(hdev);
+ unsigned long flags;
- if (data[0] != RYUJIN_CMD_PREFIX)
+ if (size < 2 || data[0] != RYUJIN_CMD_PREFIX)
return 0;
+ spin_lock_irqsave(&priv->status_report_request_lock, flags);
+
if (data[1] == RYUJIN_GET_COOLER_STATUS_CMD_RESPONSE) {
+ if (size <= priv->info->temp_offset + 1 ||
+ size <= priv->info->pump_speed_offset + 1 ||
+ size <= priv->info->fan_speed_offset + 1)
+ goto unlock;
+
/* Received coolant temp and speeds of pump and internal fan */
- priv->temp_input[0] =
- data[RYUJIN_TEMP_SENSOR_1] * 1000 + data[RYUJIN_TEMP_SENSOR_2] * 100;
- priv->speed_input[0] = get_unaligned_le16(data + RYUJIN_PUMP_SPEED);
- priv->speed_input[1] = get_unaligned_le16(data + RYUJIN_INTERNAL_FAN_SPEED);
+ priv->temp_input[0] = data[priv->info->temp_offset] * 1000 +
+ data[priv->info->temp_offset + 1] * 100;
+ priv->speed_input[0] =
+ get_unaligned_le16(data + priv->info->pump_speed_offset);
+ priv->speed_input[1] =
+ get_unaligned_le16(data + priv->info->fan_speed_offset);
if (!completion_done(&priv->cooler_status_received))
complete_all(&priv->cooler_status_received);
} else if (data[1] == RYUJIN_GET_CONTROLLER_SPEED_CMD_RESPONSE) {
+ if (size <= RYUJIN_CONTROLLER_SPEED_3 + 1)
+ goto unlock;
+
/* Received speeds of four fans attached to the controller */
priv->speed_input[2] = get_unaligned_le16(data + RYUJIN_CONTROLLER_SPEED_1);
priv->speed_input[3] = get_unaligned_le16(data + RYUJIN_CONTROLLER_SPEED_2);
@@ -411,6 +447,9 @@ static int rog_ryujin_raw_event(struct hid_device *hdev, struct hid_report *repo
if (!completion_done(&priv->controller_status_received))
complete_all(&priv->controller_status_received);
} else if (data[1] == RYUJIN_GET_COOLER_DUTY_CMD_RESPONSE) {
+ if (size <= RYUJIN_INTERNAL_FAN_DUTY)
+ goto unlock;
+
/* Received report for pump and internal fan duties (in %) */
if (data[RYUJIN_PUMP_DUTY] == 0 && data[RYUJIN_INTERNAL_FAN_DUTY] == 0) {
/*
@@ -428,7 +467,7 @@ static int rog_ryujin_raw_event(struct hid_device *hdev, struct hid_report *repo
* We're expecting a report, so parse it.
*/
goto read_cooler_duty;
- return 0;
+ goto unlock;
}
read_cooler_duty:
priv->duty_input[0] = rog_ryujin_percent_to_pwm(data[RYUJIN_PUMP_DUTY]);
@@ -437,6 +476,9 @@ static int rog_ryujin_raw_event(struct hid_device *hdev, struct hid_report *repo
if (!completion_done(&priv->cooler_duty_received))
complete_all(&priv->cooler_duty_received);
} else if (data[1] == RYUJIN_GET_CONTROLLER_DUTY_CMD_RESPONSE) {
+ if (size <= RYUJIN_CONTROLLER_DUTY)
+ goto unlock;
+
/* Received report for controller duty for fans (in PWM) */
if (data[RYUJIN_CONTROLLER_DUTY] == 0) {
/*
@@ -454,7 +496,7 @@ static int rog_ryujin_raw_event(struct hid_device *hdev, struct hid_report *repo
* We're expecting a report, so parse it.
*/
goto read_controller_duty;
- return 0;
+ goto unlock;
}
read_controller_duty:
priv->duty_input[2] = data[RYUJIN_CONTROLLER_DUTY];
@@ -463,6 +505,8 @@ static int rog_ryujin_raw_event(struct hid_device *hdev, struct hid_report *repo
complete_all(&priv->controller_duty_received);
}
+unlock:
+ spin_unlock_irqrestore(&priv->status_report_request_lock, flags);
return 0;
}
@@ -471,11 +515,15 @@ static int rog_ryujin_probe(struct hid_device *hdev, const struct hid_device_id
struct rog_ryujin_data *priv;
int ret;
+ if (!id->driver_data)
+ return -EINVAL;
+
priv = devm_kzalloc(&hdev->dev, sizeof(*priv), GFP_KERNEL);
if (!priv)
return -ENOMEM;
priv->hdev = hdev;
+ priv->info = (const struct rog_ryujin_device_info *)id->driver_data;
hid_set_drvdata(hdev, priv);
/*
@@ -546,7 +594,8 @@ static void rog_ryujin_remove(struct hid_device *hdev)
}
static const struct hid_device_id rog_ryujin_table[] = {
- { HID_USB_DEVICE(USB_VENDOR_ID_ASUS_ROG, USB_PRODUCT_ID_RYUJIN_AIO) },
+ { HID_USB_DEVICE(USB_VENDOR_ID_ASUS_ROG, USB_PRODUCT_ID_RYUJIN_AIO),
+ .driver_data = (kernel_ulong_t)&rog_ryujin_ii_360_info },
{ }
};
diff --git a/drivers/hwmon/chipcap2.c b/drivers/hwmon/chipcap2.c
index 4aecf463180f..4b6244e13e9e 100644
--- a/drivers/hwmon/chipcap2.c
+++ b/drivers/hwmon/chipcap2.c
@@ -89,11 +89,6 @@ struct cc2_data {
bool process_irqs;
};
-enum cc2_chan_addr {
- CC2_CHAN_TEMP = 0,
- CC2_CHAN_HUMIDITY,
-};
-
/* %RH as a per cent mille from a register value */
static long cc2_rh_convert(u16 data)
{
@@ -492,7 +487,7 @@ static irqreturn_t cc2_low_interrupt(int irq, void *data)
if (cc2->process_irqs) {
hwmon_notify_event(cc2->hwmon, hwmon_humidity,
- hwmon_humidity_min_alarm, CC2_CHAN_HUMIDITY);
+ hwmon_humidity_min_alarm, 0);
cc2->rh_alarm.low_alarm = true;
}
@@ -505,7 +500,7 @@ static irqreturn_t cc2_high_interrupt(int irq, void *data)
if (cc2->process_irqs) {
hwmon_notify_event(cc2->hwmon, hwmon_humidity,
- hwmon_humidity_max_alarm, CC2_CHAN_HUMIDITY);
+ hwmon_humidity_max_alarm, 0);
cc2->rh_alarm.high_alarm = true;
}
diff --git a/drivers/hwmon/corsair-cpro.c b/drivers/hwmon/corsair-cpro.c
index 8354a002f4c5..c09645152613 100644
--- a/drivers/hwmon/corsair-cpro.c
+++ b/drivers/hwmon/corsair-cpro.c
@@ -566,21 +566,18 @@ static int bootloader_show(struct seq_file *seqf, void *unused)
}
DEFINE_SHOW_ATTRIBUTE(bootloader);
-static void ccp_debugfs_init(struct ccp_device *ccp)
+static void ccp_debugfs_init(struct ccp_device *ccp, bool fw_valid, bool bl_valid)
{
char name[32];
- int ret;
scnprintf(name, sizeof(name), "corsaircpro-%s", dev_name(&ccp->hdev->dev));
ccp->debugfs = debugfs_create_dir(name, NULL);
- ret = get_fw_version(ccp);
- if (!ret)
+ if (fw_valid)
debugfs_create_file("firmware_version", 0444,
ccp->debugfs, ccp, &firmware_fops);
- ret = get_bl_version(ccp);
- if (!ret)
+ if (bl_valid)
debugfs_create_file("bootloader_version", 0444,
ccp->debugfs, ccp, &bootloader_fops);
}
@@ -588,6 +585,7 @@ static void ccp_debugfs_init(struct ccp_device *ccp)
static int ccp_probe(struct hid_device *hdev, const struct hid_device_id *id)
{
struct ccp_device *ccp;
+ bool fw_valid, bl_valid;
int ret;
ccp = devm_kzalloc(&hdev->dev, sizeof(*ccp), GFP_KERNEL);
@@ -632,17 +630,27 @@ static int ccp_probe(struct hid_device *hdev, const struct hid_device_id *id)
if (ret)
goto out_hw_close;
- ccp_debugfs_init(ccp);
+ /*
+ * Query the versions before registering the hwmon device: they send
+ * USB commands without holding ccp->mutex, which is only safe while
+ * nothing else can call send_usb_cmd().
+ */
+ fw_valid = !get_fw_version(ccp);
+ bl_valid = !get_bl_version(ccp);
ccp->hwmon_dev = hwmon_device_register_with_info(&hdev->dev, "corsaircpro",
ccp, &ccp_chip_info, NULL);
if (IS_ERR(ccp->hwmon_dev)) {
ret = PTR_ERR(ccp->hwmon_dev);
- goto out_hw_close;
+ goto out_debugfs_remove;
}
+ ccp_debugfs_init(ccp, fw_valid, bl_valid);
+
return 0;
+out_debugfs_remove:
+ debugfs_remove_recursive(ccp->debugfs);
out_hw_close:
hid_hw_close(hdev);
hid_device_io_stop(hdev);
diff --git a/drivers/hwmon/gpio-fan.c b/drivers/hwmon/gpio-fan.c
index 084828e1e281..df8bd9707605 100644
--- a/drivers/hwmon/gpio-fan.c
+++ b/drivers/hwmon/gpio-fan.c
@@ -12,6 +12,7 @@
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
+#include <linux/devm-helpers.h>
#include <linux/platform_device.h>
#include <linux/err.h>
#include <linux/kstrtox.h>
@@ -84,6 +85,7 @@ static DEVICE_ATTR_RO(fan1_alarm);
static int fan_alarm_init(struct gpio_fan_data *fan_data)
{
int alarm_irq;
+ int err;
struct device *dev = fan_data->dev;
/*
@@ -94,7 +96,11 @@ static int fan_alarm_init(struct gpio_fan_data *fan_data)
if (alarm_irq <= 0)
return 0;
- INIT_WORK(&fan_data->alarm_work, fan_alarm_notify);
+ err = devm_work_autocancel(dev, &fan_data->alarm_work,
+ fan_alarm_notify);
+ if (err)
+ return err;
+
irq_set_irq_type(alarm_irq, IRQ_TYPE_EDGE_BOTH);
return devm_request_irq(dev, alarm_irq, fan_alarm_irq_handler,
IRQF_SHARED, "GPIO fan alarm", fan_data);
@@ -606,8 +612,11 @@ static void gpio_fan_shutdown(struct platform_device *pdev)
{
struct gpio_fan_data *fan_data = platform_get_drvdata(pdev);
- if (fan_data->gpios)
+ if (fan_data->gpios) {
+ mutex_lock(&fan_data->lock);
set_fan_speed(fan_data, 0);
+ mutex_unlock(&fan_data->lock);
+ }
}
static int gpio_fan_runtime_suspend(struct device *dev)
diff --git a/drivers/hwmon/hwmon.c b/drivers/hwmon/hwmon.c
index 29dc90a2c3fe..8cf717ae8b1d 100644
--- a/drivers/hwmon/hwmon.c
+++ b/drivers/hwmon/hwmon.c
@@ -371,18 +371,17 @@ static ssize_t pec_store(struct device *dev, struct device_attribute *devattr,
* handling is not required.
*/
hwdev = to_hwmon_device(hdev);
- guard(mutex)(&hwdev->lock);
- if (hwdev->chip->ops->write) {
- err = hwdev->chip->ops->write(hdev, hwmon_chip, hwmon_chip_pec, 0, val);
- if (err && err != -EOPNOTSUPP)
- goto put;
+ scoped_guard(mutex, &hwdev->lock) {
+ if (hwdev->chip->ops->write) {
+ err = hwdev->chip->ops->write(hdev, hwmon_chip, hwmon_chip_pec, 0, val);
+ if (err && err != -EOPNOTSUPP)
+ goto put;
+ }
+ if (!val)
+ client->flags &= ~I2C_CLIENT_PEC;
+ else
+ client->flags |= I2C_CLIENT_PEC;
}
-
- if (!val)
- client->flags &= ~I2C_CLIENT_PEC;
- else
- client->flags |= I2C_CLIENT_PEC;
-
err = count;
put:
put_device(hdev);
diff --git a/drivers/hwmon/ina2xx.c b/drivers/hwmon/ina2xx.c
index 449a72c6b40b..958a1e3ec728 100644
--- a/drivers/hwmon/ina2xx.c
+++ b/drivers/hwmon/ina2xx.c
@@ -8,6 +8,7 @@
*/
#include <linux/bitfield.h>
+#include <linux/bitops.h>
#include <linux/bits.h>
#include <linux/delay.h>
#include <linux/device.h>
@@ -128,6 +129,17 @@ enum ina2xx_ids {
sy24655
};
+enum ina2xx_alert_type {
+ INA2XX_ALERT_NONE,
+ INA2XX_ALERT_CURRENT_LOW,
+ INA2XX_ALERT_CURRENT_HIGH,
+ INA2XX_ALERT_POWER_HIGH,
+ INA2XX_ALERT_BUS_VOLTAGE_LOW,
+ INA2XX_ALERT_BUS_VOLTAGE_HIGH,
+ INA2XX_ALERT_SHUNT_VOLTAGE_LOW,
+ INA2XX_ALERT_SHUNT_VOLTAGE_HIGH,
+};
+
struct ina2xx_config {
u16 config_default;
bool has_alerts; /* chip supports alerts and limits */
@@ -147,6 +159,7 @@ struct ina2xx_data {
const struct ina2xx_config *config;
enum ina2xx_ids chip;
+ enum ina2xx_alert_type active_alert;
long rshunt;
long current_lsb_uA;
long power_lsb_uW;
@@ -413,16 +426,78 @@ static u16 ina226_alert_to_reg(struct ina2xx_data *data, int reg, long val)
}
}
-static int ina226_alert_limit_read(struct ina2xx_data *data, u32 mask, int reg, long *val)
+static u32 ina2xx_alert_type_to_mask(enum ina2xx_alert_type alert)
+{
+ switch (alert) {
+ case INA2XX_ALERT_CURRENT_LOW:
+ case INA2XX_ALERT_SHUNT_VOLTAGE_LOW:
+ return INA226_SHUNT_UNDER_VOLTAGE_MASK;
+ case INA2XX_ALERT_CURRENT_HIGH:
+ case INA2XX_ALERT_SHUNT_VOLTAGE_HIGH:
+ return INA226_SHUNT_OVER_VOLTAGE_MASK;
+ case INA2XX_ALERT_BUS_VOLTAGE_LOW:
+ return INA226_BUS_UNDER_VOLTAGE_MASK;
+ case INA2XX_ALERT_BUS_VOLTAGE_HIGH:
+ return INA226_BUS_OVER_VOLTAGE_MASK;
+ case INA2XX_ALERT_POWER_HIGH:
+ return INA226_POWER_OVER_LIMIT_MASK;
+ case INA2XX_ALERT_NONE:
+ return 0;
+ default:
+ /* programmer error */
+ WARN_ON_ONCE(1);
+ return 0;
+ }
+}
+
+static enum ina2xx_alert_type ina2xx_mask_to_alert_type(u32 mask)
+{
+ int top_bit = fls(mask & INA226_ALERT_CONFIG_MASK);
+
+ if (!top_bit)
+ return INA2XX_ALERT_NONE;
+
+ /*
+ * Multiple bits may be set, with the highest-set function taking
+ * precedence according to the datasheet. Shunt voltage masks are
+ * assumed to map to voltage monitoring rather than current monitoring,
+ * since the latter isn't directly implemented in the hardware.
+ */
+ switch (BIT(top_bit - 1)) {
+ case INA226_SHUNT_OVER_VOLTAGE_MASK:
+ return INA2XX_ALERT_SHUNT_VOLTAGE_HIGH;
+ case INA226_SHUNT_UNDER_VOLTAGE_MASK:
+ return INA2XX_ALERT_SHUNT_VOLTAGE_LOW;
+ case INA226_BUS_OVER_VOLTAGE_MASK:
+ return INA2XX_ALERT_BUS_VOLTAGE_HIGH;
+ case INA226_BUS_UNDER_VOLTAGE_MASK:
+ return INA2XX_ALERT_BUS_VOLTAGE_LOW;
+ case INA226_POWER_OVER_LIMIT_MASK:
+ return INA2XX_ALERT_POWER_HIGH;
+ default:
+ return INA2XX_ALERT_NONE;
+ }
+}
+
+static int ina226_alert_limit_read(struct ina2xx_data *data, enum ina2xx_alert_type alert,
+ int reg, long *val)
{
struct regmap *regmap = data->regmap;
int regval;
+ u32 mask;
int ret;
+ /* Avoid nonzero reads from inactive alerts caused by shared limit register */
+ if (data->active_alert != alert) {
+ *val = 0;
+ return 0;
+ }
+
ret = regmap_read(regmap, INA226_MASK_ENABLE, ®val);
if (ret)
return ret;
+ mask = ina2xx_alert_type_to_mask(alert);
if (regval & mask) {
ret = regmap_read(regmap, INA226_ALERT_LIMIT, ®val);
if (ret)
@@ -434,9 +509,11 @@ static int ina226_alert_limit_read(struct ina2xx_data *data, u32 mask, int reg,
return 0;
}
-static int ina226_alert_limit_write(struct ina2xx_data *data, u32 mask, int reg, long val)
+static int ina226_alert_limit_write(struct ina2xx_data *data, enum ina2xx_alert_type alert,
+ int reg, long val)
{
struct regmap *regmap = data->regmap;
+ u32 mask;
int ret;
if (val < 0)
@@ -451,15 +528,22 @@ static int ina226_alert_limit_write(struct ina2xx_data *data, u32 mask, int reg,
INA226_ALERT_CONFIG_MASK, 0);
if (ret < 0)
return ret;
+ data->active_alert = INA2XX_ALERT_NONE;
ret = regmap_write(regmap, INA226_ALERT_LIMIT,
ina226_alert_to_reg(data, reg, val));
if (ret < 0)
return ret;
- if (val)
- return regmap_update_bits(regmap, INA226_MASK_ENABLE,
- INA226_ALERT_CONFIG_MASK, mask);
+ if (val) {
+ mask = ina2xx_alert_type_to_mask(alert);
+ ret = regmap_update_bits(regmap, INA226_MASK_ENABLE,
+ INA226_ALERT_CONFIG_MASK, mask);
+ if (ret < 0)
+ return ret;
+ data->active_alert = alert;
+ }
+
return 0;
}
@@ -483,15 +567,26 @@ static int ina2xx_chip_read(struct device *dev, u32 attr, long *val)
return 0;
}
-static int ina226_alert_read(struct regmap *regmap, u32 mask, long *val)
+static int ina226_alert_read(struct ina2xx_data *data, enum ina2xx_alert_type alert, long *val)
{
unsigned int regval;
+ u32 mask;
int ret;
- ret = regmap_read_bypassed(regmap, INA226_MASK_ENABLE, ®val);
+ /*
+ * With alert latching, reading alerts from hardware also clears the
+ * alert, so return early if the alert is inactive.
+ */
+ if (data->active_alert != alert) {
+ *val = 0;
+ return 0;
+ }
+
+ ret = regmap_read_bypassed(data->regmap, INA226_MASK_ENABLE, ®val);
if (ret)
return ret;
+ mask = ina2xx_alert_type_to_mask(alert);
*val = (regval & mask) && (regval & INA226_ALERT_FUNCTION_FLAG);
return 0;
@@ -500,10 +595,10 @@ static int ina226_alert_read(struct regmap *regmap, u32 mask, long *val)
static int ina2xx_in_read(struct device *dev, u32 attr, int channel, long *val)
{
int voltage_reg = channel ? INA2XX_BUS_VOLTAGE : INA2XX_SHUNT_VOLTAGE;
- u32 under_voltage_mask = channel ? INA226_BUS_UNDER_VOLTAGE_MASK
- : INA226_SHUNT_UNDER_VOLTAGE_MASK;
- u32 over_voltage_mask = channel ? INA226_BUS_OVER_VOLTAGE_MASK
- : INA226_SHUNT_OVER_VOLTAGE_MASK;
+ enum ina2xx_alert_type under_voltage_alert = channel ? INA2XX_ALERT_BUS_VOLTAGE_LOW
+ : INA2XX_ALERT_SHUNT_VOLTAGE_LOW;
+ enum ina2xx_alert_type over_voltage_alert = channel ? INA2XX_ALERT_BUS_VOLTAGE_HIGH
+ : INA2XX_ALERT_SHUNT_VOLTAGE_HIGH;
struct ina2xx_data *data = dev_get_drvdata(dev);
struct regmap *regmap = data->regmap;
unsigned int regval;
@@ -517,15 +612,15 @@ static int ina2xx_in_read(struct device *dev, u32 attr, int channel, long *val)
*val = ina2xx_get_value(data, voltage_reg, regval);
break;
case hwmon_in_lcrit:
- return ina226_alert_limit_read(data, under_voltage_mask,
+ return ina226_alert_limit_read(data, under_voltage_alert,
voltage_reg, val);
case hwmon_in_crit:
- return ina226_alert_limit_read(data, over_voltage_mask,
+ return ina226_alert_limit_read(data, over_voltage_alert,
voltage_reg, val);
case hwmon_in_lcrit_alarm:
- return ina226_alert_read(regmap, under_voltage_mask, val);
+ return ina226_alert_read(data, under_voltage_alert, val);
case hwmon_in_crit_alarm:
- return ina226_alert_read(regmap, over_voltage_mask, val);
+ return ina226_alert_read(data, over_voltage_alert, val);
default:
return -EOPNOTSUPP;
}
@@ -579,10 +674,10 @@ static int ina2xx_power_read(struct device *dev, u32 attr, long *val)
case hwmon_power_average:
return sy24655_average_power_read(data, SY24655_EIN, val);
case hwmon_power_crit:
- return ina226_alert_limit_read(data, INA226_POWER_OVER_LIMIT_MASK,
+ return ina226_alert_limit_read(data, INA2XX_ALERT_POWER_HIGH,
INA2XX_POWER, val);
case hwmon_power_crit_alarm:
- return ina226_alert_read(data->regmap, INA226_POWER_OVER_LIMIT_MASK, val);
+ return ina226_alert_read(data, INA2XX_ALERT_POWER_HIGH, val);
default:
return -EOPNOTSUPP;
}
@@ -618,15 +713,15 @@ static int ina2xx_curr_read(struct device *dev, u32 attr, long *val)
*val = ina2xx_get_value(data, INA2XX_CURRENT, regval);
return 0;
case hwmon_curr_lcrit:
- return ina226_alert_limit_read(data, INA226_SHUNT_UNDER_VOLTAGE_MASK,
+ return ina226_alert_limit_read(data, INA2XX_ALERT_CURRENT_LOW,
INA2XX_CURRENT, val);
case hwmon_curr_crit:
- return ina226_alert_limit_read(data, INA226_SHUNT_OVER_VOLTAGE_MASK,
+ return ina226_alert_limit_read(data, INA2XX_ALERT_CURRENT_HIGH,
INA2XX_CURRENT, val);
case hwmon_curr_lcrit_alarm:
- return ina226_alert_read(regmap, INA226_SHUNT_UNDER_VOLTAGE_MASK, val);
+ return ina226_alert_read(data, INA2XX_ALERT_CURRENT_LOW, val);
case hwmon_curr_crit_alarm:
- return ina226_alert_read(regmap, INA226_SHUNT_OVER_VOLTAGE_MASK, val);
+ return ina226_alert_read(data, INA2XX_ALERT_CURRENT_HIGH, val);
default:
return -EOPNOTSUPP;
}
@@ -670,12 +765,12 @@ static int ina2xx_in_write(struct device *dev, u32 attr, int channel, long val)
switch (attr) {
case hwmon_in_lcrit:
return ina226_alert_limit_write(data,
- channel ? INA226_BUS_UNDER_VOLTAGE_MASK : INA226_SHUNT_UNDER_VOLTAGE_MASK,
+ channel ? INA2XX_ALERT_BUS_VOLTAGE_LOW : INA2XX_ALERT_SHUNT_VOLTAGE_LOW,
channel ? INA2XX_BUS_VOLTAGE : INA2XX_SHUNT_VOLTAGE,
val);
case hwmon_in_crit:
return ina226_alert_limit_write(data,
- channel ? INA226_BUS_OVER_VOLTAGE_MASK : INA226_SHUNT_OVER_VOLTAGE_MASK,
+ channel ? INA2XX_ALERT_BUS_VOLTAGE_HIGH : INA2XX_ALERT_SHUNT_VOLTAGE_HIGH,
channel ? INA2XX_BUS_VOLTAGE : INA2XX_SHUNT_VOLTAGE,
val);
default:
@@ -690,7 +785,7 @@ static int ina2xx_power_write(struct device *dev, u32 attr, long val)
switch (attr) {
case hwmon_power_crit:
- return ina226_alert_limit_write(data, INA226_POWER_OVER_LIMIT_MASK,
+ return ina226_alert_limit_write(data, INA2XX_ALERT_POWER_HIGH,
INA2XX_POWER, val);
default:
return -EOPNOTSUPP;
@@ -704,10 +799,10 @@ static int ina2xx_curr_write(struct device *dev, u32 attr, long val)
switch (attr) {
case hwmon_curr_lcrit:
- return ina226_alert_limit_write(data, INA226_SHUNT_UNDER_VOLTAGE_MASK,
+ return ina226_alert_limit_write(data, INA2XX_ALERT_CURRENT_LOW,
INA2XX_CURRENT, val);
case hwmon_curr_crit:
- return ina226_alert_limit_write(data, INA226_SHUNT_OVER_VOLTAGE_MASK,
+ return ina226_alert_limit_write(data, INA2XX_ALERT_CURRENT_HIGH,
INA2XX_CURRENT, val);
default:
return -EOPNOTSUPP;
@@ -868,8 +963,12 @@ static ssize_t shunt_resistor_show(struct device *dev,
struct device_attribute *da, char *buf)
{
struct ina2xx_data *data = dev_get_drvdata(dev);
+ long rshunt;
- return sysfs_emit(buf, "%li\n", data->rshunt);
+ scoped_guard(hwmon_lock, dev) {
+ rshunt = data->rshunt;
+ }
+ return sysfs_emit(buf, "%li\n", rshunt);
}
static ssize_t shunt_resistor_store(struct device *dev,
@@ -925,6 +1024,16 @@ static int ina2xx_init(struct device *dev, struct ina2xx_data *data)
if (data->config->has_alerts) {
bool active_high = device_property_read_bool(dev, "ti,alert-polarity-active-high");
+ unsigned int mask_enable;
+
+ /*
+ * Infer active alert from MASK_ENABLE in case it's already
+ * configured (e.g., by a past probe or firmware)
+ */
+ ret = regmap_read(regmap, INA226_MASK_ENABLE, &mask_enable);
+ if (ret < 0)
+ return ret;
+ data->active_alert = ina2xx_mask_to_alert_type(mask_enable);
regmap_update_bits(regmap, INA226_MASK_ENABLE,
INA226_ALERT_LATCH_ENABLE | INA226_ALERT_POLARITY,
diff --git a/drivers/hwmon/ltc4282.c b/drivers/hwmon/ltc4282.c
index b1675dc5b3c7..54ba4b8542e9 100644
--- a/drivers/hwmon/ltc4282.c
+++ b/drivers/hwmon/ltc4282.c
@@ -1106,7 +1106,7 @@ static const struct clk_ops ltc4282_ops = {
static int ltc428_clk_provider_setup(struct ltc4282_state *st,
struct device *dev)
{
- struct clk_init_data init;
+ struct clk_init_data init = {};
int ret;
if (!IS_ENABLED(CONFIG_COMMON_CLK))
diff --git a/drivers/hwmon/mcp9982.c b/drivers/hwmon/mcp9982.c
index 9e19e2697e25..3918dc36c946 100644
--- a/drivers/hwmon/mcp9982.c
+++ b/drivers/hwmon/mcp9982.c
@@ -395,6 +395,8 @@ static int mcp9982_read(struct device *dev, enum hwmon_sensor_types type, u32 at
reg_status, !(reg_status & MCP9982_STATUS_BUSY),
MCP9982_WAKE_UP_TIME_US,
MCP9982_WAKE_UP_TIME_US * 10);
+ if (ret)
+ return ret;
break;
}
break;
diff --git a/drivers/hwmon/nct6694-hwmon.c b/drivers/hwmon/nct6694-hwmon.c
index 6dcf22ca5018..9a9a4db434c4 100644
--- a/drivers/hwmon/nct6694-hwmon.c
+++ b/drivers/hwmon/nct6694-hwmon.c
@@ -159,6 +159,9 @@ static inline s8 temp_to_reg(long val)
#define NCT6694_HWMON_TEMP_CONFIG (HWMON_T_INPUT | HWMON_T_ENABLE | \
HWMON_T_MAX | HWMON_T_MAX_HYST | \
HWMON_T_MAX_ALARM)
+#define NCT6694_HWMON_DTIN_CONFIG (HWMON_T_INPUT | \
+ HWMON_T_MAX | HWMON_T_MAX_HYST | \
+ HWMON_T_MAX_ALARM)
#define NCT6694_HWMON_FAN_CONFIG (HWMON_F_INPUT | HWMON_F_ENABLE | \
HWMON_F_MIN | HWMON_F_MIN_ALARM)
#define NCT6694_HWMON_PWM_CONFIG (HWMON_PWM_INPUT | HWMON_PWM_ENABLE | \
@@ -193,22 +196,22 @@ static const struct hwmon_channel_info *nct6694_info[] = {
NCT6694_HWMON_TEMP_CONFIG, /* TDP2 */
NCT6694_HWMON_TEMP_CONFIG, /* TDP3 */
NCT6694_HWMON_TEMP_CONFIG, /* TDP4 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN0 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN1 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN2 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN3 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN4 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN5 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN6 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN7 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN8 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN9 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN10 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN11 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN12 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN13 */
- NCT6694_HWMON_TEMP_CONFIG, /* DTIN14 */
- NCT6694_HWMON_TEMP_CONFIG), /* DTIN15 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN0 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN1 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN2 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN3 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN4 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN5 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN6 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN7 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN8 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN9 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN10 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN11 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN12 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN13 */
+ NCT6694_HWMON_DTIN_CONFIG, /* DTIN14 */
+ NCT6694_HWMON_DTIN_CONFIG), /* DTIN15 */
HWMON_CHANNEL_INFO(fan,
NCT6694_HWMON_FAN_CONFIG, /* FIN0 */
diff --git a/drivers/hwmon/pmbus/pmbus_core.c b/drivers/hwmon/pmbus/pmbus_core.c
index 3b219ed5386f..a73a118fca82 100644
--- a/drivers/hwmon/pmbus/pmbus_core.c
+++ b/drivers/hwmon/pmbus/pmbus_core.c
@@ -1192,7 +1192,9 @@ static int pmbus_get_boolean(struct i2c_client *client, struct pmbus_boolean *b,
regval = status & mask;
if (regval) {
- if (data->revision >= PMBUS_REV_12) {
+ /* Generic STATUS_WORD alarms are not individually clearable. */
+ if (data->revision >= PMBUS_REV_12 &&
+ reg != PMBUS_STATUS_WORD) {
ret = _pmbus_write_byte_data(client, page, reg, regval);
if (ret)
return ret;
diff --git a/drivers/hwmon/sht4x.c b/drivers/hwmon/sht4x.c
index 9cace0e8acda..a97dda9e92dc 100644
--- a/drivers/hwmon/sht4x.c
+++ b/drivers/hwmon/sht4x.c
@@ -277,6 +277,8 @@ static ssize_t heater_enable_store(struct device *dev,
heating_time_bound = 1100;
}
+ guard(hwmon_lock)(dev);
+
if (time_before(jiffies, data->heating_complete))
return -EBUSY;
@@ -286,7 +288,7 @@ static ssize_t heater_enable_store(struct device *dev,
data->heating_complete = jiffies + msecs_to_jiffies(heating_time_bound);
data->data_pending = true;
- return 0;
+ return count;
}
static ssize_t heater_power_show(struct device *dev,
@@ -314,6 +316,8 @@ static ssize_t heater_power_store(struct device *dev,
if (power != 20 && power != 110 && power != 200)
return -EINVAL;
+ guard(hwmon_lock)(dev);
+
data->heater_power = power;
return count;
@@ -344,6 +348,8 @@ static ssize_t heater_time_store(struct device *dev,
if (time != 100 && time != 1000)
return -EINVAL;
+ guard(hwmon_lock)(dev);
+
data->heater_time = time;
return count;
diff --git a/drivers/hwmon/yogafan.c b/drivers/hwmon/yogafan.c
index 605cc928f21f..314ee61955dd 100644
--- a/drivers/hwmon/yogafan.c
+++ b/drivers/hwmon/yogafan.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* yoga_fan.c - Lenovo Yoga/Legion Fan Hardware Monitoring Driver
*
* Provides fan speed monitoring for Lenovo Yoga, Legion, and IdeaPad
diff --git a/drivers/i2c/busses/i2c-designware-common.c b/drivers/i2c/busses/i2c-designware-common.c
index e4dfa2ec58bb..a1eca6cd4b75 100644
--- a/drivers/i2c/busses/i2c-designware-common.c
+++ b/drivers/i2c/busses/i2c-designware-common.c
@@ -33,6 +33,8 @@
#include <linux/types.h>
#include <linux/units.h>
+#include <linux/designware_i2c.h>
+
#include "i2c-designware-core.h"
#define DW_IC_DEFAULT_BUS_CAPACITANCE_pF 100
diff --git a/drivers/i2c/busses/i2c-designware-core.h b/drivers/i2c/busses/i2c-designware-core.h
index c71aa2dd368d..2c929a6e8da2 100644
--- a/drivers/i2c/busses/i2c-designware-core.h
+++ b/drivers/i2c/busses/i2c-designware-core.h
@@ -18,6 +18,8 @@
#include <linux/regmap.h>
#include <linux/types.h>
+#include <linux/designware_i2c.h>
+
#define DW_IC_DEFAULT_FUNCTIONALITY (I2C_FUNC_I2C | \
I2C_FUNC_SMBUS_BYTE | \
I2C_FUNC_SMBUS_BYTE_DATA | \
@@ -25,23 +27,6 @@
I2C_FUNC_SMBUS_BLOCK_DATA | \
I2C_FUNC_SMBUS_I2C_BLOCK)
-#define DW_IC_CON_MASTER BIT(0)
-#define DW_IC_CON_SPEED_STD (1 << 1)
-#define DW_IC_CON_SPEED_FAST (2 << 1)
-#define DW_IC_CON_SPEED_HIGH (3 << 1)
-#define DW_IC_CON_SPEED_MASK GENMASK(2, 1)
-#define DW_IC_CON_10BITADDR_SLAVE BIT(3)
-#define DW_IC_CON_10BITADDR_MASTER BIT(4)
-#define DW_IC_CON_RESTART_EN BIT(5)
-#define DW_IC_CON_SLAVE_DISABLE BIT(6)
-#define DW_IC_CON_STOP_DET_IFADDRESSED BIT(7)
-#define DW_IC_CON_TX_EMPTY_CTRL BIT(8)
-#define DW_IC_CON_RX_FIFO_FULL_HLD_CTRL BIT(9)
-#define DW_IC_CON_BUS_CLEAR_CTRL BIT(11)
-
-#define DW_IC_DATA_CMD_DAT GENMASK(7, 0)
-#define DW_IC_DATA_CMD_FIRST_DATA_BYTE BIT(11)
-
/*
* Register access parameters
*/
@@ -55,65 +40,9 @@
#define DW_IC_FIFO_RX_FIELD GENMASK(15, 8)
#define DW_IC_FIFO_MIN_DEPTH 2
-/*
- * Registers offset
- */
-#define DW_IC_CON 0x00
-#define DW_IC_TAR 0x04
-#define DW_IC_SAR 0x08
-#define DW_IC_DATA_CMD 0x10
-#define DW_IC_SS_SCL_HCNT 0x14
-#define DW_IC_SS_SCL_LCNT 0x18
-#define DW_IC_FS_SCL_HCNT 0x1c
-#define DW_IC_FS_SCL_LCNT 0x20
-#define DW_IC_HS_SCL_HCNT 0x24
-#define DW_IC_HS_SCL_LCNT 0x28
-#define DW_IC_INTR_STAT 0x2c
-#define DW_IC_INTR_MASK 0x30
-#define DW_IC_RAW_INTR_STAT 0x34
-#define DW_IC_RX_TL 0x38
-#define DW_IC_TX_TL 0x3c
-#define DW_IC_CLR_INTR 0x40
-#define DW_IC_CLR_RX_UNDER 0x44
-#define DW_IC_CLR_RX_OVER 0x48
-#define DW_IC_CLR_TX_OVER 0x4c
-#define DW_IC_CLR_RD_REQ 0x50
-#define DW_IC_CLR_TX_ABRT 0x54
-#define DW_IC_CLR_RX_DONE 0x58
-#define DW_IC_CLR_ACTIVITY 0x5c
-#define DW_IC_CLR_STOP_DET 0x60
-#define DW_IC_CLR_START_DET 0x64
-#define DW_IC_CLR_GEN_CALL 0x68
-#define DW_IC_ENABLE 0x6c
-#define DW_IC_STATUS 0x70
-#define DW_IC_TXFLR 0x74
-#define DW_IC_RXFLR 0x78
-#define DW_IC_SDA_HOLD 0x7c
-#define DW_IC_TX_ABRT_SOURCE 0x80
-#define DW_IC_ENABLE_STATUS 0x9c
-#define DW_IC_CLR_RESTART_DET 0xa8
-#define DW_IC_SMBUS_INTR_MASK 0xcc
-#define DW_IC_COMP_PARAM_1 0xf4
-#define DW_IC_COMP_VERSION 0xf8
#define DW_IC_SDA_HOLD_MIN_VERS 0x3131312A /* "111*" == v1.11* */
-#define DW_IC_COMP_TYPE 0xfc
#define DW_IC_COMP_TYPE_VALUE 0x44570140 /* "DW" + 0x0140 */
-#define DW_IC_INTR_RX_UNDER BIT(0)
-#define DW_IC_INTR_RX_OVER BIT(1)
-#define DW_IC_INTR_RX_FULL BIT(2)
-#define DW_IC_INTR_TX_OVER BIT(3)
-#define DW_IC_INTR_TX_EMPTY BIT(4)
-#define DW_IC_INTR_RD_REQ BIT(5)
-#define DW_IC_INTR_TX_ABRT BIT(6)
-#define DW_IC_INTR_RX_DONE BIT(7)
-#define DW_IC_INTR_ACTIVITY BIT(8)
-#define DW_IC_INTR_STOP_DET BIT(9)
-#define DW_IC_INTR_START_DET BIT(10)
-#define DW_IC_INTR_GEN_CALL BIT(11)
-#define DW_IC_INTR_RESTART_DET BIT(12)
-#define DW_IC_INTR_MST_ON_HOLD BIT(13)
-
#define DW_IC_INTR_DEFAULT_MASK (DW_IC_INTR_RX_FULL | \
DW_IC_INTR_TX_ABRT | \
DW_IC_INTR_STOP_DET)
@@ -123,16 +52,6 @@
DW_IC_INTR_RX_UNDER | \
DW_IC_INTR_RD_REQ)
-#define DW_IC_ENABLE_ENABLE BIT(0)
-#define DW_IC_ENABLE_ABORT BIT(1)
-
-#define DW_IC_STATUS_ACTIVITY BIT(0)
-#define DW_IC_STATUS_TFE BIT(2)
-#define DW_IC_STATUS_RFNE BIT(3)
-#define DW_IC_STATUS_MASTER_ACTIVITY BIT(5)
-#define DW_IC_STATUS_SLAVE_ACTIVITY BIT(6)
-#define DW_IC_STATUS_MASTER_HOLD_TX_FIFO_EMPTY BIT(7)
-
#define DW_IC_SDA_HOLD_RX_SHIFT 16
#define DW_IC_SDA_HOLD_RX_MASK GENMASK(23, 16)
diff --git a/drivers/i2c/busses/i2c-designware-master.c b/drivers/i2c/busses/i2c-designware-master.c
index 7a301c8b604e..a1bcc3797e4f 100644
--- a/drivers/i2c/busses/i2c-designware-master.c
+++ b/drivers/i2c/busses/i2c-designware-master.c
@@ -25,6 +25,8 @@
#include <linux/regmap.h>
#include <linux/reset.h>
+#include <linux/designware_i2c.h>
+
#include "i2c-designware-core.h"
#define AMD_TIMEOUT_MIN_US 25
diff --git a/drivers/i2c/busses/i2c-designware-slave.c b/drivers/i2c/busses/i2c-designware-slave.c
index ad0d5fbfa6d5..0abcc7757b23 100644
--- a/drivers/i2c/busses/i2c-designware-slave.c
+++ b/drivers/i2c/busses/i2c-designware-slave.c
@@ -19,6 +19,8 @@
#include <linux/pm_runtime.h>
#include <linux/regmap.h>
+#include <linux/designware_i2c.h>
+
#include "i2c-designware-core.h"
int i2c_dw_reg_slave(struct i2c_client *slave)
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index 2563ebe9f246..3509be3fc744 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -906,7 +906,9 @@ static void free_ga_log(struct amd_iommu *iommu)
{
#ifdef CONFIG_IRQ_REMAP
iommu_free_pages(iommu->ga_log);
+ iommu->ga_log = NULL;
iommu_free_pages(iommu->ga_log_tail);
+ iommu->ga_log_tail = NULL;
#endif
}
@@ -953,6 +955,9 @@ static int iommu_init_ga_log(struct amd_iommu *iommu)
if (WARN_ON_ONCE(!AMD_IOMMU_GUEST_IR_VAPIC(amd_iommu_guest_ir)))
return -EINVAL;
+ if (iommu->ga_log && iommu->ga_log_tail)
+ return 0;
+
iommu->ga_log = iommu_alloc_pages_node_sz(nid, GFP_KERNEL, GA_LOG_SIZE);
if (!iommu->ga_log)
goto err_out;
@@ -1914,19 +1919,20 @@ static int __init init_iommu_one(struct amd_iommu *iommu, struct ivhd_header *h,
else
iommu->mmio_phys_end = MMIO_CNTR_CONF_OFFSET;
+ if (h->efr_attr & BIT(IOMMU_IVHD_ATTR_HATDIS_SHIFT)) {
+ pr_warn_once("Host Address Translation is not supported.\n");
+ amd_iommu_hatdis = true;
+ }
+
/* XT and GAM require GA mode. */
if ((h->efr_reg & (0x1 << IOMMU_EFR_GASUP_SHIFT)) == 0) {
amd_iommu_guest_ir = AMD_IOMMU_GUEST_IR_LEGACY;
+ break;
} else {
if (h->efr_reg & BIT(IOMMU_EFR_XTSUP_SHIFT))
amd_iommu_xt_mode = IRQ_REMAP_X2APIC_MODE;
}
- if (h->efr_attr & BIT(IOMMU_IVHD_ATTR_HATDIS_SHIFT)) {
- pr_warn_once("Host Address Translation is not supported.\n");
- amd_iommu_hatdis = true;
- }
-
early_iommu_features_init(iommu, h);
break;
diff --git a/drivers/iommu/amd/nested.c b/drivers/iommu/amd/nested.c
index 63b53b29e029..f1c7987fc585 100644
--- a/drivers/iommu/amd/nested.c
+++ b/drivers/iommu/amd/nested.c
@@ -96,7 +96,7 @@ struct iommu_domain *
amd_iommu_alloc_domain_nested(struct iommufd_viommu *viommu, u32 flags,
const struct iommu_user_data *user_data)
{
- int ret;
+ int ret, hdom_id;
unsigned long irqflags;
struct nested_domain *ndom;
struct guest_domain_mapping_info *gdom_info;
@@ -161,8 +161,8 @@ amd_iommu_alloc_domain_nested(struct iommufd_viommu *viommu, u32 flags,
}
/* The gDomID does not exist. We allocate new hdom_id */
- gdom_info->hdom_id = amd_iommu_pdom_id_alloc();
- if (gdom_info->hdom_id <= 0) {
+ hdom_id = amd_iommu_pdom_id_alloc();
+ if (hdom_id <= 0) {
__xa_cmpxchg(&aviommu->gdomid_array,
ndom->gdom_id, gdom_info, NULL, GFP_ATOMIC);
xa_unlock_irqrestore(&aviommu->gdomid_array, irqflags);
@@ -170,6 +170,7 @@ amd_iommu_alloc_domain_nested(struct iommufd_viommu *viommu, u32 flags,
goto out_err_gdom_info;
}
+ gdom_info->hdom_id = hdom_id;
ndom->gdom_info = gdom_info;
refcount_set(&gdom_info->users, 1);
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 35b7b2fd4a12..b52dae11fbbe 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -4741,6 +4741,8 @@ static void arm_smmu_disable_action(void *data)
{
struct arm_smmu_device *smmu = data;
+ if (smmu->impl_ops && smmu->impl_ops->device_disable)
+ smmu->impl_ops->device_disable(smmu);
arm_smmu_device_disable(smmu);
}
@@ -5555,8 +5557,6 @@ static void arm_smmu_device_shutdown(struct platform_device *pdev)
{
struct arm_smmu_device *smmu = platform_get_drvdata(pdev);
- if (smmu->impl_ops && smmu->impl_ops->device_disable)
- smmu->impl_ops->device_disable(smmu);
arm_smmu_device_disable(smmu);
}
diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c
index cec3ddd7ab10..fe8e6d0f8a23 100644
--- a/drivers/iommu/riscv/iommu.c
+++ b/drivers/iommu/riscv/iommu.c
@@ -382,78 +382,97 @@ static int riscv_iommu_queue_wait(struct riscv_iommu_queue *queue,
(int)(cons - index) > 0, 0, timeout_us);
}
-/* Enqueue an entry and wait to be processed if timeout_us > 0
- *
- * Error handling for IOMMU hardware not responding in reasonable time
- * will be added as separate patch series along with other RAS features.
- * For now, only report hardware failure and continue.
- */
-static unsigned int riscv_iommu_queue_send(struct riscv_iommu_queue *queue,
- void *entry, size_t entry_size)
+static int riscv_iommu_queue_wait_for_space(struct riscv_iommu_queue *queue,
+ unsigned int last)
{
- unsigned int prod;
unsigned int head;
unsigned int tail;
+ unsigned int hw_head;
unsigned long flags;
+ int ret;
- /* Do not preempt submission flow. */
- local_irq_save(flags);
+ ret = riscv_iommu_readl_timeout(queue->iommu, Q_HEAD(queue), hw_head,
+ !(hw_head & ~queue->mask) && hw_head != last,
+ 0, RISCV_IOMMU_QUEUE_TIMEOUT);
+ if (ret)
+ return ret;
- /* 1. Allocate some space in the queue */
- prod = atomic_inc_return(&queue->prod) - 1;
+ raw_spin_lock_irqsave(&queue->lock, flags);
head = atomic_read(&queue->head);
+ tail = atomic_read(&queue->tail);
+ if ((tail - head) >= queue->mask) {
+ last = Q_ITEM(queue, head);
+ /*
+ * Re-read hw_head under the lock so that it is consistent with
+ * the freshly computed 'last'. Using the pre-lock snapshot
+ * could produce a stale value that wraps around relative to the
+ * new 'last', advancing the shadow head past entries that have
+ * not yet been consumed by the hardware.
+ */
+ hw_head = riscv_iommu_readl(queue->iommu, Q_HEAD(queue));
+ if (!(hw_head & ~queue->mask) && hw_head != last)
+ atomic_add((hw_head - last) & queue->mask, &queue->head);
+ }
+ raw_spin_unlock_irqrestore(&queue->lock, flags);
- /* 2. Wait for space availability. */
- if ((prod - head) > queue->mask) {
- if (readx_poll_timeout(atomic_read, &queue->head,
- head, (prod - head) < queue->mask,
- 0, RISCV_IOMMU_QUEUE_TIMEOUT))
- goto err_busy;
- } else if ((prod - head) == queue->mask) {
- const unsigned int last = Q_ITEM(queue, head);
+ return 0;
+}
+
+/* Enqueue an entry and publish it to the hardware queue. */
+static int riscv_iommu_queue_send(struct riscv_iommu_queue *queue,
+ void *entry, size_t entry_size,
+ unsigned int *out_prod)
+{
+ unsigned int prod;
+ unsigned int head;
+ unsigned long flags;
+ int ret;
+
+ /* 1. Wait for space availability and reserve the next slot. */
+ for (;;) {
+ raw_spin_lock_irqsave(&queue->lock, flags);
+
+ prod = atomic_read(&queue->tail);
+ head = atomic_read(&queue->head);
+
+ if ((prod - head) < queue->mask)
+ break;
+
+ head = Q_ITEM(queue, head);
+ raw_spin_unlock_irqrestore(&queue->lock, flags);
- if (riscv_iommu_readl_timeout(queue->iommu, Q_HEAD(queue), head,
- !(head & ~queue->mask) && head != last,
- 0, RISCV_IOMMU_QUEUE_TIMEOUT))
+ ret = riscv_iommu_queue_wait_for_space(queue, head);
+ if (ret)
goto err_busy;
- atomic_add((head - last) & queue->mask, &queue->head);
}
- /* 3. Store entry in the ring buffer */
+ /* 2. Store entry in the ring buffer. */
memcpy(queue->base + Q_ITEM(queue, prod) * entry_size, entry, entry_size);
- /* 4. Wait for all previous entries to be ready */
- if (readx_poll_timeout(atomic_read, &queue->tail, tail, prod == tail,
- 0, RISCV_IOMMU_QUEUE_TIMEOUT))
- goto err_busy;
-
- /*
- * 5. Make sure the ring buffer update (whether in normal or I/O memory) is
- * completed and visible before signaling the tail doorbell to fetch
- * the next command. 'fence ow, ow'
- */
+ /* 3. Make sure the entry is visible before updating the queue tail. */
dma_wmb();
riscv_iommu_writel(queue->iommu, Q_TAIL(queue), Q_ITEM(queue, prod + 1));
/*
- * 6. Make sure the doorbell write to the device has finished before updating
- * the shadow tail index in normal memory. 'fence o, w'
+ * 4. Make sure the doorbell write to the device has finished before
+ * updating the shadow tail index in normal memory. 'fence o, w'
*/
#ifdef CONFIG_MMIOWB
mmiowb();
#endif
- atomic_inc(&queue->tail);
+ atomic_set(&queue->tail, prod + 1);
+ atomic_set(&queue->prod, prod + 1);
- /* 7. Complete submission and restore local interrupts */
- local_irq_restore(flags);
+ if (out_prod)
+ *out_prod = prod;
- return prod;
+ raw_spin_unlock_irqrestore(&queue->lock, flags);
+ return 0;
err_busy:
- local_irq_restore(flags);
+ /* Report the failure and continue; full RAS recovery is not implemented. */
dev_err_once(queue->iommu->dev, "Hardware error: command enqueue failed\n");
-
- return prod;
+ return ret;
}
/*
@@ -492,7 +511,7 @@ static irqreturn_t riscv_iommu_cmdq_process(int irq, void *data)
static void riscv_iommu_cmd_send(struct riscv_iommu_device *iommu,
struct riscv_iommu_command *cmd)
{
- riscv_iommu_queue_send(&iommu->cmdq, cmd, sizeof(*cmd));
+ riscv_iommu_queue_send(&iommu->cmdq, cmd, sizeof(*cmd), NULL);
}
/* Send IOFENCE.C command and wait for all scheduled commands to complete. */
@@ -501,9 +520,12 @@ static void riscv_iommu_cmd_sync(struct riscv_iommu_device *iommu,
{
struct riscv_iommu_command cmd;
unsigned int prod;
+ int ret;
riscv_iommu_cmd_iofence(&cmd);
- prod = riscv_iommu_queue_send(&iommu->cmdq, &cmd, sizeof(cmd));
+ ret = riscv_iommu_queue_send(&iommu->cmdq, &cmd, sizeof(cmd), &prod);
+ if (ret)
+ return;
if (!timeout_us)
return;
@@ -1560,6 +1582,7 @@ int riscv_iommu_init(struct riscv_iommu_device *iommu)
int rc;
RISCV_IOMMU_QUEUE_INIT(&iommu->cmdq, CQ);
+ raw_spin_lock_init(&iommu->cmdq.lock);
RISCV_IOMMU_QUEUE_INIT(&iommu->fltq, FQ);
rc = riscv_iommu_init_check(iommu);
diff --git a/drivers/iommu/riscv/iommu.h b/drivers/iommu/riscv/iommu.h
index 46df79dd5495..5676001548cc 100644
--- a/drivers/iommu/riscv/iommu.h
+++ b/drivers/iommu/riscv/iommu.h
@@ -12,6 +12,7 @@
#define _RISCV_IOMMU_H_
#include <linux/iommu.h>
+#include <linux/spinlock.h>
#include <linux/types.h>
#include <linux/iopoll.h>
@@ -23,6 +24,7 @@ struct riscv_iommu_queue {
atomic_t prod; /* unbounded producer allocation index */
atomic_t head; /* unbounded shadow ring buffer consumer index */
atomic_t tail; /* unbounded shadow ring buffer producer index */
+ raw_spinlock_t lock; /* serialize queue publishing */
unsigned int mask; /* index mask, queue length - 1 */
unsigned int irq; /* allocated interrupt number */
struct riscv_iommu_device *iommu; /* iommu device handling the queue when active */
diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c
index f148f559ac56..58ca7727b7f2 100644
--- a/drivers/iommu/s390-iommu.c
+++ b/drivers/iommu/s390-iommu.c
@@ -974,6 +974,8 @@ static unsigned long *get_rto_from_iova(struct s390_domain *domain,
case ZPCI_TABLE_TYPE_RFX:
case ZPCI_TABLE_TYPE_RSX:
rso = get_rso_from_iova(domain, iova);
+ if (!rso)
+ return NULL;
rsx = calc_rsx(iova);
rse = READ_ONCE(rso[rsx]);
if (!reg_entry_isvalid(rse))
diff --git a/drivers/irqchip/irq-gic-v5.c b/drivers/irqchip/irq-gic-v5.c
index d6f1c0c8b747..21c34e758732 100644
--- a/drivers/irqchip/irq-gic-v5.c
+++ b/drivers/irqchip/irq-gic-v5.c
@@ -974,7 +974,8 @@ static void gicv5_cpu_disable_interrupts(void)
{
u64 cr0;
- cr0 = FIELD_PREP(ICC_CR0_EL1_EN, 0);
+ cr0 = read_sysreg_s(SYS_ICC_CR0_EL1);
+ cr0 &= ~ICC_CR0_EL1_EN_MASK;
write_sysreg_s(cr0, SYS_ICC_CR0_EL1);
isb();
}
@@ -991,7 +992,8 @@ static void gicv5_cpu_enable_interrupts(void)
pcr = FIELD_PREP(ICC_PCR_EL1_PRIORITY, GICV5_IRQ_PRI_MI);
write_sysreg_s(pcr, SYS_ICC_PCR_EL1);
- cr0 = FIELD_PREP(ICC_CR0_EL1_EN, 1);
+ cr0 = read_sysreg_s(SYS_ICC_CR0_EL1);
+ cr0 |= ICC_CR0_EL1_EN_MASK;
write_sysreg_s(cr0, SYS_ICC_CR0_EL1);
}
diff --git a/drivers/leds/leds-st1202.c b/drivers/leds/leds-st1202.c
index 2593ff39f22a..168df5ecf27b 100644
--- a/drivers/leds/leds-st1202.c
+++ b/drivers/leds/leds-st1202.c
@@ -237,6 +237,12 @@ static int st1202_led_pattern_set(struct led_classdev *ldev,
if (len > ST1202_MAX_PATTERNS)
return -EINVAL;
+ for (int patt = 0; patt < len; patt++) {
+ if (pattern[patt].delta_t < ST1202_MILLIS_PATTERN_DUR_MIN ||
+ pattern[patt].delta_t > ST1202_MILLIS_PATTERN_DUR_MAX)
+ return -EINVAL;
+ }
+
guard(mutex)(&chip->lock);
ret = st1202_write_reg(chip, ST1202_CONFIG_REG, ST1202_CONFIG_REG_SHFT);
@@ -244,10 +250,6 @@ static int st1202_led_pattern_set(struct led_classdev *ldev,
return ret;
for (int patt = 0; patt < len; patt++) {
- if (pattern[patt].delta_t < ST1202_MILLIS_PATTERN_DUR_MIN ||
- pattern[patt].delta_t > ST1202_MILLIS_PATTERN_DUR_MAX)
- return -EINVAL;
-
ret = st1202_pwm_pattern_write(chip, led->led_num, patt, pattern[patt].brightness);
if (ret != 0)
return ret;
diff --git a/drivers/media/i2c/mt9p031.c b/drivers/media/i2c/mt9p031.c
index d21510caf45a..2b09e8315c8e 100644
--- a/drivers/media/i2c/mt9p031.c
+++ b/drivers/media/i2c/mt9p031.c
@@ -452,11 +452,7 @@ static int mt9p031_set_params(struct mt9p031 *mt9p031)
ret = mt9p031_write(client, MT9P031_HORIZONTAL_BLANK, hblank - 1);
if (ret < 0)
return ret;
- ret = mt9p031_write(client, MT9P031_VERTICAL_BLANK, vblank - 1);
- if (ret < 0)
- return ret;
-
- return ret;
+ return mt9p031_write(client, MT9P031_VERTICAL_BLANK, vblank - 1);
}
static int mt9p031_s_stream(struct v4l2_subdev *subdev, int enable)
diff --git a/drivers/media/pci/intel/ipu-bridge.c b/drivers/media/pci/intel/ipu-bridge.c
index ad3ddfb9ccb1..510c0eca4484 100644
--- a/drivers/media/pci/intel/ipu-bridge.c
+++ b/drivers/media/pci/intel/ipu-bridge.c
@@ -173,6 +173,19 @@ static const struct acpi_device_id ivsc_acpi_ids[] = {
{ "INTC10E1" }, /* PTL */
};
+/*
+ * The subset of ivsc_acpi_ids[] which are IVSC, rather than CVS, devices. The
+ * CVS IDs are deliberately not listed here: new ones keep being added, whereas
+ * this list is complete.
+ */
+static const struct acpi_device_id ivsc_only_acpi_ids[] = {
+ { "INTC1059" },
+ { "INTC1095" },
+ { "INTC100A" },
+ { "INTC10CF" },
+ { }
+};
+
static struct acpi_device *ipu_bridge_get_ivsc_acpi_dev(struct acpi_device *adev)
{
unsigned int i;
@@ -224,6 +237,17 @@ static struct device *ipu_bridge_get_ivsc_csi_dev(struct acpi_device *adev)
return csi_dev;
}
+ /*
+ * The lookups below match on the ACPI companion alone. That is fine for
+ * CVS, which binds a driver to that very device, but not for IVSC: there
+ * the ACPI device also has a driverless platform device, which would be
+ * returned instead of the mei-csi client. Return NULL for IVSC so that
+ * the caller fails and the probe is retried once the IVSC device shows
+ * up.
+ */
+ if (!acpi_match_device_ids(adev, ivsc_only_acpi_ids))
+ return NULL;
+
/* Try to locate CVS device on the I2C bus */
csi_dev = bus_find_device_by_acpi_dev(&i2c_bus_type, adev);
if (csi_dev)
diff --git a/drivers/media/platform/mediatek/vcodec/decoder/vdec/vdec_av1_req_lat_if.c b/drivers/media/platform/mediatek/vcodec/decoder/vdec/vdec_av1_req_lat_if.c
index 2d622e85f827..49d9b4a72387 100644
--- a/drivers/media/platform/mediatek/vcodec/decoder/vdec/vdec_av1_req_lat_if.c
+++ b/drivers/media/platform/mediatek/vcodec/decoder/vdec/vdec_av1_req_lat_if.c
@@ -1299,11 +1299,12 @@ static void vdec_av1_slice_setup_tile(struct vdec_av1_slice_frame *frame,
tile->uniform_tile_spacing_flag =
BIT_FLAG(ctrl_tile, V4L2_AV1_TILE_INFO_FLAG_UNIFORM_TILE_SPACING);
- for (i = 0; i < tile->tile_cols + 1; i++)
+ /* Bound the copy to the mi_col_starts[]/mi_row_starts[] capacity. */
+ for (i = 0; i < tile->tile_cols + 1 && i < V4L2_AV1_MAX_TILE_COLS + 1; i++)
tile->mi_col_starts[i] =
ALIGN(ctrl_tile->mi_col_starts[i], BIT(mib_size_log2)) >> mib_size_log2;
- for (i = 0; i < tile->tile_rows + 1; i++)
+ for (i = 0; i < tile->tile_rows + 1 && i < V4L2_AV1_MAX_TILE_ROWS + 1; i++)
tile->mi_row_starts[i] =
ALIGN(ctrl_tile->mi_row_starts[i], BIT(mib_size_log2)) >> mib_size_log2;
}
diff --git a/drivers/media/platform/microchip/microchip-sama7g5-isc.c b/drivers/media/platform/microchip/microchip-sama7g5-isc.c
index b0302dfc3278..7383341ec51d 100644
--- a/drivers/media/platform/microchip/microchip-sama7g5-isc.c
+++ b/drivers/media/platform/microchip/microchip-sama7g5-isc.c
@@ -598,13 +598,8 @@ static int __maybe_unused xisc_runtime_suspend(struct device *dev)
static int __maybe_unused xisc_runtime_resume(struct device *dev)
{
struct isc_device *isc = dev_get_drvdata(dev);
- int ret;
-
- ret = clk_prepare_enable(isc->hclock);
- if (ret)
- return ret;
- return ret;
+ return clk_prepare_enable(isc->hclock);
}
static const struct dev_pm_ops microchip_xisc_dev_pm_ops = {
diff --git a/drivers/media/platform/qcom/iris/iris_resources.c b/drivers/media/platform/qcom/iris/iris_resources.c
index 773f6548370a..6d3339423eec 100644
--- a/drivers/media/platform/qcom/iris/iris_resources.c
+++ b/drivers/media/platform/qcom/iris/iris_resources.c
@@ -78,24 +78,21 @@ int iris_enable_power_domains(struct iris_core *core, struct device *pd_dev)
if (ret)
return ret;
- ret = pm_runtime_get_sync(pd_dev);
- if (ret < 0)
- return ret;
-
- return ret;
+ return pm_runtime_resume_and_get(pd_dev);
}
int iris_disable_power_domains(struct iris_core *core, struct device *pd_dev)
{
int ret;
+ int pm_ret;
ret = iris_opp_set_rate(core->dev, 0);
- if (ret)
- return ret;
- pm_runtime_put_sync(pd_dev);
+ pm_ret = pm_runtime_put_sync(pd_dev);
+ if (!ret)
+ ret = pm_ret;
- return 0;
+ return ret;
}
static struct clk *iris_get_clk_by_type(struct iris_core *core, enum platform_clk_type clk_type)
diff --git a/drivers/media/platform/qcom/venus/pm_helpers.c b/drivers/media/platform/qcom/venus/pm_helpers.c
index be1cbd5cfe84..e88e66be4f6d 100644
--- a/drivers/media/platform/qcom/venus/pm_helpers.c
+++ b/drivers/media/platform/qcom/venus/pm_helpers.c
@@ -781,7 +781,6 @@ static int decide_core(struct venus_inst *inst)
unsigned long max_freq = ULONG_MAX;
struct device *dev = core->dev;
struct dev_pm_opp *opp;
- int ret = 0;
if (legacy_binding) {
if (inst->session_type == VIDC_SESSION_TYPE_DEC)
@@ -829,11 +828,7 @@ static int decide_core(struct venus_inst *inst)
}
done:
- ret = hfi_session_set_property(inst, ptype, &cu);
- if (ret)
- return ret;
-
- return ret;
+ return hfi_session_set_property(inst, ptype, &cu);
}
static int acquire_core(struct venus_inst *inst)
diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
index 7305cc4a04cb..6635f5782175 100644
--- a/drivers/media/platform/renesas/rcar-csi2.c
+++ b/drivers/media/platform/renesas/rcar-csi2.c
@@ -2273,11 +2273,7 @@ static int rcsi2_init_phtw_v3u(struct rcar_csi2 *priv,
return ret;
}
- ret = rcsi2_phtw_write_array(priv, step4, ARRAY_SIZE(step4));
- if (ret)
- return ret;
-
- return ret;
+ return rcsi2_phtw_write_array(priv, step4, ARRAY_SIZE(step4));
}
/* -----------------------------------------------------------------------------
diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
index 9c4a6093af32..2b8e04dd1572 100644
--- a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
+++ b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
@@ -16,6 +16,7 @@
*/
#include <linux/v4l2-common.h>
+#include <media/v4l2-hevc.h>
#include <media/v4l2-mem2mem.h>
#include "rkvdec.h"
@@ -37,15 +38,17 @@ void compute_tiles_uniform(struct rkvdec_hevc_run *run, u16 log2_min_cb_size,
s32 pic_in_cts_height, u16 *column_width, u16 *row_height)
{
const struct v4l2_ctrl_hevc_pps *pps = run->pps;
+ unsigned int num_cols = v4l2_hevc_pps_num_tile_columns(pps);
+ unsigned int num_rows = v4l2_hevc_pps_num_tile_rows(pps);
int i;
- for (i = 0; i < pps->num_tile_columns_minus1 + 1; i++)
+ for (i = 0; i < num_cols; i++)
column_width[i] = ((i + 1) * pic_in_cts_width) /
(pps->num_tile_columns_minus1 + 1) -
(i * pic_in_cts_width) /
(pps->num_tile_columns_minus1 + 1);
- for (i = 0; i < pps->num_tile_rows_minus1 + 1; i++)
+ for (i = 0; i < num_rows; i++)
row_height[i] = ((i + 1) * pic_in_cts_height) /
(pps->num_tile_rows_minus1 + 1) -
(i * pic_in_cts_height) /
@@ -57,17 +60,20 @@ void compute_tiles_non_uniform(struct rkvdec_hevc_run *run, u16 log2_min_cb_size
s32 pic_in_cts_height, u16 *column_width, u16 *row_height)
{
const struct v4l2_ctrl_hevc_pps *pps = run->pps;
+ unsigned int num_cols = v4l2_hevc_pps_num_tile_columns(pps);
+ unsigned int num_rows = v4l2_hevc_pps_num_tile_rows(pps);
s32 sum = 0;
int i;
- for (i = 0; i < pps->num_tile_columns_minus1; i++) {
+ /* The last tile entry is written after the loop, so iterate one less. */
+ for (i = 0; i < num_cols - 1; i++) {
column_width[i] = pps->column_width_minus1[i] + 1;
sum += column_width[i];
}
column_width[i] = pic_in_cts_width - sum;
sum = 0;
- for (i = 0; i < pps->num_tile_rows_minus1; i++) {
+ for (i = 0; i < num_rows - 1; i++) {
row_height[i] = pps->row_height_minus1[i] + 1;
sum += row_height[i];
}
diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc.c b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc.c
index 87abf93dfd5e..92373db50a46 100644
--- a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc.c
+++ b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc.c
@@ -12,6 +12,7 @@
* Jeffy Chen <jeffy.chen@rock-chips.com>
*/
+#include <media/v4l2-hevc.h>
#include <media/v4l2-mem2mem.h>
#include "rkvdec.h"
@@ -135,6 +136,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
* packet unit). so the driver copy SPS/PPS information to the exact PPS
* packet unit for HW accessing.
*/
+ if (pps->pic_parameter_set_id >= ARRAY_SIZE(priv_tbl->param_set))
+ return;
hw_ps = &priv_tbl->param_set[pps->pic_parameter_set_id];
memset(hw_ps, 0, sizeof(*hw_ps));
@@ -253,9 +256,9 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
if (pps->flags & V4L2_HEVC_PPS_FLAG_TILES_ENABLED) {
/* Userspace also provide column width and row height for uniform spacing */
- for (i = 0; i <= pps->num_tile_columns_minus1; i++)
+ for (i = 0; i < v4l2_hevc_pps_num_tile_columns(pps); i++)
WRITE_PPS(pps->column_width_minus1[i], COLUMN_WIDTH(i));
- for (i = 0; i <= pps->num_tile_rows_minus1; i++)
+ for (i = 0; i < v4l2_hevc_pps_num_tile_rows(pps); i++)
WRITE_PPS(pps->row_height_minus1[i], ROW_HEIGHT(i));
} else {
WRITE_PPS(((sps->pic_width_in_luma_samples + ctb_size_y - 1) / ctb_size_y) - 1,
diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c b/drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c
index fe6414a17551..6dafa1dd2850 100644
--- a/drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c
+++ b/drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c
@@ -145,6 +145,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
* packet unit). so the driver copy SPS/PPS information to the exact PPS
* packet unit for HW accessing.
*/
+ if (pps->pic_parameter_set_id >= ARRAY_SIZE(priv_tbl->param_set))
+ return;
hw_ps = &priv_tbl->param_set[pps->pic_parameter_set_id];
memset(hw_ps, 0, sizeof(*hw_ps));
diff --git a/drivers/media/platform/samsung/s3c-camif/camif-core.c b/drivers/media/platform/samsung/s3c-camif/camif-core.c
index 221e3c447f36..a3c8b0a10eb3 100644
--- a/drivers/media/platform/samsung/s3c-camif/camif-core.c
+++ b/drivers/media/platform/samsung/s3c-camif/camif-core.c
@@ -302,7 +302,6 @@ static int camif_media_dev_init(struct camif_dev *camif)
struct media_device *md = &camif->media_dev;
struct v4l2_device *v4l2_dev = &camif->v4l2_dev;
unsigned int ip_rev = camif->variant->ip_revision;
- int ret;
memset(md, 0, sizeof(*md));
snprintf(md->model, sizeof(md->model), "Samsung S3C%s CAMIF",
@@ -317,11 +316,7 @@ static int camif_media_dev_init(struct camif_dev *camif)
media_device_init(md);
- ret = v4l2_device_register(camif->dev, v4l2_dev);
- if (ret < 0)
- return ret;
-
- return ret;
+ return v4l2_device_register(camif->dev, v4l2_dev);
}
static void camif_clk_put(struct camif_dev *camif)
diff --git a/drivers/media/platform/verisilicon/hantro_g2_hevc_dec.c b/drivers/media/platform/verisilicon/hantro_g2_hevc_dec.c
index e8c2e83379de..e7a7c7a42467 100644
--- a/drivers/media/platform/verisilicon/hantro_g2_hevc_dec.c
+++ b/drivers/media/platform/verisilicon/hantro_g2_hevc_dec.c
@@ -5,6 +5,8 @@
* Copyright (C) 2020 Safran Passenger Innovations LLC
*/
+#include <media/v4l2-hevc.h>
+
#include "hantro_hw.h"
#include "hantro_g2_regs.h"
@@ -15,8 +17,8 @@ static void prepare_tile_info_buffer(struct hantro_ctx *ctx)
const struct v4l2_ctrl_hevc_pps *pps = ctrls->pps;
const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
u16 *p = (u16 *)((u8 *)ctx->hevc_dec.tile_sizes.cpu);
- unsigned int num_tile_rows = pps->num_tile_rows_minus1 + 1;
- unsigned int num_tile_cols = pps->num_tile_columns_minus1 + 1;
+ unsigned int num_tile_rows = v4l2_hevc_pps_num_tile_rows(pps);
+ unsigned int num_tile_cols = v4l2_hevc_pps_num_tile_columns(pps);
unsigned int pic_width_in_ctbs, pic_height_in_ctbs;
unsigned int max_log2_ctb_size, ctb_size;
bool tiles_enabled, uniform_spacing;
diff --git a/drivers/media/platform/verisilicon/rockchip_vpu981_hw_av1_dec.c b/drivers/media/platform/verisilicon/rockchip_vpu981_hw_av1_dec.c
index e4e21ad37323..00aa566a4ccd 100644
--- a/drivers/media/platform/verisilicon/rockchip_vpu981_hw_av1_dec.c
+++ b/drivers/media/platform/verisilicon/rockchip_vpu981_hw_av1_dec.c
@@ -431,20 +431,39 @@ static int rockchip_vpu981_av1_dec_prepare_run(struct hantro_ctx *ctx)
{
struct hantro_av1_dec_hw_ctx *av1_dec = &ctx->av1_dec;
struct hantro_av1_dec_ctrls *ctrls = &av1_dec->ctrls;
+ const struct v4l2_av1_tile_info *tile_info;
+ struct v4l2_ctrl *tge;
+ u32 num_tiles;
ctrls->sequence = hantro_get_ctrl(ctx, V4L2_CID_STATELESS_AV1_SEQUENCE);
if (WARN_ON(!ctrls->sequence))
return -EINVAL;
- ctrls->tile_group_entry =
- hantro_get_ctrl(ctx, V4L2_CID_STATELESS_AV1_TILE_GROUP_ENTRY);
- if (WARN_ON(!ctrls->tile_group_entry))
+ tge = v4l2_ctrl_find(&ctx->ctrl_handler,
+ V4L2_CID_STATELESS_AV1_TILE_GROUP_ENTRY);
+ if (WARN_ON(!tge))
return -EINVAL;
+ ctrls->tile_group_entry = tge->p_cur.p;
ctrls->frame = hantro_get_ctrl(ctx, V4L2_CID_STATELESS_AV1_FRAME);
if (WARN_ON(!ctrls->frame))
return -EINVAL;
+ /*
+ * rockchip_vpu981_av1_dec_set_tile_info() indexes the tile group
+ * entry array by tile1 * tile_cols + tile0, so it reads up to
+ * tile_cols * tile_rows entries, and lays out one descriptor per tile
+ * in the AV1_MAX_TILES tile_info buffer while programming the real
+ * tile geometry into the hardware. Reject a frame that claims more
+ * tiles than userspace submitted, or more than the hardware tile
+ * buffer holds, so the read stays in bounds and the programmed
+ * geometry matches the descriptors written.
+ */
+ tile_info = &ctrls->frame->tile_info;
+ num_tiles = (u32)tile_info->tile_cols * tile_info->tile_rows;
+ if (num_tiles > tge->elems || num_tiles > AV1_MAX_TILES)
+ return -EINVAL;
+
ctrls->film_grain =
hantro_get_ctrl(ctx, V4L2_CID_STATELESS_AV1_FILM_GRAIN);
@@ -578,16 +597,30 @@ static void rockchip_vpu981_av1_dec_set_tile_info(struct hantro_ctx *ctx)
const struct v4l2_av1_tile_info *tile_info = &ctrls->frame->tile_info;
const struct v4l2_ctrl_av1_tile_group_entry *group_entry =
ctrls->tile_group_entry;
- int context_update_y =
- tile_info->context_update_tile_id / tile_info->tile_cols;
- int context_update_x =
- tile_info->context_update_tile_id % tile_info->tile_cols;
- int context_update_tile_id =
- context_update_x * tile_info->tile_rows + context_update_y;
+ int context_update_y = 0;
+ int context_update_x = 0;
+ int context_update_tile_id = 0;
u8 *dst = av1_dec->tile_info.cpu;
+ u8 *dst_end = dst + av1_dec->tile_info.size;
struct hantro_dev *vpu = ctx->dev;
int tile0, tile1;
+ /*
+ * tile_cols and tile_rows are bounded by the V4L2 control validation
+ * (V4L2_AV1_MAX_TILE_{COLS,ROWS} and V4L2_AV1_MAX_TILE_COUNT). Guard
+ * the divisor here, and keep the descriptor writes within the
+ * AV1_MAX_TILES tile_info buffer below; the register values use the
+ * unmodified tile geometry.
+ */
+ if (tile_info->tile_cols) {
+ context_update_y =
+ tile_info->context_update_tile_id / tile_info->tile_cols;
+ context_update_x =
+ tile_info->context_update_tile_id % tile_info->tile_cols;
+ context_update_tile_id =
+ context_update_x * tile_info->tile_rows + context_update_y;
+ }
+
memset(dst, 0, av1_dec->tile_info.size);
for (tile0 = 0; tile0 < tile_info->tile_cols; tile0++) {
@@ -598,6 +631,10 @@ static void rockchip_vpu981_av1_dec_set_tile_info(struct hantro_ctx *ctx)
tile_info->height_in_sbs_minus_1[tile1] + 1;
u32 x0 = tile_info->width_in_sbs_minus_1[tile0] + 1;
+ /* Stop once the tile_info descriptor buffer is full. */
+ if (dst + 16 > dst_end)
+ break;
+
/* tile size in SB units (width,height) */
*dst++ = x0;
*dst++ = 0;
@@ -622,6 +659,8 @@ static void rockchip_vpu981_av1_dec_set_tile_info(struct hantro_ctx *ctx)
*dst++ = (end >> 16) & 255;
*dst++ = (end >> 24) & 255;
}
+ if (dst + 16 > dst_end)
+ break;
}
hantro_reg_write(vpu, &av1_multicore_expect_context_update, !!(context_update_x == 0));
diff --git a/drivers/media/usb/as102/as102_usb_drv.c b/drivers/media/usb/as102/as102_usb_drv.c
index a11024451ceb..be2f8be560fa 100644
--- a/drivers/media/usb/as102/as102_usb_drv.c
+++ b/drivers/media/usb/as102/as102_usb_drv.c
@@ -24,35 +24,33 @@ static void as102_usb_stop_stream(struct as102_dev_t *dev);
static int as102_open(struct inode *inode, struct file *file);
static int as102_release(struct inode *inode, struct file *file);
-static const struct usb_device_id as102_usb_id_table[] = {
- { USB_DEVICE(AS102_USB_DEVICE_VENDOR_ID, AS102_USB_DEVICE_PID_0001) },
- { USB_DEVICE(PCTV_74E_USB_VID, PCTV_74E_USB_PID) },
- { USB_DEVICE(ELGATO_EYETV_DTT_USB_VID, ELGATO_EYETV_DTT_USB_PID) },
- { USB_DEVICE(NBOX_DVBT_DONGLE_USB_VID, NBOX_DVBT_DONGLE_USB_PID) },
- { USB_DEVICE(SKY_IT_DIGITAL_KEY_USB_VID, SKY_IT_DIGITAL_KEY_USB_PID) },
- { } /* Terminating entry */
+struct as102_dev_info {
+ const char *name;
+ /*
+ * eLNA configuration: devices built on the reference design work best
+ * with 0xA0, while custom designs seem to require 0xC0
+ */
+ uint8_t elna_cfg;
};
-/* Note that this table must always have the same number of entries as the
- as102_usb_id_table struct */
-static const char * const as102_device_names[] = {
- AS102_REFERENCE_DESIGN,
- AS102_PCTV_74E,
- AS102_ELGATO_EYETV_DTT_NAME,
- AS102_NBOX_DVBT_DONGLE_NAME,
- AS102_SKY_IT_DIGITAL_KEY_NAME,
- NULL /* Terminating entry */
-};
+#define DRIVER_INFO(dev_name, dev_elna_cfg) \
+ .driver_info = (kernel_ulong_t)&(const struct as102_dev_info){ \
+ .name = (dev_name), \
+ .elna_cfg = (dev_elna_cfg), \
+ }
-/* eLNA configuration: devices built on the reference design work best
- with 0xA0, while custom designs seem to require 0xC0 */
-static uint8_t const as102_elna_cfg[] = {
- 0xA0,
- 0xC0,
- 0xC0,
- 0xA0,
- 0xA0,
- 0x00 /* Terminating entry */
+static const struct usb_device_id as102_usb_id_table[] = {
+ { USB_DEVICE(AS102_USB_DEVICE_VENDOR_ID, AS102_USB_DEVICE_PID_0001),
+ DRIVER_INFO(AS102_REFERENCE_DESIGN, 0xA0) },
+ { USB_DEVICE(PCTV_74E_USB_VID, PCTV_74E_USB_PID),
+ DRIVER_INFO(AS102_PCTV_74E, 0xC0) },
+ { USB_DEVICE(ELGATO_EYETV_DTT_USB_VID, ELGATO_EYETV_DTT_USB_PID),
+ DRIVER_INFO(AS102_ELGATO_EYETV_DTT_NAME, 0xC0) },
+ { USB_DEVICE(NBOX_DVBT_DONGLE_USB_VID, NBOX_DVBT_DONGLE_USB_PID),
+ DRIVER_INFO(AS102_NBOX_DVBT_DONGLE_NAME, 0xA0) },
+ { USB_DEVICE(SKY_IT_DIGITAL_KEY_USB_VID, SKY_IT_DIGITAL_KEY_USB_PID),
+ DRIVER_INFO(AS102_SKY_IT_DIGITAL_KEY_NAME, 0xA0) },
+ { } /* Terminating entry */
};
struct usb_driver as102_usb_driver = {
@@ -336,29 +334,18 @@ static int as102_usb_probe(struct usb_interface *intf,
{
int ret;
struct as102_dev_t *as102_dev;
- int i;
-
- /* This should never actually happen */
- if (ARRAY_SIZE(as102_usb_id_table) !=
- (sizeof(as102_device_names) / sizeof(const char *))) {
- pr_err("Device names table invalid size");
- return -EINVAL;
- }
+ const struct as102_dev_info *info = (const struct as102_dev_info *)id->driver_info;
as102_dev = kzalloc_obj(struct as102_dev_t);
if (as102_dev == NULL)
return -ENOMEM;
- /* Assign the user-friendly device name */
- for (i = 0; i < ARRAY_SIZE(as102_usb_id_table); i++) {
- if (id == &as102_usb_id_table[i]) {
- as102_dev->name = as102_device_names[i];
- as102_dev->elna_cfg = as102_elna_cfg[i];
- }
- }
-
- if (as102_dev->name == NULL)
+ if (info) {
+ as102_dev->name = info->name;
+ as102_dev->elna_cfg = info->elna_cfg;
+ } else {
as102_dev->name = "Unknown AS102 device";
+ }
/* set private callback functions */
as102_dev->bus_adap.ops = &as102_priv_ops;
diff --git a/drivers/media/usb/dvb-usb-v2/mxl111sf.c b/drivers/media/usb/dvb-usb-v2/mxl111sf.c
index 870ac3c8b085..6404eb74db32 100644
--- a/drivers/media/usb/dvb-usb-v2/mxl111sf.c
+++ b/drivers/media/usb/dvb-usb-v2/mxl111sf.c
@@ -987,11 +987,7 @@ static int mxl111sf_frontend_attach_atsc_mh(struct dvb_usb_adapter *adap)
if (ret < 0)
return ret;
- ret = mxl111sf_lg2160_frontend_attach(adap, 2);
- if (ret < 0)
- return ret;
-
- return ret;
+ return mxl111sf_lg2160_frontend_attach(adap, 2);
}
static int mxl111sf_frontend_attach_mercury(struct dvb_usb_adapter *adap)
@@ -1007,11 +1003,7 @@ static int mxl111sf_frontend_attach_mercury(struct dvb_usb_adapter *adap)
if (ret < 0)
return ret;
- ret = mxl111sf_lg2161_ep6_frontend_attach(adap, 2);
- if (ret < 0)
- return ret;
-
- return ret;
+ return mxl111sf_lg2161_ep6_frontend_attach(adap, 2);
}
static int mxl111sf_frontend_attach_mercury_mh(struct dvb_usb_adapter *adap)
diff --git a/drivers/media/usb/gspca/jl2005bcd.c b/drivers/media/usb/gspca/jl2005bcd.c
index a408fcc3a060..4988fbf5005e 100644
--- a/drivers/media/usb/gspca/jl2005bcd.c
+++ b/drivers/media/usb/gspca/jl2005bcd.c
@@ -148,17 +148,12 @@ static int jl2005c_start_new_frame(struct gspca_dev *gspca_dev)
static int jl2005c_write_reg(struct gspca_dev *gspca_dev, unsigned char reg,
unsigned char value)
{
- int retval;
u8 instruction[2];
instruction[0] = reg;
instruction[1] = value;
- retval = jl2005c_write2(gspca_dev, instruction);
- if (retval < 0)
- return retval;
-
- return retval;
+ return jl2005c_write2(gspca_dev, instruction);
}
static int jl2005c_get_firmware_id(struct gspca_dev *gspca_dev)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index 5b8a594fb9e2..648b88c868bc 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -793,10 +793,30 @@ static int validate_av1_film_grain(struct v4l2_ctrl_av1_film_grain *fg)
return 0;
}
+static int validate_av1_tile_info(struct v4l2_av1_tile_info *t)
+{
+ /*
+ * tile_cols and tile_rows index the per-tile descriptor arrays and
+ * bound the tile loops in the stateless AV1 drivers; the product
+ * bounds the total tile descriptor count.
+ */
+ if (t->tile_cols > V4L2_AV1_MAX_TILE_COLS ||
+ t->tile_rows > V4L2_AV1_MAX_TILE_ROWS)
+ return -EINVAL;
+
+ if ((u32)t->tile_cols * t->tile_rows > V4L2_AV1_MAX_TILE_COUNT)
+ return -EINVAL;
+
+ return 0;
+}
+
static int validate_av1_frame(struct v4l2_ctrl_av1_frame *f)
{
int ret = 0;
+ ret = validate_av1_tile_info(&f->tile_info);
+ if (ret)
+ return ret;
ret = validate_av1_quantization(&f->quantization);
if (ret)
return ret;
@@ -1253,6 +1273,18 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
p_hevc_pps->flags &=
~V4L2_HEVC_PPS_FLAG_LOOP_FILTER_ACROSS_TILES_ENABLED;
+ } else {
+ /*
+ * These count the entries the stateless HEVC drivers
+ * read from column_width_minus1[] / row_height_minus1[]
+ * and use as tile-loop bounds.
+ */
+ if (p_hevc_pps->num_tile_columns_minus1 >=
+ ARRAY_SIZE(p_hevc_pps->column_width_minus1))
+ return -EINVAL;
+ if (p_hevc_pps->num_tile_rows_minus1 >=
+ ARRAY_SIZE(p_hevc_pps->row_height_minus1))
+ return -EINVAL;
}
if (p_hevc_pps->flags &
diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
index c00197d095e7..2323f559c6a3 100644
--- a/drivers/media/v4l2-core/v4l2-h264.c
+++ b/drivers/media/v4l2-core/v4l2-h264.c
@@ -440,7 +440,8 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
}
if (builder->num_valid > 1 &&
- !memcmp(b1_reflist, b0_reflist, builder->num_valid))
+ !memcmp(b1_reflist, b0_reflist,
+ sizeof(b1_reflist[0]) * builder->num_valid))
swap(b1_reflist[0], b1_reflist[1]);
print_ref_list_b(builder, b0_reflist, 0);
diff --git a/drivers/mfd/ab8500-core.c b/drivers/mfd/ab8500-core.c
index f0bc0b5a6f4a..86fa99022cb3 100644
--- a/drivers/mfd/ab8500-core.c
+++ b/drivers/mfd/ab8500-core.c
@@ -19,7 +19,7 @@
#include <linux/mfd/core.h>
#include <linux/mfd/abx500.h>
#include <linux/mfd/abx500/ab8500.h>
-#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/mfd/db8500-prcmu.h>
#include <linux/of.h>
/*
diff --git a/drivers/mfd/db8500-prcmu.c b/drivers/mfd/db8500-prcmu.c
index 21e68a382b11..6672c55f2ebc 100644
--- a/drivers/mfd/db8500-prcmu.c
+++ b/drivers/mfd/db8500-prcmu.c
@@ -32,7 +32,7 @@
#include <linux/platform_device.h>
#include <linux/uaccess.h>
#include <linux/mfd/core.h>
-#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/mfd/db8500-prcmu.h>
#include <linux/mfd/abx500/ab8500.h>
#include <linux/regulator/db8500-prcmu.h>
#include <linux/regulator/machine.h>
@@ -2285,7 +2285,7 @@ void db8500_prcmu_system_reset(u16 reset_code)
/**
* db8500_prcmu_get_reset_code - Retrieve SW reset reason code
*
- * Retrieves the reset reason code stored by prcmu_system_reset() before
+ * Retrieves the reset reason code stored by db8500_prcmu_system_reset() before
* last restart.
*/
u16 db8500_prcmu_get_reset_code(void)
@@ -3041,7 +3041,7 @@ static int db8500_prcmu_probe(struct platform_device *pdev)
db8500_irq_init(np);
- prcmu_config_esram0_deep_sleep(ESRAM0_DEEP_SLEEP_STATE_RET);
+ db8500_prcmu_config_esram0_deep_sleep(ESRAM0_DEEP_SLEEP_STATE_RET);
err = mfd_add_devices(&pdev->dev, 0, common_prcmu_devs,
ARRAY_SIZE(common_prcmu_devs), NULL, 0, db8500_irq_domain);
diff --git a/drivers/misc/amd-sbi/rmi-core.c b/drivers/misc/amd-sbi/rmi-core.c
index d4238ebad3c6..95c9109101b5 100644
--- a/drivers/misc/amd-sbi/rmi-core.c
+++ b/drivers/misc/amd-sbi/rmi-core.c
@@ -581,6 +581,8 @@ int create_misc_rmi_device(struct sbrmi_data *data,
GFP_KERNEL,
"sbrmi-%x",
data->dev_static_addr);
+ if (!data->sbrmi_misc_dev.name)
+ return -ENOMEM;
data->sbrmi_misc_dev.minor = MISC_DYNAMIC_MINOR;
data->sbrmi_misc_dev.fops = &sbrmi_fops;
data->sbrmi_misc_dev.parent = dev;
@@ -588,6 +590,8 @@ int create_misc_rmi_device(struct sbrmi_data *data,
GFP_KERNEL,
"sbrmi-%x",
data->dev_static_addr);
+ if (!data->sbrmi_misc_dev.nodename)
+ return -ENOMEM;
data->sbrmi_misc_dev.mode = 0600;
return misc_register(&data->sbrmi_misc_dev);
diff --git a/drivers/net/bonding/bond_alb.c b/drivers/net/bonding/bond_alb.c
index 839f7482dc18..43ac8e28e418 100644
--- a/drivers/net/bonding/bond_alb.c
+++ b/drivers/net/bonding/bond_alb.c
@@ -678,9 +678,15 @@ static struct slave *rlb_arp_xmit(struct sk_buff *skb, struct bonding *bond)
if (arp->op_code == htons(ARPOP_REPLY)) {
/* the arp must be sent on the selected rx channel */
tx_slave = rlb_choose_channel(skb, bond, arp);
- if (tx_slave)
+ if (tx_slave &&
+ !ether_addr_equal_64bits(arp->mac_src,
+ tx_slave->dev->dev_addr)) {
+ if (unlikely(skb_cow_head(skb, 0)))
+ return NULL;
+ arp = (struct arp_pkt *)skb_network_header(skb);
bond_hw_addr_copy(arp->mac_src, tx_slave->dev->dev_addr,
tx_slave->dev->addr_len);
+ }
netdev_dbg(bond->dev, "(slave %s): Server sent ARP Reply packet\n",
tx_slave ? tx_slave->dev->name : "NULL");
} else if (arp->op_code == htons(ARPOP_REQUEST)) {
@@ -875,7 +881,7 @@ static int rlb_initialize(struct bonding *bond)
spin_unlock_bh(&bond->mode_lock);
/* register to receive ARPs */
- bond->recv_probe = rlb_arp_recv;
+ WRITE_ONCE(bond->recv_probe, rlb_arp_recv);
return 0;
}
@@ -1281,10 +1287,10 @@ static int alb_set_mac_address(struct bonding *bond, void *addr)
}
/* determine if the packet is NA or NS */
-static bool alb_determine_nd(struct sk_buff *skb, struct bonding *bond)
+static bool alb_determine_nd(struct sk_buff *skb)
{
- struct ipv6hdr *ip6hdr;
- struct icmp6hdr *hdr;
+ const struct ipv6hdr *ip6hdr;
+ const struct icmp6hdr *hdr;
if (!pskb_network_may_pull(skb, sizeof(*ip6hdr)))
return true;
@@ -1296,7 +1302,8 @@ static bool alb_determine_nd(struct sk_buff *skb, struct bonding *bond)
if (!pskb_network_may_pull(skb, sizeof(*ip6hdr) + sizeof(*hdr)))
return true;
- hdr = icmp6_hdr(skb);
+ ip6hdr = ipv6_hdr(skb);
+ hdr = (const struct icmp6hdr *)(ip6hdr + 1);
return hdr->icmp6_type == NDISC_NEIGHBOUR_ADVERTISEMENT ||
hdr->icmp6_type == NDISC_NEIGHBOUR_SOLICITATION;
}
@@ -1339,7 +1346,6 @@ static netdev_tx_t bond_do_alb_xmit(struct sk_buff *skb, struct bonding *bond,
struct slave *tx_slave)
{
struct alb_bond_info *bond_info = &(BOND_ALB_INFO(bond));
- struct ethhdr *eth_data = eth_hdr(skb);
if (!tx_slave) {
/* unbalanced or unassigned, send through primary */
@@ -1350,7 +1356,9 @@ static netdev_tx_t bond_do_alb_xmit(struct sk_buff *skb, struct bonding *bond,
if (tx_slave && bond_slave_can_tx(tx_slave)) {
if (tx_slave != rcu_access_pointer(bond->curr_active_slave)) {
- ether_addr_copy(eth_data->h_source,
+ if (unlikely(skb_cow_head(skb, 0)))
+ return bond_tx_drop(bond->dev, skb);
+ ether_addr_copy(skb_eth_hdr(skb)->h_source,
tx_slave->dev->dev_addr);
}
@@ -1374,14 +1382,13 @@ struct slave *bond_xmit_tlb_slave_get(struct bonding *bond,
struct ethhdr *eth_data;
u32 hash_index;
- skb_reset_mac_header(skb);
- eth_data = eth_hdr(skb);
+ eth_data = skb_eth_hdr(skb);
/* Do not TX balance any multicast or broadcast */
if (!is_multicast_ether_addr(eth_data->h_dest)) {
switch (skb->protocol) {
case htons(ETH_P_IPV6):
- if (alb_determine_nd(skb, bond))
+ if (alb_determine_nd(skb))
break;
fallthrough;
case htons(ETH_P_IP):
@@ -1427,8 +1434,7 @@ struct slave *bond_xmit_alb_slave_get(struct bonding *bond,
u32 hash_index = 0;
int hash_size = 0;
- skb_reset_mac_header(skb);
- eth_data = eth_hdr(skb);
+ eth_data = skb_eth_hdr(skb);
switch (ntohs(skb->protocol)) {
case ETH_P_IP: {
@@ -1467,7 +1473,7 @@ struct slave *bond_xmit_alb_slave_get(struct bonding *bond,
break;
}
- if (alb_determine_nd(skb, bond)) {
+ if (alb_determine_nd(skb)) {
do_tx_balance = false;
break;
}
diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
index 522eab060f9e..6ea46a617ee6 100644
--- a/drivers/net/bonding/bond_main.c
+++ b/drivers/net/bonding/bond_main.c
@@ -1245,7 +1245,7 @@ static void bond_peer_notify_may_events(struct bonding *bond, bool force)
}
if (notified || force)
- bond->send_peer_notif--;
+ WRITE_ONCE(bond->send_peer_notif, bond->send_peer_notif - 1);
}
/**
@@ -2284,7 +2284,7 @@ int bond_enslave(struct net_device *bond_dev, struct net_device *slave_dev,
}
}
- bond->slave_cnt++;
+ WRITE_ONCE(bond->slave_cnt, bond->slave_cnt + 1);
netdev_compute_master_upper_features(bond->dev, true);
bond_set_carrier(bond);
@@ -2517,9 +2517,7 @@ static int __bond_release_one(struct net_device *bond_dev,
bond_alb_deinit_slave(bond, slave);
}
- if (all) {
- RCU_INIT_POINTER(bond->curr_active_slave, NULL);
- } else if (oldcurrent == slave) {
+ if (!all && oldcurrent == slave) {
/* Note that we hold RTNL over this sequence, so there
* is no concern that another slave add/remove event
* will interfere.
@@ -2533,7 +2531,7 @@ static int __bond_release_one(struct net_device *bond_dev,
unblock_netpoll_tx();
synchronize_rcu();
- bond->slave_cnt--;
+ WRITE_ONCE(bond->slave_cnt, bond->slave_cnt - 1);
if (!bond_has_slaves(bond)) {
call_netdevice_notifiers(NETDEV_CHANGEADDR, bond->dev);
@@ -4385,13 +4383,13 @@ static int bond_open(struct net_device *bond_dev)
if (bond->params.arp_interval) { /* arp interval, in milliseconds. */
queue_delayed_work(bond->wq, &bond->arp_work, 0);
- bond->recv_probe = bond_rcv_validate;
+ WRITE_ONCE(bond->recv_probe, bond_rcv_validate);
}
if (BOND_MODE(bond) == BOND_MODE_8023AD) {
queue_delayed_work(bond->wq, &bond->ad_work, 0);
/* register to receive LACPDUs */
- bond->recv_probe = bond_3ad_lacpdu_recv;
+ WRITE_ONCE(bond->recv_probe, bond_3ad_lacpdu_recv);
bond_3ad_initiate_agg_selection(bond, 1);
bond_for_each_slave(bond, slave, iter)
@@ -4413,7 +4411,7 @@ static int bond_close(struct net_device *bond_dev)
struct slave *slave;
bond_work_cancel_all(bond);
- bond->send_peer_notif = 0;
+ WRITE_ONCE(bond->send_peer_notif, 0);
WRITE_ONCE(bond->recv_probe, NULL);
/* Wait for any in-flight RX handlers */
@@ -5124,7 +5122,7 @@ static void bond_skip_slave(struct bond_up_slave *slaves,
if (skipslave == slaves->arr[idx]) {
slaves->arr[idx] =
slaves->arr[slaves->count - 1];
- slaves->count--;
+ WRITE_ONCE(slaves->count, slaves->count - 1);
break;
}
}
diff --git a/drivers/net/bonding/bond_options.c b/drivers/net/bonding/bond_options.c
index e590c8dee86e..d8cdbfca1bb1 100644
--- a/drivers/net/bonding/bond_options.c
+++ b/drivers/net/bonding/bond_options.c
@@ -1147,11 +1147,11 @@ static int bond_option_arp_interval_set(struct bonding *bond,
*/
if (!newval->value) {
if (bond->params.arp_validate)
- bond->recv_probe = NULL;
+ WRITE_ONCE(bond->recv_probe, NULL);
cancel_delayed_work_sync(&bond->arp_work);
} else {
/* arp_validate can be set only in active-backup mode */
- bond->recv_probe = bond_rcv_validate;
+ WRITE_ONCE(bond->recv_probe, bond_rcv_validate);
cancel_delayed_work_sync(&bond->mii_work);
queue_delayed_work(bond->wq, &bond->arp_work, 0);
}
diff --git a/drivers/net/dsa/bcm_sf2_cfp.c b/drivers/net/dsa/bcm_sf2_cfp.c
index 50d3a818eb1b..84a086c3e99b 100644
--- a/drivers/net/dsa/bcm_sf2_cfp.c
+++ b/drivers/net/dsa/bcm_sf2_cfp.c
@@ -1088,6 +1088,8 @@ static int bcm_sf2_cfp_rule_get_all(struct bcm_sf2_priv *priv,
unsigned int index = 1, rules_cnt = 0;
for_each_set_bit_from(index, priv->cfp.unique, priv->num_cfp_rules) {
+ if (rules_cnt == nfc->rule_cnt)
+ return -EMSGSIZE;
rule_locs[rules_cnt] = index;
rules_cnt++;
}
diff --git a/drivers/net/dsa/lantiq/lantiq_gswip.h b/drivers/net/dsa/lantiq/lantiq_gswip.h
index bc3686faad0d..0b75be14dc10 100644
--- a/drivers/net/dsa/lantiq/lantiq_gswip.h
+++ b/drivers/net/dsa/lantiq/lantiq_gswip.h
@@ -42,7 +42,7 @@
#define GSWIP_MDIO_PHY_FDUP_DIS 0x0600
#define GSWIP_MDIO_PHY_FCONTX_MASK 0x0180
#define GSWIP_MDIO_PHY_FCONTX_AUTO 0x0000
-#define GSWIP_MDIO_PHY_FCONTX_EN 0x0100
+#define GSWIP_MDIO_PHY_FCONTX_EN 0x0080
#define GSWIP_MDIO_PHY_FCONTX_DIS 0x0180
#define GSWIP_MDIO_PHY_FCONRX_MASK 0x0060
#define GSWIP_MDIO_PHY_FCONRX_AUTO 0x0000
diff --git a/drivers/net/dsa/mt7530-mmio.c b/drivers/net/dsa/mt7530-mmio.c
index 119fdd863d91..fd68b1cd0630 100644
--- a/drivers/net/dsa/mt7530-mmio.c
+++ b/drivers/net/dsa/mt7530-mmio.c
@@ -12,6 +12,7 @@
static const struct of_device_id mt7988_of_match[] = {
{ .compatible = "airoha,an7583-switch", .data = &mt753x_table[ID_AN7583], },
{ .compatible = "airoha,en7581-switch", .data = &mt753x_table[ID_EN7581], },
+ { .compatible = "econet,en7528-switch", .data = &mt753x_table[ID_EN7528], },
{ .compatible = "mediatek,mt7988-switch", .data = &mt753x_table[ID_MT7988], },
{ /* sentinel */ },
};
diff --git a/drivers/net/dsa/mt7530.c b/drivers/net/dsa/mt7530.c
index aa33d94e11b5..56ee8dc34f51 100644
--- a/drivers/net/dsa/mt7530.c
+++ b/drivers/net/dsa/mt7530.c
@@ -2902,6 +2902,30 @@ static void en7581_mac_port_get_caps(struct dsa_switch *ds, int port,
}
}
+static void en7528_mac_port_get_caps(struct dsa_switch *ds, int port,
+ struct phylink_config *config)
+{
+ switch (port) {
+ /* Ports which are connected to switch PHYs. There is no MII pinout. */
+ case 1 ... 4:
+ __set_bit(PHY_INTERFACE_MODE_INTERNAL,
+ config->supported_interfaces);
+
+ config->mac_capabilities |= MAC_10 | MAC_100 | MAC_1000FD;
+ break;
+
+ /* Port 6 is connected to SoC's GMAC at 1000 Mbps full duplex. There
+ * is no MII pinout.
+ */
+ case 6:
+ __set_bit(PHY_INTERFACE_MODE_INTERNAL,
+ config->supported_interfaces);
+
+ config->mac_capabilities |= MAC_1000FD;
+ break;
+ }
+}
+
static void
mt7530_mac_config(struct dsa_switch *ds, int port, unsigned int mode,
phy_interface_t interface)
@@ -3091,19 +3115,34 @@ static void mt753x_phylink_get_caps(struct dsa_switch *ds, int port,
struct phylink_config *config)
{
struct mt7530_priv *priv = ds->priv;
- u32 eeecr;
config->mac_capabilities = MAC_ASYM_PAUSE | MAC_SYM_PAUSE;
- config->lpi_capabilities = MAC_100FD | MAC_1000FD | MAC_2500FD;
+ priv->info->mac_port_get_caps(ds, port, config);
- eeecr = mt7530_read(priv, MT753X_PMEEECR_P(port));
- /* tx_lpi_timer should be in microseconds. The time units for
- * LPI threshold are unspecified.
+ /* The EN7528 GPHYs report EEE capability, but negotiating EEE with
+ * common link partners (e.g. Realtek GbE NICs) results in an unstable
+ * link with dropped frames. Leave the LPI capabilities empty so that
+ * phylink disables EEE on these PHYs and refuses to enable it from
+ * userspace.
*/
- config->lpi_timer_default = FIELD_GET(LPI_THRESH_MASK, eeecr);
-
- priv->info->mac_port_get_caps(ds, port, config);
+ if (priv->id != ID_EN7528 &&
+ config->mac_capabilities & (MAC_100FD | MAC_1000FD)) {
+ u32 eeecr = mt7530_read(priv, MT753X_PMEEECR_P(port));
+
+ /* LPI above 1 Gbps is not supported */
+ config->lpi_capabilities = config->mac_capabilities &
+ (MAC_100FD | MAC_1000FD);
+ phy_interface_copy(config->lpi_interfaces,
+ config->supported_interfaces);
+ __clear_bit(PHY_INTERFACE_MODE_2500BASEX,
+ config->lpi_interfaces);
+
+ /* tx_lpi_timer should be in microseconds. The time units for
+ * LPI threshold are unspecified.
+ */
+ config->lpi_timer_default = FIELD_GET(LPI_THRESH_MASK, eeecr);
+ }
}
static int mt753x_pcs_validate(struct phylink_pcs *pcs,
@@ -3244,7 +3283,8 @@ mt753x_conduit_state_change(struct dsa_switch *ds,
* forwarded to the numerically smallest CPU port whose conduit
* interface is up.
*/
- if (priv->id != ID_MT7530 && priv->id != ID_MT7621)
+ if (priv->id != ID_MT7530 && priv->id != ID_MT7621 &&
+ priv->id != ID_EN7528)
return;
mask = BIT(cpu_dp->index);
@@ -3449,6 +3489,16 @@ const struct mt753x_info mt753x_table[] = {
.phy_write_c45 = mt7531_ind_c45_phy_write,
.mac_port_get_caps = en7581_mac_port_get_caps,
},
+ [ID_EN7528] = {
+ .id = ID_EN7528,
+ .pcs_ops = &mt7530_pcs_ops,
+ .sw_setup = mt7988_setup,
+ .phy_read_c22 = mt7531_ind_c22_phy_read,
+ .phy_write_c22 = mt7531_ind_c22_phy_write,
+ .phy_read_c45 = mt7531_ind_c45_phy_read,
+ .phy_write_c45 = mt7531_ind_c45_phy_write,
+ .mac_port_get_caps = en7528_mac_port_get_caps,
+ },
};
EXPORT_SYMBOL_GPL(mt753x_table);
diff --git a/drivers/net/dsa/mt7530.h b/drivers/net/dsa/mt7530.h
index dd33b0df3419..5f1e841f42c0 100644
--- a/drivers/net/dsa/mt7530.h
+++ b/drivers/net/dsa/mt7530.h
@@ -21,6 +21,7 @@ enum mt753x_id {
ID_MT7988 = 3,
ID_EN7581 = 4,
ID_AN7583 = 5,
+ ID_EN7528 = 6,
};
#define NUM_TRGMII_CTRL 5
diff --git a/drivers/net/dsa/mv88e6xxx/chip.c b/drivers/net/dsa/mv88e6xxx/chip.c
index 80b877c74513..7f68a0c55802 100644
--- a/drivers/net/dsa/mv88e6xxx/chip.c
+++ b/drivers/net/dsa/mv88e6xxx/chip.c
@@ -2438,6 +2438,7 @@ static int mv88e6xxx_get_rxnfc(struct dsa_switch *ds, int port,
struct ethtool_rx_flow_spec *fs = &rxnfc->fs;
struct mv88e6xxx_chip *chip = ds->priv;
struct mv88e6xxx_policy *policy;
+ u32 cnt = 0;
int err;
int id;
@@ -2463,11 +2464,18 @@ static int mv88e6xxx_get_rxnfc(struct dsa_switch *ds, int port,
break;
case ETHTOOL_GRXCLSRLALL:
rxnfc->data = 0;
- rxnfc->rule_cnt = 0;
- idr_for_each_entry(&chip->policies, policy, id)
- if (policy->port == port)
- rule_locs[rxnfc->rule_cnt++] = id;
err = 0;
+ idr_for_each_entry(&chip->policies, policy, id) {
+ if (policy->port != port)
+ continue;
+ if (cnt == rxnfc->rule_cnt) {
+ err = -EMSGSIZE;
+ break;
+ }
+ rule_locs[cnt++] = id;
+ }
+ if (!err)
+ rxnfc->rule_cnt = cnt;
break;
default:
err = -EOPNOTSUPP;
diff --git a/drivers/net/ethernet/airoha/airoha_eth.h b/drivers/net/ethernet/airoha/airoha_eth.h
index b894828b1375..70f16189b258 100644
--- a/drivers/net/ethernet/airoha/airoha_eth.h
+++ b/drivers/net/ethernet/airoha/airoha_eth.h
@@ -538,7 +538,7 @@ struct airoha_wdma_info {
/* RX queue to IRQ mapping: BIT(q) in IRQ(n) */
#define RX_IRQ0_BANK_PIN_MASK 0x839f
-#define RX_IRQ1_BANK_PIN_MASK 0x7fe00000
+#define RX_IRQ1_BANK_PIN_MASK 0xffe00000
#define RX_IRQ2_BANK_PIN_MASK 0x20
#define RX_IRQ3_BANK_PIN_MASK 0x40
#define RX_IRQ_BANK_PIN_MASK(_n) \
diff --git a/drivers/net/ethernet/allwinner/sun4i-emac.c b/drivers/net/ethernet/allwinner/sun4i-emac.c
index a297874f4a81..9382b4b329bd 100644
--- a/drivers/net/ethernet/allwinner/sun4i-emac.c
+++ b/drivers/net/ethernet/allwinner/sun4i-emac.c
@@ -1067,6 +1067,7 @@ static int emac_probe(struct platform_device *pdev)
return 0;
out_release_sram:
+ of_node_put(db->phy_node);
sunxi_sram_release(&pdev->dev);
out_clk_disable_unprepare:
clk_disable_unprepare(db->clk);
@@ -1094,6 +1095,7 @@ static void emac_remove(struct platform_device *pdev)
}
unregister_netdev(ndev);
+ of_node_put(db->phy_node);
sunxi_sram_release(&pdev->dev);
clk_disable_unprepare(db->clk);
irq_dispose_mapping(ndev->irq);
diff --git a/drivers/net/ethernet/amd/pds_core/fw.c b/drivers/net/ethernet/amd/pds_core/fw.c
index fa626719e68d..59d262b0d29c 100644
--- a/drivers/net/ethernet/amd/pds_core/fw.c
+++ b/drivers/net/ethernet/amd/pds_core/fw.c
@@ -107,8 +107,10 @@ int pdsc_firmware_update(struct pdsc *pdsc, const struct firmware *fw,
dev_info(pdsc->dev, "Installing firmware\n");
- if (!pdsc->cmd_regs)
+ if (!pdsc->cmd_regs) {
+ NL_SET_ERR_MSG_MOD(extack, "BARs not mapped");
return -ENXIO;
+ }
dl = priv_to_devlink(pdsc);
devlink_flash_update_status_notify(dl, "Preparing to flash",
@@ -134,6 +136,12 @@ int pdsc_firmware_update(struct pdsc *pdsc, const struct firmware *fw,
copy_sz = min_t(unsigned int, buf_sz, fw->size - offset);
mutex_lock(&pdsc->devcmd_lock);
+ if (!pdsc->cmd_regs) {
+ mutex_unlock(&pdsc->devcmd_lock);
+ err = -ENXIO;
+ NL_SET_ERR_MSG_MOD(extack, "Device reset during flash");
+ goto err_out;
+ }
memcpy_toio(&pdsc->cmd_regs->data, fw->data + offset, copy_sz);
err = pdsc_devcmd_fw_download_locked(pdsc, data_addr,
offset, copy_sz);
diff --git a/drivers/net/ethernet/amd/pds_core/main.c b/drivers/net/ethernet/amd/pds_core/main.c
index 9a2c64198d03..a19269f6b66c 100644
--- a/drivers/net/ethernet/amd/pds_core/main.c
+++ b/drivers/net/ethernet/amd/pds_core/main.c
@@ -507,8 +507,12 @@ static void pdsc_reset_prepare(struct pci_dev *pdev)
pdsc_auxbus_dev_del(pdsc, pdsc, &pdsc->padev);
}
- pdsc_unmap_bars(pdsc);
- pci_release_regions(pdev);
+ if (!pdev->is_virtfn) {
+ mutex_lock(&pdsc->devcmd_lock);
+ pdsc_unmap_bars(pdsc);
+ mutex_unlock(&pdsc->devcmd_lock);
+ pci_release_regions(pdev);
+ }
if (pci_is_enabled(pdev))
pci_disable_device(pdev);
}
@@ -536,7 +540,9 @@ static void pdsc_reset_done(struct pci_dev *pdev)
return;
}
+ mutex_lock(&pdsc->devcmd_lock);
err = pdsc_map_bars(pdsc);
+ mutex_unlock(&pdsc->devcmd_lock);
if (err)
return;
}
diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-dev.c b/drivers/net/ethernet/amd/xgbe/xgbe-dev.c
index 2de974213090..e2e850c1b90b 100644
--- a/drivers/net/ethernet/amd/xgbe/xgbe-dev.c
+++ b/drivers/net/ethernet/amd/xgbe/xgbe-dev.c
@@ -3400,7 +3400,7 @@ static void xgbe_enable_rx(struct xgbe_prv_data *pdata)
XGMAC_IOWRITE(pdata, MAC_RQC0R, reg_val);
/* Enable MAC Rx */
- XGMAC_IOWRITE_BITS(pdata, MAC_RCR, DCRCC, 1);
+ XGMAC_IOWRITE_BITS(pdata, MAC_RCR, DCRCC, 0);
XGMAC_IOWRITE_BITS(pdata, MAC_RCR, CST, 1);
XGMAC_IOWRITE_BITS(pdata, MAC_RCR, ACS, 1);
XGMAC_IOWRITE_BITS(pdata, MAC_RCR, RE, 1);
@@ -3411,7 +3411,6 @@ static void xgbe_disable_rx(struct xgbe_prv_data *pdata)
unsigned int i;
/* Disable MAC Rx */
- XGMAC_IOWRITE_BITS(pdata, MAC_RCR, DCRCC, 0);
XGMAC_IOWRITE_BITS(pdata, MAC_RCR, CST, 0);
XGMAC_IOWRITE_BITS(pdata, MAC_RCR, ACS, 0);
XGMAC_IOWRITE_BITS(pdata, MAC_RCR, RE, 0);
diff --git a/drivers/net/ethernet/broadcom/asp2/bcmasp_intf.c b/drivers/net/ethernet/broadcom/asp2/bcmasp_intf.c
index ed0977832ce4..f2176ef3a127 100644
--- a/drivers/net/ethernet/broadcom/asp2/bcmasp_intf.c
+++ b/drivers/net/ethernet/broadcom/asp2/bcmasp_intf.c
@@ -148,8 +148,9 @@ static int tx_spb_ring_full(struct bcmasp_intf *intf, int cnt)
int next_index, i;
/* Check if we have enough room for cnt descriptors */
+ next_index = intf->tx_spb_index;
for (i = 0; i < cnt; i++) {
- next_index = incr_ring(intf->tx_spb_index, DESC_RING_COUNT);
+ next_index = incr_ring(next_index, DESC_RING_COUNT);
if (next_index == intf->tx_spb_clean_index)
return 1;
}
@@ -301,6 +302,7 @@ static netdev_tx_t bcmasp_xmit(struct sk_buff *skb, struct net_device *dev)
txcb->bytes_sent = total_bytes;
dma_unmap_addr_set(txcb, dma_addr, mapping);
dma_unmap_len_set(txcb, dma_len, size);
+ txcb->last = false;
if (!i) {
desc->flags |= DESC_SOF;
if (csum_hw)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index fd2ee24f3b0b..ced998399efc 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -462,6 +462,16 @@ u16 bnxt_xmit_get_cfa_action(struct sk_buff *skb)
static void bnxt_txr_db_kick(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
u16 prod)
{
+ /* If the most recent BD has its completion suppressed, unset the bit
+ * so that a completion is generated, otherwise nothing is left to
+ * clean the ring and wake the queue.
+ */
+ if (txr->kick_txbd0) {
+ txr->kick_txbd0->tx_bd_len_flags_type &=
+ cpu_to_le32(~TX_BD_FLAGS_NO_CMPL);
+ txr->kick_txbd0 = NULL;
+ }
+
/* Sync BD data before updating doorbell */
wmb();
bnxt_db_write(bp, &txr->tx_db, prod);
@@ -485,7 +495,6 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)
struct bnxt_sw_tx_bd *tx_buf;
__le32 lflags = 0;
skb_frag_t *frag;
- netdev_tx_t ret;
i = skb_get_queue_mapping(skb);
if (unlikely(i >= bp->tx_nr_rings)) {
@@ -509,11 +518,22 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)
if (skb_is_gso(skb) &&
(skb_shinfo(skb)->gso_type & SKB_GSO_UDP_L4) &&
!(bp->flags & BNXT_FLAG_UDP_GSO_CAP)) {
- ret = bnxt_sw_udp_gso_xmit(bp, txr, txq, skb);
- if (txr->kick_pending)
+ int rc = bnxt_sw_udp_gso_xmit(bp, txr, txq, skb);
+
+ /* if SW USO queued a packet, the doorbell will be written
+ * below and there is no reason to track the last BD with
+ * suppressed completions
+ */
+ if (rc > 0)
+ txr->kick_txbd0 = NULL;
+
+ /* if a packet was queued by SW USO or a doorbell was pending
+ * from a previous xmit that was deferred, write the doorbell.
+ */
+ if (rc > 0 || txr->kick_pending)
bnxt_txr_db_kick(bp, txr, txr->tx_prod);
- return ret;
+ return rc < 0 ? NETDEV_TX_BUSY : NETDEV_TX_OK;
}
free_size = bnxt_tx_avail(bp, txr);
@@ -751,23 +771,23 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)
prod = NEXT_TX(prod);
WRITE_ONCE(txr->tx_prod, prod);
+ txr->kick_txbd0 = NULL;
if (!netdev_xmit_more() || netif_xmit_stopped(txq)) {
bnxt_txr_db_kick(bp, txr, prod);
} else {
- if (free_size >= bp->tx_wake_thresh)
+ if (free_size >= bp->tx_wake_thresh) {
txbd0->tx_bd_len_flags_type |=
cpu_to_le32(TX_BD_FLAGS_NO_CMPL);
+ txr->kick_txbd0 = txbd0;
+ }
txr->kick_pending = 1;
}
tx_done:
if (unlikely(bnxt_tx_avail(bp, txr) <= MAX_SKB_FRAGS + 1)) {
- if (netdev_xmit_more() && !tx_buf->is_push) {
- txbd0->tx_bd_len_flags_type &=
- cpu_to_le32(~TX_BD_FLAGS_NO_CMPL);
+ if (txr->kick_pending)
bnxt_txr_db_kick(bp, txr, prod);
- }
netif_txq_try_stop(txq, bnxt_tx_avail(bp, txr),
bp->tx_wake_thresh);
@@ -1514,14 +1534,16 @@ static int bnxt_discard_rx(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,
return 0;
}
-static u16 bnxt_alloc_agg_idx(struct bnxt_rx_ring_info *rxr, u16 agg_id)
+static u16 bnxt_alloc_agg_idx(struct bnxt *bp, struct bnxt_rx_ring_info *rxr,
+ u16 agg_id)
{
struct bnxt_tpa_idx_map *map = rxr->rx_tpa_idx_map;
- u16 idx = agg_id & MAX_TPA_P5_MASK;
+ u16 idx = agg_id & (bp->max_tpa_roundup_size - 1);
if (test_bit(idx, map->agg_idx_bmap)) {
- idx = find_first_zero_bit(map->agg_idx_bmap, MAX_TPA_P5);
- if (idx >= MAX_TPA_P5)
+ idx = find_first_zero_bit(map->agg_idx_bmap,
+ bp->max_tpa_roundup_size);
+ if (idx >= bp->max_tpa_roundup_size)
return INVALID_HW_RING_ID;
}
__set_bit(idx, map->agg_idx_bmap);
@@ -1586,7 +1608,7 @@ static void bnxt_tpa_start(struct bnxt *bp, struct bnxt_rx_ring_info *rxr,
if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
agg_id = TPA_START_AGG_ID_P5(tpa_start);
- agg_id = bnxt_alloc_agg_idx(rxr, agg_id);
+ agg_id = bnxt_alloc_agg_idx(bp, rxr, agg_id);
if (unlikely(agg_id == INVALID_HW_RING_ID)) {
netdev_warn(bp->dev, "Unable to allocate agg ID for ring %d, agg 0x%x\n",
rxr->bnapi->index,
@@ -3584,7 +3606,7 @@ static void bnxt_free_one_tpa_info_data(struct bnxt *bp,
{
int i;
- for (i = 0; i < bp->max_tpa; i++) {
+ for (i = 0; i < bp->max_tpa_roundup_size; i++) {
struct bnxt_tpa_info *tpa_info = &rxr->rx_tpa[i];
u8 *data = tpa_info->data;
@@ -3781,7 +3803,7 @@ static void bnxt_free_one_tpa_info(struct bnxt *bp,
kfree(rxr->rx_tpa_idx_map);
rxr->rx_tpa_idx_map = NULL;
if (rxr->rx_tpa) {
- for (i = 0; i < bp->max_tpa; i++) {
+ for (i = 0; i < bp->max_tpa_roundup_size; i++) {
kfree(rxr->rx_tpa[i].agg_arr);
rxr->rx_tpa[i].agg_arr = NULL;
}
@@ -3807,13 +3829,14 @@ static int bnxt_alloc_one_tpa_info(struct bnxt *bp,
struct rx_agg_cmp *agg;
int i;
- rxr->rx_tpa = kzalloc_objs(struct bnxt_tpa_info, bp->max_tpa);
+ rxr->rx_tpa = kzalloc_objs(struct bnxt_tpa_info,
+ bp->max_tpa_roundup_size);
if (!rxr->rx_tpa)
return -ENOMEM;
if (!(bp->flags & BNXT_FLAG_CHIP_P5_PLUS))
return 0;
- for (i = 0; i < bp->max_tpa; i++) {
+ for (i = 0; i < bp->max_tpa_roundup_size; i++) {
agg = kzalloc_objs(*agg, MAX_SKB_FRAGS);
if (!agg)
return -ENOMEM;
@@ -3832,6 +3855,9 @@ static int bnxt_alloc_tpa_info(struct bnxt *bp)
bp->max_tpa = MAX_TPA;
if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
+ /* TPA is not supported at all, so there is nothing to
+ * allocate.
+ */
if (!bp->max_tpa_v2)
return 0;
bp->max_tpa = min_t(u16, bp->max_tpa_v2, MAX_TPA_P5);
@@ -3839,6 +3865,7 @@ static int bnxt_alloc_tpa_info(struct bnxt *bp)
if (bp->max_tpa <= 32 && BNXT_CHIP_P5(bp) && !BNXT_NPAR(bp))
bp->max_tpa = MAX_TPA_P5;
}
+ bp->max_tpa_roundup_size = roundup_pow_of_two(bp->max_tpa);
for (i = 0; i < bp->rx_nr_rings; i++) {
struct bnxt_rx_ring_info *rxr = &bp->rx_ring[i];
@@ -4551,7 +4578,7 @@ static int bnxt_alloc_one_tpa_info_data(struct bnxt *bp,
u8 *data;
int i;
- for (i = 0; i < bp->max_tpa; i++) {
+ for (i = 0; i < bp->max_tpa_roundup_size; i++) {
data = __bnxt_alloc_rx_frag(bp, &mapping, rxr,
GFP_KERNEL);
if (!data)
@@ -5006,7 +5033,8 @@ void bnxt_set_rx_skb_mode(struct bnxt *bp, bool page_mode)
bnxt_get_max_rings(bp, &rx, &tx, true);
if (rx > 1) {
bp->flags &= ~BNXT_FLAG_NO_AGG_RINGS;
- bp->dev->hw_features |= NETIF_F_LRO;
+ if (BNXT_SUPPORTS_TPA(bp))
+ bp->dev->hw_features |= NETIF_F_LRO;
}
}
@@ -5427,6 +5455,8 @@ static void bnxt_clear_ring_indices(struct bnxt *bp)
txr->tx_prod = 0;
txr->tx_cons = 0;
txr->tx_hw_cons = 0;
+ txr->kick_pending = 0;
+ txr->kick_txbd0 = NULL;
}
rxr = bnapi->rx_ring;
@@ -11340,8 +11370,13 @@ static int bnxt_shutdown_nic(struct bnxt *bp, bool irq_re_init)
static int bnxt_init_nic(struct bnxt *bp, bool irq_re_init)
{
+ int rc;
+
bnxt_init_cp_rings(bp);
- bnxt_init_rx_rings(bp);
+ rc = bnxt_init_rx_rings(bp);
+ if (rc)
+ return rc;
+
bnxt_init_tx_rings(bp);
bnxt_init_ring_grps(bp, irq_re_init);
bnxt_init_vnics(bp);
@@ -11772,6 +11807,8 @@ static int bnxt_tx_queue_start(struct bnxt *bp, int idx)
txr->tx_prod = 0;
txr->tx_cons = 0;
txr->tx_hw_cons = 0;
+ txr->kick_pending = 0;
+ txr->kick_txbd0 = NULL;
start_tx:
WRITE_ONCE(txr->dev_state, 0);
synchronize_net();
@@ -14563,7 +14600,14 @@ static void bnxt_rx_ring_reset(struct bnxt *bp)
rxr->rx_sw_agg_prod = 0;
rxr->rx_next_cons = 0;
rxr->bnapi->in_reset = false;
- bnxt_alloc_one_rx_ring(bp, i);
+ rc = bnxt_alloc_one_rx_ring(bp, i);
+ if (rc) {
+ netdev_warn(bp->dev, "RX ring reset failed to allocate buffers, rc = %d, falling back to global reset\n",
+ rc);
+ bnxt_reset_task(bp, true);
+ bnxt_rtnl_unlock_sp(bp);
+ return;
+ }
cpr = &rxr->bnapi->cp_ring;
cpr->sw_stats->rx.rx_resets++;
if (bp->flags & BNXT_FLAG_AGG_RINGS)
@@ -16289,6 +16333,8 @@ static int bnxt_queue_mem_alloc(struct net_device *dev,
clone->need_head_pool = false;
clone->rx_page_size = qcfg->rx_page_size;
clone->rx_agg_bmap = NULL;
+ clone->rx_tpa = NULL;
+ clone->rx_tpa_idx_map = NULL;
rc = bnxt_alloc_rx_page_pool(bp, clone, rxr->page_pool->p.nid);
if (rc)
@@ -16332,11 +16378,16 @@ static int bnxt_queue_mem_alloc(struct net_device *dev,
bnxt_alloc_one_rx_ring_skb(bp, clone, idx);
if (bp->flags & BNXT_FLAG_AGG_RINGS)
bnxt_alloc_one_rx_ring_netmem(bp, clone, idx);
- if (bp->flags & BNXT_FLAG_TPA)
- bnxt_alloc_one_tpa_info_data(bp, clone);
+ if (bp->flags & BNXT_FLAG_TPA) {
+ rc = bnxt_alloc_one_tpa_info_data(bp, clone);
+ if (rc)
+ goto err_free_rx_ring_skbs;
+ }
return 0;
+err_free_rx_ring_skbs:
+ bnxt_free_one_rx_ring_skbs(bp, clone);
err_free_tpa_info:
bnxt_free_one_tpa_info(bp, clone);
err_free_rx_agg_ring:
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index dc8ec5e5733e..4dbcd08ef7c3 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -789,7 +789,6 @@ struct nqe_cn {
#define MAX_TPA 64
#define MAX_TPA_P5 256
-#define MAX_TPA_P5_MASK (MAX_TPA_P5 - 1)
#define MAX_TPA_SEGS_P5 0x3f
#if (BNXT_PAGE_SHIFT == 16)
@@ -993,6 +992,7 @@ struct bnxt_tx_ring_info {
u16 txq_index;
u8 tx_napi_idx;
u8 kick_pending;
+ struct tx_bd *kick_txbd0;
struct bnxt_db_info tx_db;
struct tx_bd *tx_desc_ring[MAX_TX_PAGES];
@@ -2382,6 +2382,7 @@ struct bnxt {
u16 max_tpa_v2;
u16 max_tpa;
+ u16 max_tpa_roundup_size;
u32 rx_buf_size;
u32 rx_buf_use_size; /* useable size */
u16 rx_offset;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c
index f7e18bea0fb8..6c1060fa2ea5 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c
@@ -31,10 +31,14 @@ static u32 bnxt_sw_gso_lhint(unsigned int len)
return TX_BD_FLAGS_LHINT_2048_AND_LARGER;
}
-netdev_tx_t bnxt_sw_udp_gso_xmit(struct bnxt *bp,
- struct bnxt_tx_ring_info *txr,
- struct netdev_queue *txq,
- struct sk_buff *skb)
+/* Transmit an skb requiring software UDP segmentation.
+ *
+ * Returns 1 if the skb was queued and new BDs were produced, 0 if the skb
+ * was dropped, or -1 if the ring is full and the skb should be retried.
+ * The caller owns the doorbell for all three cases.
+ */
+int bnxt_sw_udp_gso_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ struct netdev_queue *txq, struct sk_buff *skb)
{
unsigned int last_unmap_len __maybe_unused = 0;
dma_addr_t last_unmap_addr __maybe_unused = 0;
@@ -69,7 +73,7 @@ netdev_tx_t bnxt_sw_udp_gso_xmit(struct bnxt *bp,
if (unlikely(bnxt_tx_avail(bp, txr) < bds_needed)) {
netif_txq_try_stop(txq, bnxt_tx_avail(bp, txr),
bp->tx_wake_thresh);
- return NETDEV_TX_BUSY;
+ return -1;
}
/* BD backpressure alone cannot prevent overwriting in-flight
@@ -77,7 +81,7 @@ netdev_tx_t bnxt_sw_udp_gso_xmit(struct bnxt *bp,
*/
if (!netif_txq_maybe_stop(txq, bnxt_inline_avail(txr),
num_segs, num_segs))
- return NETDEV_TX_BUSY;
+ return -1;
if (unlikely(tso_dma_map_init(&map, &pdev->dev, skb, hdr_len)))
goto drop;
@@ -223,16 +227,15 @@ netdev_tx_t bnxt_sw_udp_gso_xmit(struct bnxt *bp,
netdev_tx_sent_queue(txq, skb->len);
WRITE_ONCE(txr->tx_prod, prod);
- txr->kick_pending = 1;
if (unlikely(bnxt_tx_avail(bp, txr) <= bp->tx_wake_thresh))
netif_txq_try_stop(txq, bnxt_tx_avail(bp, txr),
bp->tx_wake_thresh);
- return NETDEV_TX_OK;
+ return 1;
drop:
dev_kfree_skb_any(skb);
dev_core_stats_tx_dropped_inc(bp->dev);
- return NETDEV_TX_OK;
+ return 0;
}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.h
index 47528c20f311..77d9af97cc22 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.h
@@ -38,9 +38,7 @@ static inline int bnxt_min_tx_desc_cnt(struct bnxt *bp,
return BNXT_MIN_TX_DESC_CNT;
}
-netdev_tx_t bnxt_sw_udp_gso_xmit(struct bnxt *bp,
- struct bnxt_tx_ring_info *txr,
- struct netdev_queue *txq,
- struct sk_buff *skb);
+int bnxt_sw_udp_gso_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ struct netdev_queue *txq, struct sk_buff *skb);
#endif
diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h
index 2de56017ee0d..f8bff3e71b68 100644
--- a/drivers/net/ethernet/cadence/macb.h
+++ b/drivers/net/ethernet/cadence/macb.h
@@ -974,6 +974,8 @@ struct macb_dma_desc_ptp {
* of the frame
* @mapping: DMA address of the skb's fragment buffer
* @size: size of the DMA mapped buffer
+ * @fcs_len: FCS bytes appended in software, 0 or ETH_FCS_LEN, only
+ * set for the last buffer of the frame
* @mapped_as_page: true when buffer was mapped with skb_frag_dma_map(),
* false when buffer was mapped with dma_map_single()
*/
@@ -981,6 +983,7 @@ struct macb_tx_skb {
struct sk_buff *skb;
dma_addr_t mapping;
size_t size;
+ u8 fcs_len;
bool mapped_as_page;
};
@@ -1207,11 +1210,11 @@ struct macb_or_gem_ops {
/* MACB-PTP interface: adapt to platform needs. */
struct macb_ptp_info {
- void (*ptp_init)(struct net_device *ndev);
- void (*ptp_remove)(struct net_device *ndev);
+ void (*ptp_init)(struct net_device *netdev);
+ void (*ptp_remove)(struct net_device *netdev);
s32 (*get_ptp_max_adj)(void);
unsigned int (*get_tsu_rate)(struct macb *bp);
- int (*get_ts_info)(struct net_device *dev,
+ int (*get_ts_info)(struct net_device *netdev,
struct kernel_ethtool_ts_info *info);
int (*get_hwtst)(struct net_device *netdev,
struct kernel_hwtstamp_config *tstamp_config);
@@ -1326,7 +1329,7 @@ struct macb {
struct clk *tx_clk;
struct clk *rx_clk;
struct clk *tsu_clk;
- struct net_device *dev;
+ struct net_device *netdev;
/* Protects hw_stats and ethtool_stats */
spinlock_t stats_lock;
union {
@@ -1406,8 +1409,8 @@ enum macb_bd_control {
TSTAMP_ALL_FRAMES,
};
-void gem_ptp_init(struct net_device *ndev);
-void gem_ptp_remove(struct net_device *ndev);
+void gem_ptp_init(struct net_device *netdev);
+void gem_ptp_remove(struct net_device *netdev);
void gem_ptp_txstamp(struct macb *bp, struct sk_buff *skb, struct macb_dma_desc *desc);
void gem_ptp_rxstamp(struct macb *bp, struct sk_buff *skb, struct macb_dma_desc *desc);
static inline void gem_ptp_do_txstamp(struct macb *bp, struct sk_buff *skb, struct macb_dma_desc *desc)
@@ -1426,14 +1429,14 @@ static inline void gem_ptp_do_rxstamp(struct macb *bp, struct sk_buff *skb, stru
gem_ptp_rxstamp(bp, skb, desc);
}
-int gem_get_hwtst(struct net_device *dev,
+int gem_get_hwtst(struct net_device *netdev,
struct kernel_hwtstamp_config *tstamp_config);
-int gem_set_hwtst(struct net_device *dev,
+int gem_set_hwtst(struct net_device *netdev,
struct kernel_hwtstamp_config *tstamp_config,
struct netlink_ext_ack *extack);
#else
-static inline void gem_ptp_init(struct net_device *ndev) { }
-static inline void gem_ptp_remove(struct net_device *ndev) { }
+static inline void gem_ptp_init(struct net_device *netdev) { }
+static inline void gem_ptp_remove(struct net_device *netdev) { }
static inline void gem_ptp_do_txstamp(struct macb *bp, struct sk_buff *skb, struct macb_dma_desc *desc) { }
static inline void gem_ptp_do_rxstamp(struct macb *bp, struct sk_buff *skb, struct macb_dma_desc *desc) { }
diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
index d394f1f43b68..8085a3b3846d 100644
--- a/drivers/net/ethernet/cadence/macb_main.c
+++ b/drivers/net/ethernet/cadence/macb_main.c
@@ -252,9 +252,9 @@ static void macb_set_hwaddr(struct macb *bp)
u32 bottom;
u16 top;
- bottom = get_unaligned_le32(bp->dev->dev_addr);
+ bottom = get_unaligned_le32(bp->netdev->dev_addr);
macb_or_gem_writel(bp, SA1B, bottom);
- top = get_unaligned_le16(bp->dev->dev_addr + 4);
+ top = get_unaligned_le16(bp->netdev->dev_addr + 4);
macb_or_gem_writel(bp, SA1T, top);
if (gem_has_ptp(bp)) {
@@ -291,13 +291,13 @@ static void macb_get_hwaddr(struct macb *bp)
addr[5] = (top >> 8) & 0xff;
if (is_valid_ether_addr(addr)) {
- eth_hw_addr_set(bp->dev, addr);
+ eth_hw_addr_set(bp->netdev, addr);
return;
}
}
dev_info(&bp->pdev->dev, "invalid hw address, using random\n");
- eth_hw_addr_random(bp->dev);
+ eth_hw_addr_random(bp->netdev);
}
static int macb_mdio_wait_for_idle(struct macb *bp)
@@ -509,12 +509,12 @@ static void macb_set_tx_clk(struct macb *bp, int speed)
ferr = abs(rate_rounded - rate);
ferr = DIV_ROUND_UP(ferr, rate / 100000);
if (ferr > 5)
- netdev_warn(bp->dev,
+ netdev_warn(bp->netdev,
"unable to generate target frequency: %ld Hz\n",
rate);
if (clk_set_rate(bp->tx_clk, rate_rounded))
- netdev_err(bp->dev, "adjusting tx_clk failed.\n");
+ netdev_err(bp->netdev, "adjusting tx_clk failed.\n");
}
static void macb_usx_pcs_link_up(struct phylink_pcs *pcs, unsigned int neg_mode,
@@ -697,8 +697,8 @@ static void macb_tx_lpi_wake(struct macb *bp)
static void macb_mac_disable_tx_lpi(struct phylink_config *config)
{
- struct net_device *ndev = to_net_dev(config->dev);
- struct macb *bp = netdev_priv(ndev);
+ struct net_device *netdev = to_net_dev(config->dev);
+ struct macb *bp = netdev_priv(netdev);
unsigned long flags;
cancel_delayed_work_sync(&bp->tx_lpi_work);
@@ -712,8 +712,8 @@ static void macb_mac_disable_tx_lpi(struct phylink_config *config)
static int macb_mac_enable_tx_lpi(struct phylink_config *config, u32 timer,
bool tx_clk_stop)
{
- struct net_device *ndev = to_net_dev(config->dev);
- struct macb *bp = netdev_priv(ndev);
+ struct net_device *netdev = to_net_dev(config->dev);
+ struct macb *bp = netdev_priv(netdev);
unsigned long flags;
spin_lock_irqsave(&bp->lock, flags);
@@ -732,8 +732,8 @@ static int macb_mac_enable_tx_lpi(struct phylink_config *config, u32 timer,
static void macb_mac_config(struct phylink_config *config, unsigned int mode,
const struct phylink_link_state *state)
{
- struct net_device *ndev = to_net_dev(config->dev);
- struct macb *bp = netdev_priv(ndev);
+ struct net_device *netdev = to_net_dev(config->dev);
+ struct macb *bp = netdev_priv(netdev);
unsigned long flags;
u32 old_ctrl, ctrl;
u32 old_ncr, ncr;
@@ -774,8 +774,8 @@ static void macb_mac_config(struct phylink_config *config, unsigned int mode,
static void macb_mac_link_down(struct phylink_config *config, unsigned int mode,
phy_interface_t interface)
{
- struct net_device *ndev = to_net_dev(config->dev);
- struct macb *bp = netdev_priv(ndev);
+ struct net_device *netdev = to_net_dev(config->dev);
+ struct macb *bp = netdev_priv(netdev);
struct macb_queue *queue;
unsigned int q;
u32 ctrl;
@@ -789,7 +789,7 @@ static void macb_mac_link_down(struct phylink_config *config, unsigned int mode,
ctrl = macb_readl(bp, NCR) & ~(MACB_BIT(RE) | MACB_BIT(TE));
macb_writel(bp, NCR, ctrl);
- netif_tx_stop_all_queues(ndev);
+ netif_tx_stop_all_queues(netdev);
}
/* Use juggling algorithm to left rotate tx ring and tx skb array */
@@ -884,13 +884,13 @@ static void gem_shuffle_tx_rings(struct macb *bp)
}
static void macb_mac_link_up(struct phylink_config *config,
- struct phy_device *phy,
+ struct phy_device *phydev,
unsigned int mode, phy_interface_t interface,
int speed, int duplex,
bool tx_pause, bool rx_pause)
{
- struct net_device *ndev = to_net_dev(config->dev);
- struct macb *bp = netdev_priv(ndev);
+ struct net_device *netdev = to_net_dev(config->dev);
+ struct macb *bp = netdev_priv(netdev);
struct macb_queue *queue;
unsigned long flags;
unsigned int q;
@@ -946,14 +946,14 @@ static void macb_mac_link_up(struct phylink_config *config,
macb_writel(bp, NCR, ctrl | MACB_BIT(RE) | MACB_BIT(TE));
- netif_tx_wake_all_queues(ndev);
+ netif_tx_wake_all_queues(netdev);
}
static struct phylink_pcs *macb_mac_select_pcs(struct phylink_config *config,
phy_interface_t interface)
{
- struct net_device *ndev = to_net_dev(config->dev);
- struct macb *bp = netdev_priv(ndev);
+ struct net_device *netdev = to_net_dev(config->dev);
+ struct macb *bp = netdev_priv(netdev);
if (interface == PHY_INTERFACE_MODE_10GBASER)
return &bp->phylink_usx_pcs;
@@ -982,7 +982,7 @@ static bool macb_phy_handle_exists(struct device_node *dn)
static int macb_phylink_connect(struct macb *bp)
{
struct device_node *dn = bp->pdev->dev.of_node;
- struct net_device *dev = bp->dev;
+ struct net_device *netdev = bp->netdev;
struct phy_device *phydev;
int ret;
@@ -992,7 +992,7 @@ static int macb_phylink_connect(struct macb *bp)
if (!dn || (ret && !macb_phy_handle_exists(dn))) {
phydev = phy_find_first(bp->mii_bus);
if (!phydev) {
- netdev_err(dev, "no PHY found\n");
+ netdev_err(netdev, "no PHY found\n");
return -ENXIO;
}
@@ -1001,7 +1001,7 @@ static int macb_phylink_connect(struct macb *bp)
}
if (ret) {
- netdev_err(dev, "Could not attach PHY (%d)\n", ret);
+ netdev_err(netdev, "Could not attach PHY (%d)\n", ret);
return ret;
}
@@ -1013,21 +1013,21 @@ static int macb_phylink_connect(struct macb *bp)
static void macb_get_pcs_fixed_state(struct phylink_config *config,
struct phylink_link_state *state)
{
- struct net_device *ndev = to_net_dev(config->dev);
- struct macb *bp = netdev_priv(ndev);
+ struct net_device *netdev = to_net_dev(config->dev);
+ struct macb *bp = netdev_priv(netdev);
state->link = (macb_readl(bp, NSR) & MACB_BIT(NSR_LINK)) != 0;
}
/* based on au1000_eth. c*/
-static int macb_mii_probe(struct net_device *dev)
+static int macb_mii_probe(struct net_device *netdev)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
bp->phylink_sgmii_pcs.ops = &macb_phylink_pcs_ops;
bp->phylink_usx_pcs.ops = &macb_phylink_usx_pcs_ops;
- bp->phylink_config.dev = &dev->dev;
+ bp->phylink_config.dev = &netdev->dev;
bp->phylink_config.type = PHYLINK_NETDEV;
bp->phylink_config.mac_managed_pm = true;
@@ -1086,7 +1086,7 @@ static int macb_mii_probe(struct net_device *dev)
bp->phylink = phylink_create(&bp->phylink_config, bp->pdev->dev.fwnode,
bp->phy_interface, &macb_phylink_ops);
if (IS_ERR(bp->phylink)) {
- netdev_err(dev, "Could not create a phylink instance (%ld)\n",
+ netdev_err(netdev, "Could not create a phylink instance (%ld)\n",
PTR_ERR(bp->phylink));
return PTR_ERR(bp->phylink);
}
@@ -1133,7 +1133,7 @@ static int macb_mii_init(struct macb *bp)
*/
mdio_np = of_get_child_by_name(np, "mdio");
if (!mdio_np && of_phy_is_fixed_link(np))
- return macb_mii_probe(bp->dev);
+ return macb_mii_probe(bp->netdev);
/* Enable management port */
macb_writel(bp, NCR, MACB_BIT(MPE));
@@ -1154,16 +1154,18 @@ static int macb_mii_init(struct macb *bp)
bp->mii_bus->priv = bp;
bp->mii_bus->parent = &bp->pdev->dev;
- dev_set_drvdata(&bp->dev->dev, bp->mii_bus);
+ dev_set_drvdata(&bp->netdev->dev, bp->mii_bus);
err = macb_mdiobus_register(bp, mdio_np);
if (err)
goto err_out_free_mdiobus;
- err = macb_mii_probe(bp->dev);
+ err = macb_mii_probe(bp->netdev);
if (err)
goto err_out_unregister_bus;
+ of_node_put(mdio_np);
+
return 0;
err_out_unregister_bus:
@@ -1268,8 +1270,8 @@ static void macb_tx_error_task(struct work_struct *work)
unsigned long flags;
queue_index = queue - bp->queues;
- netdev_vdbg(bp->dev, "macb_tx_error_task: q = %u, t = %u, h = %u\n",
- queue_index, queue->tx_tail, queue->tx_head);
+ netdev_vdbg(bp->netdev, "%s: q = %u, t = %u, h = %u\n",
+ __func__, queue_index, queue->tx_tail, queue->tx_head);
/* Prevent the queue NAPI TX poll from running, as it calls
* macb_tx_complete(), which in turn may call netif_wake_subqueue().
@@ -1281,14 +1283,14 @@ static void macb_tx_error_task(struct work_struct *work)
spin_lock_irqsave(&bp->lock, flags);
/* Make sure nobody is trying to queue up new packets */
- netif_tx_stop_all_queues(bp->dev);
+ netif_tx_stop_all_queues(bp->netdev);
/* Stop transmission now
* (in case we have just queued new packets)
* macb/gem must be halted to write TBQP register
*/
if (macb_halt_tx(bp)) {
- netdev_err(bp->dev, "BUG: halt tx timed out\n");
+ netdev_err(bp->netdev, "BUG: halt tx timed out\n");
macb_writel(bp, NCR, macb_readl(bp, NCR) & (~MACB_BIT(TE)));
halt_timeout = true;
}
@@ -1317,14 +1319,14 @@ static void macb_tx_error_task(struct work_struct *work)
* since it's the only one written back by the hardware
*/
if (!(ctrl & MACB_BIT(TX_BUF_EXHAUSTED))) {
- netdev_vdbg(bp->dev, "txerr skb %u (data %p) TX complete\n",
+ netdev_vdbg(bp->netdev, "txerr skb %u (data %p) TX complete\n",
macb_tx_ring_wrap(bp, tail),
skb->data);
- bp->dev->stats.tx_packets++;
+ bp->netdev->stats.tx_packets++;
queue->stats.tx_packets++;
packets++;
- bp->dev->stats.tx_bytes += skb->len;
- queue->stats.tx_bytes += skb->len;
+ bp->netdev->stats.tx_bytes += skb->len - tx_skb->fcs_len;
+ queue->stats.tx_bytes += skb->len - tx_skb->fcs_len;
bytes += skb->len;
}
} else {
@@ -1333,7 +1335,7 @@ static void macb_tx_error_task(struct work_struct *work)
* those. Statistics are updated by hardware.
*/
if (ctrl & MACB_BIT(TX_BUF_EXHAUSTED))
- netdev_err(bp->dev,
+ netdev_err(bp->netdev,
"BUG: TX buffers exhausted mid-frame\n");
desc->ctrl = ctrl | MACB_BIT(TX_USED);
@@ -1342,7 +1344,7 @@ static void macb_tx_error_task(struct work_struct *work)
macb_tx_unmap(bp, tx_skb, 0);
}
- netdev_tx_completed_queue(netdev_get_tx_queue(bp->dev, queue_index),
+ netdev_tx_completed_queue(netdev_get_tx_queue(bp->netdev, queue_index),
packets, bytes);
/* Set end of TX queue */
@@ -1367,7 +1369,7 @@ static void macb_tx_error_task(struct work_struct *work)
macb_writel(bp, NCR, macb_readl(bp, NCR) | MACB_BIT(TE));
/* Now we are ready to start transmission again */
- netif_tx_start_all_queues(bp->dev);
+ netif_tx_start_all_queues(bp->netdev);
macb_writel(bp, NCR, macb_readl(bp, NCR) | MACB_BIT(TSTART));
spin_unlock_irqrestore(&bp->lock, flags);
@@ -1446,13 +1448,13 @@ static int macb_tx_complete(struct macb_queue *queue, int budget)
!ptp_one_step_sync(skb))
gem_ptp_do_txstamp(bp, skb, desc);
- netdev_vdbg(bp->dev, "skb %u (data %p) TX complete\n",
+ netdev_vdbg(bp->netdev, "skb %u (data %p) TX complete\n",
macb_tx_ring_wrap(bp, tail),
skb->data);
- bp->dev->stats.tx_packets++;
+ bp->netdev->stats.tx_packets++;
queue->stats.tx_packets++;
- bp->dev->stats.tx_bytes += skb->len;
- queue->stats.tx_bytes += skb->len;
+ bp->netdev->stats.tx_bytes += skb->len - tx_skb->fcs_len;
+ queue->stats.tx_bytes += skb->len - tx_skb->fcs_len;
packets++;
bytes += skb->len;
}
@@ -1469,14 +1471,14 @@ static int macb_tx_complete(struct macb_queue *queue, int budget)
}
}
- netdev_tx_completed_queue(netdev_get_tx_queue(bp->dev, queue_index),
+ netdev_tx_completed_queue(netdev_get_tx_queue(bp->netdev, queue_index),
packets, bytes);
queue->tx_tail = tail;
- if (__netif_subqueue_stopped(bp->dev, queue_index) &&
+ if (__netif_subqueue_stopped(bp->netdev, queue_index) &&
CIRC_CNT(queue->tx_head, queue->tx_tail,
bp->tx_ring_size) <= MACB_TX_WAKEUP_THRESH(bp))
- netif_wake_subqueue(bp->dev, queue_index);
+ netif_wake_subqueue(bp->netdev, queue_index);
spin_unlock_irqrestore(&queue->tx_ptr_lock, flags);
if (packets)
@@ -1504,9 +1506,9 @@ static void gem_rx_refill(struct macb_queue *queue)
if (!queue->rx_skbuff[entry]) {
/* allocate sk_buff for this free entry in ring */
- skb = netdev_alloc_skb(bp->dev, bp->rx_buffer_size);
+ skb = netdev_alloc_skb(bp->netdev, bp->rx_buffer_size);
if (unlikely(!skb)) {
- netdev_err(bp->dev,
+ netdev_err(bp->netdev,
"Unable to allocate sk_buff\n");
break;
}
@@ -1555,8 +1557,8 @@ static void gem_rx_refill(struct macb_queue *queue)
/* Make descriptor updates visible to hardware */
wmb();
- netdev_vdbg(bp->dev, "rx ring: queue: %p, prepared head %d, tail %d\n",
- queue, queue->rx_prepared_head, queue->rx_tail);
+ netdev_vdbg(bp->netdev, "rx ring: queue: %p, prepared head %d, tail %d\n",
+ queue, queue->rx_prepared_head, queue->rx_tail);
}
/* Mark DMA descriptors from begin up to and not including end as unused */
@@ -1616,17 +1618,17 @@ static int gem_rx(struct macb_queue *queue, struct napi_struct *napi,
count++;
if (!(ctrl & MACB_BIT(RX_SOF) && ctrl & MACB_BIT(RX_EOF))) {
- netdev_err(bp->dev,
+ netdev_err(bp->netdev,
"not whole frame pointed by descriptor\n");
- bp->dev->stats.rx_dropped++;
+ bp->netdev->stats.rx_dropped++;
queue->stats.rx_dropped++;
break;
}
skb = queue->rx_skbuff[entry];
if (unlikely(!skb)) {
- netdev_err(bp->dev,
+ netdev_err(bp->netdev,
"inconsistent Rx descriptor chain\n");
- bp->dev->stats.rx_dropped++;
+ bp->netdev->stats.rx_dropped++;
queue->stats.rx_dropped++;
break;
}
@@ -1634,28 +1636,29 @@ static int gem_rx(struct macb_queue *queue, struct napi_struct *napi,
queue->rx_skbuff[entry] = NULL;
len = ctrl & bp->rx_frm_len_mask;
- netdev_vdbg(bp->dev, "gem_rx %u (len %u)\n", entry, len);
+ netdev_vdbg(bp->netdev, "%s %u (len %u)\n",
+ __func__, entry, len);
skb_put(skb, len);
dma_unmap_single(&bp->pdev->dev, addr,
bp->rx_buffer_size, DMA_FROM_DEVICE);
- skb->protocol = eth_type_trans(skb, bp->dev);
+ skb->protocol = eth_type_trans(skb, bp->netdev);
skb_checksum_none_assert(skb);
- if (bp->dev->features & NETIF_F_RXCSUM &&
- !(bp->dev->flags & IFF_PROMISC) &&
+ if (bp->netdev->features & NETIF_F_RXCSUM &&
+ !(bp->netdev->flags & IFF_PROMISC) &&
GEM_BFEXT(RX_CSUM, ctrl) & GEM_RX_CSUM_CHECKED_MASK)
skb->ip_summed = CHECKSUM_UNNECESSARY;
- bp->dev->stats.rx_packets++;
+ bp->netdev->stats.rx_packets++;
queue->stats.rx_packets++;
- bp->dev->stats.rx_bytes += skb->len;
+ bp->netdev->stats.rx_bytes += skb->len;
queue->stats.rx_bytes += skb->len;
gem_ptp_do_rxstamp(bp, skb, desc);
#if defined(DEBUG) && defined(VERBOSE_DEBUG)
- netdev_vdbg(bp->dev, "received skb of length %u, csum: %08x\n",
+ netdev_vdbg(bp->netdev, "received skb of length %u, csum: %08x\n",
skb->len, skb->csum);
print_hex_dump(KERN_DEBUG, " mac: ", DUMP_PREFIX_ADDRESS, 16, 1,
skb_mac_header(skb), 16, true);
@@ -1684,9 +1687,10 @@ static int macb_rx_frame(struct macb_queue *queue, struct napi_struct *napi,
desc = macb_rx_desc(queue, last_frag);
len = desc->ctrl & bp->rx_frm_len_mask;
- netdev_vdbg(bp->dev, "macb_rx_frame frags %u - %u (len %u)\n",
- macb_rx_ring_wrap(bp, first_frag),
- macb_rx_ring_wrap(bp, last_frag), len);
+ netdev_vdbg(bp->netdev, "%s frags %u - %u (len %u)\n",
+ __func__,
+ macb_rx_ring_wrap(bp, first_frag),
+ macb_rx_ring_wrap(bp, last_frag), len);
/* The ethernet header starts NET_IP_ALIGN bytes into the
* first buffer. Since the header is 14 bytes, this makes the
@@ -1696,9 +1700,9 @@ static int macb_rx_frame(struct macb_queue *queue, struct napi_struct *napi,
* the two padding bytes into the skb so that we avoid hitting
* the slowpath in memcpy(), and pull them off afterwards.
*/
- skb = netdev_alloc_skb(bp->dev, len + NET_IP_ALIGN);
+ skb = netdev_alloc_skb(bp->netdev, len + NET_IP_ALIGN);
if (!skb) {
- bp->dev->stats.rx_dropped++;
+ bp->netdev->stats.rx_dropped++;
for (frag = first_frag; ; frag++) {
desc = macb_rx_desc(queue, frag);
desc->addr &= ~MACB_BIT(RX_USED);
@@ -1742,11 +1746,11 @@ static int macb_rx_frame(struct macb_queue *queue, struct napi_struct *napi,
wmb();
__skb_pull(skb, NET_IP_ALIGN);
- skb->protocol = eth_type_trans(skb, bp->dev);
+ skb->protocol = eth_type_trans(skb, bp->netdev);
- bp->dev->stats.rx_packets++;
- bp->dev->stats.rx_bytes += skb->len;
- netdev_vdbg(bp->dev, "received skb of length %u, csum: %08x\n",
+ bp->netdev->stats.rx_packets++;
+ bp->netdev->stats.rx_bytes += skb->len;
+ netdev_vdbg(bp->netdev, "received skb of length %u, csum: %08x\n",
skb->len, skb->csum);
napi_gro_receive(napi, skb);
@@ -1826,7 +1830,7 @@ static int macb_rx(struct macb_queue *queue, struct napi_struct *napi,
unsigned long flags;
u32 ctrl;
- netdev_err(bp->dev, "RX queue corruption: reset it\n");
+ netdev_err(bp->netdev, "RX queue corruption: reset it\n");
spin_lock_irqsave(&bp->lock, flags);
@@ -1873,7 +1877,7 @@ static int macb_rx_poll(struct napi_struct *napi, int budget)
work_done = bp->macbgem_ops.mog_rx(queue, napi, budget);
- netdev_vdbg(bp->dev, "RX poll: queue = %u, work_done = %d, budget = %d\n",
+ netdev_vdbg(bp->netdev, "RX poll: queue = %u, work_done = %d, budget = %d\n",
(unsigned int)(queue - bp->queues), work_done, budget);
if (work_done < budget && napi_complete_done(napi, work_done)) {
@@ -1892,7 +1896,7 @@ static int macb_rx_poll(struct napi_struct *napi, int budget)
if (macb_rx_pending(queue)) {
queue_writel(queue, IDR, bp->rx_intr_mask);
macb_queue_isr_clear(bp, queue, MACB_BIT(RCOMP));
- netdev_vdbg(bp->dev, "poll: packets pending, reschedule\n");
+ netdev_vdbg(bp->netdev, "poll: packets pending, reschedule\n");
napi_schedule(napi);
}
}
@@ -1956,11 +1960,11 @@ static int macb_tx_poll(struct napi_struct *napi, int budget)
rmb(); // ensure txubr_pending is up to date
if (queue->txubr_pending) {
queue->txubr_pending = false;
- netdev_vdbg(bp->dev, "poll: tx restart\n");
+ netdev_vdbg(bp->netdev, "poll: tx restart\n");
macb_tx_restart(queue);
}
- netdev_vdbg(bp->dev, "TX poll: queue = %u, work_done = %d, budget = %d\n",
+ netdev_vdbg(bp->netdev, "TX poll: queue = %u, work_done = %d, budget = %d\n",
(unsigned int)(queue - bp->queues), work_done, budget);
if (work_done < budget && napi_complete_done(napi, work_done)) {
@@ -1979,7 +1983,7 @@ static int macb_tx_poll(struct napi_struct *napi, int budget)
if (macb_tx_complete_pending(queue)) {
queue_writel(queue, IDR, MACB_BIT(TCOMP));
macb_queue_isr_clear(bp, queue, MACB_BIT(TCOMP));
- netdev_vdbg(bp->dev, "TX poll: packets pending, reschedule\n");
+ netdev_vdbg(bp->netdev, "TX poll: packets pending, reschedule\n");
napi_schedule(napi);
}
}
@@ -1990,7 +1994,7 @@ static int macb_tx_poll(struct napi_struct *napi, int budget)
static void macb_hresp_error_task(struct work_struct *work)
{
struct macb *bp = from_work(bp, work, hresp_err_bh_work);
- struct net_device *dev = bp->dev;
+ struct net_device *netdev = bp->netdev;
struct macb_queue *queue;
unsigned int q;
u32 ctrl;
@@ -2004,8 +2008,8 @@ static void macb_hresp_error_task(struct work_struct *work)
ctrl &= ~(MACB_BIT(RE) | MACB_BIT(TE));
macb_writel(bp, NCR, ctrl);
- netif_tx_stop_all_queues(dev);
- netif_carrier_off(dev);
+ netif_tx_stop_all_queues(netdev);
+ netif_carrier_off(netdev);
bp->macbgem_ops.mog_init_rings(bp);
@@ -2022,8 +2026,8 @@ static void macb_hresp_error_task(struct work_struct *work)
ctrl |= MACB_BIT(RE) | MACB_BIT(TE);
macb_writel(bp, NCR, ctrl);
- netif_carrier_on(dev);
- netif_tx_start_all_queues(dev);
+ netif_carrier_on(netdev);
+ netif_tx_start_all_queues(netdev);
}
static void macb_wol_interrupt(struct macb_queue *queue, u32 status)
@@ -2032,7 +2036,7 @@ static void macb_wol_interrupt(struct macb_queue *queue, u32 status)
queue_writel(queue, IDR, MACB_BIT(WOL));
macb_writel(bp, WOL, 0);
- netdev_vdbg(bp->dev, "MACB WoL: queue = %u, isr = 0x%08lx\n",
+ netdev_vdbg(bp->netdev, "MACB WoL: queue = %u, isr = 0x%08lx\n",
(unsigned int)(queue - bp->queues),
(unsigned long)status);
macb_queue_isr_clear(bp, queue, MACB_BIT(WOL));
@@ -2045,7 +2049,7 @@ static void gem_wol_interrupt(struct macb_queue *queue, u32 status)
queue_writel(queue, IDR, GEM_BIT(WOL));
gem_writel(bp, WOL, 0);
- netdev_vdbg(bp->dev, "GEM WoL: queue = %u, isr = 0x%08lx\n",
+ netdev_vdbg(bp->netdev, "GEM WoL: queue = %u, isr = 0x%08lx\n",
(unsigned int)(queue - bp->queues),
(unsigned long)status);
macb_queue_isr_clear(bp, queue, GEM_BIT(WOL));
@@ -2055,10 +2059,10 @@ static void gem_wol_interrupt(struct macb_queue *queue, u32 status)
static int macb_interrupt_misc(struct macb_queue *queue, u32 status)
{
struct macb *bp = queue->bp;
- struct net_device *dev;
+ struct net_device *netdev;
u32 ctrl;
- dev = bp->dev;
+ netdev = bp->netdev;
if (unlikely(status & (MACB_TX_ERR_FLAGS))) {
queue_writel(queue, IDR, MACB_TX_INT_FLAGS);
@@ -2099,7 +2103,7 @@ static int macb_interrupt_misc(struct macb_queue *queue, u32 status)
if (status & MACB_BIT(HRESP)) {
queue_work(system_bh_wq, &bp->hresp_err_bh_work);
- netdev_err(dev, "DMA bus error: HRESP not OK\n");
+ netdev_err(netdev, "DMA bus error: HRESP not OK\n");
macb_queue_isr_clear(bp, queue, MACB_BIT(HRESP));
}
@@ -2118,7 +2122,7 @@ static irqreturn_t macb_interrupt(int irq, void *dev_id)
{
struct macb_queue *queue = dev_id;
struct macb *bp = queue->bp;
- struct net_device *dev = bp->dev;
+ struct net_device *netdev = bp->netdev;
u32 status;
status = queue_readl(queue, ISR);
@@ -2130,13 +2134,13 @@ static irqreturn_t macb_interrupt(int irq, void *dev_id)
while (status) {
/* close possible race with dev_close */
- if (unlikely(!netif_running(dev))) {
+ if (unlikely(!netif_running(netdev))) {
queue_writel(queue, IDR, -1);
macb_queue_isr_clear(bp, queue, -1);
break;
}
- netdev_vdbg(bp->dev, "queue = %u, isr = 0x%08lx\n",
+ netdev_vdbg(netdev, "queue = %u, isr = 0x%08lx\n",
(unsigned int)(queue - bp->queues),
(unsigned long)status);
@@ -2181,16 +2185,16 @@ static irqreturn_t macb_interrupt(int irq, void *dev_id)
/* Polling receive - used by netconsole and other diagnostic tools
* to allow network i/o with interrupts disabled.
*/
-static void macb_poll_controller(struct net_device *dev)
+static void macb_poll_controller(struct net_device *netdev)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
struct macb_queue *queue;
unsigned long flags;
unsigned int q;
local_irq_save(flags);
for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue)
- macb_interrupt(dev->irq, queue);
+ macb_interrupt(netdev->irq, queue);
local_irq_restore(flags);
}
#endif
@@ -2198,7 +2202,8 @@ static void macb_poll_controller(struct net_device *dev)
static unsigned int macb_tx_map(struct macb *bp,
struct macb_queue *queue,
struct sk_buff *skb,
- unsigned int hdrlen)
+ unsigned int hdrlen,
+ u8 fcs_len)
{
unsigned int f, nr_frags = skb_shinfo(skb)->nr_frags;
unsigned int len, i, tx_head = queue->tx_head;
@@ -2277,12 +2282,13 @@ static unsigned int macb_tx_map(struct macb *bp,
/* Should never happen */
if (unlikely(!tx_skb)) {
- netdev_err(bp->dev, "BUG! empty skb!\n");
+ netdev_err(bp->netdev, "BUG! empty skb!\n");
return 0;
}
/* This is the last buffer of the frame: save socket buffer */
tx_skb->skb = skb;
+ tx_skb->fcs_len = fcs_len;
/* Update TX ring: update buffer descriptors in reverse order
* to avoid race condition
@@ -2328,7 +2334,7 @@ static unsigned int macb_tx_map(struct macb *bp,
if (i == queue->tx_head) {
ctrl |= MACB_BF(TX_LSO, lso_ctrl);
ctrl |= MACB_BF(TX_TCP_SEQ_SRC, seq_ctrl);
- if ((bp->dev->features & NETIF_F_HW_CSUM) &&
+ if ((bp->netdev->features & NETIF_F_HW_CSUM) &&
skb->ip_summed != CHECKSUM_PARTIAL && !lso_ctrl &&
!ptp_one_step_sync(skb))
ctrl |= MACB_BIT(TX_NOCRC);
@@ -2352,7 +2358,7 @@ static unsigned int macb_tx_map(struct macb *bp,
return 0;
dma_error:
- netdev_err(bp->dev, "TX DMA map failed\n");
+ netdev_err(bp->netdev, "TX DMA map failed\n");
for (i = queue->tx_head; i != tx_head; i++) {
tx_skb = macb_tx_skb(queue, i);
@@ -2364,7 +2370,7 @@ static unsigned int macb_tx_map(struct macb *bp,
}
static netdev_features_t macb_features_check(struct sk_buff *skb,
- struct net_device *dev,
+ struct net_device *netdev,
netdev_features_t features)
{
unsigned int nr_frags, f;
@@ -2416,7 +2422,8 @@ static inline int macb_clear_csum(struct sk_buff *skb)
return 0;
}
-static int macb_pad_and_fcs(struct sk_buff **skb, struct net_device *ndev)
+/* Returns a negative errno, or the FCS bytes appended (0 or ETH_FCS_LEN). */
+static int macb_pad_and_fcs(struct sk_buff **skb, struct net_device *netdev)
{
bool cloned = skb_cloned(*skb) || skb_header_cloned(*skb) ||
skb_is_nonlinear(*skb);
@@ -2425,7 +2432,7 @@ static int macb_pad_and_fcs(struct sk_buff **skb, struct net_device *ndev)
struct sk_buff *nskb;
u32 fcs;
- if (!(ndev->features & NETIF_F_HW_CSUM) ||
+ if (!(netdev->features & NETIF_F_HW_CSUM) ||
!((*skb)->ip_summed != CHECKSUM_PARTIAL) ||
skb_shinfo(*skb)->gso_size || ptp_one_step_sync(*skb))
return 0;
@@ -2464,17 +2471,19 @@ static int macb_pad_and_fcs(struct sk_buff **skb, struct net_device *ndev)
skb_put_u8(*skb, (fcs >> 16) & 0xff);
skb_put_u8(*skb, (fcs >> 24) & 0xff);
- return 0;
+ return ETH_FCS_LEN;
}
-static netdev_tx_t macb_start_xmit(struct sk_buff *skb, struct net_device *dev)
+static netdev_tx_t macb_start_xmit(struct sk_buff *skb,
+ struct net_device *netdev)
{
u16 queue_index = skb_get_queue_mapping(skb);
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
struct macb_queue *queue = &bp->queues[queue_index];
unsigned int desc_cnt, nr_frags, frag_size, f;
unsigned int hdrlen;
unsigned long flags;
+ int fcs_len;
bool is_lso;
netdev_tx_t ret = NETDEV_TX_OK;
@@ -2483,7 +2492,8 @@ static netdev_tx_t macb_start_xmit(struct sk_buff *skb, struct net_device *dev)
return ret;
}
- if (macb_pad_and_fcs(&skb, dev)) {
+ fcs_len = macb_pad_and_fcs(&skb, netdev);
+ if (fcs_len < 0) {
dev_kfree_skb_any(skb);
return ret;
}
@@ -2502,7 +2512,7 @@ static netdev_tx_t macb_start_xmit(struct sk_buff *skb, struct net_device *dev)
else
hdrlen = skb_tcp_all_headers(skb);
if (skb_headlen(skb) < hdrlen) {
- netdev_err(bp->dev, "Error - LSO headers fragmented!!!\n");
+ netdev_err(bp->netdev, "Error - LSO headers fragmented!!!\n");
/* if this is required, would need to copy to single buffer */
return NETDEV_TX_BUSY;
}
@@ -2510,7 +2520,7 @@ static netdev_tx_t macb_start_xmit(struct sk_buff *skb, struct net_device *dev)
hdrlen = umin(skb_headlen(skb), bp->max_tx_length);
#if defined(DEBUG) && defined(VERBOSE_DEBUG)
- netdev_vdbg(bp->dev,
+ netdev_vdbg(bp->netdev,
"start_xmit: queue %hu len %u head %p data %p tail %p end %p\n",
queue_index, skb->len, skb->head, skb->data,
skb_tail_pointer(skb), skb_end_pointer(skb));
@@ -2538,15 +2548,15 @@ static netdev_tx_t macb_start_xmit(struct sk_buff *skb, struct net_device *dev)
/* This is a hard error, log it. */
if (CIRC_SPACE(queue->tx_head, queue->tx_tail,
bp->tx_ring_size) < desc_cnt) {
- netif_stop_subqueue(dev, queue_index);
- netdev_dbg(bp->dev, "tx_head = %u, tx_tail = %u\n",
+ netif_stop_subqueue(netdev, queue_index);
+ netdev_dbg(netdev, "tx_head = %u, tx_tail = %u\n",
queue->tx_head, queue->tx_tail);
ret = NETDEV_TX_BUSY;
goto unlock;
}
/* Map socket buffer for DMA transfer */
- if (macb_tx_map(bp, queue, skb, hdrlen)) {
+ if (macb_tx_map(bp, queue, skb, hdrlen, fcs_len)) {
dev_kfree_skb_any(skb);
goto unlock;
}
@@ -2554,7 +2564,7 @@ static netdev_tx_t macb_start_xmit(struct sk_buff *skb, struct net_device *dev)
/* Make newly initialized descriptor visible to hardware */
wmb();
skb_tx_timestamp(skb);
- netdev_tx_sent_queue(netdev_get_tx_queue(bp->dev, queue_index),
+ netdev_tx_sent_queue(netdev_get_tx_queue(bp->netdev, queue_index),
skb->len);
spin_lock(&bp->lock);
@@ -2563,7 +2573,7 @@ static netdev_tx_t macb_start_xmit(struct sk_buff *skb, struct net_device *dev)
spin_unlock(&bp->lock);
if (CIRC_SPACE(queue->tx_head, queue->tx_tail, bp->tx_ring_size) < 1)
- netif_stop_subqueue(dev, queue_index);
+ netif_stop_subqueue(netdev, queue_index);
unlock:
spin_unlock_irqrestore(&queue->tx_ptr_lock, flags);
@@ -2579,7 +2589,7 @@ static void macb_init_rx_buffer_size(struct macb *bp, size_t size)
bp->rx_buffer_size = MIN(size, RX_BUFFER_MAX);
if (bp->rx_buffer_size % RX_BUFFER_MULTIPLE) {
- netdev_dbg(bp->dev,
+ netdev_dbg(bp->netdev,
"RX buffer must be multiple of %d bytes, expanding\n",
RX_BUFFER_MULTIPLE);
bp->rx_buffer_size =
@@ -2587,8 +2597,8 @@ static void macb_init_rx_buffer_size(struct macb *bp, size_t size)
}
}
- netdev_dbg(bp->dev, "mtu [%u] rx_buffer_size [%zu]\n",
- bp->dev->mtu, bp->rx_buffer_size);
+ netdev_dbg(bp->netdev, "mtu [%u] rx_buffer_size [%zu]\n",
+ bp->netdev->mtu, bp->rx_buffer_size);
}
static void gem_free_rx_buffers(struct macb *bp)
@@ -2679,7 +2689,7 @@ static void macb_free_consistent(struct macb *bp)
}
queue->stats.tx_dropped += dropped;
- bp->dev->stats.tx_dropped += dropped;
+ bp->netdev->stats.tx_dropped += dropped;
kfree(queue->tx_skb);
queue->tx_skb = NULL;
@@ -2704,7 +2714,7 @@ static int gem_alloc_rx_buffers(struct macb *bp)
if (!queue->rx_skbuff)
return -ENOMEM;
else
- netdev_dbg(bp->dev,
+ netdev_dbg(bp->netdev,
"Allocated %d RX struct sk_buff entries at %p\n",
bp->rx_ring_size, queue->rx_skbuff);
}
@@ -2722,7 +2732,7 @@ static int macb_alloc_rx_buffers(struct macb *bp)
if (!queue->rx_buffers)
return -ENOMEM;
- netdev_dbg(bp->dev,
+ netdev_dbg(bp->netdev,
"Allocated RX buffers of %d bytes at %08lx (mapped %p)\n",
size, (unsigned long)queue->rx_buffers_dma, queue->rx_buffers);
return 0;
@@ -2748,14 +2758,14 @@ static int macb_alloc_consistent(struct macb *bp)
tx = dma_alloc_coherent(dev, size, &tx_dma, GFP_KERNEL);
if (!tx || upper_32_bits(tx_dma) != upper_32_bits(tx_dma + size - 1))
goto out_err;
- netdev_dbg(bp->dev, "Allocated %zu bytes for %u TX rings at %08lx (mapped %p)\n",
+ netdev_dbg(bp->netdev, "Allocated %zu bytes for %u TX rings at %08lx (mapped %p)\n",
size, bp->num_queues, (unsigned long)tx_dma, tx);
size = bp->num_queues * macb_rx_ring_size_per_queue(bp);
rx = dma_alloc_coherent(dev, size, &rx_dma, GFP_KERNEL);
if (!rx || upper_32_bits(rx_dma) != upper_32_bits(rx_dma + size - 1))
goto out_err;
- netdev_dbg(bp->dev, "Allocated %zu bytes for %u RX rings at %08lx (mapped %p)\n",
+ netdev_dbg(bp->netdev, "Allocated %zu bytes for %u RX rings at %08lx (mapped %p)\n",
size, bp->num_queues, (unsigned long)rx_dma, rx);
for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
@@ -2983,7 +2993,7 @@ static void macb_configure_dma(struct macb *bp)
else
dmacfg |= GEM_BIT(ENDIA_DESC); /* CPU in big endian */
- if (bp->dev->features & NETIF_F_HW_CSUM)
+ if (bp->netdev->features & NETIF_F_HW_CSUM)
dmacfg |= GEM_BIT(TXCOEN);
else
dmacfg &= ~GEM_BIT(TXCOEN);
@@ -2993,7 +3003,7 @@ static void macb_configure_dma(struct macb *bp)
dmacfg |= GEM_BIT(ADDR64);
if (macb_dma_ptp(bp))
dmacfg |= GEM_BIT(RXEXT) | GEM_BIT(TXEXT);
- netdev_dbg(bp->dev, "Cadence configure DMA with 0x%08x\n",
+ netdev_dbg(bp->netdev, "Cadence configure DMA with 0x%08x\n",
dmacfg);
gem_writel(bp, DMACFG, dmacfg);
}
@@ -3017,11 +3027,11 @@ static void macb_init_hw(struct macb *bp)
config |= MACB_BIT(JFRAME); /* Enable jumbo frames */
else
config |= MACB_BIT(BIG); /* Receive oversized frames */
- if (bp->dev->flags & IFF_PROMISC)
+ if (bp->netdev->flags & IFF_PROMISC)
config |= MACB_BIT(CAF); /* Copy All Frames */
- else if (macb_is_gem(bp) && bp->dev->features & NETIF_F_RXCSUM)
+ else if (macb_is_gem(bp) && bp->netdev->features & NETIF_F_RXCSUM)
config |= GEM_BIT(RXCOEN);
- if (!(bp->dev->flags & IFF_BROADCAST))
+ if (!(bp->netdev->flags & IFF_BROADCAST))
config |= MACB_BIT(NBC); /* No BroadCast */
config |= macb_dbw(bp);
macb_writel(bp, NCFGR, config);
@@ -3095,17 +3105,17 @@ static int hash_get_index(__u8 *addr)
}
/* Add multicast addresses to the internal multicast-hash table. */
-static void macb_sethashtable(struct net_device *dev)
+static void macb_sethashtable(struct net_device *netdev)
{
struct netdev_hw_addr *ha;
unsigned long mc_filter[2];
unsigned int bitnr;
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
mc_filter[0] = 0;
mc_filter[1] = 0;
- netdev_for_each_mc_addr(ha, dev) {
+ netdev_for_each_mc_addr(ha, netdev) {
bitnr = hash_get_index(ha->addr);
mc_filter[bitnr >> 5] |= 1 << (bitnr & 31);
}
@@ -3115,14 +3125,14 @@ static void macb_sethashtable(struct net_device *dev)
}
/* Enable/Disable promiscuous and multicast modes. */
-static void macb_set_rx_mode(struct net_device *dev)
+static void macb_set_rx_mode(struct net_device *netdev)
{
unsigned long cfg;
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
cfg = macb_readl(bp, NCFGR);
- if (dev->flags & IFF_PROMISC) {
+ if (netdev->flags & IFF_PROMISC) {
/* Enable promiscuous mode */
cfg |= MACB_BIT(CAF);
@@ -3134,20 +3144,20 @@ static void macb_set_rx_mode(struct net_device *dev)
cfg &= ~MACB_BIT(CAF);
/* Enable RX checksum offload only if requested */
- if (macb_is_gem(bp) && dev->features & NETIF_F_RXCSUM)
+ if (macb_is_gem(bp) && netdev->features & NETIF_F_RXCSUM)
cfg |= GEM_BIT(RXCOEN);
}
- if (dev->flags & IFF_ALLMULTI) {
+ if (netdev->flags & IFF_ALLMULTI) {
/* Enable all multicast mode */
macb_or_gem_writel(bp, HRB, -1);
macb_or_gem_writel(bp, HRT, -1);
cfg |= MACB_BIT(NCFGR_MTI);
- } else if (!netdev_mc_empty(dev)) {
+ } else if (!netdev_mc_empty(netdev)) {
/* Enable specific multicasts */
- macb_sethashtable(dev);
+ macb_sethashtable(netdev);
cfg |= MACB_BIT(NCFGR_MTI);
- } else if (dev->flags & (~IFF_ALLMULTI)) {
+ } else if (netdev->flags & (~IFF_ALLMULTI)) {
/* Disable all multicast mode */
macb_or_gem_writel(bp, HRB, 0);
macb_or_gem_writel(bp, HRT, 0);
@@ -3157,15 +3167,15 @@ static void macb_set_rx_mode(struct net_device *dev)
macb_writel(bp, NCFGR, cfg);
}
-static int macb_open(struct net_device *dev)
+static int macb_open(struct net_device *netdev)
{
- size_t bufsz = dev->mtu + ETH_HLEN + ETH_FCS_LEN + NET_IP_ALIGN;
- struct macb *bp = netdev_priv(dev);
+ size_t bufsz = netdev->mtu + ETH_HLEN + ETH_FCS_LEN + NET_IP_ALIGN;
+ struct macb *bp = netdev_priv(netdev);
struct macb_queue *queue;
unsigned int q;
int err;
- netdev_dbg(bp->dev, "open\n");
+ netdev_dbg(bp->netdev, "open\n");
err = pm_runtime_resume_and_get(&bp->pdev->dev);
if (err < 0)
@@ -3176,7 +3186,7 @@ static int macb_open(struct net_device *dev)
err = macb_alloc_consistent(bp);
if (err) {
- netdev_err(dev, "Unable to allocate DMA memory (error %d)\n",
+ netdev_err(netdev, "Unable to allocate DMA memory (error %d)\n",
err);
goto pm_exit;
}
@@ -3203,10 +3213,10 @@ static int macb_open(struct net_device *dev)
if (err)
goto phy_off;
- netif_tx_start_all_queues(dev);
+ netif_tx_start_all_queues(netdev);
if (bp->ptp_info)
- bp->ptp_info->ptp_init(dev);
+ bp->ptp_info->ptp_init(netdev);
return 0;
@@ -3225,19 +3235,19 @@ static int macb_open(struct net_device *dev)
return err;
}
-static int macb_close(struct net_device *dev)
+static int macb_close(struct net_device *netdev)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
struct macb_queue *queue;
unsigned long flags;
unsigned int q;
- netif_tx_stop_all_queues(dev);
+ netif_tx_stop_all_queues(netdev);
for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
napi_disable(&queue->napi_rx);
napi_disable(&queue->napi_tx);
- netdev_tx_reset_queue(netdev_get_tx_queue(dev, q));
+ netdev_tx_reset_queue(netdev_get_tx_queue(netdev, q));
}
cancel_delayed_work_sync(&bp->tx_lpi_work);
@@ -3249,38 +3259,38 @@ static int macb_close(struct net_device *dev)
spin_lock_irqsave(&bp->lock, flags);
macb_reset_hw(bp);
- netif_carrier_off(dev);
+ netif_carrier_off(netdev);
spin_unlock_irqrestore(&bp->lock, flags);
macb_free_consistent(bp);
if (bp->ptp_info)
- bp->ptp_info->ptp_remove(dev);
+ bp->ptp_info->ptp_remove(netdev);
pm_runtime_put(&bp->pdev->dev);
return 0;
}
-static int macb_change_mtu(struct net_device *dev, int new_mtu)
+static int macb_change_mtu(struct net_device *netdev, int new_mtu)
{
- if (netif_running(dev))
+ if (netif_running(netdev))
return -EBUSY;
- WRITE_ONCE(dev->mtu, new_mtu);
+ WRITE_ONCE(netdev->mtu, new_mtu);
return 0;
}
-static int macb_set_mac_addr(struct net_device *dev, void *addr)
+static int macb_set_mac_addr(struct net_device *netdev, void *addr)
{
int err;
- err = eth_mac_addr(dev, addr);
+ err = eth_mac_addr(netdev, addr);
if (err < 0)
return err;
- macb_set_hwaddr(netdev_priv(dev));
+ macb_set_hwaddr(netdev_priv(netdev));
return 0;
}
@@ -3318,7 +3328,7 @@ static void gem_get_stats(struct macb *bp, struct rtnl_link_stats64 *nstat)
struct gem_stats *hwstat = &bp->hw_stats.gem;
spin_lock_irq(&bp->stats_lock);
- if (netif_running(bp->dev))
+ if (netif_running(bp->netdev))
gem_update_stats(bp);
nstat->rx_errors = (hwstat->rx_frame_check_sequence_errors +
@@ -3351,10 +3361,10 @@ static void gem_get_stats(struct macb *bp, struct rtnl_link_stats64 *nstat)
spin_unlock_irq(&bp->stats_lock);
}
-static void gem_get_ethtool_stats(struct net_device *dev,
+static void gem_get_ethtool_stats(struct net_device *netdev,
struct ethtool_stats *stats, u64 *data)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
spin_lock_irq(&bp->stats_lock);
gem_update_stats(bp);
@@ -3363,9 +3373,9 @@ static void gem_get_ethtool_stats(struct net_device *dev,
spin_unlock_irq(&bp->stats_lock);
}
-static int gem_get_sset_count(struct net_device *dev, int sset)
+static int gem_get_sset_count(struct net_device *netdev, int sset)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
switch (sset) {
case ETH_SS_STATS:
@@ -3375,9 +3385,9 @@ static int gem_get_sset_count(struct net_device *dev, int sset)
}
}
-static void gem_get_ethtool_strings(struct net_device *dev, u32 sset, u8 *p)
+static void gem_get_ethtool_strings(struct net_device *netdev, u32 sset, u8 *p)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
struct macb_queue *queue;
unsigned int i;
unsigned int q;
@@ -3396,13 +3406,13 @@ static void gem_get_ethtool_strings(struct net_device *dev, u32 sset, u8 *p)
}
}
-static void macb_get_stats(struct net_device *dev,
+static void macb_get_stats(struct net_device *netdev,
struct rtnl_link_stats64 *nstat)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
struct macb_stats *hwstat = &bp->hw_stats.macb;
- netdev_stats_to_stats64(nstat, &bp->dev->stats);
+ netdev_stats_to_stats64(nstat, &bp->netdev->stats);
if (macb_is_gem(bp)) {
gem_get_stats(bp, nstat);
return;
@@ -3446,10 +3456,10 @@ static void macb_get_stats(struct net_device *dev,
spin_unlock_irq(&bp->stats_lock);
}
-static void macb_get_pause_stats(struct net_device *dev,
+static void macb_get_pause_stats(struct net_device *netdev,
struct ethtool_pause_stats *pause_stats)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
struct macb_stats *hwstat = &bp->hw_stats.macb;
spin_lock_irq(&bp->stats_lock);
@@ -3459,10 +3469,10 @@ static void macb_get_pause_stats(struct net_device *dev,
spin_unlock_irq(&bp->stats_lock);
}
-static void gem_get_pause_stats(struct net_device *dev,
+static void gem_get_pause_stats(struct net_device *netdev,
struct ethtool_pause_stats *pause_stats)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
struct gem_stats *hwstat = &bp->hw_stats.gem;
spin_lock_irq(&bp->stats_lock);
@@ -3472,10 +3482,10 @@ static void gem_get_pause_stats(struct net_device *dev,
spin_unlock_irq(&bp->stats_lock);
}
-static void macb_get_eth_mac_stats(struct net_device *dev,
+static void macb_get_eth_mac_stats(struct net_device *netdev,
struct ethtool_eth_mac_stats *mac_stats)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
struct macb_stats *hwstat = &bp->hw_stats.macb;
spin_lock_irq(&bp->stats_lock);
@@ -3497,10 +3507,10 @@ static void macb_get_eth_mac_stats(struct net_device *dev,
spin_unlock_irq(&bp->stats_lock);
}
-static void gem_get_eth_mac_stats(struct net_device *dev,
+static void gem_get_eth_mac_stats(struct net_device *netdev,
struct ethtool_eth_mac_stats *mac_stats)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
struct gem_stats *hwstat = &bp->hw_stats.gem;
spin_lock_irq(&bp->stats_lock);
@@ -3530,10 +3540,10 @@ static void gem_get_eth_mac_stats(struct net_device *dev,
}
/* TODO: Report SQE test errors when added to phy_stats */
-static void macb_get_eth_phy_stats(struct net_device *dev,
+static void macb_get_eth_phy_stats(struct net_device *netdev,
struct ethtool_eth_phy_stats *phy_stats)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
struct macb_stats *hwstat = &bp->hw_stats.macb;
spin_lock_irq(&bp->stats_lock);
@@ -3542,10 +3552,10 @@ static void macb_get_eth_phy_stats(struct net_device *dev,
spin_unlock_irq(&bp->stats_lock);
}
-static void gem_get_eth_phy_stats(struct net_device *dev,
+static void gem_get_eth_phy_stats(struct net_device *netdev,
struct ethtool_eth_phy_stats *phy_stats)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
struct gem_stats *hwstat = &bp->hw_stats.gem;
spin_lock_irq(&bp->stats_lock);
@@ -3554,11 +3564,11 @@ static void gem_get_eth_phy_stats(struct net_device *dev,
spin_unlock_irq(&bp->stats_lock);
}
-static void macb_get_rmon_stats(struct net_device *dev,
+static void macb_get_rmon_stats(struct net_device *netdev,
struct ethtool_rmon_stats *rmon_stats,
const struct ethtool_rmon_hist_range **ranges)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
struct macb_stats *hwstat = &bp->hw_stats.macb;
spin_lock_irq(&bp->stats_lock);
@@ -3580,11 +3590,11 @@ static const struct ethtool_rmon_hist_range gem_rmon_ranges[] = {
{ },
};
-static void gem_get_rmon_stats(struct net_device *dev,
+static void gem_get_rmon_stats(struct net_device *netdev,
struct ethtool_rmon_stats *rmon_stats,
const struct ethtool_rmon_hist_range **ranges)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
struct gem_stats *hwstat = &bp->hw_stats.gem;
spin_lock_irq(&bp->stats_lock);
@@ -3615,10 +3625,10 @@ static int macb_get_regs_len(struct net_device *netdev)
return MACB_GREGS_NBR * sizeof(u32);
}
-static void macb_get_regs(struct net_device *dev, struct ethtool_regs *regs,
+static void macb_get_regs(struct net_device *netdev, struct ethtool_regs *regs,
void *p)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
unsigned int tail, head;
u32 *regs_buff = p;
@@ -3735,16 +3745,16 @@ static int macb_set_ringparam(struct net_device *netdev,
return 0;
}
- if (netif_running(bp->dev)) {
+ if (netif_running(bp->netdev)) {
reset = 1;
- macb_close(bp->dev);
+ macb_close(bp->netdev);
}
bp->rx_ring_size = new_rx_size;
bp->tx_ring_size = new_tx_size;
if (reset)
- macb_open(bp->dev);
+ macb_open(bp->netdev);
return 0;
}
@@ -3771,13 +3781,13 @@ static s32 gem_get_ptp_max_adj(void)
return 64000000;
}
-static int gem_get_ts_info(struct net_device *dev,
+static int gem_get_ts_info(struct net_device *netdev,
struct kernel_ethtool_ts_info *info)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
if (!macb_dma_ptp(bp)) {
- ethtool_op_get_ts_info(dev, info);
+ ethtool_op_get_ts_info(netdev, info);
return 0;
}
@@ -3824,7 +3834,7 @@ static int macb_get_ts_info(struct net_device *netdev,
static void gem_enable_flow_filters(struct macb *bp, bool enable)
{
- struct net_device *netdev = bp->dev;
+ struct net_device *netdev = bp->netdev;
struct ethtool_rx_fs_item *item;
u32 t2_scr;
int num_t2_scr;
@@ -4154,16 +4164,16 @@ static const struct ethtool_ops macb_ethtool_ops = {
.set_ringparam = macb_set_ringparam,
};
-static int macb_get_eee(struct net_device *dev, struct ethtool_keee *eee)
+static int macb_get_eee(struct net_device *netdev, struct ethtool_keee *eee)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
return phylink_ethtool_get_eee(bp->phylink, eee);
}
-static int macb_set_eee(struct net_device *dev, struct ethtool_keee *eee)
+static int macb_set_eee(struct net_device *netdev, struct ethtool_keee *eee)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
return phylink_ethtool_set_eee(bp->phylink, eee);
}
@@ -4194,43 +4204,43 @@ static const struct ethtool_ops gem_ethtool_ops = {
.set_eee = macb_set_eee,
};
-static int macb_ioctl(struct net_device *dev, struct ifreq *rq, int cmd)
+static int macb_ioctl(struct net_device *netdev, struct ifreq *rq, int cmd)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
- if (!netif_running(dev))
+ if (!netif_running(netdev))
return -EINVAL;
return phylink_mii_ioctl(bp->phylink, rq, cmd);
}
-static int macb_hwtstamp_get(struct net_device *dev,
+static int macb_hwtstamp_get(struct net_device *netdev,
struct kernel_hwtstamp_config *cfg)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
- if (!netif_running(dev))
+ if (!netif_running(netdev))
return -EINVAL;
if (!bp->ptp_info)
return -EOPNOTSUPP;
- return bp->ptp_info->get_hwtst(dev, cfg);
+ return bp->ptp_info->get_hwtst(netdev, cfg);
}
-static int macb_hwtstamp_set(struct net_device *dev,
+static int macb_hwtstamp_set(struct net_device *netdev,
struct kernel_hwtstamp_config *cfg,
struct netlink_ext_ack *extack)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
- if (!netif_running(dev))
+ if (!netif_running(netdev))
return -EINVAL;
if (!bp->ptp_info)
return -EOPNOTSUPP;
- return bp->ptp_info->set_hwtst(dev, cfg, extack);
+ return bp->ptp_info->set_hwtst(netdev, cfg, extack);
}
static inline void macb_set_txcsum_feature(struct macb *bp,
@@ -4253,7 +4263,7 @@ static inline void macb_set_txcsum_feature(struct macb *bp,
static inline void macb_set_rxcsum_feature(struct macb *bp,
netdev_features_t features)
{
- struct net_device *netdev = bp->dev;
+ struct net_device *netdev = bp->netdev;
u32 val;
if (!macb_is_gem(bp))
@@ -4300,7 +4310,7 @@ static int macb_set_features(struct net_device *netdev,
static void macb_restore_features(struct macb *bp)
{
- struct net_device *netdev = bp->dev;
+ struct net_device *netdev = bp->netdev;
netdev_features_t features = netdev->features;
struct ethtool_rx_fs_item *item;
@@ -4317,15 +4327,15 @@ static void macb_restore_features(struct macb *bp)
macb_set_rxflow_feature(bp, features);
}
-static int macb_taprio_setup_replace(struct net_device *ndev,
+static int macb_taprio_setup_replace(struct net_device *netdev,
struct tc_taprio_qopt_offload *conf)
{
u64 total_on_time = 0, start_time_sec = 0, start_time = conf->base_time;
u32 configured_queues = 0, speed = 0, start_time_nsec;
struct macb_queue_enst_config *enst_queue;
+ struct ethtool_link_ksettings kset = {};
+ struct macb *bp = netdev_priv(netdev);
struct tc_taprio_sched_entry *entry;
- struct macb *bp = netdev_priv(ndev);
- struct ethtool_link_ksettings kset;
struct macb_queue *queue;
u32 queue_mask;
u8 queue_id;
@@ -4333,13 +4343,13 @@ static int macb_taprio_setup_replace(struct net_device *ndev,
int err;
if (conf->num_entries > bp->num_queues) {
- netdev_err(ndev, "Too many TAPRIO entries: %zu > %d queues\n",
+ netdev_err(netdev, "Too many TAPRIO entries: %zu > %d queues\n",
conf->num_entries, bp->num_queues);
return -EINVAL;
}
if (conf->base_time < 0) {
- netdev_err(ndev, "Invalid base_time: must be 0 or positive, got %lld\n",
+ netdev_err(netdev, "Invalid base_time: must be 0 or positive, got %lld\n",
conf->base_time);
return -ERANGE;
}
@@ -4347,13 +4357,13 @@ static int macb_taprio_setup_replace(struct net_device *ndev,
/* Get the current link speed */
err = phylink_ethtool_ksettings_get(bp->phylink, &kset);
if (unlikely(err)) {
- netdev_err(ndev, "Failed to get link settings: %d\n", err);
+ netdev_err(netdev, "Failed to get link settings: %d\n", err);
return err;
}
speed = kset.base.speed;
- if (unlikely(speed <= 0)) {
- netdev_err(ndev, "Invalid speed: %d\n", speed);
+ if (unlikely(speed == SPEED_UNKNOWN || !speed)) {
+ netdev_err(netdev, "Invalid speed %d, link-down?\n", speed);
return -EINVAL;
}
@@ -4366,7 +4376,7 @@ static int macb_taprio_setup_replace(struct net_device *ndev,
entry = &conf->entries[i];
if (entry->command != TC_TAPRIO_CMD_SET_GATES) {
- netdev_err(ndev, "Entry %zu: unsupported command %d\n",
+ netdev_err(netdev, "Entry %zu: unsupported command %d\n",
i, entry->command);
err = -EOPNOTSUPP;
goto cleanup;
@@ -4374,7 +4384,7 @@ static int macb_taprio_setup_replace(struct net_device *ndev,
/* Validate gate_mask: must be nonzero, single queue, and within range */
if (!is_power_of_2(entry->gate_mask)) {
- netdev_err(ndev, "Entry %zu: gate_mask 0x%x is not a power of 2 (only one queue per entry allowed)\n",
+ netdev_err(netdev, "Entry %zu: gate_mask 0x%x is not a power of 2 (only one queue per entry allowed)\n",
i, entry->gate_mask);
err = -EINVAL;
goto cleanup;
@@ -4383,7 +4393,7 @@ static int macb_taprio_setup_replace(struct net_device *ndev,
/* gate_mask must not select queues outside the valid queues */
queue_id = order_base_2(entry->gate_mask);
if (queue_id >= bp->num_queues) {
- netdev_err(ndev, "Entry %zu: gate_mask 0x%x exceeds queue range (max_queues=%d)\n",
+ netdev_err(netdev, "Entry %zu: gate_mask 0x%x exceeds queue range (max_queues=%d)\n",
i, entry->gate_mask, bp->num_queues);
err = -EINVAL;
goto cleanup;
@@ -4393,7 +4403,7 @@ static int macb_taprio_setup_replace(struct net_device *ndev,
start_time_sec = start_time;
start_time_nsec = do_div(start_time_sec, NSEC_PER_SEC);
if (start_time_sec > GENMASK(GEM_START_TIME_SEC_SIZE - 1, 0)) {
- netdev_err(ndev, "Entry %zu: Start time %llu s exceeds hardware limit\n",
+ netdev_err(netdev, "Entry %zu: Start time %llu s exceeds hardware limit\n",
i, start_time_sec);
err = -ERANGE;
goto cleanup;
@@ -4401,7 +4411,7 @@ static int macb_taprio_setup_replace(struct net_device *ndev,
/* Check for on time limit */
if (entry->interval > enst_max_hw_interval(speed)) {
- netdev_err(ndev, "Entry %zu: interval %u ns exceeds hardware limit %llu ns\n",
+ netdev_err(netdev, "Entry %zu: interval %u ns exceeds hardware limit %llu ns\n",
i, entry->interval, enst_max_hw_interval(speed));
err = -ERANGE;
goto cleanup;
@@ -4409,7 +4419,7 @@ static int macb_taprio_setup_replace(struct net_device *ndev,
/* Check for off time limit*/
if ((conf->cycle_time - entry->interval) > enst_max_hw_interval(speed)) {
- netdev_err(ndev, "Entry %zu: off_time %llu ns exceeds hardware limit %llu ns\n",
+ netdev_err(netdev, "Entry %zu: off_time %llu ns exceeds hardware limit %llu ns\n",
i, conf->cycle_time - entry->interval,
enst_max_hw_interval(speed));
err = -ERANGE;
@@ -4432,13 +4442,13 @@ static int macb_taprio_setup_replace(struct net_device *ndev,
/* Check total interval doesn't exceed cycle time */
if (total_on_time > conf->cycle_time) {
- netdev_err(ndev, "Total ON %llu ns exceeds cycle time %llu ns\n",
+ netdev_err(netdev, "Total ON %llu ns exceeds cycle time %llu ns\n",
total_on_time, conf->cycle_time);
err = -EINVAL;
goto cleanup;
}
- netdev_dbg(ndev, "TAPRIO setup: %zu entries, base_time=%lld ns, cycle_time=%llu ns\n",
+ netdev_dbg(netdev, "TAPRIO setup: %zu entries, base_time=%lld ns, cycle_time=%llu ns\n",
conf->num_entries, conf->base_time, conf->cycle_time);
/* All validations passed - proceed with hardware configuration */
@@ -4463,7 +4473,7 @@ static int macb_taprio_setup_replace(struct net_device *ndev,
gem_writel(bp, ENST_CONTROL, configured_queues);
}
- netdev_info(ndev, "TAPRIO configuration completed successfully: %zu entries, %d queues configured\n",
+ netdev_info(netdev, "TAPRIO configuration completed successfully: %zu entries, %d queues configured\n",
conf->num_entries, hweight32(configured_queues));
cleanup:
@@ -4471,14 +4481,14 @@ static int macb_taprio_setup_replace(struct net_device *ndev,
return err;
}
-static void macb_taprio_destroy(struct net_device *ndev)
+static void macb_taprio_destroy(struct net_device *netdev)
{
- struct macb *bp = netdev_priv(ndev);
+ struct macb *bp = netdev_priv(netdev);
struct macb_queue *queue;
u32 queue_mask;
unsigned int q;
- netdev_reset_tc(ndev);
+ netdev_reset_tc(netdev);
queue_mask = BIT_U32(bp->num_queues) - 1;
scoped_guard(spinlock_irqsave, &bp->lock) {
@@ -4493,30 +4503,30 @@ static void macb_taprio_destroy(struct net_device *ndev)
queue_writel(queue, ENST_OFF_TIME, 0);
}
}
- netdev_info(ndev, "TAPRIO destroy: All gates disabled\n");
+ netdev_info(netdev, "TAPRIO destroy: All gates disabled\n");
}
-static int macb_setup_taprio(struct net_device *ndev,
+static int macb_setup_taprio(struct net_device *netdev,
struct tc_taprio_qopt_offload *taprio)
{
- struct macb *bp = netdev_priv(ndev);
+ struct macb *bp = netdev_priv(netdev);
int err = 0;
- if (unlikely(!(ndev->hw_features & NETIF_F_HW_TC)))
+ if (unlikely(!(netdev->hw_features & NETIF_F_HW_TC)))
return -EOPNOTSUPP;
/* Check if Device is in runtime suspend */
if (unlikely(pm_runtime_suspended(&bp->pdev->dev))) {
- netdev_err(ndev, "Device is in runtime suspend\n");
+ netdev_err(netdev, "Device is in runtime suspend\n");
return -EOPNOTSUPP;
}
switch (taprio->cmd) {
case TAPRIO_CMD_REPLACE:
- err = macb_taprio_setup_replace(ndev, taprio);
+ err = macb_taprio_setup_replace(netdev, taprio);
break;
case TAPRIO_CMD_DESTROY:
- macb_taprio_destroy(ndev);
+ macb_taprio_destroy(netdev);
break;
default:
err = -EOPNOTSUPP;
@@ -4525,23 +4535,23 @@ static int macb_setup_taprio(struct net_device *ndev,
return err;
}
-static int macb_setup_tc(struct net_device *dev, enum tc_setup_type type,
+static int macb_setup_tc(struct net_device *netdev, enum tc_setup_type type,
void *type_data)
{
- if (!dev || !type_data)
+ if (!netdev || !type_data)
return -EINVAL;
switch (type) {
case TC_SETUP_QDISC_TAPRIO:
- return macb_setup_taprio(dev, type_data);
+ return macb_setup_taprio(netdev, type_data);
default:
return -EOPNOTSUPP;
}
}
-static void macb_tx_timeout(struct net_device *dev, unsigned int q)
+static void macb_tx_timeout(struct net_device *netdev, unsigned int q)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
macb_tx_restart(&bp->queues[q]);
}
@@ -4749,9 +4759,9 @@ static int macb_clk_init(struct platform_device *pdev, struct clk **pclk,
static int macb_init_dflt(struct platform_device *pdev)
{
- struct net_device *dev = platform_get_drvdata(pdev);
+ struct net_device *netdev = platform_get_drvdata(pdev);
unsigned int hw_q, q;
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
struct macb_queue *queue;
int err;
u32 val, reg;
@@ -4767,8 +4777,8 @@ static int macb_init_dflt(struct platform_device *pdev)
queue = &bp->queues[q];
queue->bp = bp;
spin_lock_init(&queue->tx_ptr_lock);
- netif_napi_add(dev, &queue->napi_rx, macb_rx_poll);
- netif_napi_add_tx(dev, &queue->napi_tx, macb_tx_poll);
+ netif_napi_add(netdev, &queue->napi_rx, macb_rx_poll);
+ netif_napi_add_tx(netdev, &queue->napi_tx, macb_tx_poll);
if (hw_q) {
queue->ISR = GEM_ISR(hw_q - 1);
queue->IER = GEM_IER(hw_q - 1);
@@ -4798,7 +4808,7 @@ static int macb_init_dflt(struct platform_device *pdev)
*/
queue->irq = platform_get_irq(pdev, q);
err = devm_request_irq(&pdev->dev, queue->irq, macb_interrupt,
- IRQF_SHARED, dev->name, queue);
+ IRQF_SHARED, netdev->name, queue);
if (err) {
dev_err(&pdev->dev,
"Unable to request IRQ %d (error %d)\n",
@@ -4810,7 +4820,7 @@ static int macb_init_dflt(struct platform_device *pdev)
q++;
}
- dev->netdev_ops = &macb_netdev_ops;
+ netdev->netdev_ops = &macb_netdev_ops;
/* setup appropriated routines according to adapter type */
if (macb_is_gem(bp)) {
@@ -4818,39 +4828,39 @@ static int macb_init_dflt(struct platform_device *pdev)
bp->macbgem_ops.mog_free_rx_buffers = gem_free_rx_buffers;
bp->macbgem_ops.mog_init_rings = gem_init_rings;
bp->macbgem_ops.mog_rx = gem_rx;
- dev->ethtool_ops = &gem_ethtool_ops;
+ netdev->ethtool_ops = &gem_ethtool_ops;
} else {
bp->macbgem_ops.mog_alloc_rx_buffers = macb_alloc_rx_buffers;
bp->macbgem_ops.mog_free_rx_buffers = macb_free_rx_buffers;
bp->macbgem_ops.mog_init_rings = macb_init_rings;
bp->macbgem_ops.mog_rx = macb_rx;
- dev->ethtool_ops = &macb_ethtool_ops;
+ netdev->ethtool_ops = &macb_ethtool_ops;
}
- netdev_sw_irq_coalesce_default_on(dev);
+ netdev_sw_irq_coalesce_default_on(netdev);
- dev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
+ netdev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
/* Set features */
- dev->hw_features = NETIF_F_SG;
+ netdev->hw_features = NETIF_F_SG;
/* Check LSO capability; runtime detection can be overridden by a cap
* flag if the hardware is known to be buggy
*/
if (!(bp->caps & MACB_CAPS_NO_LSO) &&
GEM_BFEXT(PBUF_LSO, gem_readl(bp, DCFG6)))
- dev->hw_features |= MACB_NETIF_LSO;
+ netdev->hw_features |= MACB_NETIF_LSO;
/* Checksum offload is only available on gem with packet buffer */
if (macb_is_gem(bp) && !(bp->caps & MACB_CAPS_FIFO_MODE))
- dev->hw_features |= NETIF_F_HW_CSUM | NETIF_F_RXCSUM;
+ netdev->hw_features |= NETIF_F_HW_CSUM | NETIF_F_RXCSUM;
if (bp->caps & MACB_CAPS_SG_DISABLED)
- dev->hw_features &= ~NETIF_F_SG;
+ netdev->hw_features &= ~NETIF_F_SG;
/* Enable HW_TC if hardware supports QBV */
if (bp->caps & MACB_CAPS_QBV)
- dev->hw_features |= NETIF_F_HW_TC;
+ netdev->hw_features |= NETIF_F_HW_TC;
- dev->features = dev->hw_features;
+ netdev->features = netdev->hw_features;
/* Check RX Flow Filters support.
* Max Rx flows set by availability of screeners & compare regs:
@@ -4868,7 +4878,7 @@ static int macb_init_dflt(struct platform_device *pdev)
reg = GEM_BFINS(ETHTCMP, (uint16_t)ETH_P_IP, reg);
gem_writel_n(bp, ETHT, SCRT2_ETHT, reg);
/* Filtering is supported in hw but don't enable it in kernel now */
- dev->hw_features |= NETIF_F_NTUPLE;
+ netdev->hw_features |= NETIF_F_NTUPLE;
/* init Rx flow definitions */
bp->rx_fs_list.count = 0;
spin_lock_init(&bp->rx_fs_lock);
@@ -5078,9 +5088,9 @@ static void at91ether_stop(struct macb *lp)
}
/* Open the ethernet interface */
-static int at91ether_open(struct net_device *dev)
+static int at91ether_open(struct net_device *netdev)
{
- struct macb *lp = netdev_priv(dev);
+ struct macb *lp = netdev_priv(netdev);
u32 ctl;
int ret;
@@ -5102,7 +5112,7 @@ static int at91ether_open(struct net_device *dev)
if (ret)
goto stop;
- netif_start_queue(dev);
+ netif_start_queue(netdev);
return 0;
@@ -5114,11 +5124,11 @@ static int at91ether_open(struct net_device *dev)
}
/* Close the interface */
-static int at91ether_close(struct net_device *dev)
+static int at91ether_close(struct net_device *netdev)
{
- struct macb *lp = netdev_priv(dev);
+ struct macb *lp = netdev_priv(netdev);
- netif_stop_queue(dev);
+ netif_stop_queue(netdev);
phylink_stop(lp->phylink);
phylink_disconnect_phy(lp->phylink);
@@ -5132,14 +5142,14 @@ static int at91ether_close(struct net_device *dev)
/* Transmit packet */
static netdev_tx_t at91ether_start_xmit(struct sk_buff *skb,
- struct net_device *dev)
+ struct net_device *netdev)
{
- struct macb *lp = netdev_priv(dev);
+ struct macb *lp = netdev_priv(netdev);
if (macb_readl(lp, TSR) & MACB_BIT(RM9200_BNQ)) {
int desc = 0;
- netif_stop_queue(dev);
+ netif_stop_queue(netdev);
/* Store packet information (to free when Tx completed) */
lp->rm9200_txq[desc].skb = skb;
@@ -5148,8 +5158,8 @@ static netdev_tx_t at91ether_start_xmit(struct sk_buff *skb,
skb->len, DMA_TO_DEVICE);
if (dma_mapping_error(&lp->pdev->dev, lp->rm9200_txq[desc].mapping)) {
dev_kfree_skb_any(skb);
- dev->stats.tx_dropped++;
- netdev_err(dev, "%s: DMA mapping error\n", __func__);
+ netdev->stats.tx_dropped++;
+ netdev_err(netdev, "%s: DMA mapping error\n", __func__);
return NETDEV_TX_OK;
}
@@ -5159,7 +5169,8 @@ static netdev_tx_t at91ether_start_xmit(struct sk_buff *skb,
macb_writel(lp, TCR, skb->len);
} else {
- netdev_err(dev, "%s called, but device is busy!\n", __func__);
+ netdev_err(netdev, "%s called, but device is busy!\n",
+ __func__);
return NETDEV_TX_BUSY;
}
@@ -5169,9 +5180,9 @@ static netdev_tx_t at91ether_start_xmit(struct sk_buff *skb,
/* Extract received frame from buffer descriptors and sent to upper layers.
* (Called from interrupt context)
*/
-static void at91ether_rx(struct net_device *dev)
+static void at91ether_rx(struct net_device *netdev)
{
- struct macb *lp = netdev_priv(dev);
+ struct macb *lp = netdev_priv(netdev);
struct macb_queue *q = &lp->queues[0];
struct macb_dma_desc *desc;
unsigned char *p_recv;
@@ -5182,21 +5193,21 @@ static void at91ether_rx(struct net_device *dev)
while (desc->addr & MACB_BIT(RX_USED)) {
p_recv = q->rx_buffers + q->rx_tail * AT91ETHER_MAX_RBUFF_SZ;
pktlen = MACB_BF(RX_FRMLEN, desc->ctrl);
- skb = netdev_alloc_skb(dev, pktlen + 2);
+ skb = netdev_alloc_skb(netdev, pktlen + 2);
if (skb) {
skb_reserve(skb, 2);
skb_put_data(skb, p_recv, pktlen);
- skb->protocol = eth_type_trans(skb, dev);
- dev->stats.rx_packets++;
- dev->stats.rx_bytes += pktlen;
+ skb->protocol = eth_type_trans(skb, netdev);
+ netdev->stats.rx_packets++;
+ netdev->stats.rx_bytes += pktlen;
netif_rx(skb);
} else {
- dev->stats.rx_dropped++;
+ netdev->stats.rx_dropped++;
}
if (desc->ctrl & MACB_BIT(RX_MHASH_MATCH))
- dev->stats.multicast++;
+ netdev->stats.multicast++;
/* reset ownership bit */
desc->addr &= ~MACB_BIT(RX_USED);
@@ -5214,8 +5225,8 @@ static void at91ether_rx(struct net_device *dev)
/* MAC interrupt handler */
static irqreturn_t at91ether_interrupt(int irq, void *dev_id)
{
- struct net_device *dev = dev_id;
- struct macb *lp = netdev_priv(dev);
+ struct net_device *netdev = dev_id;
+ struct macb *lp = netdev_priv(netdev);
u32 intstatus, ctl;
unsigned int desc;
@@ -5226,13 +5237,13 @@ static irqreturn_t at91ether_interrupt(int irq, void *dev_id)
/* Receive complete */
if (intstatus & MACB_BIT(RCOMP))
- at91ether_rx(dev);
+ at91ether_rx(netdev);
/* Transmit complete */
if (intstatus & MACB_BIT(TCOMP)) {
/* The TCOM bit is set even if the transmission failed */
if (intstatus & (MACB_BIT(ISR_TUND) | MACB_BIT(ISR_RLE)))
- dev->stats.tx_errors++;
+ netdev->stats.tx_errors++;
desc = 0;
if (lp->rm9200_txq[desc].skb) {
@@ -5240,10 +5251,10 @@ static irqreturn_t at91ether_interrupt(int irq, void *dev_id)
lp->rm9200_txq[desc].skb = NULL;
dma_unmap_single(&lp->pdev->dev, lp->rm9200_txq[desc].mapping,
lp->rm9200_txq[desc].size, DMA_TO_DEVICE);
- dev->stats.tx_packets++;
- dev->stats.tx_bytes += lp->rm9200_txq[desc].size;
+ netdev->stats.tx_packets++;
+ netdev->stats.tx_bytes += lp->rm9200_txq[desc].size;
}
- netif_wake_queue(dev);
+ netif_wake_queue(netdev);
}
/* Work-around for EMAC Errata section 41.3.1 */
@@ -5255,18 +5266,18 @@ static irqreturn_t at91ether_interrupt(int irq, void *dev_id)
}
if (intstatus & MACB_BIT(ISR_ROVR))
- netdev_err(dev, "ROVR error\n");
+ netdev_err(netdev, "ROVR error\n");
return IRQ_HANDLED;
}
#ifdef CONFIG_NET_POLL_CONTROLLER
-static void at91ether_poll_controller(struct net_device *dev)
+static void at91ether_poll_controller(struct net_device *netdev)
{
unsigned long flags;
local_irq_save(flags);
- at91ether_interrupt(dev->irq, dev);
+ at91ether_interrupt(netdev->irq, netdev);
local_irq_restore(flags);
}
#endif
@@ -5313,17 +5324,17 @@ static int at91ether_clk_init(struct platform_device *pdev, struct clk **pclk,
static int at91ether_init(struct platform_device *pdev)
{
- struct net_device *dev = platform_get_drvdata(pdev);
- struct macb *bp = netdev_priv(dev);
+ struct net_device *netdev = platform_get_drvdata(pdev);
+ struct macb *bp = netdev_priv(netdev);
int err;
bp->queues[0].bp = bp;
- dev->netdev_ops = &at91ether_netdev_ops;
- dev->ethtool_ops = &macb_ethtool_ops;
+ netdev->netdev_ops = &at91ether_netdev_ops;
+ netdev->ethtool_ops = &macb_ethtool_ops;
- err = devm_request_irq(&pdev->dev, dev->irq, at91ether_interrupt,
- 0, dev->name, dev);
+ err = devm_request_irq(&pdev->dev, netdev->irq, at91ether_interrupt,
+ 0, netdev->name, netdev);
if (err)
return err;
@@ -5452,8 +5463,8 @@ static int fu540_c000_init(struct platform_device *pdev)
static int init_reset_optional(struct platform_device *pdev)
{
- struct net_device *dev = platform_get_drvdata(pdev);
- struct macb *bp = netdev_priv(dev);
+ struct net_device *netdev = platform_get_drvdata(pdev);
+ struct macb *bp = netdev_priv(netdev);
int ret;
if (bp->phy_interface == PHY_INTERFACE_MODE_SGMII) {
@@ -5761,7 +5772,7 @@ static int macb_probe(struct platform_device *pdev)
const struct macb_config *macb_config;
struct clk *tsu_clk = NULL;
phy_interface_t interface;
- struct net_device *dev;
+ struct net_device *netdev;
struct resource *regs;
u32 wtrmrk_rst_val;
void __iomem *mem;
@@ -5796,19 +5807,19 @@ static int macb_probe(struct platform_device *pdev)
goto err_disable_clocks;
}
- dev = alloc_etherdev_mq(sizeof(*bp), num_queues);
- if (!dev) {
+ netdev = alloc_etherdev_mq(sizeof(*bp), num_queues);
+ if (!netdev) {
err = -ENOMEM;
goto err_disable_clocks;
}
- dev->base_addr = regs->start;
+ netdev->base_addr = regs->start;
- SET_NETDEV_DEV(dev, &pdev->dev);
+ SET_NETDEV_DEV(netdev, &pdev->dev);
- bp = netdev_priv(dev);
+ bp = netdev_priv(netdev);
bp->pdev = pdev;
- bp->dev = dev;
+ bp->netdev = netdev;
bp->regs = mem;
bp->native_io = native_io;
if (native_io) {
@@ -5867,6 +5878,7 @@ static int macb_probe(struct platform_device *pdev)
}
spin_lock_init(&bp->lock);
spin_lock_init(&bp->stats_lock);
+ spin_lock_init(&bp->tsu_clk_lock);
/* setup capabilities */
macb_configure_caps(bp, macb_config);
@@ -5881,21 +5893,21 @@ static int macb_probe(struct platform_device *pdev)
bp->caps |= MACB_CAPS_DMA_64B;
}
#endif
- platform_set_drvdata(pdev, dev);
+ platform_set_drvdata(pdev, netdev);
- dev->irq = platform_get_irq(pdev, 0);
- if (dev->irq < 0) {
- err = dev->irq;
+ netdev->irq = platform_get_irq(pdev, 0);
+ if (netdev->irq < 0) {
+ err = netdev->irq;
goto err_out_free_netdev;
}
/* MTU range: 68 - 1518 or 10240 */
- dev->min_mtu = GEM_MTU_MIN_SIZE;
+ netdev->min_mtu = GEM_MTU_MIN_SIZE;
if ((bp->caps & MACB_CAPS_JUMBO) && bp->jumbo_max_len)
- dev->max_mtu = MIN(bp->jumbo_max_len, RX_BUFFER_MAX) -
+ netdev->max_mtu = MIN(bp->jumbo_max_len, RX_BUFFER_MAX) -
ETH_HLEN - ETH_FCS_LEN;
else
- dev->max_mtu = 1536 - ETH_HLEN - ETH_FCS_LEN;
+ netdev->max_mtu = 1536 - ETH_HLEN - ETH_FCS_LEN;
if (bp->caps & MACB_CAPS_BD_RD_PREFETCH) {
val = GEM_BFEXT(RXBD_RDBUFF, gem_readl(bp, DCFG10));
@@ -5913,7 +5925,7 @@ static int macb_probe(struct platform_device *pdev)
if (bp->caps & MACB_CAPS_NEEDS_RSTONUBR)
bp->rx_intr_mask |= MACB_BIT(RXUBR);
- err = of_get_ethdev_address(np, bp->dev);
+ err = of_get_ethdev_address(np, bp->netdev);
if (err == -EPROBE_DEFER)
goto err_out_free_netdev;
else if (err)
@@ -5935,9 +5947,9 @@ static int macb_probe(struct platform_device *pdev)
if (err)
goto err_out_phy_exit;
- netif_carrier_off(dev);
+ netif_carrier_off(netdev);
- err = register_netdev(dev);
+ err = register_netdev(netdev);
if (err) {
dev_err(&pdev->dev, "Cannot register net device, aborting.\n");
goto err_out_unregister_mdio;
@@ -5946,23 +5958,26 @@ static int macb_probe(struct platform_device *pdev)
INIT_WORK(&bp->hresp_err_bh_work, macb_hresp_error_task);
INIT_DELAYED_WORK(&bp->tx_lpi_work, macb_tx_lpi_work_fn);
- netdev_info(dev, "Cadence %s rev 0x%08x at 0x%08lx irq %d (%pM)\n",
+ netdev_info(netdev, "Cadence %s rev 0x%08x at 0x%08lx irq %d (%pM)\n",
macb_is_gem(bp) ? "GEM" : "MACB", macb_readl(bp, MID),
- dev->base_addr, dev->irq, dev->dev_addr);
+ netdev->base_addr, netdev->irq, netdev->dev_addr);
pm_runtime_put_autosuspend(&bp->pdev->dev);
return 0;
err_out_unregister_mdio:
- mdiobus_unregister(bp->mii_bus);
- mdiobus_free(bp->mii_bus);
+ if (bp->mii_bus) {
+ mdiobus_unregister(bp->mii_bus);
+ mdiobus_free(bp->mii_bus);
+ }
+ phylink_destroy(bp->phylink);
err_out_phy_exit:
phy_exit(bp->phy);
err_out_free_netdev:
- free_netdev(dev);
+ free_netdev(netdev);
err_disable_clocks:
macb_clks_disable(pclk, hclk, tx_clk, rx_clk, tsu_clk);
@@ -5975,17 +5990,19 @@ static int macb_probe(struct platform_device *pdev)
static void macb_remove(struct platform_device *pdev)
{
- struct net_device *dev;
+ struct net_device *netdev;
struct macb *bp;
- dev = platform_get_drvdata(pdev);
+ netdev = platform_get_drvdata(pdev);
- if (dev) {
- bp = netdev_priv(dev);
- unregister_netdev(dev);
+ if (netdev) {
+ bp = netdev_priv(netdev);
+ unregister_netdev(netdev);
phy_exit(bp->phy);
- mdiobus_unregister(bp->mii_bus);
- mdiobus_free(bp->mii_bus);
+ if (bp->mii_bus) {
+ mdiobus_unregister(bp->mii_bus);
+ mdiobus_free(bp->mii_bus);
+ }
device_set_wakeup_enable(&bp->pdev->dev, 0);
cancel_delayed_work_sync(&bp->tx_lpi_work);
@@ -5994,7 +6011,7 @@ static void macb_remove(struct platform_device *pdev)
pm_runtime_dont_use_autosuspend(&pdev->dev);
pm_runtime_set_suspended(&pdev->dev);
phylink_destroy(bp->phylink);
- free_netdev(dev);
+ free_netdev(netdev);
}
}
@@ -6009,7 +6026,7 @@ static int __maybe_unused macb_suspend(struct device *dev)
u32 tmp, ifa_local;
unsigned int q;
- if (!device_may_wakeup(&bp->dev->dev))
+ if (!device_may_wakeup(&bp->netdev->dev))
phy_exit(bp->phy);
if (!netif_running(netdev))
@@ -6019,7 +6036,7 @@ static int __maybe_unused macb_suspend(struct device *dev)
if (bp->wolopts & WAKE_ARP) {
/* Check for IP address in WOL ARP mode */
rcu_read_lock();
- idev = __in_dev_get_rcu(bp->dev);
+ idev = __in_dev_get_rcu(bp->netdev);
if (idev)
ifa = rcu_dereference(idev->ifa_list);
if (!ifa) {
@@ -6121,7 +6138,7 @@ static int __maybe_unused macb_resume(struct device *dev)
unsigned long flags;
unsigned int q;
- if (!device_may_wakeup(&bp->dev->dev))
+ if (!device_may_wakeup(&bp->netdev->dev))
phy_init(bp->phy);
if (!netif_running(netdev))
diff --git a/drivers/net/ethernet/cadence/macb_pci.c b/drivers/net/ethernet/cadence/macb_pci.c
index b79dec17e6b0..ac009007118f 100644
--- a/drivers/net/ethernet/cadence/macb_pci.c
+++ b/drivers/net/ethernet/cadence/macb_pci.c
@@ -24,48 +24,48 @@
#define GEM_PCLK_RATE 50000000
#define GEM_HCLK_RATE 50000000
-static int macb_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+static int macb_probe(struct pci_dev *pci, const struct pci_device_id *id)
{
int err;
- struct platform_device *plat_dev;
+ struct platform_device *pdev;
struct platform_device_info plat_info;
struct macb_platform_data plat_data;
struct resource res[2];
/* enable pci device */
- err = pcim_enable_device(pdev);
+ err = pcim_enable_device(pci);
if (err < 0) {
- dev_err(&pdev->dev, "Enabling PCI device has failed: %d", err);
+ dev_err(&pci->dev, "Enabling PCI device has failed: %d", err);
return err;
}
- pci_set_master(pdev);
+ pci_set_master(pci);
/* set up resources */
memset(res, 0x00, sizeof(struct resource) * ARRAY_SIZE(res));
- res[0].start = pci_resource_start(pdev, 0);
- res[0].end = pci_resource_end(pdev, 0);
+ res[0].start = pci_resource_start(pci, 0);
+ res[0].end = pci_resource_end(pci, 0);
res[0].name = PCI_DRIVER_NAME;
res[0].flags = IORESOURCE_MEM;
- res[1].start = pci_irq_vector(pdev, 0);
+ res[1].start = pci_irq_vector(pci, 0);
res[1].name = PCI_DRIVER_NAME;
res[1].flags = IORESOURCE_IRQ;
- dev_info(&pdev->dev, "EMAC physical base addr: %pa\n",
+ dev_info(&pci->dev, "EMAC physical base addr: %pa\n",
&res[0].start);
/* set up macb platform data */
memset(&plat_data, 0, sizeof(plat_data));
/* initialize clocks */
- plat_data.pclk = clk_register_fixed_rate(&pdev->dev, "pclk", NULL, 0,
+ plat_data.pclk = clk_register_fixed_rate(&pci->dev, "pclk", NULL, 0,
GEM_PCLK_RATE);
if (IS_ERR(plat_data.pclk)) {
err = PTR_ERR(plat_data.pclk);
goto err_pclk_register;
}
- plat_data.hclk = clk_register_fixed_rate(&pdev->dev, "hclk", NULL, 0,
+ plat_data.hclk = clk_register_fixed_rate(&pci->dev, "hclk", NULL, 0,
GEM_HCLK_RATE);
if (IS_ERR(plat_data.hclk)) {
err = PTR_ERR(plat_data.hclk);
@@ -74,24 +74,24 @@ static int macb_probe(struct pci_dev *pdev, const struct pci_device_id *id)
/* set up platform device info */
memset(&plat_info, 0, sizeof(plat_info));
- plat_info.parent = &pdev->dev;
- plat_info.fwnode = pdev->dev.fwnode;
+ plat_info.parent = &pci->dev;
+ plat_info.fwnode = pci->dev.fwnode;
plat_info.name = PLAT_DRIVER_NAME;
- plat_info.id = pdev->devfn;
+ plat_info.id = pci->devfn;
plat_info.res = res;
plat_info.num_res = ARRAY_SIZE(res);
plat_info.data = &plat_data;
plat_info.size_data = sizeof(plat_data);
- plat_info.dma_mask = pdev->dma_mask;
+ plat_info.dma_mask = pci->dma_mask;
/* register platform device */
- plat_dev = platform_device_register_full(&plat_info);
- if (IS_ERR(plat_dev)) {
- err = PTR_ERR(plat_dev);
+ pdev = platform_device_register_full(&plat_info);
+ if (IS_ERR(pdev)) {
+ err = PTR_ERR(pdev);
goto err_plat_dev_register;
}
- pci_set_drvdata(pdev, plat_dev);
+ pci_set_drvdata(pci, pdev);
return 0;
@@ -105,14 +105,14 @@ static int macb_probe(struct pci_dev *pdev, const struct pci_device_id *id)
return err;
}
-static void macb_remove(struct pci_dev *pdev)
+static void macb_remove(struct pci_dev *pci)
{
- struct platform_device *plat_dev = pci_get_drvdata(pdev);
- struct macb_platform_data *plat_data = dev_get_platdata(&plat_dev->dev);
+ struct platform_device *pdev = pci_get_drvdata(pci);
+ struct macb_platform_data *plat_data = dev_get_platdata(&pdev->dev);
struct clk *pclk = plat_data->pclk;
struct clk *hclk = plat_data->hclk;
- platform_device_unregister(plat_dev);
+ platform_device_unregister(pdev);
clk_unregister_fixed_rate(pclk);
clk_unregister_fixed_rate(hclk);
}
diff --git a/drivers/net/ethernet/cadence/macb_ptp.c b/drivers/net/ethernet/cadence/macb_ptp.c
index d91f7b1aa39c..6d9166389988 100644
--- a/drivers/net/ethernet/cadence/macb_ptp.c
+++ b/drivers/net/ethernet/cadence/macb_ptp.c
@@ -324,9 +324,9 @@ void gem_ptp_txstamp(struct macb *bp, struct sk_buff *skb,
skb_tstamp_tx(skb, &shhwtstamps);
}
-void gem_ptp_init(struct net_device *dev)
+void gem_ptp_init(struct net_device *netdev)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
bp->ptp_clock_info = gem_ptp_caps_template;
@@ -334,7 +334,8 @@ void gem_ptp_init(struct net_device *dev)
bp->tsu_rate = bp->ptp_info->get_tsu_rate(bp);
bp->ptp_clock_info.max_adj = bp->ptp_info->get_ptp_max_adj();
gem_ptp_init_timer(bp);
- bp->ptp_clock = ptp_clock_register(&bp->ptp_clock_info, &dev->dev);
+ gem_ptp_init_tsu(bp);
+ bp->ptp_clock = ptp_clock_register(&bp->ptp_clock_info, &netdev->dev);
if (IS_ERR(bp->ptp_clock)) {
pr_err("ptp clock register failed: %ld\n",
PTR_ERR(bp->ptp_clock));
@@ -345,17 +346,13 @@ void gem_ptp_init(struct net_device *dev)
return;
}
- spin_lock_init(&bp->tsu_clk_lock);
-
- gem_ptp_init_tsu(bp);
-
dev_info(&bp->pdev->dev, "%s ptp clock registered.\n",
GEM_PTP_TIMER_NAME);
}
-void gem_ptp_remove(struct net_device *ndev)
+void gem_ptp_remove(struct net_device *netdev)
{
- struct macb *bp = netdev_priv(ndev);
+ struct macb *bp = netdev_priv(netdev);
if (bp->ptp_clock) {
ptp_clock_unregister(bp->ptp_clock);
@@ -378,10 +375,10 @@ static int gem_ptp_set_ts_mode(struct macb *bp,
return 0;
}
-int gem_get_hwtst(struct net_device *dev,
+int gem_get_hwtst(struct net_device *netdev,
struct kernel_hwtstamp_config *tstamp_config)
{
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
*tstamp_config = bp->tstamp_config;
if (!macb_dma_ptp(bp))
@@ -402,13 +399,13 @@ static void gem_ptp_set_one_step_sync(struct macb *bp, u8 enable)
macb_writel(bp, NCR, reg_val & ~MACB_BIT(OSSMODE));
}
-int gem_set_hwtst(struct net_device *dev,
+int gem_set_hwtst(struct net_device *netdev,
struct kernel_hwtstamp_config *tstamp_config,
struct netlink_ext_ack *extack)
{
enum macb_bd_control tx_bd_control = TSTAMP_DISABLED;
enum macb_bd_control rx_bd_control = TSTAMP_DISABLED;
- struct macb *bp = netdev_priv(dev);
+ struct macb *bp = netdev_priv(netdev);
u32 regval;
if (!macb_dma_ptp(bp))
diff --git a/drivers/net/ethernet/cortina/gemini.c b/drivers/net/ethernet/cortina/gemini.c
index 4c762229ce42..f08de623e6f7 100644
--- a/drivers/net/ethernet/cortina/gemini.c
+++ b/drivers/net/ethernet/cortina/gemini.c
@@ -124,6 +124,7 @@ struct gemini_ethernet_port {
unsigned int rx_coalesce_nsecs;
struct sk_buff *rx_skb;
unsigned int rx_frag_nr;
+ bool rx_dropping;
unsigned int freeq_refill;
struct gmac_txq txq[TX_QUEUE_NUM];
@@ -1439,7 +1440,8 @@ static struct sk_buff *gmac_skb_if_good_frame(struct gemini_ethernet_port *port,
return skb;
}
-static unsigned int gmac_rx(struct net_device *netdev, unsigned int budget)
+static unsigned int gmac_rx(struct net_device *netdev, unsigned int budget,
+ unsigned int *freeq_consumed)
{
struct gemini_ethernet_port *port = netdev_priv(netdev);
unsigned short m = (1 << port->rxq_order) - 1;
@@ -1447,9 +1449,12 @@ static unsigned int gmac_rx(struct net_device *netdev, unsigned int budget)
void __iomem *ptr_reg = port->rxq_rwptr;
unsigned int frag_nr = port->rx_frag_nr;
struct sk_buff *skb = port->rx_skb;
+ unsigned int consumed = 0;
unsigned int frame_len, frag_len;
struct gmac_rxdesc *rx = NULL;
struct gmac_queue_page *gpage;
+ unsigned int received = 0;
+ bool dropping = port->rx_dropping;
union gmac_rxdesc_0 word0;
union gmac_rxdesc_1 word1;
union gmac_rxdesc_3 word3;
@@ -1471,6 +1476,7 @@ static unsigned int gmac_rx(struct net_device *netdev, unsigned int budget)
w = rw.bits.wptr;
while (budget && w != r) {
+ page = NULL;
rx = port->rxq_ring + r;
word0 = rx->word0;
word1 = rx->word1;
@@ -1479,11 +1485,22 @@ static unsigned int gmac_rx(struct net_device *netdev, unsigned int budget)
r++;
r &= m;
+ consumed++;
frag_len = word0.bits.buffer_size;
frame_len = word1.bits.byte_count;
page_offs = mapping & ~PAGE_MASK;
+ if (word3.bits32 & SOF_BIT) {
+ if (skb) {
+ napi_free_frags(&port->napi);
+ port->stats.rx_dropped++;
+ skb = NULL;
+ frag_nr = 0;
+ }
+ dropping = false;
+ }
+
if (!mapping) {
netdev_err(netdev,
"rxq[%u]: HW BUG: zero DMA desc\n", r);
@@ -1494,24 +1511,11 @@ static unsigned int gmac_rx(struct net_device *netdev, unsigned int budget)
gpage = gmac_get_queue_page(geth, port, mapping + PAGE_SIZE);
if (!gpage) {
dev_err(geth->dev, "could not find mapping\n");
- port->stats.rx_dropped++;
- if (skb) {
- napi_free_frags(&port->napi);
- skb = NULL;
- frag_nr = 0;
- }
- continue;
+ goto err_drop;
}
page = gpage->page;
if (word3.bits32 & SOF_BIT) {
- if (skb) {
- napi_free_frags(&port->napi);
- port->stats.rx_dropped++;
- skb = NULL;
- frag_nr = 0;
- }
-
skb = gmac_skb_if_good_frame(port, word0, frame_len);
if (!skb)
goto err_drop;
@@ -1521,8 +1525,7 @@ static unsigned int gmac_rx(struct net_device *netdev, unsigned int budget)
frag_nr = 0;
} else if (!skb) {
- put_page(page);
- continue;
+ goto err_drop;
}
if (word3.bits32 & EOF_BIT)
@@ -1545,9 +1548,8 @@ static unsigned int gmac_rx(struct net_device *netdev, unsigned int budget)
napi_gro_frags(&port->napi);
skb = NULL;
frag_nr = 0;
- --budget;
}
- continue;
+ goto next_desc;
err_drop:
if (skb) {
@@ -1556,16 +1558,29 @@ static unsigned int gmac_rx(struct net_device *netdev, unsigned int budget)
frag_nr = 0;
}
- if (mapping)
+ if (page)
put_page(page);
- port->stats.rx_dropped++;
+ if (!dropping) {
+ port->stats.rx_dropped++;
+ dropping = true;
+ }
+
+next_desc:
+ /* Final or single-descriptor fragment, advance things */
+ if (word3.bits32 & EOF_BIT) {
+ budget--;
+ received++;
+ dropping = false;
+ }
}
port->rx_skb = skb;
port->rx_frag_nr = frag_nr;
+ port->rx_dropping = dropping;
+ *freeq_consumed = consumed;
writew(r, ptr_reg);
- return budget;
+ return received;
}
static int gmac_napi_poll(struct napi_struct *napi, int budget)
@@ -1573,26 +1588,27 @@ static int gmac_napi_poll(struct napi_struct *napi, int budget)
struct gemini_ethernet_port *port = netdev_priv(napi->dev);
struct gemini_ethernet *geth = port->geth;
unsigned int freeq_threshold;
+ unsigned int freeq_consumed;
unsigned int received;
freeq_threshold = 1 << (geth->freeq_order - 1);
u64_stats_update_begin(&port->rx_stats_syncp);
- received = gmac_rx(napi->dev, budget);
- if (received < budget) {
- napi_gro_flush(napi, false);
- napi_complete_done(napi, received);
- gmac_enable_rx_irq(napi->dev, 1);
+ received = gmac_rx(napi->dev, budget, &freeq_consumed);
+ if (received < budget)
++port->rx_napi_exits;
- }
- port->freeq_refill += (budget - received);
+ u64_stats_update_end(&port->rx_stats_syncp);
+
+ port->freeq_refill += freeq_consumed;
if (port->freeq_refill > freeq_threshold) {
port->freeq_refill -= freeq_threshold;
geth_fill_freeq(geth, true);
}
- u64_stats_update_end(&port->rx_stats_syncp);
+ if (received < budget && napi_complete_done(napi, received))
+ gmac_enable_rx_irq(napi->dev, 1);
+
return received;
}
@@ -1893,6 +1909,7 @@ static int gmac_stop(struct net_device *netdev)
napi_disable(&port->napi);
port->rx_skb = NULL;
port->rx_frag_nr = 0;
+ port->rx_dropping = false;
gmac_enable_irq(netdev, 0);
gmac_cleanup_rxq(netdev);
diff --git a/drivers/net/ethernet/huawei/hinic/hinic_hw_mbox.c b/drivers/net/ethernet/huawei/hinic/hinic_hw_mbox.c
index 2784127327e6..6e67a6c9578e 100644
--- a/drivers/net/ethernet/huawei/hinic/hinic_hw_mbox.c
+++ b/drivers/net/ethernet/huawei/hinic/hinic_hw_mbox.c
@@ -128,6 +128,7 @@ enum hinic_mbox_tx_status {
#define SEQ_ID_START_VAL 0
#define SEQ_ID_MAX_VAL 42
+#define MBOX_LAST_SEG_MAX_LEN (MBOX_MAX_BUF_SZ - SEQ_ID_MAX_VAL * MBOX_SEG_LEN)
#define NO_DMA_ATTRIBUTE_VAL 0
@@ -372,7 +373,8 @@ recv_pf_from_vf_mbox_handler(struct hinic_mbox_func_to_func *func_to_func,
static bool check_mbox_seq_id_and_seg_len(struct hinic_recv_mbox *recv_mbox,
u8 seq_id, u8 seg_len)
{
- if (seq_id > SEQ_ID_MAX_VAL || seg_len > MBOX_SEG_LEN)
+ if (seq_id > SEQ_ID_MAX_VAL || seg_len > MBOX_SEG_LEN ||
+ (seq_id == SEQ_ID_MAX_VAL && seg_len > MBOX_LAST_SEG_MAX_LEN))
return false;
if (seq_id == 0) {
diff --git a/drivers/net/ethernet/intel/ice/ice_common.c b/drivers/net/ethernet/intel/ice/ice_common.c
index ef1ce106f81b..04633103e3e6 100644
--- a/drivers/net/ethernet/intel/ice/ice_common.c
+++ b/drivers/net/ethernet/intel/ice/ice_common.c
@@ -1051,14 +1051,13 @@ int ice_init_hw(struct ice_hw *hw)
hw->evb_veb = true;
- /* init xarray for identifying scheduling nodes uniquely */
- xa_init_flags(&hw->port_info->sched_node_ids, XA_FLAGS_ALLOC);
+ xa_init_flags(&hw->sched_node_ids, XA_FLAGS_ALLOC);
/* Query the allocated resources for Tx scheduler */
status = ice_sched_query_res_alloc(hw);
if (status) {
ice_debug(hw, ICE_DBG_SCHED, "Failed to get scheduler allocated resources\n");
- goto err_unroll_alloc;
+ goto err_unroll_xarray;
}
ice_sched_get_psm_clk_freq(hw);
@@ -1146,6 +1145,8 @@ int ice_init_hw(struct ice_hw *hw)
ice_cleanup_fltr_mgmt_struct(hw);
err_unroll_sched:
ice_sched_cleanup_all(hw);
+err_unroll_xarray:
+ xa_destroy(&hw->sched_node_ids);
err_unroll_alloc:
devm_kfree(ice_hw_to_dev(hw), hw->port_info);
err_unroll_cqinit:
@@ -1186,6 +1187,8 @@ void ice_deinit_hw(struct ice_hw *hw)
/* Clear VSI contexts if not already cleared */
ice_clear_all_vsi_ctx(hw);
+
+ xa_destroy(&hw->sched_node_ids);
}
/**
diff --git a/drivers/net/ethernet/intel/ice/ice_sched.c b/drivers/net/ethernet/intel/ice/ice_sched.c
index fff0c1afdb41..ffa18d86729a 100644
--- a/drivers/net/ethernet/intel/ice/ice_sched.c
+++ b/drivers/net/ethernet/intel/ice/ice_sched.c
@@ -371,7 +371,7 @@ void ice_free_sched_node(struct ice_port_info *pi, struct ice_sched_node *node)
devm_kfree(ice_hw_to_dev(hw), node->children);
kfree(node->name);
- xa_erase(&pi->sched_node_ids, node->id);
+ xa_erase(&hw->sched_node_ids, node->id);
devm_kfree(ice_hw_to_dev(hw), node);
}
@@ -977,7 +977,7 @@ ice_sched_add_elems(struct ice_port_info *pi, struct ice_sched_node *tc_node,
if (!new_node->name)
return -ENOMEM;
- status = xa_alloc(&pi->sched_node_ids, &new_node->id, NULL, XA_LIMIT(0, UINT_MAX),
+ status = xa_alloc(&hw->sched_node_ids, &new_node->id, NULL, XA_LIMIT(0, UINT_MAX),
GFP_KERNEL);
if (status) {
ice_debug(hw, ICE_DBG_SCHED, "xa_alloc failed for sched node status =%d\n",
diff --git a/drivers/net/ethernet/intel/ice/ice_trace.h b/drivers/net/ethernet/intel/ice/ice_trace.h
index 4f35ef8d6b29..7568c917cdbe 100644
--- a/drivers/net/ethernet/intel/ice/ice_trace.h
+++ b/drivers/net/ethernet/intel/ice/ice_trace.h
@@ -63,23 +63,33 @@
DECLARE_EVENT_CLASS(ice_rx_dim_template,
TP_PROTO(struct ice_q_vector *q_vector, struct dim *dim),
TP_ARGS(q_vector, dim),
- TP_STRUCT__entry(__field(struct ice_q_vector *, q_vector)
- __field(struct dim *, dim)
+ TP_STRUCT__entry(__field(u16, q_index)
+ __field(u8, state)
+ __field(u8, profile_ix)
+ __field(u8, tune_state)
+ __field(u8, steps_right)
+ __field(u8, steps_left)
+ __field(u8, tired)
__string(devname, q_vector->rx.rx_ring->netdev->name)),
- TP_fast_assign(__entry->q_vector = q_vector;
- __entry->dim = dim;
+ TP_fast_assign(__entry->q_index = q_vector->rx.rx_ring->q_index;
+ __entry->state = dim->state;
+ __entry->profile_ix = dim->profile_ix;
+ __entry->tune_state = dim->tune_state;
+ __entry->steps_right = dim->steps_right;
+ __entry->steps_left = dim->steps_left;
+ __entry->tired = dim->tired;
__assign_str(devname);),
TP_printk("netdev: %s Rx-Q: %d dim-state: %d dim-profile: %d dim-tune: %d dim-st-right: %d dim-st-left: %d dim-tired: %d",
__get_str(devname),
- __entry->q_vector->rx.rx_ring->q_index,
- __entry->dim->state,
- __entry->dim->profile_ix,
- __entry->dim->tune_state,
- __entry->dim->steps_right,
- __entry->dim->steps_left,
- __entry->dim->tired)
+ __entry->q_index,
+ __entry->state,
+ __entry->profile_ix,
+ __entry->tune_state,
+ __entry->steps_right,
+ __entry->steps_left,
+ __entry->tired)
);
DEFINE_EVENT(ice_rx_dim_template, ice_rx_dim_work,
@@ -90,23 +100,33 @@ DEFINE_EVENT(ice_rx_dim_template, ice_rx_dim_work,
DECLARE_EVENT_CLASS(ice_tx_dim_template,
TP_PROTO(struct ice_q_vector *q_vector, struct dim *dim),
TP_ARGS(q_vector, dim),
- TP_STRUCT__entry(__field(struct ice_q_vector *, q_vector)
- __field(struct dim *, dim)
+ TP_STRUCT__entry(__field(u16, q_index)
+ __field(u8, state)
+ __field(u8, profile_ix)
+ __field(u8, tune_state)
+ __field(u8, steps_right)
+ __field(u8, steps_left)
+ __field(u8, tired)
__string(devname, q_vector->tx.tx_ring->netdev->name)),
- TP_fast_assign(__entry->q_vector = q_vector;
- __entry->dim = dim;
+ TP_fast_assign(__entry->q_index = q_vector->tx.tx_ring->q_index;
+ __entry->state = dim->state;
+ __entry->profile_ix = dim->profile_ix;
+ __entry->tune_state = dim->tune_state;
+ __entry->steps_right = dim->steps_right;
+ __entry->steps_left = dim->steps_left;
+ __entry->tired = dim->tired;
__assign_str(devname);),
TP_printk("netdev: %s Tx-Q: %d dim-state: %d dim-profile: %d dim-tune: %d dim-st-right: %d dim-st-left: %d dim-tired: %d",
__get_str(devname),
- __entry->q_vector->tx.tx_ring->q_index,
- __entry->dim->state,
- __entry->dim->profile_ix,
- __entry->dim->tune_state,
- __entry->dim->steps_right,
- __entry->dim->steps_left,
- __entry->dim->tired)
+ __entry->q_index,
+ __entry->state,
+ __entry->profile_ix,
+ __entry->tune_state,
+ __entry->steps_right,
+ __entry->steps_left,
+ __entry->tired)
);
DEFINE_EVENT(ice_tx_dim_template, ice_tx_dim_work,
diff --git a/drivers/net/ethernet/intel/ice/ice_type.h b/drivers/net/ethernet/intel/ice/ice_type.h
index d9a5c1aae7c2..cf147a212707 100644
--- a/drivers/net/ethernet/intel/ice/ice_type.h
+++ b/drivers/net/ethernet/intel/ice/ice_type.h
@@ -765,7 +765,6 @@ struct ice_port_info {
/* List contain profile ID(s) and other params per layer */
struct list_head rl_prof_list[ICE_AQC_TOPO_MAX_LEVEL_NUM];
struct ice_qos_cfg qos_cfg;
- struct xarray sched_node_ids;
u8 is_vf:1;
u8 is_custom_tx_enabled:1;
};
@@ -930,6 +929,7 @@ struct ice_hw {
u8 sw_entry_point_layer;
u16 max_children[ICE_AQC_TOPO_MAX_LEVEL_NUM];
struct list_head agg_list; /* lists all aggregator */
+ struct xarray sched_node_ids;
struct ice_vsi_ctx *vsi_ctx[ICE_MAX_VSI];
u8 evb_veb; /* true for VEB, false for VEPA */
diff --git a/drivers/net/ethernet/intel/idpf/idpf_txrx.c b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
index 91ca75e45463..81e83a98a8e2 100644
--- a/drivers/net/ethernet/intel/idpf/idpf_txrx.c
+++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
@@ -3299,6 +3299,7 @@ static int idpf_rx_rsc(struct idpf_rx_queue *rxq, struct sk_buff *skb,
struct libeth_rx_pt decoded)
{
u16 rsc_segments, rsc_seg_len;
+ u16 l3_start = 0;
bool ipv4, ipv6;
int len;
@@ -3321,7 +3322,10 @@ static int idpf_rx_rsc(struct idpf_rx_queue *rxq, struct sk_buff *skb,
NAPI_GRO_CB(skb)->count = rsc_segments;
skb_shinfo(skb)->gso_size = rsc_seg_len;
- skb_reset_network_header(skb);
+ if (unlikely(eth_type_vlan(skb->protocol)))
+ l3_start = VLAN_HLEN;
+
+ skb_set_network_header(skb, l3_start);
if (ipv4) {
struct iphdr *ipv4h = ip_hdr(skb);
@@ -3329,7 +3333,7 @@ static int idpf_rx_rsc(struct idpf_rx_queue *rxq, struct sk_buff *skb,
skb_shinfo(skb)->gso_type = SKB_GSO_TCPV4;
/* Reset and set transport header offset in skb */
- skb_set_transport_header(skb, sizeof(struct iphdr));
+ skb_set_transport_header(skb, l3_start + sizeof(struct iphdr));
len = skb->len - skb_transport_offset(skb);
/* Compute the TCP pseudo header checksum*/
@@ -3339,7 +3343,7 @@ static int idpf_rx_rsc(struct idpf_rx_queue *rxq, struct sk_buff *skb,
struct ipv6hdr *ipv6h = ipv6_hdr(skb);
skb_shinfo(skb)->gso_type = SKB_GSO_TCPV6;
- skb_set_transport_header(skb, sizeof(struct ipv6hdr));
+ skb_set_transport_header(skb, l3_start + sizeof(struct ipv6hdr));
len = skb->len - skb_transport_offset(skb);
tcp_hdr(skb)->check =
~tcp_v6_check(len, &ipv6h->saddr, &ipv6h->daddr, 0);
@@ -4145,6 +4149,26 @@ static void idpf_vport_intr_ena_irq_all(struct idpf_vport *vport,
writel(rsrc->noirq_dyn_ctl_ena, rsrc->noirq_dyn_ctl);
}
+/**
+ * idpf_vport_intr_dis_dim_all - Disable DIM work for all q_vectors
+ * @rsrc: pointer to queue and vector resources
+ *
+ * The DIM works are embedded in the q_vector array that
+ * idpf_vport_intr_rel() frees, and the poll arms them after
+ * napi_complete_done() has already cleared NAPI_STATE_SCHED. Disable
+ * rather than just cancel, so that a poll tail still running past
+ * napi_disable() cannot queue them again behind the drain.
+ */
+static void idpf_vport_intr_dis_dim_all(struct idpf_q_vec_rsrc *rsrc)
+{
+ for (u16 v_idx = 0; v_idx < rsrc->num_q_vectors; v_idx++) {
+ struct idpf_q_vector *q_vector = &rsrc->q_vectors[v_idx];
+
+ disable_work_sync(&q_vector->tx_dim.work);
+ disable_work_sync(&q_vector->rx_dim.work);
+ }
+}
+
/**
* idpf_vport_intr_deinit - Release all vector associations for the vport
* @vport: main vport structure
@@ -4155,6 +4179,7 @@ void idpf_vport_intr_deinit(struct idpf_vport *vport,
{
idpf_vport_intr_dis_irq_all(rsrc);
idpf_vport_intr_napi_dis_all(rsrc);
+ idpf_vport_intr_dis_dim_all(rsrc);
idpf_vport_intr_napi_del_all(rsrc);
idpf_vport_intr_rel_irq(vport, rsrc);
}
@@ -4235,7 +4260,6 @@ static void idpf_vport_intr_napi_ena_all(struct idpf_q_vec_rsrc *rsrc)
for (u16 q_idx = 0; q_idx < rsrc->num_q_vectors; q_idx++) {
struct idpf_q_vector *q_vector = &rsrc->q_vectors[q_idx];
- idpf_init_dim(q_vector);
napi_enable(&q_vector->napi);
}
}
@@ -4578,6 +4602,8 @@ int idpf_vport_intr_alloc(struct idpf_vport *vport,
q_coal = &user_config->q_coalesce[v_idx];
q_vector->vport = vport;
+ idpf_init_dim(q_vector);
+
q_vector->tx_itr_value = q_coal->tx_coalesce_usecs;
q_vector->tx_intr_mode = q_coal->tx_intr_mode;
q_vector->tx_itr_idx = VIRTCHNL2_ITR_IDX_1;
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mcs.c b/drivers/net/ethernet/marvell/octeontx2/af/mcs.c
index a07e0b3d8d00..211c10aa5880 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/mcs.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/mcs.c
@@ -1417,6 +1417,16 @@ static int mcs_x2p_calibration(struct mcs *mcs)
int i, err = 0;
u64 val;
+ /* Clear any stale calibration state left by firmware/bootloader.
+ * Some firmware versions may leave MCSX_MIL_GLOBAL bit 5 set,
+ * preventing the hardware from detecting the rising edge needed to
+ * trigger X2P calibration.
+ */
+ val = mcs_reg_read(mcs, MCSX_MIL_GLOBAL);
+ val &= ~BIT_ULL(5);
+ mcs_reg_write(mcs, MCSX_MIL_GLOBAL, val);
+ usleep_range(100, 200);
+
/* set X2P calibration */
val = mcs_reg_read(mcs, MCSX_MIL_GLOBAL);
val |= BIT_ULL(5);
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c
index 74c041ab5280..937b085582b5 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c
@@ -3468,6 +3468,8 @@ int rvu_get_num_lbk_chans(void)
return ret;
}
+#define PCI_DEVID_OCTEONTX2_RVU_AFVF 0xA0F8
+
static int rvu_enable_sriov(struct rvu *rvu)
{
struct pci_dev *pdev = rvu->pdev;
@@ -3486,24 +3488,27 @@ static int rvu_enable_sriov(struct rvu *rvu)
return 0;
pci_read_config_word(pdev, pos + PCI_SRIOV_VF_DID, &rvu->vf_devid);
- chans = rvu_get_num_lbk_chans();
- if (chans < 0)
- return chans;
-
vfs = pci_sriov_get_totalvfs(pdev);
-
- /* Limit VFs in case we have more VFs than LBK channels available. */
- if (vfs > chans)
- vfs = chans;
-
if (!vfs)
return 0;
- /* LBK channel number 63 is used for switching packets between
- * CGX mapped VFs. Hence limit LBK pairs till 62 only.
- */
- if (vfs > 62)
- vfs = 62;
+ if (rvu->vf_devid == PCI_DEVID_OCTEONTX2_RVU_AFVF) {
+ chans = rvu_get_num_lbk_chans();
+ if (chans < 0)
+ return chans;
+
+ /* The last LBK channel is reserved for switching packets between
+ * CGX mapped VFs. Also, since LBK VFs work in pairs, limit VF
+ * count to available LBK channels minus 2.
+ */
+ vfs = min(vfs, chans - 2);
+
+ if (vfs <= 0) {
+ dev_warn(&pdev->dev,
+ "Skipping SRIOV enablement, not enough LBK channels available\n");
+ return 0;
+ }
+ }
/* Save VFs number for reference in VF interrupts handlers.
* Since interrupts might start arriving during SRIOV enablement
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
index 22ee99676879..904374baae6f 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
@@ -829,19 +829,25 @@ static int rvu_dbg_rvu_pf_cgx_map_display(struct seq_file *filp, void *unused)
int pf, domain, blkid;
u8 cgx_id, lmac_id;
u16 pcifunc;
+ u8 start;
- domain = 2;
+ domain = pci_domain_nr(rvu->pdev->bus);
mac_ops = get_mac_ops(rvu_first_cgx_pdata(rvu));
/* There can be no CGX devices at all */
if (!mac_ops)
return 0;
seq_printf(filp, "PCI dev\t\tRVU PF Func\tNIX block\t%s\tLMAC\tCHAN\n",
mac_ops->name);
+
+ /* All the PF devices are on contiguous PCI bus numbers, but the PF0(AF)
+ * may not start from 1 always. Hence get domain and bus from PCI device.
+ */
+ start = rvu->pdev->bus->number;
for (pf = 0; pf < rvu->hw->total_pfs; pf++) {
if (!is_pf_cgxmapped(rvu, pf))
continue;
- pdev = pci_get_domain_bus_and_slot(domain, pf + 1, 0);
+ pdev = pci_get_domain_bus_and_slot(domain, pf + start, 0);
if (!pdev)
continue;
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc.c
index 38554d51164e..6f2e6bce164d 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc.c
@@ -2957,10 +2957,9 @@ static void npc_mcam_free_all_entries(struct rvu *rvu, struct npc_mcam *mcam,
}
}
- /* Disable the entry */
- npc_enable_mcam_entry(rvu, mcam, blkaddr, index, false);
-
if (!cn20k_dft_rl) {
+ /* Disable the entry */
+ npc_enable_mcam_entry(rvu, mcam, blkaddr, index, false);
mcam->entry2pfvf_map[index] = NPC_MCAM_INVALID_MAP;
/* Free the entry in bitmap */
npc_mcam_clear_bit(mcam, index);
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/qos.c b/drivers/net/ethernet/marvell/octeontx2/nic/qos.c
index 69c0911e28e9..f160b1618efa 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/qos.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/qos.c
@@ -235,13 +235,63 @@ static int otx2_qos_txschq_set_parent_topology(struct otx2_nic *pfvf,
return rc;
}
+static int otx2_qos_reset_schq_topology(struct otx2_nic *pfvf, u16 lvl,
+ u16 schq)
+{
+ struct mbox *mbox = &pfvf->mbox;
+ struct nix_txschq_config *cfg;
+ int rc;
+
+ if (lvl < NIX_TXSCH_LVL_TL4 || lvl >= NIX_TXSCH_LVL_TL1)
+ return 0;
+
+ mutex_lock(&mbox->lock);
+
+ cfg = otx2_mbox_alloc_msg_nix_txschq_cfg(mbox);
+ if (!cfg) {
+ mutex_unlock(&mbox->lock);
+ return -ENOMEM;
+ }
+
+ cfg->lvl = lvl;
+ cfg->num_regs = 1;
+
+ if (lvl == NIX_TXSCH_LVL_TL4)
+ cfg->reg[0] = NIX_AF_TL4X_TOPOLOGY(schq);
+ else if (lvl == NIX_TXSCH_LVL_TL3)
+ cfg->reg[0] = NIX_AF_TL3X_TOPOLOGY(schq);
+ else if (lvl == NIX_TXSCH_LVL_TL2)
+ cfg->reg[0] = NIX_AF_TL2X_TOPOLOGY(schq);
+
+ cfg->regval[0] = 0;
+
+ rc = otx2_sync_mbox_msg(mbox);
+
+ mutex_unlock(&mbox->lock);
+
+ return rc;
+}
+
+static void otx2_qos_free_hw_schq(struct otx2_nic *pfvf, u16 lvl, u16 schq)
+{
+ int err;
+
+ err = otx2_qos_reset_schq_topology(pfvf, lvl, schq);
+ if (err)
+ netdev_warn(pfvf->netdev,
+ "QoS: failed to reset topology for schq %u at level %u: %d\n",
+ schq, lvl, err);
+
+ otx2_txschq_free_one(pfvf, lvl, schq);
+}
+
static void otx2_qos_free_hw_node_schq(struct otx2_nic *pfvf,
struct otx2_qos_node *parent)
{
struct otx2_qos_node *node;
list_for_each_entry_reverse(node, &parent->child_schq_list, list)
- otx2_txschq_free_one(pfvf, node->level, node->schq);
+ otx2_qos_free_hw_schq(pfvf, node->level, node->schq);
}
static void otx2_qos_free_hw_node(struct otx2_nic *pfvf,
@@ -252,7 +302,7 @@ static void otx2_qos_free_hw_node(struct otx2_nic *pfvf,
list_for_each_entry_safe(node, tmp, &parent->child_list, list) {
otx2_qos_free_hw_node(pfvf, node);
otx2_qos_free_hw_node_schq(pfvf, node);
- otx2_txschq_free_one(pfvf, node->level, node->schq);
+ otx2_qos_free_hw_schq(pfvf, node->level, node->schq);
}
}
@@ -266,7 +316,7 @@ static void otx2_qos_free_hw_cfg(struct otx2_nic *pfvf,
otx2_qos_free_hw_node_schq(pfvf, node);
/* free node hw mappings */
- otx2_txschq_free_one(pfvf, node->level, node->schq);
+ otx2_qos_free_hw_schq(pfvf, node->level, node->schq);
mutex_unlock(&pfvf->qos.qos_lock);
}
@@ -913,7 +963,7 @@ static void otx2_qos_free_cfg(struct otx2_nic *pfvf, struct otx2_qos_cfg *cfg)
for (lvl = 0; lvl < NIX_TXSCH_LVL_CNT; lvl++) {
for (idx = 0; idx < cfg->schq[lvl]; idx++) {
schq = cfg->schq_list[lvl][idx];
- otx2_txschq_free_one(pfvf, lvl, schq);
+ otx2_qos_free_hw_schq(pfvf, lvl, schq);
}
}
@@ -921,7 +971,7 @@ static void otx2_qos_free_cfg(struct otx2_nic *pfvf, struct otx2_qos_cfg *cfg)
for (idx = 0; idx < cfg->schq_contig[lvl]; idx++) {
if (cfg->schq_index_used[lvl][idx]) {
schq = cfg->schq_contig_list[lvl][idx];
- otx2_txschq_free_one(pfvf, lvl, schq);
+ otx2_qos_free_hw_schq(pfvf, lvl, schq);
}
}
}
diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.c b/drivers/net/ethernet/mediatek/mtk_eth_soc.c
index 351444fb4871..b2473df74dff 100644
--- a/drivers/net/ethernet/mediatek/mtk_eth_soc.c
+++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.c
@@ -4827,7 +4827,7 @@ static int mtk_add_mac(struct mtk_eth *eth, struct device_node *np)
phy_interface_t phy_mode;
struct phylink *phylink;
struct mtk_mac *mac;
- int id, err;
+ int id, err, i;
int txqs = 1;
u32 val;
@@ -4906,8 +4906,8 @@ static int mtk_add_mac(struct mtk_eth *eth, struct device_node *np)
mac->phylink_config.type = PHYLINK_NETDEV;
mac->phylink_config.mac_capabilities = MAC_ASYM_PAUSE | MAC_SYM_PAUSE |
MAC_10 | MAC_100 | MAC_1000 | MAC_2500FD;
- mac->phylink_config.lpi_capabilities = MAC_100FD | MAC_1000FD |
- MAC_2500FD;
+ /* LPI above 1 Gbps is not supported */
+ mac->phylink_config.lpi_capabilities = MAC_100FD | MAC_1000FD;
mac->phylink_config.lpi_timer_default = 1000;
/* MT7623 gmac0 is now missing its speed-specific PLL configuration
@@ -4965,6 +4965,18 @@ static int mtk_add_mac(struct mtk_eth *eth, struct device_node *np)
__set_bit(PHY_INTERFACE_MODE_INTERNAL,
mac->phylink_config.supported_interfaces);
+ /* LPI wake-up timing is only verified on MTK_GMAC_EEE SoCs */
+ if (MTK_HAS_CAPS(eth->soc->caps, MTK_GMAC_EEE)) {
+ phy_interface_copy(mac->phylink_config.lpi_interfaces,
+ mac->phylink_config.supported_interfaces);
+ __clear_bit(PHY_INTERFACE_MODE_2500BASEX,
+ mac->phylink_config.lpi_interfaces);
+ for (i = 0; i < PHY_INTERFACE_MODE_MAX; i++)
+ if (mtk_interface_mode_is_xgmii(eth, i))
+ __clear_bit(i,
+ mac->phylink_config.lpi_interfaces);
+ }
+
phylink = phylink_create(&mac->phylink_config,
of_fwnode_handle(mac->of_node),
phy_mode, mac_ops);
diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.h b/drivers/net/ethernet/mediatek/mtk_eth_soc.h
index 0168e2fbc619..88a9b3b23bea 100644
--- a/drivers/net/ethernet/mediatek/mtk_eth_soc.h
+++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.h
@@ -994,6 +994,7 @@ enum mkt_eth_capabilities {
MTK_U3_COPHY_V2_BIT,
MTK_SRAM_BIT,
MTK_36BIT_DMA_BIT,
+ MTK_GMAC_EEE_BIT,
/* MUX BITS*/
MTK_ETH_MUX_GDM1_TO_GMAC1_ESW_BIT,
@@ -1034,6 +1035,7 @@ enum mkt_eth_capabilities {
#define MTK_U3_COPHY_V2 BIT_ULL(MTK_U3_COPHY_V2_BIT)
#define MTK_SRAM BIT_ULL(MTK_SRAM_BIT)
#define MTK_36BIT_DMA BIT_ULL(MTK_36BIT_DMA_BIT)
+#define MTK_GMAC_EEE BIT_ULL(MTK_GMAC_EEE_BIT)
#define MTK_ETH_MUX_GDM1_TO_GMAC1_ESW \
BIT_ULL(MTK_ETH_MUX_GDM1_TO_GMAC1_ESW_BIT)
@@ -1117,7 +1119,7 @@ enum mkt_eth_capabilities {
#define MT7981_CAPS (MTK_GMAC1_SGMII | MTK_GMAC2_SGMII | MTK_GMAC2_GEPHY | \
MTK_MUX_GMAC12_TO_GEPHY_SGMII | MTK_QDMA | \
MTK_MUX_U3_GMAC2_TO_QPHY | MTK_U3_COPHY_V2 | \
- MTK_RSTCTRL_PPE1 | MTK_SRAM)
+ MTK_RSTCTRL_PPE1 | MTK_SRAM | MTK_GMAC_EEE)
#define MT7986_CAPS (MTK_GMAC1_SGMII | MTK_GMAC2_SGMII | \
MTK_MUX_GMAC12_TO_GEPHY_SGMII | MTK_QDMA | \
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/port.c b/drivers/net/ethernet/mellanox/mlx5/core/en/port.c
index 6049ccf475bc..a4c096a4fed2 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/port.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/port.c
@@ -557,6 +557,7 @@ int mlx5e_set_fec_mode(struct mlx5_core_dev *dev, u16 fec_policy)
u32 in[MLX5_ST_SZ_DW(pplm_reg)] = {};
int sz = MLX5_ST_SZ_BYTES(pplm_reg);
u16 fec_policy_auto = 0;
+ bool fec_set = false;
int err;
int i;
@@ -569,9 +570,6 @@ int mlx5e_set_fec_mode(struct mlx5_core_dev *dev, u16 fec_policy)
if (fec_policy >= (1 << MLX5E_FEC_LLRS_272_257_1) && !fec_50g_per_lane)
return -EOPNOTSUPP;
- if (fec_policy && !mlx5e_fec_in_caps(dev, fec_policy))
- return -EOPNOTSUPP;
-
MLX5_SET(pplm_reg, in, local_port, 1);
err = mlx5_core_access_reg(dev, in, sz, out, sz, MLX5_REG_PPLM, 0, 0);
if (err)
@@ -591,12 +589,17 @@ int mlx5e_set_fec_mode(struct mlx5_core_dev *dev, u16 fec_policy)
mlx5e_get_fec_cap_field(out, &fec_caps, i);
/* policy supported for link speed */
- if (fec_caps & conf_fec)
+ if (fec_caps & conf_fec) {
mlx5e_fec_admin_field(out, &conf_fec, 1, i);
- else
- /* set FEC to auto*/
+ fec_set = true;
+ } else {
+ /* set FEC to auto */
mlx5e_fec_admin_field(out, &fec_policy_auto, 1, i);
+ }
}
+ if (fec_policy && !fec_set)
+ return -EOPNOTSUPP;
+
return mlx5_core_access_reg(dev, out, sz, out, sz, MLX5_REG_PPLM, 0, 1);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/port.h b/drivers/net/ethernet/mellanox/mlx5/core/en/port.h
index fa2283dd383b..53dbdf77bcce 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/port.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/port.h
@@ -66,4 +66,8 @@ enum {
MLX5E_FEC_LLRS_272_257_1 = 9,
};
+#define MLX5E_FEC_RS_MASK (BIT(MLX5E_FEC_RS_528_514) | \
+ BIT(MLX5E_FEC_RS_544_514_INTERLEAVED_QUAD) | \
+ BIT(MLX5E_FEC_RS_544_514))
+
#endif
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/sample.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/sample.c
index 89490f687a9c..93c62d3f3e5b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc/sample.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc/sample.c
@@ -311,12 +311,15 @@ sample_restore_get(struct mlx5e_tc_psample *tc_psample, u32 obj_id,
static void
sample_restore_put(struct mlx5e_tc_psample *tc_psample, struct mlx5e_sample_restore *restore)
{
+ bool last;
+
mutex_lock(&tc_psample->restore_lock);
- if (--restore->count == 0)
+ last = --restore->count == 0;
+ if (last)
hash_del(&restore->hlist);
mutex_unlock(&tc_psample->restore_lock);
- if (!restore->count) {
+ if (last) {
mlx5_del_flow_rules(restore->rule);
mlx5_modify_header_dealloc(tc_psample->esw->dev, restore->modify_hdr);
kfree(restore);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_dcbnl.c b/drivers/net/ethernet/mellanox/mlx5/core/en_dcbnl.c
index 00e706e1ede1..741f75b5bfec 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_dcbnl.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_dcbnl.c
@@ -148,7 +148,7 @@ static int mlx5e_dcbnl_ieee_getets(struct net_device *netdev,
if (err)
return err;
- if (ets->tc_tx_bw[i] < MLX5E_MAX_BW_ALLOC &&
+ if (priv->dcbx.tc_tsa[i] == IEEE_8021QAZ_TSA_ETS &&
tc_group[i] == (MLX5E_LOWEST_PRIO_GROUP + 1))
is_zero_bw_ets_tc = true;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
index 112926d07634..3ed59ced0407 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
@@ -1002,9 +1002,9 @@ static u32 pplm2ethtool_fec(u_long fec_mode, unsigned long size)
return 0;
}
-#define MLX5E_ADVERTISE_SUPPORTED_FEC(mlx5_fec, ethtool_fec) \
+#define MLX5E_ADVERTISE_SUPPORTED_FEC(fec_mask, ethtool_fec) \
do { \
- if (mlx5e_fec_in_caps(dev, 1 << (mlx5_fec))) \
+ if (mlx5e_fec_in_caps(dev, fec_mask)) \
__set_bit(ethtool_fec, \
link_ksettings->link_modes.supported);\
} while (0)
@@ -1013,6 +1013,7 @@ static const u32 pplm_fec_2_ethtool_linkmodes[] = {
[MLX5E_FEC_NOFEC] = ETHTOOL_LINK_MODE_FEC_NONE_BIT,
[MLX5E_FEC_FIRECODE] = ETHTOOL_LINK_MODE_FEC_BASER_BIT,
[MLX5E_FEC_RS_528_514] = ETHTOOL_LINK_MODE_FEC_RS_BIT,
+ [MLX5E_FEC_RS_544_514_INTERLEAVED_QUAD] = ETHTOOL_LINK_MODE_FEC_RS_BIT,
[MLX5E_FEC_RS_544_514] = ETHTOOL_LINK_MODE_FEC_RS_BIT,
[MLX5E_FEC_LLRS_272_257_1] = ETHTOOL_LINK_MODE_FEC_LLRS_BIT,
};
@@ -1029,13 +1030,13 @@ static int get_fec_supported_advertised(struct mlx5_core_dev *dev,
if (err)
return (err == -EOPNOTSUPP) ? 0 : err;
- MLX5E_ADVERTISE_SUPPORTED_FEC(MLX5E_FEC_NOFEC,
+ MLX5E_ADVERTISE_SUPPORTED_FEC(BIT(MLX5E_FEC_NOFEC),
ETHTOOL_LINK_MODE_FEC_NONE_BIT);
- MLX5E_ADVERTISE_SUPPORTED_FEC(MLX5E_FEC_FIRECODE,
+ MLX5E_ADVERTISE_SUPPORTED_FEC(BIT(MLX5E_FEC_FIRECODE),
ETHTOOL_LINK_MODE_FEC_BASER_BIT);
- MLX5E_ADVERTISE_SUPPORTED_FEC(MLX5E_FEC_RS_528_514,
+ MLX5E_ADVERTISE_SUPPORTED_FEC(MLX5E_FEC_RS_MASK,
ETHTOOL_LINK_MODE_FEC_RS_BIT);
- MLX5E_ADVERTISE_SUPPORTED_FEC(MLX5E_FEC_LLRS_272_257_1,
+ MLX5E_ADVERTISE_SUPPORTED_FEC(BIT(MLX5E_FEC_LLRS_272_257_1),
ETHTOOL_LINK_MODE_FEC_LLRS_BIT);
active_fec_long = active_fec;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
index 3d544fe4e6f7..eb3160bfbb29 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
@@ -56,7 +56,6 @@
#include "lib/vxlan.h"
#define CREATE_TRACE_POINTS
#include "diag/en_rep_tracepoint.h"
-#include "diag/reporter_vnic.h"
#include "en_accel/ipsec.h"
#include "en/tc/int_port.h"
#include "en/ptp.h"
@@ -1435,51 +1434,6 @@ static unsigned int mlx5e_ul_rep_stats_grps_num(struct mlx5e_priv *priv)
return ARRAY_SIZE(mlx5e_ul_rep_stats_grps);
}
-static int
-mlx5e_rep_vnic_reporter_diagnose(struct devlink_health_reporter *reporter,
- struct devlink_fmsg *fmsg,
- struct netlink_ext_ack *extack)
-{
- struct mlx5e_rep_priv *rpriv = devlink_health_reporter_priv(reporter);
- struct mlx5_eswitch_rep *rep = rpriv->rep;
-
- mlx5_reporter_vnic_diagnose_counters(rep->esw->dev, fmsg, rep->vport,
- true);
- return 0;
-}
-
-static const struct devlink_health_reporter_ops mlx5_rep_vnic_reporter_ops = {
- .name = "vnic",
- .diagnose = mlx5e_rep_vnic_reporter_diagnose,
-};
-
-static void mlx5e_rep_vnic_reporter_create(struct mlx5e_priv *priv,
- struct devlink_port *dl_port)
-{
- struct mlx5e_rep_priv *rpriv = priv->ppriv;
- struct devlink_health_reporter *reporter;
-
- reporter = devl_port_health_reporter_create(dl_port,
- &mlx5_rep_vnic_reporter_ops,
- rpriv);
- if (IS_ERR(reporter)) {
- mlx5_core_err(priv->mdev,
- "Failed to create representor vnic reporter, err = %pe\n",
- reporter);
- return;
- }
-
- rpriv->rep_vnic_reporter = reporter;
-}
-
-static void mlx5e_rep_vnic_reporter_destroy(struct mlx5e_priv *priv)
-{
- struct mlx5e_rep_priv *rpriv = priv->ppriv;
-
- if (!IS_ERR_OR_NULL(rpriv->rep_vnic_reporter))
- devl_health_reporter_destroy(rpriv->rep_vnic_reporter);
-}
-
static const struct mlx5e_profile mlx5e_rep_profile = {
.init = mlx5e_init_rep,
.cleanup = mlx5e_cleanup_rep,
@@ -1603,10 +1557,8 @@ mlx5e_vport_vf_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep)
dl_port = mlx5_esw_offloads_devlink_port(dev->priv.eswitch,
rpriv->rep->vport);
- if (!IS_ERR(dl_port)) {
+ if (!IS_ERR(dl_port))
SET_NETDEV_DEVLINK_PORT(netdev, dl_port);
- mlx5e_rep_vnic_reporter_create(priv, dl_port);
- }
err = register_netdev(netdev);
if (err) {
@@ -1619,7 +1571,6 @@ mlx5e_vport_vf_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep)
return 0;
err_detach_netdev:
- mlx5e_rep_vnic_reporter_destroy(priv);
mlx5e_detach_netdev(netdev_priv(netdev));
err_cleanup_profile:
priv->profile->cleanup(priv);
@@ -1677,7 +1628,6 @@ mlx5e_vport_rep_unload(struct mlx5_eswitch_rep *rep)
}
unregister_netdev(netdev);
- mlx5e_rep_vnic_reporter_destroy(priv);
mlx5e_detach_netdev(priv);
priv->profile->cleanup(priv);
mlx5e_destroy_netdev(netdev);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.h b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.h
index 70640fa1ad7b..bcd7b4e814d0 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.h
@@ -118,7 +118,6 @@ struct mlx5e_rep_priv {
struct rtnl_link_stats64 prev_vf_vport_stats;
struct mlx5_flow_handle *send_to_vport_meta_rule;
struct rhashtable tc_ht;
- struct devlink_health_reporter *rep_vnic_reporter;
};
static inline
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
index e7e6db7f6bf1..cd94bb44f6ab 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
@@ -1199,50 +1199,39 @@ void mlx5e_stats_rmon_get(struct mlx5e_priv *priv,
void mlx5e_stats_ts_get(struct mlx5e_priv *priv,
struct ethtool_ts_stats *ts_stats)
{
- int i, j;
+ u16 nch = mlx5e_stats_nch_read(priv);
+ int i, tc;
- mutex_lock(&priv->state_lock);
+ ts_stats->pkts = 0;
- if (priv->tx_ptp_opened) {
- struct mlx5e_ptp *ptp = priv->channels.ptp;
+ for (i = 0; i < nch; i++) {
+ struct mlx5e_channel_stats *channel_stats =
+ priv->channel_stats[i];
- ts_stats->pkts = 0;
+ for (tc = 0; tc < priv->max_opened_tc; tc++)
+ ts_stats->pkts += channel_stats->sq[tc].timestamps;
+ }
+
+ /* Accumulate DMA and port timestamp counters so values stay monotonic
+ * across channel teardown and mode switches.
+ */
+ if (priv->tx_ptp_opened) {
+ /* Err and Lost stats are only relevant for port timestamping,
+ * as the DMA layer will always successfully timestamp packets.
+ */
ts_stats->err = 0;
ts_stats->lost = 0;
- if (!ptp)
- goto out;
-
- /* Aggregate stats across all TCs */
- for (i = 0; i < ptp->num_tc; i++) {
+ for (tc = 0; tc < priv->max_opened_tc; tc++) {
struct mlx5e_ptp_cq_stats *stats =
- ptp->ptpsq[i].cq_stats;
+ &priv->ptp_stats.cq[tc];
ts_stats->pkts += stats->cqe;
ts_stats->err += stats->abort + stats->err_cqe +
- stats->late_cqe;
+ stats->late_cqe;
ts_stats->lost += stats->lost_cqe;
}
- } else {
- /* DMA layer will always successfully timestamp packets. Other
- * counters do not make sense for this layer.
- */
- ts_stats->pkts = 0;
-
- /* Aggregate stats across all SQs */
- for (j = 0; j < priv->channels.num; j++) {
- struct mlx5e_channel *c = priv->channels.c[j];
-
- for (i = 0; i < c->num_tc; i++) {
- struct mlx5e_sq_stats *stats = c->sq[i].stats;
-
- ts_stats->pkts += stats->timestamps;
- }
- }
}
-
-out:
- mutex_unlock(&priv->state_lock);
}
#define PPORT_PHY_LAYER_OFF(c) \
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
index 6e50311faa27..2e7eac4bca3d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
@@ -4,6 +4,26 @@
#include <linux/mlx5/driver.h>
#include "eswitch.h"
#include "devlink.h"
+#include "diag/reporter_vnic.h"
+
+static int
+mlx5_esw_rep_vnic_reporter_diagnose(struct devlink_health_reporter *reporter,
+ struct devlink_fmsg *fmsg,
+ struct netlink_ext_ack *extack)
+{
+ struct mlx5_vport *vport = devlink_health_reporter_priv(reporter);
+
+ mlx5_reporter_vnic_diagnose_counters(vport->dev, fmsg, vport->vport,
+ true);
+
+ return 0;
+}
+
+static const
+struct devlink_health_reporter_ops mlx5_esw_rep_vnic_reporter_ops = {
+ .name = "vnic",
+ .diagnose = mlx5_esw_rep_vnic_reporter_diagnose,
+};
static void
mlx5_esw_get_port_parent_id(struct mlx5_core_dev *dev, struct netdev_phys_item_id *ppid)
@@ -217,6 +237,7 @@ static void mlx5_esw_devlink_port_res_unregister(struct devlink_port *dl_port)
int mlx5_esw_offloads_devlink_port_register(struct mlx5_eswitch *esw, struct mlx5_vport *vport)
{
+ struct devlink_health_reporter *reporter;
struct mlx5_core_dev *dev = esw->dev;
const struct devlink_port_ops *ops;
struct mlx5_devlink_port *dl_port;
@@ -252,6 +273,16 @@ int mlx5_esw_offloads_devlink_port_register(struct mlx5_eswitch *esw, struct mlx
mlx5_core_dbg(dev, "Failed to register port resources: %d\n",
err);
+ reporter = devl_port_health_reporter_create(
+ &dl_port->dl_port, &mlx5_esw_rep_vnic_reporter_ops,
+ vport);
+ if (IS_ERR(reporter))
+ mlx5_core_err(dev,
+ "Failed to create vnic health reporter for vport %d: %pe\n",
+ vport_num, reporter);
+ else
+ dl_port->vnic_reporter = reporter;
+
return 0;
rate_err:
@@ -266,6 +297,12 @@ void mlx5_esw_offloads_devlink_port_unregister(struct mlx5_vport *vport)
if (!vport->dl_port)
return;
dl_port = vport->dl_port;
+
+ if (dl_port->vnic_reporter) {
+ devl_health_reporter_destroy(dl_port->vnic_reporter);
+ dl_port->vnic_reporter = NULL;
+ }
+
mlx5_esw_devlink_port_res_unregister(&dl_port->dl_port);
mlx5_esw_qos_vport_update_parent(vport, NULL, NULL);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index a0e2ca87b8d8..eadc44119b1f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -1040,13 +1040,19 @@ void mlx5_esw_vport_disable(struct mlx5_eswitch *esw, struct mlx5_vport *vport)
(vport->info.ipsec_crypto_enabled || vport->info.ipsec_packet_enabled))
esw->enabled_ipsec_vf_count--;
+ /* Clear rx-mode before esw_vport_change_handle_locked(): on
+ * MLX5_VPORT_PROMISC_CHANGE it calls esw_update_vport_mc_promisc()
+ * when vport->allmulti_rule is set, repopulating mc_list with FDB
+ * rules that dangle once the FDB is destroyed. NULL allmulti_rule
+ * here skips that path.
+ */
+ esw_apply_vport_rx_mode(esw, vport, false, false);
/* We don't assume VFs will cleanup after themselves.
* Calling vport change handler while vport is disabled will cleanup
* the vport resources.
*/
esw_vport_change_handle_locked(vport);
vport->enabled_events = 0;
- esw_apply_vport_rx_mode(esw, vport, false, false);
esw_vport_cleanup(esw, vport);
esw->enabled_vports--;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index fea72b1dedab..e8817d45763a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -189,6 +189,7 @@ struct mlx5_vport;
struct mlx5_devlink_port {
struct devlink_port dl_port;
struct mlx5_vport *vport;
+ struct devlink_health_reporter *vnic_reporter;
};
static inline void mlx5_devlink_port_init(struct mlx5_devlink_port *dl_port,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads_termtbl.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads_termtbl.c
index 19f65d4c4def..d43f07360159 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads_termtbl.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads_termtbl.c
@@ -163,12 +163,15 @@ void
mlx5_eswitch_termtbl_put(struct mlx5_eswitch *esw,
struct mlx5_termtbl_handle *tt)
{
+ bool last;
+
mutex_lock(&esw->offloads.termtbl_mutex);
- if (--tt->ref_count == 0)
+ last = (--tt->ref_count == 0);
+ if (last)
hash_del(&tt->termtbl_hlist);
mutex_unlock(&esw->offloads.termtbl_mutex);
- if (!tt->ref_count) {
+ if (last) {
mlx5_del_flow_rules(tt->rule);
mlx5_destroy_flow_table(tt->termtbl);
kfree(tt);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
index 2285c889c215..c655f6e32e9b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
@@ -63,14 +63,15 @@ static int get_port_sel_mode(enum mlx5_lag_mode mode, unsigned long flags)
return MLX5_LAG_PORT_SELECT_MODE_QUEUE_AFFINITY;
}
-static u8 lag_active_port_bits(struct mlx5_lag *ldev)
+static u8 lag_active_port_bits(struct mlx5_lag *ldev,
+ struct lag_tracker *tracker)
{
u8 enabled_ports[MLX5_MAX_PORTS] = {};
u8 active_port = 0;
int num_enabled;
int idx;
- mlx5_infer_tx_enabled(&ldev->tracker, ldev, enabled_ports,
+ mlx5_infer_tx_enabled(tracker, ldev, enabled_ports,
&num_enabled);
for (idx = 0; idx < num_enabled; idx++)
active_port |= BIT_MASK(enabled_ports[idx]);
@@ -79,7 +80,8 @@ static u8 lag_active_port_bits(struct mlx5_lag *ldev)
}
static int mlx5_cmd_create_lag(struct mlx5_core_dev *dev, struct mlx5_lag *ldev,
- int mode, unsigned long flags)
+ struct lag_tracker *tracker, int mode,
+ unsigned long flags)
{
bool fdb_sel_mode = test_bit(MLX5_LAG_MODE_FLAG_FDB_SEL_MODE_NATIVE,
&flags);
@@ -108,7 +110,7 @@ static int mlx5_cmd_create_lag(struct mlx5_core_dev *dev, struct mlx5_lag *ldev,
break;
MLX5_SET(lagc, lag_ctx, active_port,
- lag_active_port_bits(mlx5_lag_dev(dev)));
+ lag_active_port_bits(ldev, tracker));
break;
default:
break;
@@ -787,7 +789,8 @@ static int mlx5_cmd_modify_active_port(struct mlx5_core_dev *dev, u8 ports)
return mlx5_cmd_exec_in(dev, modify_lag, in);
}
-static int _mlx5_modify_lag(struct mlx5_lag *ldev, u8 *ports)
+static int _mlx5_modify_lag(struct mlx5_lag *ldev,
+ struct lag_tracker *tracker, u8 *ports)
{
int idx = mlx5_lag_get_dev_index_by_seq(ldev, MLX5_LAG_P1);
struct mlx5_core_dev *dev0;
@@ -804,7 +807,7 @@ static int _mlx5_modify_lag(struct mlx5_lag *ldev, u8 *ports)
!MLX5_CAP_PORT_SELECTION(dev0, port_select_flow_table_bypass))
return ret;
- active_ports = lag_active_port_bits(ldev);
+ active_ports = lag_active_port_bits(ldev, tracker);
return mlx5_cmd_modify_active_port(dev0, active_ports);
}
@@ -868,7 +871,7 @@ void mlx5_modify_lag(struct mlx5_lag *ldev,
idx = i * ldev->buckets + j;
if (ports[idx] == ldev->v2p_map[idx])
continue;
- err = _mlx5_modify_lag(ldev, ports);
+ err = _mlx5_modify_lag(ldev, tracker, ports);
if (err) {
mlx5_core_err(dev0,
"Failed to modify LAG (%d)\n",
@@ -976,7 +979,7 @@ static int mlx5_create_lag(struct mlx5_lag *ldev,
mlx5_core_info(dev0, "shared_fdb:%d mode:%s\n",
shared_fdb, mlx5_get_str_port_sel_mode(mode, flags));
- err = mlx5_cmd_create_lag(dev0, ldev, mode, flags);
+ err = mlx5_cmd_create_lag(dev0, ldev, tracker, mode, flags);
if (err) {
mlx5_core_err(dev0,
"Failed to create LAG (%d)\n",
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.c
index 9939749c47bc..9c5862f4e16a 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_ptp.c
@@ -572,6 +572,38 @@ mlxsw_sp1_ptp_unmatched_remove(struct mlxsw_sp *mlxsw_sp,
mlxsw_sp1_ptp_unmatched_ht_params);
}
+/* mlxsw_sp1_ptp_packet_finish() is reached both from the NAPI poll context
+ * (mlxsw_sp1_ptp_got_packet(), mlxsw_sp1_ptp_got_piece() and
+ * mlxsw_sp1_packet_timestamp()) and from process context, by way of the GC
+ * workqueue (mlxsw_sp1_ptp_ht_gc_collect() ->
+ * mlxsw_sp1_ptp_unmatched_finish()).
+ *
+ * mlxsw_sp_rx_listener_no_mark_func() ends in napi_gro_receive(), using the
+ * NAPI pointer that was placed in the SKB control block when the trapped
+ * packet was received in the NAPI context. That pointer may only be used
+ * from its own poll context, which this call site cannot guarantee.
+ *
+ * netif_receive_skb(), unlike napi_gro_receive(), can be called from outside
+ * of the NAPI instance's poll context. RX stats accounting and the skb->dev
+ * assignment are still preserved; the only change is the delivery call.
+ */
+static void mlxsw_sp1_ptp_rx_finish(struct mlxsw_sp_port *mlxsw_sp_port,
+ struct sk_buff *skb)
+{
+ struct mlxsw_sp_port_pcpu_stats *pcpu_stats;
+
+ skb->dev = mlxsw_sp_port->dev;
+
+ pcpu_stats = this_cpu_ptr(mlxsw_sp_port->pcpu_stats);
+ u64_stats_update_begin(&pcpu_stats->syncp);
+ pcpu_stats->rx_packets++;
+ pcpu_stats->rx_bytes += skb->len;
+ u64_stats_update_end(&pcpu_stats->syncp);
+
+ skb->protocol = eth_type_trans(skb, skb->dev);
+ netif_receive_skb(skb);
+}
+
/* This function is called in the following scenarios:
*
* 1) When a packet is matched with its timestamp.
@@ -600,7 +632,7 @@ static void mlxsw_sp1_ptp_packet_finish(struct mlxsw_sp *mlxsw_sp,
if (ingress) {
if (hwtstamps)
*skb_hwtstamps(skb) = *hwtstamps;
- mlxsw_sp_rx_listener_no_mark_func(skb, local_port, mlxsw_sp);
+ mlxsw_sp1_ptp_rx_finish(mlxsw_sp_port, skb);
} else {
/* skb_tstamp_tx() allows hwtstamps to be NULL. */
skb_tstamp_tx(skb, hwtstamps);
diff --git a/drivers/net/ethernet/micrel/ks8851_common.c b/drivers/net/ethernet/micrel/ks8851_common.c
index 4afbb40bc0e4..d49f281c7867 100644
--- a/drivers/net/ethernet/micrel/ks8851_common.c
+++ b/drivers/net/ethernet/micrel/ks8851_common.c
@@ -143,9 +143,6 @@ static int ks8851_write_mac_addr(struct net_device *dev)
ks8851_wrreg16(ks, KS_MAR(i), val);
}
- if (!netif_running(dev))
- ks8851_set_powermode(ks, PMECR_PM_SOFTDOWN);
-
ks8851_unlock(ks);
return 0;
@@ -478,8 +475,7 @@ static int ks8851_net_open(struct net_device *dev)
* @dev: The device being closed.
*
* Called to close down a network device which has been active. Cancel any
- * work, shutdown the RX and TX process and then place the chip into a low
- * power state whilst it is not being used.
+ * work and shutdown the RX and TX process.
*/
static int ks8851_net_stop(struct net_device *dev)
{
@@ -506,8 +502,6 @@ static int ks8851_net_stop(struct net_device *dev)
/* shutdown TX process */
ks8851_wrreg16(ks, KS_TXCR, 0x0000);
- /* set powermode to soft power down to save power */
- ks8851_set_powermode(ks, PMECR_PM_SOFTDOWN);
ks8851_unlock(ks);
/* ensure any queued tx buffers are dumped */
diff --git a/drivers/net/ethernet/microsoft/mana/mana_bpf.c b/drivers/net/ethernet/microsoft/mana/mana_bpf.c
index 53308e139cbe..5c9961ee9747 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_bpf.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_bpf.c
@@ -208,6 +208,7 @@ static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,
if (err) {
NL_SET_ERR_MSG_MOD(extack,
"XDP: Insufficient memory for tx/rx re-config");
+ apc->bpf_prog = old_prog;
return err;
}
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index 92bb55935c1c..8fb29dcddf8e 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -2756,6 +2756,10 @@ static int mana_alloc_rx_wqe(struct mana_port_context *apc,
*cq_size += COMP_ENTRY_SIZE;
}
+ /* Reserve an extra slot for Fence completion
+ * event (CQE_RX_OBJECT_FENCE) in case RX CQ is full.
+ */
+ *cq_size += COMP_ENTRY_SIZE;
return 0;
}
@@ -2850,7 +2854,7 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc,
goto out;
rq_size = MANA_PAGE_ALIGN(rq_size);
- cq_size = MANA_PAGE_ALIGN(cq_size);
+ cq_size = MANA_PAGE_ALIGN(roundup_pow_of_two(cq_size));
/* Create RQ */
memset(&spec, 0, sizeof(spec));
@@ -3728,7 +3732,8 @@ static void mana_rdma_service_handle(struct work_struct *work)
struct device *dev = gd->gdma_context->dev;
int ret;
- if (READ_ONCE(gd->rdma_teardown))
+ /* Pairs with the smp_store_release() in mana_rdma_probe(). */
+ if (smp_load_acquire(&gd->rdma_teardown))
goto out;
switch (serv_work->event) {
@@ -4024,6 +4029,21 @@ int mana_rdma_probe(struct gdma_dev *gd)
if (err)
return err;
+ /* Clear the state left by a previous mana_rdma_remove() so servicing
+ * events are handled again after a reset cycle.
+ */
+ gd->is_suspended = false;
+
+ /* Publish is_suspended before re-opening the gate, so the handler
+ * cannot observe an open gate with a stale is_suspended. Pairs
+ * with the smp_load_acquire() in mana_rdma_service_handle(). This
+ * matters on the reset path, where mana_rdma_remove() closed the
+ * gate and drained the workqueue; on the initial probe path the
+ * gate was never closed and both flags are already clear. It does
+ * not order gd->adev, which add_adev() publishes below.
+ */
+ smp_store_release(&gd->rdma_teardown, false);
+
err = add_adev(gd, "rdma");
if (err)
mana_gd_deregister_device(gd);
diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c b/drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c
index a2a89d48e3ca..4e83637715e0 100644
--- a/drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c
+++ b/drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c
@@ -1421,7 +1421,8 @@ static int nfp_net_get_fs_rule(struct nfp_net *nn, struct ethtool_rxnfc *cmd)
return -ENOENT;
}
-static int nfp_net_get_fs_loc(struct nfp_net *nn, u32 *rule_locs)
+static int nfp_net_get_fs_loc(struct nfp_net *nn, struct ethtool_rxnfc *cmd,
+ u32 *rule_locs)
{
struct nfp_fs_entry *entry;
u32 count = 0;
@@ -1429,8 +1430,12 @@ static int nfp_net_get_fs_loc(struct nfp_net *nn, u32 *rule_locs)
if (!(nn->cap_w1 & NFP_NET_CFG_CTRL_FLOW_STEER))
return -EOPNOTSUPP;
- list_for_each_entry(entry, &nn->fs.list, node)
+ list_for_each_entry(entry, &nn->fs.list, node) {
+ if (count == cmd->rule_cnt)
+ return -EMSGSIZE;
rule_locs[count++] = entry->loc;
+ }
+ cmd->rule_cnt = count;
return 0;
}
@@ -1455,7 +1460,7 @@ static int nfp_net_get_rxnfc(struct net_device *netdev,
return nfp_net_get_fs_rule(nn, cmd);
case ETHTOOL_GRXCLSRLALL:
cmd->data = NFP_FS_MAX_ENTRY;
- return nfp_net_get_fs_loc(nn, rule_locs);
+ return nfp_net_get_fs_loc(nn, cmd, rule_locs);
default:
return -EOPNOTSUPP;
}
@@ -1698,8 +1703,14 @@ static int nfp_net_fs_add(struct nfp_net *nn, struct ethtool_rxnfc *cmd)
nn->fs.count--;
err = nfp_net_fs_add_hw(nn, new);
- if (err)
+ if (err) {
+ /* mbox broken, adding the old rule back will
+ * likely also fail.
+ */
+ list_del(&entry->node);
+ kfree(entry);
goto err;
+ }
nn->fs.count++;
list_replace(&entry->node, &new->node);
diff --git a/drivers/net/ethernet/oa_tc6.c b/drivers/net/ethernet/oa_tc6.c
index 0727d53345a3..f552165e503e 100644
--- a/drivers/net/ethernet/oa_tc6.c
+++ b/drivers/net/ethernet/oa_tc6.c
@@ -12,46 +12,6 @@
#include <linux/phy.h>
#include <linux/oa_tc6.h>
-/* OPEN Alliance TC6 registers */
-/* Standard Capabilities Register */
-#define OA_TC6_REG_STDCAP 0x0002
-#define STDCAP_DIRECT_PHY_REG_ACCESS BIT(8)
-
-/* Reset Control and Status Register */
-#define OA_TC6_REG_RESET 0x0003
-#define RESET_SWRESET BIT(0) /* Software Reset */
-
-/* Configuration Register #0 */
-#define OA_TC6_REG_CONFIG0 0x0004
-#define CONFIG0_SYNC BIT(15)
-#define CONFIG0_ZARFE_ENABLE BIT(12)
-
-/* Status Register #0 */
-#define OA_TC6_REG_STATUS0 0x0008
-#define STATUS0_RESETC BIT(6) /* Reset Complete */
-#define STATUS0_HEADER_ERROR BIT(5)
-#define STATUS0_LOSS_OF_FRAME_ERROR BIT(4)
-#define STATUS0_RX_BUFFER_OVERFLOW_ERROR BIT(3)
-#define STATUS0_TX_PROTOCOL_ERROR BIT(0)
-
-/* Buffer Status Register */
-#define OA_TC6_REG_BUFFER_STATUS 0x000B
-#define BUFFER_STATUS_TX_CREDITS_AVAILABLE GENMASK(15, 8)
-#define BUFFER_STATUS_RX_CHUNKS_AVAILABLE GENMASK(7, 0)
-
-/* Interrupt Mask Register #0 */
-#define OA_TC6_REG_INT_MASK0 0x000C
-#define INT_MASK0_HEADER_ERR_MASK BIT(5)
-#define INT_MASK0_LOSS_OF_FRAME_ERR_MASK BIT(4)
-#define INT_MASK0_RX_BUFFER_OVERFLOW_ERR_MASK BIT(3)
-#define INT_MASK0_TX_PROTOCOL_ERR_MASK BIT(0)
-#define INT_MASK0_ALL_INTERRUPTS (GENMASK(5, 0) | \
- GENMASK(12, 7))
-
-/* PHY Clause 22 registers base address and mask */
-#define OA_TC6_PHY_STD_REG_ADDR_BASE 0xFF00
-#define OA_TC6_PHY_STD_REG_ADDR_MASK 0x1F
-
/* Control command header */
#define OA_TC6_CTRL_HEADER_DATA_NOT_CTRL BIT(31)
#define OA_TC6_CTRL_HEADER_WRITE_NOT_READ BIT(29)
@@ -81,23 +41,17 @@
#define OA_TC6_DATA_FOOTER_END_BYTE_OFFSET GENMASK(13, 8)
#define OA_TC6_DATA_FOOTER_TX_CREDITS GENMASK(5, 1)
-/* PHY – Clause 45 registers memory map selector (MMS) as per table 6 in the
- * OPEN Alliance specification.
- */
-#define OA_TC6_PHY_C45_PCS_MMS2 2 /* MMD 3 */
-#define OA_TC6_PHY_C45_PMA_PMD_MMS3 3 /* MMD 1 */
-#define OA_TC6_PHY_C45_VS_PLCA_MMS4 4 /* MMD 31 */
-#define OA_TC6_PHY_C45_AUTO_NEG_MMS5 5 /* MMD 7 */
-#define OA_TC6_PHY_C45_POWER_UNIT_MMS6 6 /* MMD 13 */
-
+#define OA_TC6_CTRL_PROT_REPLY_SIZE 4
#define OA_TC6_CTRL_HEADER_SIZE 4
#define OA_TC6_CTRL_REG_VALUE_SIZE 4
#define OA_TC6_CTRL_IGNORED_SIZE 4
#define OA_TC6_CTRL_MAX_REGISTERS 128
-#define OA_TC6_CTRL_SPI_BUF_SIZE (OA_TC6_CTRL_HEADER_SIZE +\
- (OA_TC6_CTRL_MAX_REGISTERS *\
- OA_TC6_CTRL_REG_VALUE_SIZE) +\
- OA_TC6_CTRL_IGNORED_SIZE)
+#define OA_TC6_CTRL_SPI_BUF_SIZE (OA_TC6_CTRL_HEADER_SIZE +\
+ (OA_TC6_CTRL_MAX_REGISTERS *\
+ (OA_TC6_CTRL_REG_VALUE_SIZE +\
+ OA_TC6_CTRL_PROT_REPLY_SIZE)) +\
+ OA_TC6_CTRL_IGNORED_SIZE)
+
#define OA_TC6_CHUNK_PAYLOAD_SIZE 64
#define OA_TC6_DATA_HEADER_SIZE 4
#define OA_TC6_CHUNK_SIZE (OA_TC6_DATA_HEADER_SIZE +\
@@ -127,9 +81,10 @@ struct oa_tc6 {
u16 spi_data_tx_buf_offset;
u16 tx_credits;
u8 rx_chunks_available;
- bool rx_buf_overflow;
+ bool wait_until_start_valid;
bool int_flag;
bool disable_traffic;
+ bool prot_ctrl;
};
enum oa_tc6_header_type {
@@ -213,25 +168,36 @@ static void oa_tc6_update_ctrl_write_data(struct oa_tc6 *tc6, u32 value[],
{
__be32 *tx_buf = tc6->spi_ctrl_tx_buf + OA_TC6_CTRL_HEADER_SIZE;
- for (int i = 0; i < length; i++)
+ for (int i = 0; i < length; i++) {
*tx_buf++ = cpu_to_be32(value[i]);
+ if (tc6->prot_ctrl)
+ *tx_buf++ = cpu_to_be32(~value[i]);
+ }
}
-static u16 oa_tc6_calculate_ctrl_buf_size(u8 length)
+static u16 oa_tc6_calculate_ctrl_buf_size(u8 length, bool ctrl_prot)
{
+ u32 reply_size = OA_TC6_CTRL_REG_VALUE_SIZE;
+
+ if (ctrl_prot)
+ reply_size += OA_TC6_CTRL_PROT_REPLY_SIZE;
+
/* Control command consists 4 bytes header + 4 bytes register value for
- * each register + 4 bytes ignored value.
+ * each register (+ 4 bytes for the register value complement in case
+ * protected mode is used) + 4 bytes ignored value.
*/
- return OA_TC6_CTRL_HEADER_SIZE + OA_TC6_CTRL_REG_VALUE_SIZE * length +
+ return OA_TC6_CTRL_HEADER_SIZE + reply_size * length +
OA_TC6_CTRL_IGNORED_SIZE;
}
static void oa_tc6_prepare_ctrl_spi_buf(struct oa_tc6 *tc6, u32 address,
u32 value[], u8 length,
- enum oa_tc6_register_op reg_op)
+ enum oa_tc6_register_op reg_op,
+ u16 buf_size)
{
__be32 *tx_buf = tc6->spi_ctrl_tx_buf;
+ memset(tx_buf, 0, buf_size);
*tx_buf = oa_tc6_prepare_ctrl_header(address, length, reg_op);
if (reg_op == OA_TC6_CTRL_REG_WRITE)
@@ -254,10 +220,12 @@ static int oa_tc6_check_ctrl_write_reply(struct oa_tc6 *tc6, u8 size)
return 0;
}
-static int oa_tc6_check_ctrl_read_reply(struct oa_tc6 *tc6, u8 size)
+static int oa_tc6_check_ctrl_read_reply(struct oa_tc6 *tc6, u8 length)
{
- u32 *rx_buf = tc6->spi_ctrl_rx_buf + OA_TC6_CTRL_IGNORED_SIZE;
- u32 *tx_buf = tc6->spi_ctrl_tx_buf;
+ __be32 *rx_buf = tc6->spi_ctrl_rx_buf + OA_TC6_CTRL_IGNORED_SIZE;
+ __be32 *tx_buf = tc6->spi_ctrl_tx_buf;
+ u32 complement;
+ u32 reply;
/* The echoed control read header must match with the one that was
* transmitted.
@@ -265,6 +233,20 @@ static int oa_tc6_check_ctrl_read_reply(struct oa_tc6 *tc6, u8 size)
if (*tx_buf != *rx_buf)
return -EPROTO;
+ if (tc6->prot_ctrl) {
+ /* Skip past the echoed header to the value/complement pairs */
+ rx_buf += 1;
+ for (int i = 0; i < length; i++) {
+ reply = be32_to_cpu(rx_buf[0]);
+ complement = be32_to_cpu(rx_buf[1]);
+
+ if (complement != ~reply)
+ return -EPROTO;
+
+ rx_buf += 2;
+ }
+ }
+
return 0;
}
@@ -274,8 +256,13 @@ static void oa_tc6_copy_ctrl_read_data(struct oa_tc6 *tc6, u32 value[],
__be32 *rx_buf = tc6->spi_ctrl_rx_buf + OA_TC6_CTRL_IGNORED_SIZE +
OA_TC6_CTRL_HEADER_SIZE;
- for (int i = 0; i < length; i++)
+ for (int i = 0; i < length; i++) {
value[i] = be32_to_cpu(*rx_buf++);
+
+ /* skip complement word */
+ if (tc6->prot_ctrl)
+ rx_buf++;
+ }
}
static int oa_tc6_perform_ctrl(struct oa_tc6 *tc6, u32 address, u32 value[],
@@ -284,10 +271,10 @@ static int oa_tc6_perform_ctrl(struct oa_tc6 *tc6, u32 address, u32 value[],
u16 size;
int ret;
- /* Prepare control command and copy to SPI control buffer */
- oa_tc6_prepare_ctrl_spi_buf(tc6, address, value, length, reg_op);
+ size = oa_tc6_calculate_ctrl_buf_size(length, tc6->prot_ctrl);
- size = oa_tc6_calculate_ctrl_buf_size(length);
+ /* Prepare control command and copy to SPI control buffer */
+ oa_tc6_prepare_ctrl_spi_buf(tc6, address, value, length, reg_op, size);
/* Perform SPI transfer */
ret = oa_tc6_spi_transfer(tc6, OA_TC6_CTRL_HEADER, size);
@@ -302,7 +289,7 @@ static int oa_tc6_perform_ctrl(struct oa_tc6 *tc6, u32 address, u32 value[],
return oa_tc6_check_ctrl_write_reply(tc6, size);
/* Check echoed/received control read command reply for errors */
- ret = oa_tc6_check_ctrl_read_reply(tc6, size);
+ ret = oa_tc6_check_ctrl_read_reply(tc6, length);
if (ret)
return ret;
@@ -408,7 +395,7 @@ static int oa_tc6_check_phy_reg_direct_access_capability(struct oa_tc6 *tc6)
if (ret)
return ret;
- if (!(regval & STDCAP_DIRECT_PHY_REG_ACCESS))
+ if (!(regval & OA_TC6_STDCAP_DIRECT_PHY_REG_ACCESS))
return -ENODEV;
return 0;
@@ -423,7 +410,7 @@ static int oa_tc6_mdiobus_read(struct mii_bus *bus, int addr, int regnum)
{
struct oa_tc6 *tc6 = bus->priv;
u32 regval;
- bool ret;
+ int ret;
ret = oa_tc6_read_register(tc6, OA_TC6_PHY_STD_REG_ADDR_BASE |
(regnum & OA_TC6_PHY_STD_REG_ADDR_MASK),
@@ -601,7 +588,7 @@ static int oa_tc6_read_status0(struct oa_tc6 *tc6)
static int oa_tc6_sw_reset_macphy(struct oa_tc6 *tc6)
{
- u32 regval = RESET_SWRESET;
+ u32 regval = OA_TC6_RESET_SWRESET;
int ret;
ret = oa_tc6_write_register(tc6, OA_TC6_REG_RESET, regval);
@@ -610,7 +597,7 @@ static int oa_tc6_sw_reset_macphy(struct oa_tc6 *tc6)
/* Poll for soft reset complete for every 1ms until 1s timeout */
ret = readx_poll_timeout(oa_tc6_read_status0, tc6, regval,
- regval & STATUS0_RESETC,
+ regval & OA_TC6_STATUS0_RESETC,
STATUS0_RESETC_POLL_DELAY,
STATUS0_RESETC_POLL_TIMEOUT);
if (ret)
@@ -629,10 +616,10 @@ static int oa_tc6_unmask_macphy_error_interrupts(struct oa_tc6 *tc6)
if (ret)
return ret;
- regval &= ~(INT_MASK0_TX_PROTOCOL_ERR_MASK |
- INT_MASK0_RX_BUFFER_OVERFLOW_ERR_MASK |
- INT_MASK0_LOSS_OF_FRAME_ERR_MASK |
- INT_MASK0_HEADER_ERR_MASK);
+ regval &= ~(OA_TC6_INT_MASK0_TX_PROTOCOL_ERR_MASK |
+ OA_TC6_INT_MASK0_RX_BUFFER_OVERFLOW_ERR_MASK |
+ OA_TC6_INT_MASK0_LOSS_OF_FRAME_ERR_MASK |
+ OA_TC6_INT_MASK0_HEADER_ERR_MASK);
return oa_tc6_write_register(tc6, OA_TC6_REG_INT_MASK0, regval);
}
@@ -647,11 +634,31 @@ static int oa_tc6_enable_data_transfer(struct oa_tc6 *tc6)
return ret;
/* Enable configuration synchronization for data transfer */
- value |= CONFIG0_SYNC;
+ value |= OA_TC6_CONFIG0_SYNC;
return oa_tc6_write_register(tc6, OA_TC6_REG_CONFIG0, value);
}
+/* Called when a frame that is meant to be transmitted, is dropped. */
+static void oa_tc6_drop_tx_skb(struct oa_tc6 *tc6, struct sk_buff *skb)
+{
+ if (skb) {
+ tc6->netdev->stats.tx_dropped++;
+ dev_kfree_skb_any(skb);
+ }
+}
+
+static struct sk_buff *oa_tc6_detach_waiting_tx_skb(struct oa_tc6 *tc6)
+{
+ struct sk_buff *skb;
+
+ lockdep_assert_held(&tc6->tx_skb_lock);
+ skb = tc6->waiting_tx_skb;
+ tc6->waiting_tx_skb = NULL;
+
+ return skb;
+}
+
static void oa_tc6_cleanup_ongoing_rx_skb(struct oa_tc6 *tc6)
{
if (tc6->rx_skb) {
@@ -663,39 +670,59 @@ static void oa_tc6_cleanup_ongoing_rx_skb(struct oa_tc6 *tc6)
static void oa_tc6_cleanup_ongoing_tx_skb(struct oa_tc6 *tc6)
{
- if (tc6->ongoing_tx_skb) {
- tc6->netdev->stats.tx_dropped++;
- kfree_skb(tc6->ongoing_tx_skb);
- tc6->ongoing_tx_skb = NULL;
- }
+ oa_tc6_drop_tx_skb(tc6, tc6->ongoing_tx_skb);
+ tc6->ongoing_tx_skb = NULL;
}
static void oa_tc6_cleanup_waiting_tx_skb(struct oa_tc6 *tc6)
{
- if (tc6->waiting_tx_skb) {
- tc6->netdev->stats.tx_dropped++;
- kfree_skb(tc6->waiting_tx_skb);
- tc6->waiting_tx_skb = NULL;
- }
+ struct sk_buff *skb;
+
+ spin_lock_bh(&tc6->tx_skb_lock);
+ skb = oa_tc6_detach_waiting_tx_skb(tc6);
+ spin_unlock_bh(&tc6->tx_skb_lock);
+
+ oa_tc6_drop_tx_skb(tc6, skb);
}
-static void oa_tc6_free_pending_skbs(struct oa_tc6 *tc6)
+static void oa_tc6_free_ongoing_skbs(struct oa_tc6 *tc6)
{
oa_tc6_cleanup_ongoing_tx_skb(tc6);
oa_tc6_cleanup_ongoing_rx_skb(tc6);
+}
+
+static void oa_tc6_free_pending_skbs(struct oa_tc6 *tc6)
+{
+ oa_tc6_free_ongoing_skbs(tc6);
oa_tc6_cleanup_waiting_tx_skb(tc6);
}
+static void oa_tc6_look_for_new_frame(struct oa_tc6 *tc6)
+{
+ tc6->wait_until_start_valid = true;
+ oa_tc6_cleanup_ongoing_rx_skb(tc6);
+}
+
/* If the failure is at SPI interface level, masking and clearing
* the interrupt of the device won't work. Since SPI interrupt is
* disabled, it should stop the repeated interrupts.
*/
static void oa_tc6_disable_traffic(struct oa_tc6 *tc6)
{
- u32 regval = INT_MASK0_ALL_INTERRUPTS;
+ u32 regval = OA_TC6_INT_MASK0_ALL_INTERRUPTS;
+ struct sk_buff *skb;
+ spin_lock_bh(&tc6->tx_skb_lock);
tc6->disable_traffic = true;
- oa_tc6_free_pending_skbs(tc6);
+ skb = oa_tc6_detach_waiting_tx_skb(tc6);
+ spin_unlock_bh(&tc6->tx_skb_lock);
+
+ /* disable_traffic, when set, is a point of no return to
+ * working state. Keeping the TX queues disabled.
+ */
+ netif_tx_disable(tc6->netdev);
+ oa_tc6_drop_tx_skb(tc6, skb);
+ oa_tc6_free_ongoing_skbs(tc6);
oa_tc6_write_register(tc6, OA_TC6_REG_INT_MASK0, regval);
oa_tc6_read_register(tc6, OA_TC6_REG_STATUS0, ®val);
oa_tc6_write_register(tc6, OA_TC6_REG_STATUS0, regval);
@@ -714,6 +741,13 @@ static int oa_tc6_process_extended_status(struct oa_tc6 *tc6)
return ret;
}
+ /* This function is called for each chunk received in a given SPI
+ * transaction. In case, extended status bit is set in more than
+ * one chunk, skip the write, if status0 is already cleared.
+ */
+ if (!value)
+ return 0;
+
/* Clear the error interrupts status */
ret = oa_tc6_write_register(tc6, OA_TC6_REG_STATUS0, value);
if (ret) {
@@ -722,25 +756,24 @@ static int oa_tc6_process_extended_status(struct oa_tc6 *tc6)
return ret;
}
- if (FIELD_GET(STATUS0_RX_BUFFER_OVERFLOW_ERROR, value)) {
- tc6->rx_buf_overflow = true;
- oa_tc6_cleanup_ongoing_rx_skb(tc6);
+ if (FIELD_GET(OA_TC6_STATUS0_RX_BUFFER_OVERFLOW_ERROR, value)) {
+ oa_tc6_look_for_new_frame(tc6);
net_err_ratelimited("%s: Receive buffer overflow error\n",
tc6->netdev->name);
return -EAGAIN;
}
- if (FIELD_GET(STATUS0_TX_PROTOCOL_ERROR, value)) {
+ if (FIELD_GET(OA_TC6_STATUS0_TX_PROTOCOL_ERROR, value)) {
netdev_err(tc6->netdev, "Transmit protocol error\n");
return -ENODEV;
}
/* TODO: Currently loss of frame and header errors are treated as
* non-recoverable errors. They will be handled in the next version.
*/
- if (FIELD_GET(STATUS0_LOSS_OF_FRAME_ERROR, value)) {
+ if (FIELD_GET(OA_TC6_STATUS0_LOSS_OF_FRAME_ERROR, value)) {
netdev_err(tc6->netdev, "Loss of frame error\n");
return -ENODEV;
}
- if (FIELD_GET(STATUS0_HEADER_ERROR, value)) {
+ if (FIELD_GET(OA_TC6_STATUS0_HEADER_ERROR, value)) {
netdev_err(tc6->netdev, "Header error\n");
return -ENODEV;
}
@@ -750,6 +783,8 @@ static int oa_tc6_process_extended_status(struct oa_tc6 *tc6)
static int oa_tc6_process_rx_chunk_footer(struct oa_tc6 *tc6, u32 footer)
{
+ int ret = 0;
+
/* Process rx chunk footer for the following,
* 1. tx credits
* 2. errors if any from MAC-PHY
@@ -760,9 +795,11 @@ static int oa_tc6_process_rx_chunk_footer(struct oa_tc6 *tc6, u32 footer)
footer);
if (FIELD_GET(OA_TC6_DATA_FOOTER_EXTENDED_STS, footer)) {
- int ret = oa_tc6_process_extended_status(tc6);
-
- if (ret)
+ ret = oa_tc6_process_extended_status(tc6);
+ /* EAGAIN error is recoverable. Move on to check
+ * HEADER and SYNC errors before returning.
+ */
+ if (ret && ret != -EAGAIN)
return ret;
}
@@ -780,7 +817,7 @@ static int oa_tc6_process_rx_chunk_footer(struct oa_tc6 *tc6, u32 footer)
return -ENODEV;
}
- return 0;
+ return ret;
}
static void oa_tc6_submit_rx_skb(struct oa_tc6 *tc6)
@@ -805,13 +842,35 @@ static void oa_tc6_submit_rx_skb(struct oa_tc6 *tc6)
tc6->rx_skb = NULL;
}
-static void oa_tc6_update_rx_skb(struct oa_tc6 *tc6, u8 *payload, u8 length)
+/* On oversubscribed traffic condition, particularly with overwhelming rx
+ * buffer overflow errors, there could be data chunk loss. If tail + length
+ * goes beyond end pointer, that is an indication that the data chunk with
+ * end_valid bit is lost. Time to look for a data chunk with start_valid bit.
+ *
+ * If rx_skb is NULL, it is time to start looking for data chunk with
+ * start_bit.
+ */
+static int oa_tc6_update_rx_skb(struct oa_tc6 *tc6, u8 *payload, u8 length)
{
+ if (!tc6->rx_skb ||
+ skb_tailroom(tc6->rx_skb) < length) {
+ oa_tc6_look_for_new_frame(tc6);
+ return -EAGAIN;
+ }
+
memcpy(skb_put(tc6->rx_skb, length), payload, length);
+ return 0;
}
+/* On overwhelming rx buffer overflow errors, due to data chunk loss, it is
+ * possible that we get two data chunks with start_valid bit set, without
+ * end_valid bit set in between. In this case, rx_skb would have a valid
+ * buffer pointer. We should release, if a valid pointer is found before
+ * allocating a new one.
+ */
static int oa_tc6_allocate_rx_skb(struct oa_tc6 *tc6)
{
+ oa_tc6_cleanup_ongoing_rx_skb(tc6);
tc6->rx_skb = netdev_alloc_skb_ip_align(tc6->netdev, tc6->netdev->mtu +
ETH_HLEN + ETH_FCS_LEN);
if (!tc6->rx_skb) {
@@ -831,7 +890,9 @@ static int oa_tc6_prcs_complete_rx_frame(struct oa_tc6 *tc6, u8 *payload,
if (ret)
return ret;
- oa_tc6_update_rx_skb(tc6, payload, size);
+ ret = oa_tc6_update_rx_skb(tc6, payload, size);
+ if (ret)
+ return ret;
oa_tc6_submit_rx_skb(tc6);
@@ -846,22 +907,24 @@ static int oa_tc6_prcs_rx_frame_start(struct oa_tc6 *tc6, u8 *payload, u16 size)
if (ret)
return ret;
- oa_tc6_update_rx_skb(tc6, payload, size);
-
- return 0;
+ return oa_tc6_update_rx_skb(tc6, payload, size);
}
-static void oa_tc6_prcs_rx_frame_end(struct oa_tc6 *tc6, u8 *payload, u16 size)
+static int oa_tc6_prcs_rx_frame_end(struct oa_tc6 *tc6, u8 *payload, u16 size)
{
- oa_tc6_update_rx_skb(tc6, payload, size);
+ int ret;
- oa_tc6_submit_rx_skb(tc6);
+ ret = oa_tc6_update_rx_skb(tc6, payload, size);
+ if (!ret)
+ oa_tc6_submit_rx_skb(tc6);
+ return ret;
}
-static void oa_tc6_prcs_ongoing_rx_frame(struct oa_tc6 *tc6, u8 *payload,
- u32 footer)
+static int oa_tc6_prcs_ongoing_rx_frame(struct oa_tc6 *tc6, u8 *payload,
+ u32 footer)
{
- oa_tc6_update_rx_skb(tc6, payload, OA_TC6_CHUNK_PAYLOAD_SIZE);
+ return oa_tc6_update_rx_skb(tc6, payload,
+ OA_TC6_CHUNK_PAYLOAD_SIZE);
}
static int oa_tc6_prcs_rx_chunk_payload(struct oa_tc6 *tc6, u8 *data,
@@ -876,10 +939,10 @@ static int oa_tc6_prcs_rx_chunk_payload(struct oa_tc6 *tc6, u8 *data,
u16 size;
/* Restart the new rx frame after receiving rx buffer overflow error */
- if (start_valid && tc6->rx_buf_overflow)
- tc6->rx_buf_overflow = false;
+ if (start_valid && tc6->wait_until_start_valid)
+ tc6->wait_until_start_valid = false;
- if (tc6->rx_buf_overflow)
+ if (tc6->wait_until_start_valid)
return 0;
/* Process the chunk with complete rx frame */
@@ -901,8 +964,7 @@ static int oa_tc6_prcs_rx_chunk_payload(struct oa_tc6 *tc6, u8 *data,
/* Process the chunk with only rx frame end */
if (end_valid && !start_valid) {
size = end_byte_offset + 1;
- oa_tc6_prcs_rx_frame_end(tc6, data, size);
- return 0;
+ return oa_tc6_prcs_rx_frame_end(tc6, data, size);
}
/* Process the chunk with previous rx frame end and next rx frame
@@ -916,6 +978,15 @@ static int oa_tc6_prcs_rx_chunk_payload(struct oa_tc6 *tc6, u8 *data,
if (tc6->rx_skb) {
size = end_byte_offset + 1;
oa_tc6_prcs_rx_frame_end(tc6, data, size);
+
+ /* Return value from oa_tc6_prcs_rx_frame_end is not
+ * checked. If it returned an error, it is to make
+ * the code to look for new frame. At this stage,
+ * code below is going to process a new frame. So,
+ * error condition is set to false, in case it is
+ * set before proceeding.
+ */
+ tc6->wait_until_start_valid = false;
}
size = OA_TC6_CHUNK_PAYLOAD_SIZE - start_byte_offset;
return oa_tc6_prcs_rx_frame_start(tc6,
@@ -924,9 +995,7 @@ static int oa_tc6_prcs_rx_chunk_payload(struct oa_tc6 *tc6, u8 *data,
}
/* Process the chunk with ongoing rx frame data */
- oa_tc6_prcs_ongoing_rx_frame(tc6, data, footer);
-
- return 0;
+ return oa_tc6_prcs_ongoing_rx_frame(tc6, data, footer);
}
static u32 oa_tc6_get_rx_chunk_footer(struct oa_tc6 *tc6, u16 footer_offset)
@@ -942,8 +1011,9 @@ static u32 oa_tc6_get_rx_chunk_footer(struct oa_tc6 *tc6, u16 footer_offset)
static int oa_tc6_process_spi_data_rx_buf(struct oa_tc6 *tc6, u16 length)
{
u16 no_of_rx_chunks = length / OA_TC6_CHUNK_SIZE;
+ bool retry = false;
+ int ret = 0;
u32 footer;
- int ret;
/* All the rx chunks in the receive SPI data buffer are examined here */
for (int i = 0; i < no_of_rx_chunks; i++) {
@@ -952,8 +1022,11 @@ static int oa_tc6_process_spi_data_rx_buf(struct oa_tc6 *tc6, u16 length)
OA_TC6_CHUNK_PAYLOAD_SIZE);
ret = oa_tc6_process_rx_chunk_footer(tc6, footer);
- if (ret)
- return ret;
+ if (ret) {
+ if (ret != -EAGAIN)
+ return ret;
+ retry = true;
+ }
/* If there is a data valid chunks then process it for the
* information needed to determine the validity and the location
@@ -965,12 +1038,35 @@ static int oa_tc6_process_spi_data_rx_buf(struct oa_tc6 *tc6, u16 length)
ret = oa_tc6_prcs_rx_chunk_payload(tc6, payload,
footer);
- if (ret)
- return ret;
+ if (ret) {
+ if (ret != -ENOMEM && ret != -EAGAIN)
+ return ret;
+ retry = true;
+ }
}
}
- return 0;
+ /* Not bailing out on recoverable error codes, -EAGAIN and
+ * -ENOMEM. If subsequent loop iterations, if any, succeeds,
+ * error code would be overwritten. retry flag helps to
+ * make the caller to continue and retry. Since recovery
+ * action for -ENOMEM and -EAGAIN are same, we are returning
+ * one of the error codes, that is -EAGAIN.
+ *
+ * Successful recovery depends on how small the frames are,
+ * how many chunks, among the received chunks triggered the
+ * error, whether data is intact even with error conditions.
+ * As a result, there is no single, best method to recover
+ * most data when error conditions hit. We do our best by
+ * processing all the chunks with good "footer header" and
+ * "data valid" bit set.
+ */
+ if (retry) {
+ ret = -EAGAIN;
+ oa_tc6_look_for_new_frame(tc6);
+ }
+
+ return ret;
}
static __be32 oa_tc6_prepare_data_header(bool data_valid, bool start_valid,
@@ -1132,12 +1228,9 @@ static int oa_tc6_try_spi_transfer(struct oa_tc6 *tc6)
}
ret = oa_tc6_process_spi_data_rx_buf(tc6, spi_len);
- if (ret) {
- if (ret == -EAGAIN)
- continue;
- oa_tc6_cleanup_ongoing_tx_skb(tc6);
- oa_tc6_cleanup_ongoing_rx_skb(tc6);
+ if (ret && ret != -EAGAIN) {
+ oa_tc6_free_ongoing_skbs(tc6);
netdev_err(tc6->netdev, "Device error: %d\n", ret);
return ret;
}
@@ -1159,15 +1252,20 @@ static irqreturn_t oa_tc6_macphy_threaded_irq(int irq, void *data)
* no need to attempt spi transfer, once it fails. Pending skbs
* are already freed.
*/
- if (!tc6->disable_traffic) {
- while (tc6->int_flag ||
- (tc6->waiting_tx_skb && tc6->tx_credits)) {
- ret = oa_tc6_try_spi_transfer(tc6);
- if (ret) {
- disable_irq_nosync(tc6->spi->irq);
- oa_tc6_disable_traffic(tc6);
- break;
- }
+ spin_lock_bh(&tc6->tx_skb_lock);
+ if (tc6->disable_traffic) {
+ spin_unlock_bh(&tc6->tx_skb_lock);
+ return IRQ_HANDLED;
+ }
+ spin_unlock_bh(&tc6->tx_skb_lock);
+
+ while (tc6->int_flag ||
+ (tc6->waiting_tx_skb && tc6->tx_credits)) {
+ ret = oa_tc6_try_spi_transfer(tc6);
+ if (ret) {
+ disable_irq_nosync(tc6->spi->irq);
+ oa_tc6_disable_traffic(tc6);
+ break;
}
}
@@ -1187,9 +1285,10 @@ static int oa_tc6_update_buffer_status_from_register(struct oa_tc6 *tc6)
if (ret)
return ret;
- tc6->tx_credits = FIELD_GET(BUFFER_STATUS_TX_CREDITS_AVAILABLE, value);
- tc6->rx_chunks_available = FIELD_GET(BUFFER_STATUS_RX_CHUNKS_AVAILABLE,
- value);
+ tc6->tx_credits = FIELD_GET(OA_TC6_BUFFER_STATUS_TX_CREDITS_AVAILABLE,
+ value);
+ tc6->rx_chunks_available =
+ FIELD_GET(OA_TC6_BUFFER_STATUS_RX_CHUNKS_AVAILABLE, value);
return 0;
}
@@ -1233,7 +1332,7 @@ int oa_tc6_zero_align_receive_frame_enable(struct oa_tc6 *tc6)
return ret;
/* Set Zero-Align Receive Frame Enable */
- regval |= CONFIG0_ZARFE_ENABLE;
+ regval |= OA_TC6_CONFIG0_ZARFE_ENABLE;
return oa_tc6_write_register(tc6, OA_TC6_REG_CONFIG0, regval);
}
@@ -1245,23 +1344,30 @@ EXPORT_SYMBOL_GPL(oa_tc6_zero_align_receive_frame_enable);
* @tc6: oa_tc6 struct.
* @skb: socket buffer in which the ethernet frame is stored.
*
- * Return: NETDEV_TX_OK if the transmit ethernet frame skb added in the tx_skb_q
- * otherwise returns NETDEV_TX_BUSY.
+ * Return: NETDEV_TX_OK either on successful queueing of the packet for
+ * transmission, or on packet getting dropped. Packet can be dropped due to
+ * failure in linearizing the buffer or disable_traffic is set due to
+ * earlier fatal error. Returns NETDEV_TX_BUSY when there is no room
+ * to queue the packet.
*/
netdev_tx_t oa_tc6_start_xmit(struct oa_tc6 *tc6, struct sk_buff *skb)
{
- if (tc6->disable_traffic || tc6->waiting_tx_skb) {
- netif_stop_queue(tc6->netdev);
- return NETDEV_TX_BUSY;
- }
-
if (skb_linearize(skb)) {
- dev_kfree_skb_any(skb);
- tc6->netdev->stats.tx_dropped++;
+ oa_tc6_drop_tx_skb(tc6, skb);
return NETDEV_TX_OK;
}
spin_lock_bh(&tc6->tx_skb_lock);
+ if (tc6->waiting_tx_skb) {
+ netif_stop_queue(tc6->netdev);
+ spin_unlock_bh(&tc6->tx_skb_lock);
+ return NETDEV_TX_BUSY;
+ }
+ if (tc6->disable_traffic) {
+ spin_unlock_bh(&tc6->tx_skb_lock);
+ oa_tc6_drop_tx_skb(tc6, skb);
+ return NETDEV_TX_OK;
+ }
tc6->waiting_tx_skb = skb;
spin_unlock_bh(&tc6->tx_skb_lock);
@@ -1272,6 +1378,20 @@ netdev_tx_t oa_tc6_start_xmit(struct oa_tc6 *tc6, struct sk_buff *skb)
}
EXPORT_SYMBOL_GPL(oa_tc6_start_xmit);
+static int oa_tc6_check_ctrl_protection(struct oa_tc6 *tc6)
+{
+ u32 regval;
+ int ret;
+
+ ret = oa_tc6_read_register(tc6, OA_TC6_REG_CONFIG0, ®val);
+ if (ret)
+ return ret;
+
+ tc6->prot_ctrl = FIELD_GET(OA_TC6_CONFIG0_PROTE, regval);
+
+ return 0;
+}
+
/**
* oa_tc6_init - allocates and initializes oa_tc6 structure.
* @spi: device with which data will be exchanged.
@@ -1324,6 +1444,14 @@ struct oa_tc6 *oa_tc6_init(struct spi_device *spi, struct net_device *netdev)
if (!tc6->spi_data_rx_buf)
return NULL;
+ /* Check the PROTE bit status so that we can reset the device */
+ ret = oa_tc6_check_ctrl_protection(tc6);
+ if (ret) {
+ dev_err(&tc6->spi->dev,
+ "Failed to check the protection mode: %d\n", ret);
+ return NULL;
+ }
+
ret = oa_tc6_sw_reset_macphy(tc6);
if (ret) {
dev_err(&tc6->spi->dev,
@@ -1393,8 +1521,10 @@ EXPORT_SYMBOL_GPL(oa_tc6_init);
*/
void oa_tc6_exit(struct oa_tc6 *tc6)
{
- tc6->disable_traffic = true;
disable_irq(tc6->spi->irq);
+ spin_lock_bh(&tc6->tx_skb_lock);
+ tc6->disable_traffic = true;
+ spin_unlock_bh(&tc6->tx_skb_lock);
oa_tc6_phy_exit(tc6);
oa_tc6_free_pending_skbs(tc6);
}
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c
index e436e3231e86..2543a8ff8547 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c
@@ -1672,15 +1672,22 @@ static int ionic_tx(struct net_device *netdev, struct ionic_queue *q,
stats->pkts++;
stats->bytes += skb->len;
+ ionic_txq_post(q, false);
+
if (likely(!ionic_txq_hwstamp_enabled(q))) {
struct netdev_queue *ndq = q_to_ndq(netdev, q);
- if (unlikely(!ionic_q_has_space(q, MAX_SKB_FRAGS + 1)))
- netif_tx_stop_queue(ndq);
+ netif_txq_maybe_stop(ndq, ionic_q_space_avail(q),
+ MAX_SKB_FRAGS + 1, MAX_SKB_FRAGS + 1);
ring_dbell = __netdev_tx_sent_queue(ndq, skb->len,
netdev_xmit_more());
}
- ionic_txq_post(q, ring_dbell);
+
+ if (ring_dbell) {
+ ionic_dbell_ring(q->lif->kern_dbpage, q->hw_type,
+ q->dbval | q->head_idx);
+ q->dbell_jiffies = jiffies;
+ }
return 0;
}
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index 3a437409f78a..f801caf65fff 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -3676,6 +3676,14 @@ static int stmmac_hw_setup(struct net_device *dev)
/* Initialize MTL*/
stmmac_mtl_configuration(priv);
+ /* Apply the RX packet parser table */
+ if (priv->tc_entries) {
+ ret = stmmac_rxp_config(priv, priv->hw->pcsr, priv->tc_entries,
+ priv->tc_entries_max);
+ if (ret)
+ return ret;
+ }
+
/* Initialize Safety Features */
stmmac_safety_feat_configuration(priv);
@@ -4319,6 +4327,7 @@ static bool stmmac_vlan_insert(struct stmmac_priv *priv, struct sk_buff *skb,
/**
* stmmac_tso_allocator - close entry point of the driver
* @priv: driver private structure
+ * @entry: TX queue buffer index
* @des: buffer start address
* @total_len: total length to fill in descriptors
* @last_segment: condition for the last descriptor
@@ -4327,8 +4336,9 @@ static bool stmmac_vlan_insert(struct stmmac_priv *priv, struct sk_buff *skb,
* This function fills descriptor and request new descriptors according to
* buffer length to fill
*/
-static void stmmac_tso_allocator(struct stmmac_priv *priv, dma_addr_t des,
- int total_len, bool last_segment, u32 queue)
+static void stmmac_tso_allocator(struct stmmac_priv *priv, u32 *entry,
+ dma_addr_t des, int total_len,
+ bool last_segment, u32 queue)
{
struct stmmac_tx_queue *tx_q = &priv->dma_conf.tx_queue[queue];
struct dma_desc *desc;
@@ -4340,14 +4350,13 @@ static void stmmac_tso_allocator(struct stmmac_priv *priv, dma_addr_t des,
while (tmp_len > 0) {
dma_addr_t curr_addr;
- tx_q->cur_tx = STMMAC_NEXT_ENTRY(tx_q->cur_tx,
- priv->dma_conf.dma_tx_size);
- WARN_ON(tx_q->tx_skbuff[tx_q->cur_tx]);
+ *entry = STMMAC_NEXT_ENTRY(*entry, priv->dma_conf.dma_tx_size);
+ WARN_ON(tx_q->tx_skbuff[*entry]);
if (tx_q->tbs & STMMAC_TBS_AVAIL)
- desc = &tx_q->dma_entx[tx_q->cur_tx].basic;
+ desc = &tx_q->dma_entx[*entry].basic;
else
- desc = &tx_q->dma_tx[tx_q->cur_tx];
+ desc = &tx_q->dma_tx[*entry];
curr_addr = des + (total_len - tmp_len);
stmmac_set_desc_addr(priv, desc, curr_addr);
@@ -4445,6 +4454,26 @@ static bool stmmac_tso_valid_packet(struct sk_buff *skb)
header_len + gso_size < 16383;
}
+static int stmmac_tso_get_num_desc(struct stmmac_tx_queue *tx_q,
+ struct sk_buff *skb, u32 pay_len)
+{
+ int i, ndesc = 1;
+
+ /* head payload */
+ ndesc += DIV_ROUND_UP(pay_len, TSO_MAX_BUFF_SIZE);
+ /* frag payload */
+ for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) {
+ const skb_frag_t *frag = &skb_shinfo(skb)->frags[i];
+
+ ndesc += DIV_ROUND_UP(skb_frag_size(frag),
+ TSO_MAX_BUFF_SIZE);
+ }
+ /* MSS update requires a new descriptor */
+ ndesc += !!(skb_shinfo(skb)->gso_size != tx_q->mss);
+
+ return ndesc;
+}
+
/**
* stmmac_tso_xmit - Tx entry point of the driver for oversized frames (TSO)
* @skb : the socket buffer
@@ -4486,12 +4515,12 @@ static netdev_tx_t stmmac_tso_xmit(struct sk_buff *skb, struct net_device *dev)
{
struct dma_desc *desc, *first, *mss_desc = NULL;
struct stmmac_priv *priv = netdev_priv(dev);
- unsigned int first_entry, tx_packets;
+ unsigned int first_entry, entry, tx_packets;
struct stmmac_txq_stats *txq_stats;
+ int i, first_tx, nfrags, ndesc;
struct stmmac_tx_queue *tx_q;
bool set_ic, is_last_segment;
u32 pay_len, mss, queue;
- int i, first_tx, nfrags;
u8 proto_hdr_len, hdr;
dma_addr_t des;
@@ -4504,14 +4533,15 @@ static netdev_tx_t stmmac_tso_xmit(struct sk_buff *skb, struct net_device *dev)
/* Compute header lengths */
proto_hdr_len = stmmac_tso_header_size(skb);
+ pay_len = skb_headlen(skb) - proto_hdr_len; /* no frags */
+
if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_L4)
hdr = sizeof(struct udphdr);
else
hdr = tcp_hdrlen(skb);
- /* Desc availability based on threshold should be enough safe */
- if (unlikely(stmmac_tx_avail(priv, queue) <
- (((skb->len - proto_hdr_len) / TSO_MAX_BUFF_SIZE + 1)))) {
+ ndesc = stmmac_tso_get_num_desc(tx_q, skb, pay_len);
+ if (unlikely(stmmac_tx_avail(priv, queue) < ndesc)) {
if (!netif_tx_queue_stopped(netdev_get_tx_queue(dev, queue))) {
netif_tx_stop_queue(netdev_get_tx_queue(priv->dev,
queue));
@@ -4523,8 +4553,6 @@ static netdev_tx_t stmmac_tso_xmit(struct sk_buff *skb, struct net_device *dev)
return NETDEV_TX_BUSY;
}
- pay_len = skb_headlen(skb) - proto_hdr_len; /* no frags */
-
mss = skb_shinfo(skb)->gso_size;
/* set new MSS value if needed */
@@ -4549,22 +4577,24 @@ static netdev_tx_t stmmac_tso_xmit(struct sk_buff *skb, struct net_device *dev)
}
first_entry = tx_q->cur_tx;
- WARN_ON(tx_q->tx_skbuff[first_entry]);
+ entry = first_entry;
+
+ WARN_ON(tx_q->tx_skbuff[entry]);
if (tx_q->tbs & STMMAC_TBS_AVAIL)
- desc = &tx_q->dma_entx[first_entry].basic;
+ desc = &tx_q->dma_entx[entry].basic;
else
- desc = &tx_q->dma_tx[first_entry];
+ desc = &tx_q->dma_tx[entry];
first = desc;
/* first descriptor: fill Headers on Buf1 */
des = dma_map_single(priv->device, skb->data, skb_headlen(skb),
DMA_TO_DEVICE);
if (dma_mapping_error(priv->device, des))
- goto dma_map_err;
+ goto error;
stmmac_set_desc_addr(priv, first, des);
- stmmac_tso_allocator(priv, des + proto_hdr_len, pay_len,
+ stmmac_tso_allocator(priv, &entry, des + proto_hdr_len, pay_len,
(nfrags == 0), queue);
/* In case two or more DMA transmit descriptors are allocated for this
@@ -4579,8 +4609,7 @@ static netdev_tx_t stmmac_tso_xmit(struct sk_buff *skb, struct net_device *dev)
* this DMA buffer right after the DMA engine completely finishes the
* full buffer transmission.
*/
- stmmac_set_tx_skb_dma_entry(tx_q, tx_q->cur_tx, des, skb_headlen(skb),
- false);
+ stmmac_set_tx_skb_dma_entry(tx_q, entry, des, skb_headlen(skb), false);
/* Prepare fragments */
for (i = 0; i < nfrags; i++) {
@@ -4590,14 +4619,15 @@ static netdev_tx_t stmmac_tso_xmit(struct sk_buff *skb, struct net_device *dev)
skb_frag_size(frag),
DMA_TO_DEVICE);
if (dma_mapping_error(priv->device, des))
- goto dma_map_err;
+ goto error_dma_unmap;
- stmmac_tso_allocator(priv, des, skb_frag_size(frag),
+ stmmac_tso_allocator(priv, &entry, des, skb_frag_size(frag),
(i == nfrags - 1), queue);
- stmmac_set_tx_skb_dma_entry(tx_q, tx_q->cur_tx, des,
+ stmmac_set_tx_skb_dma_entry(tx_q, entry, des,
skb_frag_size(frag), true);
}
+ tx_q->cur_tx = entry;
stmmac_set_tx_dma_last_segment(tx_q, tx_q->cur_tx);
@@ -4700,7 +4730,19 @@ static netdev_tx_t stmmac_tso_xmit(struct sk_buff *skb, struct net_device *dev)
return NETDEV_TX_OK;
-dma_map_err:
+error_dma_unmap:
+ for (;;) {
+ desc = stmmac_get_tx_desc(priv, tx_q, first_entry);
+ stmmac_release_tx_desc(priv, desc, priv->descriptor_mode);
+ stmmac_free_tx_buffer(priv, &priv->dma_conf, queue,
+ first_entry);
+ if (first_entry == entry)
+ break;
+
+ first_entry = STMMAC_NEXT_ENTRY(first_entry,
+ priv->dma_conf.dma_tx_size);
+ }
+error:
dev_err(priv->device, "Tx dma map failed\n");
dev_kfree_skb(skb);
priv->xstats.tx_dropped++;
@@ -8009,6 +8051,7 @@ static int __stmmac_dvr_probe(struct device *device,
stmmac_napi_add(ndev);
mutex_init(&priv->lock);
+ rwlock_init(&priv->ptp_lock);
stmmac_fpe_init(priv);
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.c
index 960249960004..3bfcc9760dce 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.c
@@ -365,7 +365,6 @@ void stmmac_ptp_register(struct stmmac_priv *priv)
if (priv->plat->crosststamp)
priv->ptp_clock_ops.getcrosststamp = stmmac_getcrosststamp;
- rwlock_init(&priv->ptp_lock);
mutex_init(&priv->aux_ts_lock);
priv->ptp_clock = ptp_clock_register(&priv->ptp_clock_ops,
diff --git a/drivers/net/gtp.c b/drivers/net/gtp.c
index 298efc76a56b..69fe5717846b 100644
--- a/drivers/net/gtp.c
+++ b/drivers/net/gtp.c
@@ -318,6 +318,11 @@ static int gtp_inner_proto(struct sk_buff *skb, unsigned int hdrlen,
static int gtp_rx(struct pdp_ctx *pctx, struct sk_buff *skb,
unsigned int hdrlen, unsigned int role, __u16 inner_proto)
{
+ if (skb_is_gso(skb)) {
+ netdev_dbg(pctx->dev, "GSO is not supported in GTP\n");
+ goto err;
+ }
+
if (!gtp_check_ms(skb, pctx, hdrlen, role, inner_proto)) {
netdev_dbg(pctx->dev, "No PDP ctx for this MS\n");
return 1;
diff --git a/drivers/net/ieee802154/cc2520.c b/drivers/net/ieee802154/cc2520.c
index 2b7034193a00..abfcfe07246a 100644
--- a/drivers/net/ieee802154/cc2520.c
+++ b/drivers/net/ieee802154/cc2520.c
@@ -1156,11 +1156,10 @@ static void cc2520_remove(struct spi_device *spi)
{
struct cc2520_private *priv = spi_get_drvdata(spi);
- mutex_destroy(&priv->buffer_mutex);
- flush_work(&priv->fifop_irqwork);
-
+ disable_work_sync(&priv->fifop_irqwork);
ieee802154_unregister_hw(priv->hw);
ieee802154_free_hw(priv->hw);
+ mutex_destroy(&priv->buffer_mutex);
}
static const struct spi_device_id cc2520_ids[] = {
diff --git a/drivers/net/ieee802154/mac802154_hwsim.c b/drivers/net/ieee802154/mac802154_hwsim.c
index 6daa0f198b9f..a9bd1555d2dc 100644
--- a/drivers/net/ieee802154/mac802154_hwsim.c
+++ b/drivers/net/ieee802154/mac802154_hwsim.c
@@ -72,6 +72,8 @@ struct hwsim_phy {
struct ieee802154_hw *hw;
u32 idx;
+ /* Serializes phy->pib_updates. */
+ spinlock_t pib_lock;
struct hwsim_pib __rcu *pib;
bool suspended;
@@ -102,8 +104,6 @@ static int hwsim_update_pib(struct ieee802154_hw *hw, u8 page, u8 channel,
if (!pib)
return -ENOMEM;
- pib_old = rtnl_dereference(phy->pib);
-
pib->page = page;
pib->channel = channel;
pib->filt.short_addr = filt->short_addr;
@@ -112,7 +112,10 @@ static int hwsim_update_pib(struct ieee802154_hw *hw, u8 page, u8 channel,
pib->filt.pan_coord = filt->pan_coord;
pib->filt_level = filt_level;
- rcu_assign_pointer(phy->pib, pib);
+ spin_lock_bh(&phy->pib_lock);
+ pib_old = rcu_replace_pointer(phy->pib, pib,
+ lockdep_is_held(&phy->pib_lock));
+ spin_unlock_bh(&phy->pib_lock);
kfree_rcu(pib_old, rcu);
return 0;
}
@@ -952,6 +955,7 @@ static int hwsim_add_one(struct genl_info *info, struct device *dev,
goto err_pib;
}
+ spin_lock_init(&phy->pib_lock);
pib->channel = 13;
pib->filt.short_addr = cpu_to_le16(IEEE802154_ADDR_BROADCAST);
pib->filt.pan_id = cpu_to_le16(IEEE802154_PANID_BROADCAST);
diff --git a/drivers/net/mctp/mctp-i3c.c b/drivers/net/mctp/mctp-i3c.c
index 88d9e36cd4a2..4e857dd5df64 100644
--- a/drivers/net/mctp/mctp-i3c.c
+++ b/drivers/net/mctp/mctp-i3c.c
@@ -288,6 +288,7 @@ __must_hold(&busdevs_lock)
static int mctp_i3c_probe(struct i3c_device *i3c)
{
struct mctp_i3c_bus *b = NULL, *mbus = NULL;
+ int rc;
/* Look for a known bus */
mutex_lock(&busdevs_lock);
@@ -296,14 +297,16 @@ static int mctp_i3c_probe(struct i3c_device *i3c)
mbus = b;
break;
}
- mutex_unlock(&busdevs_lock);
if (!mbus) {
/* probably no "mctp-controller" property on the i3c bus */
- return -ENODEV;
+ rc = -ENODEV;
+ } else {
+ rc = mctp_i3c_add_device(mbus, i3c);
}
+ mutex_unlock(&busdevs_lock);
- return mctp_i3c_add_device(mbus, i3c);
+ return rc;
}
static void mctp_i3c_remove_device(struct mctp_i3c_device *mi)
diff --git a/drivers/net/net_failover.c b/drivers/net/net_failover.c
index 3f7d31033bae..1b5213e87070 100644
--- a/drivers/net/net_failover.c
+++ b/drivers/net/net_failover.c
@@ -675,7 +675,7 @@ static int net_failover_slave_name_change(struct net_device *slave_dev,
/* We need to bring up the slave after the rename by udev in case
* open failed with EBUSY when it was registered.
*/
- dev_open(slave_dev, NULL);
+ netif_open(slave_dev, NULL);
return 0;
}
diff --git a/drivers/net/phy/dp83867.c b/drivers/net/phy/dp83867.c
index 88255e92b4cd..61a941aa02d9 100644
--- a/drivers/net/phy/dp83867.c
+++ b/drivers/net/phy/dp83867.c
@@ -1150,6 +1150,9 @@ static int dp83867_led_polarity_set(struct phy_device *phydev, int index,
case PHY_LED_ACTIVE_LOW:
polarity = 0;
break;
+ case PHY_LED_ACTIVE_HIGH:
+ polarity = DP83867_LED_POLARITY(index);
+ break;
default:
return -EINVAL;
}
diff --git a/drivers/net/phy/dp83td510.c b/drivers/net/phy/dp83td510.c
index d75dae6071ad..9e9a41bf6457 100644
--- a/drivers/net/phy/dp83td510.c
+++ b/drivers/net/phy/dp83td510.c
@@ -439,6 +439,9 @@ static int dp83td510_led_polarity_set(struct phy_device *phydev, int index,
case PHY_LED_ACTIVE_LOW:
polarity = 0;
break;
+ case PHY_LED_ACTIVE_HIGH:
+ polarity = DP83TD510E_LED_POLARITY(index);
+ break;
default:
return -EINVAL;
}
diff --git a/drivers/net/phy/mediatek/mtk-ge.c b/drivers/net/phy/mediatek/mtk-ge.c
index 73d9b72f9d9e..96d8ac5154e5 100644
--- a/drivers/net/phy/mediatek/mtk-ge.c
+++ b/drivers/net/phy/mediatek/mtk-ge.c
@@ -62,10 +62,38 @@ static void mtk_gephy_config_init(struct phy_device *phydev)
FIELD_PREP(MTK_MCC_NEARECHO_OFFSET_MASK, 0x3));
}
+static int mt7530_phy_probe(struct phy_device *phydev)
+{
+ /* The MT7530 internal GE PHY has broken EEE: with EEE advertised,
+ * some link partners fail to establish a stable link (on a 2-pair
+ * cable, 1000BASE-T training fails and the port loops instead of
+ * falling back). MediaTek recommends disabling EEE on this PHY.
+ * Clear the advertisement as early as possible, before anything
+ * can negotiate EEE with the link partner.
+ */
+ return phy_write_mmd(phydev, MDIO_MMD_AN, MDIO_AN_EEE_ADV, 0);
+}
+
static int mt7530_phy_config_init(struct phy_device *phydev)
{
+ int ret;
+
mtk_gephy_config_init(phydev);
+ /* The probe() clear alone is not durable: phy_init_hw() replays only
+ * ->config_init after a PHY reset, with the register back at its
+ * EEE-advertising hardware default, and phy_probe() zeroes
+ * eee_disabled_modes (of_set_phy_eee_broken()) after ->probe already
+ * ran. Clear the advertisement again and mark EEE disabled, so that
+ * neither phylib nor userspace can re-enable it; dp83867 disables
+ * broken EEE from config_init() the same way.
+ */
+ ret = phy_write_mmd(phydev, MDIO_MMD_AN, MDIO_AN_EEE_ADV, 0);
+ if (ret)
+ return ret;
+
+ phy_disable_eee(phydev);
+
/* Increase post_update_timer */
phy_write_paged(phydev, MTK_PHY_PAGE_EXTENDED_3,
MTK_PHY_RG_LPI_PCS_DSP_CTRL_REG11, 0x4b);
@@ -100,6 +128,7 @@ static struct phy_driver mtk_gephy_driver[] = {
{
PHY_ID_MATCH_EXACT(MTK_GPHY_ID_MT7530),
.name = "MediaTek MT7530 PHY",
+ .probe = mt7530_phy_probe,
.config_init = mt7530_phy_config_init,
/* Interrupts are handled by the switch, not the PHY
* itself.
diff --git a/drivers/net/phy/phylink.c b/drivers/net/phy/phylink.c
index 7397169236fd..27fd9aa2027a 100644
--- a/drivers/net/phy/phylink.c
+++ b/drivers/net/phy/phylink.c
@@ -1630,8 +1630,10 @@ static void phylink_resolve(struct work_struct *w)
if (pl->phylink_disable_state) {
pl->link_failed = false;
+ link_state = pl->link_config;
link_state.link = false;
} else if (pl->link_failed) {
+ link_state = pl->link_config;
link_state.link = false;
retrigger = true;
} else if (pl->act_link_an_mode == MLO_AN_FIXED) {
diff --git a/drivers/net/ppp/ppp_async.c b/drivers/net/ppp/ppp_async.c
index 93a7b0f6c4e7..6664686b1e04 100644
--- a/drivers/net/ppp/ppp_async.c
+++ b/drivers/net/ppp/ppp_async.c
@@ -65,8 +65,6 @@ struct asyncppp {
struct tasklet_struct tsk;
- refcount_t refcnt;
- struct completion dead;
struct ppp_channel chan; /* interface to generic ppp layer */
unsigned char obuf[OBUFSIZE];
};
@@ -116,38 +114,6 @@ static const struct ppp_channel_ops async_ops = {
* Routines implementing the PPP line discipline.
*/
-/*
- * We have a potential race on dereferencing tty->disc_data,
- * because the tty layer provides no locking at all - thus one
- * cpu could be running ppp_asynctty_receive while another
- * calls ppp_asynctty_close, which zeroes tty->disc_data and
- * frees the memory that ppp_asynctty_receive is using. The best
- * way to fix this is to use a rwlock in the tty struct, but for now
- * we use a single global rwlock for all ttys in ppp line discipline.
- *
- * FIXME: this is no longer true. The _close path for the ldisc is
- * now guaranteed to be sane.
- */
-static DEFINE_RWLOCK(disc_data_lock);
-
-static struct asyncppp *ap_get(struct tty_struct *tty)
-{
- struct asyncppp *ap;
-
- read_lock(&disc_data_lock);
- ap = tty->disc_data;
- if (ap != NULL)
- refcount_inc(&ap->refcnt);
- read_unlock(&disc_data_lock);
- return ap;
-}
-
-static void ap_put(struct asyncppp *ap)
-{
- if (refcount_dec_and_test(&ap->refcnt))
- complete(&ap->dead);
-}
-
/*
* Called when a tty is put into PPP line discipline. Called in process
* context.
@@ -182,9 +148,6 @@ ppp_asynctty_open(struct tty_struct *tty)
skb_queue_head_init(&ap->rqueue);
tasklet_setup(&ap->tsk, ppp_async_process);
- refcount_set(&ap->refcnt, 1);
- init_completion(&ap->dead);
-
ap->chan.private = ap;
ap->chan.ops = &async_ops;
ap->chan.mtu = PPP_MRU;
@@ -205,34 +168,18 @@ ppp_asynctty_open(struct tty_struct *tty)
}
/*
- * Called when the tty is put into another line discipline
- * or it hangs up. We have to wait for any cpu currently
- * executing in any of the other ppp_asynctty_* routines to
- * finish before we can call ppp_unregister_channel and free
- * the asyncppp struct. This routine must be called from
- * process context, not interrupt or softirq context.
+ * Called when the tty is put into another line discipline or it hangs up.
+ * This call is serialized against other ldisc functions.
*/
static void
ppp_asynctty_close(struct tty_struct *tty)
{
- struct asyncppp *ap;
+ struct asyncppp *ap = tty->disc_data;
- write_lock_irq(&disc_data_lock);
- ap = tty->disc_data;
- tty->disc_data = NULL;
- write_unlock_irq(&disc_data_lock);
if (!ap)
return;
- /*
- * We have now ensured that nobody can start using ap from now
- * on, but we have to wait for all existing users to finish.
- * Note that ppp_unregister_channel ensures that no calls to
- * our channel ops (i.e. ppp_async_send/ioctl) are in progress
- * by the time it returns.
- */
- if (!refcount_dec_and_test(&ap->refcnt))
- wait_for_completion(&ap->dead);
+ tty->disc_data = NULL;
tasklet_kill(&ap->tsk);
ppp_unregister_channel(&ap->chan);
@@ -242,17 +189,6 @@ ppp_asynctty_close(struct tty_struct *tty)
kfree(ap);
}
-/*
- * Called on tty hangup in process context.
- *
- * Wait for I/O to driver to complete and unregister PPP channel.
- * This is already done by the close routine, so just call that.
- */
-static void ppp_asynctty_hangup(struct tty_struct *tty)
-{
- ppp_asynctty_close(tty);
-}
-
/*
* Read does nothing - no data is ever available this way.
* Pppd reads and writes packets via /dev/ppp instead.
@@ -283,7 +219,7 @@ ppp_asynctty_write(struct tty_struct *tty, struct file *file, const u8 *buf,
static int
ppp_asynctty_ioctl(struct tty_struct *tty, unsigned int cmd, unsigned long arg)
{
- struct asyncppp *ap = ap_get(tty);
+ struct asyncppp *ap = tty->disc_data;
int err, val;
int __user *p = (int __user *)arg;
@@ -324,7 +260,6 @@ ppp_asynctty_ioctl(struct tty_struct *tty, unsigned int cmd, unsigned long arg)
err = tty_mode_ioctl(tty, cmd, arg);
}
- ap_put(ap);
return err;
}
@@ -333,7 +268,7 @@ static void
ppp_asynctty_receive(struct tty_struct *tty, const u8 *buf, const u8 *cflags,
size_t count)
{
- struct asyncppp *ap = ap_get(tty);
+ struct asyncppp *ap = tty->disc_data;
unsigned long flags;
if (!ap)
@@ -343,21 +278,19 @@ ppp_asynctty_receive(struct tty_struct *tty, const u8 *buf, const u8 *cflags,
spin_unlock_irqrestore(&ap->recv_lock, flags);
if (!skb_queue_empty(&ap->rqueue))
tasklet_schedule(&ap->tsk);
- ap_put(ap);
tty_unthrottle(tty);
}
static void
ppp_asynctty_wakeup(struct tty_struct *tty)
{
- struct asyncppp *ap = ap_get(tty);
+ struct asyncppp *ap = tty->disc_data;
clear_bit(TTY_DO_WRITE_WAKEUP, &tty->flags);
if (!ap)
return;
set_bit(XMIT_WAKEUP, &ap->xmit_flags);
tasklet_schedule(&ap->tsk);
- ap_put(ap);
}
@@ -367,7 +300,6 @@ static struct tty_ldisc_ops ppp_ldisc = {
.name = "ppp",
.open = ppp_asynctty_open,
.close = ppp_asynctty_close,
- .hangup = ppp_asynctty_hangup,
.read = ppp_asynctty_read,
.write = ppp_asynctty_write,
.ioctl = ppp_asynctty_ioctl,
@@ -812,11 +744,8 @@ process_input_packet(struct asyncppp *ap)
err:
/* frame had an error, remember that, reset SC_TOSS & SC_ESCAPE */
ap->state = SC_PREV_ERROR;
- if (skb) {
- /* make skb appear as freshly allocated */
- skb_trim(skb, 0);
- skb_reserve(skb, - skb_headroom(skb));
- }
+ kfree_skb(skb);
+ ap->rpkt = NULL;
}
/* Called when the tty driver has data for us. Runs parallel with the
diff --git a/drivers/net/ppp/ppp_synctty.c b/drivers/net/ppp/ppp_synctty.c
index b7f243b416f8..1b64e610f960 100644
--- a/drivers/net/ppp/ppp_synctty.c
+++ b/drivers/net/ppp/ppp_synctty.c
@@ -38,11 +38,9 @@
#include <linux/ppp-ioctl.h>
#include <linux/ppp_channel.h>
#include <linux/spinlock.h>
-#include <linux/completion.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/slab.h>
-#include <linux/refcount.h>
#include <linux/unaligned.h>
#include <linux/uaccess.h>
@@ -69,8 +67,6 @@ struct syncppp {
struct tasklet_struct tsk;
- refcount_t refcnt;
- struct completion dead_cmp;
struct ppp_channel chan; /* interface to generic ppp layer */
};
@@ -118,37 +114,6 @@ ppp_print_buffer (const char *name, const __u8 *buf, int count)
* Routines implementing the synchronous PPP line discipline.
*/
-/*
- * We have a potential race on dereferencing tty->disc_data,
- * because the tty layer provides no locking at all - thus one
- * cpu could be running ppp_synctty_receive while another
- * calls ppp_synctty_close, which zeroes tty->disc_data and
- * frees the memory that ppp_synctty_receive is using. The best
- * way to fix this is to use a rwlock in the tty struct, but for now
- * we use a single global rwlock for all ttys in ppp line discipline.
- *
- * FIXME: Fixed in tty_io nowadays.
- */
-static DEFINE_RWLOCK(disc_data_lock);
-
-static struct syncppp *sp_get(struct tty_struct *tty)
-{
- struct syncppp *ap;
-
- read_lock(&disc_data_lock);
- ap = tty->disc_data;
- if (ap != NULL)
- refcount_inc(&ap->refcnt);
- read_unlock(&disc_data_lock);
- return ap;
-}
-
-static void sp_put(struct syncppp *ap)
-{
- if (refcount_dec_and_test(&ap->refcnt))
- complete(&ap->dead_cmp);
-}
-
/*
* Called when a tty is put into sync-PPP line discipline.
*/
@@ -179,9 +144,6 @@ ppp_sync_open(struct tty_struct *tty)
skb_queue_head_init(&ap->rqueue);
tasklet_setup(&ap->tsk, ppp_sync_process);
- refcount_set(&ap->refcnt, 1);
- init_completion(&ap->dead_cmp);
-
ap->chan.private = ap;
ap->chan.ops = &sync_ops;
ap->chan.mtu = PPP_MRU;
@@ -203,34 +165,18 @@ ppp_sync_open(struct tty_struct *tty)
}
/*
- * Called when the tty is put into another line discipline
- * or it hangs up. We have to wait for any cpu currently
- * executing in any of the other ppp_synctty_* routines to
- * finish before we can call ppp_unregister_channel and free
- * the syncppp struct. This routine must be called from
- * process context, not interrupt or softirq context.
+ * Called when the tty is put into another line discipline or it hangs up.
+ * This call is serialized against other ldisc functions.
*/
static void
ppp_sync_close(struct tty_struct *tty)
{
- struct syncppp *ap;
+ struct syncppp *ap = tty->disc_data;
- write_lock_irq(&disc_data_lock);
- ap = tty->disc_data;
- tty->disc_data = NULL;
- write_unlock_irq(&disc_data_lock);
if (!ap)
return;
- /*
- * We have now ensured that nobody can start using ap from now
- * on, but we have to wait for all existing users to finish.
- * Note that ppp_unregister_channel ensures that no calls to
- * our channel ops (i.e. ppp_sync_send/ioctl) are in progress
- * by the time it returns.
- */
- if (!refcount_dec_and_test(&ap->refcnt))
- wait_for_completion(&ap->dead_cmp);
+ tty->disc_data = NULL;
tasklet_kill(&ap->tsk);
ppp_unregister_channel(&ap->chan);
@@ -239,17 +185,6 @@ ppp_sync_close(struct tty_struct *tty)
kfree(ap);
}
-/*
- * Called on tty hangup in process context.
- *
- * Wait for I/O to driver to complete and unregister PPP channel.
- * This is already done by the close routine, so just call that.
- */
-static void ppp_sync_hangup(struct tty_struct *tty)
-{
- ppp_sync_close(tty);
-}
-
/*
* Read does nothing - no data is ever available this way.
* Pppd reads and writes packets via /dev/ppp instead.
@@ -275,7 +210,7 @@ ppp_sync_write(struct tty_struct *tty, struct file *file, const u8 *buf,
static int
ppp_synctty_ioctl(struct tty_struct *tty, unsigned int cmd, unsigned long arg)
{
- struct syncppp *ap = sp_get(tty);
+ struct syncppp *ap = tty->disc_data;
int __user *p = (int __user *)arg;
int err, val;
@@ -316,7 +251,6 @@ ppp_synctty_ioctl(struct tty_struct *tty, unsigned int cmd, unsigned long arg)
break;
}
- sp_put(ap);
return err;
}
@@ -325,7 +259,7 @@ static void
ppp_sync_receive(struct tty_struct *tty, const u8 *buf, const u8 *cflags,
size_t count)
{
- struct syncppp *ap = sp_get(tty);
+ struct syncppp *ap = tty->disc_data;
unsigned long flags;
if (!ap)
@@ -335,21 +269,19 @@ ppp_sync_receive(struct tty_struct *tty, const u8 *buf, const u8 *cflags,
spin_unlock_irqrestore(&ap->recv_lock, flags);
if (!skb_queue_empty(&ap->rqueue))
tasklet_schedule(&ap->tsk);
- sp_put(ap);
tty_unthrottle(tty);
}
static void
ppp_sync_wakeup(struct tty_struct *tty)
{
- struct syncppp *ap = sp_get(tty);
+ struct syncppp *ap = tty->disc_data;
clear_bit(TTY_DO_WRITE_WAKEUP, &tty->flags);
if (!ap)
return;
set_bit(XMIT_WAKEUP, &ap->xmit_flags);
tasklet_schedule(&ap->tsk);
- sp_put(ap);
}
@@ -359,7 +291,6 @@ static struct tty_ldisc_ops ppp_sync_ldisc = {
.name = "pppsync",
.open = ppp_sync_open,
.close = ppp_sync_close,
- .hangup = ppp_sync_hangup,
.read = ppp_sync_read,
.write = ppp_sync_write,
.ioctl = ppp_synctty_ioctl,
@@ -526,17 +457,9 @@ ppp_sync_txmunge(struct syncppp *ap, struct sk_buff *skb)
/* prepend address/control fields if necessary */
if ((ap->flags & SC_COMP_AC) == 0 || islcp) {
- if (skb_headroom(skb) < 2) {
- struct sk_buff *npkt = dev_alloc_skb(skb->len + 2);
- if (npkt == NULL) {
- kfree_skb(skb);
- return NULL;
- }
- skb_reserve(npkt,2);
- skb_copy_from_linear_data(skb,
- skb_put(npkt, skb->len), skb->len);
- consume_skb(skb);
- skb = npkt;
+ if (skb_cow_head(skb, 2)) {
+ kfree_skb(skb);
+ return NULL;
}
skb_push(skb,2);
skb->data[0] = PPP_ALLSTATIONS;
diff --git a/drivers/net/usb/cx82310_eth.c b/drivers/net/usb/cx82310_eth.c
index 068acb052adb..5df657acf3d5 100644
--- a/drivers/net/usb/cx82310_eth.c
+++ b/drivers/net/usb/cx82310_eth.c
@@ -282,6 +282,7 @@ static int cx82310_rx_fixup(struct usbnet *dev, struct sk_buff *skb)
if (len == 0xffff) {
netdev_info(dev->net, "router was rebooted, re-enabling ethernet mode");
schedule_work(&priv->reenable_work);
+ return 0;
} else if (len > CX82310_MTU) {
netdev_err(dev->net, "RX packet too long: %d B\n", len);
return 0;
diff --git a/drivers/net/usb/hso.c b/drivers/net/usb/hso.c
index c1aec67688ae..71caa3764b23 100644
--- a/drivers/net/usb/hso.c
+++ b/drivers/net/usb/hso.c
@@ -1547,10 +1547,10 @@ hso_wait_modem_status(struct hso_serial *serial, unsigned long arg)
spin_unlock_irq(&serial->serial_lock);
add_wait_queue(&tiocmget->waitq, &wait);
for (;;) {
+ set_current_state(TASK_INTERRUPTIBLE);
spin_lock_irq(&serial->serial_lock);
memcpy(&cnow, &tiocmget->icount, sizeof(struct uart_icount));
spin_unlock_irq(&serial->serial_lock);
- set_current_state(TASK_INTERRUPTIBLE);
if (((arg & TIOCM_RNG) && (cnow.rng != cprev.rng)) ||
((arg & TIOCM_DSR) && (cnow.dsr != cprev.dsr)) ||
((arg & TIOCM_CD) && (cnow.dcd != cprev.dcd))) {
diff --git a/drivers/net/usb/pegasus.c b/drivers/net/usb/pegasus.c
index 8700eeb8e22d..aba1a640fc26 100644
--- a/drivers/net/usb/pegasus.c
+++ b/drivers/net/usb/pegasus.c
@@ -43,21 +43,12 @@ static bool loopback;
static bool mii_mode;
static char *devid;
-static struct usb_eth_dev usb_dev_id[] = {
-#define PEGASUS_DEV(pn, vid, pid, flags) \
- {.name = pn, .vendor = vid, .device = pid, .private = flags},
-#define PEGASUS_DEV_CLASS(pn, vid, pid, dclass, flags) \
- PEGASUS_DEV(pn, vid, pid, flags)
-#include "pegasus.h"
-#undef PEGASUS_DEV
-#undef PEGASUS_DEV_CLASS
- {NULL, 0, 0, 0},
- {NULL, 0, 0, 0}
-};
+static struct usb_eth_dev dynamic_id_info = {};
static struct usb_device_id pegasus_ids[] = {
#define PEGASUS_DEV(pn, vid, pid, flags) \
- {.match_flags = USB_DEVICE_ID_MATCH_DEVICE, .idVendor = vid, .idProduct = pid},
+ {.match_flags = USB_DEVICE_ID_MATCH_DEVICE, .idVendor = vid, .idProduct = pid, \
+ .driver_info = (kernel_ulong_t)&(const struct usb_eth_dev) {.name = pn, .private = flags}},
/*
* The Belkin F8T012xx1 bluetooth adaptor has the same vendor and product
* IDs as the Belkin F5D5050, so we need to teach the pegasus driver to
@@ -66,7 +57,8 @@ static struct usb_device_id pegasus_ids[] = {
*/
#define PEGASUS_DEV_CLASS(pn, vid, pid, dclass, flags) \
{.match_flags = (USB_DEVICE_ID_MATCH_DEVICE | USB_DEVICE_ID_MATCH_DEV_CLASS), \
- .idVendor = vid, .idProduct = pid, .bDeviceClass = dclass},
+ .idVendor = vid, .idProduct = pid, .bDeviceClass = dclass, \
+ .driver_info = (kernel_ulong_t)&(const struct usb_eth_dev) {.name = pn, .private = flags}},
#include "pegasus.h"
#undef PEGASUS_DEV
#undef PEGASUS_DEV_CLASS
@@ -402,12 +394,12 @@ static inline int reset_mac(pegasus_t *pegasus)
if (i == REG_TIMEOUT)
return -ETIMEDOUT;
- if (usb_dev_id[pegasus->dev_index].vendor == VENDOR_LINKSYS ||
- usb_dev_id[pegasus->dev_index].vendor == VENDOR_DLINK) {
+ if (le16_to_cpu(pegasus->usb->descriptor.idVendor) == VENDOR_LINKSYS ||
+ le16_to_cpu(pegasus->usb->descriptor.idVendor) == VENDOR_DLINK) {
set_register(pegasus, Gpio0, 0x24);
set_register(pegasus, Gpio0, 0x26);
}
- if (usb_dev_id[pegasus->dev_index].vendor == VENDOR_ELCON) {
+ if (le16_to_cpu(pegasus->usb->descriptor.idVendor) == VENDOR_ELCON) {
__u16 auxmode;
ret = read_mii_word(pegasus, 3, 0x1b, &auxmode);
if (ret < 0)
@@ -445,9 +437,9 @@ static int enable_net_traffic(struct net_device *dev, struct usb_device *usb)
memcpy(pegasus->eth_regs, data, sizeof(data));
ret = set_registers(pegasus, EthCtrl0, 3, data);
- if (usb_dev_id[pegasus->dev_index].vendor == VENDOR_LINKSYS ||
- usb_dev_id[pegasus->dev_index].vendor == VENDOR_LINKSYS2 ||
- usb_dev_id[pegasus->dev_index].vendor == VENDOR_DLINK) {
+ if (le16_to_cpu(pegasus->usb->descriptor.idVendor) == VENDOR_LINKSYS ||
+ le16_to_cpu(pegasus->usb->descriptor.idVendor) == VENDOR_LINKSYS2 ||
+ le16_to_cpu(pegasus->usb->descriptor.idVendor) == VENDOR_DLINK) {
u16 auxmode;
ret = read_mii_word(pegasus, 0, 0x1b, &auxmode);
if (ret < 0)
@@ -1153,7 +1145,7 @@ static int pegasus_probe(struct usb_interface *intf,
struct usb_device *dev = interface_to_usbdev(intf);
struct net_device *net;
pegasus_t *pegasus;
- int dev_index = id - pegasus_ids;
+ const struct usb_eth_dev *info = (const struct usb_eth_dev *)id->driver_info;
int res = -ENOMEM;
static const u8 bulk_ep_addr[] = {
PEGASUS_USB_EP_BULK_IN | USB_DIR_IN,
@@ -1178,7 +1170,6 @@ static int pegasus_probe(struct usb_interface *intf,
goto out;
pegasus = netdev_priv(net);
- pegasus->dev_index = dev_index;
pegasus->intf = intf;
res = alloc_urbs(pegasus);
@@ -1206,7 +1197,7 @@ static int pegasus_probe(struct usb_interface *intf,
pegasus->msg_enable = netif_msg_init(msg_level, NETIF_MSG_DRV
| NETIF_MSG_PROBE | NETIF_MSG_LINK);
- pegasus->features = usb_dev_id[dev_index].private;
+ pegasus->features = info ? info->private : DEFAULT_GPIO_RESET;
res = get_interrupt_interval(pegasus);
if (res)
goto out2;
@@ -1235,7 +1226,7 @@ static int pegasus_probe(struct usb_interface *intf,
queue_delayed_work(system_long_wq, &pegasus->carrier_check,
CARRIER_CHECK_DELAY);
dev_info(&intf->dev, "%s, %s, %pM\n", net->name,
- usb_dev_id[dev_index].name, net->dev_addr);
+ info ? info->name : "(unknown)", net->dev_addr);
return 0;
out3:
@@ -1325,8 +1316,9 @@ static struct usb_driver pegasus_driver = {
static void __init parse_id(char *id)
{
- unsigned int vendor_id = 0, device_id = 0, flags = 0, i = 0;
+ unsigned int vendor_id = 0, device_id = 0, flags = 0;
char *token, *name = NULL;
+ int dyn_id_index = ARRAY_SIZE(pegasus_ids) - 2;
token = strsep(&id, ":");
if (token)
@@ -1348,14 +1340,12 @@ static void __init parse_id(char *id)
if (device_id > 0x10000 || device_id == 0)
return;
- for (i = 0; usb_dev_id[i].name; i++);
- usb_dev_id[i].name = name;
- usb_dev_id[i].vendor = vendor_id;
- usb_dev_id[i].device = device_id;
- usb_dev_id[i].private = flags;
- pegasus_ids[i].match_flags = USB_DEVICE_ID_MATCH_DEVICE;
- pegasus_ids[i].idVendor = vendor_id;
- pegasus_ids[i].idProduct = device_id;
+ dynamic_id_info.name = name;
+ dynamic_id_info.private = flags;
+ pegasus_ids[dyn_id_index].match_flags = USB_DEVICE_ID_MATCH_DEVICE;
+ pegasus_ids[dyn_id_index].idVendor = vendor_id;
+ pegasus_ids[dyn_id_index].idProduct = device_id;
+ pegasus_ids[dyn_id_index].driver_info = (kernel_ulong_t)&dynamic_id_info;
}
static int __init pegasus_init(void)
diff --git a/drivers/net/usb/pegasus.h b/drivers/net/usb/pegasus.h
index a05b143155ba..ccdedcef52e7 100644
--- a/drivers/net/usb/pegasus.h
+++ b/drivers/net/usb/pegasus.h
@@ -85,7 +85,6 @@ typedef struct pegasus {
unsigned features;
u32 msg_enable;
u32 wolopts;
- int dev_index;
int intr_interval;
struct tasklet_struct rx_tl;
struct delayed_work carrier_check;
@@ -102,8 +101,6 @@ typedef struct pegasus {
struct usb_eth_dev {
char *name;
- __u16 vendor;
- __u16 device;
__u32 private; /* LSB is gpio reset value */
};
diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c
index 44ea8499e787..e045e1ee9e59 100644
--- a/drivers/net/vxlan/vxlan_core.c
+++ b/drivers/net/vxlan/vxlan_core.c
@@ -996,6 +996,12 @@ static int vxlan_fdb_update_existing(struct vxlan_dev *vxlan,
return -EOPNOTSUPP;
}
+ if (rcu_access_pointer(f->nh) &&
+ !(state & (NUD_PERMANENT | NUD_NOARP))) {
+ NL_SET_ERR_MSG(extack, "Cannot make a nexthop fdb dynamic");
+ return -EOPNOTSUPP;
+ }
+
/* Do not allow an externally learned entry to take over an entry added
* by the user.
*/
@@ -1257,6 +1263,11 @@ static int vxlan_fdb_add(struct ndmsg *ndm, struct nlattr *tb[],
if (err)
return err;
+ if (nhid && !(ndm->ndm_state & (NUD_PERMANENT | NUD_NOARP))) {
+ NL_SET_ERR_MSG(extack, "A nexthop fdb cannot be dynamic");
+ return -EINVAL;
+ }
+
if (vxlan->default_dst.remote_ip.sa.sa_family != ip.sa.sa_family)
return -EAFNOSUPPORT;
@@ -2362,7 +2373,7 @@ void vxlan_xmit_one(struct sk_buff *skb, struct net_device *dev,
struct ip_tunnel_key key;
struct vxlan_dev *vxlan = netdev_priv(dev);
const struct iphdr *old_iph;
- struct vxlan_metadata _md;
+ struct vxlan_metadata _md = {};
struct vxlan_metadata *md = &_md;
unsigned int pkt_len = skb->len;
__be16 src_port = 0, dst_port;
diff --git a/drivers/net/vxlan/vxlan_mdb.c b/drivers/net/vxlan/vxlan_mdb.c
index d71e1925ecfd..841f42ffecb9 100644
--- a/drivers/net/vxlan/vxlan_mdb.c
+++ b/drivers/net/vxlan/vxlan_mdb.c
@@ -354,6 +354,10 @@ static bool vxlan_mdb_is_valid_source(const struct nlattr *attr, __be16 proto,
NL_SET_ERR_MSG_MOD(extack, "IPv4 multicast source address is not allowed");
return false;
}
+ if (ipv4_is_zeronet(nla_get_in_addr(attr))) {
+ NL_SET_ERR_MSG_MOD(extack, "IPv4 all-zeros source address is not allowed");
+ return false;
+ }
break;
#if IS_ENABLED(CONFIG_IPV6)
case htons(ETH_P_IPV6): {
@@ -368,6 +372,10 @@ static bool vxlan_mdb_is_valid_source(const struct nlattr *attr, __be16 proto,
NL_SET_ERR_MSG_MOD(extack, "IPv6 multicast source address is not allowed");
return false;
}
+ if (ipv6_addr_any(&src)) {
+ NL_SET_ERR_MSG_MOD(extack, "IPv6 all-zeros source address is not allowed");
+ return false;
+ }
break;
}
#endif
diff --git a/drivers/net/wireless/ath/ath9k/hif_usb.c b/drivers/net/wireless/ath/ath9k/hif_usb.c
index 47f904e7e652..d3491ff08e6e 100644
--- a/drivers/net/wireless/ath/ath9k/hif_usb.c
+++ b/drivers/net/wireless/ath/ath9k/hif_usb.c
@@ -1087,7 +1087,7 @@ static int ath9k_hif_usb_download_fw(struct hif_device_usb *hif_dev)
}
kfree(buf);
- if (IS_AR7010_DEVICE(hif_dev->usb_device_id->driver_info))
+ if (IS_AR7010_DEVICE(hif_dev->id_info))
firm_offset = AR7010_FIRMWARE_TEXT;
else
firm_offset = AR9271_FIRMWARE_TEXT;
@@ -1182,7 +1182,7 @@ static int ath9k_hif_request_firmware(struct hif_device_usb *hif_dev,
if (MAJOR_VERSION_REQ == 1 && hif_dev->fw_minor_index == 3) {
const char *filename;
- if (IS_AR7010_DEVICE(hif_dev->usb_device_id->driver_info))
+ if (IS_AR7010_DEVICE(hif_dev->id_info))
filename = FIRMWARE_AR7010_1_1;
else
filename = FIRMWARE_AR9271;
@@ -1198,7 +1198,7 @@ static int ath9k_hif_request_firmware(struct hif_device_usb *hif_dev,
return -ENOENT;
} else {
- if (IS_AR7010_DEVICE(hif_dev->usb_device_id->driver_info))
+ if (IS_AR7010_DEVICE(hif_dev->id_info))
chip = "7010";
else
chip = "9271";
@@ -1255,9 +1255,9 @@ static void ath9k_hif_usb_firmware_cb(const struct firmware *fw, void *context)
ret = ath9k_htc_hw_init(hif_dev->htc_handle,
&hif_dev->interface->dev,
- hif_dev->usb_device_id->idProduct,
+ le16_to_cpu(hif_dev->udev->descriptor.idProduct),
hif_dev->udev->product,
- hif_dev->usb_device_id->driver_info);
+ hif_dev->id_info);
if (ret) {
ret = -EINVAL;
goto err_htc_hw_init;
@@ -1369,7 +1369,7 @@ static int ath9k_hif_usb_probe(struct usb_interface *interface,
hif_dev->udev = udev;
hif_dev->interface = interface;
- hif_dev->usb_device_id = id;
+ hif_dev->id_info = id->driver_info;
#ifdef CONFIG_PM
udev->reset_resume = 1;
#endif
diff --git a/drivers/net/wireless/ath/ath9k/hif_usb.h b/drivers/net/wireless/ath/ath9k/hif_usb.h
index dc0b0fa5c325..b3e7b0fb54b8 100644
--- a/drivers/net/wireless/ath/ath9k/hif_usb.h
+++ b/drivers/net/wireless/ath/ath9k/hif_usb.h
@@ -115,7 +115,7 @@ struct cmd_buf {
struct hif_device_usb {
struct usb_device *udev;
struct usb_interface *interface;
- const struct usb_device_id *usb_device_id;
+ int id_info;
const void *fw_data;
size_t fw_size;
struct completion fw_done;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c
index 25b9437250f7..45439652c847 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/mcu.c
@@ -415,7 +415,8 @@ static int mt7921_load_clc(struct mt792x_dev *dev, const char *fw_name)
struct mt76_dev *mdev = &dev->mt76;
struct mt792x_phy *phy = &dev->phy;
const struct firmware *fw;
- int ret, i, len, offset = 0;
+ size_t clc_len, fw_data_len, len, offset = 0;
+ int ret, i;
u8 *clc_base = NULL, hw_encap = 0;
dev->phy.clc_chan_conf = 0xff;
@@ -441,13 +442,21 @@ static int mt7921_load_clc(struct mt792x_dev *dev, const char *fw_name)
}
hdr = (const void *)(fw->data + fw->size - sizeof(*hdr));
+ if (hdr->n_region > (fw->size - sizeof(*hdr)) / sizeof(*region)) {
+ dev_err(mdev->dev, "Invalid firmware region table\n");
+ ret = -EINVAL;
+ goto out;
+ }
+ fw_data_len = fw->size - sizeof(*hdr) -
+ hdr->n_region * sizeof(*region);
+
for (i = 0; i < hdr->n_region; i++) {
region = (const void *)((const u8 *)hdr -
(hdr->n_region - i) * sizeof(*region));
len = le32_to_cpu(region->len);
/* check if we have valid buffer size */
- if (offset + len > fw->size) {
+ if (len > fw_data_len - offset) {
dev_err(mdev->dev, "Invalid firmware region\n");
ret = -EINVAL;
goto out;
@@ -464,8 +473,24 @@ static int mt7921_load_clc(struct mt792x_dev *dev, const char *fw_name)
if (!clc_base)
goto out;
- for (offset = 0; offset < len; offset += le32_to_cpu(clc->len)) {
+ for (offset = 0; offset < len; offset += clc_len) {
+ if (len - offset < sizeof(*clc)) {
+ dev_err(mdev->dev, "Invalid CLC record\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
clc = (const struct mt7921_clc *)(clc_base + offset);
+ clc_len = le32_to_cpu(clc->len);
+ if (clc_len < sizeof(*clc) || clc_len > len - offset) {
+ dev_err(mdev->dev, "Invalid CLC record\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ /* Newer firmware may add records this driver does not use yet */
+ if (clc->idx >= ARRAY_SIZE(phy->clc))
+ continue;
/* do not init buf again if chip reset triggered */
if (phy->clc[clc->idx])
@@ -477,7 +502,7 @@ static int mt7921_load_clc(struct mt792x_dev *dev, const char *fw_name)
continue;
phy->clc[clc->idx] = devm_kmemdup(mdev->dev, clc,
- le32_to_cpu(clc->len),
+ clc_len,
GFP_KERNEL);
if (!phy->clc[clc->idx]) {
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 0b80db7a0599..a73cc6a43f5e 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -148,8 +148,6 @@ static const struct class nvme_ns_chr_class = {
};
static void nvme_put_subsystem(struct nvme_subsystem *subsys);
-static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl,
- unsigned nsid);
static void nvme_update_keep_alive(struct nvme_ctrl *ctrl,
struct nvme_command *cmd);
static int nvme_get_log_lsi(struct nvme_ctrl *ctrl, u32 nsid, u8 log_page,
@@ -1594,7 +1592,7 @@ static int nvme_identify_ns_descs(struct nvme_ctrl *ctrl,
}
if (nvme_multi_css(ctrl) && !csi_seen) {
- dev_warn(ctrl->device, "Command set not reported for nsid:%d\n",
+ dev_warn(ctrl->device, "Command set not reported for nsid:%u\n",
info->nsid);
status = -EINVAL;
}
@@ -2323,14 +2321,6 @@ static int nvme_query_fdp_info(struct nvme_ns *ns, struct nvme_ns_info *info)
size_t size;
int i, ret;
- /*
- * The FDP configuration is static for the lifetime of the namespace,
- * so return immediately if we've already registered this namespace's
- * streams.
- */
- if (head->nr_plids)
- return 0;
-
ret = nvme_get_features(ctrl, NVME_FEAT_FDP, info->endgid, NULL, 0,
&fdp);
if (ret) {
@@ -2377,6 +2367,7 @@ static int nvme_query_fdp_info(struct nvme_ns *ns, struct nvme_ns_info *info)
for (i = 0; i < head->nr_plids; i++)
head->plids[i] = le16_to_cpu(ruhs->ruhsd[i].pid);
+ head->write_stream_granularity = min(info->runs, U32_MAX);
free:
kfree(ruhs);
return ret;
@@ -2424,12 +2415,6 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns,
goto out;
}
- if (ns->ctrl->ctratt & NVME_CTRL_ATTR_FDPS) {
- ret = nvme_query_fdp_info(ns, info);
- if (ret < 0)
- goto out;
- }
-
if (nvme_invalid_lba_sz(le64_to_cpu(id->nsze),
id->lbaf[lbaf].ds - SECTOR_SHIFT, &capacity)) {
dev_warn_once(ns->ctrl->device,
@@ -2489,10 +2474,7 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns,
capacity = 0;
lim.max_write_streams = ns->head->nr_plids;
- if (lim.max_write_streams)
- lim.write_stream_granularity = min(info->runs, U32_MAX);
- else
- lim.write_stream_granularity = 0;
+ lim.write_stream_granularity = ns->head->write_stream_granularity;
/*
* Only set the DEAC bit if the device guarantees that reads from
@@ -3292,6 +3274,7 @@ static bool nvme_validate_cntlid(struct nvme_subsystem *subsys,
}
static int nvme_init_subsystem(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
+ __context_unsafe(/* initialize unpublished/lock-guarded variables */)
{
struct nvme_subsystem *subsys, *found;
int ret;
@@ -3863,6 +3846,7 @@ static const struct file_operations nvme_dev_fops = {
static struct nvme_ns_head *nvme_find_ns_head(struct nvme_ctrl *ctrl,
unsigned nsid)
+ __must_hold(&ctrl->subsys->lock)
{
struct nvme_ns_head *h;
@@ -3885,6 +3869,7 @@ static struct nvme_ns_head *nvme_find_ns_head(struct nvme_ctrl *ctrl,
static int nvme_subsys_check_duplicate_ids(struct nvme_subsystem *subsys,
struct nvme_ns_ids *ids)
+ __must_hold(&subsys->lock)
{
bool has_uuid = !uuid_is_null(&ids->uuid);
bool has_nguid = memchr_inv(ids->nguid, 0, sizeof(ids->nguid));
@@ -3984,9 +3969,11 @@ static void nvme_add_ns_cdev(struct nvme_ns *ns)
set_bit(NVME_NS_CDEV_LIVE, &ns->flags);
}
-static struct nvme_ns_head *nvme_alloc_ns_head(struct nvme_ctrl *ctrl,
+static struct nvme_ns_head *nvme_alloc_ns_head(struct nvme_ns *ns,
struct nvme_ns_info *info)
+ __must_hold(&ns->ctrl->subsys->lock)
{
+ struct nvme_ctrl *ctrl = ns->ctrl;
struct nvme_ns_head *head;
size_t size = sizeof(*head);
int ret = -ENOMEM;
@@ -4014,6 +4001,7 @@ static struct nvme_ns_head *nvme_alloc_ns_head(struct nvme_ctrl *ctrl,
ratelimit_state_init(&head->rs_nuse, 5 * HZ, 1);
ratelimit_set_flags(&head->rs_nuse, RATELIMIT_MSG_ON_RELEASE);
kref_init(&head->ref);
+ ns->head = head;
if (head->ids.csi) {
ret = nvme_get_effects_log(ctrl, head->ids.csi, &head->effects);
@@ -4022,21 +4010,30 @@ static struct nvme_ns_head *nvme_alloc_ns_head(struct nvme_ctrl *ctrl,
} else
head->effects = ctrl->effects;
+ if (ctrl->ctratt & NVME_CTRL_ATTR_FDPS) {
+ ret = nvme_query_fdp_info(ns, info);
+ if (ret < 0)
+ goto out_cleanup_srcu;
+ }
+
ret = nvme_mpath_alloc_disk(ctrl, head);
if (ret)
- goto out_cleanup_srcu;
+ goto out_cleanup_fdp;
list_add_tail(&head->entry, &ctrl->subsys->nsheads);
kref_get(&ctrl->subsys->ref);
return head;
+out_cleanup_fdp:
+ kfree(head->plids);
out_cleanup_srcu:
cleanup_srcu_struct(&head->srcu);
out_ida_remove:
ida_free(&ctrl->subsys->ns_ida, head->instance);
out_free_head:
kfree(head);
+ ns->head = NULL;
out:
if (ret > 0)
ret = blk_status_to_errno(nvme_error_status(ret));
@@ -4098,13 +4095,13 @@ static int nvme_init_ns_head(struct nvme_ns *ns, struct nvme_ns_info *info)
((ns->ctrl->subsys->cmic & NVME_CTRL_CMIC_MULTI_CTRL) &&
info->is_shared)) {
dev_err(ctrl->device,
- "ignoring nsid %d because of duplicate IDs\n",
+ "ignoring nsid %u because of duplicate IDs\n",
info->nsid);
return ret;
}
dev_err(ctrl->device,
- "clearing duplicate IDs for nsid %d\n", info->nsid);
+ "clearing duplicate IDs for nsid %u\n", info->nsid);
dev_err(ctrl->device,
"use of /dev/disk/by-id/ may cause data corruption\n");
memset(&info->ids.nguid, 0, sizeof(info->ids.nguid));
@@ -4119,11 +4116,11 @@ static int nvme_init_ns_head(struct nvme_ns *ns, struct nvme_ns_info *info)
ret = nvme_subsys_check_duplicate_ids(ctrl->subsys, &info->ids);
if (ret) {
dev_err(ctrl->device,
- "duplicate IDs in subsystem for nsid %d\n",
+ "duplicate IDs in subsystem for nsid %u\n",
info->nsid);
goto out_unlock;
}
- head = nvme_alloc_ns_head(ctrl, info);
+ head = nvme_alloc_ns_head(ns, info);
if (IS_ERR(head)) {
ret = PTR_ERR(head);
goto out_unlock;
@@ -4133,20 +4130,20 @@ static int nvme_init_ns_head(struct nvme_ns *ns, struct nvme_ns_info *info)
if ((!info->is_shared || !head->shared) &&
!list_empty(&head->list)) {
dev_err(ctrl->device,
- "Duplicate unshared namespace %d\n",
+ "Duplicate unshared namespace %u\n",
info->nsid);
goto out_put_ns_head;
}
if (!nvme_ns_ids_equal(&head->ids, &info->ids)) {
dev_err(ctrl->device,
- "IDs don't match for shared namespace %d\n",
+ "IDs don't match for shared namespace %u\n",
info->nsid);
goto out_put_ns_head;
}
if (!multipath) {
dev_warn(ctrl->device,
- "Found shared namespace %d, but multipathing not supported.\n",
+ "Found shared namespace %u, but multipathing not supported.\n",
info->nsid);
dev_warn_once(ctrl->device,
"Shared namespace support requires core_nvme.multipath=Y.\n");
@@ -4395,7 +4392,7 @@ static void nvme_validate_ns(struct nvme_ns *ns, struct nvme_ns_info *info)
if (!nvme_ns_ids_equal(&ns->head->ids, &info->ids)) {
dev_err(ns->ctrl->device,
- "identifiers changed for nsid %d\n", ns->head->ns_id);
+ "identifiers changed for nsid %u\n", ns->head->ns_id);
goto out;
}
@@ -4422,7 +4419,7 @@ static void nvme_scan_ns(struct nvme_ctrl *ctrl, unsigned nsid)
if (info.ids.csi != NVME_CSI_NVM && !nvme_multi_css(ctrl)) {
dev_warn(ctrl->device,
- "command set not reported for nsid: %d\n", nsid);
+ "command set not reported for nsid: %u\n", nsid);
return;
}
@@ -4486,15 +4483,16 @@ static void nvme_scan_ns_async(void *data, async_cookie_t cookie)
nvme_scan_ns(scan_info->ctrl, nsid);
}
-static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl,
- unsigned nsid)
+static void nvme_remove_nsid_range(struct nvme_ctrl *ctrl, u32 start, u32 end)
{
struct nvme_ns *ns, *next;
LIST_HEAD(rm_list);
mutex_lock(&ctrl->namespaces_lock);
list_for_each_entry_safe(ns, next, &ctrl->namespaces, list) {
- if (ns->head->ns_id > nsid) {
+ if (ns->head->ns_id >= end)
+ break;
+ if (ns->head->ns_id > start) {
list_del_rcu(&ns->list);
synchronize_srcu(&ctrl->srcu);
list_add_tail_rcu(&ns->list, &rm_list);
@@ -4544,13 +4542,14 @@ static int nvme_scan_ns_list(struct nvme_ctrl *ctrl)
goto out;
async_schedule_domain(nvme_scan_ns_async, &scan_info,
&domain);
- while (++prev < nsid)
- nvme_ns_remove_by_nsid(ctrl, prev);
+ if (prev + 1 < nsid)
+ nvme_remove_nsid_range(ctrl, prev, nsid);
+ prev = max(prev + 1, nsid);
}
async_synchronize_full_domain(&domain);
}
out:
- nvme_remove_invalid_namespaces(ctrl, prev);
+ nvme_remove_nsid_range(ctrl, prev, UINT_MAX);
free:
async_synchronize_full_domain(&domain);
kfree(ns_list);
@@ -4570,7 +4569,7 @@ static void nvme_scan_ns_sequential(struct nvme_ctrl *ctrl)
for (i = 1; i <= nn; i++)
nvme_scan_ns(ctrl, i);
- nvme_remove_invalid_namespaces(ctrl, nn);
+ nvme_remove_nsid_range(ctrl, nn, UINT_MAX);
}
static void nvme_clear_changed_ns_log(struct nvme_ctrl *ctrl)
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 957ded0c6f53..cfdcf53db5ab 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -503,7 +503,8 @@ struct nvme_subsystem {
struct list_head entry;
struct mutex lock;
struct list_head ctrls;
- struct list_head nsheads;
+ struct list_head nsheads
+ __guarded_by(&lock);
char subnqn[NVMF_NQN_SIZE];
char serial[20];
char model[40];
@@ -567,6 +568,7 @@ struct nvme_ns_head {
u16 nr_plids;
u16 *plids;
+ u32 write_stream_granularity;
#ifdef CONFIG_NVME_MULTIPATH
struct bio_list requeue_list;
spinlock_t requeue_lock;
@@ -575,7 +577,8 @@ struct nvme_ns_head {
struct mutex lock;
unsigned long flags;
struct delayed_work remove_work;
- unsigned int delayed_removal_secs;
+ unsigned int delayed_removal_secs
+ __guarded_by(&subsys->lock);
atomic_long_t io_requeue_no_usable_path_count;
atomic_long_t io_fail_no_available_path_count;
#define NVME_NSHEAD_DISK_LIVE 0
@@ -1046,9 +1049,12 @@ static inline bool nvme_ctrl_use_ana(struct nvme_ctrl *ctrl)
return ctrl->ana_log_buf != NULL;
}
-void nvme_mpath_unfreeze(struct nvme_subsystem *subsys);
-void nvme_mpath_wait_freeze(struct nvme_subsystem *subsys);
-void nvme_mpath_start_freeze(struct nvme_subsystem *subsys);
+void nvme_mpath_unfreeze(struct nvme_subsystem *subsys)
+ __must_hold(&subsys->lock);
+void nvme_mpath_wait_freeze(struct nvme_subsystem *subsys)
+ __must_hold(&subsys->lock);
+void nvme_mpath_start_freeze(struct nvme_subsystem *subsys)
+ __must_hold(&subsys->lock);
void nvme_mpath_default_iopolicy(struct nvme_subsystem *subsys);
void nvme_failover_req(struct request *req);
void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl);
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 6909e3542794..618ee9544426 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -2003,7 +2003,7 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
struct ib_device *dev;
bool queue_ready = test_bit(NVME_RDMA_Q_LIVE, &queue->flags);
blk_status_t ret;
- int err;
+ int err = 0;
WARN_ON_ONCE(rq->tag < 0);
@@ -2059,16 +2059,18 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
err_unmap:
nvme_rdma_unmap_data(queue, rq);
err:
- if (err == -EIO)
- ret = nvme_host_path_error(rq);
- else if (err == -ENOMEM || err == -EAGAIN)
- ret = BLK_STS_RESOURCE;
- else
- ret = BLK_STS_IOERR;
- nvme_cleanup_cmd(rq);
+ if (err != -EIO) {
+ nvme_cleanup_cmd(rq);
+ if (err == -ENOMEM || err == -EAGAIN)
+ ret = BLK_STS_RESOURCE;
+ else
+ ret = BLK_STS_IOERR;
+ }
unmap_qe:
ib_dma_unmap_single(dev, req->sqe.dma, sizeof(struct nvme_command),
DMA_TO_DEVICE);
+ if (err == -EIO)
+ return nvme_host_path_error(rq);
return ret;
}
diff --git a/drivers/nvme/host/sysfs.c b/drivers/nvme/host/sysfs.c
index 75b2d69b5957..be94739e17dc 100644
--- a/drivers/nvme/host/sysfs.c
+++ b/drivers/nvme/host/sysfs.c
@@ -166,7 +166,7 @@ static DEVICE_ATTR_RO(eui);
static ssize_t nsid_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- return sysfs_emit(buf, "%d\n", dev_to_ns_head(dev)->ns_id);
+ return sysfs_emit(buf, "%u\n", dev_to_ns_head(dev)->ns_id);
}
static DEVICE_ATTR_RO(nsid);
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index ddbb34a22f68..52efc6e4f762 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -411,8 +411,13 @@ static inline void nvme_tcp_queue_request(struct nvme_tcp_request *req,
* if we're the first on the send_list and we can try to send
* directly, otherwise queue io_work. Also, only do that if we
* are on the same cpu, so we don't introduce contention.
+ *
+ * TLS kTLS send takes ctx->tx_lock while blk_mq holds set->srcu.
+ * lockdep reports circular locking via elevator_lock. Defer TLS
+ * sends to the io workqueue instead of inline from this path.
*/
if (queue->io_cpu == raw_smp_processor_id() &&
+ !nvme_tcp_queue_tls(queue) &&
empty && mutex_trylock(&queue->send_mutex)) {
nvme_tcp_send_all(queue);
mutex_unlock(&queue->send_mutex);
diff --git a/drivers/nvme/host/zns.c b/drivers/nvme/host/zns.c
index 8ed1b6a33454..9477cfbd3a51 100644
--- a/drivers/nvme/host/zns.c
+++ b/drivers/nvme/host/zns.c
@@ -48,12 +48,12 @@ int nvme_query_zone_info(struct nvme_ns *ns, unsigned lbaf,
NVME_CMD_EFFECTS_CSUPP)) {
if (test_and_clear_bit(NVME_NS_FORCE_RO, &ns->flags))
dev_warn(ns->ctrl->device,
- "Zone Append supported for zoned namespace:%d. Remove read-only mode\n",
+ "Zone Append supported for zoned namespace:%u. Remove read-only mode\n",
ns->head->ns_id);
} else {
set_bit(NVME_NS_FORCE_RO, &ns->flags);
dev_warn(ns->ctrl->device,
- "Zone Append not supported for zoned namespace:%d. Forcing to read-only mode\n",
+ "Zone Append not supported for zoned namespace:%u. Forcing to read-only mode\n",
ns->head->ns_id);
}
diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 413ee2d16d29..6286e38436dd 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -814,7 +814,7 @@ static ssize_t nvmet_ns_resv_enable_store(struct config_item *item,
mutex_lock(&ns->subsys->lock);
if (ns->enabled) {
- pr_err("the ns:%d is already enabled.\n", ns->nsid);
+ pr_err("the ns:%u is already enabled.\n", ns->nsid);
mutex_unlock(&ns->subsys->lock);
return -EINVAL;
}
@@ -880,7 +880,7 @@ static struct config_group *nvmet_ns_make(struct config_group *group,
goto out;
config_group_init_type_name(&ns->group, name, &nvmet_ns_type);
- pr_info("adding nsid %d to subsystem %s\n", nsid, subsys->subsysnqn);
+ pr_info("adding nsid %u to subsystem %s\n", nsid, subsys->subsysnqn);
return &ns->group;
out:
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 23fa90ec3c00..269daefb8885 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -558,7 +558,7 @@ static void nvmet_p2pmem_ns_add_p2p(struct nvmet_ctrl *ctrl,
if (ret < 0)
pci_dev_put(p2p_dev);
- pr_info("using p2pmem on %s for nsid %d\n", pci_name(p2p_dev),
+ pr_info("using p2pmem on %s for nsid %u\n", pci_name(p2p_dev),
ns->nsid);
}
diff --git a/drivers/nvme/target/pr.c b/drivers/nvme/target/pr.c
index 7d937093b226..ad07f55e7f17 100644
--- a/drivers/nvme/target/pr.c
+++ b/drivers/nvme/target/pr.c
@@ -145,7 +145,7 @@ static void nvmet_pr_add_resv_log(struct nvmet_ctrl *ctrl, u8 log_type,
log.nsid = cpu_to_le32(nsid);
if (!kfifo_put(&log_mgr->log_queue, log)) {
- pr_info("a reservation log lost, cntlid:%d, log_type:%d, nsid:%d\n",
+ pr_info("a reservation log lost, cntlid:%d, log_type:%d, nsid:%u\n",
ctrl->cntlid, log_type, nsid);
log_mgr->lost_count++;
}
diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
index de5a88fbb233..542138fd669f 100644
--- a/drivers/nvme/target/rdma.c
+++ b/drivers/nvme/target/rdma.c
@@ -1627,19 +1627,13 @@ static int nvmet_rdma_queue_connect(struct rdma_cm_id *cm_id,
mutex_unlock(&nvmet_rdma_queue_mutex);
if (pending > NVMET_RDMA_BACKLOG) {
ret = NVME_SC_CONNECT_CTRL_BUSY;
- goto put_device;
+ goto free_queue;
}
}
ret = nvmet_rdma_cm_accept(cm_id, queue, &event->param.conn);
- if (ret) {
- /*
- * Don't destroy the cm_id in free path, as we implicitly
- * destroy the cm_id here with non-zero ret code.
- */
- queue->cm_id = NULL;
+ if (ret)
goto free_queue;
- }
mutex_lock(&nvmet_rdma_queue_mutex);
list_add_tail(&queue->queue_list, &nvmet_rdma_queue_list);
@@ -1648,6 +1642,11 @@ static int nvmet_rdma_queue_connect(struct rdma_cm_id *cm_id,
return 0;
free_queue:
+ /*
+ * Don't destroy the cm_id in free path, as we implicitly
+ * destroy the cm_id here with non-zero ret code.
+ */
+ queue->cm_id = NULL;
nvmet_rdma_free_queue(queue);
put_device:
kref_put(&ndev->ref, nvmet_rdma_free_dev);
diff --git a/drivers/opp/of.c b/drivers/opp/of.c
index c02e20632fa6..9c4fd1f0e944 100644
--- a/drivers/opp/of.c
+++ b/drivers/opp/of.c
@@ -1039,7 +1039,7 @@ static int _of_add_opp_table_v1(struct device *dev, struct opp_table *opp_table)
val = prop->value;
while (nr) {
- unsigned long freq = be32_to_cpup(val++) * 1000;
+ unsigned long freq = (unsigned long)be32_to_cpup(val++) * 1000;
unsigned long volt = be32_to_cpup(val++);
struct dev_pm_opp_data data = {
.freq = freq,
diff --git a/drivers/perf/riscv_pmu_legacy.c b/drivers/perf/riscv_pmu_legacy.c
index 4d6461d6a74f..1b8e4789cb3c 100644
--- a/drivers/perf/riscv_pmu_legacy.c
+++ b/drivers/perf/riscv_pmu_legacy.c
@@ -110,8 +110,9 @@ static void pmu_legacy_init(struct riscv_pmu *pmu)
{
pr_info("Legacy PMU implementation is available\n");
- pmu->cmask = BIT(RISCV_PMU_LEGACY_CYCLE) |
- BIT(RISCV_PMU_LEGACY_INSTRET);
+ bitmap_zero(pmu->cmask, RISCV_MAX_COUNTERS);
+ set_bit(RISCV_PMU_LEGACY_CYCLE, pmu->cmask);
+ set_bit(RISCV_PMU_LEGACY_INSTRET, pmu->cmask);
pmu->ctr_start = pmu_legacy_ctr_start;
pmu->ctr_stop = NULL;
pmu->event_map = pmu_legacy_event_map;
diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c
index dfc886dee5ad..2e89d2fee53a 100644
--- a/drivers/perf/riscv_pmu_sbi.c
+++ b/drivers/perf/riscv_pmu_sbi.c
@@ -97,7 +97,7 @@ static unsigned int riscv_pmu_irq_mask;
static unsigned int riscv_pmu_irq;
/* Cache the available counters in a bitmask */
-static unsigned long cmask;
+static DECLARE_BITMAP(cmask, RISCV_MAX_COUNTERS);
static int pmu_event_find_cache(u64 config);
struct sbi_pmu_event_data {
@@ -359,16 +359,38 @@ static int pmu_sbi_check_event_info(void)
return result;
}
+static struct sbiret pmu_sbi_ctr_cfg_match(unsigned long cbase,
+ unsigned long ctr_mask,
+ unsigned long cflags,
+ unsigned long event_idx,
+ u64 config)
+{
+#if defined(CONFIG_32BIT)
+ return sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_CFG_MATCH, cbase,
+ ctr_mask, cflags, event_idx, config, config >> 32);
+#else
+ return sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_CFG_MATCH, cbase,
+ ctr_mask, cflags, event_idx, config, 0);
+#endif
+}
+
static void pmu_sbi_check_event(struct sbi_pmu_event_data *edata)
{
- struct sbiret ret;
+ struct sbiret ret = { .error = SBI_ERR_NOT_SUPPORTED };
+ int i;
- ret = sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_CFG_MATCH,
- 0, cmask, 0, edata->event_idx, 0, 0);
- if (!ret.error) {
- sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_STOP,
- ret.value, 0x1, SBI_PMU_STOP_FLAG_RESET, 0, 0, 0);
- } else if (ret.error == SBI_ERR_NOT_SUPPORTED) {
+ for (i = 0; i < BITS_TO_LONGS(RISCV_MAX_COUNTERS); i++) {
+ if (!cmask[i])
+ continue;
+ ret = pmu_sbi_ctr_cfg_match(i * BITS_PER_LONG, cmask[i], 0,
+ edata->event_idx, 0);
+ if (!ret.error) {
+ sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_STOP,
+ ret.value, 0x1, SBI_PMU_STOP_FLAG_RESET, 0, 0, 0);
+ return;
+ }
+ }
+ if (ret.error == SBI_ERR_NOT_SUPPORTED) {
/* This event cannot be monitored by any counter */
edata->event_idx = -ENOENT;
}
@@ -488,10 +510,10 @@ int riscv_pmu_get_hpm_info(u32 *hw_ctr_width, u32 *num_hw_ctr)
union sbi_pmu_ctr_info *info;
u32 hpm_width = 0, hpm_count = 0;
- if (!cmask)
+ if (bitmap_empty(cmask, RISCV_MAX_COUNTERS))
return -EINVAL;
- for_each_set_bit(i, &cmask, RISCV_MAX_COUNTERS) {
+ for_each_set_bit(i, cmask, RISCV_MAX_COUNTERS) {
info = &pmu_ctr_list[i];
if (!info)
continue;
@@ -540,8 +562,8 @@ static int pmu_sbi_ctr_get_idx(struct perf_event *event)
struct riscv_pmu *rvpmu = to_riscv_pmu(event->pmu);
struct cpu_hw_events *cpuc = this_cpu_ptr(rvpmu->hw_events);
struct sbiret ret;
- int idx;
- uint64_t cbase = 0, cmask = rvpmu->cmask;
+ int idx, i;
+ u64 cbase = 0, cmask = 0;
unsigned long cflags = 0;
cflags = pmu_sbi_get_filter_flags(event);
@@ -562,14 +584,21 @@ static int pmu_sbi_ctr_get_idx(struct perf_event *event)
}
/* retrieve the available counter index */
-#if defined(CONFIG_32BIT)
- ret = sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_CFG_MATCH, cbase,
- cmask, cflags, hwc->event_base, hwc->config,
- hwc->config >> 32);
-#else
- ret = sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_CFG_MATCH, cbase,
- cmask, cflags, hwc->event_base, hwc->config, 0);
-#endif
+ if (cmask) {
+ ret = pmu_sbi_ctr_cfg_match(cbase, cmask, cflags, hwc->event_base,
+ hwc->config);
+ } else {
+ ret.error = SBI_ERR_NOT_SUPPORTED;
+ for (i = 0; i < BITS_TO_LONGS(RISCV_MAX_COUNTERS); i++) {
+ if (!rvpmu->cmask[i])
+ continue;
+ cbase = i * BITS_PER_LONG;
+ ret = pmu_sbi_ctr_cfg_match(cbase, rvpmu->cmask[i], cflags,
+ hwc->event_base, hwc->config);
+ if (!ret.error)
+ break;
+ }
+ }
if (ret.error) {
pr_debug("Not able to find a counter for event %lx config %llx\n",
hwc->event_base, hwc->config);
@@ -577,7 +606,7 @@ static int pmu_sbi_ctr_get_idx(struct perf_event *event)
}
idx = ret.value;
- if (!test_bit(idx, &rvpmu->cmask) || !pmu_ctr_list[idx].value)
+ if (!test_bit(idx, rvpmu->cmask) || !pmu_ctr_list[idx].value)
return -ENOENT;
/* Additional sanity check for the counter id */
@@ -881,7 +910,7 @@ static int pmu_sbi_get_ctrinfo(int nctr, unsigned long *mask)
/* The logical counter ids are not expected to be contiguous */
continue;
- *mask |= BIT(i);
+ set_bit(i, mask);
cinfo.value = ret.value;
if (cinfo.type == SBI_PMU_CTR_TYPE_FW)
@@ -898,12 +927,19 @@ static int pmu_sbi_get_ctrinfo(int nctr, unsigned long *mask)
static inline void pmu_sbi_stop_all(struct riscv_pmu *pmu)
{
+ int i;
+
/*
* No need to check the error because we are disabling all the counters
* which may include counters that are not enabled yet.
*/
- sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_STOP,
- 0, pmu->cmask, SBI_PMU_STOP_FLAG_RESET, 0, 0, 0);
+ for (i = 0; i < BITS_TO_LONGS(RISCV_MAX_COUNTERS); i++) {
+ if (!pmu->cmask[i])
+ continue;
+ sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_STOP,
+ i * BITS_PER_LONG, pmu->cmask[i],
+ SBI_PMU_STOP_FLAG_RESET, 0, 0, 0);
+ }
}
static inline void pmu_sbi_stop_hw_ctrs(struct riscv_pmu *pmu)
@@ -1002,7 +1038,7 @@ static inline void pmu_sbi_start_ovf_ctrs_snapshot(struct cpu_hw_events *cpu_hw_
struct riscv_pmu_snapshot_data *sdata = cpu_hw_evt->snapshot_addr;
for_each_set_bit(idx, cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS) {
- if (ctr_ovf_mask & BIT(idx)) {
+ if (ctr_ovf_mask & BIT_ULL(idx)) {
event = cpu_hw_evt->events[idx];
hwc = &event->hw;
max_period = riscv_pmu_ctr_get_width_mask(event);
@@ -1050,11 +1086,13 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
u64 overflowed_ctrs = 0;
struct cpu_hw_events *cpu_hw_evt = dev;
u64 start_clock = sched_clock();
- struct riscv_pmu_snapshot_data *sdata = cpu_hw_evt->snapshot_addr;
+ struct riscv_pmu_snapshot_data *sdata;
if (WARN_ON_ONCE(!cpu_hw_evt))
return IRQ_NONE;
+ sdata = cpu_hw_evt->snapshot_addr;
+
/* Firmware counter don't support overflow yet */
fidx = find_first_bit(cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS);
if (fidx == RISCV_MAX_COUNTERS) {
@@ -1109,14 +1147,14 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
hidx = info->csr - CSR_CYCLE;
/* check if the corresponding bit is set in scountovf or overflow mask in shmem */
- if (!(overflow & BIT(hidx)))
+ if (!(overflow & BIT_ULL(hidx)))
continue;
/*
* Keep a track of overflowed counters so that they can be started
* with updated initial value.
*/
- overflowed_ctrs |= BIT(lidx);
+ overflowed_ctrs |= BIT_ULL(lidx);
hw_evt = &event->hw;
/* Update the event states here so that we know the state while reading */
hw_evt->state |= PERF_HES_STOPPED;
@@ -1442,7 +1480,7 @@ static int pmu_sbi_device_probe(struct platform_device *pdev)
}
/* cache all the information about counters now */
- if (pmu_sbi_get_ctrinfo(num_counters, &cmask))
+ if (pmu_sbi_get_ctrinfo(num_counters, cmask))
goto out_free;
ret = pmu_sbi_setup_irqs(pmu, pdev);
@@ -1454,7 +1492,7 @@ static int pmu_sbi_device_probe(struct platform_device *pdev)
pmu->pmu.attr_groups = riscv_pmu_attr_groups;
pmu->pmu.parent = &pdev->dev;
- pmu->cmask = cmask;
+ bitmap_copy(pmu->cmask, cmask, RISCV_MAX_COUNTERS);
pmu->ctr_start = pmu_sbi_ctr_start;
pmu->ctr_stop = pmu_sbi_ctr_stop;
pmu->event_map = pmu_sbi_event_map;
diff --git a/drivers/platform/x86/asus-laptop.c b/drivers/platform/x86/asus-laptop.c
index 140ac8a10537..19024fa3796e 100644
--- a/drivers/platform/x86/asus-laptop.c
+++ b/drivers/platform/x86/asus-laptop.c
@@ -1887,7 +1887,7 @@ static int asus_acpi_probe(struct platform_device *pdev)
if (result && result != -ENODEV)
goto fail_pega_rfkill;
- result = acpi_dev_install_notify_handler(device, ACPI_DEVICE_NOTIFY,
+ result = acpi_dev_install_notify_handler(device, ACPI_ALL_NOTIFY,
asus_acpi_notify, asus);
if (result)
goto fail_pega_rfkill;
@@ -1917,7 +1917,7 @@ static void asus_acpi_remove(struct platform_device *pdev)
{
struct asus_laptop *asus = platform_get_drvdata(pdev);
- acpi_dev_remove_notify_handler(asus->device, ACPI_DEVICE_NOTIFY,
+ acpi_dev_remove_notify_handler(asus->device, ACPI_ALL_NOTIFY,
asus_acpi_notify);
asus_backlight_exit(asus);
asus_rfkill_exit(asus);
diff --git a/drivers/platform/x86/hp/hp-wmi.c b/drivers/platform/x86/hp/hp-wmi.c
index 94147102cca4..051676c7ec1a 100644
--- a/drivers/platform/x86/hp/hp-wmi.c
+++ b/drivers/platform/x86/hp/hp-wmi.c
@@ -133,11 +133,35 @@ static const struct thermal_profile_params omen_v1_no_ec_thermal_params = {
.ec_tp_offset = HP_NO_THERMAL_PROFILE_OFFSET,
};
-/*
- * A generic pointer for the currently-active board's thermal profile
- * parameters.
- */
-static struct thermal_profile_params *active_thermal_profile_params;
+struct hp_wmi_board_params {
+ const struct thermal_profile_params *thermal_profile;
+};
+
+static const struct hp_wmi_board_params victus_s_board_params = {
+ .thermal_profile = &victus_s_thermal_params,
+};
+
+static const struct hp_wmi_board_params omen_v1_board_params = {
+ .thermal_profile = &omen_v1_thermal_params,
+};
+
+static const struct hp_wmi_board_params omen_v1_legacy_board_params = {
+ .thermal_profile = &omen_v1_legacy_thermal_params,
+};
+
+static const struct hp_wmi_board_params omen_v1_no_ec_board_params = {
+ .thermal_profile = &omen_v1_no_ec_thermal_params,
+};
+
+static const struct hp_wmi_board_params *active_board_params;
+
+static const struct thermal_profile_params *hp_wmi_thermal_profile(void)
+{
+ if (!active_board_params)
+ return NULL;
+
+ return active_board_params->thermal_profile;
+}
/* DMI board names of devices that should use the omen specific path for
* thermal profiles.
@@ -187,91 +211,91 @@ static const char * const victus_thermal_profile_boards[] = {
"8A25",
};
-/* DMI Board names of Victus 16-r and Victus 16-s laptops */
-static const struct dmi_system_id victus_s_thermal_profile_boards[] __initconst = {
+/* DMI board-specific feature data for Omen and Victus laptops. */
+static const struct dmi_system_id hp_wmi_feature_boards[] __initconst = {
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8902") },
- .driver_data = (void *)&omen_v1_legacy_thermal_params,
+ .driver_data = (void *)&omen_v1_legacy_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8A44") },
- .driver_data = (void *)&omen_v1_legacy_thermal_params,
+ .driver_data = (void *)&omen_v1_legacy_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8A4D") },
- .driver_data = (void *)&omen_v1_legacy_thermal_params,
+ .driver_data = (void *)&omen_v1_legacy_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8BAB") },
- .driver_data = (void *)&omen_v1_thermal_params,
+ .driver_data = (void *)&omen_v1_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8B2F") },
- .driver_data = (void *)&victus_s_thermal_params,
+ .driver_data = (void *)&victus_s_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8BBE") },
- .driver_data = (void *)&victus_s_thermal_params,
+ .driver_data = (void *)&victus_s_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8BC2") },
- .driver_data = (void *)&omen_v1_thermal_params,
+ .driver_data = (void *)&omen_v1_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8BCA") },
- .driver_data = (void *)&omen_v1_thermal_params,
+ .driver_data = (void *)&omen_v1_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8BCD") },
- .driver_data = (void *)&omen_v1_thermal_params,
+ .driver_data = (void *)&omen_v1_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8BD4") },
- .driver_data = (void *)&victus_s_thermal_params,
+ .driver_data = (void *)&victus_s_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8BD5") },
- .driver_data = (void *)&victus_s_thermal_params,
+ .driver_data = (void *)&victus_s_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8C76") },
- .driver_data = (void *)&omen_v1_thermal_params,
+ .driver_data = (void *)&omen_v1_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8C77") },
- .driver_data = (void *)&omen_v1_thermal_params,
+ .driver_data = (void *)&omen_v1_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8C78") },
- .driver_data = (void *)&omen_v1_thermal_params,
+ .driver_data = (void *)&omen_v1_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8C99") },
- .driver_data = (void *)&victus_s_thermal_params,
+ .driver_data = (void *)&victus_s_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8C9C") },
- .driver_data = (void *)&victus_s_thermal_params,
+ .driver_data = (void *)&victus_s_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8D26") },
- .driver_data = (void *)&omen_v1_legacy_thermal_params,
+ .driver_data = (void *)&omen_v1_legacy_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8D41") },
- .driver_data = (void *)&omen_v1_no_ec_thermal_params,
+ .driver_data = (void *)&omen_v1_no_ec_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8D87") },
- .driver_data = (void *)&omen_v1_no_ec_thermal_params,
+ .driver_data = (void *)&omen_v1_no_ec_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8DD6") },
- .driver_data = (void *)&omen_v1_no_ec_thermal_params,
+ .driver_data = (void *)&omen_v1_no_ec_board_params,
},
{
.matches = { DMI_MATCH(DMI_BOARD_NAME, "8E35") },
- .driver_data = (void *)&omen_v1_legacy_thermal_params,
+ .driver_data = (void *)&omen_v1_legacy_board_params,
},
{},
};
@@ -1878,7 +1902,10 @@ static int platform_profile_victus_s_get_ec(enum platform_profile_option *profil
u8 current_dstate, current_gpu_slowdown_temp, tp;
const struct thermal_profile_params *params;
- params = active_thermal_profile_params;
+ params = hp_wmi_thermal_profile();
+ if (!params)
+ return -ENODEV;
+
if (params->ec_tp_offset == HP_EC_OFFSET_UNKNOWN ||
params->ec_tp_offset == HP_NO_THERMAL_PROFILE_OFFSET) {
*profile = active_platform_profile;
@@ -1890,10 +1917,10 @@ static int platform_profile_victus_s_get_ec(enum platform_profile_option *profil
return ret;
/*
- * We cannot use active_thermal_profile_params here, because boards
- * like 8C78 have tp == 0x0 || tp == 0x1 after cold boot, but logically
- * it should have tp == 0x30 || tp == 0x31, as corrected by the Omen
- * Gaming Hub on windows. Hence accept both of these values.
+ * Boards like 8C78 have tp == 0x0 || tp == 0x1 after cold boot,
+ * but logically it should have tp == 0x30 || tp == 0x31, as
+ * corrected by the Omen Gaming Hub on windows. Hence accept both
+ * of these values.
*/
if (tp == victus_s_thermal_params.performance ||
tp == omen_v1_thermal_params.performance) {
@@ -1928,12 +1955,12 @@ static int platform_profile_victus_s_get_ec(enum platform_profile_option *profil
static int platform_profile_victus_s_set_ec(enum platform_profile_option profile)
{
- struct thermal_profile_params *params;
+ const struct thermal_profile_params *params;
bool gpu_ctgp_enable, gpu_ppab_enable;
u8 gpu_dstate; /* Test shows 1 = 100%, 2 = 50%, 3 = 25%, 4 = 12.5% */
int err, tp;
- params = active_thermal_profile_params;
+ params = hp_wmi_thermal_profile();
if (!params)
return -ENODEV;
@@ -2199,6 +2226,7 @@ static const struct platform_profile_ops hp_wmi_platform_profile_ops = {
static int thermal_profile_setup(struct platform_device *device)
{
const struct platform_profile_ops *ops;
+ const struct thermal_profile_params *params;
int err, tp;
if (is_omen_thermal_profile()) {
@@ -2230,13 +2258,17 @@ static int thermal_profile_setup(struct platform_device *device)
ops = &platform_profile_victus_ops;
} else if (is_victus_s_thermal_profile()) {
+ params = hp_wmi_thermal_profile();
+ if (!params)
+ return -ENODEV;
+
/*
* For an unknown EC layout board, platform_profile_victus_s_get_ec(),
* behaves like a wrapper around active_platform_profile, to avoid using
* uninitialized data, we default to PLATFORM_PROFILE_BALANCED.
*/
- if (active_thermal_profile_params->ec_tp_offset == HP_EC_OFFSET_UNKNOWN ||
- active_thermal_profile_params->ec_tp_offset == HP_NO_THERMAL_PROFILE_OFFSET) {
+ if (params->ec_tp_offset == HP_EC_OFFSET_UNKNOWN ||
+ params->ec_tp_offset == HP_NO_THERMAL_PROFILE_OFFSET) {
active_platform_profile = PLATFORM_PROFILE_BALANCED;
} else {
err = platform_profile_victus_s_get_ec(&active_platform_profile);
@@ -2697,24 +2729,25 @@ static int hp_wmi_hwmon_init(void)
return 0;
}
-static void __init setup_active_thermal_profile_params(void)
+static void __init setup_active_board_params(void)
{
const struct dmi_system_id *id;
+ const struct thermal_profile_params *params;
- /*
- * Currently only victus_s devices use the
- * active_thermal_profile_params
- */
- id = dmi_first_match(victus_s_thermal_profile_boards);
+ id = dmi_first_match(hp_wmi_feature_boards);
if (id) {
+ active_board_params = id->driver_data;
+ params = hp_wmi_thermal_profile();
+ if (!params)
+ return;
+
/*
* Marking this boolean is required to ensure that
* is_victus_s_thermal_profile() behaves like a valid
* wrapper.
*/
is_victus_s_board = true;
- active_thermal_profile_params = id->driver_data;
- if (active_thermal_profile_params->ec_tp_offset == HP_EC_OFFSET_UNKNOWN) {
+ if (params->ec_tp_offset == HP_EC_OFFSET_UNKNOWN) {
pr_warn("Unknown EC layout for board %s. Thermal profile readback will be disabled. Please report this to platform-driver-x86@vger.kernel.org\n",
dmi_get_system_info(DMI_BOARD_NAME));
}
@@ -2749,10 +2782,10 @@ static int __init hp_wmi_init(void)
}
/*
- * Setup active board's thermal profile parameters before
- * starting platform driver probe.
+ * Setup active board feature data before starting platform
+ * driver probe.
*/
- setup_active_thermal_profile_params();
+ setup_active_board_params();
err = platform_driver_probe(&hp_wmi_driver, hp_wmi_bios_setup);
if (err)
goto err_unregister_device;
diff --git a/drivers/platform/x86/x86-android-tablets/core.c b/drivers/platform/x86/x86-android-tablets/core.c
index 5db794d65eb5..7751261f4b7c 100644
--- a/drivers/platform/x86/x86-android-tablets/core.c
+++ b/drivers/platform/x86/x86-android-tablets/core.c
@@ -367,6 +367,7 @@ static void gpio_secondary_unset(void *data)
struct device *dev = data;
set_secondary_fwnode(dev, NULL);
+ put_device(dev);
}
static void gpio_secondary_unregister_node_group(void *data)
@@ -380,6 +381,7 @@ static int gpio_secondary_fwnode_init(struct device *parent)
{
const struct software_node *const *swnode;
struct fwnode_handle *fwnode;
+ struct device *phys_dev;
int ret;
if (!gpiochip_node_group)
@@ -407,9 +409,15 @@ static int gpio_secondary_fwnode_init(struct device *parent)
if (WARN_ON(!fwnode))
return -ENOENT;
- set_secondary_fwnode(dev, fwnode);
+ phys_dev = acpi_get_first_physical_node(to_acpi_device(dev));
+ if (!phys_dev)
+ return dev_err_probe(parent, -ENODEV,
+ "No physical device for ACPI GPIO dev: %pfwP\n",
+ fwnode);
- ret = devm_add_action_or_reset(parent, gpio_secondary_unset, dev);
+ set_secondary_fwnode(phys_dev, fwnode);
+
+ ret = devm_add_action_or_reset(parent, gpio_secondary_unset, get_device(phys_dev));
if (ret)
return ret;
}
diff --git a/drivers/ras/amd/atl/core.c b/drivers/ras/amd/atl/core.c
index 0f7cd6dab0b0..d77dacdd4f56 100644
--- a/drivers/ras/amd/atl/core.c
+++ b/drivers/ras/amd/atl/core.c
@@ -190,7 +190,6 @@ static const struct x86_cpu_id amd_atl_cpuids[] = {
X86_MATCH_FEATURE(X86_FEATURE_ZEN, NULL),
{ }
};
-MODULE_DEVICE_TABLE(x86cpu, amd_atl_cpuids);
static int __init amd_atl_init(void)
{
diff --git a/drivers/regulator/db8500-prcmu.c b/drivers/regulator/db8500-prcmu.c
index 1ec2e1348891..751fe36580fa 100644
--- a/drivers/regulator/db8500-prcmu.c
+++ b/drivers/regulator/db8500-prcmu.c
@@ -13,7 +13,7 @@
#include <linux/err.h>
#include <linux/spinlock.h>
#include <linux/platform_device.h>
-#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/mfd/db8500-prcmu.h>
#include <linux/regulator/driver.h>
#include <linux/regulator/machine.h>
#include <linux/regulator/db8500-prcmu.h>
@@ -93,13 +93,13 @@ static int enable_epod(u16 epod_id, bool ramret)
if (ramret) {
if (!epod_on[epod_id]) {
- ret = prcmu_set_epod(epod_id, EPOD_STATE_RAMRET);
+ ret = db8500_prcmu_set_epod(epod_id, EPOD_STATE_RAMRET);
if (ret < 0)
return ret;
}
epod_ramret[epod_id] = true;
} else {
- ret = prcmu_set_epod(epod_id, EPOD_STATE_ON);
+ ret = db8500_prcmu_set_epod(epod_id, EPOD_STATE_ON);
if (ret < 0)
return ret;
epod_on[epod_id] = true;
@@ -114,18 +114,18 @@ static int disable_epod(u16 epod_id, bool ramret)
if (ramret) {
if (!epod_on[epod_id]) {
- ret = prcmu_set_epod(epod_id, EPOD_STATE_OFF);
+ ret = db8500_prcmu_set_epod(epod_id, EPOD_STATE_OFF);
if (ret < 0)
return ret;
}
epod_ramret[epod_id] = false;
} else {
if (epod_ramret[epod_id]) {
- ret = prcmu_set_epod(epod_id, EPOD_STATE_RAMRET);
+ ret = db8500_prcmu_set_epod(epod_id, EPOD_STATE_RAMRET);
if (ret < 0)
return ret;
} else {
- ret = prcmu_set_epod(epod_id, EPOD_STATE_OFF);
+ ret = db8500_prcmu_set_epod(epod_id, EPOD_STATE_OFF);
if (ret < 0)
return ret;
}
diff --git a/drivers/regulator/pf1550-regulator.c b/drivers/regulator/pf1550-regulator.c
index 610eac9bb9cb..ceee553a84b2 100644
--- a/drivers/regulator/pf1550-regulator.c
+++ b/drivers/regulator/pf1550-regulator.c
@@ -283,63 +283,61 @@ static struct pf1550_desc pf1550_regulators[] = {
PF_LDO1(PF1550, "ldo3", LDO3, 0x1f, pf1550_ldo13_volts),
};
+/*
+ * The _LS interrupts indicate an over-current event. The _HS
+ * interrupts, which are more accurate and can detect catastrophic
+ * faults, issue an error event. The current limit FAULT interrupt is
+ * similar to the _HS.
+ */
+static const struct pf1550_regulator_irq {
+ unsigned int event;
+ u8 id;
+} pf1550_regulator_irqs[] = {
+ [PF1550_PMIC_IRQ_SW1_LS] = { REGULATOR_EVENT_OVER_CURRENT_WARN, PF1550_SW1 },
+ [PF1550_PMIC_IRQ_SW2_LS] = { REGULATOR_EVENT_OVER_CURRENT_WARN, PF1550_SW2 },
+ [PF1550_PMIC_IRQ_SW3_LS] = { REGULATOR_EVENT_OVER_CURRENT_WARN, PF1550_SW3 },
+ [PF1550_PMIC_IRQ_SW1_HS] = { REGULATOR_EVENT_OVER_CURRENT, PF1550_SW1 },
+ [PF1550_PMIC_IRQ_SW2_HS] = { REGULATOR_EVENT_OVER_CURRENT, PF1550_SW2 },
+ [PF1550_PMIC_IRQ_SW3_HS] = { REGULATOR_EVENT_OVER_CURRENT, PF1550_SW3 },
+ [PF1550_PMIC_IRQ_LDO1_FAULT] = { REGULATOR_EVENT_OVER_CURRENT, PF1550_LDO1 },
+ [PF1550_PMIC_IRQ_LDO2_FAULT] = { REGULATOR_EVENT_OVER_CURRENT, PF1550_LDO2 },
+ [PF1550_PMIC_IRQ_LDO3_FAULT] = { REGULATOR_EVENT_OVER_CURRENT, PF1550_LDO3 },
+};
+
static irqreturn_t pf1550_regulator_irq_handler(int irq, void *data)
{
+ const struct pf1550_regulator_irq *map;
struct pf1550_regulator_info *info = data;
struct device *dev = info->dev;
struct platform_device *pdev = to_platform_device(dev);
int i, irq_type = -1;
- unsigned int event;
for (i = 0; i < PF1550_REGULATOR_IRQ_NR; i++)
if (irq == platform_get_irq(pdev, i))
irq_type = i;
- switch (irq_type) {
- /* The _LS interrupts indicate over-current event. The _HS interrupts
- * which are more accurate and can detect catastrophic faults, issue
- * an error event. The current limit FAULT interrupt is similar to the
- * _HS'
- */
- case PF1550_PMIC_IRQ_SW1_LS:
- case PF1550_PMIC_IRQ_SW2_LS:
- case PF1550_PMIC_IRQ_SW3_LS:
- event = REGULATOR_EVENT_OVER_CURRENT_WARN;
- for (i = 0; i < PF1550_MAX_REGULATOR; i++)
- if (!strcmp(rdev_get_name(info->rdevs[i]), "SW3"))
- regulator_notifier_call_chain(info->rdevs[i],
- event, NULL);
- break;
- case PF1550_PMIC_IRQ_SW1_HS:
- case PF1550_PMIC_IRQ_SW2_HS:
- case PF1550_PMIC_IRQ_SW3_HS:
- event = REGULATOR_EVENT_OVER_CURRENT;
- for (i = 0; i < PF1550_MAX_REGULATOR; i++)
- if (!strcmp(rdev_get_name(info->rdevs[i]), "SW3"))
- regulator_notifier_call_chain(info->rdevs[i],
- event, NULL);
- break;
- case PF1550_PMIC_IRQ_LDO1_FAULT:
- case PF1550_PMIC_IRQ_LDO2_FAULT:
- case PF1550_PMIC_IRQ_LDO3_FAULT:
- event = REGULATOR_EVENT_OVER_CURRENT;
- for (i = 0; i < PF1550_MAX_REGULATOR; i++)
- if (!strcmp(rdev_get_name(info->rdevs[i]), "LDO3"))
- regulator_notifier_call_chain(info->rdevs[i],
- event, NULL);
- break;
- case PF1550_PMIC_IRQ_TEMP_110:
- case PF1550_PMIC_IRQ_TEMP_125:
- event = REGULATOR_EVENT_OVER_TEMP;
+ /* The die temperature concerns every rail. */
+ if (irq_type == PF1550_PMIC_IRQ_TEMP_110 ||
+ irq_type == PF1550_PMIC_IRQ_TEMP_125) {
for (i = 0; i < PF1550_MAX_REGULATOR; i++)
regulator_notifier_call_chain(info->rdevs[i],
- event, NULL);
- break;
- default:
- dev_err(dev, "regulator interrupt: irq %d occurred\n",
- irq_type);
+ REGULATOR_EVENT_OVER_TEMP,
+ NULL);
+ return IRQ_HANDLED;
+ }
+
+ if (irq_type < 0 || irq_type >= (int)ARRAY_SIZE(pf1550_regulator_irqs)) {
+ dev_err(dev, "regulator interrupt: irq %d occurred\n", irq_type);
+ return IRQ_HANDLED;
}
+ map = &pf1550_regulator_irqs[irq_type];
+
+ for (i = 0; i < PF1550_MAX_REGULATOR; i++)
+ if (rdev_get_id(info->rdevs[i]) == map->id)
+ regulator_notifier_call_chain(info->rdevs[i],
+ map->event, NULL);
+
return IRQ_HANDLED;
}
diff --git a/drivers/s390/net/ism_drv.c b/drivers/s390/net/ism_drv.c
index 242da20f27e0..035b233abb4e 100644
--- a/drivers/s390/net/ism_drv.c
+++ b/drivers/s390/net/ism_drv.c
@@ -231,6 +231,7 @@ static void ism_free_dmb(struct ism_dev *ism, struct dibs_dmb *dmb)
dma_unmap_page(&ism->pdev->dev, dmb->dma_addr, dmb->dmb_len,
DMA_FROM_DEVICE);
folio_put(virt_to_folio(dmb->cpu_addr));
+ dmb->cpu_addr = NULL;
}
static int ism_alloc_dmb(struct ism_dev *ism, struct dibs_dmb *dmb)
@@ -274,7 +275,8 @@ static int ism_alloc_dmb(struct ism_dev *ism, struct dibs_dmb *dmb)
return 0;
out_free:
- kfree(dmb->cpu_addr);
+ folio_put(folio);
+ dmb->cpu_addr = NULL;
out_bit:
clear_bit(dmb->idx, ism->sba_bitmap);
return rc;
diff --git a/drivers/s390/net/qeth_l2.h b/drivers/s390/net/qeth_l2.h
index 7c646e2fed7e..f94975e970ca 100644
--- a/drivers/s390/net/qeth_l2.h
+++ b/drivers/s390/net/qeth_l2.h
@@ -13,7 +13,8 @@ extern const struct attribute_group *qeth_l2_attr_groups[];
int qeth_bridgeport_query_ports(struct qeth_card *card,
enum qeth_sbp_roles *role,
- enum qeth_sbp_states *state);
+ enum qeth_sbp_states *state,
+ bool *os_mismatch);
int qeth_bridgeport_setrole(struct qeth_card *card, enum qeth_sbp_roles role);
int qeth_bridgeport_an_set(struct qeth_card *card, int enable);
diff --git a/drivers/s390/net/qeth_l2_main.c b/drivers/s390/net/qeth_l2_main.c
index a9e7d1d637a2..2935c2ecc314 100644
--- a/drivers/s390/net/qeth_l2_main.c
+++ b/drivers/s390/net/qeth_l2_main.c
@@ -1158,7 +1158,7 @@ static void qeth_l2_setup_bridgeport_attrs(struct qeth_card *card)
qeth_bridgeport_setrole(card, card->options.sbp.role);
/* Let the callback function refresh the stored role value. */
qeth_bridgeport_query_ports(card, &card->options.sbp.role,
- NULL);
+ NULL, NULL);
}
if (card->options.sbp.hostnotification) {
if (qeth_bridgeport_an_set(card, 1))
@@ -1545,6 +1545,7 @@ struct _qeth_sbp_cbctl {
struct {
enum qeth_sbp_roles *role;
enum qeth_sbp_states *state;
+ bool *os_mismatch;
} qports;
} data;
};
@@ -1721,10 +1722,19 @@ static int qeth_bridgeport_query_ports_cb(struct qeth_card *card,
struct qeth_ipa_cmd *cmd = (struct qeth_ipa_cmd *) data;
struct _qeth_sbp_cbctl *cbctl = (struct _qeth_sbp_cbctl *)reply->param;
struct qeth_sbp_port_data *qports;
+ u16 sbp_rc;
int rc;
QETH_CARD_TEXT(card, 2, "brqprtcb");
- rc = qeth_bridgeport_makerc(card, cmd);
+ sbp_rc = cmd->data.sbp.hdr.return_code;
+
+ /* on OS family mismatch, query still returns valid port data;
+ * treat as success
+ */
+ if (sbp_rc == IPA_RC_SBP_IQD_OS_MISMATCH && !cmd->hdr.return_code)
+ rc = 0;
+ else
+ rc = qeth_bridgeport_makerc(card, cmd);
if (rc)
return rc;
@@ -1740,6 +1750,9 @@ static int qeth_bridgeport_query_ports_cb(struct qeth_card *card,
if (cbctl->data.qports.state)
*cbctl->data.qports.state = qports->entry[0].state;
}
+ if (cbctl->data.qports.os_mismatch)
+ *cbctl->data.qports.os_mismatch =
+ (sbp_rc == IPA_RC_SBP_IQD_OS_MISMATCH);
return 0;
}
@@ -1748,13 +1761,17 @@ static int qeth_bridgeport_query_ports_cb(struct qeth_card *card,
* @card: qeth_card structure pointer.
* @role: Role of the port: 0-none, 1-primary, 2-secondary.
* @state: State of the port: 0-inactive, 1-standby, 2-active.
+ * @os_mismatch: if non-NULL, set to true when firmware reports
+ * OS family mismatch.
*
* Returns negative errno-compatible error indication or 0 on success.
*
- * 'role' and 'state' are not updated in case of hardware operation failure.
+ * 'role', 'state' and 'os_mismatch' are not updated in case of
+ * hardware operation failure.
*/
int qeth_bridgeport_query_ports(struct qeth_card *card,
- enum qeth_sbp_roles *role, enum qeth_sbp_states *state)
+ enum qeth_sbp_roles *role, enum qeth_sbp_states *state,
+ bool *os_mismatch)
{
struct qeth_cmd_buffer *iob;
struct _qeth_sbp_cbctl cbctl = {
@@ -1762,6 +1779,7 @@ int qeth_bridgeport_query_ports(struct qeth_card *card,
.qports = {
.role = role,
.state = state,
+ .os_mismatch = os_mismatch,
},
},
};
diff --git a/drivers/s390/net/qeth_l2_sys.c b/drivers/s390/net/qeth_l2_sys.c
index 7f592f912517..7101be62eb1d 100644
--- a/drivers/s390/net/qeth_l2_sys.c
+++ b/drivers/s390/net/qeth_l2_sys.c
@@ -15,6 +15,7 @@ static ssize_t qeth_bridge_port_role_state_show(struct device *dev,
{
struct qeth_card *card = dev_get_drvdata(dev);
enum qeth_sbp_states state = QETH_SBP_STATE_INACTIVE;
+ bool os_mismatch = false;
int rc = 0;
char *word;
@@ -25,7 +26,7 @@ static ssize_t qeth_bridge_port_role_state_show(struct device *dev,
if (qeth_card_hw_is_reachable(card) &&
card->options.sbp.supported_funcs)
rc = qeth_bridgeport_query_ports(card,
- &card->options.sbp.role, &state);
+ &card->options.sbp.role, &state, &os_mismatch);
if (!rc) {
if (show_state)
switch (state) {
@@ -52,6 +53,10 @@ static ssize_t qeth_bridge_port_role_state_show(struct device *dev,
if (rc)
QETH_CARD_TEXT_(card, 2, "SBP%02x:%02x",
card->options.sbp.role, state);
+ else if (!show_state &&
+ card->options.sbp.role == QETH_SBP_ROLE_NONE &&
+ os_mismatch)
+ rc = sysfs_emit(buf, "%s (OS family mismatch)\n", word);
else
rc = sysfs_emit(buf, "%s\n", word);
}
diff --git a/drivers/scsi/mpi3mr/mpi3mr_transport.c b/drivers/scsi/mpi3mr/mpi3mr_transport.c
index 240f67a8e2e3..232af978d737 100644
--- a/drivers/scsi/mpi3mr/mpi3mr_transport.c
+++ b/drivers/scsi/mpi3mr/mpi3mr_transport.c
@@ -1428,9 +1428,15 @@ static struct mpi3mr_sas_port *mpi3mr_sas_port_add(struct mpi3mr_ioc *mrioc,
}
port = sas_port_alloc_num(mr_sas_node->parent_dev);
+ if (!port) {
+ ioc_err(mrioc, "failure at %s:%d/%s()!\n",
+ __FILE__, __LINE__, __func__);
+ goto out_fail;
+ }
if ((sas_port_add(port))) {
ioc_err(mrioc, "failure at %s:%d/%s()!\n",
__FILE__, __LINE__, __func__);
+ sas_port_free(port);
goto out_fail;
}
@@ -1501,6 +1507,8 @@ static struct mpi3mr_sas_port *mpi3mr_sas_port_add(struct mpi3mr_ioc *mrioc,
list_for_each_entry_safe(mr_sas_phy, next, &mr_sas_port->phy_list,
port_siblings)
list_del(&mr_sas_phy->port_siblings);
+ if (tgtdev)
+ mpi3mr_tgtdev_put(tgtdev);
kfree(mr_sas_port);
return NULL;
}
diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c
index 11bcb8440e1c..ae2f7d9c281f 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_base.c
+++ b/drivers/scsi/mpt3sas/mpt3sas_base.c
@@ -3238,7 +3238,10 @@ _base_assign_reply_queues(struct MPT3SAS_ADAPTER *ioc)
* corresponding to high iops queues.
*/
if (ioc->high_iops_queues) {
- mask = cpumask_of_node(dev_to_node(&ioc->pdev->dev));
+ int node = dev_to_node(&ioc->pdev->dev);
+
+ mask = (node == NUMA_NO_NODE) ?
+ cpu_online_mask : cpumask_of_node(node);
for (index = 0; index < ioc->high_iops_queues;
index++) {
irq = pci_irq_vector(ioc->pdev, index);
diff --git a/drivers/scsi/qla2xxx/qla_bsg.c b/drivers/scsi/qla2xxx/qla_bsg.c
index 5b15fc28ddd7..79144530fa24 100644
--- a/drivers/scsi/qla2xxx/qla_bsg.c
+++ b/drivers/scsi/qla2xxx/qla_bsg.c
@@ -160,6 +160,12 @@ qla24xx_proc_fcp_prio_cfg_cmd(struct bsg_job *bsg_job)
goto exit_fcp_prio_cfg;
}
+ if (bsg_job->request_len <
+ sizeof(struct fc_bsg_request) + 2 * sizeof(uint32_t)) {
+ ret = -EINVAL;
+ goto exit_fcp_prio_cfg;
+ }
+
/* Get the sub command */
oper = bsg_request->rqst_data.h_vendor.vendor_cmd[1];
@@ -758,6 +764,10 @@ qla2x00_process_loopback(struct bsg_job *bsg_job)
return -EIO;
}
+ if (bsg_job->request_len <
+ sizeof(struct fc_bsg_request) + 3 * sizeof(uint32_t))
+ return -EINVAL;
+
memset(&elreq, 0, sizeof(elreq));
elreq.req_sg_cnt = dma_map_sg(&ha->pdev->dev,
@@ -990,6 +1000,10 @@ qla84xx_reset(struct bsg_job *bsg_job)
return -EINVAL;
}
+ if (bsg_job->request_len <
+ sizeof(struct fc_bsg_request) + 2 * sizeof(uint32_t))
+ return -EINVAL;
+
flag = bsg_request->rqst_data.h_vendor.vendor_cmd[1];
rval = qla84xx_reset_chip(vha, flag == A84_ISSUE_RESET_DIAG_FW);
@@ -1034,6 +1048,10 @@ qla84xx_updatefw(struct bsg_job *bsg_job)
return -EINVAL;
}
+ if (bsg_job->request_len <
+ sizeof(struct fc_bsg_request) + 2 * sizeof(uint32_t))
+ return -EINVAL;
+
sg_cnt = dma_map_sg(&ha->pdev->dev, bsg_job->request_payload.sg_list,
bsg_job->request_payload.sg_cnt, DMA_TO_DEVICE);
if (!sg_cnt) {
@@ -1484,6 +1502,10 @@ qla2x00_read_optrom(struct bsg_job *bsg_job)
struct qla_hw_data *ha = vha->hw;
int rval = 0;
+ if (bsg_job->request_len <
+ sizeof(struct fc_bsg_request) + 2 * sizeof(uint32_t))
+ return -EINVAL;
+
if (ha->flags.nic_core_reset_hdlr_active)
return -EBUSY;
@@ -1521,6 +1543,10 @@ qla2x00_update_optrom(struct bsg_job *bsg_job)
struct qla_hw_data *ha = vha->hw;
int rval = 0;
+ if (bsg_job->request_len <
+ sizeof(struct fc_bsg_request) + 2 * sizeof(uint32_t))
+ return -EINVAL;
+
mutex_lock(&ha->optrom_mutex);
rval = qla2x00_optrom_setup(bsg_job, vha, 1);
if (rval) {
@@ -2012,6 +2038,11 @@ qlafx00_mgmt_cmd(struct bsg_job *bsg_job)
struct fc_port *fcport;
char *type = "FC_BSG_HST_FX_MGMT";
+ if (bsg_job->request_len <
+ sizeof(struct fc_bsg_request) + sizeof(uint32_t) +
+ sizeof(struct qla_mt_iocb_rqst_fx00))
+ return -EINVAL;
+
/* Copy the IOCB specific information */
piocb_rqst = (struct qla_mt_iocb_rqst_fx00 *)
&bsg_request->rqst_data.h_vendor.vendor_cmd[1];
@@ -2925,6 +2956,13 @@ qla2x00_process_vendor_specific(struct scsi_qla_host *vha, struct bsg_job *bsg_j
{
struct fc_bsg_request *bsg_request = bsg_job->request;
+ if (bsg_job->request_len <
+ sizeof(struct fc_bsg_request) + sizeof(uint32_t)) {
+ ql_log(ql_log_warn, vha, 0x7000,
+ "BSG request too small for vendor cmd.\n");
+ return -EINVAL;
+ }
+
ql_dbg(ql_dbg_edif, vha, 0x911b, "%s FC_BSG_HST_VENDOR cmd[0]=0x%x\n",
__func__, bsg_request->rqst_data.h_vendor.vendor_cmd[0]);
@@ -3056,8 +3094,11 @@ qla24xx_bsg_request(struct bsg_job *bsg_job)
}
/* Disable port will bring down the chip, allow enable command */
- if (bsg_request->rqst_data.h_vendor.vendor_cmd[0] == QL_VND_MANAGE_HOST_PORT ||
- bsg_request->rqst_data.h_vendor.vendor_cmd[0] == QL_VND_GET_HOST_STATS)
+ if (bsg_request->msgcode == FC_BSG_HST_VENDOR &&
+ bsg_job->request_len >=
+ sizeof(struct fc_bsg_request) + sizeof(uint32_t) &&
+ (bsg_request->rqst_data.h_vendor.vendor_cmd[0] == QL_VND_MANAGE_HOST_PORT ||
+ bsg_request->rqst_data.h_vendor.vendor_cmd[0] == QL_VND_GET_HOST_STATS))
goto skip_chip_chk;
if (vha->hw->flags.port_isolated) {
@@ -3366,6 +3407,10 @@ static int qla28xx_validate_flash_image(struct bsg_job *bsg_job)
if (!IS_QLA28XX(ha) || vha->vp_idx != 0)
return -EPERM;
+ if (bsg_job->request_len <
+ sizeof(struct fc_bsg_request) + 2 * sizeof(uint32_t))
+ return -EINVAL;
+
mutex_lock(&ha->optrom_mutex);
rval = qla28xx_do_validate_flash_image(bsg_job, &state);
if (rval)
diff --git a/drivers/scsi/qla2xxx/qla_def.h b/drivers/scsi/qla2xxx/qla_def.h
index 5593ad7fad27..8be3dd40f6d7 100644
--- a/drivers/scsi/qla2xxx/qla_def.h
+++ b/drivers/scsi/qla2xxx/qla_def.h
@@ -2591,6 +2591,8 @@ typedef struct fc_port {
struct list_head list;
struct scsi_qla_host *vha;
struct list_head unsol_ctx_head;
+ /* Serializes unsol_ctx_head against ISR, DPC and NVMe transport. */
+ spinlock_t unsol_ctx_lock;
unsigned int conf_compl_supported:1;
unsigned int deleted:2;
diff --git a/drivers/scsi/qla2xxx/qla_fw.h b/drivers/scsi/qla2xxx/qla_fw.h
index f307beed9d29..e7ded95afb9a 100644
--- a/drivers/scsi/qla2xxx/qla_fw.h
+++ b/drivers/scsi/qla2xxx/qla_fw.h
@@ -1442,6 +1442,10 @@ struct vp_ctrl_entry_24xx {
uint8_t reserved_5[24];
};
+/* vp_idx_map is a 128-bit (16-byte) bitmap selecting target VPs. */
+#define VP_CTRL_IDX_MAP_BITS \
+ (sizeof_field(struct vp_ctrl_entry_24xx, vp_idx_map) * 8)
+
/*
* Modify Virtual Port Configuration IOCB
*/
diff --git a/drivers/scsi/qla2xxx/qla_init.c b/drivers/scsi/qla2xxx/qla_init.c
index d38bf74ea7a5..22374f844e08 100644
--- a/drivers/scsi/qla2xxx/qla_init.c
+++ b/drivers/scsi/qla2xxx/qla_init.c
@@ -4421,6 +4421,19 @@ qla2x00_setup_chip(scsi_qla_host_t *vha)
MIN_MULTI_ID_FABRIC))
ha->max_npiv_vports =
MIN_MULTI_ID_FABRIC - 1;
+
+ /*
+ * The VP_CTRL IOCB selects target VPs
+ * through the fixed vp_idx_map bitmap,
+ * so a vp_index beyond it can be enabled
+ * via VP_CONFIG but never disabled via
+ * VP_CTRL, leaking the VP. Cap the count
+ * to the bitmap capacity.
+ */
+ if (ha->max_npiv_vports >=
+ VP_CTRL_IDX_MAP_BITS)
+ ha->max_npiv_vports =
+ VP_CTRL_IDX_MAP_BITS - 1;
}
qlt_config_nvram_with_fw_version(vha);
qla2x00_get_resource_cnts(vha);
@@ -5650,6 +5663,7 @@ qla2x00_alloc_fcport(scsi_qla_host_t *vha, gfp_t flags)
INIT_LIST_HEAD(&fcport->gnl_entry);
INIT_LIST_HEAD(&fcport->list);
INIT_LIST_HEAD(&fcport->unsol_ctx_head);
+ spin_lock_init(&fcport->unsol_ctx_lock);
INIT_LIST_HEAD(&fcport->sess_cmd_list);
spin_lock_init(&fcport->sess_cmd_lock);
diff --git a/drivers/scsi/qla2xxx/qla_iocb.c b/drivers/scsi/qla2xxx/qla_iocb.c
index dbe3cd4e274c..1a58dd4e8ec2 100644
--- a/drivers/scsi/qla2xxx/qla_iocb.c
+++ b/drivers/scsi/qla2xxx/qla_iocb.c
@@ -11,6 +11,10 @@
#include <scsi/scsi_tcq.h>
+/* All adapters supported by this tree use the 64-byte request ring. */
+#define qla_req_ring_slot(ha, req) ((req)->ring_ptr)
+#define qla_req_entry_size(ha) REQUEST_ENTRY_SIZE
+
static int qla_start_scsi_type6(srb_t *sp);
/**
* qla2x00_get_cmd_direction() - Determine control_flag data direction.
@@ -2286,10 +2290,9 @@ __qla2x00_alloc_iocbs(struct qla_qpair *qpair, srb_t *sp)
struct req_que *req = qpair->req;
device_reg_t *reg = ISP_QUE_REG(ha, req->id);
uint32_t handle;
- request_t *pkt;
uint16_t cnt, req_cnt;
+ request_t *pkt = NULL;
- pkt = NULL;
req_cnt = 1;
handle = 0;
@@ -2343,14 +2346,18 @@ __qla2x00_alloc_iocbs(struct qla_qpair *qpair, srb_t *sp)
sp->handle = handle;
}
- /* Prep packet */
+ /*
+ * Prep the current request-ring slot.
+ */
req->cnt -= req_cnt;
- pkt = req->ring_ptr;
- memset(pkt, 0, REQUEST_ENTRY_SIZE);
+ pkt = qla_req_ring_slot(ha, req);
if (IS_QLAFX00(ha)) {
+ memset_io((void __iomem __force *)pkt, 0,
+ qla_req_entry_size(ha));
wrt_reg_byte((u8 __force __iomem *)&pkt->entry_count, req_cnt);
wrt_reg_dword((__le32 __force __iomem *)&pkt->handle, handle);
} else {
+ memset(pkt, 0, qla_req_entry_size(ha));
pkt->entry_count = req_cnt;
pkt->handle = handle;
}
@@ -3826,6 +3833,12 @@ qla25xx_ctrlvp_iocb(srb_t *sp, struct vp_ctrl_entry_24xx *vce)
*/
map = (sp->u.iocb_cmd.u.ctrlvp.vp_index - 1) / 8;
pos = (sp->u.iocb_cmd.u.ctrlvp.vp_index - 1) & 7;
+ if (map >= ARRAY_SIZE(vce->vp_idx_map)) {
+ ql_log(ql_log_warn, sp->vha, 0x307c,
+ "ctrlvp: vp_index %u exceeds vp_idx_map capacity\n",
+ sp->u.iocb_cmd.u.ctrlvp.vp_index);
+ return;
+ }
vce->vp_idx_map[map] |= 1 << pos;
}
diff --git a/drivers/scsi/qla2xxx/qla_isr.c b/drivers/scsi/qla2xxx/qla_isr.c
index 612e00ca1ada..81997119fd07 100644
--- a/drivers/scsi/qla2xxx/qla_isr.c
+++ b/drivers/scsi/qla2xxx/qla_isr.c
@@ -921,14 +921,6 @@ qla27xx_copy_multiple_pkt(struct scsi_qla_host *vha, void **pkt,
do {
while ((total_bytes > 0) && (entry_count_remaining > 0)) {
- if (rsp_q->ring_ptr->signature == RESPONSE_PROCESSED) {
- ql_dbg(ql_dbg_async, vha, 0x5084,
- "Ran out of IOCBs, partial data 0x%x\n",
- buffer_copy_offset);
- cpu_relax();
- continue;
- }
-
new_pkt = (sts_cont_entry_t *)rsp_q->ring_ptr;
*pkt = new_pkt;
@@ -1205,14 +1197,6 @@ qla27xx_copy_fpin_pkt(struct scsi_qla_host *vha, void **pkt,
do {
while ((total_bytes > 0) && (entry_count_remaining > 0)) {
- if (rsp_q->ring_ptr->signature == RESPONSE_PROCESSED) {
- ql_dbg(ql_dbg_async, vha, 0x5084,
- "Ran out of IOCBs, partial data 0x%x\n",
- buffer_copy_offset);
- cpu_relax();
- continue;
- }
-
new_pkt = (sts_cont_entry_t *)rsp_q->ring_ptr;
*pkt = new_pkt;
@@ -4142,9 +4126,24 @@ void qla24xx_process_response_queue(struct scsi_qla_host *vha,
"SCM not active for this port\n");
break;
}
+ if (qla_chk_cont_iocb_avail(vha, rsp,
+ (response_t *)pkt, rsp_in)) {
+ /*
+ * ring_ptr and ring_index were
+ * pre-incremented above. Reset them
+ * back to current. Wait for next
+ * interrupt with all IOCBs to arrive
+ * and re-process.
+ */
+ rsp->ring_ptr = (response_t *)pkt;
+ rsp->ring_index = cur_ring_index;
+
+ ql_dbg(ql_dbg_init, vha, 0x5095,
+ "Defer processing FPIN...\n");
+ return;
+ }
pure_item = qla27xx_copy_fpin_pkt(vha,
(void **)&pkt, &rsp);
- __update_rsp_in(is_shadow_hba, rsp, rsp_in);
if (!pure_item)
break;
qla24xx_queue_purex_item(vha, pure_item,
diff --git a/drivers/scsi/qla2xxx/qla_mbx.c b/drivers/scsi/qla2xxx/qla_mbx.c
index f604aab8b161..aec8e9ff3f3e 100644
--- a/drivers/scsi/qla2xxx/qla_mbx.c
+++ b/drivers/scsi/qla2xxx/qla_mbx.c
@@ -4204,6 +4204,8 @@ qla24xx_report_id_acquisition(scsi_qla_host_t *vha,
spin_lock_irqsave(&ha->vport_slock, flags);
list_for_each_entry(vp, &ha->vp_list, list) {
if (rptid_entry->vp_idx == vp->vp_idx) {
+ if (test_bit(VPORT_DELETE, &vp->dpc_flags))
+ break;
found = 1;
atomic_inc(&vp->vref_count);
break;
@@ -5715,7 +5717,7 @@ qla2x00_get_data_rate(scsi_qla_host_t *vha)
ql_dbg(ql_dbg_mbx, vha, 0x1107,
"Failed=%x mb[0]=%x.\n", rval, mcp->mb[0]);
} else {
- if (mcp->mb[1] != 0x7)
+ if (mcp->mb[1] != 0x7 || IS_QLA28XX(ha))
ha->link_data_rate = mcp->mb[1];
if (IS_QLA83XX(ha) || IS_QLA27XX(ha) || IS_QLA28XX(ha)) {
@@ -5726,8 +5728,6 @@ qla2x00_get_data_rate(scsi_qla_host_t *vha)
ql_dbg(ql_dbg_mbx + ql_dbg_verbose, vha, 0x1108,
"Done %s.\n", __func__);
- if (mcp->mb[1] != 0x7)
- ha->link_data_rate = mcp->mb[1];
}
return rval;
diff --git a/drivers/scsi/qla2xxx/qla_mid.c b/drivers/scsi/qla2xxx/qla_mid.c
index c563133f751e..21e2f9dbe0a9 100644
--- a/drivers/scsi/qla2xxx/qla_mid.c
+++ b/drivers/scsi/qla2xxx/qla_mid.c
@@ -574,16 +574,19 @@ qla25xx_free_req_que(struct scsi_qla_host *vha, struct req_que *req)
{
struct qla_hw_data *ha = vha->hw;
uint16_t que_id = req->id;
+ size_t req_entry_size = sizeof(request_t);
- dma_free_coherent(&ha->pdev->dev, (req->length + 1) *
- sizeof(request_t), req->ring, req->dma);
+ if (req->ring)
+ dma_free_coherent(&ha->pdev->dev,
+ (req->length + 1) * req_entry_size,
+ req->ring, req->dma);
req->ring = NULL;
req->dma = 0;
if (que_id) {
+ mutex_lock(&ha->mq_lock);
ha->req_q_map[que_id] = NULL;
- mutex_lock(&ha->vport_lock);
clear_bit(que_id, ha->req_qid_map);
- mutex_unlock(&ha->vport_lock);
+ mutex_unlock(&ha->mq_lock);
}
kfree(req->outstanding_cmds);
kfree(req);
@@ -594,6 +597,7 @@ qla25xx_free_rsp_que(struct scsi_qla_host *vha, struct rsp_que *rsp)
{
struct qla_hw_data *ha = vha->hw;
uint16_t que_id = rsp->id;
+ size_t rsp_entry_size = sizeof(response_t);
if (rsp->msix && rsp->msix->have_irq) {
free_irq(rsp->msix->vector, rsp->msix->handle);
@@ -601,15 +605,22 @@ qla25xx_free_rsp_que(struct scsi_qla_host *vha, struct rsp_que *rsp)
rsp->msix->in_use = 0;
rsp->msix->handle = NULL;
}
- dma_free_coherent(&ha->pdev->dev, (rsp->length + 1) *
- sizeof(response_t), rsp->ring, rsp->dma);
+
+ /* Flush any queued response work before freeing the queue/qpair. */
+ if (rsp->qpair && ha->wq)
+ cancel_work_sync(&rsp->qpair->q_work);
+
+ if (rsp->ring)
+ dma_free_coherent(&ha->pdev->dev,
+ (rsp->length + 1) * rsp_entry_size,
+ rsp->ring, rsp->dma);
rsp->ring = NULL;
rsp->dma = 0;
if (que_id) {
+ mutex_lock(&ha->mq_lock);
ha->rsp_q_map[que_id] = NULL;
- mutex_lock(&ha->vport_lock);
clear_bit(que_id, ha->rsp_qid_map);
- mutex_unlock(&ha->vport_lock);
+ mutex_unlock(&ha->mq_lock);
}
kfree(rsp);
}
@@ -794,9 +805,6 @@ qla25xx_create_req_que(struct qla_hw_data *ha, uint16_t options,
if (ret != QLA_SUCCESS) {
ql_log(ql_log_fatal, base_vha, 0x00df,
"%s failed.\n", __func__);
- mutex_lock(&ha->mq_lock);
- clear_bit(que_id, ha->req_qid_map);
- mutex_unlock(&ha->mq_lock);
goto que_failed;
}
vha->flags.qpairs_req_created = 1;
@@ -908,9 +916,6 @@ qla25xx_create_rsp_que(struct qla_hw_data *ha, uint16_t options,
if (ret != QLA_SUCCESS) {
ql_log(ql_log_fatal, base_vha, 0x00e7,
"%s failed.\n", __func__);
- mutex_lock(&ha->mq_lock);
- clear_bit(que_id, ha->rsp_qid_map);
- mutex_unlock(&ha->mq_lock);
goto que_failed;
}
vha->flags.qpairs_rsp_created = 1;
@@ -957,6 +962,22 @@ int qla24xx_control_vp(scsi_qla_host_t *vha, int cmd)
if (vp_index == 0 || vp_index >= ha->max_npiv_vports)
return QLA_PARAMETER_ERROR;
+ /*
+ * The VP_CTRL IOCB selects the target VP through a fixed 128-bit
+ * (16-byte) vp_idx_map bitmap, so vp_index must fit within it even
+ * if firmware advertises more NPIV vports.
+ */
+ if (vp_index > sizeof_field(struct vp_ctrl_entry_24xx, vp_idx_map) * 8)
+ return QLA_PARAMETER_ERROR;
+
+ /*
+ * The VP_CTRL IOCB selects the target VP through a fixed 128-bit
+ * (16-byte) vp_idx_map bitmap, so vp_index must fit within it even
+ * if firmware advertises more NPIV vports.
+ */
+ if (vp_index > VP_CTRL_IDX_MAP_BITS)
+ return QLA_PARAMETER_ERROR;
+
/* ref: INIT */
sp = qla2x00_get_sp(base_vha, NULL, GFP_KERNEL);
if (!sp)
diff --git a/drivers/scsi/qla2xxx/qla_nvme.c b/drivers/scsi/qla2xxx/qla_nvme.c
index b84fe71ea340..717722268196 100644
--- a/drivers/scsi/qla2xxx/qla_nvme.c
+++ b/drivers/scsi/qla2xxx/qla_nvme.c
@@ -257,7 +257,9 @@ static void qla_nvme_release_lsrsp_cmd_kref(struct kref *kref)
fd_rsp = uctx->fd_rsp;
+ spin_lock_irqsave(&uctx->fcport->unsol_ctx_lock, flags);
list_del(&uctx->elem);
+ spin_unlock_irqrestore(&uctx->fcport->unsol_ctx_lock, flags);
fd_rsp->done(fd_rsp);
kfree(uctx);
@@ -446,6 +448,9 @@ static int qla_nvme_xmt_ls_rsp(struct nvme_fc_local_port *lport,
qla_nvme_ls_reject_iocb(vha, ha->base_qpair, &a, true);
spin_unlock_irqrestore(ha->base_qpair->qp_lock_ptr, flags);
}
+ spin_lock_irqsave(&uctx->fcport->unsol_ctx_lock, flags);
+ list_del(&uctx->elem);
+ spin_unlock_irqrestore(&uctx->fcport->unsol_ctx_lock, flags);
kfree(uctx);
return rval;
}
@@ -1215,7 +1220,9 @@ qla2xxx_process_purls_pkt(struct scsi_qla_host *vha, struct purex_item *item)
spin_unlock_irqrestore(vha->hw->base_qpair->qp_lock_ptr,
flags);
}
+ spin_lock_irqsave(&uctx->fcport->unsol_ctx_lock, flags);
list_del(&uctx->elem);
+ spin_unlock_irqrestore(&uctx->fcport->unsol_ctx_lock, flags);
kfree(uctx);
}
}
@@ -1257,6 +1264,7 @@ void qla2xxx_process_purls_iocb(void **pkt, struct rsp_que **rsp)
struct purex_item *item;
port_id_t d_id = {0};
port_id_t id = {0};
+ unsigned long flags;
u8 *opcode;
bool xmt_reject = false;
@@ -1322,7 +1330,9 @@ void qla2xxx_process_purls_iocb(void **pkt, struct rsp_que **rsp)
uctx->ox_id = p->ox_id;
qla_rport->uctx = uctx;
INIT_LIST_HEAD(&uctx->elem);
+ spin_lock_irqsave(&fcport->unsol_ctx_lock, flags);
list_add_tail(&uctx->elem, &fcport->unsol_ctx_head);
+ spin_unlock_irqrestore(&fcport->unsol_ctx_lock, flags);
item->purls_context = (void *)uctx;
ql_dbg(ql_dbg_unsol, vha, 0x2121,
diff --git a/drivers/scsi/qla2xxx/qla_os.c b/drivers/scsi/qla2xxx/qla_os.c
index d853b4006f4e..e0c5732037ad 100644
--- a/drivers/scsi/qla2xxx/qla_os.c
+++ b/drivers/scsi/qla2xxx/qla_os.c
@@ -4484,25 +4484,40 @@ qla2x00_mem_alloc(struct qla_hw_data *ha, uint16_t req_len, uint16_t rsp_len,
fail_lsrjt:
dma_free_coherent(&ha->pdev->dev, ha->elsrej.size,
ha->elsrej.c, ha->elsrej.cdma);
+ ha->elsrej.c = NULL;
+ ha->elsrej.cdma = 0;
fail_elsrej:
dma_pool_destroy(ha->purex_dma_pool);
+ ha->purex_dma_pool = NULL;
fail_flt:
dma_free_coherent(&ha->pdev->dev, sizeof(struct qla_flt_header) + FLT_REGIONS_SIZE,
ha->flt, ha->flt_dma);
+ ha->flt = NULL;
+ ha->flt_dma = 0;
fail_flt_buffer:
dma_free_coherent(&ha->pdev->dev, SFP_DEV_SIZE,
ha->sfp_data, ha->sfp_data_dma);
+ ha->sfp_data = NULL;
+ ha->sfp_data_dma = 0;
fail_sfp_data:
kfree(ha->loop_id_map);
+ ha->loop_id_map = NULL;
fail_loop_id_map:
dma_pool_free(ha->s_dma_pool, ha->async_pd, ha->async_pd_dma);
+ ha->async_pd = NULL;
+ ha->async_pd_dma = 0;
fail_async_pd:
dma_pool_free(ha->s_dma_pool, ha->sf_init_cb, ha->sf_init_cb_dma);
+ ha->sf_init_cb = NULL;
+ ha->sf_init_cb_dma = 0;
fail_sf_init_cb:
dma_pool_free(ha->s_dma_pool, ha->ex_init_cb, ha->ex_init_cb_dma);
+ ha->ex_init_cb = NULL;
+ ha->ex_init_cb_dma = 0;
fail_ex_init_cb:
kfree(ha->npiv_info);
+ ha->npiv_info = NULL;
fail_npiv_info:
dma_free_coherent(&ha->pdev->dev, ((*rsp)->length + 1) *
sizeof(response_t), (*rsp)->ring, (*rsp)->dma);
diff --git a/drivers/staging/fbtft/fbtft-core.c b/drivers/staging/fbtft/fbtft-core.c
index ca0c38221c16..7925d974de80 100644
--- a/drivers/staging/fbtft/fbtft-core.c
+++ b/drivers/staging/fbtft/fbtft-core.c
@@ -298,14 +298,15 @@ static void fbtft_mkdirty(struct fb_info *info, int y, int height)
{
struct fbtft_par *par = info->par;
struct fb_deferred_io *fbdefio = info->fbdefio;
+ unsigned long flags;
/* Mark display lines/area as dirty */
- spin_lock(&par->dirty_lock);
+ spin_lock_irqsave(&par->dirty_lock, flags);
if (y < par->dirty_lines_start)
par->dirty_lines_start = y;
if (y + height - 1 > par->dirty_lines_end)
par->dirty_lines_end = y + height - 1;
- spin_unlock(&par->dirty_lock);
+ spin_unlock_irqrestore(&par->dirty_lock, flags);
/* Schedule deferred_io to update display (no-op if already on queue)*/
schedule_delayed_work(&info->deferred_work, fbdefio->delay);
@@ -318,13 +319,13 @@ static void fbtft_deferred_io(struct fb_info *info, struct list_head *pagereflis
struct fb_deferred_io_pageref *pageref;
unsigned int y_low = 0, y_high = 0;
- spin_lock(&par->dirty_lock);
+ spin_lock_irq(&par->dirty_lock);
dirty_lines_start = par->dirty_lines_start;
dirty_lines_end = par->dirty_lines_end;
/* set display line markers as clean */
par->dirty_lines_start = par->info->var.yres - 1;
par->dirty_lines_end = 0;
- spin_unlock(&par->dirty_lock);
+ spin_unlock_irq(&par->dirty_lock);
/* Mark display lines as dirty */
list_for_each_entry(pageref, pagereflist, list) {
diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
index 62ada3a52210..124ff269b8e7 100644
--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -1533,8 +1533,10 @@ __iscsit_check_dataout_hdr(struct iscsit_conn *conn, void *buf,
*/
if (se_cmd->transport_state & CMD_T_ABORTED) {
if (hdr->flags & ISCSI_FLAG_CMD_FINAL &&
- --cmd->outstanding_r2ts < 1)
+ --cmd->outstanding_r2ts < 1) {
iscsit_stop_dataout_timer(cmd);
+ target_complete_cmd(se_cmd, SAM_STAT_TASK_ABORTED);
+ }
return iscsit_dump_data_payload(conn, payload_length, 1);
}
diff --git a/drivers/thermal/db8500_thermal.c b/drivers/thermal/db8500_thermal.c
index 576f88b6a1b3..cf1706569e6d 100644
--- a/drivers/thermal/db8500_thermal.c
+++ b/drivers/thermal/db8500_thermal.c
@@ -10,7 +10,7 @@
#include <linux/cpu_cooling.h>
#include <linux/interrupt.h>
-#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/mfd/db8500-prcmu.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/platform_device.h>
@@ -82,7 +82,7 @@ static void db8500_thermal_update_config(struct db8500_thermal_zone *th,
unsigned long next_low,
unsigned long next_high)
{
- prcmu_stop_temp_sense();
+ db8500_prcmu_stop_temp_sense();
th->cur_index = idx;
th->interpolated_temp = (next_low + next_high)/2;
@@ -91,8 +91,8 @@ static void db8500_thermal_update_config(struct db8500_thermal_zone *th,
* The PRCMU accept absolute temperatures in celsius so divide
* down the millicelsius with 1000
*/
- prcmu_config_hotmon((u8)(next_low/1000), (u8)(next_high/1000));
- prcmu_start_temp_sense(PRCMU_DEFAULT_MEASURE_TIME);
+ db8500_prcmu_config_hotmon((u8)(next_low / 1000), (u8)(next_high / 1000));
+ db8500_prcmu_start_temp_sense(PRCMU_DEFAULT_MEASURE_TIME);
}
static irqreturn_t prcmu_low_irq_handler(int irq, void *irq_data)
@@ -204,7 +204,7 @@ static int db8500_thermal_probe(struct platform_device *pdev)
static int db8500_thermal_suspend(struct platform_device *pdev,
pm_message_t state)
{
- prcmu_stop_temp_sense();
+ db8500_prcmu_stop_temp_sense();
return 0;
}
diff --git a/drivers/thermal/thermal_sysfs.c b/drivers/thermal/thermal_sysfs.c
index b44abfc997ed..44bd7c50e4ac 100644
--- a/drivers/thermal/thermal_sysfs.c
+++ b/drivers/thermal/thermal_sysfs.c
@@ -400,8 +400,8 @@ static int create_trip_attrs(struct thermal_zone_device *tz)
struct thermal_trip_attrs *trip_attrs = &td->trip_attrs;
/* create trip type attribute */
- snprintf(trip_attrs->type.name, THERMAL_NAME_LENGTH,
- "trip_point_%d_type", i);
+ scnprintf(trip_attrs->type.name, sizeof(trip_attrs->type.name),
+ "trip_point_%d_type", i);
sysfs_attr_init(&trip_attrs->type.attr.attr);
trip_attrs->type.attr.attr.name = trip_attrs->type.name;
@@ -410,8 +410,8 @@ static int create_trip_attrs(struct thermal_zone_device *tz)
attrs[i] = &trip_attrs->type.attr.attr;
/* create trip temp attribute */
- snprintf(trip_attrs->temp.name, THERMAL_NAME_LENGTH,
- "trip_point_%d_temp", i);
+ scnprintf(trip_attrs->temp.name, sizeof(trip_attrs->temp.name),
+ "trip_point_%d_temp", i);
sysfs_attr_init(&trip_attrs->temp.attr.attr);
trip_attrs->temp.attr.attr.name = trip_attrs->temp.name;
@@ -423,8 +423,8 @@ static int create_trip_attrs(struct thermal_zone_device *tz)
}
attrs[i + tz->num_trips] = &trip_attrs->temp.attr.attr;
- snprintf(trip_attrs->hyst.name, THERMAL_NAME_LENGTH,
- "trip_point_%d_hyst", i);
+ scnprintf(trip_attrs->hyst.name, sizeof(trip_attrs->hyst.name),
+ "trip_point_%d_hyst", i);
sysfs_attr_init(&trip_attrs->hyst.attr.attr);
trip_attrs->hyst.attr.attr.name = trip_attrs->hyst.name;
diff --git a/drivers/ufs/host/ufshcd-pci.c b/drivers/ufs/host/ufshcd-pci.c
index f2433879b0eb..21bb11c724be 100644
--- a/drivers/ufs/host/ufshcd-pci.c
+++ b/drivers/ufs/host/ufshcd-pci.c
@@ -181,6 +181,25 @@ static int ufs_intel_lkf_pwr_change_notify(struct ufs_hba *hba,
return err;
}
+static int ufs_intel_nvl_pwr_change_notify(struct ufs_hba *hba,
+ enum ufs_notify_change_status stage,
+ struct ufs_pa_layer_attr *dev_req_params)
+{
+ int adapt_val;
+
+ if (stage != PRE_CHANGE || hba->ufs_version < ufshci_version(4, 0))
+ return 0;
+
+ if (dev_req_params->pwr_tx == FAST_MODE || dev_req_params->pwr_tx == FASTAUTO_MODE)
+ adapt_val = PA_INITIAL_ADAPT;
+ else
+ adapt_val = PA_NO_ADAPT;
+
+ ufshcd_dme_configure_adapt(hba, dev_req_params->gear_tx, adapt_val);
+
+ return 0;
+}
+
static int ufs_intel_lkf_apply_dev_quirks(struct ufs_hba *hba)
{
u32 granularity, peer_granularity;
@@ -441,6 +460,43 @@ static int ufs_intel_mtl_init(struct ufs_hba *hba)
return ufs_intel_common_init(hba);
}
+static int ufs_intel_mcq_config_resource(struct ufs_hba *hba)
+{
+ hba->mcq_base = hba->mmio_base + ufshcd_mcq_queue_cfg_addr(hba);
+
+ return 0;
+}
+
+/*
+ * This Intel UFS4.0 controller maps MCQ doorbell and interrupt-status
+ * registers into the same PCI BAR as the legacy HCI space, at this
+ * fixed offset/stride.
+ */
+#define UFS_INTEL_SQDAO0 0x2800
+#define UFS_INTEL_SQISAO0 0x2814
+#define UFS_INTEL_CQDAO0 0x281C
+#define UFS_INTEL_CQISAO0 0x2824
+#define UFS_INTEL_MCQ_STRIDE 0x30
+
+static int ufs_intel_op_runtime_config(struct ufs_hba *hba)
+{
+ struct ufshcd_mcq_opr_info_t *opr;
+ int i;
+
+ hba->mcq_opr[OPR_SQD].offset = UFS_INTEL_SQDAO0;
+ hba->mcq_opr[OPR_SQIS].offset = UFS_INTEL_SQISAO0;
+ hba->mcq_opr[OPR_CQD].offset = UFS_INTEL_CQDAO0;
+ hba->mcq_opr[OPR_CQIS].offset = UFS_INTEL_CQISAO0;
+
+ for (i = 0; i < OPR_MAX; i++) {
+ opr = &hba->mcq_opr[i];
+ opr->stride = UFS_INTEL_MCQ_STRIDE;
+ opr->base = hba->mmio_base + opr->offset;
+ }
+
+ return 0;
+}
+
static int ufs_qemu_get_hba_mac(struct ufs_hba *hba)
{
return MAX_SUPP_MAC;
@@ -527,6 +583,9 @@ static struct ufs_hba_variant_ops ufs_intel_mtl_hba_vops = {
.exit = ufs_intel_common_exit,
.hce_enable_notify = ufs_intel_hce_enable_notify,
.link_startup_notify = ufs_intel_link_startup_notify,
+ .pwr_change_notify = ufs_intel_nvl_pwr_change_notify,
+ .mcq_config_resource = ufs_intel_mcq_config_resource,
+ .op_runtime_config = ufs_intel_op_runtime_config,
.resume = ufs_intel_resume,
.device_reset = ufs_intel_device_reset,
};
diff --git a/drivers/usb/atm/xusbatm.c b/drivers/usb/atm/xusbatm.c
index 0befbf63d1cc..5c1e1f521555 100644
--- a/drivers/usb/atm/xusbatm.c
+++ b/drivers/usb/atm/xusbatm.c
@@ -79,7 +79,7 @@ static int xusbatm_bind(struct usbatm_data *usbatm,
struct usb_interface *intf, const struct usb_device_id *id)
{
struct usb_device *usb_dev = interface_to_usbdev(intf);
- int drv_ix = id - xusbatm_usb_ids;
+ int drv_ix = id->driver_info;
int rx_alt = rx_altsetting[drv_ix];
int tx_alt = tx_altsetting[drv_ix];
struct usb_interface *rx_intf = xusbatm_find_intf(usb_dev, rx_alt, rx_endpoint[drv_ix]);
@@ -168,7 +168,8 @@ static struct usb_driver xusbatm_usb_driver = {
.name = xusbatm_driver_name,
.probe = xusbatm_usb_probe,
.disconnect = usbatm_usb_disconnect,
- .id_table = xusbatm_usb_ids
+ .id_table = xusbatm_usb_ids,
+ .no_dynamic_id = 1,
};
static int __init xusbatm_init(void)
@@ -190,6 +191,7 @@ static int __init xusbatm_init(void)
xusbatm_usb_ids[i].match_flags = USB_DEVICE_ID_MATCH_DEVICE;
xusbatm_usb_ids[i].idVendor = vendor[i];
xusbatm_usb_ids[i].idProduct = product[i];
+ xusbatm_usb_ids[i].driver_info = i;
xusbatm_drivers[i].driver_name = xusbatm_driver_name;
xusbatm_drivers[i].bind = xusbatm_bind;
diff --git a/drivers/usb/class/usbtmc.c b/drivers/usb/class/usbtmc.c
index af9ae55dae14..51cd9320a736 100644
--- a/drivers/usb/class/usbtmc.c
+++ b/drivers/usb/class/usbtmc.c
@@ -71,7 +71,6 @@ struct usbtmc_dev_capabilities {
* allocated for each USBTMC device in the driver's probe function.
*/
struct usbtmc_device_data {
- const struct usb_device_id *id;
struct usb_device *usb_dev;
struct usb_interface *intf;
struct list_head file_list;
@@ -2394,7 +2393,6 @@ static int usbtmc_probe(struct usb_interface *intf,
return -ENOMEM;
data->intf = intf;
- data->id = id;
data->usb_dev = usb_get_dev(interface_to_usbdev(intf));
usb_set_intfdata(intf, data);
kref_init(&data->kref);
diff --git a/drivers/usb/serial/spcp8x5.c b/drivers/usb/serial/spcp8x5.c
index f610aef6bf59..3ef7ca2c0cf4 100644
--- a/drivers/usb/serial/spcp8x5.c
+++ b/drivers/usb/serial/spcp8x5.c
@@ -133,14 +133,14 @@ struct spcp8x5_private {
static int spcp8x5_probe(struct usb_serial *serial,
const struct usb_device_id *id)
{
- usb_set_serial_data(serial, (void *)id);
+ usb_set_serial_data(serial, (void *)id->driver_info);
return 0;
}
static int spcp8x5_port_probe(struct usb_serial_port *port)
{
- const struct usb_device_id *id = usb_get_serial_data(port->serial);
+ unsigned int quirks = (unsigned int)(unsigned long)usb_get_serial_data(port->serial);
struct spcp8x5_private *priv;
priv = kzalloc_obj(*priv);
@@ -148,7 +148,7 @@ static int spcp8x5_port_probe(struct usb_serial_port *port)
return -ENOMEM;
spin_lock_init(&priv->lock);
- priv->quirks = id->driver_info;
+ priv->quirks = quirks;
usb_set_serial_port_data(port, priv);
diff --git a/drivers/vdpa/ifcvf/ifcvf_main.c b/drivers/vdpa/ifcvf/ifcvf_main.c
index ab6d6ab3b3d8..2af1cec95884 100644
--- a/drivers/vdpa/ifcvf/ifcvf_main.c
+++ b/drivers/vdpa/ifcvf/ifcvf_main.c
@@ -724,7 +724,8 @@ static int ifcvf_vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
if (config->device_features & ~device_features) {
IFCVF_ERR(pdev, "The provisioned features 0x%llx are not supported by this device with features 0x%llx\n",
config->device_features, device_features);
- return -EINVAL;
+ ret = -EINVAL;
+ goto err;
}
device_features &= config->device_features;
}
diff --git a/drivers/vdpa/solidrun/snet_main.c b/drivers/vdpa/solidrun/snet_main.c
index 28d55315df2a..3e2cea1e45f3 100644
--- a/drivers/vdpa/solidrun/snet_main.c
+++ b/drivers/vdpa/solidrun/snet_main.c
@@ -418,11 +418,15 @@ static int snet_request_irqs(struct pci_dev *pdev, struct snet *snet)
snet->vqs[i]->irq_name, snet->vqs[i]);
if (ret) {
SNET_ERR(pdev, "Failed to request IRQ\n");
- return ret;
+ goto err_free_irqs;
}
snet->vqs[i]->irq = irq;
}
return 0;
+
+err_free_irqs:
+ snet_free_irqs(snet);
+ return ret;
}
static void snet_set_status(struct vdpa_device *vdev, u8 status)
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c b/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c
index f70f454dde8e..76dd5b0828d7 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c
@@ -79,10 +79,11 @@ static void vdpasim_blk_buffer_unlock(struct vdpasim_blk *blk)
static bool vdpasim_blk_check_range(struct vdpasim *vdpasim, u64 start_sector,
u64 num_sectors, u64 max_sectors)
{
- if (start_sector > VDPASIM_BLK_CAPACITY) {
+ if (start_sector >= VDPASIM_BLK_CAPACITY) {
dev_dbg(&vdpasim->vdpa.dev,
"starting sector exceeds the capacity - start: 0x%llx capacity: 0x%x\n",
start_sector, VDPASIM_BLK_CAPACITY);
+ return false;
}
if (num_sectors > max_sectors) {
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
index 29fd14ce5860..a6514b5ccd86 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
@@ -225,10 +225,15 @@ static void vdpasim_net_work(struct vdpasim *vdpasim)
break;
}
- ++tx_pkts;
read = vringh_iov_pull_iotlb(&txq->vring, &txq->out_iov,
net->buffer, PAGE_SIZE);
+ if (read <= 0) {
+ ++tx_errors;
+ vdpasim_net_complete(txq, 0);
+ continue;
+ }
+ ++tx_pkts;
tx_bytes += read;
if (!receive_filter(vdpasim, read)) {
diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vduse_dev.c
index 10dcf016bfb0..ed71e5835e60 100644
--- a/drivers/vdpa/vdpa_user/vduse_dev.c
+++ b/drivers/vdpa/vdpa_user/vduse_dev.c
@@ -1751,11 +1751,11 @@ static long vduse_dev_compat_ioctl(struct file *file, unsigned int cmd,
break;
}
default:
- ret = -ENOIOCTLCMD;
- break;
+ return vduse_dev_ioctl(file, cmd,
+ (unsigned long)compat_ptr(arg));
}
- return vduse_dev_ioctl(file, cmd, (unsigned long)compat_ptr(arg));
+ return ret;
}
#else
#define vduse_dev_compat_ioctl compat_ptr_ioctl
@@ -2094,7 +2094,9 @@ static bool vduse_validate_config(struct vduse_dev_config *config,
return false;
}
- if (config->vq_align > PAGE_SIZE)
+ if (config->vq_align < VRING_USED_ALIGN_SIZE ||
+ !is_power_of_2(config->vq_align) ||
+ config->vq_align > PAGE_SIZE)
return false;
if (config->config_size > PAGE_SIZE)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index 38d9c184082d..4d9d7c2216ed 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -1696,6 +1696,8 @@ static int vhost_net_set_features(struct vhost_net *n, const u64 *features)
if (virtio_features_test_bit(features, VIRTIO_F_ACCESS_PLATFORM)) {
if (vhost_init_device_iotlb(&n->dev))
goto out_unlock;
+ } else {
+ vhost_clear_device_iotlb(&n->dev);
}
for (i = 0; i < VHOST_NET_VQ_MAX; ++i) {
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index c3d913bd7cac..a31786796d4c 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -58,9 +58,12 @@ struct vhost_vdpa {
struct cdev cdev;
atomic_t opened;
u32 nvqs;
+ u16 vq_num_max;
int virtio_id;
int minor;
struct eventfd_ctx *config_ctx;
+ /* Serialises vhost_vdpa_config_cb() against config_ctx being replaced. */
+ spinlock_t config_lock;
int in_batch;
struct vdpa_iova_range range;
u32 batch_asid;
@@ -194,10 +197,12 @@ static irqreturn_t vhost_vdpa_virtqueue_cb(void *private)
static irqreturn_t vhost_vdpa_config_cb(void *private)
{
struct vhost_vdpa *v = private;
- struct eventfd_ctx *config_ctx = v->config_ctx;
+ unsigned long flags;
- if (config_ctx)
- eventfd_signal(config_ctx);
+ spin_lock_irqsave(&v->config_lock, flags);
+ if (v->config_ctx)
+ eventfd_signal(v->config_ctx);
+ spin_unlock_irqrestore(&v->config_lock, flags);
return IRQ_HANDLED;
}
@@ -236,7 +241,9 @@ static void vhost_vdpa_unsetup_vq_irq(struct vhost_vdpa *v, u16 qid)
static int _compat_vdpa_reset(struct vhost_vdpa *v)
{
struct vdpa_device *vdpa = v->vdpa;
+ const struct vdpa_config_ops *ops = vdpa->config;
u32 flags = 0;
+ int ret;
v->suspended = false;
@@ -246,7 +253,14 @@ static int _compat_vdpa_reset(struct vhost_vdpa *v)
VDPA_RESET_F_CLEAN_MAP : 0;
}
- return vdpa_reset(vdpa, flags);
+ v->vq_num_max = 0;
+ ret = vdpa_reset(vdpa, flags);
+ if (!ret) {
+ /* Some backends derive the max from mutable queue state. */
+ v->vq_num_max = ops->get_vq_num_max(vdpa);
+ }
+
+ return ret;
}
static int vhost_vdpa_reset(struct vhost_vdpa *v)
@@ -518,15 +532,22 @@ static long vhost_vdpa_get_vring_num(struct vhost_vdpa *v, u16 __user *argp)
static void vhost_vdpa_config_put(struct vhost_vdpa *v)
{
- if (v->config_ctx) {
- eventfd_ctx_put(v->config_ctx);
- v->config_ctx = NULL;
- }
+ struct eventfd_ctx *ctx;
+ unsigned long flags;
+
+ spin_lock_irqsave(&v->config_lock, flags);
+ ctx = v->config_ctx;
+ v->config_ctx = NULL;
+ spin_unlock_irqrestore(&v->config_lock, flags);
+
+ if (ctx)
+ eventfd_ctx_put(ctx);
}
static long vhost_vdpa_set_config_call(struct vhost_vdpa *v, u32 __user *argp)
{
struct vdpa_callback cb;
+ unsigned long flags;
int fd;
struct eventfd_ctx *ctx;
@@ -536,18 +557,20 @@ static long vhost_vdpa_set_config_call(struct vhost_vdpa *v, u32 __user *argp)
return -EFAULT;
ctx = fd == VHOST_FILE_UNBIND ? NULL : eventfd_ctx_fdget(fd);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ spin_lock_irqsave(&v->config_lock, flags);
swap(ctx, v->config_ctx);
+ spin_unlock_irqrestore(&v->config_lock, flags);
- if (!IS_ERR_OR_NULL(ctx))
+ /*
+ * The callback can no longer reach the old context, so this is the
+ * last reference to it.
+ */
+ if (ctx)
eventfd_ctx_put(ctx);
- if (IS_ERR(v->config_ctx)) {
- long ret = PTR_ERR(v->config_ctx);
-
- v->config_ctx = NULL;
- return ret;
- }
-
v->vdpa->config->set_config_cb(v->vdpa, &cb);
return 0;
@@ -648,9 +671,15 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
u32 idx;
long r;
- r = get_user(idx, (u32 __user *)argp);
- if (r < 0)
- return r;
+ if (cmd == VHOST_SET_VRING_NUM) {
+ if (copy_from_user(&s, argp, sizeof(s)))
+ return -EFAULT;
+ idx = s.index;
+ } else {
+ r = get_user(idx, (u32 __user *)argp);
+ if (r < 0)
+ return r;
+ }
if (idx >= v->nvqs)
return -ENOBUFS;
@@ -659,6 +688,23 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
vq = &v->vqs[idx];
switch (cmd) {
+ case VHOST_SET_VRING_NUM:
+ mutex_lock(&vq->mutex);
+ if (vq->private_data) {
+ r = -EBUSY;
+ } else if (!s.num || s.num > 0xffff ||
+ s.num > v->vq_num_max ||
+ (s.num & (s.num - 1))) {
+ r = -EINVAL;
+ } else {
+ vq->num = s.num;
+ r = 0;
+ }
+ mutex_unlock(&vq->mutex);
+ if (r)
+ return r;
+ ops->set_vq_num(vdpa, idx, s.num);
+ return 0;
case VHOST_VDPA_SET_VRING_ENABLE:
if (copy_from_user(&s, argp, sizeof(s)))
return -EFAULT;
@@ -772,9 +818,6 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
ops->set_vq_cb(vdpa, idx, &cb);
break;
- case VHOST_SET_VRING_NUM:
- ops->set_vq_num(vdpa, idx, vq->num);
- break;
}
return r;
@@ -1613,6 +1656,7 @@ static int vhost_vdpa_probe(struct vdpa_device *vdpa)
}
atomic_set(&v->opened, 0);
+ spin_lock_init(&v->config_lock);
v->minor = minor;
v->vdpa = vdpa;
v->nvqs = vdpa->nvqs;
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index 269efad90369..f9511be1fa43 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -344,6 +344,17 @@ static void __vhost_vq_meta_reset(struct vhost_virtqueue *vq)
vq->meta_iotlb[j] = NULL;
}
+/* Caller must hold the virtqueue mutex. */
+static void vhost_vq_invalidate_access(struct vhost_virtqueue *vq)
+{
+ vq->desc = NULL;
+ vq->avail = NULL;
+ vq->used = NULL;
+ vq->log_used = false;
+ vq->log_addr = -1ull;
+ __vhost_vq_meta_reset(vq);
+}
+
static void vhost_vq_meta_reset(struct vhost_dev *d)
{
int i;
@@ -1914,6 +1925,13 @@ int vq_meta_prefetch(struct vhost_virtqueue *vq)
{
unsigned int num = vq->num;
+ /*
+ * vhost_vq_invalidate_access() clears all three addresses together.
+ * A single zero address may be a valid GIOVA in IOTLB mode.
+ */
+ if (!vq->desc && !vq->avail && !vq->used)
+ return 0;
+
if (!vq->iotlb)
return 1;
@@ -2283,6 +2301,40 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg
}
EXPORT_SYMBOL_GPL(vhost_vring_ioctl);
+/* Caller must hold the device mutex. */
+void vhost_clear_device_iotlb(struct vhost_dev *d)
+{
+ struct vhost_iotlb *iotlb;
+ int i;
+
+ iotlb = d->iotlb;
+ if (!iotlb)
+ return;
+
+ vhost_dev_lock_vqs(d);
+
+ /*
+ * vhost_dev_lock_vqs() takes all VQ mutexes in index order. Drop the
+ * device-wide view while they are held, then clear each per-VQ view
+ * and its cached ring access before releasing the locks. Workers
+ * cannot observe a mixed address-space state during this handoff.
+ */
+ d->iotlb = NULL;
+
+ for (i = 0; i < d->nvqs; ++i) {
+ struct vhost_virtqueue *vq = d->vqs[i];
+
+ vq->iotlb = NULL;
+ vhost_vq_invalidate_access(vq);
+ }
+
+ vhost_dev_unlock_vqs(d);
+ vhost_clear_msg(d);
+ vhost_iotlb_free(iotlb);
+ wake_up_interruptible_poll(&d->wait, EPOLLIN | EPOLLRDNORM);
+}
+EXPORT_SYMBOL_GPL(vhost_clear_device_iotlb);
+
int vhost_init_device_iotlb(struct vhost_dev *d)
{
struct vhost_iotlb *niotlb, *oiotlb;
@@ -2303,7 +2355,10 @@ int vhost_init_device_iotlb(struct vhost_dev *d)
mutex_lock(&vq->mutex);
vq->iotlb = niotlb;
- __vhost_vq_meta_reset(vq);
+ if (oiotlb)
+ __vhost_vq_meta_reset(vq);
+ else
+ vhost_vq_invalidate_access(vq);
mutex_unlock(&vq->mutex);
}
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index 0192ade6e749..3c75e8089373 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -277,6 +277,7 @@ ssize_t vhost_chr_read_iter(struct vhost_dev *dev, struct iov_iter *to,
int noblock);
ssize_t vhost_chr_write_iter(struct vhost_dev *dev,
struct iov_iter *from);
+void vhost_clear_device_iotlb(struct vhost_dev *d);
int vhost_init_device_iotlb(struct vhost_dev *d);
void vhost_iotlb_map_free(struct vhost_iotlb *iotlb,
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index 9aaab6bb8061..abed1fbcf66c 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -868,6 +868,8 @@ static int vhost_vsock_set_features(struct vhost_vsock *vsock, u64 features)
if ((features & (1ULL << VIRTIO_F_ACCESS_PLATFORM))) {
if (vhost_init_device_iotlb(&vsock->dev))
goto err;
+ } else {
+ vhost_clear_device_iotlb(&vsock->dev);
}
vsock->seqpacket_allow = features & (1ULL << VIRTIO_VSOCK_F_SEQPACKET);
diff --git a/drivers/video/fbdev/vfb.c b/drivers/video/fbdev/vfb.c
index 5b7965f36c5e..de137b2bdaed 100644
--- a/drivers/video/fbdev/vfb.c
+++ b/drivers/video/fbdev/vfb.c
@@ -78,6 +78,13 @@ static int vfb_pan_display(struct fb_var_screeninfo *var,
static int vfb_mmap(struct fb_info *info,
struct vm_area_struct *vma);
+static void vfb_destroy(struct fb_info *info)
+{
+ vfree(info->screen_buffer);
+ fb_dealloc_cmap(&info->cmap);
+ framebuffer_release(info);
+}
+
static const struct fb_ops vfb_ops = {
.owner = THIS_MODULE,
__FB_DEFAULT_SYSMEM_OPS_RDWR,
@@ -87,6 +94,7 @@ static const struct fb_ops vfb_ops = {
.fb_pan_display = vfb_pan_display,
__FB_DEFAULT_SYSMEM_OPS_DRAW,
.fb_mmap = vfb_mmap,
+ .fb_destroy = vfb_destroy,
};
/*
@@ -485,9 +493,6 @@ static void vfb_remove(struct platform_device *dev)
if (info) {
unregister_framebuffer(info);
- vfree(videomemory);
- fb_dealloc_cmap(&info->cmap);
- framebuffer_release(info);
}
}
diff --git a/drivers/virtio/virtio.c b/drivers/virtio/virtio.c
index 75bb4ffe3b87..b6c9e927bef5 100644
--- a/drivers/virtio/virtio.c
+++ b/drivers/virtio/virtio.c
@@ -604,8 +604,8 @@ void unregister_virtio_device(struct virtio_device *dev)
{
int index = dev->index; /* save for after device release */
- device_unregister(&dev->dev);
virtio_debug_device_exit(dev);
+ device_unregister(&dev->dev);
ida_free(&virtio_index_ida, index);
}
EXPORT_SYMBOL_GPL(unregister_virtio_device);
diff --git a/drivers/virtio/virtio_input.c b/drivers/virtio/virtio_input.c
index deec24e8e682..e3bd0b9616f9 100644
--- a/drivers/virtio/virtio_input.c
+++ b/drivers/virtio/virtio_input.c
@@ -49,9 +49,12 @@ static void virtinput_recv_events(struct virtqueue *vq)
le16_to_cpu(event->code),
le32_to_cpu(event->value));
spin_lock_irqsave(&vi->lock, flags);
+ if (!vi->ready)
+ continue;
virtinput_queue_evtbuf(vi, event);
}
- virtqueue_kick(vq);
+ if (vi->ready)
+ virtqueue_kick(vq);
}
spin_unlock_irqrestore(&vi->lock, flags);
}
@@ -331,6 +334,7 @@ static int virtinput_probe(struct virtio_device *vdev)
spin_lock_irqsave(&vi->lock, flags);
vi->ready = false;
spin_unlock_irqrestore(&vi->lock, flags);
+ virtio_reset_device(vdev);
err_mt_init_slots:
input_free_device(vi->idev);
err_input_alloc:
@@ -350,8 +354,9 @@ static void virtinput_remove(struct virtio_device *vdev)
vi->ready = false;
spin_unlock_irqrestore(&vi->lock, flags);
- input_unregister_device(vi->idev);
+ /* Callbacks use vi->idev. */
virtio_reset_device(vdev);
+ input_unregister_device(vi->idev);
while ((buf = virtqueue_detach_unused_buf(vi->sts)) != NULL)
kfree(buf);
vdev->config->del_vqs(vdev);
diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index 510b7c4efdff..d5b148bde180 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -87,6 +87,9 @@ struct virtio_mmio_device {
void __iomem *base;
unsigned long version;
+
+ /* True if enable_irq_wake() succeeded for the shared IRQ. */
+ bool wake_irq_enabled;
};
/* Configuration interface */
@@ -329,11 +332,17 @@ static void vm_del_vqs(struct virtio_device *vdev)
{
struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
struct virtqueue *vq, *n;
+ int irq = platform_get_irq(vm_dev->pdev, 0);
list_for_each_entry_safe(vq, n, &vdev->vqs, list)
vm_del_vq(vq);
- free_irq(platform_get_irq(vm_dev->pdev, 0), vm_dev);
+ if (vm_dev->wake_irq_enabled) {
+ disable_irq_wake(irq);
+ vm_dev->wake_irq_enabled = false;
+ }
+
+ free_irq(irq, vm_dev);
}
static void vm_synchronize_cbs(struct virtio_device *vdev)
@@ -460,8 +469,9 @@ static int vm_find_vqs(struct virtio_device *vdev, unsigned int nvqs,
if (err)
return err;
- if (of_property_read_bool(vm_dev->pdev->dev.of_node, "wakeup-source"))
- enable_irq_wake(irq);
+ if (of_property_read_bool(vm_dev->pdev->dev.of_node, "wakeup-source") &&
+ !enable_irq_wake(irq))
+ vm_dev->wake_irq_enabled = true;
for (i = 0; i < nvqs; ++i) {
struct virtqueue_info *vqi = &vqs_info[i];
diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
index 10371ecbc054..b90c174450b2 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -120,7 +120,9 @@ static irqreturn_t vp_interrupt(int irq, void *opaque)
if (isr & VIRTIO_PCI_ISR_CONFIG)
vp_config_changed(irq, opaque);
- return vp_vring_interrupt(irq, opaque);
+ vp_vring_interrupt(irq, opaque);
+
+ return IRQ_HANDLED;
}
static int vp_request_msix_vectors(struct virtio_device *vdev, int nvectors,
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index b438dc2ce1b8..a9e1ee9536b5 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -1670,7 +1670,7 @@ static inline int virtqueue_add_packed(struct vring_virtqueue *vq,
struct scatterlist *sg;
unsigned int i, n, c, descs_used, err_idx, len;
__le16 head_flags, flags;
- u16 head, id, prev, curr, avail_used_flags;
+ u16 head, id, prev, curr, avail_used_flags, unpub_flags;
int err;
START_USE(vq);
@@ -1798,15 +1798,30 @@ static inline int virtqueue_add_packed(struct vring_virtqueue *vq,
curr = vq->free_head;
vq->packed.avail_used_flags = avail_used_flags;
+ unpub_flags = avail_used_flags ^ (1 << VRING_PACKED_DESC_F_AVAIL |
+ 1 << VRING_PACKED_DESC_F_USED);
for (n = 0; n < total_sg; n++) {
if (i == err_idx)
break;
+ /*
+ * The mapping loop made every descriptor but the head
+ * available. Stamp the previous wrap counter's AVAIL and USED
+ * bits on those, so that a later and shorter chain at this head
+ * does not leave one of them available beyond its own last
+ * descriptor. Marking them used instead would hand
+ * is_used_desc_packed() a completion we never made.
+ */
+ if (i != head)
+ desc[i].flags = cpu_to_le16(unpub_flags);
vring_unmap_extra_packed(vq, &vq->packed.desc_extra[curr]);
curr = vq->packed.desc_extra[curr].next;
i++;
- if (i >= vq->packed.vring.num)
+ if (i >= vq->packed.vring.num) {
i = 0;
+ unpub_flags ^= 1 << VRING_PACKED_DESC_F_AVAIL |
+ 1 << VRING_PACKED_DESC_F_USED;
+ }
}
END_USE(vq);
@@ -1828,7 +1843,7 @@ static inline int virtqueue_add_packed_in_order(struct vring_virtqueue *vq,
struct scatterlist *sg;
unsigned int i, n, sg_count, err_idx, total_in_len = 0;
__le16 head_flags, flags;
- u16 head, avail_used_flags;
+ u16 head, avail_used_flags, unpub_flags;
bool avail_wrap_counter;
int err;
@@ -1955,14 +1970,29 @@ static inline int virtqueue_add_packed_in_order(struct vring_virtqueue *vq,
i = head;
vq->packed.avail_used_flags = avail_used_flags;
vq->packed.avail_wrap_counter = avail_wrap_counter;
+ unpub_flags = avail_used_flags ^ (1 << VRING_PACKED_DESC_F_AVAIL |
+ 1 << VRING_PACKED_DESC_F_USED);
for (n = 0; n < total_sg; n++) {
if (i == err_idx)
break;
+ /*
+ * The mapping loop made every descriptor but the head
+ * available. Stamp the previous wrap counter's AVAIL and USED
+ * bits on those, so that a later and shorter chain at this head
+ * does not leave one of them available beyond its own last
+ * descriptor. Marking them used instead would hand
+ * is_used_desc_packed() a completion we never made.
+ */
+ if (i != head)
+ desc[i].flags = cpu_to_le16(unpub_flags);
vring_unmap_extra_packed(vq, &vq->packed.desc_extra[i]);
i++;
- if (i >= vq->packed.vring.num)
+ if (i >= vq->packed.vring.num) {
i = 0;
+ unpub_flags ^= 1 << VRING_PACKED_DESC_F_AVAIL |
+ 1 << VRING_PACKED_DESC_F_USED;
+ }
}
END_USE(vq);
diff --git a/drivers/watchdog/db8500_wdt.c b/drivers/watchdog/db8500_wdt.c
index 97148ac0aa54..70ccea13288d 100644
--- a/drivers/watchdog/db8500_wdt.c
+++ b/drivers/watchdog/db8500_wdt.c
@@ -16,7 +16,7 @@
#include <linux/watchdog.h>
#include <linux/platform_device.h>
-#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/mfd/db8500-prcmu.h>
#define WATCHDOG_TIMEOUT 600 /* 10 minutes */
@@ -37,24 +37,24 @@ MODULE_PARM_DESC(nowayout,
static int db8500_wdt_start(struct watchdog_device *wdd)
{
- return prcmu_enable_a9wdog(PRCMU_WDOG_ALL);
+ return db8500_prcmu_enable_a9wdog(PRCMU_WDOG_ALL);
}
static int db8500_wdt_stop(struct watchdog_device *wdd)
{
- return prcmu_disable_a9wdog(PRCMU_WDOG_ALL);
+ return db8500_prcmu_disable_a9wdog(PRCMU_WDOG_ALL);
}
static int db8500_wdt_keepalive(struct watchdog_device *wdd)
{
- return prcmu_kick_a9wdog(PRCMU_WDOG_ALL);
+ return db8500_prcmu_kick_a9wdog(PRCMU_WDOG_ALL);
}
static int db8500_wdt_set_timeout(struct watchdog_device *wdd,
unsigned int timeout)
{
db8500_wdt_stop(wdd);
- prcmu_load_a9wdog(PRCMU_WDOG_ALL, timeout * 1000);
+ db8500_prcmu_load_a9wdog(PRCMU_WDOG_ALL, timeout * 1000);
db8500_wdt_start(wdd);
return 0;
@@ -91,10 +91,10 @@ static int db8500_wdt_probe(struct platform_device *pdev)
watchdog_set_nowayout(&db8500_wdt, nowayout);
/* disable auto off on sleep */
- prcmu_config_a9wdog(PRCMU_WDOG_CPU1, false);
+ db8500_prcmu_config_a9wdog(PRCMU_WDOG_CPU1, false);
/* set HW initial value */
- prcmu_load_a9wdog(PRCMU_WDOG_ALL, timeout * 1000);
+ db8500_prcmu_load_a9wdog(PRCMU_WDOG_ALL, timeout * 1000);
ret = devm_watchdog_register_device(dev, &db8500_wdt);
if (ret)
@@ -110,9 +110,9 @@ static int db8500_wdt_suspend(struct platform_device *pdev,
{
if (watchdog_active(&db8500_wdt)) {
db8500_wdt_stop(&db8500_wdt);
- prcmu_config_a9wdog(PRCMU_WDOG_CPU1, true);
+ db8500_prcmu_config_a9wdog(PRCMU_WDOG_CPU1, true);
- prcmu_load_a9wdog(PRCMU_WDOG_ALL, timeout * 1000);
+ db8500_prcmu_load_a9wdog(PRCMU_WDOG_ALL, timeout * 1000);
db8500_wdt_start(&db8500_wdt);
}
return 0;
@@ -122,9 +122,9 @@ static int db8500_wdt_resume(struct platform_device *pdev)
{
if (watchdog_active(&db8500_wdt)) {
db8500_wdt_stop(&db8500_wdt);
- prcmu_config_a9wdog(PRCMU_WDOG_CPU1, false);
+ db8500_prcmu_config_a9wdog(PRCMU_WDOG_CPU1, false);
- prcmu_load_a9wdog(PRCMU_WDOG_ALL, timeout * 1000);
+ db8500_prcmu_load_a9wdog(PRCMU_WDOG_ALL, timeout * 1000);
db8500_wdt_start(&db8500_wdt);
}
return 0;
diff --git a/drivers/watchdog/msc313e_wdt.c b/drivers/watchdog/msc313e_wdt.c
index d962589e2c55..4a5cce2a16b1 100644
--- a/drivers/watchdog/msc313e_wdt.c
+++ b/drivers/watchdog/msc313e_wdt.c
@@ -31,20 +31,36 @@ struct msc313e_wdt_priv {
struct clk *clk;
};
+static u32 msc313e_wdt_get_hw_timeout(struct msc313e_wdt_priv *priv)
+{
+ u16 low, high;
+
+ low = readw(priv->base + REG_WDT_MAX_PRD_L);
+ high = readw(priv->base + REG_WDT_MAX_PRD_H);
+
+ return ((u32)high << 16) | low;
+}
+
+static void msc313e_wdt_set_hw_timeout(struct msc313e_wdt_priv *priv,
+ unsigned int timeout)
+{
+ u32 t = timeout * clk_get_rate(priv->clk);
+
+ writew(t & 0xffff, priv->base + REG_WDT_MAX_PRD_L);
+ writew((t >> 16) & 0xffff, priv->base + REG_WDT_MAX_PRD_H);
+ writew(1, priv->base + REG_WDT_CLR);
+}
+
static int msc313e_wdt_start(struct watchdog_device *wdev)
{
struct msc313e_wdt_priv *priv = watchdog_get_drvdata(wdev);
- u32 timeout;
int err;
err = clk_prepare_enable(priv->clk);
if (err)
return err;
- timeout = wdev->timeout * clk_get_rate(priv->clk);
- writew(timeout & 0xffff, priv->base + REG_WDT_MAX_PRD_L);
- writew((timeout >> 16) & 0xffff, priv->base + REG_WDT_MAX_PRD_H);
- writew(1, priv->base + REG_WDT_CLR);
+ msc313e_wdt_set_hw_timeout(priv, wdev->timeout);
return 0;
}
@@ -69,9 +85,13 @@ static int msc313e_wdt_stop(struct watchdog_device *wdev)
static int msc313e_wdt_settimeout(struct watchdog_device *wdev, unsigned int new_time)
{
+ struct msc313e_wdt_priv *priv = watchdog_get_drvdata(wdev);
+
wdev->timeout = new_time;
- return msc313e_wdt_start(wdev);
+ if (watchdog_hw_running(wdev) || watchdog_active(wdev))
+ msc313e_wdt_set_hw_timeout(priv, wdev->timeout);
+ return 0;
}
static const struct watchdog_info msc313e_wdt_ident = {
@@ -97,6 +117,8 @@ static int msc313e_wdt_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct msc313e_wdt_priv *priv;
+ unsigned long rate;
+ int ret;
priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
if (!priv)
@@ -116,27 +138,51 @@ static int msc313e_wdt_probe(struct platform_device *pdev)
priv->wdev.ops = &msc313e_wdt_ops,
priv->wdev.parent = dev;
priv->wdev.min_timeout = MSC313E_WDT_MIN_TIMEOUT;
- priv->wdev.max_timeout = U32_MAX / clk_get_rate(priv->clk);
+ rate = clk_get_rate(priv->clk);
+ if (!rate)
+ return -EINVAL;
+ priv->wdev.max_timeout = U32_MAX / rate;
priv->wdev.timeout = MSC313E_WDT_DEFAULT_TIMEOUT;
- /* If the period is non-zero the WDT is running */
- if (readw(priv->base + REG_WDT_MAX_PRD_L) | (readw(priv->base + REG_WDT_MAX_PRD_H) << 16))
- set_bit(WDOG_HW_RUNNING, &priv->wdev.status);
-
watchdog_set_drvdata(&priv->wdev, priv);
+ platform_set_drvdata(pdev, priv);
watchdog_init_timeout(&priv->wdev, timeout, dev);
watchdog_stop_on_reboot(&priv->wdev);
watchdog_stop_on_unregister(&priv->wdev);
+ watchdog_stop_ping_on_suspend(&priv->wdev);
+
+ ret = clk_prepare_enable(priv->clk);
+ if (ret)
+ return ret;
+
+ /* If the period is non-zero the WDT is running */
+ if (msc313e_wdt_get_hw_timeout(priv)) {
+ msc313e_wdt_set_hw_timeout(priv, priv->wdev.timeout);
+ set_bit(WDOG_HW_RUNNING, &priv->wdev.status);
+ /*
+ * Keep the clock enabled. The watchdog core will skip the next
+ * start() and a future stop() will balance the CCF reference
+ * count.
+ */
+ } else {
+ clk_disable_unprepare(priv->clk);
+ }
+
+ ret = devm_watchdog_register_device(dev, &priv->wdev);
+
+ /* If the WDT is running and anything goes wrong, disable the clock. */
+ if (ret && test_bit(WDOG_HW_RUNNING, &priv->wdev.status))
+ clk_disable_unprepare(priv->clk);
- return devm_watchdog_register_device(dev, &priv->wdev);
+ return ret;
}
static int __maybe_unused msc313e_wdt_suspend(struct device *dev)
{
struct msc313e_wdt_priv *priv = dev_get_drvdata(dev);
- if (watchdog_active(&priv->wdev))
+ if (watchdog_active(&priv->wdev) || watchdog_hw_running(&priv->wdev))
msc313e_wdt_stop(&priv->wdev);
return 0;
@@ -146,7 +192,7 @@ static int __maybe_unused msc313e_wdt_resume(struct device *dev)
{
struct msc313e_wdt_priv *priv = dev_get_drvdata(dev);
- if (watchdog_active(&priv->wdev))
+ if (watchdog_active(&priv->wdev) || watchdog_hw_running(&priv->wdev))
msc313e_wdt_start(&priv->wdev);
return 0;
diff --git a/drivers/watchdog/sunxi_wdt.c b/drivers/watchdog/sunxi_wdt.c
index b6c761acc3de..3db34524ed13 100644
--- a/drivers/watchdog/sunxi_wdt.c
+++ b/drivers/watchdog/sunxi_wdt.c
@@ -128,6 +128,38 @@ static int sunxi_wdt_ping(struct watchdog_device *wdt_dev)
return 0;
}
+static bool sunxi_wdt_is_running(struct watchdog_device *wdt_dev)
+{
+ struct sunxi_wdt_dev *sunxi_wdt = watchdog_get_drvdata(wdt_dev);
+ const struct sunxi_wdt_reg *regs = sunxi_wdt->wdt_regs;
+
+ return readl(sunxi_wdt->wdt_base + regs->wdt_mode) & WDT_MODE_EN;
+}
+
+static unsigned int sunxi_wdt_get_timeout(struct watchdog_device *wdt_dev)
+{
+ struct sunxi_wdt_dev *sunxi_wdt = watchdog_get_drvdata(wdt_dev);
+ const struct sunxi_wdt_reg *regs = sunxi_wdt->wdt_regs;
+ unsigned int timeout;
+ u32 interval;
+
+ interval = readl(sunxi_wdt->wdt_base + regs->wdt_mode);
+ interval >>= regs->wdt_timeout_shift;
+ interval &= WDT_TIMEOUT_MASK;
+ /* Round the 0.5-second interval up to the minimum representable timeout. */
+ if (!interval)
+ return WDT_MIN_TIMEOUT;
+
+ for (timeout = WDT_MIN_TIMEOUT;
+ timeout < ARRAY_SIZE(wdt_timeout_map); timeout++) {
+ if (wdt_timeout_map[timeout] == interval)
+ return timeout;
+ }
+
+ /* Reserved interval encoding. */
+ return 0;
+}
+
static int sunxi_wdt_set_timeout(struct watchdog_device *wdt_dev,
unsigned int timeout)
{
@@ -259,6 +291,7 @@ static int sunxi_wdt_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct sunxi_wdt_dev *sunxi_wdt;
+ unsigned int running_timeout;
int err;
sunxi_wdt = devm_kzalloc(dev, sizeof(*sunxi_wdt), GFP_KERNEL);
@@ -286,7 +319,17 @@ static int sunxi_wdt_probe(struct platform_device *pdev)
watchdog_set_drvdata(&sunxi_wdt->wdt_dev, sunxi_wdt);
- sunxi_wdt_stop(&sunxi_wdt->wdt_dev);
+ if (sunxi_wdt_is_running(&sunxi_wdt->wdt_dev)) {
+ running_timeout = sunxi_wdt_get_timeout(&sunxi_wdt->wdt_dev);
+ if (running_timeout)
+ sunxi_wdt->wdt_dev.timeout = running_timeout;
+
+ err = sunxi_wdt_start(&sunxi_wdt->wdt_dev);
+ if (err)
+ return err;
+
+ set_bit(WDOG_HW_RUNNING, &sunxi_wdt->wdt_dev.status);
+ }
watchdog_stop_on_reboot(&sunxi_wdt->wdt_dev);
err = devm_watchdog_register_device(dev, &sunxi_wdt->wdt_dev);
diff --git a/drivers/watchdog/watchdog_hrtimer_pretimeout.c b/drivers/watchdog/watchdog_hrtimer_pretimeout.c
index fbc7eecd8b20..49a05ea60c97 100644
--- a/drivers/watchdog/watchdog_hrtimer_pretimeout.c
+++ b/drivers/watchdog/watchdog_hrtimer_pretimeout.c
@@ -30,6 +30,7 @@ void watchdog_hrtimer_pretimeout_init(struct watchdog_device *wdd)
void watchdog_hrtimer_pretimeout_start(struct watchdog_device *wdd)
{
if (!(wdd->info->options & WDIOF_PRETIMEOUT) &&
+ wdd->pretimeout &&
!watchdog_pretimeout_invalid(wdd, wdd->pretimeout))
hrtimer_start(&wdd->wd_data->pretimeout_timer,
ktime_set(wdd->timeout - wdd->pretimeout, 0),
diff --git a/fs/afs/addr_list.c b/fs/afs/addr_list.c
index 63bf096b721a..73195d76b481 100644
--- a/fs/afs/addr_list.c
+++ b/fs/afs/addr_list.c
@@ -394,8 +394,11 @@ void afs_set_peer_appdata(struct afs_server *server,
struct rxrpc_peer *pn = new_alist->addrs[n].peer;
struct rxrpc_peer *po = old_alist->addrs[o].peer;
- if (pn == po)
+ if (pn == po) {
+ n++;
+ o++;
continue;
+ }
if (pn < po) {
rxrpc_kernel_set_peer_data(pn, data);
n++;
diff --git a/fs/afs/dir_edit.c b/fs/afs/dir_edit.c
index 3ead36a07048..c31303059444 100644
--- a/fs/afs/dir_edit.c
+++ b/fs/afs/dir_edit.c
@@ -442,7 +442,7 @@ void afs_edit_dir_remove(struct afs_vnode *vnode,
/* Check and clear the entry. */
de = &block->dirents[slot];
if (de->u.valid != 1)
- goto error_unmap;
+ goto error;
trace_afs_edit_dir(vnode, why, afs_edit_dir_delete, b, slot,
ntohl(de->u.vnode), ntohl(de->u.unique),
@@ -458,7 +458,6 @@ void afs_edit_dir_remove(struct afs_vnode *vnode,
/* Clear the constituent entries. */
next = de->u.hash_next;
memset(de, 0, sizeof(*de) * iter.nr_slots);
- kunmap_local(block);
/* Adjust the hash chain: if iter->prev_entry is 0, the hashtable head
* index is previous; otherwise it's slot number of the previous entry.
@@ -485,7 +484,6 @@ void afs_edit_dir_remove(struct afs_vnode *vnode,
pde = &pblock->dirents[ps];
prev_next = pde->u.hash_next;
if (prev_next != htons(entry)) {
- kunmap_local(pblock);
pr_warn("%llx:%llx:%x: not prev in chain b=%x p=%x,%x e=%x %*s",
vnode->fid.vid, vnode->fid.vnode, vnode->fid.unique,
iter.bucket, iter.prev_entry, prev_next, entry,
@@ -493,7 +491,6 @@ void afs_edit_dir_remove(struct afs_vnode *vnode,
goto error;
}
pde->u.hash_next = next;
- kunmap_local(pblock);
}
netfs_single_mark_inode_dirty(&vnode->netfs.inode);
@@ -503,18 +500,16 @@ void afs_edit_dir_remove(struct afs_vnode *vnode,
_debug("Remove %s from %u[%u]", name->name, b, slot);
out_unmap:
+ afs_dir_end_iter(&iter);
kunmap_local(meta);
_leave("");
return;
already_invalidated:
- kunmap_local(block);
trace_afs_edit_dir(vnode, why, afs_edit_dir_delete_inval,
0, 0, 0, 0, name->name);
goto out_unmap;
-error_unmap:
- kunmap_local(block);
error:
trace_afs_edit_dir(vnode, why, afs_edit_dir_delete_error,
0, 0, 0, 0, name->name);
diff --git a/fs/afs/dir_search.c b/fs/afs/dir_search.c
index 104411c0692f..11ebdfffcb1d 100644
--- a/fs/afs/dir_search.c
+++ b/fs/afs/dir_search.c
@@ -75,10 +75,7 @@ union afs_xdr_dir_block *afs_dir_find_block(struct afs_dir_iter *iter, size_t bl
_enter("%zx,%d", block, slot);
- if (iter->block) {
- kunmap_local(iter->block);
- iter->block = NULL;
- }
+ afs_dir_end_iter(iter);
if (dvnode->directory_size < blend)
goto fail;
@@ -173,12 +170,8 @@ int afs_dir_search_bucket(struct afs_dir_iter *iter, const struct qstr *name,
ret = -ENOENT;
found:
- if (iter->block) {
- kunmap_local(iter->block);
- iter->block = NULL;
- }
-
bad:
+ afs_dir_end_iter(iter);
if (ret == -ESTALE)
afs_invalidate_dir(iter->dvnode, afs_dir_invalid_iter_stale);
_leave(" = %d", ret);
diff --git a/fs/afs/fs_probe.c b/fs/afs/fs_probe.c
index a91ad1938d07..8c62334dbfe7 100644
--- a/fs/afs/fs_probe.c
+++ b/fs/afs/fs_probe.c
@@ -258,6 +258,7 @@ int afs_fs_probe_fileserver(struct afs_net *net, struct afs_server *server,
lockdep_is_held(&server->fs_lock));
if (old) {
estate->responsive_set = old->responsive_set;
+ old_alist = old->addresses;
if (!new_alist)
new_alist = old->addresses;
}
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 290873bac89b..330654ed16ec 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -1133,6 +1133,14 @@ int afs_dir_search_bucket(struct afs_dir_iter *iter, const struct qstr *name,
int afs_dir_search(struct afs_vnode *dvnode, const struct qstr *name,
struct afs_fid *_fid, afs_dataversion_t *_dir_version);
+static inline void afs_dir_end_iter(struct afs_dir_iter *iter)
+{
+ if (iter->block) {
+ kunmap_local(iter->block);
+ iter->block = NULL;
+ }
+}
+
/*
* dir_silly.c
*/
diff --git a/fs/afs/server.c b/fs/afs/server.c
index 0fe162ea2a36..189138bd6d71 100644
--- a/fs/afs/server.c
+++ b/fs/afs/server.c
@@ -242,7 +242,6 @@ struct afs_server *afs_lookup_server(struct afs_cell *cell, struct key *key,
out:
afs_put_addrlist(alist, afs_alist_trace_put_server_create);
if (candidate) {
- kfree(rcu_access_pointer(server->endpoint_state));
kfree(candidate);
afs_dec_servers_outstanding(cell->net);
}
diff --git a/fs/autofs/inode.c b/fs/autofs/inode.c
index c1e210cec436..6b15a3717ba7 100644
--- a/fs/autofs/inode.c
+++ b/fs/autofs/inode.c
@@ -323,8 +323,10 @@ static int autofs_fill_super(struct super_block *s, struct fs_context *fc)
return -ENOMEM;
root_inode = autofs_get_inode(s, S_IFDIR | 0755);
- if (!root_inode)
+ if (!root_inode) {
+ autofs_free_ino(ino);
return -ENOMEM;
+ }
root_inode->i_uid = ctx->uid;
root_inode->i_gid = ctx->gid;
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 39a01f9cf6b5..188b4ac9889f 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -3435,6 +3435,9 @@ int btrfs_finish_one_ordered(struct btrfs_ordered_extent *ordered_extent)
*/
btrfs_remove_ordered_extent(ordered_extent);
+ /* Cleanup any remaining biocs attached to the OE. */
+ btrfs_cleanup_ordered_bioc_list(ordered_extent);
+
/* once for us */
btrfs_put_ordered_extent(ordered_extent);
/* once for the tree */
diff --git a/fs/btrfs/raid-stripe-tree.c b/fs/btrfs/raid-stripe-tree.c
index b210371ce91e..d9e660447205 100644
--- a/fs/btrfs/raid-stripe-tree.c
+++ b/fs/btrfs/raid-stripe-tree.c
@@ -310,8 +310,10 @@ static int update_raid_extent_item(struct btrfs_trans_handle *trans,
ret = btrfs_search_slot(trans, trans->fs_info->stripe_root, key, path,
0, 1);
- if (ret)
- return (ret == 1 ? ret : -EINVAL);
+ if (ret > 0)
+ ret = -ENOENT;
+ if (ret < 0)
+ return ret;
leaf = path->nodes[0];
slot = path->slots[0];
@@ -337,7 +339,6 @@ int btrfs_insert_one_raid_extent(struct btrfs_trans_handle *trans,
stripe_extent = kzalloc(item_size, GFP_NOFS);
if (unlikely(!stripe_extent)) {
btrfs_abort_transaction(trans, -ENOMEM);
- btrfs_end_transaction(trans);
return -ENOMEM;
}
@@ -374,7 +375,7 @@ int btrfs_insert_raid_extent(struct btrfs_trans_handle *trans,
struct btrfs_ordered_extent *ordered_extent)
{
struct btrfs_io_context *bioc;
- int ret;
+ int ret = 0;
if (!btrfs_fs_incompat(trans->fs_info, RAID_STRIPE_TREE))
return 0;
@@ -382,17 +383,23 @@ int btrfs_insert_raid_extent(struct btrfs_trans_handle *trans,
list_for_each_entry(bioc, &ordered_extent->bioc_list, rst_ordered_entry) {
ret = btrfs_insert_one_raid_extent(trans, bioc);
if (ret)
- return ret;
+ break;
}
- while (!list_empty(&ordered_extent->bioc_list)) {
- bioc = list_first_entry(&ordered_extent->bioc_list,
+ btrfs_cleanup_ordered_bioc_list(ordered_extent);
+ return ret;
+}
+
+void btrfs_cleanup_ordered_bioc_list(struct btrfs_ordered_extent *ordered)
+{
+ while (!list_empty(&ordered->bioc_list)) {
+ struct btrfs_io_context *bioc;
+
+ bioc = list_first_entry(&ordered->bioc_list,
typeof(*bioc), rst_ordered_entry);
list_del(&bioc->rst_ordered_entry);
btrfs_put_bioc(bioc);
}
-
- return 0;
}
int btrfs_get_raid_extent_offset(struct btrfs_fs_info *fs_info,
diff --git a/fs/btrfs/raid-stripe-tree.h b/fs/btrfs/raid-stripe-tree.h
index 69942ad43140..eb02cf48511b 100644
--- a/fs/btrfs/raid-stripe-tree.h
+++ b/fs/btrfs/raid-stripe-tree.h
@@ -28,6 +28,7 @@ int btrfs_get_raid_extent_offset(struct btrfs_fs_info *fs_info,
u32 stripe_index, struct btrfs_io_stripe *stripe);
int btrfs_insert_raid_extent(struct btrfs_trans_handle *trans,
struct btrfs_ordered_extent *ordered_extent);
+void btrfs_cleanup_ordered_bioc_list(struct btrfs_ordered_extent *ordered);
#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS
int btrfs_insert_one_raid_extent(struct btrfs_trans_handle *trans,
diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c
index d2f7ac5b6e96..3b3f04521369 100644
--- a/fs/btrfs/scrub.c
+++ b/fs/btrfs/scrub.c
@@ -1041,6 +1041,10 @@ static void scrub_stripe_report_errors(struct scrub_ctx *sctx,
skip:
for_each_set_bit(sector_nr, &extent_bitmap, stripe->nr_sectors) {
+ const u64 sector_logical = stripe->logical +
+ ((u64)sector_nr << fs_info->sectorsize_bits);
+ const u64 sector_physical = physical +
+ ((u64)sector_nr << fs_info->sectorsize_bits);
bool repaired = false;
if (scrub_bitmap_test_bit_is_metadata(stripe, sector_nr)) {
@@ -1069,12 +1073,12 @@ static void scrub_stripe_report_errors(struct scrub_ctx *sctx,
if (dev) {
btrfs_err_rl(fs_info,
"scrub: fixed up error at logical %llu on dev %s physical %llu",
- stripe->logical, btrfs_dev_name(dev),
- physical);
+ sector_logical, btrfs_dev_name(dev),
+ sector_physical);
} else {
btrfs_err_rl(fs_info,
"scrub: fixed up error at logical %llu on mirror %u",
- stripe->logical, stripe->mirror_num);
+ sector_logical, stripe->mirror_num);
}
continue;
}
@@ -1083,30 +1087,30 @@ static void scrub_stripe_report_errors(struct scrub_ctx *sctx,
if (dev) {
btrfs_err_rl(fs_info,
"scrub: unable to fixup (regular) error at logical %llu on dev %s physical %llu",
- stripe->logical, btrfs_dev_name(dev),
- physical);
+ sector_logical, btrfs_dev_name(dev),
+ sector_physical);
} else {
btrfs_err_rl(fs_info,
"scrub: unable to fixup (regular) error at logical %llu on mirror %u",
- stripe->logical, stripe->mirror_num);
+ sector_logical, stripe->mirror_num);
}
if (scrub_bitmap_test_bit_io_error(stripe, sector_nr))
if (__ratelimit(&rs) && dev)
scrub_print_common_warning("i/o error", dev, false,
- stripe->logical, physical);
+ sector_logical, sector_physical);
if (scrub_bitmap_test_bit_csum_error(stripe, sector_nr))
if (__ratelimit(&rs) && dev)
scrub_print_common_warning("checksum error", dev, false,
- stripe->logical, physical);
+ sector_logical, sector_physical);
if (scrub_bitmap_test_bit_meta_error(stripe, sector_nr))
if (__ratelimit(&rs) && dev)
scrub_print_common_warning("header error", dev, false,
- stripe->logical, physical);
+ sector_logical, sector_physical);
if (scrub_bitmap_test_bit_meta_gen_error(stripe, sector_nr))
if (__ratelimit(&rs) && dev)
scrub_print_common_warning("generation error", dev, false,
- stripe->logical, physical);
+ sector_logical, sector_physical);
}
/* Update the device stats. */
diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index 3ae480c7474b..a888202397ed 100644
--- a/fs/btrfs/send.c
+++ b/fs/btrfs/send.c
@@ -2066,7 +2066,7 @@ static int will_overwrite_ref(struct send_ctx *sctx, u64 dir, u64 dir_gen,
ret = is_inode_existent(sctx, dir, dir_gen, NULL, &parent_root_dir_gen);
if (ret <= 0)
- return 0;
+ return ret;
/*
* If we have a parent root we need to verify that the parent dir was
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 80919590175f..b64c6d2855c9 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -458,8 +458,19 @@ static int record_root_in_trans(struct btrfs_trans_handle *trans,
* through btrfs_record_root_in_trans without having to take the
* lock. smp_wmb() makes sure that all the writes above are
* done before we pop in the zero below
+ *
+ * If @force is true, it means the call is from
+ * qgroup_account_snapshot(), which only requires radix tree
+ * tracking.
+ * We should not force reloc root creation here, as the root
+ * may have already been modified, and in that case
+ * root->commit_root has already been dropped.
+ *
+ * Using that commit root will cause the reloc root to refer
+ * to a deleted extent, causing extent tree corruption.
*/
- ret = btrfs_init_reloc_root(trans, root);
+ if (!force)
+ ret = btrfs_init_reloc_root(trans, root);
smp_mb__before_atomic();
clear_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state);
}
diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c
index 875e4ddc68ea..c87f6a7e8d61 100644
--- a/fs/btrfs/tree-log.c
+++ b/fs/btrfs/tree-log.c
@@ -7307,6 +7307,22 @@ static int btrfs_log_all_parents(struct btrfs_trans_handle *trans,
ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
if (ret < 0)
goto out;
+ /*
+ * There can't be an inode ref key with offset 0 because inode numbers
+ * start at BTRFS_FIRST_FREE_OBJECTID.
+ */
+ if (WARN_ON_ONCE(ret == 0)) {
+ btrfs_err(trans->fs_info,
+ "found inode ref key with offset 0 for root %llu inode %llu",
+ btrfs_root_id(root), ino);
+ ret = BTRFS_LOG_FORCE_COMMIT;
+ goto out;
+ }
+ /*
+ * Set to 0 so that in case we don't do any work below, we won't return
+ * 1 and trigger an unnecessary transaction commit.
+ */
+ ret = 0;
while (true) {
struct extent_buffer *leaf = path->nodes[0];
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 6eab4cc73ce4..b90cbd1e3ebe 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -3070,7 +3070,11 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_path
error_sysfs:
btrfs_sysfs_remove_device(device);
mutex_lock(&fs_info->fs_devices->device_list_mutex);
+ if (seeding_dev)
+ btrfs_assign_next_active_device(device, seed_devices->latest_dev);
mutex_lock(&fs_info->chunk_mutex);
+ if (!list_empty(&device->post_commit_list))
+ list_del_init(&device->post_commit_list);
list_del_rcu(&device->dev_list);
list_del(&device->dev_alloc_list);
fs_info->fs_devices->num_devices--;
diff --git a/fs/btrfs/zoned.c b/fs/btrfs/zoned.c
index a016cb471beb..7f0dde6398d4 100644
--- a/fs/btrfs/zoned.c
+++ b/fs/btrfs/zoned.c
@@ -2626,16 +2626,13 @@ static int do_zone_finish(struct btrfs_block_group *block_group, bool fully_writ
down_read(&dev_replace->rwsem);
map = block_group->physical_map;
for (i = 0; i < map->num_stripes; i++) {
-
ret = call_zone_finish(block_group, &map->stripes[i]);
- if (ret) {
- up_read(&dev_replace->rwsem);
- return ret;
- }
+ if (ret)
+ break;
}
up_read(&dev_replace->rwsem);
- if (!fully_written)
+ if (!ret && !fully_written)
btrfs_dec_block_group_ro(block_group);
spin_lock(&fs_info->zone_active_bgs_lock);
@@ -2648,7 +2645,7 @@ static int do_zone_finish(struct btrfs_block_group *block_group, bool fully_writ
clear_and_wake_up_bit(BTRFS_FS_NEED_ZONE_FINISH, &fs_info->flags);
- return 0;
+ return ret;
}
int btrfs_zone_finish(struct btrfs_block_group *block_group)
diff --git a/fs/btrfs/zstd.c b/fs/btrfs/zstd.c
index 86919293fd54..58d9ff76fe07 100644
--- a/fs/btrfs/zstd.c
+++ b/fs/btrfs/zstd.c
@@ -307,8 +307,17 @@ struct list_head *zstd_get_workspace(struct btrfs_fs_info *fs_info, int level)
DEFINE_WAIT(wait);
prepare_to_wait(&zwsm->wait, &wait, TASK_UNINTERRUPTIBLE);
- schedule();
+ /*
+ * Re-check after being queued: zstd_put_workspace() only wakes
+ * a queue that already has a sleeper, so a workspace returned
+ * since the failed allocation woke nobody.
+ */
+ ws = zstd_find_workspace(fs_info, level);
+ if (!ws)
+ schedule();
finish_wait(&zwsm->wait, &wait);
+ if (ws)
+ return ws;
goto again;
}
diff --git a/fs/cachefiles/xattr.c b/fs/cachefiles/xattr.c
index f8ae78b3f7b6..c70bf67e52b0 100644
--- a/fs/cachefiles/xattr.c
+++ b/fs/cachefiles/xattr.c
@@ -13,6 +13,7 @@
#include <linux/quotaops.h>
#include <linux/xattr.h>
#include <linux/slab.h>
+#include <linux/unaligned.h>
#include "internal.h"
#define CACHEFILES_COOKIE_TYPE_DATA 1
@@ -50,7 +51,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object)
_enter("%x,#%d", object->debug_id, len);
- buf = kmalloc(sizeof(struct cachefiles_xattr) + len, GFP_KERNEL);
+ buf = kmalloc(sizeof(struct cachefiles_xattr) + max(len, sizeof(__be64)), GFP_KERNEL);
if (!buf)
return -ENOMEM;
@@ -60,6 +61,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object)
buf->content = object->content_info;
if (test_bit(FSCACHE_COOKIE_LOCAL_WRITE, &object->cookie->flags))
buf->content = CACHEFILES_CONTENT_DIRTY;
+ put_unaligned_be64(0, (__be64 *)buf->data);
if (len > 0)
memcpy(buf->data, fscache_get_aux(object->cookie), len);
@@ -77,8 +79,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object)
trace_cachefiles_vfs_error(object, file_inode(file), ret,
cachefiles_trace_setxattr_error);
trace_cachefiles_coherency(object, file_inode(file)->i_ino,
- be64_to_cpup((__be64 *)buf->data),
- buf->content,
+ buf->data, buf->content,
cachefiles_coherency_set_fail);
if (ret != -ENOMEM)
cachefiles_io_error_obj(
@@ -86,8 +87,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object)
"Failed to set xattr with error %d", ret);
} else {
trace_cachefiles_coherency(object, file_inode(file)->i_ino,
- be64_to_cpup((__be64 *)buf->data),
- buf->content,
+ buf->data, buf->content,
cachefiles_coherency_set_ok);
}
@@ -110,9 +110,10 @@ int cachefiles_check_auxdata(struct cachefiles_object *object, struct file *file
int ret = -ESTALE;
tlen = sizeof(struct cachefiles_xattr) + len;
- buf = kmalloc(tlen, GFP_KERNEL);
+ buf = kmalloc(sizeof(struct cachefiles_xattr) + max(len, sizeof(__be64)), GFP_KERNEL);
if (!buf)
return -ENOMEM;
+ put_unaligned_be64(0, (__be64 *)buf->data);
xlen = cachefiles_inject_read_error();
if (xlen == 0)
@@ -148,8 +149,7 @@ int cachefiles_check_auxdata(struct cachefiles_object *object, struct file *file
out:
trace_cachefiles_coherency(object, file_inode(file)->i_ino,
- be64_to_cpup((__be64 *)buf->data),
- buf->content, why);
+ buf->data, buf->content, why);
kfree(buf);
return ret;
}
diff --git a/fs/configfs/dir.c b/fs/configfs/dir.c
index 3c88f13f1ca2..eda80c2a2d38 100644
--- a/fs/configfs/dir.c
+++ b/fs/configfs/dir.c
@@ -416,6 +416,15 @@ static void configfs_remove_dir(struct dentry *d)
if (d_really_is_positive(d)) {
if (unlikely(simple_rmdir(d_inode(parent), d)))
pr_warn("remove_dir (%pd): attributes remain", d);
+ else
+ /*
+ * configfs_get_config_item() takes a hashed dentry as
+ * proof that ->s_element is still alive. Our caller
+ * is about to drop the last reference to the item and
+ * the VFS will not unhash until after we return, so
+ * unhash it here.
+ */
+ d_drop(d);
}
pr_debug(" o %pd removing done (%d)\n", d, d_count(d));
diff --git a/fs/configfs/symlink.c b/fs/configfs/symlink.c
index 31eb28b27309..3b31c714400f 100644
--- a/fs/configfs/symlink.c
+++ b/fs/configfs/symlink.c
@@ -76,9 +76,9 @@ static int configfs_get_target_path(struct config_item *item,
static int create_link(struct config_item *parent_item,
struct config_item *item,
+ struct configfs_dirent *target_sd,
struct dentry *dentry)
{
- struct configfs_dirent *target_sd = item->ci_dentry->d_fsdata;
char *body;
int ret;
@@ -115,6 +115,7 @@ static int create_link(struct config_item *parent_item,
static int get_target(const char *symname, struct config_item **target,
+ struct configfs_dirent **target_sd,
struct super_block *sb)
{
struct path path __free(path_put) = {};
@@ -125,7 +126,20 @@ static int get_target(const char *symname, struct config_item **target,
return ret;
if (path.dentry->d_sb != sb)
return -EPERM;
- *target = configfs_get_config_item(path.dentry);
+ /*
+ * A hashed dentry guarantees that neither the item nor the dirent
+ * have been released yet, as removals unhash before dropping.
+ * Grab both references here. An item reference alone would not keep
+ * ->ci_dentry alive.
+ */
+ spin_lock(&path.dentry->d_lock);
+ if (!d_unhashed(path.dentry)) {
+ struct configfs_dirent *sd = path.dentry->d_fsdata;
+
+ *target = config_item_get(sd->s_element);
+ *target_sd = configfs_get(sd);
+ }
+ spin_unlock(&path.dentry->d_lock);
if (!*target)
return -ENOENT;
return 0;
@@ -139,6 +153,7 @@ int configfs_symlink(struct mnt_idmap *idmap, struct inode *dir,
struct configfs_dirent *sd;
struct config_item *parent_item;
struct config_item *target_item = NULL;
+ struct configfs_dirent *target_sd = NULL;
const struct config_item_type *type;
sd = dentry->d_parent->d_fsdata;
@@ -182,7 +197,7 @@ int configfs_symlink(struct mnt_idmap *idmap, struct inode *dir,
* AV, a thoroughly annoyed bastard.
*/
inode_unlock(dir);
- ret = get_target(symname, &target_item, dentry->d_sb);
+ ret = get_target(symname, &target_item, &target_sd, dentry->d_sb);
inode_lock(dir);
if (ret)
goto out_put;
@@ -196,13 +211,14 @@ int configfs_symlink(struct mnt_idmap *idmap, struct inode *dir,
ret = type->ct_item_ops->allow_link(parent_item, target_item);
if (!ret) {
mutex_lock(&configfs_symlink_mutex);
- ret = create_link(parent_item, target_item, dentry);
+ ret = create_link(parent_item, target_item, target_sd, dentry);
mutex_unlock(&configfs_symlink_mutex);
if (ret && type->ct_item_ops->drop_link)
type->ct_item_ops->drop_link(parent_item,
target_item);
}
+ configfs_put(target_sd);
config_item_put(target_item);
out_put:
diff --git a/fs/erofs/data.c b/fs/erofs/data.c
index 9aa48c8d67d1..b6cdffd0d893 100644
--- a/fs/erofs/data.c
+++ b/fs/erofs/data.c
@@ -61,7 +61,7 @@ void *erofs_bread(struct erofs_buf *buf, erofs_off_t offset, bool need_kmap)
return NULL;
if (!buf->base)
buf->base = kmap_local_page(buf->page);
- return buf->base + (offset & ~PAGE_MASK);
+ return buf->base + ((buf->off + offset) & ~PAGE_MASK);
}
int erofs_init_metabuf(struct erofs_buf *buf, struct super_block *sb,
diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c
index 27caf4bebddc..d387b27c4ee2 100644
--- a/fs/erofs/decompressor.c
+++ b/fs/erofs/decompressor.c
@@ -7,8 +7,6 @@
#include "compress.h"
#include <linux/lz4.h>
-#define LZ4_MAX_DISTANCE_PAGES (DIV_ROUND_UP(LZ4_DISTANCE_MAX, PAGE_SIZE) + 1)
-
static int z_erofs_load_lz4_config(struct super_block *sb,
struct erofs_super_block *dsb, void *data, int size)
{
@@ -21,8 +19,6 @@ static int z_erofs_load_lz4_config(struct super_block *sb,
erofs_err(sb, "invalid lz4 cfgs, size=%u", size);
return -EINVAL;
}
- distance = le16_to_cpu(lz4->max_distance);
-
sbi->lz4.max_pclusterblks = le16_to_cpu(lz4->max_pclusterblks);
if (!sbi->lz4.max_pclusterblks) {
sbi->lz4.max_pclusterblks = 1; /* reserved case */
@@ -39,45 +35,25 @@ static int z_erofs_load_lz4_config(struct super_block *sb,
sbi->lz4.max_pclusterblks = 1;
sbi->available_compr_algs = 1 << Z_EROFS_COMPRESSION_LZ4;
}
-
- sbi->lz4.max_distance_pages = distance ?
- DIV_ROUND_UP(distance, PAGE_SIZE) + 1 :
- LZ4_MAX_DISTANCE_PAGES;
return z_erofs_gbuf_growsize(sbi->lz4.max_pclusterblks);
}
/*
- * Fill all gaps with bounce pages if it's a sparse page list. Also check if
- * all physical pages are consecutive, which can be seen for moderate CR.
+ * Fill all gaps with bounce pages if it's a sparse page list (for example some
+ * folios are already uptodate and thus can be mapped into userspace). Also
+ * check if pages are physically consecutive, which can be seen for moderate CR.
*/
-static int z_erofs_lz4_prepare_dstpages(struct z_erofs_decompress_req *rq,
- struct page **pagepool)
+static int z_erofs_oneshot_prepare_dstpages(struct z_erofs_decompress_req *rq,
+ struct page **pagepool)
{
- struct page *availables[LZ4_MAX_DISTANCE_PAGES] = { NULL };
- unsigned long bounced[DIV_ROUND_UP(LZ4_MAX_DISTANCE_PAGES,
- BITS_PER_LONG)] = { 0 };
- unsigned int lz4_max_distance_pages =
- EROFS_SB(rq->sb)->lz4.max_distance_pages;
void *kaddr = NULL;
- unsigned int i, j, top;
+ unsigned int i;
- top = 0;
- for (i = j = 0; i < rq->outpages; ++i, ++j) {
- struct page *const page = rq->out[i];
- struct page *victim;
-
- if (j >= lz4_max_distance_pages)
- j = 0;
-
- /* 'valid' bounced can only be tested after a complete round */
- if (!rq->fillgaps && test_bit(j, bounced)) {
- DBG_BUGON(i < lz4_max_distance_pages);
- DBG_BUGON(top >= lz4_max_distance_pages);
- availables[top++] = rq->out[i - lz4_max_distance_pages];
- }
+ for (i = 0; i < rq->outpages; ++i) {
+ struct page *page, *victim;
+ page = rq->out[i];
if (page) {
- __clear_bit(j, bounced);
if (!PageHighMem(page)) {
if (!i) {
kaddr = page_address(page);
@@ -89,21 +65,14 @@ static int z_erofs_lz4_prepare_dstpages(struct z_erofs_decompress_req *rq,
continue;
}
}
- kaddr = NULL;
- continue;
- }
- kaddr = NULL;
- __set_bit(j, bounced);
-
- if (top) {
- victim = availables[--top];
} else {
victim = __erofs_allocpage(pagepool, rq->gfp, true);
if (!victim)
return -ENOMEM;
set_page_private(victim, Z_EROFS_SHORTLIVED_PAGE);
+ rq->out[i] = victim;
}
- rq->out[i] = victim;
+ kaddr = NULL;
}
return kaddr ? 1 : 0;
}
@@ -266,7 +235,7 @@ static const char *z_erofs_lz4_decompress(struct z_erofs_decompress_req *rq,
dst_maptype = 0;
} else {
/* general decoding path which can be used for all cases */
- ret = z_erofs_lz4_prepare_dstpages(rq, pagepool);
+ ret = z_erofs_oneshot_prepare_dstpages(rq, pagepool);
if (ret < 0)
return ERR_PTR(ret);
if (ret > 0) {
diff --git a/fs/erofs/decompressor_lzma.c b/fs/erofs/decompressor_lzma.c
index 6b0cdb446c6a..9d15f94cbee1 100644
--- a/fs/erofs/decompressor_lzma.c
+++ b/fs/erofs/decompressor_lzma.c
@@ -5,6 +5,7 @@
struct z_erofs_lzma {
struct z_erofs_lzma *next;
struct xz_dec_microlzma *state;
+ unsigned int dict_size;
u8 bounce[PAGE_SIZE];
};
@@ -128,11 +129,19 @@ static int z_erofs_load_lzma_config(struct super_block *sb,
err = 0;
/* 2. walk each isolated stream and grow max dict_size if needed */
for (strm = head; strm; strm = strm->next) {
+ struct xz_dec_microlzma *state;
+
+ if (strm->dict_size >= dict_size)
+ continue;
+ state = xz_dec_microlzma_alloc(XZ_PREALLOC, dict_size);
+ if (!state) {
+ err = -ENOMEM;
+ break;
+ }
if (strm->state)
xz_dec_microlzma_end(strm->state);
- strm->state = xz_dec_microlzma_alloc(XZ_PREALLOC, dict_size);
- if (!strm->state)
- err = -ENOMEM;
+ strm->state = state;
+ strm->dict_size = dict_size;
}
/* 3. push back all to the global list and update max dict_size */
@@ -142,7 +151,8 @@ static int z_erofs_load_lzma_config(struct super_block *sb,
spin_unlock(&z_erofs_lzma_lock);
wake_up_all(&z_erofs_lzma_wq);
- z_erofs_lzma_max_dictsize = dict_size;
+ if (!err)
+ z_erofs_lzma_max_dictsize = dict_size;
mutex_unlock(&lzma_resize_mutex);
return err;
}
diff --git a/fs/erofs/internal.h b/fs/erofs/internal.h
index bee2f50c8488..be98c9222bf7 100644
--- a/fs/erofs/internal.h
+++ b/fs/erofs/internal.h
@@ -71,12 +71,8 @@ struct erofs_dev_context {
bool flatdev;
};
-/* all filesystem-wide lz4 configurations */
struct erofs_sb_lz4_info {
- /* # of pages needed for EROFS lz4 rolling decompression */
- u16 max_distance_pages;
- /* maximum possible blocks for pclusters in the filesystem */
- u16 max_pclusterblks;
+ u16 max_pclusterblks; /* maximum physical blocks for LZ4 pclusters */
};
struct erofs_xattr_prefix_item {
diff --git a/fs/erofs/sysfs.c b/fs/erofs/sysfs.c
index 6734483a440f..dfcec9376cd5 100644
--- a/fs/erofs/sysfs.c
+++ b/fs/erofs/sysfs.c
@@ -95,6 +95,7 @@ EROFS_ATTR_FEATURE(sb_chksum);
EROFS_ATTR_FEATURE(ztailpacking);
EROFS_ATTR_FEATURE(fragments);
EROFS_ATTR_FEATURE(dedupe);
+EROFS_ATTR_FEATURE(xattr_prefixes);
EROFS_ATTR_FEATURE(48bit);
EROFS_ATTR_FEATURE(metabox);
@@ -108,6 +109,7 @@ static struct attribute *erofs_feat_attrs[] = {
ATTR_LIST(ztailpacking),
ATTR_LIST(fragments),
ATTR_LIST(dedupe),
+ ATTR_LIST(xattr_prefixes),
ATTR_LIST(48bit),
ATTR_LIST(metabox),
NULL,
diff --git a/fs/erofs/xattr.c b/fs/erofs/xattr.c
index df7ea019526d..57cfb7520782 100644
--- a/fs/erofs/xattr.c
+++ b/fs/erofs/xattr.c
@@ -620,8 +620,8 @@ int erofs_xattr_fill_inode_fingerprint(struct erofs_inode_fingerprint *fp,
{
struct erofs_sb_info *sbi = EROFS_SB(inode->i_sb);
struct erofs_xattr_prefix_item *prefix;
+ int domainlen, valuelen, base_index;
const char *infix;
- int valuelen, base_index;
if (!test_opt(&sbi->opt, INODE_SHARE))
return -EOPNOTSUPP;
@@ -633,17 +633,18 @@ int erofs_xattr_fill_inode_fingerprint(struct erofs_inode_fingerprint *fp,
valuelen = erofs_getxattr(inode, base_index, infix, NULL, 0);
if (valuelen <= 0 || valuelen > (1 << sbi->blkszbits))
return -EFSCORRUPTED;
- fp->size = valuelen + (domain_id ? strlen(domain_id) : 0);
+ domainlen = strlen(domain_id);
+ fp->size = domainlen + 1 + valuelen;
fp->opaque = kmalloc(fp->size, GFP_KERNEL);
if (!fp->opaque)
return -ENOMEM;
+ memcpy(fp->opaque, domain_id, domainlen + 1);
if (valuelen != erofs_getxattr(inode, base_index, infix,
- fp->opaque, valuelen)) {
+ fp->opaque + domainlen + 1, valuelen)) {
kfree(fp->opaque);
fp->opaque = NULL;
return -EFSCORRUPTED;
}
- memcpy(fp->opaque + valuelen, domain_id, fp->size - valuelen);
return 0;
}
#endif
diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
index 0e9cac8ee12d..f0759211b723 100644
--- a/fs/erofs/zdata.c
+++ b/fs/erofs/zdata.c
@@ -1264,7 +1264,7 @@ static int z_erofs_decompress_pcluster(struct z_erofs_backend *be, bool eio)
const struct z_erofs_decompressor *alg =
z_erofs_decomp[pcl->algorithmformat];
bool try_free = true;
- int i, j, jtop, err2, err = eio ? -EIO : 0;
+ int i, err2, err = eio ? -EIO : 0;
struct page *page;
bool overlapped;
const char *reason;
@@ -1353,7 +1353,6 @@ static int z_erofs_decompress_pcluster(struct z_erofs_backend *be, bool eio)
be->compressed_pages >= be->onstack_pages + Z_EROFS_ONSTACK_PAGES)
kvfree(be->compressed_pages);
- jtop = 0;
z_erofs_fill_other_copies(be, err);
for (i = 0; i < be->nr_pages; ++i) {
page = be->decompressed_pages[i];
@@ -1361,22 +1360,11 @@ static int z_erofs_decompress_pcluster(struct z_erofs_backend *be, bool eio)
continue;
DBG_BUGON(z_erofs_page_is_invalidated(page));
- if (!z_erofs_is_shortlived_page(page)) {
+ if (!z_erofs_is_shortlived_page(page))
erofs_onlinefolio_end(page_folio(page), err, true);
- continue;
- }
- if (pcl->algorithmformat != Z_EROFS_COMPRESSION_LZ4) {
+ else
erofs_pagepool_add(be->pagepool, page);
- continue;
- }
- for (j = 0; j < jtop && be->decompressed_pages[j] != page; ++j)
- ;
- if (j >= jtop) /* this bounce page is newly detected */
- be->decompressed_pages[jtop++] = page;
}
- while (jtop)
- erofs_pagepool_add(be->pagepool,
- be->decompressed_pages[--jtop]);
if (be->decompressed_pages != be->onstack_pages)
kvfree(be->decompressed_pages);
diff --git a/fs/exec.c b/fs/exec.c
index d01523d0d8b4..7ae972f3c886 100644
--- a/fs/exec.c
+++ b/fs/exec.c
@@ -1149,6 +1149,20 @@ int begin_new_exec(struct linux_binprm * bprm)
if (retval)
goto out;
+ /*
+ * We have to apply CLOEXEC before we change whether the process is
+ * dumpable (in setup_new_exec) to avoid a race with a process in userspace
+ * trying to access the should-be-closed file descriptors of a process
+ * undergoing exec(2).
+ *
+ * This can block on filesystem ->flush() handlers, including waiting
+ * for FUSE daemons, so do it before exec_mmap takes the
+ * exec_update_lock.
+ * This must happen after the point of no return, and after unsharing
+ * the FD table.
+ */
+ do_close_on_exec(me->files);
+
/*
* Must be called _before_ exec_mmap() as bprm->mm is
* not visible until then. Doing it here also ensures
@@ -1199,14 +1213,6 @@ int begin_new_exec(struct linux_binprm * bprm)
clear_syscall_work_syscall_user_dispatch(me);
- /*
- * We have to apply CLOEXEC before we change whether the process is
- * dumpable (in setup_new_exec) to avoid a race with a process in userspace
- * trying to access the should-be-closed file descriptors of a process
- * undergoing exec(2).
- */
- do_close_on_exec(me->files);
-
if (bprm->secureexec) {
/* Make sure parent cannot signal privileged process. */
me->pdeath_signal = 0;
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index bc2246abd421..b3c763a98cb8 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -4271,7 +4271,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, struct f2fs_gc_control *gc_control);
void f2fs_build_gc_manager(struct f2fs_sb_info *sbi);
int f2fs_gc_range(struct f2fs_sb_info *sbi,
unsigned int start_seg, unsigned int end_seg,
- bool dry_run, unsigned int dry_run_sections);
+ bool dry_run, unsigned int dry_run_sections, bool lock);
int f2fs_resize_fs(struct file *filp, __u64 block_count);
int __init f2fs_create_garbage_collection_cache(void);
void f2fs_destroy_garbage_collection_cache(void);
diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c
index c771adedff39..57ef3876d7af 100644
--- a/fs/f2fs/gc.c
+++ b/fs/f2fs/gc.c
@@ -2150,8 +2150,9 @@ void f2fs_build_gc_manager(struct f2fs_sb_info *sbi)
int f2fs_gc_range(struct f2fs_sb_info *sbi,
unsigned int start_seg, unsigned int end_seg,
- bool dry_run, unsigned int dry_run_sections)
+ bool dry_run, unsigned int dry_run_sections, bool lock)
{
+ struct f2fs_lock_context lc;
unsigned int segno;
unsigned int gc_secs = dry_run_sections;
@@ -2164,28 +2165,46 @@ int f2fs_gc_range(struct f2fs_sb_info *sbi,
.ilist = LIST_HEAD_INIT(gc_list.ilist),
.iroot = RADIX_TREE_INIT(gc_list.iroot, GFP_NOFS),
};
+ int err = 0;
+
+ if (lock)
+ f2fs_down_write_trace(&sbi->gc_lock, &lc);
/*
* avoid migrating empty section, as it can be allocated by
* log in parallel.
*/
if (!get_valid_blocks(sbi, segno, true))
- continue;
+ goto next;
if (is_cursec(sbi, GET_SEC_FROM_SEG(sbi, segno)))
- continue;
+ goto next;
do_garbage_collect(sbi, segno, &gc_list, FG_GC, true, false);
put_gc_inode(&gc_list);
- if (!dry_run && get_valid_blocks(sbi, segno, true))
- return -EAGAIN;
+ /* reset all pinned status during fggc */
+ f2fs_unpin_all_sections(sbi, true);
+
+ if (!dry_run && get_valid_blocks(sbi, segno, true)) {
+ err = -EAGAIN;
+ goto next;
+ }
if (dry_run && dry_run_sections &&
- !get_valid_blocks(sbi, segno, true) && --gc_secs == 0)
- break;
+ !get_valid_blocks(sbi, segno, true)) {
+ --gc_secs;
+ goto next;
+ }
if (fatal_signal_pending(current))
- return -ERESTARTSYS;
+ err = -ERESTARTSYS;
+next:
+ if (lock)
+ f2fs_up_write_trace(&sbi->gc_lock, &lc);
+ if (err)
+ return err;
+ if (dry_run && dry_run_sections && !gc_secs)
+ return 0;
}
return 0;
@@ -2194,8 +2213,9 @@ int f2fs_gc_range(struct f2fs_sb_info *sbi,
static int free_segment_range(struct f2fs_sb_info *sbi,
unsigned int secs, bool dry_run)
{
- unsigned int next_inuse, start, end;
+ unsigned int secno, next_inuse, start, end, end_secno;
struct cp_control cpc = { CP_RESIZE, 0, 0, 0 };
+ unsigned int freed_secs = 0;
int gc_mode, gc_type;
int err = 0;
int type;
@@ -2204,6 +2224,7 @@ static int free_segment_range(struct f2fs_sb_info *sbi,
MAIN_SECS(sbi) -= secs;
start = MAIN_SECS(sbi) * SEGS_PER_SEC(sbi);
end = MAIN_SEGS(sbi) - 1;
+ end_secno = GET_SEC_FROM_SEG(sbi, end);
mutex_lock(&DIRTY_I(sbi)->seglist_lock);
for (gc_mode = 0; gc_mode < MAX_GC_POLICY; gc_mode++)
@@ -2215,6 +2236,14 @@ static int free_segment_range(struct f2fs_sb_info *sbi,
sbi->next_victim_seg[gc_type] = NULL_SEGNO;
mutex_unlock(&DIRTY_I(sbi)->seglist_lock);
+ spin_lock(&FREE_I(sbi)->segmap_lock);
+ for (secno = MAIN_SECS(sbi); secno <= end_secno; secno++) {
+ if (!test_bit(secno, FREE_I(sbi)->free_secmap))
+ freed_secs++;
+ }
+ FREE_I(sbi)->free_sections -= freed_secs;
+ spin_unlock(&FREE_I(sbi)->segmap_lock);
+
/* Move out cursegs from the target range */
for (type = CURSEG_HOT_DATA; type < NR_CURSEG_TYPE; type++) {
err = f2fs_allocate_segment_for_resize(sbi, type, start, end);
@@ -2223,7 +2252,7 @@ static int free_segment_range(struct f2fs_sb_info *sbi,
}
/* do GC to move out valid blocks in the range */
- err = f2fs_gc_range(sbi, start, end, dry_run, 0);
+ err = f2fs_gc_range(sbi, start, end, dry_run, 0, false);
if (err || dry_run)
goto out;
@@ -2239,6 +2268,9 @@ static int free_segment_range(struct f2fs_sb_info *sbi,
f2fs_bug_on(sbi, 1);
}
out:
+ spin_lock(&FREE_I(sbi)->segmap_lock);
+ FREE_I(sbi)->free_sections += freed_secs;
+ spin_unlock(&FREE_I(sbi)->segmap_lock);
MAIN_SECS(sbi) += secs;
return err;
}
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 8830e1892b43..011f8172e923 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -3357,10 +3357,8 @@ int f2fs_allocate_pinning_section(struct f2fs_sb_info *sbi)
f2fs_unlock_op(sbi, &lc);
if (f2fs_sb_has_blkzoned(sbi) && err == -EAGAIN && gc_required) {
- f2fs_down_write_trace(&sbi->gc_lock, &lc);
err = f2fs_gc_range(sbi, 0, sbi->first_seq_zone_segno - 1,
- true, ZONED_PIN_SEC_REQUIRED_COUNT);
- f2fs_up_write_trace(&sbi->gc_lock, &lc);
+ true, ZONED_PIN_SEC_REQUIRED_COUNT, true);
if (err)
return err;
err = f2fs_sync_fs(sbi->sb, 1);
diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index 7fdfa4f27e34..424df70a5c30 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -54,6 +54,42 @@ static void netfs_rreq_expand(struct netfs_io_request *rreq,
}
}
+/*
+ * Drop the folio refs acquired from the readahead API.
+ */
+static void netfs_bulk_drop_ra_refs(struct netfs_io_request *rreq)
+{
+ struct folio_batch fbatch;
+ struct folio *folio;
+ pgoff_t nr_pages = DIV_ROUND_UP(rreq->len, PAGE_SIZE);
+ pgoff_t first = rreq->start / PAGE_SIZE;
+ XA_STATE(xas, &rreq->mapping->i_pages, first);
+
+ folio_batch_init(&fbatch);
+
+ rcu_read_lock();
+
+ xas_for_each(&xas, folio, first + nr_pages - 1) {
+ if (xas_retry(&xas, folio))
+ continue;
+
+ if (!folio_batch_add(&fbatch, folio))
+ folio_batch_release(&fbatch);
+ }
+
+ rcu_read_unlock();
+ folio_batch_release(&fbatch);
+ trace_netfs_rreq(rreq, netfs_rreq_trace_ra_put_ref);
+ clear_bit_unlock(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags);
+ wake_up(&rreq->waitq);
+}
+
+static void netfs_maybe_bulk_drop_ra_refs(struct netfs_io_request *rreq)
+{
+ if (test_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags))
+ netfs_bulk_drop_ra_refs(rreq);
+}
+
/*
* Begin an operation, and fetch the stored zero point value from the cookie if
* available.
@@ -74,12 +110,8 @@ static int netfs_begin_cache_read(struct netfs_io_request *rreq, struct netfs_in
*
* Returns the limited size if successful and -ENOMEM if insufficient memory
* available.
- *
- * [!] NOTE: This must be run in the same thread as ->issue_read() was called
- * in as we access the readahead_control struct.
*/
-static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq,
- struct readahead_control *ractl)
+static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq)
{
struct netfs_io_request *rreq = subreq->rreq;
size_t rsize = subreq->len;
@@ -87,30 +119,6 @@ static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq,
if (subreq->source == NETFS_DOWNLOAD_FROM_SERVER)
rsize = umin(rsize, rreq->io_streams[0].sreq_max_len);
- if (ractl) {
- /* If we don't have sufficient folios in the rolling buffer,
- * extract a folioq's worth from the readahead region at a time
- * into the buffer. Note that this acquires a ref on each page
- * that we will need to release later - but we don't want to do
- * that until after we've started the I/O.
- */
- struct folio_batch put_batch;
-
- folio_batch_init(&put_batch);
- while (rreq->submitted < subreq->start + rsize) {
- ssize_t added;
-
- added = rolling_buffer_load_from_ra(&rreq->buffer, ractl,
- &put_batch);
- if (added < 0) {
- folio_batch_release(&put_batch);
- return added;
- }
- rreq->submitted += added;
- }
- folio_batch_release(&put_batch);
- }
-
subreq->len = rsize;
if (unlikely(rreq->io_streams[0].sreq_max_segs)) {
size_t limit = netfs_limit_iter(&rreq->buffer.iter, 0, rsize,
@@ -203,17 +211,68 @@ static void netfs_issue_read(struct netfs_io_request *rreq,
}
}
+/*
+ * Mark folios that we want to copy to the cache. For filesystems that use
+ * netfslib fully, we set folio->private to NETFS_FOLIO_COPY_TO_CACHE;
+ * otherwise we set the deprecated PG_private_2.
+ */
+static void netfs_mark_copy_to_cache(struct netfs_io_request *rreq,
+ struct folio_queue **fq,
+ unsigned int *offset,
+ int *slot,
+ size_t len,
+ bool copy)
+{
+ while (len > 0) {
+ struct folio *folio;
+ size_t fsize, overlap;
+
+ if (!*fq)
+ break;
+ if (*slot >= folioq_count(*fq)) {
+ *fq = (*fq)->next;
+ *slot = 0;
+ *offset = 0;
+ continue;
+ }
+
+ /* Determine how much the subreq overlaps the folio, if at all. */
+ fsize = folioq_folio_size(*fq, *slot);
+ overlap = min(len, fsize - *offset);
+
+ if (overlap > 0 && copy) {
+ folio = folioq_folio(*fq, *slot);
+ if (unlikely(test_bit(NETFS_RREQ_USE_PGPRIV2, &rreq->flags))) {
+ if (!folio_test_private_2(folio))
+ folio_start_private_2(folio);
+ } else {
+ if (!folio_get_private(folio))
+ folio_attach_private(folio, NETFS_FOLIO_COPY_TO_CACHE);
+ }
+ trace_netfs_folio(folio, netfs_folio_trace_mark_copy);
+ }
+
+ len -= overlap;
+ *offset += overlap;
+ if (*offset >= fsize) {
+ *slot += 1;
+ *offset = 0;
+ }
+ }
+}
+
/*
* Perform a read to the pagecache from a series of sources of different types,
* slicing up the region to be read according to available cache blocks and
* network rsize.
*/
-static void netfs_read_to_pagecache(struct netfs_io_request *rreq,
- struct readahead_control *ractl)
+static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
{
+ struct folio_queue *fq = rreq->buffer.tail;
unsigned long long start = rreq->start;
+ unsigned int offset = 0;
ssize_t size = rreq->len;
- int ret = 0;
+ int ret = 0, slot = 0;
do {
struct netfs_io_subrequest *subreq;
@@ -288,7 +347,7 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq,
break;
issue:
- slice = netfs_prepare_read_iterator(subreq, ractl);
+ slice = netfs_prepare_read_iterator(subreq);
if (slice < 0) {
ret = slice;
netfs_cancel_read(subreq, ret);
@@ -301,7 +360,15 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq,
set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
}
+ if (fq) {
+ /* See if the cache indicated this should be cached. */
+ bool copy = test_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
+
+ netfs_mark_copy_to_cache(rreq, &fq, &slot, &offset, slice, copy);
+ }
+
netfs_issue_read(rreq, subreq);
+ netfs_maybe_bulk_drop_ra_refs(rreq);
if (test_bit(NETFS_RREQ_PAUSE, &rreq->flags))
netfs_wait_for_paused_read(rreq);
@@ -339,7 +406,8 @@ void netfs_readahead(struct readahead_control *ractl)
{
struct netfs_io_request *rreq;
struct netfs_inode *ictx = netfs_inode(ractl->mapping->host);
- unsigned long long start = readahead_pos(ractl);
+ ssize_t added;
+ uoff_t start = readahead_pos(ractl);
size_t size = readahead_length(ractl);
int ret;
@@ -360,11 +428,24 @@ void netfs_readahead(struct readahead_control *ractl)
netfs_rreq_expand(rreq, ractl);
- rreq->submitted = rreq->start;
- if (rolling_buffer_init(&rreq->buffer, rreq->debug_id, ITER_DEST, rreq->gfp) < 0)
+ /* Load the folios to be read into a bvecq chain. Note that this
+ * acquires a ref on each folio that we will need to release later -
+ * but we don't want to do that until after we've started the I/O.
+ */
+ added = rolling_buffer_bulk_load_from_ra(&rreq->buffer, ractl,
+ rreq->debug_id, rreq->gfp);
+ if (added < 0) {
+ ret = added;
goto cleanup_free;
- netfs_read_to_pagecache(rreq, ractl);
+ }
+ __set_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags);
+
+ rreq->submitted = rreq->start + added;
+ rreq->cleaned_to = rreq->start;
+ netfs_read_set_unlock_at(rreq);
+ netfs_read_to_pagecache(rreq);
+ netfs_maybe_bulk_drop_ra_refs(rreq);
return netfs_put_request(rreq, netfs_rreq_trace_put_return);
cleanup_free:
@@ -387,6 +468,7 @@ static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct fo
if (added < 0)
return added;
rreq->submitted = rreq->start + added;
+ rreq->progress_at = added;
return 0;
}
@@ -457,7 +539,7 @@ static int netfs_read_gaps(struct file *file, struct folio *folio)
iov_iter_bvec(&rreq->buffer.iter, ITER_DEST, bvec, i, rreq->len);
rreq->submitted = rreq->start + flen;
- netfs_read_to_pagecache(rreq, NULL);
+ netfs_read_to_pagecache(rreq);
ret = netfs_wait_for_read(rreq);
if (ret >= 0) {
@@ -532,7 +614,7 @@ int netfs_read_folio(struct file *file, struct folio *folio)
if (ret < 0)
goto discard;
- netfs_read_to_pagecache(rreq, NULL);
+ netfs_read_to_pagecache(rreq);
ret = netfs_wait_for_read(rreq);
netfs_put_request(rreq, netfs_rreq_trace_put_return);
return ret < 0 ? ret : 0;
@@ -689,7 +771,7 @@ int netfs_write_begin(struct netfs_inode *ctx,
if (ret < 0)
goto error_put;
- netfs_read_to_pagecache(rreq, NULL);
+ netfs_read_to_pagecache(rreq);
ret = netfs_wait_for_read(rreq);
netfs_put_request(rreq, netfs_rreq_trace_put_return);
if (ret < 0)
@@ -754,7 +836,7 @@ int netfs_prefetch_for_write(struct file *file, struct folio *folio,
if (ret < 0)
goto error_put;
- netfs_read_to_pagecache(rreq, NULL);
+ netfs_read_to_pagecache(rreq);
ret = netfs_wait_for_read(rreq);
netfs_put_request(rreq, netfs_rreq_trace_put_return);
return ret < 0 ? ret : 0;
diff --git a/fs/netfs/direct_write.c b/fs/netfs/direct_write.c
index c16fbad286a1..2361277416c7 100644
--- a/fs/netfs/direct_write.c
+++ b/fs/netfs/direct_write.c
@@ -21,7 +21,7 @@ static void netfs_unbuffered_write_done(struct netfs_io_request *wreq)
/* Okay, declare that all I/O is complete. */
trace_netfs_rreq(wreq, netfs_rreq_trace_write_done);
- if (!wreq->error)
+ if (wreq->transferred)
netfs_update_i_size(ictx, &ictx->inode, wreq->start, wreq->transferred);
if (wreq->origin == NETFS_DIO_WRITE &&
@@ -51,7 +51,7 @@ static void netfs_unbuffered_write_done(struct netfs_io_request *wreq)
wreq->iocb->ki_pos += written;
if (wreq->iocb->ki_complete) {
trace_netfs_rreq(wreq, netfs_rreq_trace_ki_complete);
- wreq->iocb->ki_complete(wreq->iocb, wreq->error ?: written);
+ wreq->iocb->ki_complete(wreq->iocb, written ?: wreq->error);
}
wreq->iocb = VFS_PTR_POISON;
}
@@ -95,7 +95,7 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
{
struct netfs_io_subrequest *subreq = NULL;
struct netfs_io_stream *stream = &wreq->io_streams[0];
- int ret;
+ int ret = 0;
_enter("%llx", wreq->len);
@@ -110,6 +110,11 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
if (!subreq) {
netfs_prepare_write(wreq, stream, wreq->start + wreq->transferred);
subreq = stream->construct;
+ if (!subreq) {
+ wreq->error = -ENOMEM;
+ ret = -ENOMEM;
+ break;
+ }
stream->construct = NULL;
}
@@ -121,8 +126,14 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
}
iov_iter_truncate(&subreq->io_iter, wreq->len - wreq->transferred);
- if (!iov_iter_count(&subreq->io_iter))
+ if (!iov_iter_count(&subreq->io_iter)) {
+ pr_warn("netfs: Unexpected zero-length iterator R=%08x\n",
+ wreq->debug_id);
+ __set_bit(NETFS_SREQ_FAILED, &subreq->flags);
+ netfs_write_subrequest_terminated(subreq, -EIO);
+ wreq->error = -EIO;
break;
+ }
subreq->len = netfs_limit_iter(&subreq->io_iter, 0,
stream->sreq_max_len,
@@ -139,13 +150,11 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
if (test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
retry = true;
} else if (test_bit(NETFS_SREQ_FAILED, &subreq->flags)) {
- ret = subreq->error;
- wreq->error = ret;
+ wreq->error = subreq->error;
netfs_see_subrequest(subreq, netfs_sreq_trace_see_failed);
subreq = NULL;
break;
}
- ret = 0;
if (!retry) {
netfs_unbuffered_write_collect(wreq, stream, subreq);
@@ -288,11 +297,11 @@ ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *
ret = -EIOCBQUEUED;
} else {
ret = netfs_unbuffered_write(wreq);
- if (ret < 0) {
- _debug("begin = %zd", ret);
- } else {
+ if (wreq->transferred) {
iocb->ki_pos += wreq->transferred;
- ret = wreq->transferred ?: wreq->error;
+ ret = wreq->transferred;
+ } else if (wreq->error) {
+ ret = wreq->error;
}
netfs_put_request(wreq, netfs_rreq_trace_put_complete);
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 420ee7b26580..c79c8e69d60c 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -79,6 +79,7 @@ ssize_t netfs_wait_for_read(struct netfs_io_request *rreq);
ssize_t netfs_wait_for_write(struct netfs_io_request *rreq);
void netfs_wait_for_paused_read(struct netfs_io_request *rreq);
void netfs_wait_for_paused_write(struct netfs_io_request *rreq);
+void netfs_wait_for_put_ra_refs(struct netfs_io_request *rreq);
/*
* objects.c
@@ -109,6 +110,8 @@ static inline void netfs_see_subrequest(struct netfs_io_subrequest *subreq,
/*
* read_collect.c
*/
+void netfs_cancel_copy_to_cache(struct netfs_io_request *rreq, struct folio *folio);
+void netfs_read_set_unlock_at(struct netfs_io_request *rreq);
bool netfs_read_collection(struct netfs_io_request *rreq);
void netfs_read_collection_worker(struct work_struct *work);
void netfs_cancel_read(struct netfs_io_subrequest *subreq, int error);
diff --git a/fs/netfs/misc.c b/fs/netfs/misc.c
index 5d554512ed23..f5c1c463f4ff 100644
--- a/fs/netfs/misc.c
+++ b/fs/netfs/misc.c
@@ -563,3 +563,22 @@ void netfs_wait_for_paused_write(struct netfs_io_request *rreq)
{
return netfs_wait_for_pause(rreq, netfs_write_collection);
}
+
+/*
+ * Wait for the readahead-acquired refs to be put.
+ */
+void netfs_wait_for_put_ra_refs(struct netfs_io_request *rreq)
+{
+ DEFINE_WAIT(myself);
+
+ for (;;) {
+ trace_netfs_rreq(rreq, netfs_rreq_trace_wait_put_ra_refs);
+ prepare_to_wait(&rreq->waitq, &myself, TASK_UNINTERRUPTIBLE);
+ if (!test_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags))
+ break;
+ schedule();
+ }
+
+ trace_netfs_rreq(rreq, netfs_rreq_trace_waited_put_ra_refs);
+ finish_wait(&rreq->waitq, &myself);
+}
diff --git a/fs/netfs/objects.c b/fs/netfs/objects.c
index 01461a74642d..7f6a3e912602 100644
--- a/fs/netfs/objects.c
+++ b/fs/netfs/objects.c
@@ -41,24 +41,32 @@ struct netfs_io_request *netfs_alloc_request(struct address_space *mapping,
memset(rreq, 0, kmem_cache_size(cache));
INIT_WORK(&rreq->cleanup_work, netfs_free_request);
- rreq->gfp = gfp;
- rreq->start = start;
- rreq->len = len;
- rreq->origin = origin;
- rreq->netfs_ops = ctx->ops;
- rreq->mapping = mapping;
- rreq->inode = inode;
- rreq->i_size = i_size_read(inode);
- rreq->debug_id = atomic_inc_return(&debug_ids);
- rreq->wsize = INT_MAX;
+ rreq->gfp = gfp;
+ rreq->start = start;
+ rreq->collected_to = start;
+ rreq->cleaned_to = start;
+ rreq->len = len;
+ rreq->progress_at = 0;
+ rreq->origin = origin;
+ rreq->netfs_ops = ctx->ops;
+ rreq->mapping = mapping;
+ rreq->inode = inode;
+ rreq->i_size = i_size_read(inode);
+ rreq->debug_id = atomic_inc_return(&debug_ids);
+ rreq->wsize = INT_MAX;
rreq->io_streams[0].sreq_max_len = ULONG_MAX;
rreq->io_streams[0].sreq_max_segs = 0;
spin_lock_init(&rreq->lock);
- INIT_LIST_HEAD(&rreq->io_streams[0].subrequests);
- INIT_LIST_HEAD(&rreq->io_streams[1].subrequests);
init_waitqueue_head(&rreq->waitq);
refcount_set(&rreq->ref, 2);
+ for (int s = 0; s < NR_IO_STREAMS; s++) {
+ struct netfs_io_stream *stream = &rreq->io_streams[s];
+
+ INIT_LIST_HEAD(&stream->subrequests);
+ stream->collected_to = rreq->start;
+ }
+
if (origin == NETFS_READAHEAD ||
origin == NETFS_READPAGE ||
origin == NETFS_READ_GAPS ||
diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index 23660a590124..5cf22087d243 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -19,7 +19,6 @@
#define MADE_PROGRESS 0x04 /* Made progress cleaning up a stream or the folio set */
#define BUFFERED 0x08 /* The pagecache needs cleaning up */
#define NEED_RETRY 0x10 /* A front op requests retrying */
-#define COPY_TO_CACHE 0x40 /* Need to copy subrequest to cache */
#define ABANDON_SREQ 0x80 /* Need to abandon untransferred part of subrequest */
/*
@@ -34,6 +33,30 @@ static void netfs_clear_unread(struct netfs_io_subrequest *subreq)
__set_bit(NETFS_SREQ_HIT_EOF, &subreq->flags);
}
+/*
+ * Cancel the copy-to-cache mark on a folio.
+ */
+void netfs_cancel_copy_to_cache(struct netfs_io_request *rreq, struct folio *folio)
+{
+ if (!test_bit(NETFS_RREQ_USE_PGPRIV2, &rreq->flags)) {
+ if (folio_get_private(folio) == NETFS_FOLIO_COPY_TO_CACHE) {
+ folio_detach_private(folio);
+ trace_netfs_folio(folio, netfs_folio_trace_cancel_copy);
+ } else if (netfs_folio_group(folio) == NETFS_FOLIO_COPY_TO_CACHE) {
+ struct netfs_folio *finfo = netfs_folio_info(folio);
+
+ finfo->netfs_group = NULL;
+ trace_netfs_folio(folio, netfs_folio_trace_cancel_copy);
+ }
+ } else {
+ // TODO: Use of PG_private_2 is deprecated.
+ if (folio_test_private_2(folio)) {
+ folio_end_private_2(folio);
+ trace_netfs_folio(folio, netfs_folio_trace_cancel_copy);
+ }
+ }
+}
+
/*
* Flush, mark and unlock a folio that's now completely read. If we want to
* cache the folio, we set the group to NETFS_FOLIO_COPY_TO_CACHE, mark it
@@ -48,37 +71,37 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
if (unlikely(folio_pos(folio) < rreq->abandon_to)) {
trace_netfs_folio(folio, netfs_folio_trace_abandon);
+ netfs_cancel_copy_to_cache(rreq, folio);
goto just_unlock;
}
flush_dcache_folio(folio);
folio_mark_uptodate(folio);
- if (!test_bit(NETFS_RREQ_USE_PGPRIV2, &rreq->flags)) {
- finfo = netfs_folio_info(folio);
- if (finfo) {
- trace_netfs_folio(folio, netfs_folio_trace_filled_gaps);
- if (finfo->netfs_group)
- folio_change_private(folio, finfo->netfs_group);
- else
- folio_detach_private(folio);
- kfree(finfo);
- }
+ if (unlikely(test_bit(NETFS_RREQ_CANCEL_CACHING, &rreq->flags)))
+ netfs_cancel_copy_to_cache(rreq, folio);
- if (test_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags)) {
- if (!WARN_ON_ONCE(folio_get_private(folio) != NULL)) {
- trace_netfs_folio(folio, netfs_folio_trace_copy_to_cache);
- folio_attach_private(folio, NETFS_FOLIO_COPY_TO_CACHE);
- folio_mark_dirty(folio);
- }
+ if (!test_bit(NETFS_RREQ_USE_PGPRIV2, &rreq->flags)) {
+ if (netfs_folio_group(folio) == NETFS_FOLIO_COPY_TO_CACHE) {
+ trace_netfs_folio(folio, netfs_folio_trace_sched_copy);
+ folio_mark_dirty(folio);
} else {
+ finfo = netfs_folio_info(folio);
+ if (finfo) {
+ trace_netfs_folio(folio, netfs_folio_trace_filled_gaps);
+ if (finfo->netfs_group)
+ folio_change_private(folio, finfo->netfs_group);
+ else
+ folio_detach_private(folio);
+ kfree(finfo);
+ }
trace_netfs_folio(folio, netfs_folio_trace_read_done);
}
folioq_clear(folioq, slot);
} else {
// TODO: Use of PG_private_2 is deprecated.
- if (test_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags))
+ if (folio_test_private_2(folio))
netfs_pgpriv2_copy_to_cache(rreq, folio);
}
@@ -94,6 +117,35 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
folioq_clear(folioq, slot);
}
+/*
+ * Determine how much to gather before unlocking more folios.
+ */
+void netfs_read_set_unlock_at(struct netfs_io_request *rreq)
+{
+ struct folio_queue *folioq = rreq->buffer.tail;
+ unsigned int slot = rreq->buffer.first_tail_slot;
+ size_t cleaned_to = rreq->cleaned_to - rreq->start;
+ size_t progress_at = cleaned_to;
+ size_t minimum = 256 * 1024;
+
+ while (progress_at < rreq->len) {
+ if (slot >= folioq_count(folioq)) {
+ folioq = folioq->next;
+ if (!folioq)
+ break;
+ slot = 0;
+ }
+
+ progress_at += folioq_folio_size(folioq, slot);
+ if (progress_at - cleaned_to >= minimum)
+ break;
+ slot++;
+ }
+
+ WRITE_ONCE(rreq->progress_at, progress_at);
+ trace_netfs_read_progress_at(rreq);
+}
+
/*
* Unlock any folios we've finished with.
*/
@@ -112,30 +164,31 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
if (slot >= folioq_nr_slots(folioq)) {
folioq = rolling_buffer_delete_spent(&rreq->buffer);
if (!folioq) {
- rreq->front_folio_order = 0;
+ WRITE_ONCE(rreq->progress_at, rreq->len);
return;
}
slot = 0;
}
+ /* We have to wait for readahead refs to have been released before we
+ * can unlock any folios as the ref-dropper walks i_pages and the only
+ * thing preventing these folios from being removed is the folio lock.
+ */
+ if (test_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags))
+ netfs_wait_for_put_ra_refs(rreq);
+
for (;;) {
struct folio *folio;
unsigned long long fpos, fend;
- unsigned int order;
size_t fsize;
- if (*notes & COPY_TO_CACHE)
- set_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags);
-
folio = folioq_folio(folioq, slot);
if (WARN_ONCE(!folio_test_locked(folio),
"R=%08x: folio %lx is not locked\n",
rreq->debug_id, folio->index))
trace_netfs_folio(folio, netfs_folio_trace_not_locked);
- order = folioq_folio_order(folioq, slot);
- rreq->front_folio_order = order;
- fsize = PAGE_SIZE << order;
+ fsize = folioq_folio_size(folioq, slot);
fpos = folio_pos(folio);
fend = fpos + fsize;
@@ -149,8 +202,6 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
WRITE_ONCE(rreq->cleaned_to, fpos + fsize);
*notes |= MADE_PROGRESS;
- clear_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags);
-
/* Clean up the head folioq. If we clear an entire folioq, then
* we can get rid of it provided it's not also the tail folioq
* being filled by the issuer.
@@ -172,6 +223,8 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
rreq->buffer.tail = folioq;
done:
rreq->buffer.first_tail_slot = slot;
+
+ netfs_read_set_unlock_at(rreq);
}
/*
@@ -232,7 +285,7 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
* subreqs.
*/
if (notes & BUFFERED) {
- size_t fsize = PAGE_SIZE << rreq->front_folio_order;
+ uoff_t unlock_at = rreq->start + rreq->progress_at;
/* Clear the tail of a short read. */
if (!(notes & HIT_PENDING) &&
@@ -248,16 +301,13 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
stream->collected_to = front->start + transferred;
rreq->collected_to = stream->collected_to;
- if (test_bit(NETFS_SREQ_COPY_TO_CACHE, &front->flags))
- notes |= COPY_TO_CACHE;
-
if (test_bit(NETFS_SREQ_FAILED, &front->flags)) {
rreq->abandon_to = front->start + front->len;
front->transferred = front->len;
transferred = front->len;
trace_netfs_rreq(rreq, netfs_rreq_trace_set_abandon);
}
- if (front->start + transferred >= rreq->cleaned_to + fsize ||
+ if (front->start + transferred >= unlock_at ||
test_bit(NETFS_SREQ_HIT_EOF, &front->flags))
netfs_read_unlock_folios(rreq, ¬es);
} else {
@@ -477,20 +527,22 @@ void netfs_read_collection_worker(struct work_struct *work)
void netfs_read_subreq_progress(struct netfs_io_subrequest *subreq)
{
struct netfs_io_request *rreq = subreq->rreq;
- struct netfs_io_stream *stream = &rreq->io_streams[0];
- size_t fsize = PAGE_SIZE << rreq->front_folio_order;
-
- trace_netfs_sreq(subreq, netfs_sreq_trace_progress);
+ struct netfs_io_stream *stream = &rreq->io_streams[subreq->stream_nr];
+ size_t progress_at = READ_ONCE(rreq->progress_at);
+ uoff_t update_at = rreq->start + progress_at;
+ uoff_t transferred_to = subreq->start + subreq->transferred;
/* If we are at the head of the queue, wake up the collector,
* getting a ref to it if we were the ones to do so.
*/
- if (subreq->start + subreq->transferred > rreq->cleaned_to + fsize &&
+ if (progress_at < rreq->len &&
+ transferred_to >= update_at &&
(rreq->origin == NETFS_READAHEAD ||
rreq->origin == NETFS_READPAGE ||
rreq->origin == NETFS_READ_FOR_WRITE) &&
list_is_first(&subreq->rreq_link, &stream->subrequests)
) {
+ trace_netfs_sreq(subreq, netfs_sreq_trace_progress);
__set_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
netfs_wake_collector(rreq);
}
diff --git a/fs/netfs/read_pgpriv2.c b/fs/netfs/read_pgpriv2.c
index c31190993b76..a4b7bb88cbdb 100644
--- a/fs/netfs/read_pgpriv2.c
+++ b/fs/netfs/read_pgpriv2.c
@@ -54,8 +54,8 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
/* Attach the folio to the rolling buffer. */
if (rolling_buffer_append(&creq->buffer, folio, 0, creq->gfp) < 0) {
+ set_bit(NETFS_RREQ_CANCEL_CACHING, &creq->flags);
folio_end_private_2(folio);
- clear_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &creq->flags);
return;
}
@@ -122,13 +122,14 @@ static struct netfs_io_request *netfs_pgpriv2_begin_copy_to_cache(
netfs_put_failed_request(creq);
cancel:
rreq->copy_to_cache = ERR_PTR(-ENOBUFS);
- clear_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags);
+ set_bit(NETFS_RREQ_CANCEL_CACHING, &rreq->flags);
return ERR_PTR(-ENOBUFS);
}
/*
* [DEPRECATED] Mark page as requiring copy-to-cache using PG_private_2 and add
- * it to the copy write request.
+ * it to the copy write request. PG_private_2 should already be set on the
+ * folio.
*/
void netfs_pgpriv2_copy_to_cache(struct netfs_io_request *rreq, struct folio *folio)
{
@@ -136,11 +137,13 @@ void netfs_pgpriv2_copy_to_cache(struct netfs_io_request *rreq, struct folio *fo
if (!creq)
creq = netfs_pgpriv2_begin_copy_to_cache(rreq, folio);
- if (IS_ERR(creq))
+ if (IS_ERR(creq)) {
+ set_bit(NETFS_RREQ_CANCEL_CACHING, &rreq->flags);
+ netfs_cancel_copy_to_cache(rreq, folio);
return;
+ }
- trace_netfs_folio(folio, netfs_folio_trace_copy_to_cache);
- folio_start_private_2(folio);
+ trace_netfs_folio(folio, netfs_folio_trace_pgpriv2_copy);
netfs_pgpriv2_copy_folio(creq, folio);
}
diff --git a/fs/netfs/read_retry.c b/fs/netfs/read_retry.c
index 2b42758e01ec..4f6a36c6e214 100644
--- a/fs/netfs/read_retry.c
+++ b/fs/netfs/read_retry.c
@@ -292,11 +292,22 @@ void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq)
{
struct folio_queue *p;
+ /* We have to wait for readahead refs to have been released before we
+ * can unlock any folios as the ref-dropper walks i_pages and the only
+ * thing preventing these folios from being removed is the folio lock.
+ */
+ if (test_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags))
+ netfs_wait_for_put_ra_refs(rreq);
+
for (p = rreq->buffer.tail; p; p = p->next) {
for (int slot = 0; slot < folioq_count(p); slot++) {
struct folio *folio = folioq_folio(p, slot);
- if (folio && !folioq_is_marked2(p, slot)) {
+ if (!folio)
+ continue;
+ netfs_cancel_copy_to_cache(rreq, folio);
+
+ if (!folioq_is_marked2(p, slot)) {
if (folio == rreq->no_unlock_folio &&
test_bit(NETFS_RREQ_NO_UNLOCK_FOLIO,
&rreq->flags)) {
diff --git a/fs/netfs/read_single.c b/fs/netfs/read_single.c
index 8833550d2eb6..de67ac41548d 100644
--- a/fs/netfs/read_single.c
+++ b/fs/netfs/read_single.c
@@ -170,6 +170,8 @@ ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_ite
if (IS_ERR(rreq))
return PTR_ERR(rreq);
+ rreq->progress_at = rreq->len;
+
ret = netfs_single_begin_cache_read(rreq, ictx);
if (ret == -ENOMEM || ret == -EINTR || ret == -ERESTARTSYS)
goto cleanup_free;
diff --git a/fs/netfs/rolling_buffer.c b/fs/netfs/rolling_buffer.c
index 8c0026836f9c..424e77a9a109 100644
--- a/fs/netfs/rolling_buffer.c
+++ b/fs/netfs/rolling_buffer.c
@@ -115,42 +115,65 @@ int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp)
}
/*
- * Decant the list of folios to read into a rolling buffer.
+ * Decant the entire list of folios to read into a rolling buffer.
*/
-ssize_t rolling_buffer_load_from_ra(struct rolling_buffer *roll,
- struct readahead_control *ractl,
- struct folio_batch *put_batch)
+ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
+ struct readahead_control *ractl,
+ unsigned int rreq_id, gfp_t gfp)
{
struct folio_queue *fq;
- struct page **vec;
- int nr, ix, to;
- ssize_t size = 0;
+ ssize_t loaded = 0;
- if (rolling_buffer_make_space(roll, GFP_KERNEL) < 0)
- return -ENOMEM;
+ while (ractl->_nr_pages - ractl->_batch_count > 0) {
+ unsigned int nr;
- fq = roll->head;
- vec = (struct page **)fq->vec.folios;
- nr = __readahead_batch(ractl, vec + folio_batch_count(&fq->vec),
- folio_batch_space(&fq->vec));
- ix = fq->vec.nr;
- to = ix + nr;
- fq->vec.nr = to;
- for (; ix < to; ix++) {
- struct folio *folio = folioq_folio(fq, ix);
- unsigned int order = folio_order(folio);
-
- fq->orders[ix] = order;
- size += PAGE_SIZE << order;
- trace_netfs_folio(folio, netfs_folio_trace_read);
- if (!folio_batch_add(put_batch, folio))
- folio_batch_release(put_batch);
+ /* Allocate a folioq to put some folios into and attach it to
+ * the rolling buffer.
+ */
+ fq = netfs_folioq_alloc(rreq_id, gfp,
+ netfs_trace_folioq_make_space);
+ if (!fq)
+ goto nomem_unlock;
+ fq->prev = roll->head;
+ if (!roll->tail)
+ roll->tail = fq;
+ else
+ roll->head->next = fq;
+ roll->head = fq;
+
+ /* Get a batch of folios and note their orders. */
+ nr = __readahead_batch(ractl, (struct page **)fq->vec.folios,
+ folioq_nr_slots(fq));
+ if (WARN_ON_ONCE(!nr))
+ break;
+ fq->vec.nr = nr;
+
+ for (int slot = 0; slot < nr; slot++) {
+ struct folio *folio = folioq_folio(fq, slot);
+ unsigned int order;
+
+ order = folio_order(folio);
+ fq->orders[slot] = order;
+ loaded += PAGE_SIZE << order;
+ trace_netfs_folio(folio, netfs_folio_trace_read);
+ }
}
- WRITE_ONCE(roll->iter.count, roll->iter.count + size);
- /* Store the counter after setting the slot. */
- smp_store_release(&roll->next_head_slot, to);
- return size;
+ WRITE_ONCE(roll->iter.count, loaded);
+ iov_iter_folio_queue(&roll->iter, ITER_DEST, roll->tail, 0, 0, loaded);
+ return loaded;
+
+nomem_unlock:
+ for (fq = roll->tail; fq; fq = fq->next) {
+ for (int slot = 0; slot < folioq_count(fq); slot++) {
+ folio_unlock(fq->vec.folios[slot]);
+ folioq_mark(fq, slot);
+ }
+ }
+ rolling_buffer_clear(roll);
+ roll->head = NULL;
+ roll->tail = NULL;
+ return -ENOMEM;
}
/*
diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c
index 2d9cfcd43658..851f6f93ad45 100644
--- a/fs/netfs/write_issue.c
+++ b/fs/netfs/write_issue.c
@@ -170,6 +170,8 @@ void netfs_prepare_write(struct netfs_io_request *wreq,
rolling_buffer_make_space(&wreq->buffer, wreq->gfp);
subreq = netfs_alloc_subrequest(wreq);
+ if (!subreq)
+ return;
subreq->source = stream->source;
subreq->start = start;
subreq->stream_nr = stream->stream_nr;
diff --git a/fs/ntfs/attrib.c b/fs/ntfs/attrib.c
index b4c6137c037b..918ff4db75f7 100644
--- a/fs/ntfs/attrib.c
+++ b/fs/ntfs/attrib.c
@@ -1732,8 +1732,8 @@ static struct attr_def *ntfs_attr_find_in_attrdef(const struct ntfs_volume *vol,
struct attr_def *ad;
WARN_ON(!type);
- for (ad = vol->attrdef; (u8 *)ad - (u8 *)vol->attrdef <
- vol->attrdef_size && ad->type; ++ad) {
+ for (ad = vol->attrdef; (u8 *)ad - (u8 *)vol->attrdef <=
+ vol->attrdef_size - (s32)sizeof(*ad) && ad->type; ++ad) {
/* We have not found it yet, carry on searching. */
if (likely(le32_to_cpu(ad->type) < le32_to_cpu(type)))
continue;
@@ -2495,7 +2495,7 @@ int ntfs_resident_attr_record_add(struct ntfs_inode *ni, __le32 type,
return offset;
put_err_out:
ntfs_attr_put_search_ctx(ctx);
- return -EIO;
+ return err;
}
/*
@@ -2634,7 +2634,7 @@ static int ntfs_non_resident_attr_record_add(struct ntfs_inode *ni, __le32 type,
return offset;
put_err_out:
ntfs_attr_put_search_ctx(ctx);
- return -1;
+ return err;
}
/*
@@ -5693,7 +5693,7 @@ int ntfs_attr_fallocate(struct ntfs_inode *ni, loff_t start, loff_t byte_len, bo
lcn << vol->cluster_size_bits,
alloc_cnt <<
vol->cluster_size_bits);
- if (err > 0)
+ if (err)
goto out;
}
diff --git a/fs/ntfs/ea.c b/fs/ntfs/ea.c
index 25ff159dbfd3..75a251ec4ebc 100644
--- a/fs/ntfs/ea.c
+++ b/fs/ntfs/ea.c
@@ -390,10 +390,12 @@ static int ntfs_set_ea(struct inode *inode, const char *name, size_t name_len,
*packed_ea_size = p_ea_info->ea_length;
mark_mft_record_dirty(ni);
out:
- if (ea_info_qsize > 0)
- NInoSetHasEA(ni);
- else
- NInoClearHasEA(ni);
+ if (!err) {
+ if (ea_info_qsize > 0)
+ NInoSetHasEA(ni);
+ else
+ NInoClearHasEA(ni);
+ }
kvfree(ea_buf);
kvfree(old_ea_buf);
@@ -591,7 +593,7 @@ static int ntfs_getxattr(const struct xattr_handler *handler,
if (!buffer) {
err = sizeof(u8);
} else if (size < sizeof(u8)) {
- err = -ENODATA;
+ err = -ERANGE;
} else {
err = sizeof(u8);
*(u8 *)buffer = (u8)(le32_to_cpu(ni->flags) & 0x3F);
@@ -604,7 +606,7 @@ static int ntfs_getxattr(const struct xattr_handler *handler,
if (!buffer) {
err = sizeof(u32);
} else if (size < sizeof(u32)) {
- err = -ENODATA;
+ err = -ERANGE;
} else {
err = sizeof(u32);
*(u32 *)buffer = le32_to_cpu(ni->flags);
diff --git a/fs/ntfs/lcnalloc.c b/fs/ntfs/lcnalloc.c
index aa2e017a4384..0d6cd08ee2e7 100644
--- a/fs/ntfs/lcnalloc.c
+++ b/fs/ntfs/lcnalloc.c
@@ -53,10 +53,10 @@ int ntfs_cluster_free_from_rl_nolock(struct ntfs_volume *vol,
if (rl->lcn < 0)
continue;
err = ntfs_bitmap_clear_run(lcnbmp_vi, rl->lcn, rl->length);
- if (unlikely(err && (!ret || ret == -ENOMEM) && ret != err))
- ret = err;
- else
+ if (likely(!err))
nr_freed += rl->length;
+ else if (!ret || ret == -ENOMEM)
+ ret = err;
}
ntfs_inc_free_clusters(vol, nr_freed);
ntfs_debug("Done.");
@@ -1045,8 +1045,9 @@ s64 __ntfs_cluster_free(struct ntfs_inode *ni, const s64 start_vcn, s64 count,
"Failed to rollback (error %i). Leaving inconsistent metadata! Unmount and run chkdsk.",
(int)delta);
NVolSetErrors(vol);
+ } else {
+ ntfs_dec_free_clusters(vol, delta);
}
- ntfs_dec_free_clusters(vol, delta);
up_write(&vol->lcnbmp_lock);
memalloc_nofs_restore(memalloc_flags);
ntfs_error(vol->sb, "Aborting (error %i).", err);
diff --git a/fs/ntfs/mft.c b/fs/ntfs/mft.c
index 271a26549128..3ad739179c7a 100644
--- a/fs/ntfs/mft.c
+++ b/fs/ntfs/mft.c
@@ -580,7 +580,7 @@ int write_mft_record_nolock(struct ntfs_inode *ni, struct mft_record *m, int syn
err = pre_write_mst_fixup((struct ntfs_record *)fixup_m, vol->mft_record_size);
if (err) {
ntfs_error(vol->sb, "Failed to apply mst fixups!");
- goto err_out;
+ goto unmap_err_out;
}
folio_size = vol->mft_record_size / ni->mft_lcn_count;
@@ -645,6 +645,8 @@ int write_mft_record_nolock(struct ntfs_inode *ni, struct mft_record *m, int syn
return 0;
put_bio_out:
bio_put(bio);
+unmap_err_out:
+ kunmap_local(kaddr);
err_out:
/*
* The caller should mark the base inode as bad so no more I/O
diff --git a/fs/ntfs/reparse.c b/fs/ntfs/reparse.c
index fa523dc3691e..f8b88a110cb6 100644
--- a/fs/ntfs/reparse.c
+++ b/fs/ntfs/reparse.c
@@ -332,7 +332,7 @@ unsigned int ntfs_reparse_tag_dt_types(struct ntfs_volume *vol, unsigned long mr
vi = ntfs_iget(vol->sb, mref);
if (IS_ERR(vi))
- return PTR_ERR(vi);
+ return DT_UNKNOWN;
reparse_attr = (struct reparse_point *)ntfs_attr_readall(NTFS_I(vi),
AT_REPARSE_POINT, NULL, 0, &attr_size);
@@ -620,8 +620,9 @@ static int update_reparse_data(struct ntfs_inode *ni, struct ntfs_index_context
goto put_rp_inode;
}
- if (set_reparse_index(ni, xr, ((const struct reparse_point *)value)->reparse_tag) &&
- oldsize > 0) {
+ err = set_reparse_index(ni, xr,
+ ((const struct reparse_point *)value)->reparse_tag);
+ if (err && oldsize > 0) {
/*
* If cannot index, try to remove the reparse
* data and log the error. There will be an
diff --git a/fs/ntfs/super.c b/fs/ntfs/super.c
index cd8fa2c13370..63aa83ff77f5 100644
--- a/fs/ntfs/super.c
+++ b/fs/ntfs/super.c
@@ -557,8 +557,8 @@ static bool is_boot_sector_ntfs(const struct super_block *sb,
* Check sectors per cluster value is valid and the cluster size
* is not above the maximum (2MB).
*/
- if (b->bpb.sectors_per_cluster > 0x80 &&
- b->bpb.sectors_per_cluster < 0xf4)
+ if (b->bpb.sectors_per_cluster < 0xf4 &&
+ !is_power_of_2(b->bpb.sectors_per_cluster))
goto not_ntfs;
/* Check reserved/unused fields are really zero. */
@@ -1241,9 +1241,9 @@ static bool load_and_init_attrdef(struct ntfs_volume *vol)
goto failed;
}
NInoSetSparseDisabled(NTFS_I(ino));
- /* The size of FILE_AttrDef must be above 0 and fit inside 31 bits. */
+ /* FILE_AttrDef must hold at least one entry and fit inside 31 bits. */
i_size = i_size_read(ino);
- if (i_size <= 0 || i_size > 0x7fffffff)
+ if (i_size < (s64)sizeof(struct attr_def) || i_size > 0x7fffffff)
goto iput_failed;
vol->attrdef = kvzalloc(i_size, GFP_NOFS);
if (!vol->attrdef)
@@ -1862,7 +1862,8 @@ static int ntfs_sync_fs(struct super_block *sb, int wait)
return 0;
/* If there are some dirty buffers in the bdev inode */
- if (ntfs_clear_volume_flags(vol, VOLUME_IS_DIRTY)) {
+ if (!NVolErrors(vol) &&
+ ntfs_clear_volume_flags(vol, VOLUME_IS_DIRTY)) {
ntfs_warning(sb, "Failed to clear dirty bit in volume information flags. Run chkdsk.");
err = -EIO;
}
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 60b808b85fc4..43eb0c053dac 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -1543,7 +1543,7 @@ int ovl_fill_super(struct super_block *sb, struct fs_context *fc)
struct ovl_fs *ofs = sb->s_fs_info;
int err;
- err = -EIO;
+ err = -EINVAL;
/* The fscontext fd may have been passed to another user namespace. */
if (fc->user_ns != current_user_ns())
goto out_err;
diff --git a/fs/smb/client/cifsacl.c b/fs/smb/client/cifsacl.c
index 12005f46307d..c5e47a835f99 100644
--- a/fs/smb/client/cifsacl.c
+++ b/fs/smb/client/cifsacl.c
@@ -100,8 +100,23 @@ cifs_idmap_key_destroy(struct key *key)
kfree(key->payload.data[0]);
}
+static int
+cifs_idmap_key_vet_description(const char *description)
+{
+ /*
+ * cifs.idmap descriptions are authority-bearing inputs to the
+ * cifs.idmap upcall helper. Only allow the kernel to create this
+ * type of key using the private root_cred installed in
+ * init_cifs_idmap; reject userspace request_key(2)/add_key(2).
+ */
+ if (current_cred() != root_cred)
+ return -EPERM;
+ return 0;
+}
+
static struct key_type cifs_idmap_key_type = {
.name = "cifs.idmap",
+ .vet_description = cifs_idmap_key_vet_description,
.instantiate = cifs_idmap_key_instantiate,
.destroy = cifs_idmap_key_destroy,
.describe = user_describe,
@@ -1081,13 +1096,13 @@ unsigned int setup_special_user_owner_ACE(struct smb_ace *pntace)
static void populate_new_aces(char *nacl_base,
struct smb_sid *pownersid,
struct smb_sid *pgrpsid,
- __u64 *pnmode, u16 *pnum_aces, u16 *pnsize,
+ __u64 *pnmode, u16 *pnum_aces, u32 *pnsize,
bool modefromsid,
bool posix)
{
__u64 nmode;
u16 num_aces = 0;
- u16 nsize = 0;
+ u32 nsize = 0;
__u64 user_mode;
__u64 group_mode;
__u64 other_mode;
@@ -1186,17 +1201,17 @@ static void populate_new_aces(char *nacl_base,
*pnsize = nsize;
}
-static __u16 replace_sids_and_copy_aces(struct smb_acl *pdacl, struct smb_acl *pndacl,
- struct smb_sid *pownersid, struct smb_sid *pgrpsid,
- struct smb_sid *pnownersid, struct smb_sid *pngrpsid,
- int *aclflag)
+static int replace_sids_and_copy_aces(struct smb_acl *pdacl, struct smb_acl *pndacl,
+ struct smb_sid *pownersid, struct smb_sid *pgrpsid,
+ struct smb_sid *pnownersid, struct smb_sid *pngrpsid,
+ int *aclflag, u16 *pnsize)
{
int i;
u16 size = 0;
struct smb_ace *pntace = NULL;
char *acl_base = NULL;
u16 src_num_aces = 0;
- u16 nsize = 0;
+ u32 nsize = 0;
struct smb_ace *pnntace = NULL;
char *nacl_base = NULL;
u16 ace_size = 0;
@@ -1225,9 +1240,12 @@ static __u16 replace_sids_and_copy_aces(struct smb_acl *pdacl, struct smb_acl *p
size += le16_to_cpu(pntace->size);
nsize += ace_size;
+ if (nsize > U16_MAX)
+ return -EOVERFLOW;
}
- return nsize;
+ *pnsize = nsize;
+ return 0;
}
static int set_chmod_dacl(struct smb_acl *pdacl, struct smb_acl *pndacl,
@@ -1239,7 +1257,7 @@ static int set_chmod_dacl(struct smb_acl *pdacl, struct smb_acl *pndacl,
struct smb_ace *pntace = NULL;
char *acl_base = NULL;
u16 src_num_aces = 0;
- u16 nsize = 0;
+ u32 nsize = 0;
struct smb_ace *pnntace = NULL;
char *nacl_base = NULL;
u16 num_aces = 0;
@@ -1290,6 +1308,8 @@ static int set_chmod_dacl(struct smb_acl *pdacl, struct smb_acl *pndacl,
nsize += cifs_copy_ace(pnntace, pntace, NULL);
num_aces++;
+ if (nsize > U16_MAX)
+ return -EOVERFLOW;
next_ace:
size += le16_to_cpu(pntace->size);
@@ -1306,6 +1326,10 @@ static int set_chmod_dacl(struct smb_acl *pdacl, struct smb_acl *pndacl,
}
finalize_dacl:
+ /* The DACL size field is 16-bit on the wire, see MS-DTYP 2.4.5 */
+ if (nsize > U16_MAX)
+ return -EOVERFLOW;
+
pndacl->num_aces = cpu_to_le16(num_aces);
pndacl->size = cpu_to_le16(nsize);
@@ -1331,6 +1355,7 @@ static int parse_sec_desc(struct cifs_sb_info *cifs_sb,
{
int rc = 0;
struct smb_sid *owner_sid_ptr, *group_sid_ptr;
+ unsigned int sbflags = cifs_sb_flags(cifs_sb);
struct smb_acl *dacl_ptr; /* no need for SACL ptr */
char *end_of_acl;
__u32 dacloffset, osidoffset, gsidoffset;
@@ -1349,17 +1374,21 @@ static int parse_sec_desc(struct cifs_sb_info *cifs_sb,
cifs_dbg(NOISY, "revision %d type 0x%x ooffset 0x%x goffset 0x%x sacloffset 0x%x dacloffset 0x%x\n",
pntsd->revision, pntsd->type, osidoffset, gsidoffset,
le32_to_cpu(pntsd->sacloffset), dacloffset);
-/* cifs_dump_mem("owner_sid: ", owner_sid_ptr, 64); */
+ fattr->cf_uid = cifs_sb->ctx->linux_uid;
+ fattr->cf_gid = cifs_sb->ctx->linux_gid;
+
rc = sid_from_sd(pntsd, acl_len, osidoffset, &owner_sid_ptr);
if (rc) {
cifs_dbg(FYI, "%s: Error %d parsing Owner SID\n", __func__, rc);
return rc;
}
- rc = sid_to_id(cifs_sb, owner_sid_ptr, fattr, SIDOWNER);
- if (rc) {
- cifs_dbg(FYI, "%s: Error %d mapping Owner SID to uid\n",
- __func__, rc);
- return rc;
+ if (!(sbflags & CIFS_MOUNT_OVERR_UID)) {
+ rc = sid_to_id(cifs_sb, owner_sid_ptr, fattr, SIDOWNER);
+ if (rc) {
+ cifs_dbg(FYI, "%s: Error %d mapping Owner SID to uid\n",
+ __func__, rc);
+ return rc;
+ }
}
rc = sid_from_sd(pntsd, acl_len, gsidoffset, &group_sid_ptr);
@@ -1368,11 +1397,13 @@ static int parse_sec_desc(struct cifs_sb_info *cifs_sb,
__func__, rc);
return rc;
}
- rc = sid_to_id(cifs_sb, group_sid_ptr, fattr, SIDGROUP);
- if (rc) {
- cifs_dbg(FYI, "%s: Error %d mapping Group SID to gid\n",
- __func__, rc);
- return rc;
+ if (!(sbflags & CIFS_MOUNT_OVERR_GID)) {
+ rc = sid_to_id(cifs_sb, group_sid_ptr, fattr, SIDGROUP);
+ if (rc) {
+ cifs_dbg(FYI, "%s: Error %d mapping Group SID to gid\n",
+ __func__, rc);
+ return rc;
+ }
}
if (dacloffset) {
@@ -1451,6 +1482,8 @@ static int build_sec_desc(struct smb_ntsd *pntsd, struct smb_ntsd *pnntsd,
rc = set_chmod_dacl(dacl_ptr, ndacl_ptr, owner_sid_ptr, group_sid_ptr,
pnmode, mode_from_sid, posix);
+ if (rc)
+ return rc;
sidsoffset = ndacloffset + le16_to_cpu(ndacl_ptr->size);
/* copy the non-dacl portion of secdesc */
@@ -1526,10 +1559,12 @@ static int build_sec_desc(struct smb_ntsd *pntsd, struct smb_ntsd *pnntsd,
if (dacloffset) {
/* Replace ACEs for old owner with new one */
- size = replace_sids_and_copy_aces(dacl_ptr, ndacl_ptr,
- owner_sid_ptr, group_sid_ptr,
- nowner_sid_ptr, ngroup_sid_ptr,
- aclflag);
+ rc = replace_sids_and_copy_aces(dacl_ptr, ndacl_ptr,
+ owner_sid_ptr, group_sid_ptr,
+ nowner_sid_ptr, ngroup_sid_ptr,
+ aclflag, &size);
+ if (rc)
+ goto chown_chgrp_exit;
ndacl_ptr->size = cpu_to_le16(size);
}
@@ -1815,11 +1850,13 @@ id_mode_to_cifs_acl(struct inode *inode, const char *path, __u64 *pnmode,
cifs_put_tlink(tlink);
return rc;
}
- if (mode_from_sid)
- nsecdesclen +=
- le16_to_cpu(dacl_ptr->num_aces) * sizeof(struct smb_ace);
- else /* cifsacl */
- nsecdesclen += le16_to_cpu(dacl_ptr->size);
+ /*
+ * Worst case: every ACE is rewritten with a new SID of
+ * SID_MAX_SUB_AUTHORITIES sub-auths -> sizeof(smb_ace) each,
+ * plus the smb_acl header replace_sids_and_copy_aces() emits.
+ */
+ nsecdesclen += sizeof(struct smb_acl) +
+ le16_to_cpu(dacl_ptr->num_aces) * sizeof(struct smb_ace);
}
}
diff --git a/fs/smb/client/cifssmb.c b/fs/smb/client/cifssmb.c
index f5aad5f61dce..a1da1ade8c10 100644
--- a/fs/smb/client/cifssmb.c
+++ b/fs/smb/client/cifssmb.c
@@ -1719,8 +1719,17 @@ CIFSSMBRead(const unsigned int xid, struct cifs_io_parms *io_parms,
pSMBr = (READ_RSP *)rsp_iov.iov_base;
if (rc) {
cifs_dbg(VFS, "Send error in read = %d\n", rc);
+ } else if (rsp_iov.iov_len < tcon->ses->server->vals->read_rsp_size) {
+ /* check that the received response can hold a whole READ_RSP */
+ cifs_dbg(FYI, "%s: server returned short header. got=%zu expected=%zu\n",
+ __func__, rsp_iov.iov_len,
+ tcon->ses->server->vals->read_rsp_size);
+ rc = smb_EIO2(smb_eio_trace_read_rsp_short,
+ rsp_iov.iov_len, tcon->ses->server->vals->read_rsp_size);
+ *nbytes = 0;
} else {
- int data_length = le16_to_cpu(pSMBr->DataLengthHigh);
+ unsigned int data_length = le16_to_cpu(pSMBr->DataLengthHigh);
+ __u16 data_offset = le16_to_cpu(pSMBr->DataOffset);
data_length = data_length << 16;
data_length += le16_to_cpu(pSMBr->DataLength);
*nbytes = data_length;
@@ -1728,14 +1737,21 @@ CIFSSMBRead(const unsigned int xid, struct cifs_io_parms *io_parms,
/*check that DataLength would not go beyond end of SMB */
if ((data_length > CIFSMaxBufSize)
|| (data_length > count)) {
- cifs_dbg(FYI, "bad length %d for count %d\n",
- data_length, count);
+ cifs_dbg(FYI, "%s: bad length %u for count %u\n",
+ __func__, data_length, count);
rc = smb_EIO2(smb_eio_trace_read_overlarge,
data_length, count);
*nbytes = 0;
+ } else if (data_offset < sizeof(*pSMBr) ||
+ (size_t)data_offset + data_length > rsp_iov.iov_len) {
+ /* check that the data lies within the received response */
+ cifs_dbg(FYI, "%s: bad data offset %u length %u for response of %zu\n",
+ __func__, data_offset, data_length, rsp_iov.iov_len);
+ rc = smb_EIO2(smb_eio_trace_read_bad_offset,
+ data_offset, data_length);
+ *nbytes = 0;
} else {
- pReadData = (char *) (&pSMBr->hdr.Protocol) +
- le16_to_cpu(pSMBr->DataOffset);
+ pReadData = (char *) (&pSMBr->hdr.Protocol) + data_offset;
/* if (rc = copy_to_user(buf, pReadData, data_length)) {
cifs_dbg(VFS, "Faulting on read rc = %d\n",rc);
rc = -EFAULT;
diff --git a/fs/smb/client/file.c b/fs/smb/client/file.c
index 389083f9ce00..8886e9483c91 100644
--- a/fs/smb/client/file.c
+++ b/fs/smb/client/file.c
@@ -1491,11 +1491,18 @@ int cifs_close(struct inode *inode, struct file *file)
trace_smb3_close_cached(tcon->tid, tcon->ses->Suid,
cfile->fid.persistent_fid,
cifs_sb->ctx->closetimeo);
- queue_delayed_work(deferredclose_wq,
- &cfile->deferred, cifs_sb->ctx->closetimeo);
- cfile->deferred_close_scheduled = true;
- spin_unlock(&cinode->deferred_lock);
- return 0;
+ /*
+ * Each queued execution owns one reference.
+ * If nothing was queued, the reference of
+ * the closing file is dropped below.
+ */
+ if (queue_delayed_work(deferredclose_wq,
+ &cfile->deferred,
+ cifs_sb->ctx->closetimeo)) {
+ cfile->deferred_close_scheduled = true;
+ spin_unlock(&cinode->deferred_lock);
+ return 0;
+ }
}
spin_unlock(&cinode->deferred_lock);
_cifsFileInfo_put(cfile, true, false);
@@ -3324,8 +3331,11 @@ void cifs_oplock_break(struct work_struct *work)
TASK_UNINTERRUPTIBLE);
tlink = cifs_sb_tlink(cifs_sb);
- if (IS_ERR(tlink))
+ if (IS_ERR(tlink)) {
+ /* drop the reference taken when the break was queued */
+ _cifsFileInfo_put(cfile, false /* do not wait for ourself */, false);
goto out;
+ }
tcon = tlink_tcon(tlink);
server = tcon->ses->server;
diff --git a/fs/smb/client/inode.c b/fs/smb/client/inode.c
index 85ef8fff0e84..db797a562156 100644
--- a/fs/smb/client/inode.c
+++ b/fs/smb/client/inode.c
@@ -851,6 +851,7 @@ static void smb311_posix_info_to_fattr(struct cifs_fattr *fattr,
struct smb311_posix_qinfo *info = &data->posix_fi;
struct cifs_sb_info *cifs_sb = CIFS_SB(sb);
struct cifs_tcon *tcon = cifs_sb_master_tcon(cifs_sb);
+ unsigned int sbflags = cifs_sb_flags(cifs_sb);
memset(fattr, 0, sizeof(*fattr));
@@ -895,8 +896,12 @@ static void smb311_posix_info_to_fattr(struct cifs_fattr *fattr,
fattr->cf_symlink_target = data->symlink_target;
data->symlink_target = NULL;
}
- sid_to_id(cifs_sb, &data->posix_owner, fattr, SIDOWNER);
- sid_to_id(cifs_sb, &data->posix_group, fattr, SIDGROUP);
+ fattr->cf_uid = cifs_sb->ctx->linux_uid;
+ fattr->cf_gid = cifs_sb->ctx->linux_gid;
+ if (!(sbflags & CIFS_MOUNT_OVERR_UID))
+ sid_to_id(cifs_sb, &data->posix_owner, fattr, SIDOWNER);
+ if (!(sbflags & CIFS_MOUNT_OVERR_GID))
+ sid_to_id(cifs_sb, &data->posix_group, fattr, SIDGROUP);
cifs_dbg(FYI, "POSIX query info: mode 0x%x uniqueid 0x%llx nlink %d\n",
fattr->cf_mode, fattr->cf_uniqueid, fattr->cf_nlink);
@@ -2983,14 +2988,14 @@ int cifs_getattr(struct mnt_idmap *idmap, const struct path *path,
stat->attributes |= STATX_ATTR_ENCRYPTED;
/*
- * If on a multiuser mount without unix extensions or cifsacl being
- * enabled, and the admin hasn't overridden them, set the ownership
- * to the fsuid/fsgid of the current process.
+ * If on a multiuser mount without unix extensions, posix extensions
+ * or cifsacl being enabled, and the admin hasn't overridden them,
+ * set the ownership to the fsuid/fsgid of the current process.
*/
sbflags = cifs_sb_flags(cifs_sb);
if ((sbflags & CIFS_MOUNT_MULTIUSER) &&
!(sbflags & CIFS_MOUNT_CIFS_ACL) &&
- !tcon->unix_ext) {
+ !tcon->unix_ext && !tcon->posix_extensions) {
if (!(sbflags & CIFS_MOUNT_OVERR_UID))
stat->uid = current_fsuid();
if (!(sbflags & CIFS_MOUNT_OVERR_GID))
diff --git a/fs/smb/client/misc.c b/fs/smb/client/misc.c
index 46e1382e8e04..945194fe7a97 100644
--- a/fs/smb/client/misc.c
+++ b/fs/smb/client/misc.c
@@ -378,10 +378,11 @@ void cifs_queue_oplock_break(struct cifsFileInfo *cfile)
* open_file_lock to enforce the validity of it for the oplock
* break handler. The matching put is done at the end of the
* handler.
+ *
+ * Only take a reference if the work is actually queued.
*/
- cifsFileInfo_get(cfile);
-
- queue_work(cifsoplockd_wq, &cfile->oplock_break);
+ if (queue_work(cifsoplockd_wq, &cfile->oplock_break))
+ cifsFileInfo_get(cfile);
}
void cifs_done_oplock_break(struct cifsInodeInfo *cinode)
@@ -891,8 +892,14 @@ static void tcon_super_cb(struct super_block *sb, void *arg)
t1->ses->dfs_root_ses == t2->ses->dfs_root_ses) &&
t1->ses->server == t2->ses->server &&
t2->origin_fullpath &&
- dfs_src_pathname_equal(t2->origin_fullpath, t1->origin_fullpath))
+ dfs_src_pathname_equal(t2->origin_fullpath, t1->origin_fullpath)) {
+ /*
+ * Take the active reference while iterate_supers_type() still
+ * holds s_umount shared.
+ */
+ cifs_sb_active(sb);
sd->sb = sb;
+ }
spin_unlock(&t2->tc_lock);
}
@@ -909,15 +916,8 @@ static struct super_block *__cifs_get_super(void (*f)(struct super_block *, void
for (; *fs_type; fs_type++) {
iterate_supers_type(*fs_type, f, &sd);
- if (sd.sb) {
- /*
- * Grab an active reference in order to prevent automounts (DFS links)
- * of expiring and then freeing up our cifs superblock pointer while
- * we're doing failover.
- */
- cifs_sb_active(sd.sb);
+ if (sd.sb)
return sd.sb;
- }
}
pr_warn_once("%s: could not find dfs superblock\n", __func__);
return ERR_PTR(-EINVAL);
diff --git a/fs/smb/client/readdir.c b/fs/smb/client/readdir.c
index 32a75afca8f5..9530e5b01564 100644
--- a/fs/smb/client/readdir.c
+++ b/fs/smb/client/readdir.c
@@ -242,9 +242,11 @@ static void
cifs_posix_to_fattr(struct cifs_fattr *fattr, struct smb2_posix_info *info,
struct cifs_sb_info *cifs_sb)
{
+ unsigned int sbflags = cifs_sb_flags(cifs_sb);
struct smb2_posix_info_parsed parsed;
+ int rc;
- posix_info_parse(info, NULL, &parsed);
+ rc = posix_info_parse(info, NULL, &parsed);
memset(fattr, 0, sizeof(*fattr));
fattr->cf_uniqueid = le64_to_cpu(info->Inode);
@@ -281,8 +283,17 @@ cifs_posix_to_fattr(struct cifs_fattr *fattr, struct smb2_posix_info *info,
le32_to_cpu(info->ReparseTag),
le32_to_cpu(info->Mode));
- sid_to_id(cifs_sb, &parsed.owner, fattr, SIDOWNER);
- sid_to_id(cifs_sb, &parsed.group, fattr, SIDGROUP);
+ fattr->cf_uid = cifs_sb->ctx->linux_uid;
+ fattr->cf_gid = cifs_sb->ctx->linux_gid;
+ if (rc < 0) {
+ cifs_dbg(VFS, "%s: failed to parse SIDs: %d\n",
+ __func__, rc);
+ } else {
+ if (!(sbflags & CIFS_MOUNT_OVERR_UID))
+ sid_to_id(cifs_sb, &parsed.owner, fattr, SIDOWNER);
+ if (!(sbflags & CIFS_MOUNT_OVERR_GID))
+ sid_to_id(cifs_sb, &parsed.group, fattr, SIDGROUP);
+ }
}
static void __dir_info_to_fattr(struct cifs_fattr *fattr, const void *info)
diff --git a/fs/smb/client/reparse.c b/fs/smb/client/reparse.c
index 5cc5b0410d48..8a1b9e8be5ba 100644
--- a/fs/smb/client/reparse.c
+++ b/fs/smb/client/reparse.c
@@ -971,7 +971,8 @@ int smb2_parse_native_symlink(char **target, const char *buf, unsigned int len,
linux_target[i*3 + 1] = '.';
linux_target[i*3 + 2] = sep;
}
- memcpy(linux_target + levels*3, smb_target+1, smb_target_len); /* +1 to skip leading sep */
+ /* +1 to skip leading sep */
+ memcpy(linux_target + levels*3, smb_target+1, smb_target_len-1);
} else {
/*
* This is either an absolute symlink in POSIX-style format
@@ -1137,10 +1138,15 @@ static bool wsl_to_fattr(struct cifs_open_info_data *data,
struct cifs_sb_info *cifs_sb,
u32 tag, struct cifs_fattr *fattr)
{
+ unsigned int sbflags = cifs_sb_flags(cifs_sb);
struct smb2_file_full_ea_info *ea;
bool have_xattr_dev = false;
u32 next = 0;
+ fattr->cf_uid = cifs_sb->ctx->linux_uid;
+ fattr->cf_gid = cifs_sb->ctx->linux_gid;
+
+ fattr->cf_mode &= ~S_IFMT;
switch (tag) {
case IO_REPARSE_TAG_LX_SYMLINK:
fattr->cf_mode |= S_IFLNK;
@@ -1177,11 +1183,13 @@ static bool wsl_to_fattr(struct cifs_open_info_data *data,
nlen = ea->ea_name_length;
v = (void *)((u8 *)ea->ea_data + ea->ea_name_length + 1);
- if (!strncmp(name, SMB2_WSL_XATTR_UID, nlen))
- fattr->cf_uid = wsl_make_kuid(cifs_sb, v);
- else if (!strncmp(name, SMB2_WSL_XATTR_GID, nlen))
- fattr->cf_gid = wsl_make_kgid(cifs_sb, v);
- else if (!strncmp(name, SMB2_WSL_XATTR_MODE, nlen)) {
+ if (!strncmp(name, SMB2_WSL_XATTR_UID, nlen)) {
+ if (!(sbflags & CIFS_MOUNT_OVERR_UID))
+ fattr->cf_uid = wsl_make_kuid(cifs_sb, v);
+ } else if (!strncmp(name, SMB2_WSL_XATTR_GID, nlen)) {
+ if (!(sbflags & CIFS_MOUNT_OVERR_GID))
+ fattr->cf_gid = wsl_make_kgid(cifs_sb, v);
+ } else if (!strncmp(name, SMB2_WSL_XATTR_MODE, nlen)) {
/* File type in reparse point tag and in xattr mode must match. */
if (S_DT(fattr->cf_mode) != S_DT(le32_to_cpu(*(__le32 *)v)))
return false;
@@ -1205,6 +1213,7 @@ static bool posix_reparse_to_fattr(struct cifs_sb_info *cifs_sb,
struct cifs_open_info_data *data)
{
struct reparse_nfs_data_buffer *buf = (struct reparse_nfs_data_buffer *)data->reparse.buf;
+ umode_t ftype;
if (buf == NULL)
return true;
@@ -1220,7 +1229,7 @@ static bool posix_reparse_to_fattr(struct cifs_sb_info *cifs_sb,
WARN_ON_ONCE(1);
return false;
}
- fattr->cf_mode |= S_IFCHR;
+ ftype = S_IFCHR;
fattr->cf_rdev = reparse_mkdev(buf->DataBuffer);
break;
case NFS_SPECFILE_BLK:
@@ -1228,22 +1237,23 @@ static bool posix_reparse_to_fattr(struct cifs_sb_info *cifs_sb,
WARN_ON_ONCE(1);
return false;
}
- fattr->cf_mode |= S_IFBLK;
+ ftype = S_IFBLK;
fattr->cf_rdev = reparse_mkdev(buf->DataBuffer);
break;
case NFS_SPECFILE_FIFO:
- fattr->cf_mode |= S_IFIFO;
+ ftype = S_IFIFO;
break;
case NFS_SPECFILE_SOCK:
- fattr->cf_mode |= S_IFSOCK;
+ ftype = S_IFSOCK;
break;
case NFS_SPECFILE_LNK:
- fattr->cf_mode |= S_IFLNK;
+ ftype = S_IFLNK;
break;
default:
WARN_ON_ONCE(1);
return false;
}
+ fattr->cf_mode = (fattr->cf_mode & ~S_IFMT) | ftype;
return true;
}
@@ -1271,6 +1281,7 @@ bool cifs_reparse_point_to_fattr(struct cifs_sb_info *cifs_sb,
break;
case 0: /* SMB1 symlink */
case IO_REPARSE_TAG_SYMLINK:
+ fattr->cf_mode &= ~S_IFMT;
fattr->cf_mode |= S_IFLNK;
break;
default:
diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c
index 0e872d58fae7..5f616384c75c 100644
--- a/fs/smb/client/smb2ops.c
+++ b/fs/smb/client/smb2ops.c
@@ -1839,31 +1839,31 @@ smb2_ioctl_query_info(const unsigned int xid,
*
* @tcon: destination file tcon
* @bytes_left: how many bytes are left to copy
+ * @chunk_size: maximum size of a single chunk
*
* Return: maximum number of chunks with which Chunks[] can be filled.
*/
static inline u32
-calc_chunk_count(struct cifs_tcon *tcon, u64 bytes_left)
+calc_chunk_count(struct cifs_tcon *tcon, u64 bytes_left, u32 chunk_size)
{
u32 max_chunks = READ_ONCE(tcon->max_chunks);
u32 max_bytes_copy = READ_ONCE(tcon->max_bytes_copy);
- u32 max_bytes_chunk = READ_ONCE(tcon->max_bytes_chunk);
u64 need;
u32 allowed;
- if (!max_bytes_chunk || !max_bytes_copy || !max_chunks)
+ if (!chunk_size || !max_bytes_copy || !max_chunks)
return 0;
/* chunks needed for the remaining bytes */
- need = DIV_ROUND_UP_ULL(bytes_left, max_bytes_chunk);
+ need = DIV_ROUND_UP_ULL(bytes_left, chunk_size);
/* chunks allowed per cc request */
- allowed = DIV_ROUND_UP(max_bytes_copy, max_bytes_chunk);
+ allowed = DIV_ROUND_UP(max_bytes_copy, chunk_size);
return (u32)umin(need, umin(max_chunks, allowed));
}
/**
- * smb2_copychunk_range - server-side copy of data range
+ * __smb2_copychunk_range - server-side copy of data range
*
* @xid: transaction id
* @src_file: source file
@@ -1875,15 +1875,15 @@ calc_chunk_count(struct cifs_tcon *tcon, u64 bytes_left)
* Obtains a resume key for @src_file and issues FSCTL_SRV_COPYCHUNK_WRITE
* IOCTLs, splitting the request into chunks limited by tcon->max_*.
*
- * Return: @len on success; negative errno on failure.
+ * Return: 0 on success; negative errno on failure.
*/
-static ssize_t
-smb2_copychunk_range(const unsigned int xid,
- struct cifsFileInfo *src_file,
- struct cifsFileInfo *dst_file,
- u64 src_off,
- u64 len,
- u64 dst_off)
+static int
+__smb2_copychunk_range(const unsigned int xid,
+ struct cifsFileInfo *src_file,
+ struct cifsFileInfo *dst_file,
+ u64 src_off,
+ u64 len,
+ u64 dst_off)
{
int rc = 0;
unsigned int ret_data_len = 0;
@@ -1891,12 +1891,14 @@ smb2_copychunk_range(const unsigned int xid,
struct copychunk_ioctl_rsp *cc_rsp = NULL;
struct cifs_tcon *tcon;
struct srv_copychunk *chunk;
- u32 chunks, chunk_count, chunk_bytes;
+ u32 chunks, chunk_count, chunk_bytes, chunk_size;
u32 copy_bytes, copy_bytes_left;
u32 chunks_written, bytes_written;
u64 total_bytes_left = len;
u64 src_off_prev, dst_off_prev;
+ u64 max_chunk = 0;
u32 retries = 0;
+ bool reverse = false;
tcon = tlink_tcon(dst_file->tlink);
@@ -1904,8 +1906,50 @@ smb2_copychunk_range(const unsigned int xid,
dst_file->fid.volatile_fid, tcon->tid,
tcon->ses->Suid, src_off, dst_off, len);
+ /*
+ * Same-file left shifts are safe in forward order. For a right shift,
+ * let L be the copy length, delta the distance between the source and
+ * destination, and C the normal chunk size:
+ *
+ * delta >= L: copy forwards using C
+ * delta < L:
+ * delta >= C: copy backwards using C
+ * delta < C: copy backwards with chunks limited to delta
+ *
+ * Copying backwards prevents one chunk from overwriting data needed by
+ * a later chunk. Limiting the chunk size to delta prevents an individual
+ * chunk from overlapping itself.
+ * This limit can be removed once all supported servers handle overlapping
+ * descriptors safely.
+ *
+ * A small right shift over a large range may therefore require many
+ * chunks.
+ */
+ if (src_file == dst_file && dst_off > src_off) {
+ u64 delta = dst_off - src_off;
+
+ if (delta < len) {
+ reverse = true;
+ max_chunk = delta;
+ }
+ }
+
+ /*
+ * A backward copy walks the offsets down from the end of the range.
+ * Do this once, outside the retry loop, so a retry does not move the
+ * offsets again.
+ */
+ if (reverse) {
+ src_off += len;
+ dst_off += len;
+ }
+
retry:
- chunk_count = calc_chunk_count(tcon, total_bytes_left);
+ chunk_size = READ_ONCE(tcon->max_bytes_chunk);
+ if (max_chunk && max_chunk < chunk_size)
+ chunk_size = (u32)max_chunk;
+
+ chunk_count = calc_chunk_count(tcon, total_bytes_left, chunk_size);
if (!chunk_count) {
rc = -EOPNOTSUPP;
goto out;
@@ -1946,16 +1990,21 @@ smb2_copychunk_range(const unsigned int xid,
while (copy_bytes_left > 0 && chunks < chunk_count) {
chunk = &cc_req->Chunks[chunks++];
+ chunk_bytes = umin(copy_bytes_left, chunk_size);
+ if (reverse) {
+ src_off -= chunk_bytes;
+ dst_off -= chunk_bytes;
+ }
+
chunk->SourceOffset = cpu_to_le64(src_off);
chunk->TargetOffset = cpu_to_le64(dst_off);
-
- chunk_bytes = umin(copy_bytes_left, tcon->max_bytes_chunk);
-
chunk->Length = cpu_to_le32(chunk_bytes);
/* Buffer is zeroed, no need to set chunk->Reserved = 0 */
- src_off += chunk_bytes;
- dst_off += chunk_bytes;
+ if (!reverse) {
+ src_off += chunk_bytes;
+ dst_off += chunk_bytes;
+ }
copy_bytes_left -= chunk_bytes;
copy_bytes += chunk_bytes;
@@ -2003,6 +2052,18 @@ smb2_copychunk_range(const unsigned int xid,
goto out;
}
+ /*
+ * A successful COPYCHUNK should copy every descriptor (MS-SMB2
+ * 3.3.5.15.6). Reject a short backward copy because the rewind
+ * below only supports forward copying.
+ */
+ if (unlikely(reverse && bytes_written < copy_bytes)) {
+ cifs_tcon_dbg(VFS, "Copychunk short write %u/%u (reverse)\n",
+ bytes_written, copy_bytes);
+ rc = -EIO;
+ goto out;
+ }
+
/* Partial write: rewind */
if (bytes_written < copy_bytes) {
u32 delta = copy_bytes - bytes_written;
@@ -2064,10 +2125,27 @@ smb2_copychunk_range(const unsigned int xid,
trace_smb3_copychunk_done(xid, src_file->fid.volatile_fid,
dst_file->fid.volatile_fid, tcon->tid,
tcon->ses->Suid, src_off, dst_off, len);
- return len;
+ return 0;
}
}
+static ssize_t
+smb2_copychunk_range(const unsigned int xid,
+ struct cifsFileInfo *src_file,
+ struct cifsFileInfo *dst_file,
+ u64 src_off,
+ u64 len,
+ u64 dst_off)
+{
+ int rc;
+
+ rc = __smb2_copychunk_range(xid, src_file, dst_file, src_off, len,
+ dst_off);
+ if (rc)
+ return rc;
+ return len;
+}
+
static int
smb2_flush_file(const unsigned int xid, struct cifs_tcon *tcon,
struct cifs_fid *fid)
@@ -3441,6 +3519,13 @@ static long smb3_zero_range(struct file *file, struct cifs_tcon *tcon,
trace_smb3_zero_enter(xid, cfile->fid.persistent_fid, tcon->tid,
ses->Suid, offset, len);
+ new_size = offset + len;
+ if (!keep_size && i_size_read(inode) < new_size) {
+ rc = inode_newsize_ok(inode, new_size);
+ if (rc)
+ goto out;
+ }
+
filemap_invalidate_lock(inode->i_mapping);
netfs_read_sizes(inode, &i_size, &remote_i_size, &zero_point);
@@ -3464,6 +3549,9 @@ static long smb3_zero_range(struct file *file, struct cifs_tcon *tcon,
if (keep_size == false && !CIFS_CACHE_READ(cifsi))
goto zero_range_exit;
+ fscache_invalidate(cifs_inode_cookie(inode), NULL,
+ i_size_read(inode), 0);
+
rc = smb3_zero_data(file, tcon, offset, len, xid);
if (rc < 0)
goto zero_range_exit;
@@ -3471,7 +3559,6 @@ static long smb3_zero_range(struct file *file, struct cifs_tcon *tcon,
/*
* do we also need to change the size of the file?
*/
- new_size = offset + len;
if (keep_size == false && (unsigned long long)i_size_read(inode) < new_size) {
rc = SMB2_set_eof(xid, tcon, cfile->fid.persistent_fid,
cfile->fid.volatile_fid, cfile->pid, new_size);
@@ -3488,6 +3575,7 @@ static long smb3_zero_range(struct file *file, struct cifs_tcon *tcon,
zero_range_exit:
filemap_invalidate_unlock(inode->i_mapping);
+ out:
free_xid(xid);
if (rc)
trace_smb3_zero_err(xid, cfile->fid.persistent_fid, tcon->tid,
@@ -3533,6 +3621,8 @@ static long smb3_punch_hole(struct file *file, struct cifs_tcon *tcon,
*/
truncate_pagecache_range(inode, offset, offset + len - 1);
netfs_wait_for_outstanding_io(inode);
+ fscache_invalidate(cifs_inode_cookie(inode), NULL,
+ i_size_read(inode), 0);
cifs_dbg(FYI, "Offset %lld len %lld\n", offset, len);
@@ -3938,18 +4028,26 @@ static long smb3_collapse_range(struct file *file, struct cifs_tcon *tcon,
}
filemap_invalidate_lock(inode->i_mapping);
- rc = filemap_write_and_wait_range(inode->i_mapping, off, old_eof - 1);
+ rc = filemap_write_and_wait_range(inode->i_mapping,
+ round_down(off, PAGE_SIZE),
+ old_eof - 1);
if (rc < 0)
goto out_2;
- truncate_pagecache_range(inode, off, old_eof);
+ netfs_wait_for_outstanding_io(inode);
+ /*
+ * Invalidate cached folios from the page containing off to EOF before
+ * moving data on the server, so subsequent reads do not see stale data.
+ */
+ truncate_pagecache_range(inode, round_down(off, PAGE_SIZE), -1);
+ fscache_invalidate(cifs_inode_cookie(inode), NULL, old_eof, 0);
+
spin_lock(&inode->i_lock);
netfs_write_zero_point(inode, old_eof);
spin_unlock(&inode->i_lock);
- netfs_wait_for_outstanding_io(inode);
- rc = smb2_copychunk_range(xid, cfile, cfile, off + len,
- old_eof - off - len, off);
+ rc = __smb2_copychunk_range(xid, cfile, cfile, off + len,
+ old_eof - off - len, off);
if (rc < 0)
goto out_2;
@@ -3982,7 +4080,7 @@ static long smb3_insert_range(struct file *file, struct cifs_tcon *tcon,
struct cifsFileInfo *cfile = file->private_data;
struct inode *inode = file_inode(file);
struct cifsInodeInfo *cifsi = CIFS_I(inode);
- __u64 count, old_eof, new_eof;
+ loff_t old_eof, new_eof;
xid = get_xid();
@@ -3992,15 +4090,32 @@ static long smb3_insert_range(struct file *file, struct cifs_tcon *tcon,
goto out;
}
- count = old_eof - off;
- new_eof = old_eof + len;
+ if (check_add_overflow(old_eof, len, &new_eof)) {
+ rc = -EFBIG;
+ goto out;
+ }
+ rc = inode_newsize_ok(inode, new_eof);
+ if (rc)
+ goto out;
+
+ /* SET_ZERO_DATA creates a hole only in a sparse file. */
+ rc = smb2_set_sparse(xid, tcon, cfile, inode, true);
+ if (rc)
+ goto out;
filemap_invalidate_lock(inode->i_mapping);
- rc = filemap_write_and_wait_range(inode->i_mapping, off, new_eof - 1);
+ rc = filemap_write_and_wait_range(inode->i_mapping,
+ round_down(off, PAGE_SIZE),
+ old_eof - 1);
if (rc < 0)
goto out_2;
- truncate_pagecache_range(inode, off, old_eof);
netfs_wait_for_outstanding_io(inode);
+ /*
+ * Invalidate cached folios from the page containing off to EOF before
+ * moving data on the server, so subsequent reads do not see stale data.
+ */
+ truncate_pagecache_range(inode, round_down(off, PAGE_SIZE), -1);
+ fscache_invalidate(cifs_inode_cookie(inode), NULL, old_eof, 0);
rc = SMB2_set_eof(xid, tcon, cfile->fid.persistent_fid,
cfile->fid.volatile_fid, cfile->pid, new_eof);
@@ -4013,7 +4128,12 @@ static long smb3_insert_range(struct file *file, struct cifs_tcon *tcon,
spin_unlock(&inode->i_lock);
fscache_resize_cookie(cifs_inode_cookie(inode), i_size_read(inode));
- rc = smb2_copychunk_range(xid, cfile, cfile, off, count, off + len);
+ /*
+ * Move [off, old_eof) right by len. The helper copies backwards if the
+ * source and destination ranges overlap.
+ */
+ rc = __smb2_copychunk_range(xid, cfile, cfile, off, old_eof - off,
+ off + len);
if (rc < 0)
goto out_2;
spin_lock(&inode->i_lock);
diff --git a/fs/smb/client/trace.h b/fs/smb/client/trace.h
index 12241abb8e2e..b442cccd1530 100644
--- a/fs/smb/client/trace.h
+++ b/fs/smb/client/trace.h
@@ -79,6 +79,7 @@
EM(smb_eio_trace_qreparse_setup_count, "qreparse_setup_count") \
EM(smb_eio_trace_qreparse_sizes_wrong, "qreparse_sizes_wrong") \
EM(smb_eio_trace_qsym_bcc_too_small, "qsym_bcc_too_small") \
+ EM(smb_eio_trace_read_bad_offset, "read_bad_offset") \
EM(smb_eio_trace_read_mid_state_unknown, "read_mid_state_unknown") \
EM(smb_eio_trace_read_overlarge, "read_overlarge") \
EM(smb_eio_trace_read_rsp_malformed, "read_rsp_malformed") \
diff --git a/fs/smb/client/transport.c b/fs/smb/client/transport.c
index fdf4e50c27ce..e266859818a4 100644
--- a/fs/smb/client/transport.c
+++ b/fs/smb/client/transport.c
@@ -101,12 +101,11 @@ void __release_mid(struct TCP_Server_Info *server, struct mid_q_entry *midEntry)
trace_smb3_slow_rsp(smb_cmd, midEntry->mid, midEntry->pid,
midEntry->when_sent, midEntry->when_received);
if (cifsFYI & CIFS_TIMER) {
- pr_debug("slow rsp: cmd %d mid %llu",
- midEntry->command, midEntry->mid);
- cifs_info("A: 0x%lx S: 0x%lx R: 0x%lx\n",
- now - midEntry->when_alloc,
- now - midEntry->when_sent,
- now - midEntry->when_received);
+ pr_debug("slow rsp: cmd %d mid %llu A: 0x%lx S: 0x%lx R: 0x%lx\n",
+ midEntry->command, midEntry->mid,
+ now - midEntry->when_alloc,
+ now - midEntry->when_sent,
+ now - midEntry->when_received);
}
}
#endif
diff --git a/fs/smb/server/connection.c b/fs/smb/server/connection.c
index af73c2ed5d24..39ac777fd452 100644
--- a/fs/smb/server/connection.c
+++ b/fs/smb/server/connection.c
@@ -13,6 +13,7 @@
#include "mgmt/ksmbd_ida.h"
#include "mgmt/user_session.h"
#include "connection.h"
+#include "vfs_cache.h"
#include "compress.h"
#include "transport_tcp.h"
#include "transport_rdma.h"
@@ -294,6 +295,26 @@ void ksmbd_conn_try_dequeue_request(struct ksmbd_work *work)
wake_up_all(&conn->req_running_q);
}
+static void ksmbd_conn_cancel_async_requests(struct ksmbd_conn *conn)
+{
+ struct ksmbd_work *work, *tmp;
+
+ ksmbd_debug(CONN, "Cancel pending async requests on releasing connection\n");
+ spin_lock(&conn->request_lock);
+ list_for_each_entry_safe(work, tmp, &conn->async_requests,
+ async_request_entry) {
+ if (cmpxchg(&work->state, KSMBD_WORK_ACTIVE,
+ KSMBD_WORK_CANCELLED) != KSMBD_WORK_ACTIVE)
+ continue;
+
+ ksmbd_debug(CONN, "Cancel async request id %d\n",
+ work->async_id);
+ if (work->cancel_fn)
+ work->cancel_fn(work->cancel_argv);
+ }
+ spin_unlock(&conn->request_lock);
+}
+
void ksmbd_conn_lock(struct ksmbd_conn *conn)
{
mutex_lock(&conn->srv_mutex);
@@ -371,6 +392,9 @@ int ksmbd_conn_wait_idle_sess(struct ksmbd_conn *curr_conn,
if (retry_count >= max_timeout)
return -EIO;
+ /* A blocked byte-range lock cannot drain until teardown wakes it. */
+ ksmbd_wake_session_blocked_works(sess);
+
down_read(&conn_list_lock);
hash_for_each(conn_list, bkt, conn, hlist) {
if (ksmbd_session_is_bound_to_conn(sess, conn)) {
@@ -608,6 +632,7 @@ int ksmbd_conn_handler_loop(void *p)
}
ksmbd_conn_set_releasing(conn);
+ ksmbd_conn_cancel_async_requests(conn);
/* Wait till all reference dropped to the Server object*/
ksmbd_debug(CONN, "Wait for all pending requests(%d)\n", atomic_read(&conn->r_count));
wait_event(conn->r_count_q, atomic_read(&conn->r_count) == 0);
diff --git a/fs/smb/server/ksmbd_work.h b/fs/smb/server/ksmbd_work.h
index 5f1d3ebab4fb..0844aa929f55 100644
--- a/fs/smb/server/ksmbd_work.h
+++ b/fs/smb/server/ksmbd_work.h
@@ -82,7 +82,7 @@ struct ksmbd_work {
/* Contiguous SMB2 compression transform owned by this work item. */
void *compress_buf;
- unsigned char state;
+ unsigned int state;
/* No response for cancelled request */
bool send_no_response:1;
/* Request is encrypted */
diff --git a/fs/smb/server/mgmt/user_session.c b/fs/smb/server/mgmt/user_session.c
index d91dde3f9e61..10b31df185a6 100644
--- a/fs/smb/server/mgmt/user_session.c
+++ b/fs/smb/server/mgmt/user_session.c
@@ -650,10 +650,21 @@ void destroy_previous_session(struct ksmbd_conn *conn,
memcmp(user->passkey, prev_user->passkey, user->passkey_sz))
goto out;
+ down_write(&prev_sess->chann_lock);
+ if (prev_sess->tearing_down) {
+ up_write(&prev_sess->chann_lock);
+ goto out;
+ }
+ prev_sess->tearing_down = true;
+ up_write(&prev_sess->chann_lock);
+
ksmbd_all_conn_set_status(prev_sess, KSMBD_SESS_NEED_RECONNECT);
err = ksmbd_conn_wait_idle_sess(conn, prev_sess);
if (err) {
- ksmbd_all_conn_set_status(prev_sess, KSMBD_SESS_NEED_SETUP);
+ down_write(&prev_sess->chann_lock);
+ prev_sess->tearing_down = false;
+ up_write(&prev_sess->chann_lock);
+ ksmbd_all_conn_set_status(prev_sess, KSMBD_SESS_GOOD);
goto out;
}
diff --git a/fs/smb/server/mgmt/user_session.h b/fs/smb/server/mgmt/user_session.h
index f8a24c33f7fe..3e52d4cc1324 100644
--- a/fs/smb/server/mgmt/user_session.h
+++ b/fs/smb/server/mgmt/user_session.h
@@ -42,6 +42,7 @@ struct ksmbd_session {
bool sign;
bool enc;
+ bool tearing_down;
int state;
__u8 *Preauth_HashValue;
diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c
index 4cf7083f35cc..184b28072a6f 100644
--- a/fs/smb/server/smb2pdu.c
+++ b/fs/smb/server/smb2pdu.c
@@ -95,6 +95,11 @@ static int register_session_channel(struct ksmbd_session *sess,
int rc = 0;
down_write(&sess->chann_lock);
+ if (sess->tearing_down) {
+ rc = -ESHUTDOWN;
+ goto out;
+ }
+
if (xa_load(&sess->ksmbd_chann_list, (long)conn))
goto out;
@@ -2812,17 +2817,41 @@ int smb2_session_logoff(struct ksmbd_work *work)
smb2_set_err_rsp(work);
return -ENOENT;
}
+
+ down_write(&sess->chann_lock);
+ if (sess->tearing_down) {
+ up_write(&sess->chann_lock);
+ ksmbd_conn_unlock(conn);
+ rsp->hdr.Status = STATUS_USER_SESSION_DELETED;
+ smb2_set_err_rsp(work);
+ return -ENOENT;
+ }
+ sess->tearing_down = true;
+ up_write(&sess->chann_lock);
+
ksmbd_all_conn_set_status(sess, KSMBD_SESS_NEED_RECONNECT);
ksmbd_conn_unlock(conn);
+ err = ksmbd_conn_wait_idle_sess(conn, sess);
+ if (err) {
+ down_write(&sess->chann_lock);
+ sess->tearing_down = false;
+ up_write(&sess->chann_lock);
+ ksmbd_all_conn_set_status(sess, KSMBD_SESS_GOOD);
+ rsp->hdr.Status = STATUS_UNEXPECTED_IO_ERROR;
+ smb2_set_err_rsp(work);
+ return err;
+ }
+
ksmbd_close_session_fds(work);
- ksmbd_conn_wait_idle(conn);
if (ksmbd_tree_conn_session_logoff(sess)) {
ksmbd_debug(SMB, "Invalid tid %d\n", req->hdr.Id.SyncId.TreeId);
rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED;
smb2_set_err_rsp(work);
- return -ENOENT;
+ err = -ENOENT;
+ } else {
+ err = 0;
}
down_write(&conn->session_lock);
@@ -2832,6 +2861,9 @@ int smb2_session_logoff(struct ksmbd_work *work)
ksmbd_all_conn_set_status(sess, KSMBD_SESS_NEED_SETUP);
+ if (err)
+ return err;
+
rsp->StructureSize = cpu_to_le16(4);
err = ksmbd_iov_pin_rsp(work, rsp, sizeof(struct smb2_logoff_rsp));
if (err) {
@@ -8488,14 +8520,14 @@ int smb2_cancel(struct ksmbd_work *work)
* still on conn->async_requests with a live cancel_fn
* pointing at the freed file_lock.
*/
- if (iter->state != KSMBD_WORK_ACTIVE)
+ if (cmpxchg(&iter->state, KSMBD_WORK_ACTIVE,
+ KSMBD_WORK_CANCELLED) != KSMBD_WORK_ACTIVE)
break;
ksmbd_debug(SMB,
"smb2 with AsyncId %llu cancelled command = 0x%x\n",
le64_to_cpu(hdr->Id.AsyncId),
le16_to_cpu(chdr->Command));
- iter->state = KSMBD_WORK_CANCELLED;
if (iter->cancel_fn == smb2_notify_cancel_fn)
cancelled_notify =
smb2_notify_cancel_claim(iter->cancel_argv);
@@ -8524,11 +8556,16 @@ int smb2_cancel(struct ksmbd_work *work)
iter == work)
continue;
+ if (cmpxchg(&iter->state, KSMBD_WORK_ACTIVE,
+ KSMBD_WORK_CANCELLED) != KSMBD_WORK_ACTIVE)
+ break;
+
ksmbd_debug(SMB,
"smb2 with mid %llu cancelled command = 0x%x\n",
le64_to_cpu(hdr->MessageId),
le16_to_cpu(chdr->Command));
- iter->state = KSMBD_WORK_CANCELLED;
+ if (iter->cancel_fn)
+ iter->cancel_fn(iter->cancel_argv);
break;
}
spin_unlock(&conn->request_lock);
diff --git a/fs/smb/server/smbacl.c b/fs/smb/server/smbacl.c
index 33825c1356b4..ae1527aa6269 100644
--- a/fs/smb/server/smbacl.c
+++ b/fs/smb/server/smbacl.c
@@ -380,10 +380,10 @@ void free_acl_state(struct posix_acl_state *state)
kfree(state->groups);
}
-static void parse_dacl(struct mnt_idmap *idmap,
- struct smb_acl *pdacl, char *end_of_acl,
- struct smb_sid *pownersid, struct smb_sid *pgrpsid,
- struct smb_fattr *fattr)
+static int parse_dacl(struct mnt_idmap *idmap,
+ struct smb_acl *pdacl, char *end_of_acl,
+ struct smb_sid *pownersid, struct smb_sid *pgrpsid,
+ struct smb_fattr *fattr)
{
int i, ret;
u16 num_aces = 0;
@@ -397,13 +397,13 @@ static void parse_dacl(struct mnt_idmap *idmap,
bool owner_found = false, group_found = false, others_found = false;
if (!pdacl)
- return;
+ return 0;
/* validate that we do not go past end of acl */
if (end_of_acl < (char *)pdacl + sizeof(struct smb_acl) ||
end_of_acl < (char *)pdacl + le16_to_cpu(pdacl->size)) {
pr_err("ACL too small to parse DACL\n");
- return;
+ return -EINVAL;
}
ksmbd_debug(SMB, "DACL revision %d size %d num aces %d\n",
@@ -415,31 +415,31 @@ static void parse_dacl(struct mnt_idmap *idmap,
num_aces = le16_to_cpu(pdacl->num_aces);
if (num_aces <= 0)
- return;
+ return 0;
dacl_size = le16_to_cpu(pdacl->size);
if (dacl_size < sizeof(struct smb_acl))
- return;
+ return -EINVAL;
if (num_aces > (dacl_size - sizeof(struct smb_acl)) /
(offsetof(struct smb_ace, sid) +
offsetof(struct smb_sid, sub_auth) + sizeof(__le16)))
- return;
+ return -EINVAL;
ret = init_acl_state(&acl_state, num_aces);
if (ret)
- return;
+ return ret;
ret = init_acl_state(&default_acl_state, num_aces);
if (ret) {
free_acl_state(&acl_state);
- return;
+ return ret;
}
ppace = kmalloc_objs(struct smb_ace *, num_aces, KSMBD_DEFAULT_GFP);
if (!ppace) {
free_acl_state(&default_acl_state);
free_acl_state(&acl_state);
- return;
+ return -ENOMEM;
}
/*
@@ -448,8 +448,10 @@ static void parse_dacl(struct mnt_idmap *idmap,
* user/group/other have no permissions
*/
for (i = 0; i < num_aces; ++i) {
- if (end_of_acl - acl_base < acl_size)
- break;
+ if (end_of_acl - acl_base < acl_size) {
+ ret = -EINVAL;
+ goto out;
+ }
ppace[i] = (struct smb_ace *)(acl_base + acl_size);
acl_base = (char *)ppace[i];
@@ -462,8 +464,10 @@ static void parse_dacl(struct mnt_idmap *idmap,
(end_of_acl - acl_base <
acl_size + sizeof(__le32) * ppace[i]->sid.num_subauth) ||
(le16_to_cpu(ppace[i]->size) <
- acl_size + sizeof(__le32) * ppace[i]->sid.num_subauth))
- break;
+ acl_size + sizeof(__le32) * ppace[i]->sid.num_subauth)) {
+ ret = -EINVAL;
+ goto out;
+ }
acl_size = le16_to_cpu(ppace[i]->size);
ppace[i]->access_req =
@@ -521,8 +525,8 @@ static void parse_dacl(struct mnt_idmap *idmap,
temp_fattr.cf_uid = INVALID_UID;
ret = sid_to_id(idmap, &ppace[i]->sid, SIDOWNER, &temp_fattr);
if (ret || uid_eq(temp_fattr.cf_uid, INVALID_UID)) {
- pr_err("%s: Error %d mapping Owner SID to uid\n",
- __func__, ret);
+ pr_err_ratelimited("%s: Error %d mapping Owner SID to uid\n",
+ __func__, ret);
continue;
}
@@ -538,7 +542,6 @@ static void parse_dacl(struct mnt_idmap *idmap,
((acl_mode & 0700) >> 6) | 0004;
}
}
- kfree(ppace);
if (owner_found) {
/* The owner must be set to at least read-only. */
@@ -581,10 +584,12 @@ static void parse_dacl(struct mnt_idmap *idmap,
fattr->cf_acls =
posix_acl_alloc(acl_state.users->n +
acl_state.groups->n + 4, KSMBD_DEFAULT_GFP);
- if (fattr->cf_acls) {
- cf_pace = fattr->cf_acls->a_entries;
- posix_state_to_acl(&acl_state, cf_pace);
+ if (!fattr->cf_acls) {
+ ret = -ENOMEM;
+ goto out;
}
+ cf_pace = fattr->cf_acls->a_entries;
+ posix_state_to_acl(&acl_state, cf_pace);
}
}
@@ -595,14 +600,20 @@ static void parse_dacl(struct mnt_idmap *idmap,
fattr->cf_dacls =
posix_acl_alloc(default_acl_state.users->n +
default_acl_state.groups->n + 4, KSMBD_DEFAULT_GFP);
- if (fattr->cf_dacls) {
- cf_pdace = fattr->cf_dacls->a_entries;
- posix_state_to_acl(&default_acl_state, cf_pdace);
+ if (!fattr->cf_dacls) {
+ ret = -ENOMEM;
+ goto out;
}
+ cf_pdace = fattr->cf_dacls->a_entries;
+ posix_state_to_acl(&default_acl_state, cf_pdace);
}
}
+ ret = 0;
+out:
+ kfree(ppace);
free_acl_state(&acl_state);
free_acl_state(&default_acl_state);
+ return ret;
}
static void set_posix_acl_entries_dacl(struct mnt_idmap *idmap,
@@ -963,8 +974,10 @@ int parse_sec_desc(struct mnt_idmap *idmap, struct smb_ntsd *pntsd,
if (dacloffset < sizeof(struct smb_ntsd))
return -EINVAL;
- parse_dacl(idmap, dacl_ptr, end_of_acl,
- owner_sid_ptr, group_sid_ptr, fattr);
+ rc = parse_dacl(idmap, dacl_ptr, end_of_acl,
+ owner_sid_ptr, group_sid_ptr, fattr);
+ if (rc)
+ return rc;
}
return 0;
diff --git a/fs/smb/server/transport_tcp.c b/fs/smb/server/transport_tcp.c
index 990b14e5d3ea..668fc333c5be 100644
--- a/fs/smb/server/transport_tcp.c
+++ b/fs/smb/server/transport_tcp.c
@@ -39,6 +39,7 @@ struct tcp_transport {
static const struct ksmbd_transport_ops ksmbd_tcp_transport_ops;
static void tcp_stop_kthread(struct task_struct *kthread);
+static void ksmbd_tcp_stop_listener(struct interface *iface);
static struct interface *alloc_iface(char *ifname);
static void ksmbd_tcp_disconnect(struct ksmbd_transport *t);
@@ -315,13 +316,20 @@ static int ksmbd_tcp_run_kthread(struct interface *iface)
int rc;
struct task_struct *kthread;
- kthread = kthread_run(ksmbd_kthread_fn, (void *)iface, "ksmbd-%s",
- iface->name);
+ kthread = kthread_create(ksmbd_kthread_fn, (void *)iface, "ksmbd-%s",
+ iface->name);
if (IS_ERR(kthread)) {
rc = PTR_ERR(kthread);
return rc;
}
+
+ /*
+ * The listener can exit after its socket is shutdown, so keep the
+ * task_struct alive until the caller has stopped it.
+ */
+ get_task_struct(kthread);
iface->ksmbd_kthread = kthread;
+ wake_up_process(kthread);
return 0;
}
@@ -585,12 +593,7 @@ static int ksmbd_netdev_event(struct notifier_block *nb, unsigned long event,
if (iface && iface->state == IFACE_STATE_CONFIGURED) {
ksmbd_debug(CONN, "netdev-down event: netdev(%s) is going down\n",
iface->name);
- kernel_sock_shutdown(iface->ksmbd_socket, SHUT_RDWR);
- tcp_stop_kthread(iface->ksmbd_kthread);
- iface->ksmbd_kthread = NULL;
- sock_release(iface->ksmbd_socket);
- iface->ksmbd_socket = NULL;
-
+ ksmbd_tcp_stop_listener(iface);
iface->state = IFACE_STATE_DOWN;
break;
}
@@ -618,11 +621,25 @@ static void tcp_stop_kthread(struct task_struct *kthread)
if (!kthread)
return;
- ret = kthread_stop(kthread);
+ ret = kthread_stop_put(kthread);
if (ret)
pr_err("failed to stop forker thread\n");
}
+static void ksmbd_tcp_stop_listener(struct interface *iface)
+{
+ if (iface->ksmbd_socket)
+ kernel_sock_shutdown(iface->ksmbd_socket, SHUT_RDWR);
+
+ tcp_stop_kthread(iface->ksmbd_kthread);
+ iface->ksmbd_kthread = NULL;
+
+ if (iface->ksmbd_socket) {
+ sock_release(iface->ksmbd_socket);
+ iface->ksmbd_socket = NULL;
+ }
+}
+
void ksmbd_tcp_destroy(void)
{
struct interface *iface, *tmp;
@@ -630,6 +647,7 @@ void ksmbd_tcp_destroy(void)
unregister_netdevice_notifier(&ksmbd_netdev_notifier);
list_for_each_entry_safe(iface, tmp, &iface_list, entry) {
+ ksmbd_tcp_stop_listener(iface);
list_del(&iface->entry);
kfree(iface->name);
kfree(iface);
diff --git a/fs/smb/server/vfs_cache.c b/fs/smb/server/vfs_cache.c
index 293dab9b43be..a23bd9b1a68b 100644
--- a/fs/smb/server/vfs_cache.c
+++ b/fs/smb/server/vfs_cache.c
@@ -727,12 +727,25 @@ static void set_close_state_blocked_works(struct ksmbd_file *fp)
spin_lock(&fp->f_lock);
list_for_each_entry(cancel_work, &fp->blocked_works,
fp_entry) {
- cancel_work->state = KSMBD_WORK_CLOSED;
- cancel_work->cancel_fn(cancel_work->cancel_argv);
+ if (xchg(&cancel_work->state, KSMBD_WORK_CLOSED) ==
+ KSMBD_WORK_ACTIVE)
+ cancel_work->cancel_fn(cancel_work->cancel_argv);
}
spin_unlock(&fp->f_lock);
}
+void ksmbd_wake_session_blocked_works(struct ksmbd_session *sess)
+{
+ struct ksmbd_file_table *ft = &sess->file_table;
+ struct ksmbd_file *fp;
+ unsigned int id;
+
+ read_lock(&ft->lock);
+ idr_for_each_entry(ft->idr, fp, id)
+ set_close_state_blocked_works(fp);
+ read_unlock(&ft->lock);
+}
+
int ksmbd_close_fd(struct ksmbd_work *work, u64 id)
{
struct ksmbd_file *fp;
diff --git a/fs/smb/server/vfs_cache.h b/fs/smb/server/vfs_cache.h
index 8bb4396f9523..d6a9cd4b6d56 100644
--- a/fs/smb/server/vfs_cache.h
+++ b/fs/smb/server/vfs_cache.h
@@ -211,6 +211,7 @@ void ksmbd_launch_ksmbd_durable_scavenger(void);
void ksmbd_stop_durable_scavenger(void);
void ksmbd_close_tree_conn_fds(struct ksmbd_work *work);
void ksmbd_close_session_fds(struct ksmbd_work *work);
+void ksmbd_wake_session_blocked_works(struct ksmbd_session *sess);
int ksmbd_close_inode_fds(struct ksmbd_work *work, struct inode *inode);
int ksmbd_init_global_file_table(void);
void ksmbd_free_global_file_table(void);
diff --git a/fs/super.c b/fs/super.c
index d94f3a3e988d..1159a73856db 100644
--- a/fs/super.c
+++ b/fs/super.c
@@ -2122,11 +2122,14 @@ static int thaw_super_locked(struct super_block *sb, enum freeze_holder who,
goto out_unlock;
/*
- * All freezers share a single active reference.
- * So just unlock in case there are any left.
+ * All freezers share a single active reference. If other freezers
+ * remain, drop our hold and report success; the superblock stays
+ * frozen until the last holder thaws it.
*/
- if (freeze_dec(sb, who))
+ if (freeze_dec(sb, who)) {
+ error = 0;
goto out_unlock;
+ }
if (sb_rdonly(sb)) {
sb->s_writers.frozen = SB_UNFROZEN;
diff --git a/fs/ufs/cylinder.c b/fs/ufs/cylinder.c
index a2813270c303..b930ee1cf853 100644
--- a/fs/ufs/cylinder.c
+++ b/fs/ufs/cylinder.c
@@ -68,6 +68,16 @@ static bool ufs_read_cylinder(struct super_block *sb,
ucpi->c_clustersumoff = fs32_to_cpu(sb, ucg->cg_u.cg_44.cg_clustersumoff);
ucpi->c_clusteroff = fs32_to_cpu(sb, ucg->cg_u.cg_44.cg_clusteroff);
ucpi->c_nclusterblks = fs32_to_cpu(sb, ucg->cg_u.cg_44.cg_nclusterblks);
+
+ /* these on-disk values become array and bitmap indices */
+ if (ucpi->c_cgx != cgno ||
+ ucpi->c_rotor >= uspi->s_fpg ||
+ ucpi->c_frotor >= uspi->s_fpg ||
+ ucpi->c_irotor >= uspi->s_ipg) {
+ ufs_error(sb, __func__,
+ "inconsistent metadata in cylinder group %u\n", cgno);
+ goto failed;
+ }
UFSD("EXIT\n");
return true;
diff --git a/fs/ufs/dir.c b/fs/ufs/dir.c
index e62fe5667671..ce43cf20b07c 100644
--- a/fs/ufs/dir.c
+++ b/fs/ufs/dir.c
@@ -590,7 +590,7 @@ int ufs_empty_dir(struct inode * inode)
kaddr = ufs_get_folio(inode, i, &folio);
if (IS_ERR(kaddr))
- continue;
+ return 0;
de = (struct ufs_dir_entry *)kaddr;
kaddr += ufs_last_byte(inode, i) - UFS_DIR_REC_LEN(1);
diff --git a/fs/ufs/super.c b/fs/ufs/super.c
index c4831a8b9b3f..7deecb3954e2 100644
--- a/fs/ufs/super.c
+++ b/fs/ufs/super.c
@@ -1199,6 +1199,15 @@ static int ufs_fill_super(struct super_block *sb, struct fs_context *fc)
sb->s_maxbytes = ufs_max_bytes(sb);
sb->s_max_links = UFS_LINK_MAX;
+ ufs_setup_cstotal(sb);
+ /*
+ * Read cylinder group structures
+ */
+ if (!sb_rdonly(sb))
+ if (!ufs_read_cylinder_structures(sb))
+ goto failed;
+
+ /* create the root dentry last, once UFS_SB(sb) is fully set up */
inode = ufs_iget(sb, UFS_ROOTINO);
if (IS_ERR(inode)) {
ret = PTR_ERR(inode);
@@ -1210,14 +1219,6 @@ static int ufs_fill_super(struct super_block *sb, struct fs_context *fc)
goto failed;
}
- ufs_setup_cstotal(sb);
- /*
- * Read cylinder group structures
- */
- if (!sb_rdonly(sb))
- if (!ufs_read_cylinder_structures(sb))
- goto failed;
-
UFSD("EXIT\n");
return 0;
diff --git a/fs/xfs/libxfs/xfs_btree_mem.c b/fs/xfs/libxfs/xfs_btree_mem.c
index 37136a70e56d..1d83a4251cee 100644
--- a/fs/xfs/libxfs/xfs_btree_mem.c
+++ b/fs/xfs/libxfs/xfs_btree_mem.c
@@ -117,6 +117,7 @@ xfbtree_init(
struct xfs_buftarg *btp,
const struct xfs_btree_ops *ops)
{
+ unsigned long long owner = xfbt->owner;
unsigned int blocklen = xfbtree_rec_bytes(mp, ops);
unsigned int keyptr_len;
int error;
@@ -133,6 +134,7 @@ xfbtree_init(
memset(xfbt, 0, sizeof(*xfbt));
xfbt->target = btp;
+ xfbt->owner = owner;
/* Set up min/maxrecs for this btree. */
keyptr_len = ops->key_len + sizeof(__be64);
diff --git a/fs/xfs/libxfs/xfs_btree_staging.c b/fs/xfs/libxfs/xfs_btree_staging.c
index c3c7ea54895a..c5b7613e5216 100644
--- a/fs/xfs/libxfs/xfs_btree_staging.c
+++ b/fs/xfs/libxfs/xfs_btree_staging.c
@@ -337,8 +337,10 @@ xfs_btree_bload_prep_block(
xfs_btree_set_sibling(cur, *blockp, &new_ptr, XFS_BB_RIGHTSIB);
ret = xfs_btree_bload_drop_buf(bbl, buffers_list, bpp);
- if (ret)
+ if (ret) {
+ xfs_buf_relse(new_bp);
return ret;
+ }
/* Initialize the new btree block. */
xfs_btree_init_block_cur(cur, new_bp, level, nr_this_block);
diff --git a/fs/xfs/libxfs/xfs_da_btree.c b/fs/xfs/libxfs/xfs_da_btree.c
index 9debb95d86fa..98f02d68e938 100644
--- a/fs/xfs/libxfs/xfs_da_btree.c
+++ b/fs/xfs/libxfs/xfs_da_btree.c
@@ -130,7 +130,7 @@ xfs_da_state_reset(
state->mp = state->args->dp->i_mount;
}
-static inline int xfs_dabuf_nfsb(struct xfs_mount *mp, int whichfork)
+inline int xfs_dabuf_nfsb(struct xfs_mount *mp, int whichfork)
{
if (whichfork == XFS_DATA_FORK)
return mp->m_dir_geo->fsbcount;
diff --git a/fs/xfs/libxfs/xfs_da_btree.h b/fs/xfs/libxfs/xfs_da_btree.h
index afcf2d3c7a21..a718b1ceb0aa 100644
--- a/fs/xfs/libxfs/xfs_da_btree.h
+++ b/fs/xfs/libxfs/xfs_da_btree.h
@@ -244,4 +244,6 @@ xfs_failaddr_t xfs_da3_node_header_check(struct xfs_buf *bp, xfs_ino_t owner);
extern struct kmem_cache *xfs_da_state_cache;
+int xfs_dabuf_nfsb(struct xfs_mount *mp, int whichfork);
+
#endif /* __XFS_DA_BTREE_H__ */
diff --git a/fs/xfs/libxfs/xfs_defer.c b/fs/xfs/libxfs/xfs_defer.c
index 89501e8bd2f8..843c33304441 100644
--- a/fs/xfs/libxfs/xfs_defer.c
+++ b/fs/xfs/libxfs/xfs_defer.c
@@ -583,7 +583,7 @@ xfs_defer_finish_one(
const struct xfs_defer_op_type *ops = dfp->dfp_ops;
struct xfs_btree_cur *state = NULL;
struct list_head *li, *n;
- int error;
+ int error = 0;
trace_xfs_defer_pending_finish(tp->t_mountp, dfp);
diff --git a/fs/xfs/libxfs/xfs_exchmaps.c b/fs/xfs/libxfs/xfs_exchmaps.c
index 3efed37cb98a..49eda8d0994d 100644
--- a/fs/xfs/libxfs/xfs_exchmaps.c
+++ b/fs/xfs/libxfs/xfs_exchmaps.c
@@ -959,6 +959,16 @@ xmi_can_exchange_reflink_flags(
{
struct xfs_mount *mp = req->ip1->i_mount;
+ /*
+ * The INO1_WRITTEN optimization can skip exchanging hole and
+ * unwritten mappings, which means we cannot guarantee that all
+ * shared extents actually moved to the other file. Clearing the
+ * reflink flag of an inode that still holds shared extents breaks
+ * the CoW write path, so refuse to exchange the flags in that case.
+ */
+ if (req->flags & XFS_EXCHMAPS_INO1_WRITTEN)
+ return false;
+
/*
* The INO1_WRITTEN optimization can skip exchanging hole and
* unwritten mappings, which means we cannot guarantee that all
diff --git a/fs/xfs/libxfs/xfs_parent.c b/fs/xfs/libxfs/xfs_parent.c
index 8d111c9b6527..a2f2f5fa640e 100644
--- a/fs/xfs/libxfs/xfs_parent.c
+++ b/fs/xfs/libxfs/xfs_parent.c
@@ -193,7 +193,7 @@ xfs_parent_addname(
const struct xfs_name *parent_name,
struct xfs_inode *child)
{
- int error;
+ int error, local;
error = xfs_parent_iread_extents(tp, child);
if (error)
@@ -203,6 +203,10 @@ xfs_parent_addname(
xfs_parent_da_args_init(&ppargs->args, tp, &ppargs->rec, child,
I_INO(child), parent_name);
+ /* Growing the attr fork needs a real reservation in args->total. */
+ ppargs->args.total = xfs_attr_calc_size(&ppargs->args, &local);
+ ASSERT(local);
+
return xfs_attr_setname(&ppargs->args, 0);
}
@@ -239,7 +243,7 @@ xfs_parent_replacename(
const struct xfs_name *new_name,
struct xfs_inode *child)
{
- int error;
+ int error, local;
error = xfs_parent_iread_extents(tp, child);
if (error)
@@ -249,6 +253,10 @@ xfs_parent_replacename(
xfs_parent_da_args_init(&ppargs->args, tp, &ppargs->rec, child,
I_INO(child), old_name);
+ /* Growing the attr fork needs a real reservation in args->total. */
+ ppargs->args.total = xfs_attr_calc_size(&ppargs->args, &local);
+ ASSERT(local);
+
xfs_inode_to_parent_rec(&ppargs->new_rec, new_dp);
ppargs->args.new_name = new_name->name;
diff --git a/fs/xfs/libxfs/xfs_rtrefcount_btree.c b/fs/xfs/libxfs/xfs_rtrefcount_btree.c
index 22acc1411aac..e2950dbe2068 100644
--- a/fs/xfs/libxfs/xfs_rtrefcount_btree.c
+++ b/fs/xfs/libxfs/xfs_rtrefcount_btree.c
@@ -489,8 +489,11 @@ xfs_rtrefcountbt_maxlevels_ondisk(void)
minrecs[0] = xfs_rtrefcountbt_block_maxrecs(blocklen, true) / 2;
minrecs[1] = xfs_rtrefcountbt_block_maxrecs(blocklen, false) / 2;
- /* We need at most one record for every block in an rt group. */
- return xfs_btree_compute_maxlevels(minrecs, XFS_MAX_RGBLOCKS);
+ /*
+ * We need at most one record for every block in an rt group, and
+ * one extra level for the inode root.
+ */
+ return xfs_btree_compute_maxlevels(minrecs, XFS_MAX_RGBLOCKS) + 1;
}
int __init
diff --git a/fs/xfs/libxfs/xfs_rtrmap_btree.c b/fs/xfs/libxfs/xfs_rtrmap_btree.c
index c264bc5651c0..a15e460a1ec7 100644
--- a/fs/xfs/libxfs/xfs_rtrmap_btree.c
+++ b/fs/xfs/libxfs/xfs_rtrmap_btree.c
@@ -618,7 +618,7 @@ xfs_rtrmapbt_mem_cursor(
struct xfs_btree_cur *cur;
cur = xfs_btree_alloc_cursor(mp, tp, &xfs_rtrmapbt_mem_ops,
- mp->m_rtrmap_maxlevels, xfs_rtrmapbt_cur_cache);
+ xfs_rtrmapbt_maxlevels_ondisk(), xfs_rtrmapbt_cur_cache);
cur->bc_mem.xfbtree = xfbt;
cur->bc_nlevels = xfbt->nlevels;
cur->bc_group = xfs_group_hold(rtg_group(rtg));
@@ -716,10 +716,12 @@ xfs_rtrmapbt_maxlevels_ondisk(void)
* happens, which means that we must compute the max height based on
* what the btree will look like if it consumes almost all the blocks
* in the data device due to maximal sharing factor.
+ *
+ * Add one extra level for the inode root.
*/
max_dblocks = -1U; /* max ag count */
max_dblocks *= XFS_MAX_CRC_AG_BLOCKS;
- return xfs_btree_space_to_height(minrecs, max_dblocks);
+ return xfs_btree_space_to_height(minrecs, max_dblocks) + 1;
}
int __init
diff --git a/fs/xfs/scrub/agheader.c b/fs/xfs/scrub/agheader.c
index 1fa66aa68e16..fa5d32ec020a 100644
--- a/fs/xfs/scrub/agheader.c
+++ b/fs/xfs/scrub/agheader.c
@@ -418,6 +418,13 @@ xchk_superblock(
xchk_block_set_corrupt(sc, bp);
}
+ if (xfs_has_zoned(mp)) {
+ if (sb->sb_rtstart != cpu_to_be64(mp->m_sb.sb_rtstart))
+ xchk_block_set_corrupt(sc, bp);
+ if (sb->sb_rtreserved != cpu_to_be64(mp->m_sb.sb_rtreserved))
+ xchk_block_set_corrupt(sc, bp);
+ }
+
/* Everything else must be zero. */
sblen = xchk_superblock_ondisk_size(mp);
if (memchr_inv((char *)sb + sblen, 0, BBTOB(bp->b_length) - sblen))
diff --git a/fs/xfs/scrub/agheader_repair.c b/fs/xfs/scrub/agheader_repair.c
index 2104512f1ee1..a66b611588c4 100644
--- a/fs/xfs/scrub/agheader_repair.c
+++ b/fs/xfs/scrub/agheader_repair.c
@@ -668,14 +668,16 @@ xrep_agfl_init_header(
struct xfs_scrub *sc,
struct xfs_buf *agfl_bp,
struct xagb_bitmap *agfl_extents,
- xfs_agblock_t flcount)
+ xfs_agblock_t flcount,
+ struct xfs_agfl *old_agfl)
{
struct xrep_agfl_fill af = {
.sc = sc,
.flcount = flcount,
};
struct xfs_mount *mp = sc->mp;
- struct xfs_agfl *agfl;
+ struct xfs_agfl *agfl = XFS_BUF_TO_AGFL(agfl_bp);
+ const size_t agfl_sz = BBTOB(agfl_bp->b_length);
int error;
ASSERT(flcount <= xfs_agfl_size(mp));
@@ -684,8 +686,8 @@ xrep_agfl_init_header(
* Start rewriting the header by setting the bno[] array to
* NULLAGBLOCK, then setting AGFL header fields.
*/
- agfl = XFS_BUF_TO_AGFL(agfl_bp);
- memset(agfl, 0xFF, BBTOB(agfl_bp->b_length));
+ memcpy(old_agfl, agfl, agfl_sz);
+ memset(agfl, 0xFF, agfl_sz);
agfl->agfl_magicnum = cpu_to_be32(XFS_AGFL_MAGIC);
agfl->agfl_seqno = cpu_to_be32(pag_agno(sc->sa.pag));
uuid_copy(&agfl->agfl_uuid, &mp->m_sb.sb_meta_uuid);
@@ -697,16 +699,23 @@ xrep_agfl_init_header(
*/
xagb_bitmap_init(&af.used_extents);
af.agfl_bno = xfs_buf_to_agfl_bno(agfl_bp);
- xagb_bitmap_walk(agfl_extents, xrep_agfl_fill, &af);
+ error = xagb_bitmap_walk(agfl_extents, xrep_agfl_fill, &af);
+ if (error && error != -ECANCELED)
+ goto err_undo;
error = xagb_bitmap_disunion(agfl_extents, &af.used_extents);
if (error)
- return error;
+ goto err_undo;
/* Write new AGFL to disk. */
xfs_trans_buf_set_type(sc->tp, agfl_bp, XFS_BLFT_AGFL_BUF);
- xfs_trans_log_buf(sc->tp, agfl_bp, 0, BBTOB(agfl_bp->b_length) - 1);
+ xfs_trans_log_buf(sc->tp, agfl_bp, 0, agfl_sz - 1);
xagb_bitmap_destroy(&af.used_extents);
return 0;
+
+err_undo:
+ xagb_bitmap_destroy(&af.used_extents);
+ memcpy(agfl, old_agfl, agfl_sz);
+ return error;
}
/* Repair the AGFL. */
@@ -718,6 +727,7 @@ xrep_agfl(
struct xfs_mount *mp = sc->mp;
struct xfs_buf *agf_bp;
struct xfs_buf *agfl_bp;
+ struct xfs_agfl *old_agfl;
xfs_agblock_t flcount;
int error;
@@ -725,6 +735,10 @@ xrep_agfl(
if (!xfs_has_rmapbt(mp))
return -EOPNOTSUPP;
+ old_agfl = kzalloc(BBTOB(XFS_FSS_TO_BB(mp, 1)), XCHK_GFP_FLAGS);
+ if (!old_agfl)
+ return -ENOMEM;
+
xagb_bitmap_init(&agfl_extents);
/*
@@ -734,7 +748,7 @@ xrep_agfl(
*/
error = xfs_alloc_read_agf(sc->sa.pag, sc->tp, 0, &agf_bp);
if (error)
- return error;
+ goto err_old_agfl;
/*
* Make sure we have the AGFL buffer, as scrub might have decided it
@@ -745,7 +759,7 @@ xrep_agfl(
XFS_AGFL_DADDR(mp)),
XFS_FSS_TO_BB(mp, 1), 0, &agfl_bp, NULL);
if (error)
- return error;
+ goto err_old_agfl;
agfl_bp->b_ops = &xfs_agfl_buf_ops;
/* Gather all the extents we're going to put on the new AGFL. */
@@ -762,10 +776,11 @@ xrep_agfl(
* we adjust the AGF flcount (which can fail) so avoid updating any
* buffers until we know that part works.
*/
- xrep_agfl_update_agf(sc, agf_bp, flcount);
- error = xrep_agfl_init_header(sc, agfl_bp, &agfl_extents, flcount);
+ error = xrep_agfl_init_header(sc, agfl_bp, &agfl_extents, flcount,
+ old_agfl);
if (error)
goto err;
+ xrep_agfl_update_agf(sc, agf_bp, flcount);
/*
* Ok, the AGFL should be ready to go now. Roll the transaction to
@@ -785,6 +800,8 @@ xrep_agfl(
err:
xagb_bitmap_destroy(&agfl_extents);
+err_old_agfl:
+ kfree(old_agfl);
return error;
}
diff --git a/fs/xfs/scrub/alloc_repair.c b/fs/xfs/scrub/alloc_repair.c
index dce6ab0429dc..95e318e4f3a6 100644
--- a/fs/xfs/scrub/alloc_repair.c
+++ b/fs/xfs/scrub/alloc_repair.c
@@ -571,7 +571,7 @@ xrep_abt_dispose_one(
* allocation, and blocks that didn't get used can be freed via the usual
* (deferred) means.
*/
-STATIC void
+STATIC int
xrep_abt_dispose_reservations(
struct xrep_abt *ra,
int error)
@@ -582,9 +582,13 @@ xrep_abt_dispose_reservations(
goto junkit;
list_for_each_entry_safe(resv, n, &ra->new_bnobt.resv_list, list) {
- error = xrep_abt_dispose_one(ra, resv);
- if (error)
+ int error2 = xrep_abt_dispose_one(ra, resv);
+
+ if (error2) {
+ if (!error)
+ error = error2;
goto junkit;
+ }
}
junkit:
@@ -596,6 +600,7 @@ xrep_abt_dispose_reservations(
xrep_newbt_cancel(&ra->new_bnobt);
xrep_newbt_cancel(&ra->new_cntbt);
+ return error;
}
/* Retrieve free space data for bulk load. */
@@ -801,7 +806,9 @@ xrep_abt_build_new_trees(
goto err_newbt;
/* Dispose of any unused blocks and the accounting information. */
- xrep_abt_dispose_reservations(ra, error);
+ error = xrep_abt_dispose_reservations(ra, error);
+ if (error)
+ return error;
return xrep_roll_ag_trans(sc);
@@ -812,8 +819,7 @@ xrep_abt_build_new_trees(
xfs_btree_del_cursor(cnt_cur, error);
xfs_btree_del_cursor(bno_cur, error);
err_newbt:
- xrep_abt_dispose_reservations(ra, error);
- return error;
+ return xrep_abt_dispose_reservations(ra, error);
}
/*
diff --git a/fs/xfs/scrub/attr_repair.c b/fs/xfs/scrub/attr_repair.c
index 6e6af142f1fb..28f92e9ba72b 100644
--- a/fs/xfs/scrub/attr_repair.c
+++ b/fs/xfs/scrub/attr_repair.c
@@ -1294,7 +1294,7 @@ xrep_xattr_swap_prep(
.geo = sc->mp->m_attr_geo,
.whichfork = XFS_ATTR_FORK,
.trans = sc->tp,
- .total = 1,
+ .total = xfs_dabuf_nfsb(sc->mp, XFS_ATTR_FORK),
.owner = I_INO(sc->ip),
};
diff --git a/fs/xfs/scrub/bmap.c b/fs/xfs/scrub/bmap.c
index 401c278725d2..3b0f1dbd9147 100644
--- a/fs/xfs/scrub/bmap.c
+++ b/fs/xfs/scrub/bmap.c
@@ -274,7 +274,7 @@ xchk_bmap_xref_rmap_cow(
unsigned long long rmap_end;
uint64_t owner = XFS_RMAP_OWN_COW;
- if (!info->sc->sa.rmap_cur || xchk_skip_xref(info->sc->sm))
+ if (xchk_skip_xref(info->sc->sm))
return;
/* Find the rmap record for this irec. */
diff --git a/fs/xfs/scrub/dir_repair.c b/fs/xfs/scrub/dir_repair.c
index 1c088cfba10e..2cfcf1c35679 100644
--- a/fs/xfs/scrub/dir_repair.c
+++ b/fs/xfs/scrub/dir_repair.c
@@ -484,18 +484,24 @@ xrep_dir_recover_data(
while (offset < end) {
struct xfs_dir2_data_unused *dup = bp->b_addr + offset;
struct xfs_dir2_data_entry *dep = bp->b_addr + offset;
+ unsigned int advance;
if (xchk_should_terminate(rd->sc, &error))
return error;
/* Skip unused entries. */
if (be16_to_cpu(dup->freetag) == XFS_DIR2_DATA_FREE_TAG) {
+ if (!dup->length)
+ break;
offset += be16_to_cpu(dup->length);
continue;
}
/* Don't walk off the end of the block. */
- offset += xfs_dir2_data_entsize(rd->sc->mp, dep->namelen);
+ advance = xfs_dir2_data_entsize(rd->sc->mp, dep->namelen);
+ if (!advance)
+ break;
+ offset += advance;
if (offset > end)
break;
@@ -721,7 +727,7 @@ xrep_dir_replay_removename(
const struct xfs_name *name,
xfs_extlen_t total)
{
- struct xfs_inode *dp = rd->args.dp;
+ struct xfs_inode *dp = rd->sc->tempip;
ASSERT(S_ISDIR(VFS_I(dp)->i_mode));
@@ -1375,9 +1381,24 @@ xrep_dir_live_update(
if (p->delta > 0)
error = xrep_dir_stash_createname(rd, p->name,
I_INO(p->ip));
- else
- error = xrep_dir_stash_removename(rd, p->name,
+ else {
+ /*
+ * xfs_dentry_to_name in unlink or rename-exchange can
+ * pass us names with ftype FT_UNKNOWN, but we really
+ * must know the ftype of the child that is being
+ * removed so that we can do nlink updates correctly
+ * without holding inode references.
+ */
+ struct xfs_name name = {
+ .name = p->name->name,
+ .len = p->name->len,
+ .type = xfs_mode_to_ftype(
+ VFS_IC(p->ip)->i_mode),
+ };
+
+ error = xrep_dir_stash_removename(rd, &name,
I_INO(p->ip));
+ }
mutex_unlock(&rd->pscan.lock);
if (error)
goto out_abort;
@@ -1467,7 +1488,7 @@ xrep_dir_swap_prep(
.geo = sc->mp->m_dir_geo,
.whichfork = XFS_DATA_FORK,
.trans = sc->tp,
- .total = 1,
+ .total = xfs_dabuf_nfsb(sc->mp, XFS_DATA_FORK),
.owner = I_INO(sc->ip),
};
diff --git a/fs/xfs/scrub/dirtree.c b/fs/xfs/scrub/dirtree.c
index b2cf6e5439d9..9b0ab2316612 100644
--- a/fs/xfs/scrub/dirtree.c
+++ b/fs/xfs/scrub/dirtree.c
@@ -259,6 +259,7 @@ xchk_dirtree_create_path(
dl->nr_paths++;
return 0;
out_path:
+ xino_bitmap_destroy(&path->seen_inodes);
kfree(path);
return error;
}
@@ -368,12 +369,38 @@ xchk_dirpath_step_up(
struct xfs_inode *dp;
xfs_ino_t parent_ino = be64_to_cpu(dl->pptr_rec.p_ino);
unsigned int lock_mode;
- int error;
+ int error = 0;
+
+ if (xchk_should_terminate(sc, &error))
+ return error;
/* Grab and lock the parent directory. */
error = xchk_iget(sc, parent_ino, &dp);
- if (error)
+ switch (error) {
+ case -EINVAL:
+ case -ENOENT:
+ mutex_lock(&dl->lock);
+
+ if (dl->stale) {
+ /* live update detected a change in this path */
+ error = -ESTALE;
+ } else {
+ /* inode doesn't exist, path invalid */
+ error = -EFSCORRUPTED;
+
+ trace_xchk_dirpath_badino(dl->sc, path->path_nr,
+ path->nr_steps, &dl->xname,
+ &dl->pptr_rec);
+ }
+
+ mutex_unlock(&dl->lock);
+ return error;
+ case 0:
+ /* keep going */
+ break;
+ default:
return error;
+ }
lock_mode = xfs_ilock_attr_map_shared(dp);
mutex_lock(&dl->lock);
diff --git a/fs/xfs/scrub/dirtree_repair.c b/fs/xfs/scrub/dirtree_repair.c
index bbf6acf6fd40..1d1eafcf6eb5 100644
--- a/fs/xfs/scrub/dirtree_repair.c
+++ b/fs/xfs/scrub/dirtree_repair.c
@@ -479,6 +479,7 @@ xrep_dirtree_unlink(
}
if (xfs_has_parent(sc->mp)) {
+ memset(&dl->ppargs, 0, sizeof(dl->ppargs));
error = xfs_parent_removename(sc->tp, &dl->ppargs, dp,
&dl->xname, sc->ip);
if (error)
@@ -618,6 +619,7 @@ xrep_dirtree_create_adoption_path(
return 0;
out_path:
+ xino_bitmap_destroy(&path->seen_inodes);
kfree(path);
return error;
}
diff --git a/fs/xfs/scrub/findparent.c b/fs/xfs/scrub/findparent.c
index 04b6b96b0a30..eab3ac2704be 100644
--- a/fs/xfs/scrub/findparent.c
+++ b/fs/xfs/scrub/findparent.c
@@ -139,32 +139,52 @@ xrep_findparent_dirent(
return 0;
}
-/*
- * If this is a directory, walk the dirents looking for any that point to the
- * scrub target inode.
- */
-STATIC int
-xrep_findparent_walk_directory(
- struct xrep_findparent_info *fpi)
+static inline bool
+xrep_findparent_want_scan_file(
+ const struct xrep_findparent_info *fpi)
{
- struct xfs_scrub *sc = fpi->sc;
- struct xfs_inode *dp = fpi->dp;
- unsigned int lock_mode;
- int error = 0;
+ const struct xfs_scrub *sc = fpi->sc;
+ const struct xfs_inode *dp = fpi->dp;
+
+ /* Only directories can be parents */
+ if (!S_ISDIR(VFS_IC(dp)->i_mode))
+ return false;
/*
* The inode being scanned cannot be its own parent, nor can any
* temporary directory we created to stage this repair.
*/
if (dp == sc->ip || dp == sc->tempip)
- return 0;
+ return false;
/*
* Similarly, temporary files created to stage a repair cannot be the
* parent of this inode.
*/
if (xrep_is_tempfile(dp))
+ return false;
+
+ return true;
+}
+
+/*
+ * If this is a directory, walk the dirents looking for any that point to the
+ * scrub target inode.
+ */
+STATIC int
+xrep_findparent_walk_file(
+ struct xrep_findparent_info *fpi)
+{
+ struct xfs_scrub *sc = fpi->sc;
+ struct xfs_inode *dp = fpi->dp;
+ unsigned int lock_mode;
+ int error = 0;
+
+ if (!xrep_findparent_want_scan_file(fpi)) {
+ if (fpi->parent_scan)
+ xchk_iscan_mark_visited(&fpi->parent_scan->iscan, dp);
return 0;
+ }
/*
* Scan the directory to see if there it contains an entry pointing to
@@ -201,6 +221,8 @@ xrep_findparent_walk_directory(
goto out_unlock;
out_unlock:
+ if (fpi->parent_scan)
+ xchk_iscan_mark_visited(&fpi->parent_scan->iscan, dp);
xfs_iunlock(dp, lock_mode);
return error;
}
@@ -308,11 +330,7 @@ xrep_findparent_scan(
ASSERT(S_ISDIR(VFS_IC(sc->ip)->i_mode));
while ((ret = xchk_iscan_iter(&pscan->iscan, &fpi.dp)) == 1) {
- if (S_ISDIR(VFS_I(fpi.dp)->i_mode))
- ret = xrep_findparent_walk_directory(&fpi);
- else
- ret = 0;
- xchk_iscan_mark_visited(&pscan->iscan, fpi.dp);
+ ret = xrep_findparent_walk_file(&fpi);
xchk_irele(sc, fpi.dp);
if (ret)
break;
@@ -401,7 +419,7 @@ xrep_findparent_confirm(
goto out_rele;
}
- error = xrep_findparent_walk_directory(&fpi);
+ error = xrep_findparent_walk_file(&fpi);
if (error)
goto out_rele;
diff --git a/fs/xfs/scrub/ialloc.c b/fs/xfs/scrub/ialloc.c
index 19c0b1b2a787..9270ad075fe0 100644
--- a/fs/xfs/scrub/ialloc.c
+++ b/fs/xfs/scrub/ialloc.c
@@ -85,6 +85,8 @@ xchk_inobt_xref_finobt(
goto no_record;
error = xfs_inobt_get_rec(cur, &frec, &has_record);
+ if (error)
+ return error;
if (!has_record)
return -EFSCORRUPTED;
@@ -188,6 +190,8 @@ xchk_finobt_xref_inobt(
goto no_record;
error = xfs_inobt_get_rec(cur, &irec, &has_record);
+ if (error)
+ return error;
if (!has_record)
return -EFSCORRUPTED;
diff --git a/fs/xfs/scrub/metapath.c b/fs/xfs/scrub/metapath.c
index ff1ff762b300..c1fa0e179653 100644
--- a/fs/xfs/scrub/metapath.c
+++ b/fs/xfs/scrub/metapath.c
@@ -397,7 +397,7 @@ xrep_metapath_unlink(
/* Figure out if we're removing a parent pointer too. */
if (xfs_has_parent(mp)) {
- xfs_inode_to_parent_rec(&rec, ip);
+ xfs_inode_to_parent_rec(&rec, mpath->dp);
error = xfs_parent_lookup(sc->tp, ip, &mpath->xname, &rec,
&mpath->pptr_args);
switch (error) {
@@ -556,6 +556,8 @@ xrep_metapath_try_unlink(
error = xchk_metapath_ilock_parent_and_child(mpath, ip);
if (error) {
xchk_trans_cancel(sc);
+ if (ip)
+ xchk_irele(sc, ip);
return error;
}
xfs_trans_ijoin(sc->tp, mpath->dp, 0);
diff --git a/fs/xfs/scrub/quota_repair.c b/fs/xfs/scrub/quota_repair.c
index 487bd4f68ebb..59302e8afc7e 100644
--- a/fs/xfs/scrub/quota_repair.c
+++ b/fs/xfs/scrub/quota_repair.c
@@ -325,7 +325,7 @@ xrep_quota_block(
* If there's nothing that would impede a dqiterate, we're
* done.
*/
- if ((ddq->d_type & XFS_DQTYPE_REC_MASK) != dqtype ||
+ if ((ddq->d_type & XFS_DQTYPE_REC_MASK) == dqtype &&
id == be32_to_cpu(ddq->d_id)) {
xfs_trans_brelse(sc->tp, bp);
return 0;
@@ -363,11 +363,18 @@ xrep_quota_block(
ddq->d_rtbcount, &ddq->d_rtbtimer,
defq->rtb.time);
+ /*
+ * This transaction operates on raw disk buffers, so we don't
+ * have a dquot log item to assign the LSN for us. Instead,
+ * set it to zero so that log recovery will always replay any
+ * logged dquot item atop this buffer.
+ */
+ dqblk->dd_lsn = 0;
+
/* We only support v5 filesystems so always set these. */
uuid_copy(&dqblk->dd_uuid, &sc->mp->m_sb.sb_meta_uuid);
xfs_update_cksum((char *)dqblk, sizeof(struct xfs_dqblk),
XFS_DQUOT_CRC_OFF);
- dqblk->dd_lsn = 0;
}
switch (dqtype) {
case XFS_DQTYPE_USER:
@@ -455,8 +462,7 @@ xrep_quota_data_fork(
if (truncate) {
/* Erase everything after the block containing the max dquot */
- error = xfs_bunmapi_range(&sc->tp, sc->ip, 0,
- max_dqid_off * sc->mp->m_sb.sb_blocksize,
+ error = xfs_bunmapi_range(&sc->tp, sc->ip, 0, max_dqid_off + 1,
XFS_MAX_FILEOFF);
if (error)
goto out;
diff --git a/fs/xfs/scrub/quotacheck.c b/fs/xfs/scrub/quotacheck.c
index c199d128538e..c32030a05440 100644
--- a/fs/xfs/scrub/quotacheck.c
+++ b/fs/xfs/scrub/quotacheck.c
@@ -263,8 +263,10 @@ xqcheck_mod_live_ino_dqtrx(
dqa->tx_id = p->tx_id;
error = rhashtable_insert_fast(&xqc->shadow_dquot_acct,
&dqa->hash, xqcheck_dqacct_hash_params);
- if (error)
+ if (error) {
+ kfree(dqa);
goto out_abort;
+ }
}
/* Find the shadow dqtrx (or an empty slot) here. */
diff --git a/fs/xfs/scrub/reap.c b/fs/xfs/scrub/reap.c
index fcd14c1703ea..f698b9be3dd1 100644
--- a/fs/xfs/scrub/reap.c
+++ b/fs/xfs/scrub/reap.c
@@ -601,7 +601,7 @@ xreap_configure_agextent_limits(
/* Maximum overhead of invalidating one buffer. */
const unsigned int per_binval =
- xfs_buf_inval_log_space(1, XFS_B_TO_FSBT(mp, max_binval));
+ xfs_buf_inval_log_space(1, XFS_FSB_TO_B(mp, max_binval));
/*
* For each transaction in a reap chain, we can delete some number of
@@ -680,7 +680,7 @@ xreap_configure_agcow_limits(
/* Overhead of invalidating one buffer */
const unsigned int per_binval =
- xfs_buf_inval_log_space(1, XFS_B_TO_FSBT(mp, max_binval));
+ xfs_buf_inval_log_space(1, XFS_FSB_TO_B(mp, max_binval));
/*
* For each transaction in a reap chain, we can delete some number of
@@ -1399,7 +1399,7 @@ xreap_bmapi_binval(
* far we've gotten.
*/
if (!xreap_inc_binval(rs)) {
- imap->br_blockcount = agbno_next - bno;
+ imap->br_blockcount = bno - agbno;
goto out;
}
}
diff --git a/fs/xfs/scrub/refcount.c b/fs/xfs/scrub/refcount.c
index 4e1bf23e5b89..f8c51d8fbb3d 100644
--- a/fs/xfs/scrub/refcount.c
+++ b/fs/xfs/scrub/refcount.c
@@ -410,7 +410,7 @@ xchk_refcount_mergeable(
const struct xfs_refcount_irec *r1 = &rrc->prev_rec;
/* Ignore if prev_rec is not yet initialized. */
- if (r1->rc_blockcount > 0)
+ if (r1->rc_blockcount == 0)
return false;
if (r1->rc_domain != r2->rc_domain)
@@ -581,8 +581,12 @@ xchk_xref_is_cow_staging(
if (rc.rc_domain != XFS_REFC_DOMAIN_COW)
xchk_btree_xref_set_corrupt(sc, sc->sa.refc_cur, 0);
+ /* Can't start after bno */
+ if (rc.rc_startblock > agbno)
+ xchk_btree_xref_set_corrupt(sc, sc->sa.refc_cur, 0);
+
/* Must be at least as long as what was passed in */
- if (rc.rc_blockcount < len)
+ if (rc.rc_startblock + rc.rc_blockcount < agbno + len)
xchk_btree_xref_set_corrupt(sc, sc->sa.refc_cur, 0);
}
diff --git a/fs/xfs/scrub/rgsuper.c b/fs/xfs/scrub/rgsuper.c
index 2bd2c0351b35..6e2abe5dc27c 100644
--- a/fs/xfs/scrub/rgsuper.c
+++ b/fs/xfs/scrub/rgsuper.c
@@ -36,8 +36,10 @@ xchk_rgsuperblock_xref(
if (sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT)
return;
- xchk_xref_is_used_rt_space(sc, xfs_rgbno_to_rtb(sc->sr.rtg, 0), 1);
- xchk_xref_is_only_rt_owned_by(sc, 0, 1, &XFS_RMAP_OINFO_FS);
+ xchk_xref_is_used_rt_space(sc, xfs_rgbno_to_rtb(sc->sr.rtg, 0),
+ sc->mp->m_sb.sb_rextsize);
+ xchk_xref_is_only_rt_owned_by(sc, 0, sc->mp->m_sb.sb_rextsize,
+ &XFS_RMAP_OINFO_FS);
}
int
diff --git a/fs/xfs/scrub/rtrefcount.c b/fs/xfs/scrub/rtrefcount.c
index 4e7c540c8d23..3d916d71a135 100644
--- a/fs/xfs/scrub/rtrefcount.c
+++ b/fs/xfs/scrub/rtrefcount.c
@@ -20,6 +20,7 @@
#include "xfs_metafile.h"
#include "xfs_rtrefcount_btree.h"
#include "xfs_rtalloc.h"
+#include "xfs_ag.h"
#include "scrub/scrub.h"
#include "scrub/common.h"
#include "scrub/btree.h"
@@ -375,7 +376,7 @@ xchk_rtrefcount_mergeable(
const struct xfs_refcount_irec *r1 = &rrc->prev_rec;
/* Ignore if prev_rec is not yet initialized. */
- if (r1->rc_blockcount > 0)
+ if (r1->rc_blockcount == 0)
return false;
if (r1->rc_startblock + r1->rc_blockcount != r2->rc_startblock)
@@ -428,7 +429,7 @@ static inline void
xchk_rtrefcountbt_xref_gaps(
struct xfs_scrub *sc,
struct xchk_rtrefcbt_records *rrc,
- xfs_rtblock_t bno)
+ xfs_rgblock_t bno)
{
struct xfs_rmap_irec low;
struct xfs_rmap_irec high;
@@ -504,30 +505,75 @@ xchk_rtrefcountbt_rec(
return 0;
}
+/* Count the number of blocks used by the rtrefcount btree file in this AG. */
+static int
+xchk_rtrefcount_count_agblocks(
+ struct xfs_scrub *sc,
+ xfs_agnumber_t agno,
+ const struct xfs_owner_info *btree_oinfo,
+ xfs_filblks_t *blocks)
+{
+ xfs_filblks_t agblocks = 0;
+ int error;
+
+ error = xchk_ag_init_existing(sc, agno, &sc->sa);
+ if (error)
+ goto out_free;
+
+ /*
+ * If we don't have an rmap cursor, we can't complete the cross
+ * referencing, so return EFSCORRUPTED to end the loop and trigger the
+ * XFAIL flag.
+ */
+ if (!sc->sa.rmap_cur) {
+ error = -EFSCORRUPTED;
+ goto out_free;
+ }
+
+ error = xchk_count_rmap_ownedby_ag(sc, sc->sa.rmap_cur, btree_oinfo,
+ &agblocks);
+ if (error)
+ goto out_free;
+
+ *blocks += agblocks;
+out_free:
+ xchk_ag_free(sc, &sc->sa);
+ return error;
+}
+
/* Make sure we have as many refc blocks as the rmap says. */
STATIC void
-xchk_refcount_xref_rmap(
+xchk_rtrefcount_xref_rmap(
struct xfs_scrub *sc,
const struct xfs_owner_info *btree_oinfo,
xfs_extlen_t cow_blocks)
{
xfs_filblks_t refcbt_blocks = 0;
- xfs_filblks_t blocks;
- int error;
+ xfs_filblks_t blocks = 1; /* one for the iroot */
+ xfs_agnumber_t agno;
+ int error = 0;
- if (!sc->sr.rmap_cur || !sc->sa.rmap_cur || xchk_skip_xref(sc->sm))
+ if (!xfs_has_rmapbt(sc->mp) || xchk_skip_xref(sc->sm))
return;
/* Check that we saw as many refcbt blocks as the rmap knows about. */
error = xfs_btree_count_blocks(sc->sr.refc_cur, &refcbt_blocks);
if (!xchk_btree_process_error(sc, sc->sr.refc_cur, 0, &error))
return;
- error = xchk_count_rmap_ownedby_ag(sc, sc->sa.rmap_cur, btree_oinfo,
- &blocks);
- if (!xchk_should_check_xref(sc, &error, &sc->sa.rmap_cur))
+
+ for (agno = 0; agno < sc->mp->m_sb.sb_agcount; agno++) {
+ error = xchk_rtrefcount_count_agblocks(sc, agno, btree_oinfo,
+ &blocks);
+ if (error)
+ break;
+ }
+ if (!xchk_fblock_xref_process_error(sc, XFS_DATA_FORK, 0, &error))
return;
if (blocks != refcbt_blocks)
- xchk_btree_xref_set_corrupt(sc, sc->sa.rmap_cur, 0);
+ xchk_fblock_xref_set_corrupt(sc, XFS_DATA_FORK, 0);
+
+ if (!sc->sr.rmap_cur || xchk_skip_xref(sc->sm))
+ return;
/* Check that we saw as many cow blocks as the rmap knows about. */
error = xchk_count_rmap_ownedby_ag(sc, sc->sr.rmap_cur,
@@ -538,7 +584,7 @@ xchk_refcount_xref_rmap(
xchk_btree_xref_set_corrupt(sc, sc->sr.rmap_cur, 0);
}
-/* Scrub the refcount btree for some AG. */
+/* Scrub the refcount btree for some rtgroup. */
int
xchk_rtrefcountbt(
struct xfs_scrub *sc)
@@ -564,11 +610,11 @@ xchk_rtrefcountbt(
/*
* Check that all blocks between the last refcount > 1 record and the
- * end of the rt volume have at most one reverse mapping.
+ * end of the rtgroup have at most one reverse mapping.
*/
- xchk_rtrefcountbt_xref_gaps(sc, &rrc, sc->mp->m_sb.sb_rblocks);
-
- xchk_refcount_xref_rmap(sc, &btree_oinfo, rrc.cow_blocks);
+ xchk_rtrefcountbt_xref_gaps(sc, &rrc,
+ xfs_rtx_to_rgbno(sc->sr.rtg, sc->mp->m_sb.sb_rgextents));
+ xchk_rtrefcount_xref_rmap(sc, &btree_oinfo, rrc.cow_blocks);
return 0;
}
@@ -609,8 +655,12 @@ xchk_xref_is_rt_cow_staging(
if (rc.rc_domain != XFS_REFC_DOMAIN_COW)
xchk_btree_xref_set_corrupt(sc, sc->sr.refc_cur, 0);
+ /* Can't start after bno */
+ if (rc.rc_startblock > bno)
+ xchk_btree_xref_set_corrupt(sc, sc->sr.refc_cur, 0);
+
/* Must be at least as long as what was passed in */
- if (rc.rc_blockcount < len)
+ if (rc.rc_startblock + rc.rc_blockcount < bno + len)
xchk_btree_xref_set_corrupt(sc, sc->sr.refc_cur, 0);
}
diff --git a/fs/xfs/scrub/rtsummary_repair.c b/fs/xfs/scrub/rtsummary_repair.c
index f065c3e51ce2..ed763290aec1 100644
--- a/fs/xfs/scrub/rtsummary_repair.c
+++ b/fs/xfs/scrub/rtsummary_repair.c
@@ -164,9 +164,10 @@ xrep_rtsummary(
/*
* Now exchange the contents. Nothing in repair uses the temporary
* buffer, so we can reuse it for the tempfile exchrange information.
+ * Use XFS_MAX_FILEOFF here so that we correct the rtsummary file size.
*/
error = xrep_tempexch_trans_reserve(sc, XFS_DATA_FORK, 0,
- rts->rsumblocks, &rts->tempexch);
+ XFS_MAX_FILEOFF, &rts->tempexch);
if (error)
return error;
diff --git a/fs/xfs/scrub/scrub.c b/fs/xfs/scrub/scrub.c
index 8742445c86f4..12c228b7f477 100644
--- a/fs/xfs/scrub/scrub.c
+++ b/fs/xfs/scrub/scrub.c
@@ -765,8 +765,7 @@ xfs_scrub_metadata(
out_teardown:
error = xchk_teardown(sc, error);
out_sc:
- if (error != -ENOENT)
- xchk_stats_merge(mp, sm, &run);
+ xchk_stats_merge(mp, sm, error, &run);
kfree(sc);
out:
trace_xchk_done(XFS_I(file_inode(file)), sm, error);
diff --git a/fs/xfs/scrub/stats.c b/fs/xfs/scrub/stats.c
index ef3f6abdb706..da0c05ffe5cd 100644
--- a/fs/xfs/scrub/stats.c
+++ b/fs/xfs/scrub/stats.c
@@ -84,6 +84,7 @@ static const char *name_map[XFS_SCRUB_TYPE_NR] = {
[XFS_SCRUB_TYPE_RGSUPER] = "rgsuper",
[XFS_SCRUB_TYPE_RTRMAPBT] = "rtrmapbt",
[XFS_SCRUB_TYPE_RTREFCBT] = "rtrefcountbt",
+ [XFS_SCRUB_TYPE_HEALTHY] = "healthy",
};
/* Format the scrub stats into a text buffer, similar to pcp style. */
@@ -99,25 +100,31 @@ xchk_stats_format(
int ret = 0;
for (i = 0; i < XFS_SCRUB_TYPE_NR; i++, css++) {
+ struct xchk_scrub_stats fss;
+
if (!name_map[i])
continue;
+ spin_lock(&css->css_lock);
+ memcpy(&fss, css, offsetof(struct xchk_scrub_stats, css_lock));
+ spin_unlock(&css->css_lock);
+
ret = scnprintf(buf, remaining,
"%s %u %u %u %u %u %u %u %u %u %llu %u %u %llu\n",
name_map[i],
- (unsigned int)css->invocations,
- (unsigned int)css->clean,
- (unsigned int)css->corrupt,
- (unsigned int)css->preen,
- (unsigned int)css->xfail,
- (unsigned int)css->xcorrupt,
- (unsigned int)css->incomplete,
- (unsigned int)css->warning,
- (unsigned int)css->retries,
- (unsigned long long)css->checktime_us,
- (unsigned int)css->repair_invocations,
- (unsigned int)css->repair_success,
- (unsigned long long)css->repairtime_us);
+ (unsigned int)fss.invocations,
+ (unsigned int)fss.clean,
+ (unsigned int)fss.corrupt,
+ (unsigned int)fss.preen,
+ (unsigned int)fss.xfail,
+ (unsigned int)fss.xcorrupt,
+ (unsigned int)fss.incomplete,
+ (unsigned int)fss.warning,
+ (unsigned int)fss.retries,
+ (unsigned long long)fss.checktime_us,
+ (unsigned int)fss.repair_invocations,
+ (unsigned int)fss.repair_success,
+ (unsigned long long)fss.repairtime_us);
if (ret <= 0)
break;
@@ -188,31 +195,37 @@ STATIC void
xchk_stats_merge_one(
struct xchk_stats *cs,
const struct xfs_scrub_metadata *sm,
+ int error,
const struct xchk_stats_run *run)
{
struct xchk_scrub_stats *css;
+ unsigned int sm_flags = sm->sm_flags;
if (sm->sm_type >= XFS_SCRUB_TYPE_NR) {
ASSERT(sm->sm_type < XFS_SCRUB_TYPE_NR);
return;
}
+ /* caller applies this same transformation after we return */
+ if (error == -EFSCORRUPTED || error == -EFSBADCRC)
+ sm_flags |= XFS_SCRUB_OFLAG_CORRUPT;
+
css = &cs->cs_stats[sm->sm_type];
spin_lock(&css->css_lock);
css->invocations++;
- if (!(sm->sm_flags & XFS_SCRUB_OFLAG_UNCLEAN))
+ if (!(sm_flags & XFS_SCRUB_OFLAG_UNCLEAN))
css->clean++;
- if (sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT)
+ if (sm_flags & XFS_SCRUB_OFLAG_CORRUPT)
css->corrupt++;
- if (sm->sm_flags & XFS_SCRUB_OFLAG_PREEN)
+ if (sm_flags & XFS_SCRUB_OFLAG_PREEN)
css->preen++;
- if (sm->sm_flags & XFS_SCRUB_OFLAG_XFAIL)
+ if (sm_flags & XFS_SCRUB_OFLAG_XFAIL)
css->xfail++;
- if (sm->sm_flags & XFS_SCRUB_OFLAG_XCORRUPT)
+ if (sm_flags & XFS_SCRUB_OFLAG_XCORRUPT)
css->xcorrupt++;
- if (sm->sm_flags & XFS_SCRUB_OFLAG_INCOMPLETE)
+ if (sm_flags & XFS_SCRUB_OFLAG_INCOMPLETE)
css->incomplete++;
- if (sm->sm_flags & XFS_SCRUB_OFLAG_WARNING)
+ if (sm_flags & XFS_SCRUB_OFLAG_WARNING)
css->warning++;
css->retries += run->retries;
css->checktime_us += howmany_64(run->scrub_ns, NSEC_PER_USEC);
@@ -230,10 +243,14 @@ void
xchk_stats_merge(
struct xfs_mount *mp,
const struct xfs_scrub_metadata *sm,
+ int error,
const struct xchk_stats_run *run)
{
- xchk_stats_merge_one(&global_stats, sm, run);
- xchk_stats_merge_one(mp->m_scrub_stats, sm, run);
+ if (error == -ENOENT)
+ return;
+
+ xchk_stats_merge_one(&global_stats, sm, error, run);
+ xchk_stats_merge_one(mp->m_scrub_stats, sm, error, run);
}
/* debugfs boilerplate */
diff --git a/fs/xfs/scrub/stats.h b/fs/xfs/scrub/stats.h
index b358ad8d8b90..221052b95dd0 100644
--- a/fs/xfs/scrub/stats.h
+++ b/fs/xfs/scrub/stats.h
@@ -27,7 +27,7 @@ void xchk_stats_register(struct xchk_stats *cs, struct dentry *parent);
void xchk_stats_unregister(struct xchk_stats *cs);
void xchk_stats_merge(struct xfs_mount *mp, const struct xfs_scrub_metadata *sm,
- const struct xchk_stats_run *run);
+ int error, const struct xchk_stats_run *run);
static inline u64 xchk_stats_now(void) { return ktime_get_ns(); }
static inline u64 xchk_stats_elapsed_ns(u64 since)
@@ -53,7 +53,7 @@ static inline u64 xchk_stats_elapsed_ns(u64 since)
# define xchk_stats_unregister(cs) ((void)0)
# define xchk_stats_now() (0)
# define xchk_stats_elapsed_ns(x) (0 * (x))
-# define xchk_stats_merge(mp, sm, run) ((void)0)
+# define xchk_stats_merge(mp, sm, error, run) ((void)0)
#endif /* CONFIG_XFS_ONLINE_SCRUB_STATS */
#endif /* __XFS_SCRUB_STATS_H__ */
diff --git a/fs/xfs/scrub/symlink_repair.c b/fs/xfs/scrub/symlink_repair.c
index 91c86ea0e0f1..181961364233 100644
--- a/fs/xfs/scrub/symlink_repair.c
+++ b/fs/xfs/scrub/symlink_repair.c
@@ -291,7 +291,7 @@ xrep_symlink_swap_prep(
if (error)
return error;
- xfs_trans_log_inode(sc->tp, sc->ip, 0);
+ xfs_trans_log_inode(sc->tp, sc->tempip, logflags);
error = xfs_defer_finish(&sc->tp);
if (error)
diff --git a/fs/xfs/scrub/tempfile.c b/fs/xfs/scrub/tempfile.c
index 98820003b929..59a9213a3c7d 100644
--- a/fs/xfs/scrub/tempfile.c
+++ b/fs/xfs/scrub/tempfile.c
@@ -649,6 +649,19 @@ xrep_tempexch_prep_request(
return 0;
}
+static inline unsigned int
+xrep_tempexch_estimate_sf_resblks(
+ struct xfs_scrub *sc,
+ int whichfork)
+{
+ /* repairing a symlink target */
+ if (S_ISLNK(VFS_I(sc->ip)->i_mode) && whichfork == XFS_DATA_FORK)
+ return 1;
+
+ /* everything else is a directory or an xattr structure */
+ return xfs_dabuf_nfsb(sc->mp, whichfork);
+}
+
/*
* Fill out the mapping exchange resource estimation structures in preparation
* for exchanging the contents of a metadata file that we've rebuilt in the
@@ -663,6 +676,8 @@ xrep_tempexch_estimate(
struct xfs_ifork *ifp;
struct xfs_ifork *tifp;
int whichfork = xfs_exchmaps_reqfork(req);
+ unsigned int sf_resblks =
+ xrep_tempexch_estimate_sf_resblks(sc, whichfork);
int state = 0;
/*
@@ -693,9 +708,9 @@ xrep_tempexch_estimate(
* plus the block we converted.
*/
req->ip1_bcount = sc->tempip->i_nblocks;
- req->ip2_bcount = 1;
+ req->ip2_bcount = sf_resblks;
req->nr_exchanges = 1 + tifp->if_nextents;
- req->resblks = 1;
+ req->resblks = sf_resblks;
break;
case 2:
/*
@@ -707,10 +722,10 @@ xrep_tempexch_estimate(
* is (worst case) the extent count of the file being repaired
* plus the block we converted.
*/
- req->ip1_bcount = 1;
+ req->ip1_bcount = sf_resblks;
req->ip2_bcount = sc->ip->i_nblocks;
req->nr_exchanges = 1 + ifp->if_nextents;
- req->resblks = 1;
+ req->resblks = sf_resblks;
break;
case 3:
/*
@@ -722,10 +737,10 @@ xrep_tempexch_estimate(
* fileoff 0. Presumably, the caller could not exchange the
* two inode fork areas directly.
*/
- req->ip1_bcount = 1;
- req->ip2_bcount = 1;
+ req->ip1_bcount = sf_resblks;
+ req->ip2_bcount = sf_resblks;
req->nr_exchanges = 1;
- req->resblks = 2;
+ req->resblks = 2 * sf_resblks;
break;
}
diff --git a/fs/xfs/scrub/trace.h b/fs/xfs/scrub/trace.h
index 14aa0ec1f09e..0f5adc293962 100644
--- a/fs/xfs/scrub/trace.h
+++ b/fs/xfs/scrub/trace.h
@@ -1640,7 +1640,7 @@ DECLARE_EVENT_CLASS(xchk_pptr_class,
__entry->dev = ip->i_mount->m_super->s_dev;
__entry->ino = I_INO(ip);
__entry->namelen = name->len;
- memcpy(__get_str(name), name, name->len);
+ memcpy(__get_str(name), name->name, name->len);
__entry->far_ino = far_ino;
),
TP_printk("dev %d:%d ino 0x%llx name '%.*s' far_ino 0x%llx",
@@ -1706,6 +1706,39 @@ DEFINE_EVENT(xchk_dirtree_class, name, \
DEFINE_XCHK_DIRTREE_EVENT(xchk_dirtree_create_path);
DEFINE_XCHK_DIRTREE_EVENT(xchk_dirpath_walk_upwards);
+TRACE_EVENT(xchk_dirpath_badino,
+ TP_PROTO(struct xfs_scrub *sc, unsigned int path_nr,
+ unsigned int step_nr, const struct xfs_name *name,
+ const struct xfs_parent_rec *pptr),
+ TP_ARGS(sc, path_nr, step_nr, name, pptr),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(unsigned int, path_nr)
+ __field(unsigned int, step_nr)
+ __field(xfs_ino_t, parent_ino)
+ __field(unsigned int, parent_gen)
+ __field(unsigned int, namelen)
+ __dynamic_array(char, name, name->len)
+ ),
+ TP_fast_assign(
+ __entry->dev = sc->mp->m_super->s_dev;
+ __entry->path_nr = path_nr;
+ __entry->step_nr = step_nr;
+ __entry->parent_ino = be64_to_cpu(pptr->p_ino);
+ __entry->parent_gen = be32_to_cpu(pptr->p_gen);
+ __entry->namelen = name->len;
+ memcpy(__get_str(name), name->name, name->len);
+ ),
+ TP_printk("dev %d:%d path %u step %u parent_ino 0x%llx parent_gen 0x%x name '%.*s'",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->path_nr,
+ __entry->step_nr,
+ __entry->parent_ino,
+ __entry->parent_gen,
+ __entry->namelen,
+ __get_str(name))
+);
+
DECLARE_EVENT_CLASS(xchk_dirpath_class,
TP_PROTO(struct xfs_scrub *sc, struct xfs_inode *ip,
unsigned int path_nr, unsigned int step_nr,
diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c
index 48d7dfd3e15f..44e344fc51e9 100644
--- a/fs/xfs/xfs_buf.c
+++ b/fs/xfs/xfs_buf.c
@@ -108,7 +108,7 @@ xfs_buf_free(
ASSERT(list_empty(&bp->b_lru));
if (!xfs_buftarg_is_mem(bp->b_target) && size >= PAGE_SIZE)
- mm_account_reclaimed_pages(howmany(size, PAGE_SHIFT));
+ mm_account_reclaimed_pages(howmany(size, PAGE_SIZE));
if (is_vmalloc_addr(bp->b_addr))
vfree(bp->b_addr);
diff --git a/fs/xfs/xfs_exchmaps_item.c b/fs/xfs/xfs_exchmaps_item.c
index c3745d33e54e..dd5d92ca1010 100644
--- a/fs/xfs/xfs_exchmaps_item.c
+++ b/fs/xfs/xfs_exchmaps_item.c
@@ -344,7 +344,17 @@ xfs_xmi_validate(
if (!xfs_verify_fileext(mp, xlf->xmi_startoff1, xlf->xmi_blockcount))
return false;
- return xfs_verify_fileext(mp, xlf->xmi_startoff2, xlf->xmi_blockcount);
+ if (!xfs_verify_fileext(mp, xlf->xmi_startoff2, xlf->xmi_blockcount))
+ return false;
+
+ if (xlf->xmi_flags & XFS_EXCHMAPS_SET_SIZES) {
+ if ((int64_t)xlf->xmi_isize1 < 0)
+ return false;
+ if ((int64_t)xlf->xmi_isize2 < 0)
+ return false;
+ }
+
+ return true;
}
/*
@@ -403,6 +413,13 @@ xfs_xmi_item_recover_intent(
*ipp1 = ip1;
*ipp2 = ip2;
xmi = xfs_exchmaps_init_intent(req);
+
+ /* Restore intended file sizes from recovered logged item */
+ if (req->flags & XFS_EXCHMAPS_SET_SIZES) {
+ xmi->xmi_isize1 = xlf->xmi_isize1;
+ xmi->xmi_isize2 = xlf->xmi_isize2;
+ }
+
xfs_defer_add_item(dfp, &xmi->xmi_list);
return xmi;
diff --git a/fs/xfs/xfs_exchrange.c b/fs/xfs/xfs_exchrange.c
index 94965a6c2187..c69ecd6a19de 100644
--- a/fs/xfs/xfs_exchrange.c
+++ b/fs/xfs/xfs_exchrange.c
@@ -504,6 +504,9 @@ xfs_exchange_range_finish(
{
int error;
+ if (fxr->flags & XFS_EXCHANGE_RANGE_DRY_RUN)
+ return 0;
+
error = file_remove_privs(fxr->file1);
if (error)
return error;
@@ -783,9 +786,12 @@ xfs_exchange_range(
if (ret)
return ret;
- fsnotify_modify(fxr->file1);
- if (fxr->file2 != fxr->file1)
- fsnotify_modify(fxr->file2);
+ if (!(fxr->flags & XFS_EXCHANGE_RANGE_DRY_RUN)) {
+ fsnotify_modify(fxr->file1);
+ if (fxr->file2 != fxr->file1)
+ fsnotify_modify(fxr->file2);
+ }
+
return 0;
}
diff --git a/fs/xfs/xfs_healthmon.c b/fs/xfs/xfs_healthmon.c
index 4521ffdab9f1..acb85a1249a8 100644
--- a/fs/xfs/xfs_healthmon.c
+++ b/fs/xfs/xfs_healthmon.c
@@ -272,12 +272,13 @@ __xfs_healthmon_insert(
{
struct timespec64 now;
+ lockdep_assert_held(&hm->lock);
+
ktime_get_coarse_real_ts64(&now);
event->time_ns = (now.tv_sec * NSEC_PER_SEC) + now.tv_nsec;
event->next = hm->first_event;
- if (!hm->first_event)
- hm->first_event = event;
+ hm->first_event = event;
if (!hm->last_event)
hm->last_event = event;
xfs_healthmon_bump_events(hm);
@@ -294,6 +295,8 @@ __xfs_healthmon_push(
{
struct timespec64 now;
+ lockdep_assert_held(&hm->lock);
+
ktime_get_coarse_real_ts64(&now);
event->time_ns = (now.tv_sec * NSEC_PER_SEC) + now.tv_nsec;
@@ -330,8 +333,10 @@ xfs_healthmon_clear_lost_prev(
if (hm->events < XFS_HEALTHMON_MAX_EVENTS)
event = kmemdup(&lost_event, sizeof(struct xfs_healthmon_event),
GFP_NOFS);
- if (!event)
+ if (!event) {
+ xfs_healthmon_bump_lost(hm);
return -ENOMEM;
+ }
__xfs_healthmon_push(hm, event);
cleared:
@@ -415,8 +420,10 @@ xfs_healthmon_unmount(
* There's nothing actionable for userspace after an unmount. Once
* we've inserted the unmount event, hm no longer owns that event.
*/
+ mutex_lock(&hm->lock);
__xfs_healthmon_insert(hm, hm->unmount_event);
hm->unmount_event = NULL;
+ mutex_unlock(&hm->lock);
xfs_healthmon_detach(hm);
xfs_healthmon_put(hm);
@@ -738,6 +745,13 @@ static const unsigned int type_map[] = {
[XFS_HEALTHMON_DATALOST] = XFS_HEALTH_MONITOR_TYPE_DATALOST,
};
+static inline bool
+xfs_healthmon_check_outbuffer_space(const struct xfs_healthmon *hm)
+{
+ return hm->bufhead + sizeof(struct xfs_health_monitor_event) <=
+ hm->bufsize;
+}
+
/* Render event as a V0 structure */
STATIC int
xfs_healthmon_format_v0(
@@ -804,10 +818,10 @@ xfs_healthmon_format_v0(
break;
}
- ASSERT(hm->bufhead + sizeof(hme) <= hm->bufsize);
+ ASSERT(xfs_healthmon_check_outbuffer_space(hm));
/* copy formatted object to the outbuf */
- if (hm->bufhead + sizeof(hme) <= hm->bufsize) {
+ if (xfs_healthmon_check_outbuffer_space(hm)) {
memcpy(hm->buffer + hm->bufhead, &hme, sizeof(hme));
hm->bufhead += sizeof(hme);
}
@@ -890,7 +904,11 @@ xfs_healthmon_format_pop(
{
struct xfs_healthmon_event *event;
- if (hm->bufhead + sizeof(*event) > hm->bufsize)
+ /*
+ * Don't bother if there's not enough space to format even one event in
+ * the outbuffer.
+ */
+ if (!xfs_healthmon_check_outbuffer_space(hm))
return NULL;
mutex_lock(&hm->lock);
diff --git a/fs/xfs/xfs_verify_media.c b/fs/xfs/xfs_verify_media.c
index 5ead3976d511..b75c81f8fcc0 100644
--- a/fs/xfs/xfs_verify_media.c
+++ b/fs/xfs/xfs_verify_media.c
@@ -268,6 +268,8 @@ xfs_verify_media(
struct xfs_buftarg *btp = NULL;
struct bio *bio;
struct folio *folio;
+ xfs_daddr_t dev_start = 0;
+ xfs_daddr_t dev_end = 0;
xfs_daddr_t daddr;
uint64_t bbcount;
int error = 0;
@@ -277,24 +279,33 @@ xfs_verify_media(
switch (me->me_dev) {
case XFS_DEV_DATA:
btp = mp->m_ddev_targp;
+ dev_end = XFS_FSB_TO_BB(mp, mp->m_sb.sb_dblocks);
break;
case XFS_DEV_LOG:
- if (mp->m_logdev_targp != mp->m_ddev_targp)
+ if (mp->m_logdev_targp != mp->m_ddev_targp) {
btp = mp->m_logdev_targp;
+ dev_end = XFS_FSB_TO_BB(mp, mp->m_sb.sb_logblocks);
+ }
break;
case XFS_DEV_RT:
btp = mp->m_rtdev_targp;
+ dev_start = XFS_FSB_TO_BB(mp, mp->m_sb.sb_rtstart);
+ dev_end = XFS_FSB_TO_BB(mp, mp->m_sb.sb_rtstart +
+ mp->m_sb.sb_rblocks);
break;
}
if (!btp)
return -ENODEV;
/*
- * If the caller told us to verify beyond the end of the disk, tell the
- * user exactly where that was.
+ * If the caller told us to verify before the start or beyond the end
+ * of the disk volume, tell the user exactly where the volume starts
+ * and ends.
*/
- if (me->me_end_daddr > btp->bt_nr_sectors)
- me->me_end_daddr = btp->bt_nr_sectors;
+ if (me->me_end_daddr > dev_end)
+ me->me_end_daddr = dev_end;
+ if (me->me_start_daddr < dev_start)
+ me->me_start_daddr = dev_start;
/* start and end have to be aligned to the lba size */
if (!IS_ALIGNED(BBTOB(me->me_start_daddr | me->me_end_daddr),
@@ -323,8 +334,7 @@ xfs_verify_media(
* verifying.
*/
daddr = me->me_start_daddr;
- bbcount = min_t(sector_t, me->me_end_daddr, btp->bt_nr_sectors) -
- me->me_start_daddr;
+ bbcount = me->me_end_daddr - me->me_start_daddr;
folio = xfs_verify_alloc_folio(xfs_verify_iosize(me, btp, bbcount));
if (!folio)
diff --git a/include/drm/drm_pagemap.h b/include/drm/drm_pagemap.h
index 95eb4b66b057..ebbd3b0ddf36 100644
--- a/include/drm/drm_pagemap.h
+++ b/include/drm/drm_pagemap.h
@@ -2,6 +2,7 @@
#ifndef _DRM_PAGEMAP_H_
#define _DRM_PAGEMAP_H_
+#include <linux/bits.h>
#include <linux/dma-direction.h>
#include <linux/hmm.h>
#include <linux/memremap.h>
@@ -339,6 +340,9 @@ struct drm_pagemap_migrate_details {
#if IS_ENABLED(CONFIG_ZONE_DEVICE)
+#define DRM_PAGEMAP_ZDD_FLAG_MIGRATED BIT(0)
+#define DRM_PAGEMAP_ZDD_FLAG_MASK DRM_PAGEMAP_ZDD_FLAG_MIGRATED
+
int drm_pagemap_migrate_to_devmem(struct drm_pagemap_devmem *devmem_allocation,
struct mm_struct *mm,
unsigned long start, unsigned long end,
@@ -373,7 +377,9 @@ static inline struct drm_pagemap_zdd *drm_pagemap_page_zone_device_data(struct p
{
struct folio *folio = page_folio(page);
- return folio_zone_device_data(folio);
+ return (struct drm_pagemap_zdd *)
+ ((unsigned long)folio_zone_device_data(folio) &
+ ~DRM_PAGEMAP_ZDD_FLAG_MASK);
}
#else
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 77e2075f77c7..c2c327dc0a72 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -894,6 +894,7 @@ enum bpf_arg_type {
ARG_PTR_TO_CTX, /* pointer to context */
ARG_ANYTHING, /* any (initialized) argument is ok */
+ ARG_SCALAR, /* scalar argument */
ARG_PTR_TO_SPIN_LOCK, /* pointer to bpf_spin_lock */
ARG_PTR_TO_SOCK_COMMON, /* pointer to sock_common */
ARG_PTR_TO_SOCKET, /* pointer to bpf_sock (fullsock) */
@@ -4184,7 +4185,7 @@ static inline int bpf_map_check_op_flags(struct bpf_map *map, u64 flags, u64 all
return -EINVAL;
cpu = flags >> 32;
- if ((flags & BPF_F_CPU) && cpu >= num_possible_cpus())
+ if ((flags & BPF_F_CPU) && (cpu >= nr_cpu_ids || !cpu_possible(cpu)))
return -ERANGE;
}
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 39a851e690ec..7ad03ffb9300 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -385,6 +385,8 @@ struct bpf_func_state {
* | number of simulations is tracked in frame N
*/
u32 callback_depth;
+ /* Instructions processed in this frame and callees on the current path. */
+ u32 insns_subtotal;
/* The following fields should be last. See copy_func_state() */
/* The state of the stack. Each element of the array describes BPF_REG_SIZE
@@ -704,6 +706,8 @@ struct bpf_insn_aux_data {
*/
u32 calls_callback:1;
u32 indirect_target:1; /* if it is an indirect jump target */
+ /* true if some jump or call instruction targets this instruction */
+ u32 jump_target:1;
/*
* CFG strongly connected component this instruction belongs to,
* zero if it is a singleton SCC.
@@ -803,7 +807,8 @@ struct bpf_subprog_info {
u32 exit_idx; /* Index of one of the BPF_EXIT instructions in this subprogram */
u16 stack_depth; /* max. stack depth used by this function */
u16 stack_extra;
- u32 insn_processed;
+ u32 insns_total;
+ u32 insns_self;
/* offsets in range [stack_depth .. fastcall_stack_off)
* are used for bpf_fastcall spills and fills.
*/
@@ -1115,6 +1120,16 @@ static inline void mark_jmp_point(struct bpf_verifier_env *env, int idx)
env->insn_aux_data[idx].jmp_point = true;
}
+static inline void mark_jump_target(struct bpf_verifier_env *env, int idx)
+{
+ env->insn_aux_data[idx].jump_target = true;
+}
+
+static inline bool bpf_is_jump_target(struct bpf_verifier_env *env, int insn_idx)
+{
+ return env->insn_aux_data[insn_idx].jump_target;
+}
+
static inline struct bpf_func_state *cur_func(struct bpf_verifier_env *env)
{
struct bpf_verifier_state *cur = env->cur_state;
@@ -1307,7 +1322,9 @@ static inline bool bpf_type_has_unsafe_modifiers(u32 type)
static inline bool type_is_ptr_alloc_obj(u32 type)
{
- return base_type(type) == PTR_TO_BTF_ID && type_flag(type) & MEM_ALLOC;
+ return base_type(type) == PTR_TO_BTF_ID &&
+ type_flag(type) & MEM_ALLOC &&
+ !(type_flag(type) & PTR_UNTRUSTED);
}
static inline bool type_is_non_owning_ref(u32 type)
diff --git a/include/linux/designware_i2c.h b/include/linux/designware_i2c.h
new file mode 100644
index 000000000000..53f37f18a722
--- /dev/null
+++ b/include/linux/designware_i2c.h
@@ -0,0 +1,107 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Synopsys DesignWare I2C register definitions
+ *
+ * Copyright (C) 2026, Intel Corporation
+ */
+
+#ifndef __LINUX_DESIGNWARE_I2C_H
+#define __LINUX_DESIGNWARE_I2C_H
+
+#include <linux/bits.h>
+
+/*
+ * Registers offset
+ */
+#define DW_IC_CON 0x00
+#define DW_IC_TAR 0x04
+#define DW_IC_SAR 0x08
+#define DW_IC_DATA_CMD 0x10
+#define DW_IC_SS_SCL_HCNT 0x14
+#define DW_IC_SS_SCL_LCNT 0x18
+#define DW_IC_FS_SCL_HCNT 0x1c
+#define DW_IC_FS_SCL_LCNT 0x20
+#define DW_IC_HS_SCL_HCNT 0x24
+#define DW_IC_HS_SCL_LCNT 0x28
+#define DW_IC_INTR_STAT 0x2c
+#define DW_IC_INTR_MASK 0x30
+#define DW_IC_RAW_INTR_STAT 0x34
+#define DW_IC_RX_TL 0x38
+#define DW_IC_TX_TL 0x3c
+#define DW_IC_CLR_INTR 0x40
+#define DW_IC_CLR_RX_UNDER 0x44
+#define DW_IC_CLR_RX_OVER 0x48
+#define DW_IC_CLR_TX_OVER 0x4c
+#define DW_IC_CLR_RD_REQ 0x50
+#define DW_IC_CLR_TX_ABRT 0x54
+#define DW_IC_CLR_RX_DONE 0x58
+#define DW_IC_CLR_ACTIVITY 0x5c
+#define DW_IC_CLR_STOP_DET 0x60
+#define DW_IC_CLR_START_DET 0x64
+#define DW_IC_CLR_GEN_CALL 0x68
+#define DW_IC_ENABLE 0x6c
+#define DW_IC_STATUS 0x70
+#define DW_IC_TXFLR 0x74
+#define DW_IC_RXFLR 0x78
+#define DW_IC_SDA_HOLD 0x7c
+#define DW_IC_TX_ABRT_SOURCE 0x80
+#define DW_IC_ENABLE_STATUS 0x9c
+#define DW_IC_CLR_RESTART_DET 0xa8
+#define DW_IC_SMBUS_INTR_STAT 0xc8
+#define DW_IC_SMBUS_INTR_MASK 0xcc
+#define DW_IC_CLR_SMBUS_INTR 0xd4
+#define DW_IC_COMP_PARAM_1 0xf4
+#define DW_IC_COMP_VERSION 0xf8
+#define DW_IC_COMP_TYPE 0xfc
+
+/* DW_IC_CON bits */
+#define DW_IC_CON_MASTER BIT(0)
+#define DW_IC_CON_SPEED_STD (1 << 1)
+#define DW_IC_CON_SPEED_FAST (2 << 1)
+#define DW_IC_CON_SPEED_HIGH (3 << 1)
+#define DW_IC_CON_SPEED_MASK GENMASK(2, 1)
+#define DW_IC_CON_10BITADDR_SLAVE BIT(3)
+#define DW_IC_CON_10BITADDR_MASTER BIT(4)
+#define DW_IC_CON_RESTART_EN BIT(5)
+#define DW_IC_CON_SLAVE_DISABLE BIT(6)
+#define DW_IC_CON_STOP_DET_IFADDRESSED BIT(7)
+#define DW_IC_CON_TX_EMPTY_CTRL BIT(8)
+#define DW_IC_CON_RX_FIFO_FULL_HLD_CTRL BIT(9)
+#define DW_IC_CON_BUS_CLEAR_CTRL BIT(11)
+
+/* DW_IC_DATA_CMD bits */
+#define DW_IC_DATA_CMD_DAT GENMASK(7, 0)
+#define DW_IC_DATA_CMD_FIRST_DATA_BYTE BIT(11)
+
+/* DW_IC_INTR_* bits */
+#define DW_IC_INTR_RX_UNDER BIT(0)
+#define DW_IC_INTR_RX_OVER BIT(1)
+#define DW_IC_INTR_RX_FULL BIT(2)
+#define DW_IC_INTR_TX_OVER BIT(3)
+#define DW_IC_INTR_TX_EMPTY BIT(4)
+#define DW_IC_INTR_RD_REQ BIT(5)
+#define DW_IC_INTR_TX_ABRT BIT(6)
+#define DW_IC_INTR_RX_DONE BIT(7)
+#define DW_IC_INTR_ACTIVITY BIT(8)
+#define DW_IC_INTR_STOP_DET BIT(9)
+#define DW_IC_INTR_START_DET BIT(10)
+#define DW_IC_INTR_GEN_CALL BIT(11)
+#define DW_IC_INTR_RESTART_DET BIT(12)
+#define DW_IC_INTR_MST_ON_HOLD BIT(13)
+
+/* DW_IC_ENABLE bits */
+#define DW_IC_ENABLE_ENABLE BIT(0)
+#define DW_IC_ENABLE_ABORT BIT(1)
+
+/* DW_IC_STATUS bits */
+#define DW_IC_STATUS_ACTIVITY BIT(0)
+#define DW_IC_STATUS_TFE BIT(2)
+#define DW_IC_STATUS_RFNE BIT(3)
+#define DW_IC_STATUS_MASTER_ACTIVITY BIT(5)
+#define DW_IC_STATUS_SLAVE_ACTIVITY BIT(6)
+#define DW_IC_STATUS_MASTER_HOLD_TX_FIFO_EMPTY BIT(7)
+
+/* DW_IC_SMBUS_INTR_* bits */
+#define DW_IC_SMBUS_INTR_ALERT BIT(10)
+
+#endif /* __LINUX_DESIGNWARE_I2C_H */
diff --git a/include/linux/dma-fence-array.h b/include/linux/dma-fence-array.h
index 1b1d87579c38..0c49d7ccefb6 100644
--- a/include/linux/dma-fence-array.h
+++ b/include/linux/dma-fence-array.h
@@ -28,7 +28,6 @@ struct dma_fence_array_cb {
/**
* struct dma_fence_array - fence to represent an array of fences
* @base: fence base class
- * @lock: spinlock for fence handling
* @num_fences: number of fences in the array
* @num_pending: fences in the array still pending
* @fences: array of the fences
diff --git a/include/linux/dma-fence-chain.h b/include/linux/dma-fence-chain.h
index df3beadf1515..705c4394ac0d 100644
--- a/include/linux/dma-fence-chain.h
+++ b/include/linux/dma-fence-chain.h
@@ -20,7 +20,6 @@
* @prev: previous fence of the chain
* @prev_seqno: original previous seqno before garbage collection
* @fence: encapsulated fence
- * @lock: spinlock for fence handling
*/
struct dma_fence_chain {
struct dma_fence base;
@@ -81,9 +80,8 @@ dma_fence_chain_contained(struct dma_fence *fence)
}
/**
- * dma_fence_chain_alloc
- *
- * Returns a new struct dma_fence_chain object or NULL on failure.
+ * dma_fence_chain_alloc - Returns a new &struct dma_fence_chain object or
+ * %NULL on failure.
*
* This specialized allocator has to be a macro for its allocations to be
* accounted separately (to have a separate alloc_tag). The typecast is
@@ -93,7 +91,8 @@ dma_fence_chain_contained(struct dma_fence *fence)
kmalloc_obj(struct dma_fence_chain)
/**
- * dma_fence_chain_free
+ * dma_fence_chain_free - Frees an allocated but not used
+ * &struct dma_fence_chain object.
* @chain: chain node to free
*
* Frees up an allocated but not used struct dma_fence_chain object. This
diff --git a/include/linux/igmp.h b/include/linux/igmp.h
index 3a2d35a9f307..a0cf0398519f 100644
--- a/include/linux/igmp.h
+++ b/include/linux/igmp.h
@@ -57,20 +57,21 @@ struct ip_mc_socklist {
};
struct ip_sf_list {
- struct ip_sf_list *sf_next;
+ struct ip_sf_list __rcu *sf_next;
unsigned long sf_count[2]; /* include/exclude counts */
__be32 sf_inaddr;
unsigned char sf_gsresp; /* include in g & s response? */
unsigned char sf_oldin; /* change state */
unsigned char sf_crcount; /* retrans. left to send */
+ struct rcu_head rcu;
};
struct ip_mc_list {
struct in_device *interface;
__be32 multiaddr;
unsigned int sfmode;
- struct ip_sf_list *sources;
- struct ip_sf_list *tomb;
+ struct ip_sf_list __rcu *sources;
+ struct ip_sf_list __rcu *tomb;
unsigned long sfcount[2];
union {
struct ip_mc_list *next;
diff --git a/include/linux/mfd/db8500-prcmu.h b/include/linux/mfd/db8500-prcmu.h
index a62de3d155ed..c939c9a1170a 100644
--- a/include/linux/mfd/db8500-prcmu.h
+++ b/include/linux/mfd/db8500-prcmu.h
@@ -12,6 +12,9 @@
#include <linux/interrupt.h>
#include <linux/bitops.h>
+#include <linux/err.h>
+
+#include <dt-bindings/mfd/dbx500-prcmu.h> /* For clock identifiers */
/*
* Registers
@@ -24,6 +27,38 @@
#define DB8500_PRCM_DSI_SW_RESET_DSI1_SW_RESETN BIT(1)
#define DB8500_PRCM_DSI_SW_RESET_DSI2_SW_RESETN BIT(2)
+/* Offset for the firmware version within the TCPM */
+#define DB8500_PRCMU_FW_VERSION_OFFSET 0xA4
+
+#define DB8500_PRCMU_LEGACY_OFFSET 0xDD4
+
+/*
+ * CLKOUT sources
+ */
+#define PRCMU_CLKSRC_CLK38M 0x00
+#define PRCMU_CLKSRC_ACLK 0x01
+#define PRCMU_CLKSRC_SYSCLK 0x02
+#define PRCMU_CLKSRC_LCDCLK 0x03
+#define PRCMU_CLKSRC_SDMMCCLK 0x04
+#define PRCMU_CLKSRC_TVCLK 0x05
+#define PRCMU_CLKSRC_TIMCLK 0x06
+#define PRCMU_CLKSRC_CLK009 0x07
+/* These are only valid for CLKOUT1: */
+#define PRCMU_CLKSRC_SIAMMDSPCLK 0x40
+#define PRCMU_CLKSRC_I2CCLK 0x41
+#define PRCMU_CLKSRC_MSP02CLK 0x42
+#define PRCMU_CLKSRC_ARMPLL_OBSCLK 0x43
+#define PRCMU_CLKSRC_HSIRXCLK 0x44
+#define PRCMU_CLKSRC_HSITXCLK 0x45
+#define PRCMU_CLKSRC_ARMCLKFIX 0x46
+#define PRCMU_CLKSRC_HDMICLK 0x47
+
+/*
+ * Definitions for controlling ESRAM0 in deep sleep.
+ */
+#define ESRAM0_DEEP_SLEEP_STATE_OFF 1
+#define ESRAM0_DEEP_SLEEP_STATE_RET 2
+
/* This portion previously known as <mach/prcmu-fw-defs_v1.h> */
/**
@@ -451,10 +486,173 @@ enum prcmu_power_status {
PRCMU_ARMPENDINGIT_ER = 0x93,
};
+/* PRCMU Wakeup defines */
+enum prcmu_wakeup_index {
+ PRCMU_WAKEUP_INDEX_RTC,
+ PRCMU_WAKEUP_INDEX_RTT0,
+ PRCMU_WAKEUP_INDEX_RTT1,
+ PRCMU_WAKEUP_INDEX_HSI0,
+ PRCMU_WAKEUP_INDEX_HSI1,
+ PRCMU_WAKEUP_INDEX_USB,
+ PRCMU_WAKEUP_INDEX_ABB,
+ PRCMU_WAKEUP_INDEX_ABB_FIFO,
+ PRCMU_WAKEUP_INDEX_ARM,
+ PRCMU_WAKEUP_INDEX_CD_IRQ,
+ NUM_PRCMU_WAKEUP_INDICES
+};
+
+#define PRCMU_WAKEUP(_name) (BIT(PRCMU_WAKEUP_INDEX_##_name))
+
+/**
+ * enum prcmu_wdog_id - PRCMU watchdog IDs
+ * @PRCMU_WDOG_ALL: use all timers
+ * @PRCMU_WDOG_CPU1: use first CPU timer only
+ * @PRCMU_WDOG_CPU2: use second CPU timer conly
+ */
+enum prcmu_wdog_id {
+ PRCMU_WDOG_ALL = 0x00,
+ PRCMU_WDOG_CPU1 = 0x01,
+ PRCMU_WDOG_CPU2 = 0x02,
+};
+
+/**
+ * enum ape_opp - APE OPP states definition
+ * @APE_OPP_INIT:
+ * @APE_NO_CHANGE: The APE operating point is unchanged
+ * @APE_100_OPP: The new APE operating point is ape100opp
+ * @APE_50_OPP: 50%
+ * @APE_50_PARTLY_25_OPP: 50%, except some clocks at 25%.
+ */
+enum ape_opp {
+ APE_OPP_INIT = 0x00,
+ APE_NO_CHANGE = 0x01,
+ APE_100_OPP = 0x02,
+ APE_50_OPP = 0x03,
+ APE_50_PARTLY_25_OPP = 0xFF,
+};
+
+/**
+ * enum arm_opp - ARM OPP states definition
+ * @ARM_OPP_INIT:
+ * @ARM_NO_CHANGE: The ARM operating point is unchanged
+ * @ARM_100_OPP: The new ARM operating point is arm100opp
+ * @ARM_50_OPP: The new ARM operating point is arm50opp
+ * @ARM_MAX_OPP: Operating point is "max" (more than 100)
+ * @ARM_MAX_FREQ100OPP: Set max opp if available, else 100
+ * @ARM_EXTCLK: The new ARM operating point is armExtClk
+ */
+enum arm_opp {
+ ARM_OPP_INIT = 0x00,
+ ARM_NO_CHANGE = 0x01,
+ ARM_100_OPP = 0x02,
+ ARM_50_OPP = 0x03,
+ ARM_MAX_OPP = 0x04,
+ ARM_MAX_FREQ100OPP = 0x05,
+ ARM_EXTCLK = 0x07
+};
+
+/**
+ * enum ddr_opp - DDR OPP states definition
+ * @DDR_100_OPP: The new DDR operating point is ddr100opp
+ * @DDR_50_OPP: The new DDR operating point is ddr50opp
+ * @DDR_25_OPP: The new DDR operating point is ddr25opp
+ */
+enum ddr_opp {
+ DDR_100_OPP = 0x00,
+ DDR_50_OPP = 0x01,
+ DDR_25_OPP = 0x02,
+};
+
+/**
+ * enum ddr_pwrst - DDR power states definition
+ * @DDR_PWR_STATE_UNCHANGED: SDRAM and DDR controller state is unchanged
+ * @DDR_PWR_STATE_ON:
+ * @DDR_PWR_STATE_OFFLOWLAT:
+ * @DDR_PWR_STATE_OFFHIGHLAT:
+ */
+enum ddr_pwrst {
+ DDR_PWR_STATE_UNCHANGED = 0x00,
+ DDR_PWR_STATE_ON = 0x01,
+ DDR_PWR_STATE_OFFLOWLAT = 0x02,
+ DDR_PWR_STATE_OFFHIGHLAT = 0x03
+};
+
/*
* Definitions for autonomous power management configuration.
*/
+/* EPOD (power domain) IDs */
+
+/*
+ * DB8500 EPODs
+ * - EPOD_ID_SVAMMDSP: power domain for SVA MMDSP
+ * - EPOD_ID_SVAPIPE: power domain for SVA pipe
+ * - EPOD_ID_SIAMMDSP: power domain for SIA MMDSP
+ * - EPOD_ID_SIAPIPE: power domain for SIA pipe
+ * - EPOD_ID_SGA: power domain for SGA
+ * - EPOD_ID_B2R2_MCDE: power domain for B2R2 and MCDE
+ * - EPOD_ID_ESRAM12: power domain for ESRAM 1 and 2
+ * - EPOD_ID_ESRAM34: power domain for ESRAM 3 and 4
+ * - NUM_EPOD_ID: number of power domains
+ *
+ * TODO: These should be prefixed.
+ */
+#define EPOD_ID_SVAMMDSP 0
+#define EPOD_ID_SVAPIPE 1
+#define EPOD_ID_SIAMMDSP 2
+#define EPOD_ID_SIAPIPE 3
+#define EPOD_ID_SGA 4
+#define EPOD_ID_B2R2_MCDE 5
+#define EPOD_ID_ESRAM12 6
+#define EPOD_ID_ESRAM34 7
+#define NUM_EPOD_ID 8
+
+/*
+ * state definition for EPOD (power domain)
+ * - EPOD_STATE_NO_CHANGE: The EPOD should remain unchanged
+ * - EPOD_STATE_OFF: The EPOD is switched off
+ * - EPOD_STATE_RAMRET: The EPOD is switched off with its internal RAM in
+ * retention
+ * - EPOD_STATE_ON_CLK_OFF: The EPOD is switched on, clock is still off
+ * - EPOD_STATE_ON: Same as above, but with clock enabled
+ */
+#define EPOD_STATE_NO_CHANGE 0x00
+#define EPOD_STATE_OFF 0x01
+#define EPOD_STATE_RAMRET 0x02
+#define EPOD_STATE_ON_CLK_OFF 0x03
+#define EPOD_STATE_ON 0x04
+
+#define PRCMU_FW_PROJECT_U8500 2
+#define PRCMU_FW_PROJECT_U8400 3
+#define PRCMU_FW_PROJECT_U9500 4 /* Customer specific */
+#define PRCMU_FW_PROJECT_U8500_MBB 5
+#define PRCMU_FW_PROJECT_U8500_C1 6
+#define PRCMU_FW_PROJECT_U8500_C2 7
+#define PRCMU_FW_PROJECT_U8500_C3 8
+#define PRCMU_FW_PROJECT_U8500_C4 9
+#define PRCMU_FW_PROJECT_U9500_MBL 10
+#define PRCMU_FW_PROJECT_U8500_SSG1 11 /* Samsung specific */
+#define PRCMU_FW_PROJECT_U8500_MBL2 12 /* Customer specific */
+#define PRCMU_FW_PROJECT_U8520 13
+#define PRCMU_FW_PROJECT_U8420 14
+#define PRCMU_FW_PROJECT_U8500_SSG2 15 /* Samsung specific */
+#define PRCMU_FW_PROJECT_U8420_SYSCLK 17
+#define PRCMU_FW_PROJECT_A9420 20
+/* [32..63] 9540 and derivatives */
+#define PRCMU_FW_PROJECT_U9540 32
+/* [64..95] 8540 and derivatives */
+#define PRCMU_FW_PROJECT_L8540 64
+/* [96..126] 8580 and derivatives */
+#define PRCMU_FW_PROJECT_L8580 96
+
+#define PRCMU_FW_PROJECT_NAME_LEN 20
+
+/* PRCMU QoS APE OPP class */
+#define PRCMU_QOS_APE_OPP 1
+#define PRCMU_QOS_DDR_OPP 2
+#define PRCMU_QOS_ARM_OPP 3
+#define PRCMU_QOS_DEFAULT_VALUE -1
+
#define PRCMU_AUTO_PM_OFF 0
#define PRCMU_AUTO_PM_ON 1
@@ -469,6 +667,14 @@ enum prcmu_auto_pm_policy {
PRCMU_AUTO_PM_POLICY_DSP_CLK_OFF_HWP_CLK_OFF,
};
+struct prcmu_fw_version {
+ u32 project; /* Notice, project shifted with 8 on ux540 */
+ u8 api_version;
+ u8 func_version;
+ u8 errata;
+ char project_name[PRCMU_FW_PROJECT_NAME_LEN];
+};
+
/**
* struct prcmu_auto_pm_config - Autonomous power management configuration.
* @sia_auto_pm_enable: SIA autonomous pm enable. (PRCMU_AUTO_PM_{OFF,ON})
@@ -501,6 +707,9 @@ void prcmu_configure_auto_pm(struct prcmu_auto_pm_config *sleep,
bool prcmu_is_auto_pm_enabled(void);
int prcmu_config_clkout(u8 clkout, u8 source, u8 div);
+unsigned long prcmu_clock_rate(u8 clock);
+long prcmu_round_clock_rate(u8 clock, unsigned long rate);
+int prcmu_set_clock_rate(u8 clock, unsigned long rate);
int prcmu_set_clock_divider(u8 clock, u8 divider);
int db8500_prcmu_config_hotdog(u8 threshold);
int db8500_prcmu_config_hotmon(u8 low, u8 high);
@@ -508,6 +717,8 @@ int db8500_prcmu_start_temp_sense(u16 cycles32k);
int db8500_prcmu_stop_temp_sense(void);
int prcmu_abb_read(u8 slave, u8 reg, u8 *value, u8 size);
int prcmu_abb_write(u8 slave, u8 reg, u8 *value, u8 size);
+int prcmu_abb_write_masked(u8 slave, u8 reg, u8 *value,
+ u8 *mask, u8 size);
int prcmu_ac_wake_req(void);
void prcmu_ac_sleep_req(void);
@@ -610,6 +821,21 @@ static inline int prcmu_config_clkout(u8 clkout, u8 source, u8 div)
return 0;
}
+static inline unsigned long prcmu_clock_rate(u8 clock)
+{
+ return 0;
+}
+
+static inline long prcmu_round_clock_rate(u8 clock, unsigned long rate)
+{
+ return 0;
+}
+
+static inline int prcmu_set_clock_rate(u8 clock, unsigned long rate)
+{
+ return 0;
+}
+
static inline int prcmu_set_clock_divider(u8 clock, u8 divider)
{
return 0;
@@ -637,12 +863,18 @@ static inline int db8500_prcmu_stop_temp_sense(void)
static inline int prcmu_abb_read(u8 slave, u8 reg, u8 *value, u8 size)
{
- return -ENOSYS;
+ return -EINVAL;
}
static inline int prcmu_abb_write(u8 slave, u8 reg, u8 *value, u8 size)
{
- return -ENOSYS;
+ return -EINVAL;
+}
+
+static inline int prcmu_abb_write_masked(u8 slave, u8 reg,
+ u8 *value, u8 *mask, u8 size)
+{
+ return -EINVAL;
}
static inline int prcmu_ac_wake_req(void)
@@ -745,4 +977,20 @@ static inline void db8500_prcmu_write_masked(unsigned int reg, u32 mask,
#endif /* !CONFIG_MFD_DB8500_PRCMU */
+static inline int prcmu_qos_add_requirement(int prcmu_qos_class,
+ char *name, s32 value)
+{
+ return 0;
+}
+
+static inline int prcmu_qos_update_requirement(int prcmu_qos_class,
+ char *name, s32 new_value)
+{
+ return 0;
+}
+
+static inline void prcmu_qos_remove_requirement(int prcmu_qos_class, char *name)
+{
+}
+
#endif /* __MFD_DB8500_PRCMU_H */
diff --git a/include/linux/mfd/dbx500-prcmu.h b/include/linux/mfd/dbx500-prcmu.h
deleted file mode 100644
index 828362b7860c..000000000000
--- a/include/linux/mfd/dbx500-prcmu.h
+++ /dev/null
@@ -1,575 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-/*
- * Copyright (C) ST Ericsson SA 2011
- *
- * STE Ux500 PRCMU API
- */
-#ifndef __MACH_PRCMU_H
-#define __MACH_PRCMU_H
-
-#include <linux/interrupt.h>
-#include <linux/notifier.h>
-#include <linux/err.h>
-
-#include <dt-bindings/mfd/dbx500-prcmu.h> /* For clock identifiers */
-
-/* Offset for the firmware version within the TCPM */
-#define DB8500_PRCMU_FW_VERSION_OFFSET 0xA4
-#define DBX540_PRCMU_FW_VERSION_OFFSET 0xA8
-
-/* PRCMU Wakeup defines */
-enum prcmu_wakeup_index {
- PRCMU_WAKEUP_INDEX_RTC,
- PRCMU_WAKEUP_INDEX_RTT0,
- PRCMU_WAKEUP_INDEX_RTT1,
- PRCMU_WAKEUP_INDEX_HSI0,
- PRCMU_WAKEUP_INDEX_HSI1,
- PRCMU_WAKEUP_INDEX_USB,
- PRCMU_WAKEUP_INDEX_ABB,
- PRCMU_WAKEUP_INDEX_ABB_FIFO,
- PRCMU_WAKEUP_INDEX_ARM,
- PRCMU_WAKEUP_INDEX_CD_IRQ,
- NUM_PRCMU_WAKEUP_INDICES
-};
-#define PRCMU_WAKEUP(_name) (BIT(PRCMU_WAKEUP_INDEX_##_name))
-
-/* EPOD (power domain) IDs */
-
-/*
- * DB8500 EPODs
- * - EPOD_ID_SVAMMDSP: power domain for SVA MMDSP
- * - EPOD_ID_SVAPIPE: power domain for SVA pipe
- * - EPOD_ID_SIAMMDSP: power domain for SIA MMDSP
- * - EPOD_ID_SIAPIPE: power domain for SIA pipe
- * - EPOD_ID_SGA: power domain for SGA
- * - EPOD_ID_B2R2_MCDE: power domain for B2R2 and MCDE
- * - EPOD_ID_ESRAM12: power domain for ESRAM 1 and 2
- * - EPOD_ID_ESRAM34: power domain for ESRAM 3 and 4
- * - NUM_EPOD_ID: number of power domains
- *
- * TODO: These should be prefixed.
- */
-#define EPOD_ID_SVAMMDSP 0
-#define EPOD_ID_SVAPIPE 1
-#define EPOD_ID_SIAMMDSP 2
-#define EPOD_ID_SIAPIPE 3
-#define EPOD_ID_SGA 4
-#define EPOD_ID_B2R2_MCDE 5
-#define EPOD_ID_ESRAM12 6
-#define EPOD_ID_ESRAM34 7
-#define NUM_EPOD_ID 8
-
-/*
- * state definition for EPOD (power domain)
- * - EPOD_STATE_NO_CHANGE: The EPOD should remain unchanged
- * - EPOD_STATE_OFF: The EPOD is switched off
- * - EPOD_STATE_RAMRET: The EPOD is switched off with its internal RAM in
- * retention
- * - EPOD_STATE_ON_CLK_OFF: The EPOD is switched on, clock is still off
- * - EPOD_STATE_ON: Same as above, but with clock enabled
- */
-#define EPOD_STATE_NO_CHANGE 0x00
-#define EPOD_STATE_OFF 0x01
-#define EPOD_STATE_RAMRET 0x02
-#define EPOD_STATE_ON_CLK_OFF 0x03
-#define EPOD_STATE_ON 0x04
-
-/*
- * CLKOUT sources
- */
-#define PRCMU_CLKSRC_CLK38M 0x00
-#define PRCMU_CLKSRC_ACLK 0x01
-#define PRCMU_CLKSRC_SYSCLK 0x02
-#define PRCMU_CLKSRC_LCDCLK 0x03
-#define PRCMU_CLKSRC_SDMMCCLK 0x04
-#define PRCMU_CLKSRC_TVCLK 0x05
-#define PRCMU_CLKSRC_TIMCLK 0x06
-#define PRCMU_CLKSRC_CLK009 0x07
-/* These are only valid for CLKOUT1: */
-#define PRCMU_CLKSRC_SIAMMDSPCLK 0x40
-#define PRCMU_CLKSRC_I2CCLK 0x41
-#define PRCMU_CLKSRC_MSP02CLK 0x42
-#define PRCMU_CLKSRC_ARMPLL_OBSCLK 0x43
-#define PRCMU_CLKSRC_HSIRXCLK 0x44
-#define PRCMU_CLKSRC_HSITXCLK 0x45
-#define PRCMU_CLKSRC_ARMCLKFIX 0x46
-#define PRCMU_CLKSRC_HDMICLK 0x47
-
-/**
- * enum prcmu_wdog_id - PRCMU watchdog IDs
- * @PRCMU_WDOG_ALL: use all timers
- * @PRCMU_WDOG_CPU1: use first CPU timer only
- * @PRCMU_WDOG_CPU2: use second CPU timer conly
- */
-enum prcmu_wdog_id {
- PRCMU_WDOG_ALL = 0x00,
- PRCMU_WDOG_CPU1 = 0x01,
- PRCMU_WDOG_CPU2 = 0x02,
-};
-
-/**
- * enum ape_opp - APE OPP states definition
- * @APE_OPP_INIT:
- * @APE_NO_CHANGE: The APE operating point is unchanged
- * @APE_100_OPP: The new APE operating point is ape100opp
- * @APE_50_OPP: 50%
- * @APE_50_PARTLY_25_OPP: 50%, except some clocks at 25%.
- */
-enum ape_opp {
- APE_OPP_INIT = 0x00,
- APE_NO_CHANGE = 0x01,
- APE_100_OPP = 0x02,
- APE_50_OPP = 0x03,
- APE_50_PARTLY_25_OPP = 0xFF,
-};
-
-/**
- * enum arm_opp - ARM OPP states definition
- * @ARM_OPP_INIT:
- * @ARM_NO_CHANGE: The ARM operating point is unchanged
- * @ARM_100_OPP: The new ARM operating point is arm100opp
- * @ARM_50_OPP: The new ARM operating point is arm50opp
- * @ARM_MAX_OPP: Operating point is "max" (more than 100)
- * @ARM_MAX_FREQ100OPP: Set max opp if available, else 100
- * @ARM_EXTCLK: The new ARM operating point is armExtClk
- */
-enum arm_opp {
- ARM_OPP_INIT = 0x00,
- ARM_NO_CHANGE = 0x01,
- ARM_100_OPP = 0x02,
- ARM_50_OPP = 0x03,
- ARM_MAX_OPP = 0x04,
- ARM_MAX_FREQ100OPP = 0x05,
- ARM_EXTCLK = 0x07
-};
-
-/**
- * enum ddr_opp - DDR OPP states definition
- * @DDR_100_OPP: The new DDR operating point is ddr100opp
- * @DDR_50_OPP: The new DDR operating point is ddr50opp
- * @DDR_25_OPP: The new DDR operating point is ddr25opp
- */
-enum ddr_opp {
- DDR_100_OPP = 0x00,
- DDR_50_OPP = 0x01,
- DDR_25_OPP = 0x02,
-};
-
-/*
- * Definitions for controlling ESRAM0 in deep sleep.
- */
-#define ESRAM0_DEEP_SLEEP_STATE_OFF 1
-#define ESRAM0_DEEP_SLEEP_STATE_RET 2
-
-/**
- * enum ddr_pwrst - DDR power states definition
- * @DDR_PWR_STATE_UNCHANGED: SDRAM and DDR controller state is unchanged
- * @DDR_PWR_STATE_ON:
- * @DDR_PWR_STATE_OFFLOWLAT:
- * @DDR_PWR_STATE_OFFHIGHLAT:
- */
-enum ddr_pwrst {
- DDR_PWR_STATE_UNCHANGED = 0x00,
- DDR_PWR_STATE_ON = 0x01,
- DDR_PWR_STATE_OFFLOWLAT = 0x02,
- DDR_PWR_STATE_OFFHIGHLAT = 0x03
-};
-
-#define DB8500_PRCMU_LEGACY_OFFSET 0xDD4
-
-#define PRCMU_FW_PROJECT_U8500 2
-#define PRCMU_FW_PROJECT_U8400 3
-#define PRCMU_FW_PROJECT_U9500 4 /* Customer specific */
-#define PRCMU_FW_PROJECT_U8500_MBB 5
-#define PRCMU_FW_PROJECT_U8500_C1 6
-#define PRCMU_FW_PROJECT_U8500_C2 7
-#define PRCMU_FW_PROJECT_U8500_C3 8
-#define PRCMU_FW_PROJECT_U8500_C4 9
-#define PRCMU_FW_PROJECT_U9500_MBL 10
-#define PRCMU_FW_PROJECT_U8500_SSG1 11 /* Samsung specific */
-#define PRCMU_FW_PROJECT_U8500_MBL2 12 /* Customer specific */
-#define PRCMU_FW_PROJECT_U8520 13
-#define PRCMU_FW_PROJECT_U8420 14
-#define PRCMU_FW_PROJECT_U8500_SSG2 15 /* Samsung specific */
-#define PRCMU_FW_PROJECT_U8420_SYSCLK 17
-#define PRCMU_FW_PROJECT_A9420 20
-/* [32..63] 9540 and derivatives */
-#define PRCMU_FW_PROJECT_U9540 32
-/* [64..95] 8540 and derivatives */
-#define PRCMU_FW_PROJECT_L8540 64
-/* [96..126] 8580 and derivatives */
-#define PRCMU_FW_PROJECT_L8580 96
-
-#define PRCMU_FW_PROJECT_NAME_LEN 20
-struct prcmu_fw_version {
- u32 project; /* Notice, project shifted with 8 on ux540 */
- u8 api_version;
- u8 func_version;
- u8 errata;
- char project_name[PRCMU_FW_PROJECT_NAME_LEN];
-};
-
-#include <linux/mfd/db8500-prcmu.h>
-
-#if defined(CONFIG_UX500_SOC_DB8500)
-
-static inline void __init prcmu_early_init(void)
-{
- db8500_prcmu_early_init();
-}
-
-static inline int prcmu_set_power_state(u8 state, bool keep_ulp_clk,
- bool keep_ap_pll)
-{
- return db8500_prcmu_set_power_state(state, keep_ulp_clk,
- keep_ap_pll);
-}
-
-static inline u8 prcmu_get_power_state_result(void)
-{
- return db8500_prcmu_get_power_state_result();
-}
-
-static inline int prcmu_set_epod(u16 epod_id, u8 epod_state)
-{
- return db8500_prcmu_set_epod(epod_id, epod_state);
-}
-
-static inline void prcmu_enable_wakeups(u32 wakeups)
-{
- db8500_prcmu_enable_wakeups(wakeups);
-}
-
-static inline void prcmu_disable_wakeups(void)
-{
- prcmu_enable_wakeups(0);
-}
-
-static inline void prcmu_config_abb_event_readout(u32 abb_events)
-{
- db8500_prcmu_config_abb_event_readout(abb_events);
-}
-
-static inline void prcmu_get_abb_event_buffer(void __iomem **buf)
-{
- db8500_prcmu_get_abb_event_buffer(buf);
-}
-
-int prcmu_abb_read(u8 slave, u8 reg, u8 *value, u8 size);
-int prcmu_abb_write(u8 slave, u8 reg, u8 *value, u8 size);
-int prcmu_abb_write_masked(u8 slave, u8 reg, u8 *value, u8 *mask, u8 size);
-
-int prcmu_config_clkout(u8 clkout, u8 source, u8 div);
-
-static inline int prcmu_request_clock(u8 clock, bool enable)
-{
- return db8500_prcmu_request_clock(clock, enable);
-}
-
-unsigned long prcmu_clock_rate(u8 clock);
-long prcmu_round_clock_rate(u8 clock, unsigned long rate);
-int prcmu_set_clock_rate(u8 clock, unsigned long rate);
-
-static inline int prcmu_get_ddr_opp(void)
-{
- return db8500_prcmu_get_ddr_opp();
-}
-
-static inline int prcmu_set_arm_opp(u8 opp)
-{
- return db8500_prcmu_set_arm_opp(opp);
-}
-
-static inline int prcmu_get_arm_opp(void)
-{
- return db8500_prcmu_get_arm_opp();
-}
-
-static inline int prcmu_set_ape_opp(u8 opp)
-{
- return db8500_prcmu_set_ape_opp(opp);
-}
-
-static inline int prcmu_get_ape_opp(void)
-{
- return db8500_prcmu_get_ape_opp();
-}
-
-static inline int prcmu_request_ape_opp_100_voltage(bool enable)
-{
- return db8500_prcmu_request_ape_opp_100_voltage(enable);
-}
-
-static inline void prcmu_system_reset(u16 reset_code)
-{
- db8500_prcmu_system_reset(reset_code);
-}
-
-static inline u16 prcmu_get_reset_code(void)
-{
- return db8500_prcmu_get_reset_code();
-}
-
-int prcmu_ac_wake_req(void);
-void prcmu_ac_sleep_req(void);
-static inline void prcmu_modem_reset(void)
-{
- db8500_prcmu_modem_reset();
-}
-
-static inline bool prcmu_is_ac_wake_requested(void)
-{
- return db8500_prcmu_is_ac_wake_requested();
-}
-
-static inline int prcmu_config_esram0_deep_sleep(u8 state)
-{
- return db8500_prcmu_config_esram0_deep_sleep(state);
-}
-
-static inline int prcmu_config_hotdog(u8 threshold)
-{
- return db8500_prcmu_config_hotdog(threshold);
-}
-
-static inline int prcmu_config_hotmon(u8 low, u8 high)
-{
- return db8500_prcmu_config_hotmon(low, high);
-}
-
-static inline int prcmu_start_temp_sense(u16 cycles32k)
-{
- return db8500_prcmu_start_temp_sense(cycles32k);
-}
-
-static inline int prcmu_stop_temp_sense(void)
-{
- return db8500_prcmu_stop_temp_sense();
-}
-
-static inline u32 prcmu_read(unsigned int reg)
-{
- return db8500_prcmu_read(reg);
-}
-
-static inline void prcmu_write(unsigned int reg, u32 value)
-{
- db8500_prcmu_write(reg, value);
-}
-
-static inline void prcmu_write_masked(unsigned int reg, u32 mask, u32 value)
-{
- db8500_prcmu_write_masked(reg, mask, value);
-}
-
-static inline int prcmu_enable_a9wdog(u8 id)
-{
- return db8500_prcmu_enable_a9wdog(id);
-}
-
-static inline int prcmu_disable_a9wdog(u8 id)
-{
- return db8500_prcmu_disable_a9wdog(id);
-}
-
-static inline int prcmu_kick_a9wdog(u8 id)
-{
- return db8500_prcmu_kick_a9wdog(id);
-}
-
-static inline int prcmu_load_a9wdog(u8 id, u32 timeout)
-{
- return db8500_prcmu_load_a9wdog(id, timeout);
-}
-
-static inline int prcmu_config_a9wdog(u8 num, bool sleep_auto_off)
-{
- return db8500_prcmu_config_a9wdog(num, sleep_auto_off);
-}
-#else
-
-static inline void prcmu_early_init(void) {}
-
-static inline int prcmu_set_power_state(u8 state, bool keep_ulp_clk,
- bool keep_ap_pll)
-{
- return 0;
-}
-
-static inline int prcmu_set_epod(u16 epod_id, u8 epod_state)
-{
- return 0;
-}
-
-static inline void prcmu_enable_wakeups(u32 wakeups) {}
-
-static inline void prcmu_disable_wakeups(void) {}
-
-static inline int prcmu_abb_read(u8 slave, u8 reg, u8 *value, u8 size)
-{
- return -ENOSYS;
-}
-
-static inline int prcmu_abb_write(u8 slave, u8 reg, u8 *value, u8 size)
-{
- return -ENOSYS;
-}
-
-static inline int prcmu_abb_write_masked(u8 slave, u8 reg, u8 *value, u8 *mask,
- u8 size)
-{
- return -ENOSYS;
-}
-
-static inline int prcmu_config_clkout(u8 clkout, u8 source, u8 div)
-{
- return 0;
-}
-
-static inline int prcmu_request_clock(u8 clock, bool enable)
-{
- return 0;
-}
-
-static inline long prcmu_round_clock_rate(u8 clock, unsigned long rate)
-{
- return 0;
-}
-
-static inline int prcmu_set_clock_rate(u8 clock, unsigned long rate)
-{
- return 0;
-}
-
-static inline unsigned long prcmu_clock_rate(u8 clock)
-{
- return 0;
-}
-
-static inline int prcmu_set_ape_opp(u8 opp)
-{
- return 0;
-}
-
-static inline int prcmu_get_ape_opp(void)
-{
- return APE_100_OPP;
-}
-
-static inline int prcmu_request_ape_opp_100_voltage(bool enable)
-{
- return 0;
-}
-
-static inline int prcmu_set_arm_opp(u8 opp)
-{
- return 0;
-}
-
-static inline int prcmu_get_arm_opp(void)
-{
- return ARM_100_OPP;
-}
-
-static inline int prcmu_get_ddr_opp(void)
-{
- return DDR_100_OPP;
-}
-
-static inline void prcmu_system_reset(u16 reset_code) {}
-
-static inline u16 prcmu_get_reset_code(void)
-{
- return 0;
-}
-
-static inline int prcmu_ac_wake_req(void)
-{
- return 0;
-}
-
-static inline void prcmu_ac_sleep_req(void) {}
-
-static inline void prcmu_modem_reset(void) {}
-
-static inline bool prcmu_is_ac_wake_requested(void)
-{
- return false;
-}
-
-static inline int prcmu_config_esram0_deep_sleep(u8 state)
-{
- return 0;
-}
-
-static inline void prcmu_config_abb_event_readout(u32 abb_events) {}
-
-static inline void prcmu_get_abb_event_buffer(void __iomem **buf)
-{
- *buf = NULL;
-}
-
-static inline int prcmu_config_hotdog(u8 threshold)
-{
- return 0;
-}
-
-static inline int prcmu_config_hotmon(u8 low, u8 high)
-{
- return 0;
-}
-
-static inline int prcmu_start_temp_sense(u16 cycles32k)
-{
- return 0;
-}
-
-static inline int prcmu_stop_temp_sense(void)
-{
- return 0;
-}
-
-static inline u32 prcmu_read(unsigned int reg)
-{
- return 0;
-}
-
-static inline void prcmu_write(unsigned int reg, u32 value) {}
-
-static inline void prcmu_write_masked(unsigned int reg, u32 mask, u32 value) {}
-
-#endif
-
-static inline void prcmu_set(unsigned int reg, u32 bits)
-{
- prcmu_write_masked(reg, bits, bits);
-}
-
-static inline void prcmu_clear(unsigned int reg, u32 bits)
-{
- prcmu_write_masked(reg, bits, 0);
-}
-
-/* PRCMU QoS APE OPP class */
-#define PRCMU_QOS_APE_OPP 1
-#define PRCMU_QOS_DDR_OPP 2
-#define PRCMU_QOS_ARM_OPP 3
-#define PRCMU_QOS_DEFAULT_VALUE -1
-
-static inline int prcmu_qos_add_requirement(int prcmu_qos_class,
- char *name, s32 value)
-{
- return 0;
-}
-
-static inline int prcmu_qos_update_requirement(int prcmu_qos_class,
- char *name, s32 new_value)
-{
- return 0;
-}
-
-static inline void prcmu_qos_remove_requirement(int prcmu_qos_class, char *name)
-{
-}
-
-#endif /* __MACH_PRCMU_H */
diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
index 6b5c2390cc30..e1d348d430c1 100644
--- a/include/linux/mmap_lock.h
+++ b/include/linux/mmap_lock.h
@@ -622,6 +622,8 @@ static inline void mmap_read_unlock(struct mm_struct *mm)
DEFINE_GUARD(mmap_read_lock, struct mm_struct *,
mmap_read_lock(_T), mmap_read_unlock(_T))
DEFINE_GUARD_COND(mmap_read_lock, _try, mmap_read_trylock(_T))
+DEFINE_GUARD(mmap_write_lock, struct mm_struct *,
+ mmap_write_lock(_T), mmap_write_unlock(_T))
static inline void mmap_read_unlock_non_owner(struct mm_struct *mm)
{
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index d0b62d53eea9..67c2d9ef7c62 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -247,6 +247,7 @@ struct netfs_io_request {
unsigned long long submitted; /* Amount submitted for I/O so far */
unsigned long long len; /* Length of the request */
size_t transferred; /* Amount to be indicated as transferred */
+ size_t progress_at; /* Report read progress when hit this much read */
long error; /* 0 or error that occurred */
unsigned long long i_size; /* Size of the file */
unsigned long long start; /* Start position */
@@ -263,7 +264,6 @@ struct netfs_io_request {
atomic_t subreq_counter; /* Next subreq->debug_index */
unsigned int nr_group_rel; /* Number of refs to release on ->group */
spinlock_t lock; /* Lock for queuing subreqs */
- unsigned char front_folio_order; /* Order (size) of front folio */
enum netfs_io_origin origin; /* Origin of the request */
bool direct_bv_unpin; /* T if direct_bv[] must be unpinned */
refcount_t ref;
@@ -276,9 +276,10 @@ struct netfs_io_request {
#define NETFS_RREQ_SHORT_TRANSFER 5 /* Set if we have a short transfer */
#define NETFS_RREQ_OFFLOAD_COLLECTION 8 /* Offload collection to workqueue */
#define NETFS_RREQ_NO_UNLOCK_FOLIO 9 /* Don't unlock no_unlock_folio on completion */
-#define NETFS_RREQ_FOLIO_COPY_TO_CACHE 10 /* Copy current folio to cache from read */
+#define NETFS_RREQ_CANCEL_CACHING 10 /* Set to cancel caching */
#define NETFS_RREQ_UPLOAD_TO_SERVER 11 /* Need to write to the server */
#define NETFS_RREQ_USE_IO_ITER 12 /* Use ->io_iter rather than ->i_pages */
+#define NETFS_RREQ_NEED_PUT_RA_REFS 17 /* Need to put the folio refs RA gave us */
#define NETFS_RREQ_USE_PGPRIV2 31 /* [DEPRECATED] Use PG_private_2 to mark
* write to cache on read */
const struct netfs_request_ops *netfs_ops;
diff --git a/include/linux/oa_tc6.h b/include/linux/oa_tc6.h
index 15f58e3c56c7..71cffcb041b1 100644
--- a/include/linux/oa_tc6.h
+++ b/include/linux/oa_tc6.h
@@ -10,6 +10,56 @@
#include <linux/etherdevice.h>
#include <linux/spi/spi.h>
+/* OPEN Alliance TC6 registers */
+/* Standard Capabilities Register */
+#define OA_TC6_REG_STDCAP 0x0002
+#define OA_TC6_STDCAP_DIRECT_PHY_REG_ACCESS BIT(8)
+
+/* Reset Control and Status Register */
+#define OA_TC6_REG_RESET 0x0003
+#define OA_TC6_RESET_SWRESET BIT(0) /* Software Reset */
+
+/* Configuration Register #0 */
+#define OA_TC6_REG_CONFIG0 0x0004
+#define OA_TC6_CONFIG0_SYNC BIT(15)
+#define OA_TC6_CONFIG0_ZARFE_ENABLE BIT(12)
+#define OA_TC6_CONFIG0_PROTE BIT(5)
+
+/* Status Register #0 */
+#define OA_TC6_REG_STATUS0 0x0008
+#define OA_TC6_STATUS0_RESETC BIT(6) /* Reset Complete */
+#define OA_TC6_STATUS0_HEADER_ERROR BIT(5)
+#define OA_TC6_STATUS0_LOSS_OF_FRAME_ERROR BIT(4)
+#define OA_TC6_STATUS0_RX_BUFFER_OVERFLOW_ERROR BIT(3)
+#define OA_TC6_STATUS0_TX_PROTOCOL_ERROR BIT(0)
+
+/* Buffer Status Register */
+#define OA_TC6_REG_BUFFER_STATUS 0x000B
+#define OA_TC6_BUFFER_STATUS_TX_CREDITS_AVAILABLE GENMASK(15, 8)
+#define OA_TC6_BUFFER_STATUS_RX_CHUNKS_AVAILABLE GENMASK(7, 0)
+
+/* Interrupt Mask Register #0 */
+#define OA_TC6_REG_INT_MASK0 0x000C
+#define OA_TC6_INT_MASK0_HEADER_ERR_MASK BIT(5)
+#define OA_TC6_INT_MASK0_LOSS_OF_FRAME_ERR_MASK BIT(4)
+#define OA_TC6_INT_MASK0_RX_BUFFER_OVERFLOW_ERR_MASK BIT(3)
+#define OA_TC6_INT_MASK0_TX_PROTOCOL_ERR_MASK BIT(0)
+#define OA_TC6_INT_MASK0_ALL_INTERRUPTS (GENMASK(5, 0) | \
+ GENMASK(12, 7))
+
+/* PHY Clause 22 registers base address and mask */
+#define OA_TC6_PHY_STD_REG_ADDR_BASE 0xFF00
+#define OA_TC6_PHY_STD_REG_ADDR_MASK 0x1F
+
+/* PHY – Clause 45 registers memory map selector (MMS) as per table 6 in the
+ * OPEN Alliance specification.
+ */
+#define OA_TC6_PHY_C45_PCS_MMS2 2 /* MMD 3 */
+#define OA_TC6_PHY_C45_PMA_PMD_MMS3 3 /* MMD 1 */
+#define OA_TC6_PHY_C45_VS_PLCA_MMS4 4 /* MMD 31 */
+#define OA_TC6_PHY_C45_AUTO_NEG_MMS5 5 /* MMD 7 */
+#define OA_TC6_PHY_C45_POWER_UNIT_MMS6 6 /* MMD 13 */
+
struct oa_tc6;
struct oa_tc6 *oa_tc6_init(struct spi_device *spi, struct net_device *netdev);
diff --git a/include/linux/perf/riscv_pmu.h b/include/linux/perf/riscv_pmu.h
index f82a28040594..ecaa40370830 100644
--- a/include/linux/perf/riscv_pmu.h
+++ b/include/linux/perf/riscv_pmu.h
@@ -55,7 +55,7 @@ struct riscv_pmu {
irqreturn_t (*handle_irq)(int irq_num, void *dev);
- unsigned long cmask;
+ DECLARE_BITMAP(cmask, RISCV_MAX_COUNTERS);
u64 (*ctr_read)(struct perf_event *event);
int (*ctr_get_idx)(struct perf_event *event);
int (*ctr_get_width)(int idx);
diff --git a/include/linux/rbtree_augmented.h b/include/linux/rbtree_augmented.h
index 6dbc5a1bf6a8..d2fa1c41bfd2 100644
--- a/include/linux/rbtree_augmented.h
+++ b/include/linux/rbtree_augmented.h
@@ -87,18 +87,18 @@ rb_add_augmented_cached(struct rb_node *node, struct rb_root_cached *tree,
}
/*
- * Template for declaring augmented rbtree callbacks (generic case)
+ * Template for declaring augmented rbtree callbacks (generic multi fields)
*
* RBSTATIC: 'static' or empty
* RBNAME: name of the rb_augment_callbacks structure
* RBSTRUCT: struct type of the tree nodes
* RBFIELD: name of struct rb_node field within RBSTRUCT
- * RBAUGMENTED: name of field within RBSTRUCT holding data for subtree
- * RBCOMPUTE: name of function that recomputes the RBAUGMENTED data
+ * RBCOPY: name of function that copies the RBAUGMENTED datas
+ * RBCOMPUTE: name of function that recomputes the RBAUGMENTED datas
*/
-#define RB_DECLARE_CALLBACKS(RBSTATIC, RBNAME, \
- RBSTRUCT, RBFIELD, RBAUGMENTED, RBCOMPUTE) \
+#define RB_DECLARE_CALLBACKS_MULTI(RBSTATIC, RBNAME, \
+ RBSTRUCT, RBFIELD, RBCOPY, RBCOMPUTE) \
static inline void \
RBNAME ## _propagate(struct rb_node *rb, struct rb_node *stop) \
{ \
@@ -114,14 +114,14 @@ RBNAME ## _copy(struct rb_node *rb_old, struct rb_node *rb_new) \
{ \
RBSTRUCT *old = rb_entry(rb_old, RBSTRUCT, RBFIELD); \
RBSTRUCT *new = rb_entry(rb_new, RBSTRUCT, RBFIELD); \
- new->RBAUGMENTED = old->RBAUGMENTED; \
+ RBCOPY(new, old); \
} \
static void \
RBNAME ## _rotate(struct rb_node *rb_old, struct rb_node *rb_new) \
{ \
RBSTRUCT *old = rb_entry(rb_old, RBSTRUCT, RBFIELD); \
RBSTRUCT *new = rb_entry(rb_new, RBSTRUCT, RBFIELD); \
- new->RBAUGMENTED = old->RBAUGMENTED; \
+ RBCOPY(new, old); \
RBCOMPUTE(old, false); \
} \
RBSTATIC const struct rb_augment_callbacks RBNAME = { \
@@ -130,6 +130,27 @@ RBSTATIC const struct rb_augment_callbacks RBNAME = { \
.rotate = RBNAME ## _rotate \
};
+/*
+ * Template for declaring augmented rbtree callbacks (generic single field)
+ *
+ * RBSTATIC: 'static' or empty
+ * RBNAME: name of the rb_augment_callbacks structure
+ * RBSTRUCT: struct type of the tree nodes
+ * RBFIELD: name of struct rb_node field within RBSTRUCT
+ * RBAUGMENTED: name of field within RBSTRUCT holding data for subtree
+ * RBCOMPUTE: name of function that recomputes the RBAUGMENTED data
+ */
+
+#define RB_DECLARE_CALLBACKS(RBSTATIC, RBNAME, \
+ RBSTRUCT, RBFIELD, RBAUGMENTED, RBCOMPUTE) \
+static inline void \
+RBNAME ## _copy_single(RBSTRUCT *new, RBSTRUCT *old) \
+{ \
+ new->RBAUGMENTED = old->RBAUGMENTED; \
+} \
+RB_DECLARE_CALLBACKS_MULTI(RBSTATIC, RBNAME, \
+ RBSTRUCT, RBFIELD, RBNAME ## _copy_single, RBCOMPUTE)
+
/*
* Template for declaring augmented rbtree callbacks,
* computing RBAUGMENTED scalar as max(RBCOMPUTE(node)) for all subtree nodes.
diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h
index 0670742b2d60..eac3e9080c3c 100644
--- a/include/linux/ring_buffer.h
+++ b/include/linux/ring_buffer.h
@@ -3,8 +3,9 @@
#define _LINUX_RING_BUFFER_H
#include <linux/mm.h>
-#include <linux/seq_file.h>
#include <linux/poll.h>
+#include <linux/ring_buffer_types.h>
+#include <linux/seq_file.h>
#include <uapi/linux/trace_mmap.h>
@@ -218,14 +219,15 @@ bool ring_buffer_time_stamp_abs(struct trace_buffer *buffer);
size_t ring_buffer_nr_dirty_pages(struct trace_buffer *buffer, int cpu);
struct buffer_data_read_page;
-struct buffer_data_read_page *
-ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu);
+int ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu,
+ struct buffer_data_read_page **rpage);
void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu,
struct buffer_data_read_page *page);
int ring_buffer_read_page(struct trace_buffer *buffer,
struct buffer_data_read_page *data_page,
size_t len, int cpu, int full);
void *ring_buffer_read_page_data(struct buffer_data_read_page *page);
+unsigned int ring_buffer_read_page_size(struct buffer_data_read_page *rpage);
struct trace_seq;
@@ -278,11 +280,25 @@ static inline struct ring_buffer_desc *__first_ring_buffer_desc(struct trace_buf
return (struct ring_buffer_desc *)(&desc->__data[0]);
}
+/*
+ * Returns the number of pages for a ring_buffer_desc. The caller must ensure it
+ * does not overflow ring_buffer_desc::nr_page_va.
+ */
+static inline unsigned long __calc_nr_pages_ring_buffer_desc(size_t size)
+{
+ /* Takes into account the reader page */
+ return max(DIV_ROUND_UP(size, PAGE_SIZE - BUF_PAGE_HDR_SIZE), 2UL) + 1;
+}
+
static inline size_t trace_buffer_desc_size(size_t buffer_size, unsigned int nr_cpus)
{
- unsigned int nr_pages = max(DIV_ROUND_UP(buffer_size, PAGE_SIZE), 2UL) + 1;
+ unsigned long nr_pages = __calc_nr_pages_ring_buffer_desc(buffer_size);
struct ring_buffer_desc *rbdesc;
+ /* Capped by ring_buffer_desc::nr_page_va */
+ if (nr_pages > UINT_MAX)
+ return SIZE_MAX;
+
return size_add(offsetof(struct trace_buffer_desc, __data),
size_mul(nr_cpus, struct_size(rbdesc, page_va, nr_pages)));
}
diff --git a/include/linux/rolling_buffer.h b/include/linux/rolling_buffer.h
index 9e5dad29669c..a97f7cfaacaa 100644
--- a/include/linux/rolling_buffer.h
+++ b/include/linux/rolling_buffer.h
@@ -45,9 +45,9 @@ struct rolling_buffer_snapshot {
int rolling_buffer_init(struct rolling_buffer *roll, unsigned int rreq_id,
unsigned int direction, gfp_t gfp);
int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp);
-ssize_t rolling_buffer_load_from_ra(struct rolling_buffer *roll,
- struct readahead_control *ractl,
- struct folio_batch *put_batch);
+ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
+ struct readahead_control *ractl,
+ unsigned int rreq_id, gfp_t gfp);
ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
unsigned int flags, gfp_t gfp);
struct folio_queue *rolling_buffer_delete_spent(struct rolling_buffer *roll);
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 7a53c15cecb5..9341d647bbd1 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1775,7 +1775,7 @@ static inline bool is_lazy_mmu_mode_active(void)
}
#endif
-extern struct pid *cad_pid;
+extern struct pid __rcu *cad_pid;
/*
* Per process flags
diff --git a/include/linux/sched/signal.h b/include/linux/sched/signal.h
index 584ae88b435e..d45a5476b97d 100644
--- a/include/linux/sched/signal.h
+++ b/include/linux/sched/signal.h
@@ -562,10 +562,7 @@ static inline sigset_t *sigmask_to_save(void)
return res;
}
-static inline int kill_cad_pid(int sig, int priv)
-{
- return kill_pid(cad_pid, sig, priv);
-}
+int kill_cad_pid(int sig, int priv);
/* These can be the second arg to send_sig_info/send_group_sig_info. */
#define SEND_SIG_NOINFO ((struct kernel_siginfo *) 0)
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index ed6a2bc23db5..f7dd3db9459c 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -3082,6 +3082,11 @@ static inline bool skb_transport_header_was_set(const struct sk_buff *skb)
return skb->transport_header != (typeof(skb->transport_header))~0U;
}
+static inline void skb_unset_transport_header(struct sk_buff *skb)
+{
+ skb->transport_header = (typeof(skb->transport_header))~0U;
+}
+
static inline unsigned char *skb_transport_header(const struct sk_buff *skb)
{
DEBUG_NET_WARN_ON_ONCE(!skb_transport_header_was_set(skb));
diff --git a/include/media/v4l2-hevc.h b/include/media/v4l2-hevc.h
new file mode 100644
index 000000000000..973c96be16be
--- /dev/null
+++ b/include/media/v4l2-hevc.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Helper functions for HEVC stateless codecs.
+ */
+
+#ifndef _MEDIA_V4L2_HEVC_H
+#define _MEDIA_V4L2_HEVC_H
+
+#include <linux/minmax.h>
+#include <media/v4l2-ctrls.h>
+
+/**
+ * v4l2_hevc_pps_num_tile_columns - number of HEVC tile columns, bounded
+ * @pps: the V4L2 HEVC PPS control
+ *
+ * Return the number of tile columns (num_tile_columns_minus1 + 1) clamped to
+ * the capacity of column_width_minus1[]. The control validation already
+ * rejects out-of-range counts; this keeps the consuming drivers bounded too.
+ */
+static inline unsigned int
+v4l2_hevc_pps_num_tile_columns(const struct v4l2_ctrl_hevc_pps *pps)
+{
+ return min_t(unsigned int, pps->num_tile_columns_minus1 + 1,
+ ARRAY_SIZE(pps->column_width_minus1));
+}
+
+/**
+ * v4l2_hevc_pps_num_tile_rows - number of HEVC tile rows, bounded
+ * @pps: the V4L2 HEVC PPS control
+ *
+ * Return the number of tile rows (num_tile_rows_minus1 + 1) clamped to the
+ * capacity of row_height_minus1[].
+ */
+static inline unsigned int
+v4l2_hevc_pps_num_tile_rows(const struct v4l2_ctrl_hevc_pps *pps)
+{
+ return min_t(unsigned int, pps->num_tile_rows_minus1 + 1,
+ ARRAY_SIZE(pps->row_height_minus1));
+}
+
+#endif /* _MEDIA_V4L2_HEVC_H */
diff --git a/include/net/cfg802154.h b/include/net/cfg802154.h
index 76d2cd2e2b30..2e960441ea49 100644
--- a/include/net/cfg802154.h
+++ b/include/net/cfg802154.h
@@ -376,6 +376,7 @@ struct cfg802154_mac_pkt {
struct list_head node;
struct sk_buff *skb;
struct ieee802154_sub_if_data *sdata;
+ netdevice_tracker dev_tracker;
u8 page;
u8 channel;
};
diff --git a/include/net/if_inet6.h b/include/net/if_inet6.h
index 238ad3349456..795fb41b45f5 100644
--- a/include/net/if_inet6.h
+++ b/include/net/if_inet6.h
@@ -88,8 +88,6 @@ struct ip6_sf_socklist {
struct in6_addr sl_addr[] __counted_by(sl_max);
};
-#define IP6_SFBLOCK 10 /* allocate this many at once */
-
struct ipv6_mc_socklist {
struct in6_addr addr;
int ifindex;
diff --git a/include/net/ip.h b/include/net/ip.h
index a8f57b4f4aa2..6f602df72ee6 100644
--- a/include/net/ip.h
+++ b/include/net/ip.h
@@ -704,7 +704,8 @@ static inline void ip_ipgre_mc_map(__be32 naddr, const unsigned char *broadcast,
static __inline__ void inet_reset_saddr(struct sock *sk)
{
- inet_sk(sk)->inet_rcv_saddr = inet_sk(sk)->inet_saddr = 0;
+ inet_sk(sk)->inet_saddr = 0;
+ WRITE_ONCE(inet_sk(sk)->inet_rcv_saddr, 0);
#if IS_ENABLED(CONFIG_IPV6)
if (sk->sk_family == PF_INET6) {
struct ipv6_pinfo *np = inet6_sk(sk);
diff --git a/include/net/ip6_route.h b/include/net/ip6_route.h
index ac1acc0b7436..745ab62154ac 100644
--- a/include/net/ip6_route.h
+++ b/include/net/ip6_route.h
@@ -382,6 +382,8 @@ static inline unsigned int ip6_dst_mtu_maybe_forward(const struct dst_entry *dst
rcu_read_unlock();
out:
+ mtu = min_t(unsigned int, mtu, IP6_MAX_MTU);
+
return mtu - lwtunnel_headroom(dst->lwtstate, mtu);
}
diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h
index cbc248776511..f35bd06a6bad 100644
--- a/include/net/sch_generic.h
+++ b/include/net/sch_generic.h
@@ -793,7 +793,7 @@ void qdisc_offload_query_caps(struct net_device *dev,
struct Qdisc *qdisc_alloc(struct netdev_queue *dev_queue,
const struct Qdisc_ops *ops,
struct netlink_ext_ack *extack);
-void qdisc_free(struct Qdisc *qdisc);
+void qdisc_free_rcu(struct Qdisc *qdisc);
struct Qdisc *qdisc_create_dflt(struct netdev_queue *dev_queue,
const struct Qdisc_ops *ops, u32 parentid,
struct netlink_ext_ack *extack);
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 670c20876f26..436495ff2271 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -765,8 +765,7 @@ int tcp_fragment(struct sock *sk, enum tcp_queue tcp_queue,
void tcp_send_probe0(struct sock *);
int tcp_write_wakeup(struct sock *, int mib);
void tcp_send_fin(struct sock *sk);
-void tcp_send_active_reset(struct sock *sk, gfp_t priority,
- enum sk_rst_reason reason);
+void tcp_send_active_reset(struct sock *sk, enum sk_rst_reason reason);
int tcp_send_synack(struct sock *);
void tcp_push_one(struct sock *, unsigned int mss_now);
void __tcp_send_ack(struct sock *sk, u32 rcv_nxt, u16 flags);
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index 6e3b1424eea4..d002ed508a43 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -380,7 +380,7 @@ TRACE_EVENT(cachefiles_rename,
TRACE_EVENT(cachefiles_coherency,
TP_PROTO(struct cachefiles_object *obj,
ino_t ino,
- u64 disk_aux,
+ const void *disk_aux,
enum cachefiles_content content,
enum cachefiles_coherency_trace why),
@@ -397,12 +397,27 @@ TRACE_EVENT(cachefiles_coherency,
),
TP_fast_assign(
+ union {
+ __be16 s[4];
+ __be64 ll;
+ } x;
+
__entry->obj = obj->debug_id;
__entry->why = why;
__entry->content = content;
__entry->ino = ino;
__entry->aux = be64_to_cpup((__be64 *)obj->cookie->inline_aux);
- __entry->disk_aux = disk_aux;
+
+ /* cachefiles_xattr::data is 2-byte aligned but not 8-byte aligned. */
+ if (disk_aux) {
+ x.s[0] = ((__be16 *)disk_aux)[0];
+ x.s[1] = ((__be16 *)disk_aux)[1];
+ x.s[2] = ((__be16 *)disk_aux)[2];
+ x.s[3] = ((__be16 *)disk_aux)[3];
+ __entry->disk_aux = be64_to_cpu(x.ll);
+ } else {
+ __entry->disk_aux = 0;
+ }
),
TP_printk("o=%08x %s B=%llx c=%u aux=%llx dsk=%llx",
diff --git a/include/trace/events/icmp.h b/include/trace/events/icmp.h
index 31559796949a..acfcf2b132d6 100644
--- a/include/trace/events/icmp.h
+++ b/include/trace/events/icmp.h
@@ -27,17 +27,20 @@ TRACE_EVENT(icmp_send,
TP_fast_assign(
struct iphdr *iph = ip_hdr(skb);
- struct udphdr *uh = udp_hdr(skb);
- int proto_4 = iph->protocol;
+ struct udphdr _uh, *uh = NULL;
__be32 *p32;
__entry->skbaddr = skb;
__entry->type = type;
__entry->code = code;
- if (proto_4 != IPPROTO_UDP || (u8 *)uh < skb->head ||
- (u8 *)uh + sizeof(struct udphdr)
- > skb_tail_pointer(skb)) {
+ if (iph->protocol == IPPROTO_UDP)
+ uh = skb_header_pointer(skb,
+ skb_network_offset(skb) +
+ (iph->ihl << 2),
+ sizeof(_uh), &_uh);
+
+ if (!uh) {
__entry->sport = 0;
__entry->dport = 0;
__entry->ulen = 0;
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 082cb03c6131..3fec3e8f91c8 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -59,6 +59,7 @@
EM(netfs_rreq_trace_free, "FREE ") \
EM(netfs_rreq_trace_intr, "INTR ") \
EM(netfs_rreq_trace_ki_complete, "KI-CMPL") \
+ EM(netfs_rreq_trace_ra_put_ref, "RA-PUT ") \
EM(netfs_rreq_trace_recollect, "RECLLCT") \
EM(netfs_rreq_trace_redirty, "REDIRTY") \
EM(netfs_rreq_trace_resubmit, "RESUBMT") \
@@ -70,9 +71,11 @@
EM(netfs_rreq_trace_unpause, "UNPAUSE") \
EM(netfs_rreq_trace_wait_ip, "WAIT-IP") \
EM(netfs_rreq_trace_wait_pause, "--PAUSED--") \
+ EM(netfs_rreq_trace_wait_put_ra_refs, "WAIT-P-RA") \
EM(netfs_rreq_trace_wait_quiesce, "WAIT-QUIESCE") \
EM(netfs_rreq_trace_waited_ip, "DONE-IP") \
EM(netfs_rreq_trace_waited_pause, "--UNPAUSED--") \
+ EM(netfs_rreq_trace_waited_put_ra_refs, "DONE-P-RA") \
EM(netfs_rreq_trace_waited_quiesce, "DONE-QUIESCE") \
EM(netfs_rreq_trace_wake_ip, "WAKE-IP") \
EM(netfs_rreq_trace_wake_queue, "WAKE-Q ") \
@@ -195,7 +198,6 @@
EM(netfs_folio_trace_clear_cc, "clear-cc") \
EM(netfs_folio_trace_clear_g, "clear-g") \
EM(netfs_folio_trace_clear_s, "clear-s") \
- EM(netfs_folio_trace_copy_to_cache, "mark-copy") \
EM(netfs_folio_trace_end_copy, "end-copy") \
EM(netfs_folio_trace_filled_gaps, "filled-gaps") \
EM(netfs_folio_trace_invalidate_all, "inval-all") \
@@ -206,16 +208,19 @@
EM(netfs_folio_trace_kill_cc, "kill-cc") \
EM(netfs_folio_trace_kill_g, "kill-g") \
EM(netfs_folio_trace_kill_s, "kill-s") \
+ EM(netfs_folio_trace_mark_copy, "mark-copy") \
EM(netfs_folio_trace_mkwrite, "mkwrite") \
EM(netfs_folio_trace_mkwrite_plus, "mkwrite+") \
- EM(netfs_folio_trace_not_under_wback, "!wback") \
EM(netfs_folio_trace_not_locked, "!locked") \
+ EM(netfs_folio_trace_not_under_wback, "!wback") \
+ EM(netfs_folio_trace_pgpriv2_copy, "pgpriv2-copy") \
EM(netfs_folio_trace_put, "put") \
EM(netfs_folio_trace_read, "read") \
EM(netfs_folio_trace_read_done, "read-done") \
EM(netfs_folio_trace_read_gaps, "read-gaps") \
EM(netfs_folio_trace_read_unlock, "read-unlock") \
EM(netfs_folio_trace_redirtied, "redirtied") \
+ EM(netfs_folio_trace_sched_copy, "sched-copy") \
EM(netfs_folio_trace_store, "store") \
EM(netfs_folio_trace_store_copy, "store-copy") \
EM(netfs_folio_trace_store_plus, "store+") \
@@ -786,6 +791,27 @@ TRACE_EVENT(netfs_folioq,
__print_symbolic(__entry->trace, netfs_folioq_traces))
);
+TRACE_EVENT(netfs_read_progress_at,
+ TP_PROTO(const struct netfs_io_request *rreq),
+
+ TP_ARGS(rreq),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, rreq)
+ __field(size_t, progress_at)
+ __field(size_t, cleaned_to)
+ ),
+
+ TP_fast_assign(
+ __entry->rreq = rreq->debug_id;
+ __entry->cleaned_to = rreq->cleaned_to - rreq->start;
+ __entry->progress_at = rreq->progress_at;
+ ),
+
+ TP_printk("R=%08x cln=%zx prg=%zx",
+ __entry->rreq, __entry->cleaned_to, __entry->progress_at)
+ );
+
#undef EM
#undef E_
#endif /* _TRACE_NETFS_H */
diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h
index 9f3090db2f16..322aef9b54bb 100644
--- a/include/uapi/drm/amdgpu_drm.h
+++ b/include/uapi/drm/amdgpu_drm.h
@@ -1510,6 +1510,7 @@ struct drm_amdgpu_info_device {
__u64 high_va_max;
/* gfx10 pa_sc_tile_steering_override */
__u32 pa_sc_tile_steering_override;
+ __u32 pad;
/* disabled TCCs */
__u64 tcc_disabled_mask;
__u64 min_engine_clock;
@@ -1534,7 +1535,6 @@ struct drm_amdgpu_info_device {
__u32 csa_alignment;
/* Userq IP mask (1 << AMDGPU_HW_IP_*) */
__u32 userq_ip_mask;
- __u32 pad;
};
struct drm_amdgpu_info_hw_ip {
diff --git a/init/main.c b/init/main.c
index e363232b428b..dd7b2a6599fc 100644
--- a/init/main.c
+++ b/init/main.c
@@ -276,7 +276,8 @@ static void * __init get_boot_config_from_initrd(size_t *_size)
u8 *hdr;
int i;
- if (!initrd_end)
+ if (!initrd_end || initrd_end < initrd_start ||
+ initrd_end - initrd_start < BOOTCONFIG_MAGIC_LEN + 8)
return NULL;
data = (char *)initrd_end - BOOTCONFIG_MAGIC_LEN;
@@ -293,16 +294,26 @@ static void * __init get_boot_config_from_initrd(size_t *_size)
found:
hdr = (u8 *)(data - 8);
+ if ((unsigned long)hdr < initrd_start)
+ return NULL;
+
size = get_unaligned_le32(hdr);
csum = get_unaligned_le32(hdr + 4);
- data = ((void *)hdr) - size;
- if ((unsigned long)data < initrd_start) {
- pr_err("bootconfig size %d is greater than initrd size %ld\n",
+ if (size > XBC_DATA_MAX) {
+ pr_err("bootconfig size %u is greater than max size %d\n",
+ size, XBC_DATA_MAX);
+ return NULL;
+ }
+
+ if (size > ((unsigned long)hdr - initrd_start)) {
+ pr_err("bootconfig size %u is greater than initrd size %lu\n",
size, initrd_end - initrd_start);
return NULL;
}
+ data = ((void *)hdr) - size;
+
if (xbc_calc_checksum(data, size) != csum) {
pr_err("bootconfig checksum failed\n");
return NULL;
@@ -405,12 +416,6 @@ static void __init setup_boot_config(void)
return;
}
- if (size >= XBC_DATA_MAX) {
- pr_err("bootconfig size %ld greater than max size %d\n",
- (long)size, XBC_DATA_MAX);
- return;
- }
-
ret = xbc_init(data, size, &msg, &pos);
if (ret < 0) {
if (pos < 0)
@@ -1636,7 +1641,7 @@ static noinline void __init kernel_init_freeable(void)
*/
set_mems_allowed(node_states[N_MEMORY]);
- cad_pid = get_pid(task_pid(current));
+ rcu_assign_pointer(cad_pid, get_pid(task_pid(current)));
smp_prepare_cpus(setup_max_cpus);
diff --git a/io_uring/net.c b/io_uring/net.c
index a74d15f7b7d2..e6847fb084ab 100644
--- a/io_uring/net.c
+++ b/io_uring/net.c
@@ -884,7 +884,7 @@ int io_recvmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
static inline bool io_recv_finish(struct io_kiocb *req,
struct io_async_msghdr *kmsg,
struct io_br_sel *sel, bool mshot_finished,
- unsigned issue_flags)
+ unsigned issue_flags, int consumed)
{
struct io_sr_msg *sr = io_kiocb_to_cmd(req, struct io_sr_msg);
unsigned int cflags = 0;
@@ -908,7 +908,7 @@ static inline bool io_recv_finish(struct io_kiocb *req,
if (sr->flags & IORING_RECVSEND_BUNDLE) {
size_t this_ret = sel->val - sr->done_io;
- cflags |= io_put_kbufs(req, this_ret, sel->buf_list, io_bundle_nbufs(kmsg, this_ret));
+ cflags |= io_put_kbufs(req, consumed, sel->buf_list, io_bundle_nbufs(kmsg, consumed));
if (sr->flags & IORING_RECV_RETRY)
cflags = req->cqe.flags | (cflags & CQE_F_MASK);
if (sr->mshot_len && sel->val >= sr->mshot_len)
@@ -930,7 +930,7 @@ static inline bool io_recv_finish(struct io_kiocb *req,
return false;
}
} else {
- cflags |= io_put_kbuf(req, sel->val, sel->buf_list);
+ cflags |= io_put_kbuf(req, consumed, sel->buf_list);
}
/*
@@ -1058,6 +1058,8 @@ int io_recvmsg(struct io_kiocb *req, unsigned int issue_flags)
int ret, min_ret = 0;
bool force_nonblock = issue_flags & IO_URING_F_NONBLOCK;
bool mshot_finished = true;
+ int consumed = 0;
+ size_t len;
sock = sock_from_file(req->file);
if (unlikely(!sock))
@@ -1073,9 +1075,8 @@ int io_recvmsg(struct io_kiocb *req, unsigned int issue_flags)
retry_multishot:
sel.buf_list = NULL;
+ len = sr->len;
if (io_do_buffer_select(req)) {
- size_t len = sr->len;
-
sel = io_buffer_select(req, &len, sr->buf_group, issue_flags);
if (!sel.addr)
return -ENOBUFS;
@@ -1096,6 +1097,7 @@ int io_recvmsg(struct io_kiocb *req, unsigned int issue_flags)
if (req->flags & REQ_F_APOLL_MULTISHOT) {
ret = io_recvmsg_multishot(sock, sr, kmsg, flags,
&mshot_finished);
+ consumed = ret;
} else {
/* disable partial retry for recvmsg with cmsg attached */
if (flags & MSG_WAITALL && !kmsg->msg.msg_controllen)
@@ -1103,6 +1105,15 @@ int io_recvmsg(struct io_kiocb *req, unsigned int issue_flags)
ret = __sys_recvmsg_sock(sock, &kmsg->msg, sr->umsg,
kmsg->uaddr, flags);
+ /*
+ * With MSG_TRUNC, the net layer will return the full size of
+ * the packet, even if we only filled part of it in the buffers.
+ * Adjust the returned size to consume only the real part of the
+ * buffer.
+ */
+ consumed = ret;
+ if (ret > 0)
+ consumed = min_t(size_t, ret, len);
}
if (ret < min_ret) {
@@ -1129,7 +1140,7 @@ int io_recvmsg(struct io_kiocb *req, unsigned int issue_flags)
io_kbuf_recycle(req, sel.buf_list, issue_flags);
sel.val = ret;
- if (!io_recv_finish(req, kmsg, &sel, mshot_finished, issue_flags))
+ if (!io_recv_finish(req, kmsg, &sel, mshot_finished, issue_flags, consumed))
goto retry_multishot;
return sel.val;
@@ -1139,6 +1150,7 @@ static int io_recv_buf_select(struct io_kiocb *req, struct io_async_msghdr *kmsg
struct io_br_sel *sel, unsigned int issue_flags)
{
struct io_sr_msg *sr = io_kiocb_to_cmd(req, struct io_sr_msg);
+ size_t len;
int ret;
/*
@@ -1184,13 +1196,14 @@ static int io_recv_buf_select(struct io_kiocb *req, struct io_async_msghdr *kmsg
/* special case 1 vec, can be a fast path */
if (ret == 1) {
sr->buf = arg.iovs[0].iov_base;
- sr->len = arg.iovs[0].iov_len;
+ len = sr->len = arg.iovs[0].iov_len;
goto map_ubuf;
}
iov_iter_init(&kmsg->msg.msg_iter, ITER_DEST, arg.iovs, ret,
- arg.out_len);
+ arg.out_len);
+ len = arg.out_len;
} else {
- size_t len = sel->val;
+ len = sel->val;
*sel = io_buffer_select(req, &len, sr->buf_group, issue_flags);
if (!sel->addr)
@@ -1204,7 +1217,7 @@ static int io_recv_buf_select(struct io_kiocb *req, struct io_async_msghdr *kmsg
return ret;
}
- return 0;
+ return len;
}
int io_recv(struct io_kiocb *req, unsigned int issue_flags)
@@ -1214,9 +1227,10 @@ int io_recv(struct io_kiocb *req, unsigned int issue_flags)
struct io_br_sel sel;
struct socket *sock;
unsigned flags;
- int ret, min_ret = 0;
+ int ret, min_ret = 0, consumed = 0;
bool force_nonblock = issue_flags & IO_URING_F_NONBLOCK;
bool mshot_finished;
+ size_t len = 0;
sock = sock_from_file(req->file);
if (unlikely(!sock))
@@ -1244,6 +1258,7 @@ int io_recv(struct io_kiocb *req, unsigned int issue_flags)
retry_multishot:
sel.buf_list = NULL;
+ len = sr->len;
if (io_do_buffer_select(req)) {
sel.val = sr->len;
ret = io_recv_buf_select(req, kmsg, &sel, issue_flags);
@@ -1251,6 +1266,7 @@ int io_recv(struct io_kiocb *req, unsigned int issue_flags)
kmsg->msg.msg_inq = -1;
goto out_free;
}
+ len = ret;
sr->buf = NULL;
}
@@ -1281,6 +1297,17 @@ int io_recv(struct io_kiocb *req, unsigned int issue_flags)
}
mshot_finished = ret <= 0;
+
+ /*
+ * With MSG_TRUNC, the net layer will return the full size of
+ * the packet, even if we only filled part of it in the buffers.
+ * Adjust the returned size to consume only the real part of the
+ * buffer.
+ */
+ consumed = ret;
+ if (ret > 0)
+ consumed = min_t(size_t, ret, len);
+
if (ret > 0)
ret += sr->done_io;
else if (sr->done_io)
@@ -1289,7 +1316,7 @@ int io_recv(struct io_kiocb *req, unsigned int issue_flags)
io_kbuf_recycle(req, sel.buf_list, issue_flags);
sel.val = ret;
- if (!io_recv_finish(req, kmsg, &sel, mshot_finished, issue_flags))
+ if (!io_recv_finish(req, kmsg, &sel, mshot_finished, issue_flags, consumed))
goto retry_multishot;
return sel.val;
diff --git a/io_uring/rw.c b/io_uring/rw.c
index 95038cfda615..45a96e6546fc 100644
--- a/io_uring/rw.c
+++ b/io_uring/rw.c
@@ -531,20 +531,25 @@ static void io_req_end_write(struct io_kiocb *req)
}
}
-/*
- * Trigger the notifications after having done some IO, and finish the write
- * accounting, if any.
- */
-static void io_req_io_end(struct io_kiocb *req)
+/* Trigger the notifications after having done some IO. */
+static void io_req_io_notify(struct io_kiocb *req)
{
struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw);
- if (rw->kiocb.ki_flags & IOCB_WRITE) {
- io_req_end_write(req);
+ if (rw->kiocb.ki_flags & IOCB_WRITE)
fsnotify_modify(req->file);
- } else {
+ else
fsnotify_access(req->file);
- }
+}
+
+/* Finish write accounting and notify, for inline completions only. */
+static void io_req_io_end(struct io_kiocb *req)
+{
+ struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw);
+
+ if (rw->kiocb.ki_flags & IOCB_WRITE)
+ io_req_end_write(req);
+ io_req_io_notify(req);
}
static void __io_complete_rw_common(struct io_kiocb *req, long res)
@@ -577,7 +582,7 @@ void io_req_rw_complete(struct io_tw_req tw_req, io_tw_token_t tw)
{
struct io_kiocb *req = tw_req.req;
- io_req_io_end(req);
+ io_req_io_notify(req);
if (req->flags & (REQ_F_BUFFER_SELECTED|REQ_F_BUFFER_RING))
req->cqe.flags |= io_put_kbuf(req, max(req->cqe.res, 0), NULL);
@@ -591,6 +596,10 @@ static void io_complete_rw(struct kiocb *kiocb, long res)
struct io_rw *rw = container_of(kiocb, struct io_rw, kiocb);
struct io_kiocb *req = cmd_to_io_kiocb(rw);
+ /* ring owner may block in freeze_super() before task_work runs */
+ if (kiocb->ki_flags & IOCB_WRITE)
+ io_req_end_write(req);
+
__io_complete_rw_common(req, res);
io_req_set_res(req, io_fixup_rw_res(req, res), 0);
req->io_task_work.func = io_req_rw_complete;
diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c
index 248b4818178c..cc3f8c25a28b 100644
--- a/kernel/bpf/arraymap.c
+++ b/kernel/bpf/arraymap.c
@@ -405,7 +405,7 @@ int bpf_percpu_array_update(struct bpf_map *map, void *key, void *value,
void __percpu *pptr;
void *ptr, *val;
u32 size;
- int cpu;
+ int cpu, off = 0;
if (unlikely((map_flags & BPF_F_LOCK) || (u32)map_flags > BPF_F_ALL_CPUS))
/* unknown flags */
@@ -437,9 +437,10 @@ int bpf_percpu_array_update(struct bpf_map *map, void *key, void *value,
}
for_each_possible_cpu(cpu) {
ptr = per_cpu_ptr(pptr, cpu);
- val = (map_flags & BPF_F_ALL_CPUS) ? value : value + size * cpu;
+ val = (map_flags & BPF_F_ALL_CPUS) ? value : value + off;
copy_map_value(map, ptr, val);
bpf_obj_cancel_fields(map, ptr);
+ off += size;
}
unlock:
rcu_read_unlock();
diff --git a/kernel/bpf/backtrack.c b/kernel/bpf/backtrack.c
index 2e4ae0ef0860..53a1e52d73c5 100644
--- a/kernel/bpf/backtrack.c
+++ b/kernel/bpf/backtrack.c
@@ -521,37 +521,34 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
return -EFAULT;
}
} else if (opcode == BPF_EXIT) {
- bool r0_precise;
+ bool from_subprog_call, r0_precise;
+
+ /* BPF_EXIT in subprog or callback always returns
+ * right after the call instruction, so by checking
+ * whether the instruction at subseq_idx-1 is subprog
+ * call or not we can distinguish actual exit from
+ * *subprog* from exit from *callback*. In the former
+ * case, we need to propagate r0 precision, if
+ * necessary. In the former we never do that.
+ */
+ from_subprog_call = subseq_idx - 1 >= 0 &&
+ bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]);
+
+ r0_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_0);
/* Backtracking to a nested function call, 'idx' is a part of
* the inner frame 'subseq_idx' is a part of the outer frame.
* In case of a regular function call, instructions giving
* precision to registers R1-R5 should have been found already.
- * In case of a callback, it is ok to have R1-R5 marked for
- * backtracking, as these registers are set by the function
- * invoking callback.
+ * In case of a callback from bpf_loop(), R{1,4} in the calling
+ * frame would be set as precise and that is correct.
*/
- if (subseq_idx >= 0 && bpf_calls_callback(env, subseq_idx))
- for (i = BPF_REG_1; i <= BPF_REG_5; i++)
- bt_clear_reg(bt, i);
- if (bt_reg_mask(bt) & BPF_REGMASK_ARGS) {
+ if (from_subprog_call && (bt_reg_mask(bt) & BPF_REGMASK_ARGS)) {
verifier_bug(env, "backtracking exit unexpected regs %x",
bt_reg_mask(bt));
return -EFAULT;
}
- /* BPF_EXIT in subprog or callback always returns
- * right after the call instruction, so by checking
- * whether the instruction at subseq_idx-1 is subprog
- * call or not we can distinguish actual exit from
- * *subprog* from exit from *callback*. In the former
- * case, we need to propagate r0 precision, if
- * necessary. In the former we never do that.
- */
- r0_precise = subseq_idx - 1 >= 0 &&
- bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]) &&
- bt_is_reg_set(bt, BPF_REG_0);
-
bt_clear_reg(bt, BPF_REG_0);
if (bt_subprog_enter(bt))
return -EFAULT;
@@ -583,16 +580,29 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
*/
}
} else if (class == BPF_LD) {
- if (!bt_is_reg_set(bt, dreg))
- return 0;
- bt_clear_reg(bt, dreg);
/* It's ld_imm64 or ld_abs or ld_ind.
* For ld_imm64 no further tracking of precision
* into parent is necessary
*/
- if (mode == BPF_IND || mode == BPF_ABS)
- /* to be analyzed */
- return -ENOTSUPP;
+ if (mode == BPF_IMM) {
+ bt_clear_reg(bt, dreg);
+ return 0;
+ }
+ /*
+ * BPF_{IND,ABS} are modelled as two branches:
+ * - fallthrough;
+ * - implicit subprogram exit.
+ * It is necessary to switch current frame if
+ * implicit subprogram exit branch is backtracked.
+ */
+ if (mode == BPF_IND || mode == BPF_ABS) {
+ if (bt_is_reg_set(bt, dreg))
+ return -ENOTSUPP;
+ if (subseq_idx != idx + 1)
+ if (bt_subprog_enter(bt))
+ return -EFAULT;
+ return 0;
+ }
}
/* Propagate precision marks to linked registers, to account for
* registers marked as precise in this function.
diff --git a/kernel/bpf/bpf_iter.c b/kernel/bpf/bpf_iter.c
index f5eaeb2493d4..9b4f5236c5ea 100644
--- a/kernel/bpf/bpf_iter.c
+++ b/kernel/bpf/bpf_iter.c
@@ -754,7 +754,7 @@ const struct bpf_func_proto bpf_loop_proto = {
.func = bpf_loop,
.gpl_only = false,
.ret_type = RET_INTEGER,
- .arg1_type = ARG_ANYTHING,
+ .arg1_type = ARG_SCALAR,
.arg2_type = ARG_PTR_TO_FUNC,
.arg3_type = ARG_PTR_TO_STACK_OR_NULL,
.arg4_type = ARG_ANYTHING,
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 608be952717d..022d5a594dc9 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -2912,14 +2912,29 @@ static void btf_modifier_show(const struct btf *btf,
else
t = btf_type_skip_modifiers(btf, type_id, NULL);
- btf_type_ops(t)->show(btf, t, type_id, data, bits_offset, show);
+ /*
+ * A modifier can resolve to void, which has no show op; print a
+ * placeholder rather than dereferencing NULL.
+ */
+ if (!btf_type_ops(t))
+ btf_df_show(btf, t, type_id, data, bits_offset, show);
+ else
+ btf_type_ops(t)->show(btf, t, type_id, data, bits_offset, show);
}
static void btf_var_show(const struct btf *btf, const struct btf_type *t,
u32 type_id, void *data, u8 bits_offset,
struct btf_show *show)
{
- t = btf_type_id_resolve(btf, &type_id);
+ /*
+ * btf_type_id_resolve() dereferences btf->resolved_ids, which is NULL
+ * for a base BTF (e.g. the vmlinux BTF that bpf_snprintf_btf() uses).
+ * Resolve the var's type directly in that case.
+ */
+ if (btf->resolved_ids)
+ t = btf_type_id_resolve(btf, &type_id);
+ else
+ t = btf_type_skip_modifiers(btf, t->type, &type_id);
btf_type_ops(t)->show(btf, t, type_id, data, bits_offset, show);
}
@@ -6658,6 +6673,10 @@ struct bpf_raw_tp_null_args {
static const struct bpf_raw_tp_null_args raw_tp_null_args[] = {
/* sched */
{ "sched_pi_setprio", 0x10 },
+ /*
+ * do_wait() passes NULL for wait4(-1) and waitid(P_ALL).
+ */
+ { "sched_process_wait", 0x1 },
/* ... from sched_numa_pair_template event class */
{ "sched_stick_numa", 0x100 },
{ "sched_swap_numa", 0x100 },
@@ -6725,6 +6744,9 @@ static const struct bpf_raw_tp_null_args raw_tp_null_args[] = {
{ "rxrpc_resend", 0x10 },
{ "rxrpc_tq", 0x10 },
{ "rxrpc_client", 0x1 },
+ /* signal */
+ { "signal_generate", 0x20 },
+ { "signal_deliver", 0x20 },
/* skb */
{"kfree_skb", 0x1000},
/* sunrpc */
@@ -8707,6 +8729,7 @@ BPF_CALL_4(bpf_btf_find_by_name_kind, char *, name, int, name_sz, u32, kind, int
const struct bpf_func_proto bpf_btf_find_by_name_kind_proto = {
.func = bpf_btf_find_by_name_kind,
.gpl_only = false,
+ .might_sleep = true,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg2_type = ARG_CONST_SIZE,
diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c
index 26d37066465f..d5fc03f12969 100644
--- a/kernel/bpf/cfg.c
+++ b/kernel/bpf/cfg.c
@@ -120,6 +120,7 @@ static int push_insn(int t, int w, int e, struct bpf_verifier_env *env)
/* mark branch target for state pruning */
mark_prune_point(env, w);
mark_jmp_point(env, w);
+ mark_jump_target(env, w);
}
if (insn_state[w] == 0) {
@@ -378,6 +379,7 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
}
mark_jmp_point(env, w);
+ mark_jump_target(env, w);
/* EXPLORED || DISCOVERED */
if (insn_state[w])
@@ -539,6 +541,7 @@ static int visit_insn(int t, struct bpf_verifier_env *env)
mark_prune_point(env, t + off + 1);
mark_jmp_point(env, t + off + 1);
+ mark_jump_target(env, t + off + 1);
return ret;
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index cd42eb838d6c..48acb61854ed 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -13,10 +13,15 @@
#define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args)
+/*
+ * Matches BPF_PROBE_ATOMIC too: bpf_convert_ctx_accesses() rewrites arena
+ * atomics before bpf_opt_subreg_zext_lo32_rnd_hi32() runs.
+ */
static bool is_cmpxchg_insn(const struct bpf_insn *insn)
{
return BPF_CLASS(insn->code) == BPF_STX &&
- BPF_MODE(insn->code) == BPF_ATOMIC &&
+ (BPF_MODE(insn->code) == BPF_ATOMIC ||
+ BPF_MODE(insn->code) == BPF_PROBE_ATOMIC) &&
insn->imm == BPF_CMPXCHG;
}
diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index 9f394e1aa2e8..59406da06424 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -530,6 +530,9 @@ static int htab_map_check_btf(struct bpf_map *map, const struct btf *btf,
{
struct bpf_htab *htab = container_of(map, struct bpf_htab, map);
+ if (btf_type_is_void(key_type))
+ return -EINVAL;
+
if (htab_is_prealloc(htab))
return 0;
/*
@@ -1026,7 +1029,7 @@ static void pcpu_copy_value(struct bpf_htab *htab, void __percpu *pptr,
} else {
u32 size = round_up(htab->map.value_size, 8);
void *val;
- int cpu;
+ int cpu, off = 0;
if (map_flags & BPF_F_CPU) {
cpu = map_flags >> 32;
@@ -1038,9 +1041,10 @@ static void pcpu_copy_value(struct bpf_htab *htab, void __percpu *pptr,
for_each_possible_cpu(cpu) {
ptr = per_cpu_ptr(pptr, cpu);
- val = (map_flags & BPF_F_ALL_CPUS) ? value : value + size * cpu;
+ val = (map_flags & BPF_F_ALL_CPUS) ? value : value + off;
copy_map_value(&htab->map, ptr, val);
bpf_obj_cancel_fields(&htab->map, ptr);
+ off += size;
}
}
}
@@ -2865,16 +2869,6 @@ static int rhtab_map_alloc_check(union bpf_attr *attr)
return htab_map_alloc_check(attr);
}
-static void rhtab_check_and_free_fields(struct bpf_rhtab *rhtab,
- struct rhtab_elem *elem)
-{
- if (IS_ERR_OR_NULL(rhtab->map.record))
- return;
-
- bpf_obj_free_fields(rhtab->map.record,
- rhtab_elem_value(elem, rhtab->map.key_size));
-}
-
static void rhtab_mem_dtor(void *obj, void *ctx)
{
struct htab_btf_record *hrec = ctx;
@@ -2964,8 +2958,8 @@ static int rhtab_delete_elem(struct bpf_rhtab *rhtab, struct rhtab_elem *elem, v
rhtab_read_elem_value(&rhtab->map, copy, elem, flags);
check_and_init_map_value(&rhtab->map, copy);
}
- /* Release internal structs: kptr, bpf_timer, task_work, wq */
- rhtab_check_and_free_fields(rhtab, elem);
+ bpf_obj_cancel_fields(&rhtab->map,
+ rhtab_elem_value(elem, rhtab->map.key_size));
bpf_mem_cache_free_rcu(&rhtab->ma, elem);
return 0;
}
@@ -3007,7 +3001,6 @@ static int rhtab_map_lookup_and_delete_elem(struct bpf_map *map, void *key, void
static long rhtab_map_update_existing(struct bpf_map *map, struct rhtab_elem *elem, void *value,
u64 map_flags)
{
- struct bpf_rhtab *rhtab = container_of(map, struct bpf_rhtab, map);
void *old_val = rhtab_elem_value(elem, map->key_size);
if (map_flags & BPF_NOEXIST)
@@ -3027,7 +3020,7 @@ static long rhtab_map_update_existing(struct bpf_map *map, struct rhtab_elem *el
* kptrs/etc. still sit in the slot. Cancel them after the copy
* to match arraymap's update semantics.
*/
- rhtab_check_and_free_fields(rhtab, elem);
+ bpf_obj_cancel_fields(map, old_val);
return 0;
}
@@ -3068,7 +3061,6 @@ static long rhtab_map_update_elem(struct bpf_map *map, void *key, void *value, u
memcpy(elem->data, key, map->key_size);
copy_map_value(map, rhtab_elem_value(elem, map->key_size), value);
- check_and_init_map_value(map, rhtab_elem_value(elem, map->key_size));
/* Prevent deadlock for NMI programs attempting to take bucket lock */
bpf_disable_instrumentation();
@@ -3112,6 +3104,9 @@ static int rhtab_map_check_btf(struct bpf_map *map, const struct btf *btf,
{
struct bpf_rhtab *rhtab = container_of(map, struct bpf_rhtab, map);
+ if (btf_type_is_void(key_type))
+ return -EINVAL;
+
return bpf_ma_set_dtor(map, &rhtab->ma, rhtab_mem_dtor);
}
diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c
index 23267213a17f..83cd527a2542 100644
--- a/kernel/bpf/local_storage.c
+++ b/kernel/bpf/local_storage.c
@@ -220,7 +220,7 @@ int bpf_percpu_cgroup_storage_update(struct bpf_map *_map, void *key,
struct bpf_cgroup_storage *storage;
void *val;
u32 size;
- int cpu;
+ int cpu, off = 0;
if ((u32)map_flags & ~(BPF_ANY | BPF_EXIST | BPF_F_CPU | BPF_F_ALL_CPUS))
return -EINVAL;
@@ -245,8 +245,9 @@ int bpf_percpu_cgroup_storage_update(struct bpf_map *_map, void *key,
}
size = round_up(_map->value_size, 8);
for_each_possible_cpu(cpu) {
- val = (map_flags & BPF_F_ALL_CPUS) ? value : value + size * cpu;
+ val = (map_flags & BPF_F_ALL_CPUS) ? value : value + off;
copy_map_value(_map, per_cpu_ptr(storage->percpu_buf, cpu), val);
+ off += size;
}
unlock:
rcu_read_unlock();
diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c
index 142a65672c6f..e41fb1c567cd 100644
--- a/kernel/bpf/stackmap.c
+++ b/kernel/bpf/stackmap.c
@@ -880,6 +880,7 @@ BPF_CALL_4(bpf_get_stack_sleepable, struct pt_regs *, regs, void *, buf, u32, si
const struct bpf_func_proto bpf_get_stack_sleepable_proto = {
.func = bpf_get_stack_sleepable,
.gpl_only = true,
+ .might_sleep = true,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_PTR_TO_UNINIT_MEM,
@@ -933,6 +934,7 @@ BPF_CALL_4(bpf_get_task_stack_sleepable, struct task_struct *, task, void *, buf
const struct bpf_func_proto bpf_get_task_stack_sleepable_proto = {
.func = bpf_get_task_stack_sleepable,
.gpl_only = false,
+ .might_sleep = true,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_BTF_ID,
.arg1_btf_id = &btf_tracing_ids[BTF_TRACING_TYPE_TASK],
diff --git a/kernel/bpf/states.c b/kernel/bpf/states.c
index 4e6aafad33bd..66fb11b6c6a7 100644
--- a/kernel/bpf/states.c
+++ b/kernel/bpf/states.c
@@ -445,22 +445,19 @@ static void __clean_func_state(struct bpf_verifier_env *env,
struct bpf_reg_state *spill = &st->stack[i].spilled_ptr;
if (lo_live && stype == STACK_SPILL) {
- u8 val = STACK_MISC;
-
if (spill->type != SCALAR_VALUE)
continue;
-
/*
- * 8 byte spill of scalar 0 where half slot is dead
- * should become STACK_ZERO in lo 4 bytes.
+ * Can't replace with STACK_ZERO, because
+ * that requires bpf_mark_chain_precision().
*/
if (bpf_register_is_null(spill))
- val = STACK_ZERO;
+ continue;
for (j = 0; j < 4; j++) {
u8 *t = &st->stack[i].slot_type[j];
if (*t == STACK_SPILL)
- *t = val;
+ *t = STACK_MISC;
}
}
bpf_mark_reg_not_init(env, spill);
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index fb678b9dcd3e..c7cb336fb064 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -6633,6 +6633,7 @@ EXPORT_SYMBOL_NS(kern_sys_bpf, "BPF_INTERNAL");
static const struct bpf_func_proto bpf_sys_bpf_proto = {
.func = bpf_sys_bpf,
.gpl_only = false,
+ .might_sleep = true,
.ret_type = RET_INTEGER,
.arg1_type = ARG_ANYTHING,
.arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY,
@@ -6658,6 +6659,7 @@ BPF_CALL_1(bpf_sys_close, u32, fd)
static const struct bpf_func_proto bpf_sys_close_proto = {
.func = bpf_sys_close,
.gpl_only = false,
+ .might_sleep = true,
.ret_type = RET_INTEGER,
.arg1_type = ARG_ANYTHING,
};
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 14a9fd2d5472..1a0cd37b03cd 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -367,9 +367,18 @@ static bool reg_not_null(struct bpf_verifier_env *env, const struct bpf_reg_stat
if (type_may_be_null(type))
return false;
+ /*
+ * The types below guarantee a non-NULL base, an unbounded offset can
+ * still wrap base + offset to zero.
+ */
+ if (reg_smin(reg) <= -BPF_MAX_VAR_OFF || reg_smax(reg) >= BPF_MAX_VAR_OFF)
+ return false;
+
type = base_type(type);
return type == PTR_TO_SOCKET ||
type == PTR_TO_TCP_SOCK ||
+ type == PTR_TO_XDP_SOCK ||
+ type == PTR_TO_BUF ||
type == PTR_TO_MAP_VALUE ||
type == PTR_TO_MAP_KEY ||
type == PTR_TO_SOCK_COMMON ||
@@ -1611,6 +1620,8 @@ static int copy_func_state(struct bpf_func_state *dst,
const struct bpf_func_state *src)
{
memcpy(dst, src, offsetof(struct bpf_func_state, stack));
+ /* Instruction accounting is path-local, not part of verifier state. */
+ dst->insns_subtotal = 0;
return copy_stack_state(dst, src);
}
@@ -4309,6 +4320,13 @@ static int map_kptr_match_type(struct bpf_verifier_env *env,
if (type_flag(reg->type) & ~perm_flags)
goto bad_type;
+ /*
+ * A BPF_KPTR_PERCPU field is read back as MEM_PERCPU, so the value
+ * stored in it must carry the same flag.
+ */
+ if ((kptr_field->type == BPF_KPTR_PERCPU) != !!(reg->type & MEM_PERCPU))
+ goto bad_type;
+
/* We need to verify reg->type and reg->btf, before accessing reg->btf */
reg_name = btf_type_name(reg->btf, reg->btf_id);
@@ -4510,8 +4528,15 @@ static int check_map_kptr_access(struct bpf_verifier_env *env,
return ret;
} else if (class == BPF_STX) {
val_reg = reg_state(env, value_regno);
- if (!bpf_register_is_null(val_reg) &&
- map_kptr_match_type(env, kptr_field, val_reg, value_regno))
+ if (bpf_register_is_null(val_reg)) {
+ /*
+ * This store is valid only because the scalar is known to be
+ * zero. Mark it precise so another scalar cannot be pruned
+ * against this state.
+ */
+ return mark_chain_precision(env, value_regno);
+ }
+ if (map_kptr_match_type(env, kptr_field, val_reg, value_regno))
return -EACCES;
} else if (class == BPF_ST) {
if (insn->imm) {
@@ -5115,6 +5140,15 @@ static int check_max_stack_depth_subprog(struct bpf_verifier_env *env, int idx,
if (!priv_stack_supported)
subprog[idx].priv_stack_mode = NO_PRIV_STACK;
process_func:
+ if (subprog[idx].has_ld_abs) {
+ for (tmp = idx; tmp >= 0; tmp = dinfo[tmp].caller) {
+ if (subprog[tmp].is_cb) {
+ verbose(env, "cannot use BPF_LD_[ABS|IND] within callback\n");
+ return -EINVAL;
+ }
+ }
+ }
+
/* protect against potential stack overflow that might happen when
* bpf2bpf calls get combined with tailcalls. Limit the caller's stack
* depth for such case down to 256 so that the worst case scenario
@@ -7843,6 +7877,7 @@ static const struct bpf_reg_types *compatible_reg_types[__BPF_ARG_TYPE_MAX] = {
[ARG_CONST_SIZE] = &scalar_types,
[ARG_CONST_SIZE_OR_ZERO] = &scalar_types,
[ARG_CONST_ALLOC_SIZE_OR_ZERO] = &scalar_types,
+ [ARG_SCALAR] = &scalar_types,
[ARG_CONST_MAP_PTR] = &const_map_ptr_types,
[ARG_PTR_TO_CTX] = &context_types,
[ARG_PTR_TO_SOCK_COMMON] = &sock_types,
@@ -8283,11 +8318,15 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg,
return err;
}
- if (bpf_register_is_null(reg) && type_may_be_null(arg_type))
+ if (bpf_register_is_null(reg) && type_may_be_null(arg_type)) {
/* A NULL register has a SCALAR_VALUE type, so skip
* type checking.
*/
+ err = mark_chain_precision(env, regno);
+ if (err)
+ return err;
goto skip_type_check;
+ }
/* arg_btf_id and arg_size are in a union. */
if (base_type(arg_type) == ARG_PTR_TO_BTF_ID ||
@@ -9495,10 +9534,12 @@ int map_set_for_each_callback_args(struct bpf_verifier_env *env,
callee->regs[BPF_REG_2].type = PTR_TO_MAP_KEY;
__mark_reg_known_zero(&callee->regs[BPF_REG_2]);
callee->regs[BPF_REG_2].map_ptr = caller->regs[BPF_REG_1].map_ptr;
+ callee->regs[BPF_REG_2].map_uid = caller->regs[BPF_REG_1].map_uid;
callee->regs[BPF_REG_3].type = PTR_TO_MAP_VALUE;
__mark_reg_known_zero(&callee->regs[BPF_REG_3]);
callee->regs[BPF_REG_3].map_ptr = caller->regs[BPF_REG_1].map_ptr;
+ callee->regs[BPF_REG_3].map_uid = caller->regs[BPF_REG_1].map_uid;
/* pointer to stack or null */
callee->regs[BPF_REG_4] = caller->regs[BPF_REG_3];
@@ -9576,6 +9617,7 @@ static int set_timer_callback_state(struct bpf_verifier_env *env,
int insn_idx)
{
struct bpf_map *map_ptr = caller->regs[BPF_REG_1].map_ptr;
+ u32 map_uid = caller->regs[BPF_REG_1].map_uid;
/* bpf_timer_set_callback(struct bpf_timer *timer, void *callback_fn);
* callback_fn(struct bpf_map *map, void *key, void *value);
@@ -9583,14 +9625,17 @@ static int set_timer_callback_state(struct bpf_verifier_env *env,
callee->regs[BPF_REG_1].type = CONST_PTR_TO_MAP;
__mark_reg_known_zero(&callee->regs[BPF_REG_1]);
callee->regs[BPF_REG_1].map_ptr = map_ptr;
+ callee->regs[BPF_REG_1].map_uid = map_uid;
callee->regs[BPF_REG_2].type = PTR_TO_MAP_KEY;
__mark_reg_known_zero(&callee->regs[BPF_REG_2]);
callee->regs[BPF_REG_2].map_ptr = map_ptr;
+ callee->regs[BPF_REG_2].map_uid = map_uid;
callee->regs[BPF_REG_3].type = PTR_TO_MAP_VALUE;
__mark_reg_known_zero(&callee->regs[BPF_REG_3]);
callee->regs[BPF_REG_3].map_ptr = map_ptr;
+ callee->regs[BPF_REG_3].map_uid = map_uid;
/* unused */
bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_4]);
@@ -9690,6 +9735,7 @@ static int set_task_work_schedule_callback_state(struct bpf_verifier_env *env,
int insn_idx)
{
struct bpf_map *map_ptr = caller->regs[BPF_REG_3].map_ptr;
+ u32 map_uid = caller->regs[BPF_REG_3].map_uid;
/*
* callback_fn(struct bpf_map *map, void *key, void *value);
@@ -9697,14 +9743,17 @@ static int set_task_work_schedule_callback_state(struct bpf_verifier_env *env,
callee->regs[BPF_REG_1].type = CONST_PTR_TO_MAP;
__mark_reg_known_zero(&callee->regs[BPF_REG_1]);
callee->regs[BPF_REG_1].map_ptr = map_ptr;
+ callee->regs[BPF_REG_1].map_uid = map_uid;
callee->regs[BPF_REG_2].type = PTR_TO_MAP_KEY;
__mark_reg_known_zero(&callee->regs[BPF_REG_2]);
callee->regs[BPF_REG_2].map_ptr = map_ptr;
+ callee->regs[BPF_REG_2].map_uid = map_uid;
callee->regs[BPF_REG_3].type = PTR_TO_MAP_VALUE;
__mark_reg_known_zero(&callee->regs[BPF_REG_3]);
callee->regs[BPF_REG_3].map_ptr = map_ptr;
+ callee->regs[BPF_REG_3].map_uid = map_uid;
/* unused */
bpf_mark_reg_not_init(env, &callee->regs[BPF_REG_4]);
@@ -9716,9 +9765,46 @@ static int set_task_work_schedule_callback_state(struct bpf_verifier_env *env,
static bool is_rbtree_lock_required_kfunc(u32 btf_id);
-/* Are we currently verifying the callback for a rbtree helper that must
- * be called with lock held? If so, no need to complain about unreleased
- * lock
+static void account_processed_insn(struct bpf_verifier_env *env)
+{
+ struct bpf_func_state *frame = cur_func(env);
+
+ env->insn_processed++;
+ frame->insns_subtotal++;
+ env->subprog_info[frame->subprogno].insns_self++;
+}
+
+static void account_processed_insns(struct bpf_verifier_env *env,
+ struct bpf_func_state *callee,
+ struct bpf_func_state *caller)
+{
+ u32 insns;
+
+ if (!callee)
+ return;
+
+ insns = callee->insns_subtotal;
+
+ env->subprog_info[callee->subprogno].insns_total += insns;
+ if (caller)
+ caller->insns_subtotal += insns;
+ callee->insns_subtotal = 0;
+}
+
+static void account_current_path(struct bpf_verifier_env *env)
+{
+ struct bpf_verifier_state *state = env->cur_state;
+ int frame;
+
+ for (frame = state->curframe; frame >= 0; frame--)
+ account_processed_insns(env, state->frame[frame],
+ frame ? state->frame[frame - 1] : NULL);
+}
+
+/*
+ * Are we currently verifying the callback for an rbtree kfunc that must
+ * be called with a lock held, or one of that callback's subprogs? If so,
+ * no need to complain about an unreleased lock.
*/
static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env)
{
@@ -9726,17 +9812,19 @@ static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env)
struct bpf_insn *insn = env->prog->insnsi;
struct bpf_func_state *callee;
int kfunc_btf_id;
+ u32 frame;
- if (!state->curframe)
- return false;
-
- callee = state->frame[state->curframe];
+ for (frame = state->curframe; frame; frame--) {
+ callee = state->frame[frame];
+ if (!callee->in_callback_fn)
+ continue;
- if (!callee->in_callback_fn)
- return false;
+ kfunc_btf_id = insn[callee->callsite].imm;
+ if (is_rbtree_lock_required_kfunc(kfunc_btf_id))
+ return true;
+ }
- kfunc_btf_id = insn[callee->callsite].imm;
- return is_rbtree_lock_required_kfunc(kfunc_btf_id);
+ return false;
}
static bool retval_range_within(struct bpf_retval_range range, const struct bpf_reg_state *reg)
@@ -9812,6 +9900,7 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)
verbose(env, "to caller at %d:\n", *insn_idx);
print_verifier_state(env, state, caller->frameno, true);
}
+ account_processed_insns(env, callee, caller);
/* clear everything in the callee. In case of exceptional exits using
* bpf_throw, this will be done by copy_verifier_state for extra frames. */
free_func_state(callee);
@@ -10100,33 +10189,45 @@ static struct bpf_insn_aux_data *cur_aux(const struct bpf_verifier_env *env)
return &env->insn_aux_data[env->insn_idx];
}
-static bool loop_flag_is_zero(struct bpf_verifier_env *env)
+/* Returns 1 if R4 is a known zero, 0 if it is not, a negative errno on error. */
+static int loop_flag_is_zero(struct bpf_verifier_env *env)
{
struct bpf_reg_state *reg = reg_state(env, BPF_REG_4);
- bool reg_is_null = bpf_register_is_null(reg);
+ int err;
- if (reg_is_null)
- mark_chain_precision(env, BPF_REG_4);
+ if (!bpf_register_is_null(reg))
+ return 0;
- return reg_is_null;
+ err = mark_chain_precision(env, BPF_REG_4);
+ if (err)
+ return err;
+ return 1;
}
-static void update_loop_inline_state(struct bpf_verifier_env *env, u32 subprogno)
+static int update_loop_inline_state(struct bpf_verifier_env *env, u32 subprogno)
{
struct bpf_loop_inline_state *state = &cur_aux(env)->loop_inline_state;
+ int flag_is_zero;
if (!state->initialized) {
+ flag_is_zero = loop_flag_is_zero(env);
+ if (flag_is_zero < 0)
+ return flag_is_zero;
state->initialized = 1;
- state->fit_for_inline = loop_flag_is_zero(env);
+ state->fit_for_inline = flag_is_zero;
state->callback_subprogno = subprogno;
- return;
+ return 0;
}
if (!state->fit_for_inline)
- return;
+ return 0;
- state->fit_for_inline = (loop_flag_is_zero(env) &&
+ flag_is_zero = loop_flag_is_zero(env);
+ if (flag_is_zero < 0)
+ return flag_is_zero;
+ state->fit_for_inline = (flag_is_zero &&
state->callback_subprogno == subprogno);
+ return 0;
}
/* Returns whether or not the given map can potentially elide
@@ -10318,6 +10419,9 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
verbose(env, "get_local_storage() doesn't support non-zero flags\n");
return -EINVAL;
}
+ err = mark_chain_precision(env, BPF_REG_2);
+ if (err)
+ return err;
break;
case BPF_FUNC_for_each_map_elem:
err = push_callback_call(env, insn, insn_idx, meta.subprogno,
@@ -10335,7 +10439,9 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
err = check_bpf_snprintf_call(env, regs);
break;
case BPF_FUNC_loop:
- update_loop_inline_state(env, meta.subprogno);
+ err = update_loop_inline_state(env, meta.subprogno);
+ if (err)
+ return err;
/* Verifier relies on R1 value to determine if bpf_loop() iteration
* is finished, thus mark it precise.
*/
@@ -10670,6 +10776,17 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
if (env->cur_state->curframe) {
struct bpf_verifier_state *branch;
+ /*
+ * A taken tail call is modeled as a return from the current
+ * frame. A callback frame cannot be left that way because
+ * prepare_func_exit() would apply its return contract to the
+ * unknown R0 synthesized below. Stack-depth validation rejects
+ * this construct anyway.
+ */
+ if (cur_func(env)->in_callback_fn) {
+ verbose(env, "cannot tail call within callback\n");
+ return -EINVAL;
+ }
mark_reg_scratched(env, BPF_REG_0);
branch = push_stack(env, env->insn_idx + 1, env->insn_idx, false);
if (IS_ERR(branch))
@@ -12459,7 +12576,7 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
reg_arg_name(env, argno));
return -EINVAL;
}
- if (!type_is_non_owning_ref(reg->type))
+ if (!type_is_non_owning_ref(reg->type) && reg_is_referenced(env, reg))
meta->arg_owning_ref = true;
rec = reg_btf_record(reg);
@@ -12531,6 +12648,11 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
{
int flags = PROCESS_RES_LOCK;
+ if (in_rbtree_lock_required_cb(env)) {
+ verbose(env, "can't res_spin_{lock,unlock} in rbtree cb\n");
+ return -EACCES;
+ }
+
if (reg->type != PTR_TO_MAP_VALUE && reg->type != (PTR_TO_BTF_ID | MEM_ALLOC)) {
verbose(env, "%s doesn't point to map value or allocated object\n",
reg_arg_name(env, argno));
@@ -13794,9 +13916,6 @@ static int adjust_ptr_min_max_vals(struct bpf_verifier_env *env,
!check_reg_sane_offset_ptr(env, ptr_reg, ptr_reg->type))
return -EINVAL;
- /* pointer types do not carry 32-bit bounds at the moment. */
- __mark_reg32_unbounded(dst_reg);
-
if (sanitize_needed(opcode)) {
ret = sanitize_ptr_alu(env, insn, ptr_reg, off_reg, dst_reg,
&info, false);
@@ -13804,6 +13923,14 @@ static int adjust_ptr_min_max_vals(struct bpf_verifier_env *env,
return sanitize_err(env, insn, ret);
}
+ /*
+ * Pointer types do not carry 32-bit bounds at the moment. Blank r32
+ * only after sanitize_ptr_alu() may have snapshotted dst_reg into a
+ * speculative path: otherwise reg_bounds_sanity_check() might hit some
+ * constraints violations.
+ */
+ __mark_reg32_unbounded(dst_reg);
+
switch (opcode) {
case BPF_ADD:
/*
@@ -15495,6 +15622,13 @@ static int is_branch_taken(struct bpf_verifier_env *env, struct bpf_reg_state *r
if (__is_pointer_value(false, reg1) || __is_pointer_value(false, reg2)) {
u64 val;
+ /*
+ * The low 32 bits of a valid pointer may well be zero, hence
+ * nothing below applies to a 32-bit comparison.
+ */
+ if (is_jmp32)
+ return -1;
+
/* arrange that reg2 is a scalar, and reg1 is a pointer */
if (!is_reg_const(reg2, is_jmp32)) {
opcode = flip_opcode(opcode);
@@ -16061,6 +16195,16 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
return err;
}
+ /*
+ * Collect the linked registers before env->{true,false}_reg{1,2} setup,
+ * otherwise ids dropped by collect_linked_regs() would be resurrected
+ * when env->{true,false}_reg{1,2} are copied back.
+ */
+ if (BPF_SRC(insn->code) == BPF_X && src_reg->type == SCALAR_VALUE && src_reg->id)
+ collect_linked_regs(env, this_branch, src_reg->id, &linked_regs);
+ if (dst_reg->type == SCALAR_VALUE && dst_reg->id)
+ collect_linked_regs(env, this_branch, dst_reg->id, &linked_regs);
+
is_jmp32 = BPF_CLASS(insn->code) == BPF_JMP32;
env->false_reg1 = *dst_reg;
env->false_reg2 = *src_reg;
@@ -16115,10 +16259,6 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
* 'this_branch' and 'other_branch' share this history
* if parent state is created.
*/
- if (BPF_SRC(insn->code) == BPF_X && src_reg->type == SCALAR_VALUE && src_reg->id)
- collect_linked_regs(env, this_branch, src_reg->id, &linked_regs);
- if (dst_reg->type == SCALAR_VALUE && dst_reg->id)
- collect_linked_regs(env, this_branch, dst_reg->id, &linked_regs);
if (linked_regs.cnt > 1) {
err = bpf_push_jmp_history(env, this_branch, 0, 0, 0, linked_regs_pack(&linked_regs));
if (err)
@@ -16168,7 +16308,6 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
*/
if (!is_jmp32 && BPF_SRC(insn->code) == BPF_X &&
__is_pointer_value(false, src_reg) && __is_pointer_value(false, dst_reg) &&
- type_may_be_null(src_reg->type) != type_may_be_null(dst_reg->type) &&
base_type(src_reg->type) != PTR_TO_BTF_ID &&
base_type(dst_reg->type) != PTR_TO_BTF_ID) {
eq_branch_regs = NULL;
@@ -16184,9 +16323,11 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
break;
}
if (eq_branch_regs) {
- if (type_may_be_null(src_reg->type))
+ /* src == dst && dst != NULL => src != NULL */
+ if (reg_not_null(env, dst_reg) && type_may_be_null(src_reg->type))
mark_ptr_not_null_reg(&eq_branch_regs[insn->src_reg]);
- else
+ /* src == dst && src != NULL => dst != NULL */
+ if (reg_not_null(env, src_reg) && type_may_be_null(dst_reg->type))
mark_ptr_not_null_reg(&eq_branch_regs[insn->dst_reg]);
}
}
@@ -16201,6 +16342,15 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
type_may_be_null(dst_reg->type) &&
((BPF_SRC(insn->code) == BPF_K && insn->imm == 0) ||
(BPF_SRC(insn->code) == BPF_X && bpf_register_is_null(src_reg)))) {
+ /*
+ * For BPF_X the zero is a property of this execution path,
+ * hence src_reg has to be precise.
+ */
+ if (BPF_SRC(insn->code) == BPF_X) {
+ err = mark_chain_precision(env, insn->src_reg);
+ if (err)
+ return err;
+ }
/* Mark all identical registers in each branch as either
* safe or unknown depending R == 0 or R != 0 conditional.
*/
@@ -16283,6 +16433,15 @@ static int check_ld_imm(struct bpf_verifier_env *env, struct bpf_insn *insn)
verbose(env, "callback function not static\n");
return -EINVAL;
}
+ /*
+ * When env->subprog_cnt == 1 this instruction won't be rewritten
+ * to hold a real function address. Assume that no usable program
+ * combines e.g. main and timer callback and just reject here.
+ */
+ if (subprogno == 0) {
+ verbose(env, "callback function cannot be the main program\n");
+ return -EINVAL;
+ }
dst_reg->type = PTR_TO_FUNC;
dst_reg->subprogno = subprogno;
@@ -16345,6 +16504,7 @@ static bool may_access_skb(enum bpf_prog_type type)
*/
static int check_ld_abs(struct bpf_verifier_env *env, struct bpf_insn *insn)
{
+ struct bpf_verifier_state *state = env->cur_state;
struct bpf_reg_state *regs = cur_regs(env);
static const int ctx_reg = BPF_REG_6;
u8 mode = BPF_MODE(insn->code);
@@ -16355,6 +16515,13 @@ static int check_ld_abs(struct bpf_verifier_env *env, struct bpf_insn *insn)
return -EINVAL;
}
+ for (i = state->curframe; i; i--) {
+ if (state->frame[i]->in_callback_fn) {
+ verbose(env, "cannot use BPF_LD_[ABS|IND] within callback\n");
+ return -EINVAL;
+ }
+ }
+
if (!env->ops->gen_ld_abs) {
verifier_bug(env, "gen_ld_abs is null");
return -EFAULT;
@@ -16823,6 +16990,10 @@ bool bpf_get_call_summary(struct bpf_verifier_env *env, struct bpf_insn *call,
* r0 = *(u64 *)(r10 - 8); r0 += r1;
* r0 += r1; exit;
* exit;
+ *
+ * Both uses of the marks assume that a pattern is entered at its first
+ * spill and thus executes as a unit, hence a pattern is not grown past
+ * an instruction targeted by a jump.
*/
static void mark_fastcall_pattern_for_call(struct bpf_verifier_env *env,
struct bpf_subprog_info *subprog,
@@ -16861,6 +17032,10 @@ static void mark_fastcall_pattern_for_call(struct bpf_verifier_env *env,
for (i = 1, off = lowest_off; i <= ARRAY_SIZE(caller_saved); ++i, off += BPF_REG_SIZE) {
if (insn_idx - i < 0 || insn_idx + i >= env->prog->len)
break;
+ /* stx/ldx/call must not be a jump targets, a jump to the first stx is fine */
+ if (bpf_is_jump_target(env, insn_idx - i + 1) ||
+ bpf_is_jump_target(env, insn_idx + i))
+ break;
stx = &insns[insn_idx - i];
ldx = &insns[insn_idx + i];
/* must be a stack spill/fill pair */
@@ -17404,7 +17579,9 @@ static int do_check(struct bpf_verifier_env *env)
insn = &insns[env->insn_idx];
insn_aux = &env->insn_aux_data[env->insn_idx];
- if (++env->insn_processed > BPF_COMPLEXITY_LIMIT_INSNS) {
+ account_processed_insn(env);
+
+ if (env->insn_processed > BPF_COMPLEXITY_LIMIT_INSNS) {
verbose(env,
"BPF program is too large. Processed %d insn\n",
env->insn_processed);
@@ -17544,6 +17721,7 @@ static int do_check(struct bpf_verifier_env *env)
"speculation barrier after jump instruction may not have the desired effect"))
return -EFAULT;
process_bpf_exit:
+ account_current_path(env);
mark_verifier_state_scratched(env);
err = bpf_update_branch_counts(env, env->cur_state);
if (err)
@@ -18538,6 +18716,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
ret = do_check(env);
out:
+ account_current_path(env);
if (!ret && pop_log)
bpf_vlog_reset(&env->log, 0);
free_states(env);
@@ -18569,7 +18748,6 @@ static int do_check_subprogs(struct bpf_verifier_env *env)
struct bpf_prog_aux *aux = env->prog->aux;
struct bpf_func_info_aux *sub_aux;
int i, ret, new_cnt;
- u32 insn_processed;
if (!aux->func_info)
return 0;
@@ -18584,8 +18762,6 @@ static int do_check_subprogs(struct bpf_verifier_env *env)
if (!bpf_subprog_is_global(env, i))
continue;
- insn_processed = env->insn_processed;
-
sub_aux = subprog_aux(env, i);
if (!sub_aux->called || sub_aux->verified)
continue;
@@ -18593,7 +18769,6 @@ static int do_check_subprogs(struct bpf_verifier_env *env)
env->insn_idx = env->subprog_info[i].start;
WARN_ON_ONCE(env->insn_idx == 0);
ret = do_check_common(env, i);
- env->subprog_info[i].insn_processed = env->insn_processed - insn_processed;
if (ret) {
return ret;
} else if (env->log.level & BPF_LOG_LEVEL) {
@@ -18620,12 +18795,10 @@ static int do_check_subprogs(struct bpf_verifier_env *env)
static int do_check_main(struct bpf_verifier_env *env)
{
- u32 insn_processed = env->insn_processed;
int ret;
env->insn_idx = 0;
ret = do_check_common(env, 0);
- env->subprog_info[0].insn_processed = env->insn_processed - insn_processed;
if (!ret)
env->prog->aux->stack_depth = env->subprog_info[0].stack_depth;
return ret;
@@ -18644,10 +18817,10 @@ static void print_verification_stats(struct bpf_verifier_env *env)
for (i = 1; i < subprog_cnt; i++)
verbose(env, "+%d", env->subprog_info[i].stack_depth);
verbose(env, " max %d\n", env->max_stack_depth);
- verbose(env, "insns processed %d", env->subprog_info[0].insn_processed);
+ verbose(env, "insns processed %d", env->subprog_info[0].insns_total);
for (i = 1; i < subprog_cnt; i++)
if (bpf_subprog_is_global(env, i))
- verbose(env, "+%d", env->subprog_info[i].insn_processed);
+ verbose(env, "+%d", env->subprog_info[i].insns_total);
verbose(env, "\n");
}
verbose(env, "processed %d insns (limit %d) max_states_per_insn %d "
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 1c6d56800668..69ff86e03c63 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1220,6 +1220,28 @@ static void reset_partition_data(struct cpuset *cs)
cpumask_copy(cs->effective_cpus, parent->effective_cpus);
}
+/* Return true if isolated_cpus changes. */
+static bool isolated_cpu_update(int new_prs, int cpu)
+{
+ lockdep_assert_held(&callback_lock);
+ lockdep_assert_held(&cpuset_mutex);
+
+ if (new_prs == PRS_ISOLATED) {
+ if (cpumask_test_cpu(cpu, isolated_cpus))
+ return false;
+ cpumask_set_cpu(cpu, isolated_cpus);
+ return true;
+ }
+
+ /* CPUs isolated at boot must remain isolated. */
+ if (!cpumask_test_cpu(cpu,
+ housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT)) ||
+ !cpumask_test_cpu(cpu, isolated_cpus))
+ return false;
+ cpumask_clear_cpu(cpu, isolated_cpus);
+ return true;
+}
+
/*
* isolated_cpus_update - Update the isolated_cpus mask
* @old_prs: old partition_root_state
@@ -1228,19 +1250,16 @@ static void reset_partition_data(struct cpuset *cs)
*/
static void isolated_cpus_update(int old_prs, int new_prs, struct cpumask *xcpus)
{
+ bool updated = false;
+ int cpu;
+
WARN_ON_ONCE(old_prs == new_prs);
lockdep_assert_held(&callback_lock);
lockdep_assert_held(&cpuset_mutex);
- if (new_prs == PRS_ISOLATED) {
- if (cpumask_subset(xcpus, isolated_cpus))
- return;
- cpumask_or(isolated_cpus, isolated_cpus, xcpus);
- } else {
- if (!cpumask_intersects(xcpus, isolated_cpus))
- return;
- cpumask_andnot(isolated_cpus, isolated_cpus, xcpus);
- }
- update_housekeeping = true;
+ for_each_cpu(cpu, xcpus)
+ updated |= isolated_cpu_update(new_prs, cpu);
+ if (updated)
+ update_housekeeping = true;
}
/*
diff --git a/kernel/events/core.c b/kernel/events/core.c
index ed3b62b0f948..bd8c1acf59e2 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -3925,13 +3925,13 @@ static void perf_pmu_sched_task(struct task_struct *prev,
bool sched_in)
{
struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
- struct perf_cpu_pmu_context *cpc;
+ struct perf_cpu_pmu_context *cpc, *cpc2;
/* cpuctx->task_ctx will be handled in perf_event_context_sched_in/out */
if (prev == next || cpuctx->task_ctx)
return;
- list_for_each_entry(cpc, this_cpu_ptr(&sched_cb_list), sched_cb_entry)
+ list_for_each_entry_safe(cpc, cpc2, this_cpu_ptr(&sched_cb_list), sched_cb_entry)
__perf_pmu_sched_task(cpc, sched_in ? next : prev, sched_in);
}
diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c
index 9fe92161715e..1b1ffe0533e5 100644
--- a/kernel/events/ring_buffer.c
+++ b/kernel/events/ring_buffer.c
@@ -509,7 +509,10 @@ void perf_aux_output_end(struct perf_output_handle *handle, unsigned long size)
/*
* Only send RECORD_AUX if we have something useful to communicate
*
- * Note: the OVERWRITE records by themselves are not considered
+ * PMU_FORMAT bits identify the PMU type rather than an AUX event
+ * has occurred, so ignore them for zero-sized records.
+ *
+ * The OVERWRITE records by themselves are not considered
* useful, as they don't communicate any *new* information,
* aside from the short-lived offset, that becomes history at
* the next event sched-in and therefore isn't useful.
@@ -518,7 +521,9 @@ void perf_aux_output_end(struct perf_output_handle *handle, unsigned long size)
* offset. So, from now on we don't output AUX records that
* have *only* OVERWRITE flag set.
*/
- if (size || (handle->aux_flags & ~(u64)PERF_AUX_FLAG_OVERWRITE))
+ if (size ||
+ (handle->aux_flags & ~(u64)(PERF_AUX_FLAG_PMU_FORMAT_TYPE_MASK |
+ PERF_AUX_FLAG_OVERWRITE)))
perf_event_aux_event(handle->event, aux_head, size,
handle->aux_flags);
diff --git a/kernel/exit.c b/kernel/exit.c
index 2c0b1c02920f..e4cd7232015d 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -262,8 +262,11 @@ void release_task(struct task_struct *p)
pidfs_exit(p);
cgroup_task_release(p);
- /* Retrieve @thread_pid before __unhash_process() may set it to NULL. */
- thread_pid = task_pid(p);
+ /*
+ * Pin @thread_pid before __unhash_process() clears it. The last
+ * PIDTYPE detach can otherwise free it before proc_flush_pid().
+ */
+ thread_pid = get_pid(task_pid(p));
write_lock_irq(&tasklist_lock);
ptrace_release_task(p);
@@ -292,8 +295,8 @@ void release_task(struct task_struct *p)
}
write_unlock_irq(&tasklist_lock);
- /* @thread_pid can't go away until free_pids() below */
proc_flush_pid(thread_pid);
+ put_pid(thread_pid);
exit_cred_namespaces(p);
add_device_randomness(&p->se.sum_exec_runtime,
sizeof(p->se.sum_exec_runtime));
diff --git a/kernel/fork.c b/kernel/fork.c
index abc2f01ac357..3340dbaf2bb4 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -2114,6 +2114,11 @@ __latent_entropy struct task_struct *copy_process(
p = dup_task_struct(current, node);
if (!p)
goto fork_out;
+ /*
+ * Must run before the first fallible op, so error paths never
+ * free the parent's ret_stack.
+ */
+ ftrace_graph_init_task(p);
retval = copy_exec_state(clone_flags, p);
if (retval)
goto bad_fork_free;
@@ -2140,8 +2145,6 @@ __latent_entropy struct task_struct *copy_process(
*/
p->clear_child_tid = (clone_flags & CLONE_CHILD_CLEARTID) ? args->child_tid : NULL;
- ftrace_graph_init_task(p);
-
rt_mutex_init_task(p);
raw_spin_lock_init(&p->blocked_lock);
diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c
index e0de81114824..06f025abe887 100644
--- a/kernel/locking/lockdep.c
+++ b/kernel/locking/lockdep.c
@@ -947,6 +947,34 @@ look_up_lock_class(const struct lockdep_map *lock, unsigned int subclass)
return NULL;
}
+static __always_inline bool lock_class_cache_is_valid(const struct lockdep_map *lock,
+ const struct lock_class *class,
+ unsigned int subclass)
+{
+ unsigned int class_subclass;
+
+ if (!class)
+ return false;
+
+ if (unlikely(class < lock_classes || class >= lock_classes + MAX_LOCKDEP_KEYS))
+ return false;
+
+ if (unlikely(!arch_test_bit(class - lock_classes, lock_classes_in_use)))
+ return false;
+
+ if (unlikely(!lock->key))
+ return false;
+
+ class_subclass = subclass ? subclass : class->subclass;
+ if (unlikely(class_subclass >= MAX_LOCKDEP_SUBCLASSES))
+ return false;
+
+ if (unlikely(READ_ONCE(class->key) != lock->key->subkeys + class_subclass))
+ return false;
+
+ return true;
+}
+
/*
* Static locks do not have their class-keys yet - for them the key is
* the lock object itself. If the lock is in the per cpu area, the
@@ -1379,9 +1407,9 @@ register_lock_class(struct lockdep_map *lock, unsigned int subclass, int force)
out_set_class_cache:
if (!subclass || force)
- lock->class_cache[0] = class;
+ WRITE_ONCE(lock->class_cache[0], class);
else if (subclass < NR_LOCKDEP_CACHING_CLASSES)
- lock->class_cache[subclass] = class;
+ WRITE_ONCE(lock->class_cache[subclass], class);
/*
* Hash collision, did we smoke some? We found a class with a matching
@@ -4941,7 +4969,7 @@ void lockdep_init_map_type(struct lockdep_map *lock, const char *name,
int i;
for (i = 0; i < NR_LOCKDEP_CACHING_CLASSES; i++)
- lock->class_cache[i] = NULL;
+ WRITE_ONCE(lock->class_cache[i], NULL);
#ifdef CONFIG_LOCK_STAT
lock->cpu = raw_smp_processor_id();
@@ -5006,12 +5034,15 @@ EXPORT_SYMBOL_GPL(__lockdep_no_track__);
void lockdep_set_lock_cmp_fn(struct lockdep_map *lock, lock_cmp_fn cmp_fn,
lock_print_fn print_fn)
{
- struct lock_class *class = lock->class_cache[0];
+ struct lock_class *class = READ_ONCE(lock->class_cache[0]);
unsigned long flags;
raw_local_irq_save(flags);
lockdep_recursion_inc();
+ if (!lock_class_cache_is_valid(lock, class, 0))
+ class = NULL;
+
if (!class)
class = register_lock_class(lock, 0, 0);
@@ -5103,8 +5134,11 @@ static int __lock_acquire(struct lockdep_map *lock, unsigned int subclass,
if (DEBUG_LOCKS_WARN_ON(subclass >= MAX_LOCKDEP_SUBCLASSES))
return 0;
- if (subclass < NR_LOCKDEP_CACHING_CLASSES)
- class = lock->class_cache[subclass];
+ if (subclass < NR_LOCKDEP_CACHING_CLASSES) {
+ class = READ_ONCE(lock->class_cache[subclass]);
+ if (!lock_class_cache_is_valid(lock, class, subclass))
+ class = NULL;
+ }
/*
* Not cached?
*/
@@ -5307,9 +5341,9 @@ static noinstr int match_held_lock(const struct held_lock *hlock,
return 1;
if (hlock->references) {
- const struct lock_class *class = lock->class_cache[0];
+ const struct lock_class *class = READ_ONCE(lock->class_cache[0]);
- if (!class)
+ if (!lock_class_cache_is_valid(lock, class, 0))
class = look_up_lock_class(lock, 0);
/*
diff --git a/kernel/nstree.c b/kernel/nstree.c
index 6d12e5900ac0..831f279d174a 100644
--- a/kernel/nstree.c
+++ b/kernel/nstree.c
@@ -533,19 +533,13 @@ DEFINE_FREE(ns_put, struct ns_common *, if (!IS_ERR_OR_NULL(_T)) ns_put(_T))
static inline struct ns_common *__must_check legitimize_ns(const struct klistns *kls,
struct ns_common *candidate)
{
- struct ns_common *ns __free(ns_put) = NULL;
-
if (!ns_requested(kls, candidate))
return NULL;
- ns = ns_get_unless_inactive(candidate);
- if (!ns)
- return NULL;
-
- if (!may_list_ns(kls, ns))
+ if (!may_list_ns(kls, candidate))
return NULL;
- return no_free_ptr(ns);
+ return ns_get_unless_inactive(candidate);
}
static ssize_t do_listns_userns(struct klistns *kls)
diff --git a/kernel/printk/nbcon.c b/kernel/printk/nbcon.c
index 4b03b019cd5e..ad79e30afd64 100644
--- a/kernel/printk/nbcon.c
+++ b/kernel/printk/nbcon.c
@@ -1382,7 +1382,7 @@ bool nbcon_kthread_create(struct console *con)
return true;
kt = kthread_run(nbcon_kthread_func, con, "pr/%s%d", con->name, con->index);
- if (WARN_ON(IS_ERR(kt))) {
+ if (IS_ERR(kt)) {
con_printk(KERN_ERR, con, "failed to start printing thread\n");
return false;
}
@@ -1782,7 +1782,7 @@ bool nbcon_alloc(struct console *con)
}
rcuwait_init(&con->rcuwait);
- init_irq_work(&con->irq_work, nbcon_irq_work);
+ con->irq_work = IRQ_WORK_INIT_LAZY(nbcon_irq_work);
atomic_long_set(&ACCESS_PRIVATE(con, nbcon_prev_seq), -1UL);
nbcon_state_set(con, &state);
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index 6d363e42e2a0..96ff9547e888 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -3732,7 +3732,7 @@ static bool legacy_kthread_create(void)
lockdep_assert_console_list_lock_held();
kt = kthread_run(legacy_kthread_func, NULL, "pr/legacy");
- if (WARN_ON(IS_ERR(kt))) {
+ if (IS_ERR(kt)) {
pr_err("failed to start legacy printing thread\n");
return false;
}
diff --git a/kernel/reboot.c b/kernel/reboot.c
index f070c5c1103a..d177d89fcc33 100644
--- a/kernel/reboot.c
+++ b/kernel/reboot.c
@@ -13,7 +13,9 @@
#include <linux/kexec.h>
#include <linux/kmod.h>
#include <linux/kmsg_dump.h>
+#include <linux/rcupdate.h>
#include <linux/reboot.h>
+#include <linux/sched/signal.h>
#include <linux/suspend.h>
#include <linux/syscalls.h>
#include <linux/syscore_ops.h>
@@ -24,8 +26,7 @@
*/
static int C_A_D = 1;
-struct pid *cad_pid;
-EXPORT_SYMBOL(cad_pid);
+struct pid __rcu *cad_pid;
#if defined(CONFIG_ARM)
#define DEFAULT_REBOOT_MODE = REBOOT_HARD
@@ -1371,10 +1372,14 @@ static int proc_do_cad_pid(const struct ctl_table *table, int write, void *buffe
{
struct ctl_table tmp_table = *table;
struct pid *new_pid;
+ struct pid *old_pid;
pid_t tmp_pid;
int r;
- tmp_pid = pid_vnr(cad_pid);
+ rcu_read_lock();
+ tmp_pid = pid_vnr(rcu_dereference(cad_pid));
+ rcu_read_unlock();
+
tmp_table.data = &tmp_pid;
r = proc_dointvec(&tmp_table, write, buffer, lenp, ppos);
@@ -1385,7 +1390,13 @@ static int proc_do_cad_pid(const struct ctl_table *table, int write, void *buffe
if (!new_pid)
return -ESRCH;
- put_pid(xchg(&cad_pid, new_pid));
+ old_pid = unrcu_pointer(xchg(&cad_pid, RCU_INITIALIZER(new_pid)));
+ /*
+ * Wait for cad_pid readers before put_pid(). We cannot use
+ * call_rcu() here because free_pid() already owns pid->rcu.
+ */
+ synchronize_rcu();
+ put_pid(old_pid);
return 0;
}
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 87caa1290fad..35d647c8c368 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3743,11 +3743,17 @@ static inline void ttwu_do_wakeup(struct task_struct *p)
void update_rq_avg_idle(struct rq *rq)
{
- u64 delta = rq_clock(rq) - rq->idle_stamp;
- u64 max = 2*rq->max_idle_balance_cost;
+ u64 idle_stamp = rq->idle_stamp;
+ u64 delta, max;
+
+ if (!idle_stamp)
+ return;
+
+ delta = rq_clock(rq) - idle_stamp;
update_avg(&rq->avg_idle, delta);
+ max = 2 * rq->max_idle_balance_cost;
if (rq->avg_idle > max)
rq->avg_idle = max;
rq->idle_stamp = 0;
@@ -5774,8 +5780,8 @@ void sched_tick(void)
{
int cpu = smp_processor_id();
struct rq *rq = cpu_rq(cpu);
- /* accounting goes to the donor task */
- struct task_struct *donor;
+ /* scheduler accounting goes to the donor task */
+ struct task_struct *curr, *donor;
struct rq_flags rf;
unsigned long hw_pressure;
u64 resched_latency;
@@ -5786,6 +5792,7 @@ void sched_tick(void)
sched_clock_tick();
rq_lock(rq, &rf);
+ curr = rq->curr;
donor = rq->donor;
psi_account_irqtime(rq, donor, NULL);
@@ -5811,8 +5818,8 @@ void sched_tick(void)
perf_event_task_tick();
- if (donor->flags & PF_WQ_WORKER)
- wq_worker_tick(donor);
+ if (curr->flags & PF_WQ_WORKER)
+ wq_worker_tick(curr);
if (!scx_switched_all()) {
rq->idle_balance = idle_cpu(cpu);
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 857dbe3519a8..0663c00c41c0 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -3028,8 +3028,8 @@ static struct task_struct *pick_next_pushable_dl_task(struct rq *rq)
next_node = rb_first_cached(&rq->dl.pushable_dl_tasks_root);
while (next_node) {
i = __node_2_pdl(next_node);
- /* make sure task isn't on_cpu (possible with proxy-exec) */
- if (!task_on_cpu(rq, i)) {
+ /* skip tasks that cannot be migrated */
+ if (!task_on_cpu(rq, i) && !is_migration_disabled(i)) {
p = i;
break;
}
diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c
index 558be0cdebef..548f46a42788 100644
--- a/kernel/sched/ext/ext.c
+++ b/kernel/sched/ext/ext.c
@@ -7435,7 +7435,7 @@ static void scx_root_enable_workfn(struct kthread_work *work)
/*
* Enable ops for every task. Fork is excluded by scx_fork_rwsem
* preventing new tasks from being added. No need to exclude tasks
- * leaving as sched_ext_free() can handle both prepped and enabled
+ * leaving as sched_ext_dead() can handle both prepped and enabled
* tasks. Prep all tasks first and then enable them with preemption
* disabled.
*
@@ -7524,7 +7524,7 @@ static void scx_root_enable_workfn(struct kthread_work *work)
/*
* We're fully committed and can't fail. The task READY -> ENABLED
- * transitions here are synchronized against sched_ext_free() through
+ * transitions here are synchronized against sched_ext_dead() through
* scx_tasks_lock.
*/
percpu_down_write(&scx_fork_rwsem);
@@ -9019,12 +9019,6 @@ static bool scx_dsq_move(struct bpf_iter_scx_dsq_kern *kit,
if (unlikely(READ_ONCE(sch->aborting)))
return false;
- if (unlikely(!scx_task_on_sched(sch, p))) {
- scx_error(sch, "scx_bpf_dsq_move[_vtime]() on %s[%d] but the task belongs to a different scheduler",
- p->comm, p->pid);
- return false;
- }
-
/*
* Can be called from either ops.dispatch() holding the dispatched rq's
* lock or any context where no rq lock is held. If latter, lock @p's
@@ -9056,6 +9050,17 @@ static bool scx_dsq_move(struct bpf_iter_scx_dsq_kern *kit,
goto out;
}
+ /*
+ * @p has been on $src_dsq and can't move anymore. If @p is not on @sch,
+ * the caller didn't have authority over @p at the time of the call.
+ */
+ if (unlikely(!scx_task_on_sched(sch, p))) {
+ scx_error(sch, "scx_bpf_dsq_move[_vtime]() on %s[%d] but the task belongs to a different scheduler",
+ p->comm, p->pid);
+ raw_spin_unlock(&src_dsq->lock);
+ goto out;
+ }
+
/* @p is still on $src_dsq and stable, determine the destination */
dst_dsq = find_dsq_for_dispatch(sch, locked_rq ?: this_rq(), dsq_id, task_cpu(p));
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 4c7bb4b7018e..bb8a5f358ee1 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -1004,6 +1004,13 @@ static inline void __max_slice_update(struct sched_entity *se, struct rb_node *n
}
}
+static inline void min_vruntime_copy(struct sched_entity *new, struct sched_entity *old)
+{
+ new->min_vruntime = old->min_vruntime;
+ new->min_slice = old->min_slice;
+ new->max_slice = old->max_slice;
+}
+
/*
* se->min_vruntime = min(se->vruntime, {left,right}->min_vruntime)
*/
@@ -1031,8 +1038,9 @@ static inline bool min_vruntime_update(struct sched_entity *se, bool exit)
se->max_slice == old_max_slice;
}
-RB_DECLARE_CALLBACKS(static, min_vruntime_cb, struct sched_entity,
- run_node, min_vruntime, min_vruntime_update);
+
+RB_DECLARE_CALLBACKS_MULTI(static, min_vruntime_cb, struct sched_entity,
+ run_node, min_vruntime_copy, min_vruntime_update);
/*
* Enqueue an entity into the rb-tree:
@@ -1042,6 +1050,8 @@ static void __enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se)
sum_w_vruntime_add(cfs_rq, se);
se->min_vruntime = se->vruntime;
se->min_slice = se->slice;
+ se->max_slice = se->slice;
+
rb_add_augmented_cached(&se->run_node, &cfs_rq->tasks_timeline,
__entity_less, &min_vruntime_cb);
}
@@ -1363,7 +1373,6 @@ static s64 update_se(struct rq *rq, struct sched_entity *se)
se->exec_start = now;
if (entity_is_task(se)) {
- struct task_struct *donor = task_of(se);
struct task_struct *running = rq->curr;
/*
* If se is a task, we account the time against the running
@@ -1376,8 +1385,7 @@ static s64 update_se(struct rq *rq, struct sched_entity *se)
account_group_exec_runtime(running, delta_exec);
account_mm_sched(rq, running, delta_exec);
- /* cgroup time is always accounted against the donor */
- cgroup_account_cputime(donor, delta_exec);
+ cgroup_account_cputime(running, delta_exec);
} else {
/* If not task, account the time against donor se */
se->sum_exec_runtime += delta_exec;
diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index e6e5f8a2caaf..85303add726d 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -1872,8 +1872,8 @@ static struct task_struct *pick_next_pushable_task(struct rq *rq)
return NULL;
plist_for_each_entry(i, head, pushable_tasks) {
- /* make sure task isn't on_cpu (possible with proxy-exec) */
- if (!task_on_cpu(rq, i)) {
+ /* skip tasks that cannot be migrated */
+ if (!task_on_cpu(rq, i) && !is_migration_disabled(i)) {
p = i;
break;
}
diff --git a/kernel/signal.c b/kernel/signal.c
index c53f202da9da..5f34ee7b504e 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -1903,6 +1903,18 @@ int kill_pid(struct pid *pid, int sig, int priv)
}
EXPORT_SYMBOL(kill_pid);
+int kill_cad_pid(int sig, int priv)
+{
+ int ret;
+
+ rcu_read_lock();
+ ret = kill_pid(rcu_dereference(cad_pid), sig, priv);
+ rcu_read_unlock();
+
+ return ret;
+}
+EXPORT_SYMBOL(kill_cad_pid);
+
#ifdef CONFIG_POSIX_TIMERS
/*
* These functions handle POSIX timer signals. POSIX timers use
diff --git a/kernel/time/clockevents.c b/kernel/time/clockevents.c
index 0014d163f989..62ad7c000386 100644
--- a/kernel/time/clockevents.c
+++ b/kernel/time/clockevents.c
@@ -615,33 +615,40 @@ void clockevents_handle_noop(struct clock_event_device *dev)
{
}
-/**
- * clockevents_exchange_device - release and request clock devices
- * @old: device to release (can be NULL)
- * @new: device to request (can be NULL)
- *
- * Called from various tick functions with clockevents_lock held and
- * interrupts disabled.
- */
-void clockevents_exchange_device(struct clock_event_device *old,
- struct clock_event_device *new)
+void __clockevents_exchange_device(struct clock_event_device *old,
+ struct clock_event_device *new)
{
/*
* Caller releases a clock event device. We queue it into the
* released list and do a notify add later.
*/
if (old) {
- module_put(old->owner);
clockevents_switch_state(old, CLOCK_EVT_STATE_DETACHED);
list_move(&old->list, &clockevents_released);
}
if (new) {
- BUG_ON(!clockevent_state_detached(new));
+ WARN_ON(!clockevent_state_detached(new));
clockevents_shutdown(new);
}
}
+/**
+ * clockevents_exchange_device - release and request clock devices
+ * @old: device to release (can be NULL)
+ * @new: device to request (can be NULL)
+ *
+ * Called from various tick functions with clockevents_lock held and
+ * interrupts disabled.
+ */
+void clockevents_exchange_device(struct clock_event_device *old,
+ struct clock_event_device *new)
+{
+ __clockevents_exchange_device(old, new);
+ if (old)
+ module_put(old->owner);
+}
+
/**
* clockevents_suspend - suspend clock devices
*/
@@ -699,7 +706,7 @@ void tick_offline_cpu(unsigned int cpu)
if (cpumask_test_cpu(cpu, dev->cpumask) &&
cpumask_weight(dev->cpumask) == 1 &&
!tick_is_broadcast_device(dev)) {
- BUG_ON(!clockevent_state_detached(dev));
+ WARN_ON(!clockevent_state_detached(dev));
list_del(&dev->list);
}
}
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 96eef3026fba..71015e59f5de 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -1236,13 +1236,23 @@ remove_and_enqueue_same_base(struct hrtimer *timer, struct hrtimer_clock_base *b
{
bool was_first = false;
+ /*
+ * Updating the sort key while @timer is queued can temporarily
+ * make the tree inconsistent. This is safe under cpu_base->lock:
+ * no other queue operation can observe that state.
+ * hrtimer_can_update_in_place() either confirms that the new expiry
+ * fits between the neighbours or timerqueue_linked_del() removes the
+ * timer without consulting the expiry.
+ */
+ hrtimer_set_expires_range_ns(timer, expires, delta_ns);
+ expires = hrtimer_get_expires(timer);
+
/* Remove it from the timer queue if active */
if (timer->is_queued) {
was_first = !timerqueue_linked_prev(&timer->node);
/* Try to update in place to avoid the de/enqueue dance */
if (hrtimer_can_update_in_place(timer, base, expires)) {
- hrtimer_set_expires_range_ns(timer, expires, delta_ns);
trace_hrtimer_start(timer, mode, true);
if (was_first)
base->expires_next = expires;
@@ -1253,9 +1263,6 @@ remove_and_enqueue_same_base(struct hrtimer *timer, struct hrtimer_clock_base *b
timerqueue_linked_del(&base->active, &timer->node);
}
- /* Set the new expiry time */
- hrtimer_set_expires_range_ns(timer, expires, delta_ns);
-
debug_activate(timer, mode, timer->is_queued);
base->cpu_base->active_bases |= 1 << base->index;
diff --git a/kernel/time/tick-broadcast.c b/kernel/time/tick-broadcast.c
index 115e0bf01276..bda3d2391a60 100644
--- a/kernel/time/tick-broadcast.c
+++ b/kernel/time/tick-broadcast.c
@@ -165,23 +165,31 @@ static bool tick_set_oneshot_wakeup_device(struct clock_event_device *newdev,
*/
void tick_install_broadcast_device(struct clock_event_device *dev, int cpu)
{
- struct clock_event_device *cur = tick_broadcast_device.evtdev;
+ struct clock_event_device *cur;
- if (tick_set_oneshot_wakeup_device(dev, cpu))
- return;
+ scoped_guard(raw_spinlock_irqsave, &tick_broadcast_lock) {
- if (!tick_check_broadcast_device(cur, dev))
- return;
+ if (tick_set_oneshot_wakeup_device(dev, cpu))
+ return;
- if (!try_module_get(dev->owner))
- return;
+ cur = tick_broadcast_device.evtdev;
+ if (!tick_check_broadcast_device(cur, dev))
+ return;
- clockevents_exchange_device(cur, dev);
+ if (!try_module_get(dev->owner))
+ return;
+
+ __clockevents_exchange_device(cur, dev);
+ if (cur)
+ cur->event_handler = clockevents_handle_noop;
+ WRITE_ONCE(tick_broadcast_device.evtdev, dev);
+ if (!cpumask_empty(tick_broadcast_mask))
+ tick_broadcast_start_periodic(dev);
+ }
+
+ /* Module release must be outside of the lock */
if (cur)
- cur->event_handler = clockevents_handle_noop;
- tick_broadcast_device.evtdev = dev;
- if (!cpumask_empty(tick_broadcast_mask))
- tick_broadcast_start_periodic(dev);
+ module_put(cur->owner);
if (!(dev->features & CLOCK_EVT_FEAT_ONESHOT))
return;
@@ -1218,7 +1226,7 @@ int tick_broadcast_oneshot_active(void)
*/
bool tick_broadcast_oneshot_available(void)
{
- struct clock_event_device *bc = tick_broadcast_device.evtdev;
+ struct clock_event_device *bc = READ_ONCE(tick_broadcast_device.evtdev);
return bc ? bc->features & CLOCK_EVT_FEAT_ONESHOT : false;
}
@@ -1226,7 +1234,7 @@ bool tick_broadcast_oneshot_available(void)
#else
int __tick_broadcast_oneshot_control(enum tick_broadcast_state state)
{
- struct clock_event_device *bc = tick_broadcast_device.evtdev;
+ struct clock_event_device *bc = READ_ONCE(tick_broadcast_device.evtdev);
if (!bc || (bc->features & CLOCK_EVT_FEAT_HRTIMER))
return -EBUSY;
diff --git a/kernel/time/tick-internal.h b/kernel/time/tick-internal.h
index 597d816d22e8..ce1c9d1f6b96 100644
--- a/kernel/time/tick-internal.h
+++ b/kernel/time/tick-internal.h
@@ -54,6 +54,8 @@ static inline void clockevent_set_state(struct clock_event_device *dev,
}
extern void clockevents_shutdown(struct clock_event_device *dev);
+extern void __clockevents_exchange_device(struct clock_event_device *old,
+ struct clock_event_device *new);
extern void clockevents_exchange_device(struct clock_event_device *old,
struct clock_event_device *new);
extern void clockevents_switch_state(struct clock_event_device *dev,
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index a4ecba1518de..db8eeaefeb00 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -6675,7 +6675,7 @@ int update_ftrace_direct_mod(struct ftrace_ops *ops, struct ftrace_hash *hash, b
/* Enable the tmp_ops to have the same functions as the hash object. */
ftrace_ops_init(&tmp_ops);
- tmp_ops.func_hash->filter_hash = hash;
+ rcu_assign_pointer(tmp_ops.func_hash->filter_hash, hash);
err = register_ftrace_function_nolock(&tmp_ops);
if (err)
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 1925ee9cc36d..7c59e8e58619 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -330,6 +330,11 @@ struct buffer_data_read_page {
struct buffer_data_page *data; /* actual data, stored in this page */
};
+static __always_inline unsigned int rb_read_page_capacity(struct buffer_data_read_page *rpage)
+{
+ return (PAGE_SIZE << rpage->order) - BUF_PAGE_HDR_SIZE;
+}
+
/*
* Note, the buffer_page list must be first. The buffer pages
* are allocated in cache lines, which means that each buffer
@@ -514,7 +519,7 @@ struct ring_buffer_per_cpu {
int cpu;
atomic_t record_disabled;
atomic_t resize_disabled;
- struct trace_buffer *buffer;
+ struct trace_buffer *buffer;
raw_spinlock_t reader_lock; /* serialize readers */
arch_spinlock_t lock;
struct lock_class_key lock_key;
@@ -552,7 +557,6 @@ struct ring_buffer_per_cpu {
/* pages removed since last reset */
unsigned long pages_removed;
- unsigned int mapped;
unsigned int user_mapped; /* user space mapping */
struct mutex mapping_lock;
struct buffer_page **subbuf_ids; /* ID to subbuf VA */
@@ -632,6 +636,20 @@ unsigned long rb_subbuf_start(struct trace_buffer *buffer, unsigned long addr)
return addr & ~((unsigned long)(rb_subbuf_size(buffer) - 1));
}
+static bool rb_is_static(struct ring_buffer_per_cpu *cpu_buffer)
+{
+ return cpu_buffer->user_mapped || cpu_buffer->remote || cpu_buffer->ring_meta;
+}
+
+static unsigned long rb_static_max_pages(void)
+{
+ /*
+ * Static ring buffers are using bpage::id and must account for the
+ * reader page.
+ */
+ return (1UL << 30) - 1;
+}
+
struct ring_buffer_iter {
struct ring_buffer_per_cpu *cpu_buffer;
unsigned long head;
@@ -867,14 +885,13 @@ static void rb_wake_up_waiters(struct irq_work *work)
struct ring_buffer_per_cpu *cpu_buffer =
container_of(rbwork, struct ring_buffer_per_cpu, irq_work);
- /* Called from interrupt context */
- raw_spin_lock(&cpu_buffer->reader_lock);
- rbwork->wakeup_full = false;
- rbwork->full_waiters_pending = false;
+ scoped_guard(raw_spinlock_irqsave, &cpu_buffer->reader_lock) {
+ rbwork->wakeup_full = false;
+ rbwork->full_waiters_pending = false;
- /* Waking up all waiters, they will reset the shortest full */
- cpu_buffer->shortest_full = 0;
- raw_spin_unlock(&cpu_buffer->reader_lock);
+ /* Waking up all waiters, they will reset the shortest full */
+ cpu_buffer->shortest_full = 0;
+ }
wake_up_all(&rbwork->full_waiters);
}
@@ -1836,6 +1853,11 @@ static bool rb_cpu_meta_valid(struct ring_buffer_cpu_meta *meta, int cpu,
return false;
}
+ if (meta->nr_subbufs != nr_pages + 1) {
+ pr_info("Ring buffer boot meta [%d] invalid nr_subbufs\n", cpu);
+ return false;
+ }
+
buffers_start = meta->first_buffer;
buffers_end = meta->first_buffer + (subbuf_size * meta->nr_subbufs);
@@ -2562,7 +2584,6 @@ rb_allocate_cpu_buffer(struct trace_buffer *buffer, long nr_pages, int cpu)
* Range mapped buffers have the same restrictions as memory
* mapped ones do.
*/
- cpu_buffer->mapped = 1;
cpu_buffer->ring_meta = rb_range_meta(buffer, nr_pages, cpu);
bpage->page = rb_range_buffer(cpu_buffer, 0);
if (!bpage->page)
@@ -2812,6 +2833,8 @@ static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
size = end - buffers_start;
size = size / nr_cpu_ids;
+ if (size < sizeof(struct ring_buffer_cpu_meta))
+ goto fail_free_buffers;
/*
* The number of sub-buffers (nr_pages) is determined by the
* total size allocated minus the meta data size.
@@ -2821,6 +2844,10 @@ static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
*/
nr_pages = (size - sizeof(struct ring_buffer_cpu_meta)) /
(subbuf_size + sizeof(int));
+
+ if (nr_pages > rb_static_max_pages())
+ goto fail_free_buffers;
+
/* Need at least two pages plus the reader page */
if (nr_pages < 3)
goto fail_free_buffers;
@@ -2853,6 +2880,10 @@ static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
/* The writer is remote. This ring-buffer is read-only */
atomic_inc(&buffer->record_disabled);
nr_pages = desc->nr_page_va - 1;
+
+ if (nr_pages > rb_static_max_pages())
+ goto fail_free_buffers;
+
if (nr_pages < 2)
goto fail_free_buffers;
} else {
@@ -6656,12 +6687,11 @@ rb_reset_cpu(struct ring_buffer_per_cpu *cpu_buffer)
rb_head_page_activate(cpu_buffer);
cpu_buffer->pages_removed = 0;
- if (cpu_buffer->mapped) {
- rb_update_meta_page(cpu_buffer);
- if (cpu_buffer->ring_meta) {
- struct ring_buffer_cpu_meta *meta = cpu_buffer->ring_meta;
- meta->commit_buffer = meta->head_buffer;
- }
+ rb_update_meta_page(cpu_buffer);
+ if (cpu_buffer->ring_meta) {
+ struct ring_buffer_cpu_meta *meta = cpu_buffer->ring_meta;
+
+ meta->commit_buffer = meta->head_buffer;
}
}
@@ -6910,8 +6940,8 @@ int ring_buffer_swap_cpu(struct trace_buffer *buffer_a,
cpu_buffer_a = buffer_a->buffers[cpu];
cpu_buffer_b = buffer_b->buffers[cpu];
- /* It's up to the callers to not try to swap mapped buffers */
- if (WARN_ON_ONCE(cpu_buffer_a->mapped || cpu_buffer_b->mapped))
+ /* It's up to the callers to not try to swap static buffers */
+ if (WARN_ON_ONCE(rb_is_static(cpu_buffer_a) || rb_is_static(cpu_buffer_b)))
return -EBUSY;
/* At least make sure the two buffers are somewhat the same */
@@ -6977,56 +7007,78 @@ EXPORT_SYMBOL_GPL(ring_buffer_swap_cpu);
* ring_buffer_alloc_read_page - allocate a page to read from buffer
* @buffer: the buffer to allocate for.
* @cpu: the cpu buffer to allocate.
+ * @rpage: pointer to pass in an already allocated page (can be NULL)
+ * and returns the allocated page.
*
- * This function is used in conjunction with ring_buffer_read_page.
+ * This function is used in conjunction with ring_buffer_read_page().
* When reading a full page from the ring buffer, these functions
* can be used to speed up the process. The calling function should
* allocate a few pages first with this function. Then when it
* needs to get pages from the ring buffer, it passes the result
- * of this function into ring_buffer_read_page, which will swap
+ * of this function into ring_buffer_read_page(), which will swap
* the page that was allocated, with the read page of the buffer.
*
+ * If @rpage is provided, and it has a different order than the current
+ * subbuffer order, its payload will be freed and re-allocated. If it
+ * already matches the order, it is simply returned.
+ *
* Returns:
- * The page allocated, or ERR_PTR
+ * 0 on success, < 0 on error
*/
-struct buffer_data_read_page *
-ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu)
+int ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu,
+ struct buffer_data_read_page **rpage)
{
struct ring_buffer_per_cpu *cpu_buffer;
- struct buffer_data_read_page *bpage = NULL;
unsigned long flags;
+ unsigned int order;
if (!cpumask_test_cpu(cpu, buffer->cpumask))
- return ERR_PTR(-ENODEV);
+ return -ENODEV;
- bpage = kzalloc_obj(*bpage);
- if (!bpage)
- return ERR_PTR(-ENOMEM);
+ if (!rpage)
+ return -EINVAL;
+
+ order = READ_ONCE(buffer->subbuf_order);
- bpage->order = buffer->subbuf_order;
+ if (*rpage) {
+ if ((*rpage)->order == order)
+ return 0;
+
+ /* We can reuse rpage, but we discard the payload */
+ free_pages((unsigned long)(*rpage)->data, (*rpage)->order);
+ (*rpage)->data = NULL;
+ } else {
+ *rpage = kzalloc_obj(**rpage);
+ if (!*rpage)
+ return -ENOMEM;
+ }
+
+ (*rpage)->order = order;
cpu_buffer = buffer->buffers[cpu];
+
local_irq_save(flags);
arch_spin_lock(&cpu_buffer->lock);
if (cpu_buffer->free_page.data) {
- *bpage = cpu_buffer->free_page;
+ **rpage = cpu_buffer->free_page;
cpu_buffer->free_page.data = NULL;
}
arch_spin_unlock(&cpu_buffer->lock);
local_irq_restore(flags);
- if (bpage->data) {
- rb_init_data_page(bpage->data);
+ if ((*rpage)->data) {
+ rb_init_data_page((*rpage)->data);
} else {
- bpage->data = alloc_cpu_data(cpu, bpage->order);
- if (!bpage->data) {
- kfree(bpage);
- return ERR_PTR(-ENOMEM);
+ (*rpage)->data = alloc_cpu_data(cpu, (*rpage)->order);
+ if (!(*rpage)->data) {
+ kfree(*rpage);
+ *rpage = NULL;
+ return -ENOMEM;
}
}
- return bpage;
+ return 0;
}
EXPORT_SYMBOL_GPL(ring_buffer_alloc_read_page);
@@ -7034,21 +7086,30 @@ EXPORT_SYMBOL_GPL(ring_buffer_alloc_read_page);
* ring_buffer_free_read_page - free an allocated read page
* @buffer: the buffer the page was allocate for
* @cpu: the cpu buffer the page came from
- * @data_page: the page to free
+ * @rpage: the buffer_data_read_page to free
*
* Free a page allocated from ring_buffer_alloc_read_page.
*/
void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu,
- struct buffer_data_read_page *data_page)
+ struct buffer_data_read_page *rpage)
{
struct ring_buffer_per_cpu *cpu_buffer;
- struct buffer_data_page *dpage = data_page->data;
- struct page *page = virt_to_page(dpage);
+ struct buffer_data_page *dpage;
unsigned long flags;
+ struct page *page;
if (!buffer || !buffer->buffers || !buffer->buffers[cpu])
return;
+ if (!rpage)
+ return;
+
+ dpage = rpage->data;
+ if (!dpage)
+ goto out;
+
+ page = virt_to_page(dpage);
+
cpu_buffer = buffer->buffers[cpu];
/*
@@ -7056,14 +7117,14 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu,
* is different from the subbuffer order of the buffer -
* we can't reuse it
*/
- if (page_ref_count(page) > 1 || data_page->order != buffer->subbuf_order)
+ if (page_ref_count(page) > 1 || rpage->order != READ_ONCE(buffer->subbuf_order))
goto out;
local_irq_save(flags);
arch_spin_lock(&cpu_buffer->lock);
if (!cpu_buffer->free_page.data) {
- cpu_buffer->free_page = *data_page;
+ cpu_buffer->free_page = *rpage;
dpage = NULL;
}
@@ -7071,8 +7132,8 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu,
local_irq_restore(flags);
out:
- free_pages((unsigned long)dpage, data_page->order);
- kfree(data_page);
+ free_pages((unsigned long)dpage, rpage->order);
+ kfree(rpage);
}
EXPORT_SYMBOL_GPL(ring_buffer_free_read_page);
@@ -7123,7 +7184,6 @@ int ring_buffer_read_page(struct trace_buffer *buffer,
unsigned int size;
unsigned int read;
u64 save_timestamp;
- bool force_memcpy;
if (!cpumask_test_cpu(cpu, buffer->cpumask))
return -1;
@@ -7144,10 +7204,9 @@ int ring_buffer_read_page(struct trace_buffer *buffer,
if (!dpage)
return -1;
- guard(raw_spinlock_irqsave)(&cpu_buffer->reader_lock);
+ len = min_t(size_t, len, rb_read_page_capacity(data_page));
- if (data_page->order != cpu_buffer->reader_page->order)
- return -1;
+ guard(raw_spinlock_irqsave)(&cpu_buffer->reader_lock);
reader = rb_get_reader_page(cpu_buffer);
if (!reader)
@@ -7162,18 +7221,18 @@ int ring_buffer_read_page(struct trace_buffer *buffer,
/* Check if any events were dropped */
missed_events = cpu_buffer->lost_events;
- force_memcpy = cpu_buffer->mapped || cpu_buffer->remote;
-
- /*
- * If this page has been partially read or
- * if len is not big enough to read the rest of the page or
- * a writer is still on the page, then
- * we must copy the data from the page to the buffer.
- * Otherwise, we can simply swap the page with the one passed in.
- */
+ /*
+ * It is not possible to swap the reader page if:
+ * - It has been partially read
+ * - len is not big enough to read it entirely
+ * - A writer is still on it
+ * - The ring buffer is static
+ * - The order doesn't match
+ */
if (read || (len < (size - read)) ||
cpu_buffer->reader_page == cpu_buffer->commit_page ||
- force_memcpy) {
+ rb_is_static(cpu_buffer) ||
+ data_page->order != reader->order) {
struct buffer_data_page *rpage = cpu_buffer->reader_page->page;
unsigned int rpos = read;
unsigned int pos = 0;
@@ -7267,7 +7326,7 @@ int ring_buffer_read_page(struct trace_buffer *buffer,
* missed events, then record it there.
*/
if (missed_events > 0 &&
- rb_page_capacity(reader) - size >= sizeof(missed_events)) {
+ rb_read_page_capacity(data_page) - size >= sizeof(missed_events)) {
memcpy(&dpage->data[size], &missed_events,
sizeof(missed_events));
local_add(RB_MISSED_STORED, &dpage->commit);
@@ -7287,8 +7346,8 @@ int ring_buffer_read_page(struct trace_buffer *buffer,
/*
* This page may be off to user land. Zero it out here.
*/
- if (size < rb_page_capacity(reader))
- memset(&dpage->data[size], 0, rb_page_capacity(reader) - size);
+ if (size < rb_read_page_capacity(data_page))
+ memset(&dpage->data[size], 0, rb_read_page_capacity(data_page) - size);
return read;
}
@@ -7306,6 +7365,18 @@ void *ring_buffer_read_page_data(struct buffer_data_read_page *page)
}
EXPORT_SYMBOL_GPL(ring_buffer_read_page_data);
+/**
+ * ring_buffer_read_page_size - get size of the read page.
+ * @rpage: the page to get the size from
+ *
+ * Returns size of the page in bytes.
+ */
+unsigned int ring_buffer_read_page_size(struct buffer_data_read_page *rpage)
+{
+ return rpage ? PAGE_SIZE << rpage->order : 0;
+}
+EXPORT_SYMBOL_GPL(ring_buffer_read_page_size);
+
/**
* ring_buffer_subbuf_size_get - get size of the sub buffer.
* @buffer: the buffer to get the sub buffer size from
@@ -7386,12 +7457,20 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order)
old_capacity = rb_subbuf_capacity(buffer);
+ /* The mmap fast path reads subbuf_order without buffer->mutex. */
+ for_each_buffer_cpu(buffer, cpu) {
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
+ continue;
+ if (atomic_read(&buffer->buffers[cpu]->resize_disabled))
+ return -EBUSY;
+ }
+
atomic_inc(&buffer->record_disabled);
/* Make sure all commits have finished */
synchronize_rcu();
- buffer->subbuf_order = order;
+ WRITE_ONCE(buffer->subbuf_order, order);
/* Make sure all new buffers are allocated, before deleting the old ones */
for_each_buffer_cpu(buffer, cpu) {
@@ -7495,7 +7574,7 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order)
return 0;
error:
- buffer->subbuf_order = old_order;
+ WRITE_ONCE(buffer->subbuf_order, old_order);
atomic_dec(&buffer->record_disabled);
@@ -7615,11 +7694,7 @@ static int __rb_inc_dec_mapped(struct ring_buffer_per_cpu *cpu_buffer,
lockdep_assert_held(&cpu_buffer->mapping_lock);
- /* mapped is always greater or equal to user_mapped */
- if (WARN_ON(cpu_buffer->mapped < cpu_buffer->user_mapped))
- return -EINVAL;
-
- if (inc && cpu_buffer->mapped == UINT_MAX)
+ if (inc && cpu_buffer->user_mapped == UINT_MAX)
return -EBUSY;
if (WARN_ON(!inc && cpu_buffer->user_mapped == 0))
@@ -7628,13 +7703,10 @@ static int __rb_inc_dec_mapped(struct ring_buffer_per_cpu *cpu_buffer,
mutex_lock(&cpu_buffer->buffer->mutex);
raw_spin_lock_irqsave(&cpu_buffer->reader_lock, flags);
- if (inc) {
+ if (inc)
cpu_buffer->user_mapped++;
- cpu_buffer->mapped++;
- } else {
+ else
cpu_buffer->user_mapped--;
- cpu_buffer->mapped--;
- }
raw_spin_unlock_irqrestore(&cpu_buffer->reader_lock, flags);
mutex_unlock(&cpu_buffer->buffer->mutex);
@@ -7780,6 +7852,9 @@ int ring_buffer_map(struct trace_buffer *buffer, int cpu,
/* prevent another thread from changing buffer/sub-buffer sizes */
guard(mutex)(&buffer->mutex);
+ if (cpu_buffer->nr_pages > rb_static_max_pages())
+ return -E2BIG;
+
err = rb_alloc_meta_page(cpu_buffer);
if (err)
return err;
@@ -7806,7 +7881,6 @@ int ring_buffer_map(struct trace_buffer *buffer, int cpu,
if (!err) {
raw_spin_lock_irqsave(&cpu_buffer->reader_lock, flags);
/* This is the first time it is mapped by user */
- cpu_buffer->mapped++;
cpu_buffer->user_mapped = 1;
raw_spin_unlock_irqrestore(&cpu_buffer->reader_lock, flags);
} else {
@@ -7863,8 +7937,6 @@ int ring_buffer_unmap(struct trace_buffer *buffer, int cpu)
raw_spin_lock_irqsave(&cpu_buffer->reader_lock, flags);
/* This is the last user space mapping */
- if (!WARN_ON_ONCE(cpu_buffer->mapped < cpu_buffer->user_mapped))
- cpu_buffer->mapped--;
cpu_buffer->user_mapped = 0;
raw_spin_unlock_irqrestore(&cpu_buffer->reader_lock, flags);
diff --git a/kernel/trace/ring_buffer_benchmark.c b/kernel/trace/ring_buffer_benchmark.c
index 593e3b59e42e..c3d34c0e64e2 100644
--- a/kernel/trace/ring_buffer_benchmark.c
+++ b/kernel/trace/ring_buffer_benchmark.c
@@ -104,7 +104,7 @@ static enum event_status read_event(int cpu)
static enum event_status read_page(int cpu)
{
- struct buffer_data_read_page *bpage;
+ struct buffer_data_read_page *bpage = NULL;
struct ring_buffer_event *event;
struct rb_page *rpage;
unsigned long commit;
@@ -114,8 +114,8 @@ static enum event_status read_page(int cpu)
int inc;
int i;
- bpage = ring_buffer_alloc_read_page(buffer, cpu);
- if (IS_ERR(bpage))
+ ret = ring_buffer_alloc_read_page(buffer, cpu, &bpage);
+ if (ret < 0)
return EVENT_DROPPED;
page_size = ring_buffer_subbuf_size_get(buffer);
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index fc964001783f..c4fa38a1f167 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -7080,8 +7080,8 @@ ssize_t tracing_buffers_read(struct file *filp, char __user *ubuf,
{
struct ftrace_buffer_info *info = filp->private_data;
struct trace_iterator *iter = &info->iter;
+ unsigned int spare_size;
void *trace_data;
- int page_size;
ssize_t ret = 0;
ssize_t size;
@@ -7091,36 +7091,22 @@ ssize_t tracing_buffers_read(struct file *filp, char __user *ubuf,
if (iter->snapshot && tracer_uses_snapshot(iter->tr->current_trace))
return -EBUSY;
- page_size = ring_buffer_subbuf_size_get(iter->array_buffer->buffer);
+ spare_size = ring_buffer_read_page_size(info->spare);
- /* Make sure the spare matches the current sub buffer size */
- if (info->spare) {
- if (page_size != info->spare_size) {
- ring_buffer_free_read_page(iter->array_buffer->buffer,
- info->spare_cpu, info->spare);
- info->spare = NULL;
- }
- }
+again:
+ /* Do we have previous read data to read? */
+ if (info->read < spare_size)
+ goto read;
- if (!info->spare) {
- info->spare = ring_buffer_alloc_read_page(iter->array_buffer->buffer,
- iter->cpu_file);
- if (IS_ERR(info->spare)) {
- ret = PTR_ERR(info->spare);
- info->spare = NULL;
- } else {
- info->spare_cpu = iter->cpu_file;
- info->spare_size = page_size;
- }
- }
- if (!info->spare)
+ ret = ring_buffer_alloc_read_page(iter->array_buffer->buffer, iter->cpu_file,
+ &info->spare);
+ if (ret)
return ret;
- /* Do we have previous read data to read? */
- if (info->read < page_size)
- goto read;
+ spare_size = ring_buffer_read_page_size(info->spare);
+ info->read = spare_size;
+ info->spare_cpu = iter->cpu_file;
- again:
trace_access_lock(iter->cpu_file);
ret = ring_buffer_read_page(iter->array_buffer->buffer,
info->spare,
@@ -7146,8 +7132,9 @@ ssize_t tracing_buffers_read(struct file *filp, char __user *ubuf,
}
info->read = 0;
+
read:
- size = page_size - info->read;
+ size = spare_size - info->read;
if (size > count)
size = count;
trace_data = ring_buffer_read_page_data(info->spare);
@@ -7188,26 +7175,24 @@ int tracing_buffers_release(struct inode *inode, struct file *file)
__trace_array_put(iter->tr);
- if (info->spare)
- ring_buffer_free_read_page(iter->array_buffer->buffer,
- info->spare_cpu, info->spare);
+ ring_buffer_free_read_page(iter->array_buffer->buffer, info->spare_cpu, info->spare);
kvfree(info);
return 0;
}
struct buffer_ref {
- struct trace_buffer *buffer;
- void *page;
- int cpu;
- refcount_t refcount;
+ struct trace_buffer *buffer;
+ struct buffer_data_read_page *rpage;
+ int cpu;
+ refcount_t refcount;
};
static void buffer_ref_release(struct buffer_ref *ref)
{
if (!refcount_dec_and_test(&ref->refcount))
return;
- ring_buffer_free_read_page(ref->buffer, ref->cpu, ref->page);
+ ring_buffer_free_read_page(ref->buffer, ref->cpu, ref->rpage);
kfree(ref);
}
@@ -7266,25 +7251,15 @@ ssize_t tracing_buffers_splice_read(struct file *file, loff_t *ppos,
.ops = &buffer_pipe_buf_ops,
.spd_release = buffer_spd_release,
};
+ unsigned int page_size = 0;
struct buffer_ref *ref;
bool woken = false;
- int page_size;
int entries, i;
ssize_t ret = 0;
if (iter->snapshot && tracer_uses_snapshot(iter->tr->current_trace))
return -EBUSY;
- page_size = ring_buffer_subbuf_size_get(iter->array_buffer->buffer);
- if (*ppos & (page_size - 1))
- return -EINVAL;
-
- if (len & (page_size - 1)) {
- if (len < page_size)
- return -EINVAL;
- len &= (~(page_size - 1));
- }
-
if (splice_grow_spd(pipe, &spd))
return -ENOMEM;
@@ -7304,25 +7279,39 @@ ssize_t tracing_buffers_splice_read(struct file *file, loff_t *ppos,
refcount_set(&ref->refcount, 1);
ref->buffer = iter->array_buffer->buffer;
- ref->page = ring_buffer_alloc_read_page(ref->buffer, iter->cpu_file);
- if (IS_ERR(ref->page)) {
- ret = PTR_ERR(ref->page);
- ref->page = NULL;
+
+ ret = ring_buffer_alloc_read_page(ref->buffer, iter->cpu_file, &ref->rpage);
+ if (ret) {
kfree(ref);
break;
}
ref->cpu = iter->cpu_file;
- r = ring_buffer_read_page(ref->buffer, ref->page,
- len, iter->cpu_file, 1);
+ page_size = ring_buffer_read_page_size(ref->rpage);
+
+ r = -EINVAL;
+ if (IS_ALIGNED(*ppos, page_size) && len >= page_size) {
+ r = ring_buffer_read_page(ref->buffer, ref->rpage, len, iter->cpu_file, 1);
+ } else if (!i) {
+ /*
+ * If this fails to read on the first iteration, it
+ * means the length was too small and an error should
+ * be returned to user space. Otherwise, at least
+ * one sub-buffer was successfully read but this failed
+ * due to either the length was unaligned or the
+ * subbuf order changed. Either case, do not report
+ * an error.
+ */
+ ret = -EINVAL;
+ }
+
if (r < 0) {
- ring_buffer_free_read_page(ref->buffer, ref->cpu,
- ref->page);
+ ring_buffer_free_read_page(ref->buffer, ref->cpu, ref->rpage);
kfree(ref);
break;
}
- page = virt_to_page(ring_buffer_read_page_data(ref->page));
+ page = virt_to_page(ring_buffer_read_page_data(ref->rpage));
spd.pages[i] = page;
spd.partial[i].len = page_size;
@@ -7726,12 +7715,55 @@ trace_options_write(struct file *filp, const char __user *ubuf, size_t cnt,
return cnt;
}
+static bool tr_option_match(struct trace_array *tr, void *topt)
+{
+ for (int i = 0; i < tr->nr_topts; i++) {
+ struct trace_options *tr_topts = &tr->topts[i];
+
+ if (topt >= (void *)&tr_topts->topts[0] &&
+ topt < (void *)&tr_topts->topts[tr_topts->nr_topts])
+ return true;
+ }
+ return false;
+}
+
+/*
+ * The topt is the address of a trace_array->topts[] element that holds the
+ * the tracer options descriptor. But since the trace_array reference has not
+ * been taken yet, it cannot be dereferenced as it could have been freed by
+ * a rmdir of the instance the trace_array represents.
+ *
+ * Search the list of trace_arrays and compare the topt to the address of
+ * the entire trace_array topts array for each trace_array in the list.
+ * If one is matched, then take the reference and return it. If not, the
+ * trace_array no longer exits.
+ */
+static int trace_array_tracer_options_get(void *topt)
+{
+ struct trace_array *tr;
+ int ret;
+
+ ret = security_locked_down(LOCKDOWN_TRACEFS);
+ if (ret)
+ return ret;
+
+ if (tracing_disabled)
+ return -ENODEV;
+
+ guard(mutex)(&trace_types_lock);
+ list_for_each_entry(tr, &ftrace_trace_arrays, list) {
+ if (tr_option_match(tr, topt))
+ return __trace_array_get(tr);
+ }
+ return -ENODEV;
+}
+
static int tracing_open_options(struct inode *inode, struct file *filp)
{
struct trace_option_dentry *topt = inode->i_private;
int ret;
- ret = tracing_check_open_get_tr(topt->tr);
+ ret = trace_array_tracer_options_get(topt);
if (ret)
return ret;
@@ -7993,6 +8025,7 @@ create_trace_option_files(struct trace_array *tr, struct tracer *tracer,
tr->topts = tr_topts;
tr->topts[tr->nr_topts].tracer = tracer;
tr->topts[tr->nr_topts].topts = topts;
+ tr->topts[tr->nr_topts].nr_topts = cnt;
tr->nr_topts++;
for (cnt = 0; opts[cnt].name; cnt++) {
@@ -9708,6 +9741,11 @@ __init static void enable_instances(void)
if (flag_delim)
*flag_delim++ = '\0';
+ if (trace_array_find(name)) {
+ pr_warn("Tracing: Instance %s already exists\n", name);
+ continue;
+ }
+
if (backup) {
if (backup_instance_area(backup, &addr, &size) < 0)
continue;
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 3c111ca88e32..3749485a7d85 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -227,6 +227,7 @@ struct array_buffer {
struct trace_options {
struct tracer *tracer;
struct trace_option_dentry *topts;
+ int nr_topts;
};
struct trace_pid_list *trace_pid_list_alloc(void);
@@ -745,11 +746,10 @@ static inline int tracing_get_cpu(struct inode *inode)
void tracing_reset_cpu(struct array_buffer *buf, int cpu);
struct ftrace_buffer_info {
- struct trace_iterator iter;
- void *spare;
- unsigned int spare_cpu;
- unsigned int spare_size;
- unsigned int read;
+ struct trace_iterator iter;
+ struct buffer_data_read_page *spare;
+ unsigned int spare_cpu;
+ unsigned int read;
};
/**
@@ -1953,6 +1953,7 @@ struct event_trigger_data {
struct enable_trigger_data {
struct trace_event_file *file;
+ struct trace_event_call *call;
bool enable;
bool hist;
};
diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c
index e03471109e48..9c62202b0c97 100644
--- a/kernel/trace/trace_events.c
+++ b/kernel/trace/trace_events.c
@@ -1462,6 +1462,8 @@ int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set)
/* Put back the colon to allow this to be called again */
if (buf)
*(buf - 1) = ':';
+ if (mod)
+ *(mod - 5) = ':';
return ret;
}
diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c
index 6bf00710fd10..539b5f583af9 100644
--- a/kernel/trace/trace_events_hist.c
+++ b/kernel/trace/trace_events_hist.c
@@ -170,7 +170,6 @@ struct hist_field {
struct hist_field *operands[HIST_FIELD_OPERANDS_MAX];
struct hist_trigger_data *hist_data;
enum hist_field_fn fn_num;
- unsigned int ref;
unsigned int size;
unsigned int offset;
unsigned int is_signed;
@@ -1906,16 +1905,8 @@ static int contains_operator(char *str, char **sep)
return field_op;
}
-static void get_hist_field(struct hist_field *hist_field)
-{
- hist_field->ref++;
-}
-
static void __destroy_hist_field(struct hist_field *hist_field)
{
- if (--hist_field->ref > 1)
- return;
-
kfree(hist_field->var.name);
kfree(hist_field->name);
@@ -1962,8 +1953,6 @@ static struct hist_field *create_hist_field(struct hist_trigger_data *hist_data,
if (!hist_field)
return NULL;
- hist_field->ref = 1;
-
hist_field->hist_data = hist_data;
if (flags & HIST_FIELD_FL_EXPR || flags & HIST_FIELD_FL_ALIAS)
@@ -2218,10 +2207,8 @@ static struct hist_field *create_var_ref(struct hist_trigger_data *hist_data,
for (i = 0; i < hist_data->n_var_refs; i++) {
ref_field = hist_data->var_refs[i];
if (ref_field->var.idx == var_field->var.idx &&
- ref_field->var.hist_data == var_field->hist_data) {
- get_hist_field(ref_field);
+ ref_field->var.hist_data == var_field->hist_data)
return ref_field;
- }
}
/* Sanity check to avoid out-of-bound write on 'hist_data->var_refs' */
if (hist_data->n_var_refs >= TRACING_MAP_VARS_MAX)
@@ -2229,7 +2216,7 @@ static struct hist_field *create_var_ref(struct hist_trigger_data *hist_data,
ref_field = create_hist_field(var_field->hist_data, NULL, flags, NULL);
if (ref_field) {
if (init_var_ref(ref_field, var_field, system, event_name)) {
- destroy_hist_field(ref_field, 0);
+ __destroy_hist_field(ref_field);
return NULL;
}
@@ -2325,6 +2312,7 @@ parse_field(struct hist_trigger_data *hist_data, struct trace_event_file *file,
struct ftrace_event_field *field = NULL;
char *field_name, *modifier, *str;
struct trace_array *tr = file->tr;
+ bool stack_modifier = false;
modifier = str = kstrdup(field_str, GFP_KERNEL);
if (!modifier)
@@ -2347,9 +2335,10 @@ parse_field(struct hist_trigger_data *hist_data, struct trace_event_file *file,
*flags |= HIST_FIELD_FL_EXECNAME;
else if (strcmp(modifier, "syscall") == 0)
*flags |= HIST_FIELD_FL_SYSCALL;
- else if (strcmp(modifier, "stacktrace") == 0)
+ else if (strcmp(modifier, "stacktrace") == 0) {
*flags |= HIST_FIELD_FL_STACKTRACE;
- else if (strcmp(modifier, "log2") == 0)
+ stack_modifier = true;
+ } else if (strcmp(modifier, "log2") == 0)
*flags |= HIST_FIELD_FL_LOG2;
else if (strcmp(modifier, "usecs") == 0)
*flags |= HIST_FIELD_FL_TIMESTAMP_USECS;
@@ -2410,6 +2399,7 @@ parse_field(struct hist_trigger_data *hist_data, struct trace_event_file *file,
*flags |= HIST_FIELD_FL_CPU;
} else if (field && field->filter_type == FILTER_STACKTRACE) {
*flags |= HIST_FIELD_FL_STACKTRACE;
+ field = NULL;
} else if (field && field->filter_type == FILTER_COMM) {
*flags |= HIST_FIELD_FL_COMM | HIST_FIELD_FL_STRING;
} else {
@@ -2420,6 +2410,12 @@ parse_field(struct hist_trigger_data *hist_data, struct trace_event_file *file,
}
}
}
+
+ if (stack_modifier &&
+ (!field || field->filter_type != FILTER_STACKTRACE)) {
+ hist_err(tr, HIST_ERR_BAD_FIELD_MODIFIER, errpos(field_str));
+ field = ERR_PTR(-EINVAL);
+ }
out:
kfree(str);
@@ -3256,7 +3252,6 @@ static struct hist_field *create_var(struct hist_trigger_data *hist_data,
goto out;
}
- var->ref = 1;
var->flags = HIST_FIELD_FL_VAR;
var->var.idx = idx;
var->var.hist_data = var->hist_data = hist_data;
@@ -4293,8 +4288,7 @@ static int __create_val_field(struct hist_trigger_data *hist_data,
goto err;
} else {
/* Value */
- if (hist_field->flags & (HIST_FIELD_FL_GRAPH | HIST_FIELD_FL_PERCENT |
- HIST_FIELD_FL_BUCKET | HIST_FIELD_FL_LOG2 |
+ if (hist_field->flags & (HIST_FIELD_FL_BUCKET | HIST_FIELD_FL_LOG2 |
HIST_FIELD_FL_SYM | HIST_FIELD_FL_SYM_OFFSET |
HIST_FIELD_FL_SYSCALL | HIST_FIELD_FL_STACKTRACE))
goto err;
@@ -4311,6 +4305,7 @@ static int __create_val_field(struct hist_trigger_data *hist_data,
return ret;
err:
hist_err(file->tr, HIST_ERR_BAD_FIELD_MODIFIER, errpos(field_str));
+ destroy_hist_field(hist_field, 0);
return -EINVAL;
}
@@ -5670,7 +5665,7 @@ static int print_entries(struct seq_file *m,
{
struct tracing_map_sort_entry **sort_entries = NULL;
struct tracing_map *map = hist_data->map;
- int i, j, n_entries;
+ int i, j, n_entries, ret;
struct hist_val_stat *stats = NULL;
u64 val;
@@ -5680,6 +5675,8 @@ static int print_entries(struct seq_file *m,
if (n_entries < 0)
return n_entries;
+ ret = n_entries;
+
/* Calculate the max and the total for each field if needed. */
for (j = 0; j < hist_data->n_vals; j++) {
if (!(hist_data->fields[j]->flags &
@@ -5688,7 +5685,7 @@ static int print_entries(struct seq_file *m,
if (!stats) {
stats = kzalloc_objs(*stats, hist_data->n_vals);
if (!stats) {
- n_entries = -ENOMEM;
+ ret = -ENOMEM;
goto out;
}
}
@@ -5709,7 +5706,7 @@ static int print_entries(struct seq_file *m,
out:
tracing_map_destroy_sort_entries(sort_entries, n_entries);
- return n_entries;
+ return ret;
}
static void hist_trigger_show(struct seq_file *m,
@@ -6363,17 +6360,18 @@ static int event_hist_trigger_named_init(struct event_trigger_data *data)
{
int ret;
- data->ref++;
-
- save_named_trigger(data->named_data->name, data);
-
ret = event_hist_trigger_init(data->named_data);
if (ret < 0) {
kfree(data->cmd_ops);
data->cmd_ops = &trigger_hist_cmd;
+ return ret;
}
- return ret;
+ data->ref++;
+
+ save_named_trigger(data->named_data->name, data);
+
+ return 0;
}
static void event_hist_trigger_named_free(struct event_trigger_data *data)
@@ -6623,12 +6621,6 @@ static int hist_register_trigger(char *glob,
data->cmd_ops = cmd_ops;
}
- if (data->cmd_ops->init) {
- ret = data->cmd_ops->init(data);
- if (ret < 0)
- goto out;
- }
-
if (hist_data->enable_timestamps) {
char *clock = hist_data->attrs->clock;
@@ -6641,6 +6633,15 @@ static int hist_register_trigger(char *glob,
tracing_set_filter_buffering(file->tr, true);
}
+ if (data->cmd_ops->init) {
+ ret = data->cmd_ops->init(data);
+ if (ret < 0) {
+ if (hist_data->enable_timestamps)
+ tracing_set_filter_buffering(file->tr, false);
+ goto out;
+ }
+ }
+
if (named_data) {
remove_hist_vars(hist_data);
destroy_hist_data(hist_data);
@@ -6658,11 +6659,12 @@ static int hist_trigger_enable(struct event_trigger_data *data,
update_cond_flag(file);
- if (trace_event_trigger_enable_disable(file, 1) < 0) {
- list_del_rcu(&data->list);
- update_cond_flag(file);
+ /*
+ * On failure the caller undoes the registration, and
+ * hist_unregister_trigger() can only find the trigger here.
+ */
+ if (trace_event_trigger_enable_disable(file, 1) < 0)
ret--;
- }
return ret;
}
@@ -6740,13 +6742,13 @@ static void hist_unregister_trigger(char *glob,
}
}
- if (test && test->cmd_ops->free)
- test->cmd_ops->free(test);
-
if (hist_data->enable_timestamps) {
if (!hist_data->remove || test)
tracing_set_filter_buffering(file->tr, false);
}
+
+ if (test && test->cmd_ops->free)
+ test->cmd_ops->free(test);
}
static bool hist_file_check_refs(struct trace_event_file *file)
@@ -6951,6 +6953,8 @@ static int event_hist_trigger_parse(struct event_command *cmd_ops,
return ret;
out_unreg:
event_trigger_unregister(cmd_ops, file, glob+1, trigger_data);
+ /* The unregister frees trigger_data, skip out_free */
+ goto out;
out_free:
remove_hist_vars(hist_data);
diff --git a/kernel/trace/trace_events_trigger.c b/kernel/trace/trace_events_trigger.c
index ad83419cb420..1954184ab2b9 100644
--- a/kernel/trace/trace_events_trigger.c
+++ b/kernel/trace/trace_events_trigger.c
@@ -1728,7 +1728,8 @@ static void enable_trigger_private_data_free(struct event_trigger_data *data)
{
struct enable_trigger_data *enable_data = data->private_data;
- trace_event_put_ref(enable_data->file->event_call);
+ /* The file may already be freed here, only the call is kept alive */
+ trace_event_put_ref(enable_data->call);
kfree(enable_data);
}
@@ -1801,6 +1802,7 @@ int event_enable_trigger_parse(struct event_command *cmd_ops,
enable_data->hist = hist;
enable_data->enable = enable;
enable_data->file = event_enable_file;
+ enable_data->call = event_enable_file->event_call;
trigger_data = trigger_data_alloc(cmd_ops, cmd, param, enable_data);
if (!trigger_data) {
diff --git a/kernel/trace/trace_events_user.c b/kernel/trace/trace_events_user.c
index 6b89d225b189..e190dad1eb91 100644
--- a/kernel/trace/trace_events_user.c
+++ b/kernel/trace/trace_events_user.c
@@ -1122,10 +1122,9 @@ static void user_event_destroy_validators(struct user_event *user)
}
}
-static void user_event_destroy_fields(struct user_event *user)
+static void user_event_destroy_fields(struct list_head *head)
{
struct ftrace_event_field *field, *next;
- struct list_head *head = &user->fields;
list_for_each_entry_safe(field, next, head, link) {
list_del(&field->link);
@@ -1502,17 +1501,32 @@ static int user_event_set_call_visible(struct user_event *user, bool visible)
static int destroy_user_event(struct user_event *user)
{
+ LIST_HEAD(fields);
int ret = 0;
lockdep_assert_held(&event_mutex);
- /* Must destroy fields before call removal */
- user_event_destroy_fields(user);
+ /*
+ * Detach the fields before removing the call. Removing the event
+ * frees the field list memory (trace_destroy_fields() is run on
+ * successful removal and kmem_cache_free()s the fields), but the
+ * fields here are allocated and owned by user_events. Destroy
+ * them separately once removal has succeeded.
+ */
+ list_splice_init(&user->fields, &fields);
ret = user_event_set_call_visible(user, false);
- if (ret)
+ if (ret) {
+ /*
+ * Removal failed and the event stays registered, recover
+ * the fields so it is left in a consistent state.
+ */
+ list_splice(&fields, &user->fields);
return ret;
+ }
+
+ user_event_destroy_fields(&fields);
dyn_event_remove(&user->devent);
hash_del(&user->node);
@@ -2212,7 +2226,7 @@ static int user_event_parse(struct user_event_group *group, char *name,
put_user_lock:
mutex_unlock(&event_mutex);
put_user:
- user_event_destroy_fields(user);
+ user_event_destroy_fields(&user->fields);
user_event_destroy_validators(user);
kfree(user->call.print_fmt);
diff --git a/kernel/trace/trace_functions_graph.c b/kernel/trace/trace_functions_graph.c
index 0d2d3a2ea7dd..9ce68d207ef9 100644
--- a/kernel/trace/trace_functions_graph.c
+++ b/kernel/trace/trace_functions_graph.c
@@ -52,6 +52,7 @@ struct fgraph_data {
};
struct ftrace_graph_ret_entry ret;
int failed;
+ int ent_size;
int cpu;
};
@@ -1274,6 +1275,7 @@ print_graph_entry(struct ftrace_graph_ent_entry *field, struct trace_seq *s,
if (s->full) {
data->failed = 1;
data->cpu = cpu;
+ data->ent_size = iter->ent_size;
} else
data->failed = 0;
}
@@ -1457,6 +1459,7 @@ print_graph_function_flags(struct trace_iterator *iter, u32 flags)
if (data && data->failed) {
field = &data->ent.ent;
iter->cpu = data->cpu;
+ iter->ent_size = data->ent_size;
ret = print_graph_entry(field, s, iter, flags);
if (ret == TRACE_TYPE_HANDLED && iter->cpu != cpu) {
per_cpu_ptr(data->cpu_data, iter->cpu)->ignore = 1;
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index aae60fbebeaf..f3c20a093f5d 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -2108,19 +2108,60 @@ int traceprobe_set_print_fmt(struct trace_probe *tp, enum probe_print_type ptype
int traceprobe_define_arg_fields(struct trace_event_call *event_call,
size_t offset, struct trace_probe *tp)
{
+ struct trace_probe_event *tpe = trace_probe_event_from_call(event_call);
int ret, i;
+ /*
+ * A field created by trace_define_field() only stores the name and
+ * type pointers, it does not copy the strings. Here they point into
+ * the probe_arg of @tp, which is freed when @tp is removed. For an
+ * event with multiple probes attached, the field list is defined
+ * once by the first probe but kept alive by the surviving siblings,
+ * so removing that first probe would leave the fields referencing
+ * freed memory. Duplicate the strings and anchor the copies on the
+ * trace_probe_event, which lives as long as the field list itself.
+ *
+ * event_define_fields() ignores the return value of this hook, so
+ * if a previous attempt failed before creating any field, it may
+ * call here again. Release duplicates left behind by such an
+ * attempt before starting over.
+ */
+ for (i = 0; i < tpe->nr_field_strings; i++)
+ kfree(tpe->field_strings[i]);
+ kfree(tpe->field_strings);
+ tpe->field_strings = NULL;
+ tpe->nr_field_strings = 0;
+
+ if (tp->nr_args) {
+ tpe->field_strings = kcalloc(tp->nr_args * 2, sizeof(char *),
+ GFP_KERNEL);
+ if (!tpe->field_strings)
+ return -ENOMEM;
+ }
+
/* Set argument names as fields */
for (i = 0; i < tp->nr_args; i++) {
struct probe_arg *parg = &tp->args[i];
const char *fmt = parg->type->fmttype;
int size = parg->type->size;
+ char *name, *type;
if (parg->fmt)
fmt = parg->fmt;
if (parg->count)
size *= parg->count;
- ret = trace_define_field(event_call, fmt, parg->name,
+
+ name = kstrdup(parg->name, GFP_KERNEL);
+ type = kstrdup(fmt, GFP_KERNEL);
+ if (!name || !type) {
+ kfree(name);
+ kfree(type);
+ return -ENOMEM;
+ }
+ tpe->field_strings[tpe->nr_field_strings++] = name;
+ tpe->field_strings[tpe->nr_field_strings++] = type;
+
+ ret = trace_define_field(event_call, type, name,
offset + parg->offset, size,
parg->type->is_signed,
FILTER_OTHER);
@@ -2132,6 +2173,11 @@ int traceprobe_define_arg_fields(struct trace_event_call *event_call,
static void trace_probe_event_free(struct trace_probe_event *tpe)
{
+ int i;
+
+ for (i = 0; i < tpe->nr_field_strings; i++)
+ kfree(tpe->field_strings[i]);
+ kfree(tpe->field_strings);
kfree(tpe->class.system);
kfree(tpe->call.name);
kfree(tpe->call.print_fmt);
diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h
index 0f09f7aaf93f..480eb6acd883 100644
--- a/kernel/trace/trace_probe.h
+++ b/kernel/trace/trace_probe.h
@@ -255,6 +255,8 @@ struct trace_probe_event {
struct trace_event_call call;
struct list_head files;
struct list_head probes;
+ char **field_strings;
+ int nr_field_strings;
struct trace_uprobe_filter filter[];
};
diff --git a/kernel/trace/trace_remote.c b/kernel/trace/trace_remote.c
index e6724f947170..c79cd21347a0 100644
--- a/kernel/trace/trace_remote.c
+++ b/kernel/trace/trace_remote.c
@@ -980,9 +980,12 @@ int trace_remote_alloc_buffer(struct trace_buffer_desc *desc, size_t desc_size,
const struct cpumask *cpumask)
{
size_t min_desc_size = trace_buffer_desc_size(buffer_size, cpumask_weight(cpumask));
- unsigned int nr_pages = max(DIV_ROUND_UP(buffer_size, PAGE_SIZE), 2UL) + 1;
struct ring_buffer_desc *rb_desc;
int cpu, ret = -ENOMEM;
+ unsigned int nr_pages;
+
+ if (min_desc_size == SIZE_MAX)
+ return -E2BIG;
if (desc_size < min_desc_size)
return -EINVAL;
@@ -991,6 +994,7 @@ int trace_remote_alloc_buffer(struct trace_buffer_desc *desc, size_t desc_size,
desc->struct_len = min_desc_size;
rb_desc = __first_ring_buffer_desc(desc);
+ nr_pages = __calc_nr_pages_ring_buffer_desc(buffer_size);
for_each_cpu(cpu, cpumask) {
unsigned int id;
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 929c04a9581b..36aaeb38e217 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -7923,6 +7923,9 @@ static int wq_watchdog_param_set_thresh(const char *val,
if (ret)
return ret;
+ if (thresh > MAX_JIFFY_OFFSET / HZ)
+ return -ERANGE;
+
if (system_percpu_wq)
wq_watchdog_set_thresh(thresh);
else
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 34a52e9ba9e1..523873191091 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -1920,15 +1920,29 @@ ssize_t iov_iter_extract_bvecs(struct iov_iter *iter, struct bio_vec *bv,
unsigned short max_vecs, unsigned mem_align_mask,
iov_iter_extraction_t extraction_flags)
{
- unsigned long start = (unsigned long)iter_iov_addr(iter);
unsigned short entries_left = max_vecs - *nr_vecs;
unsigned short nr_pages, i = 0;
size_t left, offset, len;
struct page **pages;
ssize_t size;
- if ((start | iter_iov_len(iter)) & mem_align_mask)
+ /*
+ * DMA engines typically have both memory address and length alignment
+ * requirements, so check these against the alignment mask. For UBUF,
+ * IOVEC and KVEC, only the current segment will be extracted from; for
+ * everything else we might extract from multiple segments, so we need
+ * to check those too.
+ */
+ if (likely(iter_is_ubuf(iter) ||
+ iter_is_iovec(iter) ||
+ iov_iter_is_kvec(iter))) {
+ unsigned long start = (unsigned long)iter_iov_addr(iter);
+
+ if ((start | iter_iov_len(iter)) & mem_align_mask)
+ return -EINVAL;
+ } else if (iov_iter_alignment(iter) & mem_align_mask) {
return -EINVAL;
+ }
/*
* Move page array up in the allocated memory for the bio vecs as far as
diff --git a/mm/slub.c b/mm/slub.c
index 5e88c520522b..3791d69ba6f9 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -5612,10 +5612,12 @@ static noinline void free_to_partial_list(
*
* Fail if the slab isn't full anymore due to a concurrent free.
*/
-static bool __slab_try_return_freelist(struct kmem_cache *s, struct slab *slab,
- void *head, int cnt)
+static bool __slab_try_return_freelist(struct kmem_cache *s,
+ struct kmem_cache_node *n,
+ struct slab *slab, void *head, int cnt)
{
struct freelist_counters old, new;
+ unsigned long flags;
old.freelist = slab->freelist;
old.counters = slab->counters;
@@ -5627,9 +5629,15 @@ static bool __slab_try_return_freelist(struct kmem_cache *s, struct slab *slab,
new.counters = old.counters;
new.inuse -= cnt;
- if (!slab_update_freelist(s, slab, &old, &new, "__slab_try_return_freelist"))
+ spin_lock_irqsave(&n->list_lock, flags);
+
+ if (!slab_update_freelist(s, slab, &old, &new, "__slab_try_return_freelist")) {
+ spin_unlock_irqrestore(&n->list_lock, flags);
return false;
+ }
+ add_partial(n, slab, ADD_TO_TAIL);
+ spin_unlock_irqrestore(&n->list_lock, flags);
return true;
}
@@ -7202,10 +7210,8 @@ __refill_objects_node(struct kmem_cache *s, void **p, gfp_t gfp, unsigned int mi
void *head = object;
void *tail;
- if (__slab_try_return_freelist(s, slab, head, count)) {
- list_add(&slab->slab_list, &pc.slabs);
+ if (__slab_try_return_freelist(s, n, slab, head, count))
break;
- }
do {
tail = object;
@@ -7218,7 +7224,7 @@ __refill_objects_node(struct kmem_cache *s, void **p, gfp_t gfp, unsigned int mi
break;
}
- if (!list_empty(&pc.slabs)) {
+ if (unlikely(!list_empty(&pc.slabs))) {
spin_lock_irqsave(&n->list_lock, flags);
list_for_each_entry(slab, &pc.slabs, slab_list)
diff --git a/net/bluetooth/hci_core.c b/net/bluetooth/hci_core.c
index f346505c3f88..f747492da5f6 100644
--- a/net/bluetooth/hci_core.c
+++ b/net/bluetooth/hci_core.c
@@ -2629,11 +2629,11 @@ int hci_register_dev(struct hci_dev *hdev)
if (error)
BT_WARN("register suspend notifier failed error:%d\n", error);
- queue_work(hdev->req_workqueue, &hdev->power_on);
-
idr_init(&hdev->adv_monitors_idr);
msft_register(hdev);
+ queue_work(hdev->req_workqueue, &hdev->power_on);
+
return id;
err_wqueue:
diff --git a/net/bluetooth/hci_sysfs.c b/net/bluetooth/hci_sysfs.c
index 8957ce7c21b7..c2065abf753e 100644
--- a/net/bluetooth/hci_sysfs.c
+++ b/net/bluetooth/hci_sysfs.c
@@ -13,7 +13,10 @@ static const struct class bt_class = {
static void bt_link_release(struct device *dev)
{
struct hci_conn *conn = to_hci_conn(dev);
+ struct device *parent = dev->parent;
+
kfree(conn);
+ put_device(parent);
}
static const struct device_type bt_link = {
@@ -21,6 +24,16 @@ static const struct device_type bt_link = {
.release = bt_link_release,
};
+/*
+ * The rfcomm tty device will possibly retain even when conn
+ * is down, and sysfs doesn't support move zombie device,
+ * so we should move the device before conn device is destroyed.
+ */
+static int __match_tty(struct device *dev, const void *data)
+{
+ return !strncmp(dev_name(dev), "rfcomm", 6);
+}
+
void hci_conn_init_sysfs(struct hci_conn *conn)
{
struct hci_dev *hdev = conn->hdev;
@@ -29,7 +42,7 @@ void hci_conn_init_sysfs(struct hci_conn *conn)
conn->dev.type = &bt_link;
conn->dev.class = &bt_class;
- conn->dev.parent = &hdev->dev;
+ conn->dev.parent = get_device(&hdev->dev);
device_initialize(&conn->dev);
}
@@ -69,7 +82,7 @@ void hci_conn_del_sysfs(struct hci_conn *conn)
while (1) {
struct device *dev;
- dev = device_find_any_child(&conn->dev);
+ dev = device_find_child(&conn->dev, NULL, __match_tty);
if (!dev)
break;
device_move(dev, NULL, DPM_ORDER_DEV_LAST);
diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
index 30d7120d3a15..800f7517bfeb 100644
--- a/net/bluetooth/l2cap_core.c
+++ b/net/bluetooth/l2cap_core.c
@@ -1337,7 +1337,7 @@ static void l2cap_le_connect(struct l2cap_chan *chan)
struct l2cap_ecred_conn_data {
struct {
struct l2cap_ecred_conn_req_hdr req;
- __le16 scid[5];
+ __le16 scid[L2CAP_ECRED_CONN_SCID_MAX];
} __packed pdu;
struct l2cap_chan *chan;
struct pid *pid;
@@ -1352,7 +1352,7 @@ static void l2cap_ecred_defer_connect(struct l2cap_chan *chan, void *data)
if (chan == conn->chan)
return;
- if (!test_and_clear_bit(FLAG_DEFER_SETUP, &chan->flags))
+ if (!test_bit(FLAG_DEFER_SETUP, &chan->flags))
return;
pid = chan->ops->get_peer_pid(chan);
@@ -1362,9 +1362,16 @@ static void l2cap_ecred_defer_connect(struct l2cap_chan *chan, void *data)
chan->mode != L2CAP_MODE_EXT_FLOWCTL || chan->state != BT_CONNECT)
return;
+ if (!test_and_clear_bit(FLAG_DEFER_SETUP, &chan->flags))
+ return;
+
if (test_and_set_bit(FLAG_ECRED_CONN_REQ_SENT, &chan->flags))
return;
+ /* Unreachable, checked in l2cap_connect (+timer drops it if reached) */
+ if (WARN_ON_ONCE(conn->count >= ARRAY_SIZE(conn->pdu.scid)))
+ return;
+
l2cap_ecred_init(chan, 0);
/* Set the same ident so we can match on the rsp */
@@ -3893,6 +3900,9 @@ static void l2cap_ecred_rsp_defer(struct l2cap_chan *chan, void *data)
struct l2cap_ecred_conn_rsp *rsp_flex =
container_of(&rsp->pdu.rsp, struct l2cap_ecred_conn_rsp, hdr);
+ if (chan->mode != L2CAP_MODE_EXT_FLOWCTL)
+ return;
+
/* Check if channel for outgoing connection or if it wasn't deferred
* since in those cases it must be skipped.
*/
@@ -3903,6 +3913,10 @@ static void l2cap_ecred_rsp_defer(struct l2cap_chan *chan, void *data)
/* Reset ident so only one response is sent */
chan->ident = 0;
+ /* Unreachable, check in l2cap_ecred_conn_req. If reached, drop rest */
+ if (WARN_ON_ONCE(rsp->count >= ARRAY_SIZE(rsp->pdu.scid)))
+ rsp->pdu.rsp.result = cpu_to_le16(L2CAP_CR_LE_NO_MEM);
+
/* Include all channels pending with the same ident */
if (!rsp->pdu.rsp.result)
rsp_flex->dcid[rsp->count++] = cpu_to_le16(chan->scid);
@@ -5062,6 +5076,7 @@ static int l2cap_le_connect_req(struct l2cap_conn *conn,
__set_chan_timer(chan, chan->ops->get_sndtimeo(chan));
chan->ident = cmd->ident;
+ chan->mode = L2CAP_MODE_LE_FLOWCTL;
if (test_bit(FLAG_DEFER_SETUP, &chan->flags)) {
l2cap_state_change(chan, BT_CONNECT2);
@@ -7365,6 +7380,9 @@ int l2cap_chan_connect(struct l2cap_chan *chan, __le16 psm, u16 cid,
goto done;
}
+ mutex_lock(&conn->lock);
+ l2cap_chan_lock(chan);
+
if (chan->mode == L2CAP_MODE_EXT_FLOWCTL) {
struct l2cap_chan_data data;
@@ -7372,19 +7390,20 @@ int l2cap_chan_connect(struct l2cap_chan *chan, __le16 psm, u16 cid,
data.pid = chan->ops->get_peer_pid(chan);
data.count = 1;
- l2cap_chan_list(conn, l2cap_chan_by_pid, &data);
+ __l2cap_chan_list(conn, l2cap_chan_by_pid, &data);
+
+ /* Leave room for non-deferred channel that ends the group. */
+ if (test_bit(FLAG_DEFER_SETUP, &chan->flags))
+ data.count += 1;
/* Check if there isn't too many channels being connected */
if (data.count > L2CAP_ECRED_CONN_SCID_MAX) {
hci_conn_drop(hcon);
err = -EPROTO;
- goto done;
+ goto chan_unlock;
}
}
- mutex_lock(&conn->lock);
- l2cap_chan_lock(chan);
-
if (cid && __l2cap_get_chan_by_dcid(conn, cid)) {
hci_conn_drop(hcon);
err = -EBUSY;
diff --git a/net/bluetooth/msft.c b/net/bluetooth/msft.c
index ded68568e6c9..d9dd722db3eb 100644
--- a/net/bluetooth/msft.c
+++ b/net/bluetooth/msft.c
@@ -769,8 +769,8 @@ void msft_register(struct hci_dev *hdev)
INIT_LIST_HEAD(&msft->handle_map);
INIT_LIST_HEAD(&msft->address_filters);
- hdev->msft_data = msft;
mutex_init(&msft->filter_lock);
+ hdev->msft_data = msft;
}
void msft_release(struct hci_dev *hdev)
diff --git a/net/bridge/br_cfm.c b/net/bridge/br_cfm.c
index dea56fffa1c1..9dcc97d63a6f 100644
--- a/net/bridge/br_cfm.c
+++ b/net/bridge/br_cfm.c
@@ -367,7 +367,7 @@ static u32 ccm_tlv_extract(struct sk_buff *skb, u32 index,
}
/* note: already called with rcu_read_lock */
-static int br_cfm_frame_rx(struct net_bridge_port *port, struct sk_buff *skb)
+int br_cfm_frame_rx(struct net_bridge_port *port, struct sk_buff *skb)
{
u32 mdlevel, interval, size, index, max;
const struct br_cfm_common_hdr *hdr;
@@ -489,11 +489,6 @@ static int br_cfm_frame_rx(struct net_bridge_port *port, struct sk_buff *skb)
return 1;
}
-static struct br_frame_type cfm_frame_type __read_mostly = {
- .type = cpu_to_be16(ETH_P_CFM),
- .frame_handler = br_cfm_frame_rx,
-};
-
int br_cfm_mep_create(struct net_bridge *br,
const u32 instance,
struct br_cfm_mep_create *const create,
@@ -559,7 +554,7 @@ int br_cfm_mep_create(struct net_bridge *br,
INIT_DELAYED_WORK(&mep->ccm_tx_dwork, ccm_tx_work_expired);
if (hlist_empty(&br->mep_list))
- br_add_frame(br, &cfm_frame_type);
+ br_opt_toggle(br, BROPT_CFM_ENABLED, true);
hlist_add_tail_rcu(&mep->head, &br->mep_list);
@@ -588,7 +583,7 @@ static void mep_delete_implementation(struct net_bridge *br,
kfree_rcu(mep, rcu);
if (hlist_empty(&br->mep_list))
- br_del_frame(br, &cfm_frame_type);
+ br_opt_toggle(br, BROPT_CFM_ENABLED, false);
}
int br_cfm_mep_delete(struct net_bridge *br,
diff --git a/net/bridge/br_device.c b/net/bridge/br_device.c
index e7f343ab22d3..7b163d2b4e01 100644
--- a/net/bridge/br_device.c
+++ b/net/bridge/br_device.c
@@ -503,7 +503,6 @@ void br_dev_setup(struct net_device *dev)
spin_lock_init(&br->lock);
INIT_LIST_HEAD(&br->port_list);
INIT_HLIST_HEAD(&br->fdb_list);
- INIT_HLIST_HEAD(&br->frame_type_list);
#if IS_ENABLED(CONFIG_BRIDGE_MRP)
INIT_HLIST_HEAD(&br->mrp_list);
#endif
diff --git a/net/bridge/br_input.c b/net/bridge/br_input.c
index ddb8f002a40e..1322543bc917 100644
--- a/net/bridge/br_input.c
+++ b/net/bridge/br_input.c
@@ -317,17 +317,25 @@ static int nf_hook_bridge_pre(struct sk_buff *skb, struct sk_buff **pskb)
return RX_HANDLER_CONSUMED;
}
+#define BR_CFM_MRP_OPTS \
+ ((IS_ENABLED(CONFIG_BRIDGE_CFM) ? BIT(BROPT_CFM_ENABLED) : 0UL) | \
+ (IS_ENABLED(CONFIG_BRIDGE_MRP) ? BIT(BROPT_MRP_ENABLED) : 0UL))
+
/* Return 0 if the frame was not processed otherwise 1
* note: already called with rcu_read_lock
*/
static int br_process_frame_type(struct net_bridge_port *p,
struct sk_buff *skb)
{
- struct br_frame_type *tmp;
+ struct net_bridge *br = p->br;
+
+ if (skb->protocol == htons(ETH_P_CFM) &&
+ br_opt_get(br, BROPT_CFM_ENABLED))
+ return br_cfm_frame_rx(p, skb);
- hlist_for_each_entry_rcu(tmp, &p->br->frame_type_list, list)
- if (unlikely(tmp->type == skb->protocol))
- return tmp->frame_handler(p, skb);
+ if (skb->protocol == htons(ETH_P_MRP) &&
+ br_opt_get(br, BROPT_MRP_ENABLED))
+ return br_mrp_process(p, skb);
return 0;
}
@@ -425,7 +433,8 @@ static rx_handler_result_t br_handle_frame(struct sk_buff **pskb)
}
}
- if (unlikely(br_process_frame_type(p, skb)))
+ if (unlikely((READ_ONCE(p->br->options) & BR_CFM_MRP_OPTS) &&
+ br_process_frame_type(p, skb)))
return RX_HANDLER_PASS;
forward:
@@ -467,19 +476,3 @@ rx_handler_func_t *br_get_rx_handler(const struct net_device *dev)
return br_handle_frame;
}
-
-void br_add_frame(struct net_bridge *br, struct br_frame_type *ft)
-{
- hlist_add_head_rcu(&ft->list, &br->frame_type_list);
-}
-
-void br_del_frame(struct net_bridge *br, struct br_frame_type *ft)
-{
- struct br_frame_type *tmp;
-
- hlist_for_each_entry(tmp, &br->frame_type_list, list)
- if (ft == tmp) {
- hlist_del_rcu(&ft->list);
- return;
- }
-}
diff --git a/net/bridge/br_mrp.c b/net/bridge/br_mrp.c
index ef16d0703924..dce6efa96c4c 100644
--- a/net/bridge/br_mrp.c
+++ b/net/bridge/br_mrp.c
@@ -6,13 +6,6 @@
static const u8 mrp_test_dmac[ETH_ALEN] = { 0x1, 0x15, 0x4e, 0x0, 0x0, 0x1 };
static const u8 mrp_in_test_dmac[ETH_ALEN] = { 0x1, 0x15, 0x4e, 0x0, 0x0, 0x3 };
-static int br_mrp_process(struct net_bridge_port *p, struct sk_buff *skb);
-
-static struct br_frame_type mrp_frame_type __read_mostly = {
- .type = cpu_to_be16(ETH_P_MRP),
- .frame_handler = br_mrp_process,
-};
-
static bool br_mrp_is_ring_port(struct net_bridge_port *p_port,
struct net_bridge_port *s_port,
struct net_bridge_port *port)
@@ -486,7 +479,7 @@ static void br_mrp_del_impl(struct net_bridge *br, struct br_mrp *mrp)
kfree_rcu(mrp, rcu);
if (hlist_empty(&br->mrp_list))
- br_del_frame(br, &mrp_frame_type);
+ br_opt_toggle(br, BROPT_MRP_ENABLED, false);
}
/* Adds a new MRP instance.
@@ -536,7 +529,7 @@ int br_mrp_add(struct net_bridge *br, struct br_mrp_instance *instance)
rcu_assign_pointer(mrp->s_port, p);
if (hlist_empty(&br->mrp_list))
- br_add_frame(br, &mrp_frame_type);
+ br_opt_toggle(br, BROPT_MRP_ENABLED, true);
INIT_DELAYED_WORK(&mrp->test_work, br_mrp_test_work_expired);
INIT_DELAYED_WORK(&mrp->in_test_work, br_mrp_in_test_work_expired);
@@ -1241,7 +1234,7 @@ static int br_mrp_rcv(struct net_bridge_port *p,
* normal forwarding.
* note: already called with rcu_read_lock
*/
-static int br_mrp_process(struct net_bridge_port *p, struct sk_buff *skb)
+int br_mrp_process(struct net_bridge_port *p, struct sk_buff *skb)
{
/* If there is no MRP instance do normal forwarding */
if (likely(!test_bit(BR_MRP_AWARE_BIT, &p->flags)))
diff --git a/net/bridge/br_multicast.c b/net/bridge/br_multicast.c
index ec0339e204bd..2191d35d9fa0 100644
--- a/net/bridge/br_multicast.c
+++ b/net/bridge/br_multicast.c
@@ -1435,16 +1435,17 @@ struct net_bridge_port_group *br_multicast_new_port_group(
goto free_out;
}
- rcu_assign_pointer(p->next, next);
timer_setup(&p->timer, br_multicast_port_group_expired, 0);
timer_setup(&p->rexmit_timer, br_multicast_port_group_rexmit, 0);
- hlist_add_head(&p->mglist, &port->mglist);
if (src)
memcpy(p->eth_addr, src, ETH_ALEN);
else
eth_broadcast_addr(p->eth_addr);
+ RCU_INIT_POINTER(p->next, next);
+ hlist_add_head_rcu(&p->mglist, &port->mglist);
+
return p;
free_out:
@@ -1459,11 +1460,11 @@ void br_multicast_del_port_group(struct net_bridge_port_group *p)
struct net_bridge_port *port = p->key.port;
__u16 vid = p->key.addr.vid;
- hlist_del_init(&p->mglist);
+ hlist_del_init_rcu(&p->mglist);
if (!br_multicast_is_star_g(&p->key.addr))
rhashtable_remove_fast(&port->br->sg_port_tbl, &p->rhnode,
br_sg_port_rht_params);
- kfree(p);
+ kfree_rcu(p, rcu);
br_multicast_port_ngroups_dec(port, vid);
}
diff --git a/net/bridge/br_private.h b/net/bridge/br_private.h
index d3880f31edc4..d08e318f0455 100644
--- a/net/bridge/br_private.h
+++ b/net/bridge/br_private.h
@@ -491,12 +491,13 @@ enum net_bridge_opts {
BROPT_MST_ENABLED,
BROPT_MDB_OFFLOAD_FAIL_NOTIFICATION,
BROPT_FDB_LOCAL_VLAN_0,
+ BROPT_CFM_ENABLED,
+ BROPT_MRP_ENABLED,
};
struct net_bridge {
spinlock_t lock;
spinlock_t hash_lock;
- struct hlist_head frame_type_list;
struct net_device *dev;
unsigned long options;
/* These fields are accessed on each packet */
@@ -928,16 +929,6 @@ int nbp_backup_change(struct net_bridge_port *p, struct net_device *backup_dev);
int br_handle_frame_finish(struct net *net, struct sock *sk, struct sk_buff *skb);
rx_handler_func_t *br_get_rx_handler(const struct net_device *dev);
-struct br_frame_type {
- __be16 type;
- int (*frame_handler)(struct net_bridge_port *port,
- struct sk_buff *skb);
- struct hlist_node list;
-};
-
-void br_add_frame(struct net_bridge *br, struct br_frame_type *ft);
-void br_del_frame(struct net_bridge *br, struct br_frame_type *ft);
-
static inline bool br_rx_handler_check_rcu(const struct net_device *dev)
{
return rcu_dereference(dev->rx_handler) == br_get_rx_handler(dev);
@@ -2076,6 +2067,7 @@ int br_mrp_parse(struct net_bridge *br, struct net_bridge_port *p,
bool br_mrp_enabled(struct net_bridge *br);
void br_mrp_port_del(struct net_bridge *br, struct net_bridge_port *p);
int br_mrp_fill_info(struct sk_buff *skb, struct net_bridge *br);
+int br_mrp_process(struct net_bridge_port *p, struct sk_buff *skb);
#else
static inline int br_mrp_parse(struct net_bridge *br, struct net_bridge_port *p,
struct nlattr *attr, int cmd,
@@ -2099,6 +2091,11 @@ static inline int br_mrp_fill_info(struct sk_buff *skb, struct net_bridge *br)
return 0;
}
+static inline int br_mrp_process(struct net_bridge_port *p, struct sk_buff *skb)
+{
+ return 0;
+}
+
#endif
/* br_cfm.c */
@@ -2107,6 +2104,7 @@ int br_cfm_parse(struct net_bridge *br, struct net_bridge_port *p,
struct nlattr *attr, int cmd, struct netlink_ext_ack *extack);
bool br_cfm_created(struct net_bridge *br);
void br_cfm_port_del(struct net_bridge *br, struct net_bridge_port *p);
+int br_cfm_frame_rx(struct net_bridge_port *port, struct sk_buff *skb);
int br_cfm_config_fill_info(struct sk_buff *skb, struct net_bridge *br);
int br_cfm_status_fill_info(struct sk_buff *skb,
struct net_bridge *br,
@@ -2131,6 +2129,12 @@ static inline void br_cfm_port_del(struct net_bridge *br,
{
}
+static inline int br_cfm_frame_rx(struct net_bridge_port *port,
+ struct sk_buff *skb)
+{
+ return 0;
+}
+
static inline int br_cfm_config_fill_info(struct sk_buff *skb, struct net_bridge *br)
{
return -EOPNOTSUPP;
diff --git a/net/core/dev.c b/net/core/dev.c
index 39807b68ff26..74dd9e25f4ff 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -9932,7 +9932,7 @@ int netif_change_tx_queue_len(struct net_device *dev, unsigned long new_len)
unsigned int orig_len = dev->tx_queue_len;
int res;
- if (new_len != (unsigned int)new_len)
+ if (new_len > S16_MAX)
return -ERANGE;
if (new_len != orig_len) {
@@ -12541,7 +12541,7 @@ int __dev_change_net_namespace(struct net_device *dev, struct net *net,
const char *pat, int new_ifindex,
struct netlink_ext_ack *extack)
{
- struct netdev_name_node *name_node;
+ struct netdev_name_node *name_node, *tmp;
struct net *net_old = dev_net(dev);
char new_name[IFNAMSIZ] = {};
int err, new_nsid;
@@ -12587,13 +12587,19 @@ int __dev_change_net_namespace(struct net_device *dev, struct net *net,
}
/* Check that none of the altnames conflicts. */
err = -EEXIST;
- netdev_for_each_altname(dev, name_node) {
- if (netdev_name_in_use(net, name_node->name)) {
- NL_SET_ERR_MSG_FMT(extack,
- "An interface with the altname %s exists in the target netns",
- name_node->name);
- goto out;
+ netdev_for_each_altname_safe(dev, name_node, tmp) {
+ if (!netdev_name_in_use(net, name_node->name))
+ continue;
+
+ if (!check_net(net_old)) {
+ __netdev_name_node_alt_destroy(name_node);
+ continue;
}
+
+ NL_SET_ERR_MSG_FMT(extack,
+ "An interface with the altname %s exists in the target netns",
+ name_node->name);
+ goto out;
}
/* Check that new_ifindex isn't used yet. */
@@ -13044,7 +13050,6 @@ static struct pernet_operations __net_initdata netdev_net_ops = {
static void __net_exit default_device_exit_net(struct net *net)
{
- struct netdev_name_node *name_node, *tmp;
struct net_device *dev, *aux;
/*
* Push all migratable network devices back to the
@@ -13068,10 +13073,6 @@ static void __net_exit default_device_exit_net(struct net *net)
if (netdev_name_in_use(&init_net, fb_name))
snprintf(fb_name, IFNAMSIZ, "dev%%d");
- netdev_for_each_altname_safe(dev, name_node, tmp)
- if (netdev_name_in_use(&init_net, name_node->name))
- __netdev_name_node_alt_destroy(name_node);
-
err = dev_change_net_namespace(dev, &init_net, fb_name);
if (err) {
pr_emerg("%s: failed to move %s to init_net: %d\n",
diff --git a/net/core/gro_cells.c b/net/core/gro_cells.c
index 1b84385c04bd..d8c0a2867120 100644
--- a/net/core/gro_cells.c
+++ b/net/core/gro_cells.c
@@ -22,6 +22,8 @@ int gro_cells_receive(struct gro_cells *gcells, struct sk_buff *skb)
if (unlikely(!(dev->flags & IFF_UP)))
goto drop;
+ skb_unset_transport_header(skb);
+
if (!gcells->cells || skb_cloned(skb) || netif_elide_gro(dev)) {
res = netif_rx(skb);
goto unlock;
diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
index 12aa3aa1688b..73101ec65c9a 100644
--- a/net/core/rtnetlink.c
+++ b/net/core/rtnetlink.c
@@ -2236,6 +2236,11 @@ static int rtnl_fill_ifinfo(struct sk_buff *skb,
return -EMSGSIZE;
}
+static const struct netlink_range_validation txqlen_range = {
+ .min = 0,
+ .max = S16_MAX,
+};
+
static const struct nla_policy ifla_policy[IFLA_MAX+1] = {
[IFLA_UNSPEC] = { .strict_start_type = IFLA_DPLL_PIN },
[IFLA_IFNAME] = { .type = NLA_STRING, .len = IFNAMSIZ-1 },
@@ -2246,7 +2251,7 @@ static const struct nla_policy ifla_policy[IFLA_MAX+1] = {
[IFLA_LINK] = { .type = NLA_U32 },
[IFLA_MASTER] = { .type = NLA_U32 },
[IFLA_CARRIER] = { .type = NLA_U8 },
- [IFLA_TXQLEN] = { .type = NLA_U32 },
+ [IFLA_TXQLEN] = NLA_POLICY_FULL_RANGE(NLA_U32, &txqlen_range),
[IFLA_WEIGHT] = { .type = NLA_U32 },
[IFLA_OPERSTATE] = { .type = NLA_U8 },
[IFLA_LINKMODE] = { .type = NLA_U8 },
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index b3edb90a21d6..0e964c67a721 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -6678,6 +6678,13 @@ int skb_mpls_pop(struct sk_buff *skb, __be16 next_proto, int mac_len,
}
skb->protocol = next_proto;
+ /* The last label is gone, so the inner header recorded by
+ * skb_mpls_push() no longer describes this packet. Drop it, or a
+ * later push keeps the stale offset.
+ */
+ if (!eth_p_mpls(next_proto))
+ skb->inner_protocol = 0;
+
return 0;
}
EXPORT_SYMBOL_GPL(skb_mpls_pop);
diff --git a/net/dsa/tag_brcm.c b/net/dsa/tag_brcm.c
index 411e3b57d16a..b7c49822ca88 100644
--- a/net/dsa/tag_brcm.c
+++ b/net/dsa/tag_brcm.c
@@ -373,6 +373,7 @@ static const struct dsa_device_ops brcm_legacy_fcs_netdev_ops = {
.xmit = brcm_leg_fcs_tag_xmit,
.rcv = brcm_leg_tag_rcv,
.needed_headroom = BRCM_LEG_TAG_LEN,
+ .needed_tailroom = ETH_FCS_LEN,
};
DSA_TAG_DRIVER(brcm_legacy_fcs_netdev_ops);
diff --git a/net/hsr/hsr_slave.c b/net/hsr/hsr_slave.c
index 01c73b4b50dd..a546f70f9cc8 100644
--- a/net/hsr/hsr_slave.c
+++ b/net/hsr/hsr_slave.c
@@ -149,9 +149,12 @@ static int hsr_portdev_setup(struct hsr_priv *hsr, struct net_device *dev,
int res;
/* Don't use promiscuous mode for offload since L2 frame forward
- * happens at the offloaded hardware.
+ * happens at the offloaded hardware. The interlink port never
+ * gets forwarding offload (RedBox forwarding to/from it is done
+ * by this driver), so it still needs promiscuous mode to receive
+ * frames addressed to hsr_dev's MAC rather than its own.
*/
- if (!port->hsr->fwd_offloaded) {
+ if (!port->hsr->fwd_offloaded || port->type == HSR_PT_INTERLINK) {
res = dev_set_promiscuity(dev, 1);
if (res)
return res;
@@ -176,7 +179,7 @@ static int hsr_portdev_setup(struct hsr_priv *hsr, struct net_device *dev,
fail_rx_handler:
netdev_upper_dev_unlink(dev, hsr_dev);
fail_upper_dev_link:
- if (!port->hsr->fwd_offloaded)
+ if (!port->hsr->fwd_offloaded || port->type == HSR_PT_INTERLINK)
dev_set_promiscuity(dev, -1);
return res;
@@ -240,7 +243,7 @@ void hsr_del_port(struct hsr_port *port)
netdev_update_features(master->dev);
dev_set_mtu(master->dev, hsr_get_max_mtu(hsr));
netdev_rx_handler_unregister(port->dev);
- if (!port->hsr->fwd_offloaded)
+ if (!port->hsr->fwd_offloaded || port->type == HSR_PT_INTERLINK)
dev_set_promiscuity(port->dev, -1);
if (port->type == HSR_PT_SLAVE_A || port->type == HSR_PT_SLAVE_B)
vlan_vids_del_by_dev(port->dev, master->dev);
diff --git a/net/ieee802154/6lowpan/core.c b/net/ieee802154/6lowpan/core.c
index 018929563c6b..6a8d6852cb93 100644
--- a/net/ieee802154/6lowpan/core.c
+++ b/net/ieee802154/6lowpan/core.c
@@ -150,7 +150,7 @@ static int lowpan_newlink(struct net_device *ldev,
wdev = dev_get_by_index(dev_net(ldev), nla_get_u32(tb[IFLA_LINK]));
if (!wdev)
return -ENODEV;
- if (wdev->type != ARPHRD_IEEE802154) {
+ if (wdev->type != ARPHRD_IEEE802154 || !wdev->ieee802154_ptr) {
dev_put(wdev);
return -EINVAL;
}
diff --git a/net/ipv4/fib_rules.c b/net/ipv4/fib_rules.c
index e068a5bace73..dcbbfdb291ed 100644
--- a/net/ipv4/fib_rules.c
+++ b/net/ipv4/fib_rules.c
@@ -214,6 +214,8 @@ INDIRECT_CALLABLE_SCOPE int fib4_rule_match(struct fib_rule *rule,
return 1;
}
+#define FIB_MAX_AUTO_TABLE_ID 4096
+
static struct fib_table *fib_empty_table(struct net *net)
{
u32 id = 1;
@@ -222,7 +224,7 @@ static struct fib_table *fib_empty_table(struct net *net)
if (!fib_get_table(net, id))
return fib_new_table(net, id);
- if (id++ == RT_TABLE_MAX)
+ if (id++ == FIB_MAX_AUTO_TABLE_ID)
break;
}
return NULL;
diff --git a/net/ipv4/igmp.c b/net/ipv4/igmp.c
index bb2d4441a492..0a5b7ded23be 100644
--- a/net/ipv4/igmp.c
+++ b/net/ipv4/igmp.c
@@ -188,6 +188,10 @@ static void ip_ma_put(struct ip_mc_list *im)
}
}
+#define pmc_dereference(e, pmc) \
+ rcu_dereference_protected(e, lockdep_is_held(&(pmc)->lock) || \
+ lockdep_is_held(&(pmc)->interface->mc_tomb_lock))
+
#define for_each_pmc_rcu(in_dev, pmc) \
for (pmc = rcu_dereference(in_dev->mc_list); \
pmc != NULL; \
@@ -198,13 +202,28 @@ static void ip_ma_put(struct ip_mc_list *im)
pmc != NULL; \
pmc = rtnl_dereference(pmc->next_rcu))
+#define for_each_psf_mclock(pmc, psf) \
+ for (psf = pmc_dereference((pmc)->sources, pmc); \
+ psf; \
+ psf = pmc_dereference(psf->sf_next, pmc))
+
+#define for_each_psf_rcu(im, psf) \
+ for (psf = rcu_dereference((im)->sources); \
+ psf; \
+ psf = rcu_dereference(psf->sf_next))
+
+#define for_each_psf_tomb(pmc, psf) \
+ for (psf = pmc_dereference((pmc)->tomb, pmc); \
+ psf; \
+ psf = pmc_dereference(psf->sf_next, pmc))
+
static void ip_sf_list_clear_all(struct ip_sf_list *psf)
{
struct ip_sf_list *next;
while (psf) {
- next = psf->sf_next;
- kfree(psf);
+ next = rcu_dereference_protected(psf->sf_next, 1);
+ kfree_rcu(psf, rcu);
psf = next;
}
}
@@ -349,7 +368,7 @@ igmp_scount(struct ip_mc_list *pmc, int type, int gdeleted, int sdeleted)
struct ip_sf_list *psf;
int scount = 0;
- for (psf = pmc->sources; psf; psf = psf->sf_next) {
+ for_each_psf_mclock(pmc, psf) {
if (!is_in(pmc, psf, type, gdeleted, sdeleted))
continue;
scount++;
@@ -494,7 +513,8 @@ static struct sk_buff *add_grec(struct sk_buff *skb, struct ip_mc_list *pmc,
struct net *net = dev_net(dev);
struct igmpv3_report *pih;
struct igmpv3_grec *pgr = NULL;
- struct ip_sf_list *psf, *psf_next, *psf_prev, **psf_list;
+ struct ip_sf_list *psf, *psf_next, *psf_prev;
+ struct ip_sf_list __rcu **psf_list;
int scount, stotal, first, isquery, truncate;
unsigned int mtu;
@@ -517,7 +537,7 @@ static struct sk_buff *add_grec(struct sk_buff *skb, struct ip_mc_list *pmc,
psf_list = sdeleted ? &pmc->tomb : &pmc->sources;
- if (!*psf_list)
+ if (!rcu_access_pointer(*psf_list))
goto empty_source;
pih = skb ? igmpv3_report_hdr(skb) : NULL;
@@ -533,10 +553,12 @@ static struct sk_buff *add_grec(struct sk_buff *skb, struct ip_mc_list *pmc,
}
first = 1;
psf_prev = NULL;
- for (psf = *psf_list; psf; psf = psf_next) {
+ for (psf = pmc_dereference(*psf_list, pmc);
+ psf;
+ psf = psf_next) {
__be32 *psrc;
- psf_next = psf->sf_next;
+ psf_next = pmc_dereference(psf->sf_next, pmc);
if (!is_in(pmc, psf, type, gdeleted, sdeleted)) {
psf_prev = psf;
@@ -583,10 +605,12 @@ static struct sk_buff *add_grec(struct sk_buff *skb, struct ip_mc_list *pmc,
psf->sf_crcount--;
if ((sdeleted || gdeleted) && psf->sf_crcount == 0) {
if (psf_prev)
- psf_prev->sf_next = psf->sf_next;
+ rcu_assign_pointer(psf_prev->sf_next,
+ psf_next);
else
- *psf_list = psf->sf_next;
- kfree(psf);
+ rcu_assign_pointer(*psf_list,
+ psf_next);
+ kfree_rcu(psf, rcu);
continue;
}
}
@@ -655,28 +679,29 @@ static int igmpv3_send_report(struct in_device *in_dev, struct ip_mc_list *pmc)
/*
* remove zero-count source records from a source filter list
*/
-static void igmpv3_clear_zeros(struct ip_sf_list **ppsf)
+static void igmpv3_clear_zeros(struct ip_sf_list __rcu **ppsf)
{
struct ip_sf_list *psf_prev, *psf_next, *psf;
psf_prev = NULL;
- for (psf = *ppsf; psf; psf = psf_next) {
- psf_next = psf->sf_next;
+ for (psf = rcu_dereference_protected(*ppsf, 1); psf; psf = psf_next) {
+ psf_next = rcu_dereference_protected(psf->sf_next, 1);
if (psf->sf_crcount == 0) {
if (psf_prev)
- psf_prev->sf_next = psf->sf_next;
+ rcu_assign_pointer(psf_prev->sf_next, psf_next);
else
- *ppsf = psf->sf_next;
- kfree(psf);
- } else
+ rcu_assign_pointer(*ppsf, psf_next);
+ kfree_rcu(psf, rcu);
+ } else {
psf_prev = psf;
+ }
}
}
static void kfree_pmc(struct ip_mc_list *pmc)
{
- ip_sf_list_clear_all(pmc->sources);
- ip_sf_list_clear_all(pmc->tomb);
+ ip_sf_list_clear_all(rcu_dereference_protected(pmc->sources, 1));
+ ip_sf_list_clear_all(rcu_dereference_protected(pmc->tomb, 1));
kfree(pmc);
}
@@ -710,7 +735,8 @@ static void igmpv3_send_cr(struct in_device *in_dev)
igmpv3_clear_zeros(&pmc->sources);
}
}
- if (pmc->crcount == 0 && !pmc->tomb && !pmc->sources) {
+ if (pmc->crcount == 0 && !rcu_access_pointer(pmc->tomb) &&
+ !rcu_access_pointer(pmc->sources)) {
if (pmc_prev)
pmc_prev->next = pmc_next;
else
@@ -896,7 +922,7 @@ static int igmp_xmarksources(struct ip_mc_list *pmc, int nsrcs, __be32 *srcs)
int i, scount;
scount = 0;
- for (psf = pmc->sources; psf; psf = psf->sf_next) {
+ for_each_psf_mclock(pmc, psf) {
if (scount == nsrcs)
break;
for (i = 0; i < nsrcs; i++) {
@@ -927,7 +953,7 @@ static int igmp_marksources(struct ip_mc_list *pmc, int nsrcs, __be32 *srcs)
/* mark INCLUDE-mode sources */
scount = 0;
- for (psf = pmc->sources; psf; psf = psf->sf_next) {
+ for_each_psf_mclock(pmc, psf) {
if (scount == nsrcs)
break;
for (i = 0; i < nsrcs; i++)
@@ -1228,11 +1254,12 @@ static void igmpv3_add_delrec(struct in_device *in_dev, struct ip_mc_list *im,
if (pmc->sfmode == MCAST_INCLUDE) {
struct ip_sf_list *psf;
+ for_each_psf_mclock(im, psf)
+ psf->sf_crcount = pmc->crcount;
pmc->tomb = im->tomb;
pmc->sources = im->sources;
- im->tomb = im->sources = NULL;
- for (psf = pmc->sources; psf; psf = psf->sf_next)
- psf->sf_crcount = pmc->crcount;
+ RCU_INIT_POINTER(im->tomb, NULL);
+ RCU_INIT_POINTER(im->sources, NULL);
}
spin_unlock_bh(&im->lock);
@@ -1271,9 +1298,18 @@ static void igmpv3_del_delrec(struct in_device *in_dev, struct ip_mc_list *im)
if (pmc) {
im->interface = pmc->interface;
if (im->sfmode == MCAST_INCLUDE) {
- swap(im->tomb, pmc->tomb);
- swap(im->sources, pmc->sources);
- for (psf = im->sources; psf; psf = psf->sf_next)
+ struct ip_sf_list *sources, *tomb;
+
+ tomb = rcu_replace_pointer(im->tomb,
+ rcu_dereference_protected(pmc->tomb, 1),
+ lockdep_is_held(&im->lock));
+ rcu_assign_pointer(pmc->tomb, tomb);
+
+ sources = rcu_replace_pointer(im->sources,
+ rcu_dereference_protected(pmc->sources, 1),
+ lockdep_is_held(&im->lock));
+ rcu_assign_pointer(pmc->sources, sources);
+ for_each_psf_mclock(im, psf)
psf->sf_crcount = in_dev->mr_qrv ?:
READ_ONCE(net->ipv4.sysctl_igmp_qrv);
} else {
@@ -1310,8 +1346,8 @@ static void igmpv3_clear_delrec(struct in_device *in_dev)
struct ip_sf_list *psf;
spin_lock_bh(&pmc->lock);
- psf = pmc->tomb;
- pmc->tomb = NULL;
+ psf = pmc_dereference(pmc->tomb, pmc);
+ RCU_INIT_POINTER(pmc->tomb, NULL);
spin_unlock_bh(&pmc->lock);
ip_sf_list_clear_all(psf);
}
@@ -1988,7 +2024,7 @@ static int ip_mc_del1_src(struct ip_mc_list *pmc, int sfmode,
int rv = 0;
psf_prev = NULL;
- for (psf = pmc->sources; psf; psf = psf->sf_next) {
+ for_each_psf_mclock(pmc, psf) {
if (psf->sf_inaddr == *psfsrc)
break;
psf_prev = psf;
@@ -1997,7 +2033,7 @@ static int ip_mc_del1_src(struct ip_mc_list *pmc, int sfmode,
/* source filter not found, or count wrong => bug */
return -ESRCH;
}
- psf->sf_count[sfmode]--;
+ WRITE_ONCE(psf->sf_count[sfmode], psf->sf_count[sfmode] - 1);
if (psf->sf_count[sfmode] == 0) {
ip_rt_multicast_event(pmc->interface);
}
@@ -2009,19 +2045,28 @@ static int ip_mc_del1_src(struct ip_mc_list *pmc, int sfmode,
/* no more filters for this source */
if (psf_prev)
- psf_prev->sf_next = psf->sf_next;
+ rcu_assign_pointer(psf_prev->sf_next,
+ pmc_dereference(psf->sf_next, pmc));
else
- pmc->sources = psf->sf_next;
+ rcu_assign_pointer(pmc->sources,
+ pmc_dereference(psf->sf_next, pmc));
#ifdef CONFIG_IP_MULTICAST
if (psf->sf_oldin &&
!IGMP_V1_SEEN(in_dev) && !IGMP_V2_SEEN(in_dev)) {
- psf->sf_crcount = in_dev->mr_qrv ?: READ_ONCE(net->ipv4.sysctl_igmp_qrv);
- psf->sf_next = pmc->tomb;
- pmc->tomb = psf;
- rv = 1;
- } else
+ struct ip_sf_list *dpsf = kmalloc_obj(*dpsf, GFP_ATOMIC);
+
+ if (dpsf) {
+ *dpsf = *psf;
+ dpsf->sf_crcount = in_dev->mr_qrv ?:
+ READ_ONCE(net->ipv4.sysctl_igmp_qrv);
+ rcu_assign_pointer(dpsf->sf_next,
+ pmc_dereference(pmc->tomb, pmc));
+ rcu_assign_pointer(pmc->tomb, dpsf);
+ rv = 1;
+ }
+ }
#endif
- kfree(psf);
+ kfree_rcu(psf, rcu);
}
return rv;
}
@@ -2058,7 +2103,7 @@ static int ip_mc_del_src(struct in_device *in_dev, __be32 *pmca, int sfmode,
err = -EINVAL;
if (!pmc->sfcount[sfmode])
goto out_unlock;
- pmc->sfcount[sfmode]--;
+ WRITE_ONCE(pmc->sfcount[sfmode], pmc->sfcount[sfmode] - 1);
}
err = 0;
for (i = 0; i < sfcount; i++) {
@@ -2081,7 +2126,7 @@ static int ip_mc_del_src(struct in_device *in_dev, __be32 *pmca, int sfmode,
#ifdef CONFIG_IP_MULTICAST
pmc->crcount = in_dev->mr_qrv ?: READ_ONCE(net->ipv4.sysctl_igmp_qrv);
WRITE_ONCE(in_dev->mr_ifc_count, pmc->crcount);
- for (psf = pmc->sources; psf; psf = psf->sf_next)
+ for_each_psf_mclock(pmc, psf)
psf->sf_crcount = 0;
igmp_ifc_event(pmc->interface);
} else if (sf_setstate(pmc) || changerec) {
@@ -2102,7 +2147,7 @@ static int ip_mc_add1_src(struct ip_mc_list *pmc, int sfmode,
struct ip_sf_list *psf, *psf_prev;
psf_prev = NULL;
- for (psf = pmc->sources; psf; psf = psf->sf_next) {
+ for_each_psf_mclock(pmc, psf) {
if (psf->sf_inaddr == *psfsrc)
break;
psf_prev = psf;
@@ -2112,12 +2157,12 @@ static int ip_mc_add1_src(struct ip_mc_list *pmc, int sfmode,
if (!psf)
return -ENOBUFS;
psf->sf_inaddr = *psfsrc;
- if (psf_prev) {
- psf_prev->sf_next = psf;
- } else
- pmc->sources = psf;
+ if (psf_prev)
+ rcu_assign_pointer(psf_prev->sf_next, psf);
+ else
+ rcu_assign_pointer(pmc->sources, psf);
}
- psf->sf_count[sfmode]++;
+ WRITE_ONCE(psf->sf_count[sfmode], psf->sf_count[sfmode] + 1);
if (psf->sf_count[sfmode] == 1) {
ip_rt_multicast_event(pmc->interface);
}
@@ -2130,13 +2175,15 @@ static void sf_markstate(struct ip_mc_list *pmc)
struct ip_sf_list *psf;
int mca_xcount = pmc->sfcount[MCAST_EXCLUDE];
- for (psf = pmc->sources; psf; psf = psf->sf_next)
+ for_each_psf_mclock(pmc, psf) {
if (pmc->sfcount[MCAST_EXCLUDE]) {
psf->sf_oldin = mca_xcount ==
psf->sf_count[MCAST_EXCLUDE] &&
!psf->sf_count[MCAST_INCLUDE];
- } else
+ } else {
psf->sf_oldin = psf->sf_count[MCAST_INCLUDE] != 0;
+ }
+ }
}
static int sf_setstate(struct ip_mc_list *pmc)
@@ -2147,27 +2194,31 @@ static int sf_setstate(struct ip_mc_list *pmc)
int new_in, rv;
rv = 0;
- for (psf = pmc->sources; psf; psf = psf->sf_next) {
+ for_each_psf_mclock(pmc, psf) {
if (pmc->sfcount[MCAST_EXCLUDE]) {
new_in = mca_xcount == psf->sf_count[MCAST_EXCLUDE] &&
!psf->sf_count[MCAST_INCLUDE];
- } else
+ } else {
new_in = psf->sf_count[MCAST_INCLUDE] != 0;
+ }
if (new_in) {
if (!psf->sf_oldin) {
struct ip_sf_list *prev = NULL;
- for (dpsf = pmc->tomb; dpsf; dpsf = dpsf->sf_next) {
+ for_each_psf_tomb(pmc, dpsf) {
if (dpsf->sf_inaddr == psf->sf_inaddr)
break;
prev = dpsf;
}
if (dpsf) {
+ struct ip_sf_list *dpsf_next;
+
+ dpsf_next = pmc_dereference(dpsf->sf_next, pmc);
if (prev)
- prev->sf_next = dpsf->sf_next;
+ rcu_assign_pointer(prev->sf_next, dpsf_next);
else
- pmc->tomb = dpsf->sf_next;
- kfree(dpsf);
+ rcu_assign_pointer(pmc->tomb, dpsf_next);
+ kfree_rcu(dpsf, rcu);
}
psf->sf_crcount = qrv;
rv++;
@@ -2179,17 +2230,19 @@ static int sf_setstate(struct ip_mc_list *pmc)
* add or update "delete" records if an active filter
* is now inactive
*/
- for (dpsf = pmc->tomb; dpsf; dpsf = dpsf->sf_next)
+ for_each_psf_tomb(pmc, dpsf) {
if (dpsf->sf_inaddr == psf->sf_inaddr)
break;
+ }
if (!dpsf) {
dpsf = kmalloc_obj(*dpsf, GFP_ATOMIC);
if (!dpsf)
continue;
*dpsf = *psf;
/* pmc->lock held by callers */
- dpsf->sf_next = pmc->tomb;
- pmc->tomb = dpsf;
+ rcu_assign_pointer(dpsf->sf_next,
+ pmc_dereference(pmc->tomb, pmc));
+ rcu_assign_pointer(pmc->tomb, dpsf);
}
dpsf->sf_crcount = qrv;
rv++;
@@ -2229,7 +2282,7 @@ static int ip_mc_add_src(struct in_device *in_dev, __be32 *pmca, int sfmode,
#endif
isexclude = pmc->sfmode == MCAST_EXCLUDE;
if (!delta)
- pmc->sfcount[sfmode]++;
+ WRITE_ONCE(pmc->sfcount[sfmode], pmc->sfcount[sfmode] + 1);
err = 0;
for (i = 0; i < sfcount; i++) {
err = ip_mc_add1_src(pmc, sfmode, &psfsrc[i]);
@@ -2240,7 +2293,7 @@ static int ip_mc_add_src(struct in_device *in_dev, __be32 *pmca, int sfmode,
int j;
if (!delta)
- pmc->sfcount[sfmode]--;
+ WRITE_ONCE(pmc->sfcount[sfmode], pmc->sfcount[sfmode] - 1);
for (j = 0; j < i; j++)
(void) ip_mc_del1_src(pmc, sfmode, &psfsrc[j]);
} else if (isexclude != (pmc->sfcount[MCAST_EXCLUDE] != 0)) {
@@ -2260,7 +2313,7 @@ static int ip_mc_add_src(struct in_device *in_dev, __be32 *pmca, int sfmode,
pmc->crcount = in_dev->mr_qrv ?: READ_ONCE(net->ipv4.sysctl_igmp_qrv);
WRITE_ONCE(in_dev->mr_ifc_count, pmc->crcount);
- for (psf = pmc->sources; psf; psf = psf->sf_next)
+ for_each_psf_mclock(pmc, psf)
psf->sf_crcount = 0;
igmp_ifc_event(in_dev);
} else if (sf_setstate(pmc)) {
@@ -2276,13 +2329,13 @@ static void ip_mc_clear_src(struct ip_mc_list *pmc)
struct ip_sf_list *tomb, *sources;
spin_lock_bh(&pmc->lock);
- tomb = pmc->tomb;
- pmc->tomb = NULL;
- sources = pmc->sources;
- pmc->sources = NULL;
+ tomb = pmc_dereference(pmc->tomb, pmc);
+ RCU_INIT_POINTER(pmc->tomb, NULL);
+ sources = pmc_dereference(pmc->sources, pmc);
+ RCU_INIT_POINTER(pmc->sources, NULL);
pmc->sfmode = MCAST_EXCLUDE;
- pmc->sfcount[MCAST_INCLUDE] = 0;
- pmc->sfcount[MCAST_EXCLUDE] = 1;
+ WRITE_ONCE(pmc->sfcount[MCAST_INCLUDE], 0);
+ WRITE_ONCE(pmc->sfcount[MCAST_EXCLUDE], 1);
spin_unlock_bh(&pmc->lock);
ip_sf_list_clear_all(tomb);
@@ -2864,20 +2917,19 @@ int ip_check_mc_rcu(struct in_device *in_dev, __be32 mc_addr, __be32 src_addr, u
rv = 1;
} else if (im) {
if (src_addr) {
- spin_lock_bh(&im->lock);
- for (psf = im->sources; psf; psf = psf->sf_next) {
+ for_each_psf_rcu(im, psf) {
if (psf->sf_inaddr == src_addr)
break;
}
if (psf)
- rv = psf->sf_count[MCAST_INCLUDE] ||
- psf->sf_count[MCAST_EXCLUDE] !=
- im->sfcount[MCAST_EXCLUDE];
+ rv = READ_ONCE(psf->sf_count[MCAST_INCLUDE]) ||
+ READ_ONCE(psf->sf_count[MCAST_EXCLUDE]) !=
+ READ_ONCE(im->sfcount[MCAST_EXCLUDE]);
else
- rv = im->sfcount[MCAST_EXCLUDE] != 0;
- spin_unlock_bh(&im->lock);
- } else
+ rv = READ_ONCE(im->sfcount[MCAST_EXCLUDE]) != 0;
+ } else {
rv = 1; /* unspecified source; tentatively allow */
+ }
}
return rv;
}
@@ -3041,7 +3093,7 @@ static inline struct ip_sf_list *igmp_mcf_get_first(struct seq_file *seq)
im = rcu_dereference(idev->mc_list);
if (likely(im)) {
spin_lock_bh(&im->lock);
- psf = im->sources;
+ psf = pmc_dereference(im->sources, im);
if (likely(psf)) {
state->im = im;
state->idev = idev;
@@ -3057,7 +3109,7 @@ static struct ip_sf_list *igmp_mcf_get_next(struct seq_file *seq, struct ip_sf_l
{
struct igmp_mcf_iter_state *state = igmp_mcf_seq_private(seq);
- psf = psf->sf_next;
+ psf = pmc_dereference(psf->sf_next, state->im);
while (!psf) {
spin_unlock_bh(&state->im->lock);
state->im = state->im->next;
@@ -3073,7 +3125,7 @@ static struct ip_sf_list *igmp_mcf_get_next(struct seq_file *seq, struct ip_sf_l
state->im = rcu_dereference(state->idev->mc_list);
}
spin_lock_bh(&state->im->lock);
- psf = state->im->sources;
+ psf = pmc_dereference(state->im->sources, state->im);
}
out:
return psf;
diff --git a/net/ipv4/inet_fragment.c b/net/ipv4/inet_fragment.c
index c17e57ec7d5c..b286ee429da8 100644
--- a/net/ipv4/inet_fragment.c
+++ b/net/ipv4/inet_fragment.c
@@ -235,6 +235,8 @@ void fqdir_pre_exit(struct fqdir *fqdir)
rhashtable_walk_start(&hti);
while ((fq = rhashtable_walk_next(&hti))) {
+ int refs = 0;
+
if (IS_ERR(fq)) {
if (PTR_ERR(fq) != -EAGAIN)
break;
@@ -242,8 +244,12 @@ void fqdir_pre_exit(struct fqdir *fqdir)
}
spin_lock_bh(&fq->lock);
if (!(fq->flags & INET_FRAG_COMPLETE))
+ inet_frag_kill(fq, &refs);
+
+ if (fq->flags & INET_FRAG_HASH_DEAD)
inet_frag_queue_flush(fq, 0);
spin_unlock_bh(&fq->lock);
+ inet_frag_putn(fq, refs);
}
rhashtable_walk_stop(&hti);
diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c
index 74e095b6b7ca..a24cc8ee11d3 100644
--- a/net/ipv4/ip_output.c
+++ b/net/ipv4/ip_output.c
@@ -1303,6 +1303,7 @@ static int ip_setup_cork(struct sock *sk, struct inet_cork *cork,
cork->fragsize = ip_sk_use_pmtu(sk) ?
dst4_mtu(&rt->dst) : READ_ONCE(rt->dst.dev->mtu);
+ cork->fragsize = min(cork->fragsize, IP_MAX_MTU);
if (!inetdev_valid_mtu(cork->fragsize))
return -ENETUNREACH;
diff --git a/net/ipv4/ip_tunnel_core.c b/net/ipv4/ip_tunnel_core.c
index d3c677e9bff2..5168d546ea2f 100644
--- a/net/ipv4/ip_tunnel_core.c
+++ b/net/ipv4/ip_tunnel_core.c
@@ -268,6 +268,9 @@ static int iptunnel_pmtud_build_icmp(struct sk_buff *skb, int mtu)
eth_header(skb, skb->dev, ntohs(eh.h_proto), eh.h_source, eh.h_dest, 0);
skb_reset_mac_header(skb);
+ if (skb_valid_dst(skb))
+ skb_dst_drop(skb);
+
return skb->len;
}
@@ -371,6 +374,9 @@ static int iptunnel_pmtud_build_icmpv6(struct sk_buff *skb, int mtu)
eth_header(skb, skb->dev, ntohs(eh.h_proto), eh.h_source, eh.h_dest, 0);
skb_reset_mac_header(skb);
+ if (skb_valid_dst(skb))
+ skb_dst_drop(skb);
+
return skb->len;
}
diff --git a/net/ipv4/ipmr.c b/net/ipv4/ipmr.c
index e5f2b1c6150d..b9c544d48c45 100644
--- a/net/ipv4/ipmr.c
+++ b/net/ipv4/ipmr.c
@@ -3376,7 +3376,8 @@ int __init ip_mr_init(void)
{
int err;
- mrt_cachep = KMEM_CACHE(mfc_cache, SLAB_HWCACHE_ALIGN | SLAB_PANIC);
+ mrt_cachep = KMEM_CACHE(mfc_cache,
+ SLAB_HWCACHE_ALIGN | SLAB_PANIC | SLAB_ACCOUNT);
err = register_pernet_subsys(&ipmr_net_ops);
if (err)
diff --git a/net/ipv4/ipmr_base.c b/net/ipv4/ipmr_base.c
index 867b24beded1..a0ec6d19a237 100644
--- a/net/ipv4/ipmr_base.c
+++ b/net/ipv4/ipmr_base.c
@@ -52,7 +52,7 @@ mr_table_alloc(struct net *net, u32 id,
struct mr_table *mrt;
int err;
- mrt = kzalloc_obj(*mrt);
+ mrt = kzalloc_obj(*mrt, GFP_KERNEL_ACCOUNT);
if (!mrt)
return ERR_PTR(-ENOMEM);
mrt->id = id;
diff --git a/net/ipv4/nexthop.c b/net/ipv4/nexthop.c
index 0f1e21a5c812..901c35319682 100644
--- a/net/ipv4/nexthop.c
+++ b/net/ipv4/nexthop.c
@@ -2029,7 +2029,7 @@ remove_nh_grp_entry(struct net *net, struct nh_grp_entry *nhge,
{
struct nh_grp_entry *nhges, *new_nhges;
struct nexthop *nhp = nhge->nh_parent;
- struct netlink_ext_ack extack;
+ struct netlink_ext_ack extack = {};
struct nexthop *nh = nhge->nh;
struct nh_group *nhg, *newg;
int i, j, err;
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 10f58281be17..47a34a993972 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -3181,8 +3181,7 @@ void __tcp_close(struct sock *sk, long timeout)
/* Unread data was tossed, zap the connection. */
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPABORTONCLOSE);
tcp_set_state(sk, TCP_CLOSE);
- tcp_send_active_reset(sk, sk->sk_allocation,
- SK_RST_REASON_TCP_ABORT_ON_CLOSE);
+ tcp_send_active_reset(sk, SK_RST_REASON_TCP_ABORT_ON_CLOSE);
} else if (sock_flag(sk, SOCK_LINGER) && !sk->sk_lingertime) {
/* Check zero linger _after_ checking for unread data. */
sk->sk_prot->disconnect(sk, 0);
@@ -3256,7 +3255,7 @@ void __tcp_close(struct sock *sk, long timeout)
struct tcp_sock *tp = tcp_sk(sk);
if (READ_ONCE(tp->linger2) < 0) {
tcp_set_state(sk, TCP_CLOSE);
- tcp_send_active_reset(sk, GFP_ATOMIC,
+ tcp_send_active_reset(sk,
SK_RST_REASON_TCP_ABORT_ON_LINGER);
__NET_INC_STATS(sock_net(sk),
LINUX_MIB_TCPABORTONLINGER);
@@ -3275,7 +3274,7 @@ void __tcp_close(struct sock *sk, long timeout)
if (sk->sk_state != TCP_CLOSE) {
if (tcp_check_oom(sk, 0)) {
tcp_set_state(sk, TCP_CLOSE);
- tcp_send_active_reset(sk, GFP_ATOMIC,
+ tcp_send_active_reset(sk,
SK_RST_REASON_TCP_ABORT_ON_MEMORY);
__NET_INC_STATS(sock_net(sk),
LINUX_MIB_TCPABORTONMEMORY);
@@ -3376,14 +3375,14 @@ int tcp_disconnect(struct sock *sk, int flags)
} else if (unlikely(tp->repair)) {
WRITE_ONCE(sk->sk_err, ECONNABORTED);
} else if (tcp_need_reset(old_state)) {
- tcp_send_active_reset(sk, gfp_any(), SK_RST_REASON_TCP_STATE);
+ tcp_send_active_reset(sk, SK_RST_REASON_TCP_STATE);
WRITE_ONCE(sk->sk_err, ECONNRESET);
} else if (tp->snd_nxt != tp->write_seq &&
(1 << old_state) & (TCPF_CLOSING | TCPF_LAST_ACK)) {
/* The last check adjusts for discrepancy of Linux wrt. RFC
* states
*/
- tcp_send_active_reset(sk, gfp_any(),
+ tcp_send_active_reset(sk,
SK_RST_REASON_TCP_DISCONNECT_WITH_DATA);
WRITE_ONCE(sk->sk_err, ECONNRESET);
} else if (old_state == TCP_SYN_SENT)
@@ -5146,8 +5145,7 @@ int tcp_abort(struct sock *sk, int err)
bh_lock_sock(sk);
if (tcp_need_reset(sk->sk_state))
- tcp_send_active_reset(sk, GFP_ATOMIC,
- SK_RST_REASON_TCP_STATE);
+ tcp_send_active_reset(sk, SK_RST_REASON_TCP_STATE);
tcp_done_with_error(sk, err);
bh_unlock_sock(sk);
diff --git a/net/ipv4/tcp_offload.c b/net/ipv4/tcp_offload.c
index 3b1fdcd3cb29..e74d99ca9fac 100644
--- a/net/ipv4/tcp_offload.c
+++ b/net/ipv4/tcp_offload.c
@@ -332,6 +332,7 @@ struct sk_buff *tcp_gro_receive(struct list_head *head, struct sk_buff *skb,
flush |= skb->ip_summed != p->ip_summed;
flush |= skb->csum_level != p->csum_level;
flush |= NAPI_GRO_CB(p)->count >= 64;
+ flush |= NAPI_GRO_CB(p)->is_flist != NAPI_GRO_CB(skb)->is_flist;
skb_set_network_header(skb, skb_gro_receive_network_offset(skb));
if (flush || skb_gro_receive_list(p, skb))
@@ -395,12 +396,20 @@ static void tcp4_check_fraglist_gro(struct list_head *head, struct sk_buff *skb,
struct net *net;
int iif, sdif;
- if (likely(!(skb->dev->features & NETIF_F_GRO_FRAGLIST)))
- return;
-
p = tcp_gro_lookup(head, th);
if (p) {
- NAPI_GRO_CB(skb)->is_flist = NAPI_GRO_CB(p)->is_flist;
+ /* flist GRO applies to consecutive non-GSO skbs */
+ if (!skb_is_gso(skb) || !NAPI_GRO_CB(p)->is_flist) {
+ NAPI_GRO_CB(skb)->is_flist = NAPI_GRO_CB(p)->is_flist;
+ return;
+ }
+
+ /* Fall back to the regular GRO path */
+ if (NAPI_GRO_CB(p)->count == 1)
+ NAPI_GRO_CB(p)->is_flist = 0;
+
+ NAPI_GRO_CB(skb)->is_flist = 0;
+
return;
}
@@ -410,7 +419,7 @@ static void tcp4_check_fraglist_gro(struct list_head *head, struct sk_buff *skb,
sk = __inet_lookup_established(net, iph->saddr, th->source,
iph->daddr, ntohs(th->dest),
iif, sdif);
- NAPI_GRO_CB(skb)->is_flist = !sk;
+ NAPI_GRO_CB(skb)->is_flist = !sk && !skb_is_gso(skb);
if (sk)
sock_gen_put(sk);
}
@@ -430,7 +439,8 @@ struct sk_buff *tcp4_gro_receive(struct list_head *head, struct sk_buff *skb)
if (!th)
goto flush;
- tcp4_check_fraglist_gro(head, skb, th);
+ if (unlikely(skb->dev->features & NETIF_F_GRO_FRAGLIST))
+ tcp4_check_fraglist_gro(head, skb, th);
return tcp_gro_receive(head, skb, th);
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 8d243e91761a..f72a6b1fe749 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -3849,9 +3849,9 @@ void tcp_send_fin(struct sock *sk)
* was unread data in the receive queue. This behavior is recommended
* by RFC 2525, section 2.17. -DaveM
*/
-void tcp_send_active_reset(struct sock *sk, gfp_t priority,
- enum sk_rst_reason reason)
+void tcp_send_active_reset(struct sock *sk, enum sk_rst_reason reason)
{
+ gfp_t priority = sk_gfp_mask(sk, GFP_ATOMIC | __GFP_NOWARN);
struct sk_buff *skb;
TCP_INC_STATS(sock_net(sk), TCP_MIB_OUTRSTS);
diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c
index 4df1c9745c5d..95296b1cf0ca 100644
--- a/net/ipv4/tcp_timer.c
+++ b/net/ipv4/tcp_timer.c
@@ -126,7 +126,7 @@ static int tcp_out_of_resources(struct sock *sk, bool do_reset)
(!tp->snd_wnd && !tp->packets_out))
do_reset = true;
if (do_reset)
- tcp_send_active_reset(sk, GFP_ATOMIC,
+ tcp_send_active_reset(sk,
SK_RST_REASON_TCP_ABORT_ON_MEMORY);
tcp_done(sk);
__NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPABORTONMEMORY);
@@ -809,7 +809,7 @@ static void tcp_keepalive_timer(struct timer_list *t)
goto out;
}
}
- tcp_send_active_reset(sk, GFP_ATOMIC, SK_RST_REASON_TCP_STATE);
+ tcp_send_active_reset(sk, SK_RST_REASON_TCP_STATE);
goto death;
}
@@ -836,7 +836,7 @@ static void tcp_keepalive_timer(struct timer_list *t)
icsk->icsk_probes_out > 0) ||
(user_timeout == 0 &&
icsk->icsk_probes_out >= keepalive_probes(tp))) {
- tcp_send_active_reset(sk, GFP_ATOMIC,
+ tcp_send_active_reset(sk,
SK_RST_REASON_TCP_KEEPALIVE_TIMEOUT);
tcp_write_err(sk);
goto out;
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 70f6cbd4ef73..45e96b721989 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -2164,10 +2164,10 @@ int __udp_disconnect(struct sock *sk, int flags)
*/
sk->sk_state = TCP_CLOSE;
- inet->inet_daddr = 0;
+ WRITE_ONCE(inet->inet_daddr, 0);
inet->inet_dport = 0;
sock_rps_reset_rxhash(sk);
- sk->sk_bound_dev_if = 0;
+ WRITE_ONCE(sk->sk_bound_dev_if, 0);
if (!(sk->sk_userlocks & SOCK_BINDADDR_LOCK)) {
inet_reset_saddr(sk);
if (sk->sk_prot->rehash &&
diff --git a/net/ipv6/exthdrs.c b/net/ipv6/exthdrs.c
index 51941ad656a3..09a4552f7f08 100644
--- a/net/ipv6/exthdrs.c
+++ b/net/ipv6/exthdrs.c
@@ -445,7 +445,7 @@ static int ipv6_srh_rcv(struct sk_buff *skb, struct inet6_dev *idev)
hdr->segments_left--;
addr = hdr->segments + hdr->segments_left;
- skb_push(skb, sizeof(struct ipv6hdr));
+ skb_push(skb, -skb_network_offset(skb));
if (skb->ip_summed == CHECKSUM_COMPLETE)
seg6_update_csum(skb);
@@ -469,7 +469,7 @@ static int ipv6_srh_rcv(struct sk_buff *skb, struct inet6_dev *idev)
}
ipv6_hdr(skb)->hop_limit--;
- skb_pull(skb, sizeof(struct ipv6hdr));
+ skb_pull(skb, skb_transport_offset(skb));
goto looped_back;
}
diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
index e9fc692d4f3b..7c5daea3f096 100644
--- a/net/ipv6/ip6_fib.c
+++ b/net/ipv6/ip6_fib.c
@@ -85,7 +85,7 @@ static void fib6_walker_link(struct net *net, struct fib6_walker *w)
static void fib6_walker_unlink(struct net *net, struct fib6_walker *w)
{
write_lock_bh(&net->ipv6.fib6_walker_lock);
- list_del(&w->lh);
+ list_del_init(&w->lh);
write_unlock_bh(&net->ipv6.fib6_walker_lock);
}
@@ -2758,7 +2758,7 @@ static void *ipv6_route_seq_start(struct seq_file *seq, loff_t *pos)
static bool ipv6_route_iter_active(struct ipv6_route_iter *iter)
{
struct fib6_walker *w = &iter->w;
- return w->node && !(w->state == FWS_U && w->node == w->root);
+ return w->node && !list_empty(&w->lh);
}
static void ipv6_route_native_seq_stop(struct seq_file *seq, void *v)
diff --git a/net/ipv6/ip6_flowlabel.c b/net/ipv6/ip6_flowlabel.c
index 1ab5ad0dcf24..006585dc8b5c 100644
--- a/net/ipv6/ip6_flowlabel.c
+++ b/net/ipv6/ip6_flowlabel.c
@@ -461,6 +461,21 @@ fl_create(struct net *net, struct sock *sk, struct in6_flowlabel_req *freq,
return NULL;
}
+static bool fl_sock_at_lease_limit(const struct sock *sk)
+{
+ const struct ipv6_fl_socklist *sfl;
+ int count = 0;
+
+ rcu_read_lock();
+ for_each_sk_fl_rcu(sk, sfl) {
+ if (++count >= FL_MAX_PER_SOCK)
+ break;
+ }
+ rcu_read_unlock();
+
+ return count >= FL_MAX_PER_SOCK;
+}
+
static int mem_check(struct sock *sk)
{
const int unpriv_total_limit = FL_MAX_SIZE - (FL_MAX_SIZE / 4);
@@ -679,6 +694,10 @@ static int ipv6_flowlabel_get(struct sock *sk, struct in6_flowlabel_req *freq,
err = -ENOMEM;
if (!sfl1)
goto release;
+ err = -ENOBUFS;
+ if (fl_sock_at_lease_limit(sk) &&
+ !capable(CAP_NET_ADMIN))
+ goto release;
if (fl->linger > fl1->linger)
fl1->linger = fl->linger;
if ((long)(fl->expires - fl1->expires) > 0)
diff --git a/net/ipv6/ip6_gre.c b/net/ipv6/ip6_gre.c
index 200d0ba1a40e..678678fcb5da 100644
--- a/net/ipv6/ip6_gre.c
+++ b/net/ipv6/ip6_gre.c
@@ -878,6 +878,7 @@ static int ip6gre_xmit_other(struct sk_buff *skb, struct net_device *dev)
static netdev_tx_t ip6gre_tunnel_xmit(struct sk_buff *skb,
struct net_device *dev)
{
+ struct ip_tunnel_info *tun_info = NULL;
struct ip6_tnl *t = netdev_priv(dev);
__be16 payload_protocol;
int ret;
@@ -888,6 +889,9 @@ static netdev_tx_t ip6gre_tunnel_xmit(struct sk_buff *skb,
if (!ip6_tnl_xmit_ctl(t, &t->parms.laddr, &t->parms.raddr))
goto tx_err;
+ if (t->parms.collect_md)
+ tun_info = skb_tunnel_info_txcheck(skb);
+
payload_protocol = skb_protocol(skb, true);
switch (payload_protocol) {
case htons(ETH_P_IP):
@@ -907,7 +911,7 @@ static netdev_tx_t ip6gre_tunnel_xmit(struct sk_buff *skb,
return NETDEV_TX_OK;
tx_err:
- if (!t->parms.collect_md || !IS_ERR(skb_tunnel_info_txcheck(skb)))
+ if (!IS_ERR(tun_info))
DEV_STATS_INC(dev, tx_errors);
DEV_STATS_INC(dev, tx_dropped);
kfree_skb(skb);
diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
index 8fc4766c8da9..550965058991 100644
--- a/net/ipv6/ip6_output.c
+++ b/net/ipv6/ip6_output.c
@@ -1432,6 +1432,8 @@ static int ip6_setup_cork(struct sock *sk, struct inet_cork_full *cork,
if (frag_size && frag_size < mtu)
mtu = frag_size;
+ if (sk_is_udp(sk))
+ mtu = min(mtu, IP6_MAX_MTU);
cork->base.fragsize = mtu;
cork->base.gso_size = ipc6->gso_size;
cork->base.tx_flags = 0;
diff --git a/net/ipv6/ip6mr.c b/net/ipv6/ip6mr.c
index 3f2ed9b77deb..9d8116b5edb1 100644
--- a/net/ipv6/ip6mr.c
+++ b/net/ipv6/ip6mr.c
@@ -1427,7 +1427,7 @@ int __init ip6_mr_init(void)
{
int err;
- mrt_cachep = KMEM_CACHE(mfc6_cache, SLAB_HWCACHE_ALIGN);
+ mrt_cachep = KMEM_CACHE(mfc6_cache, SLAB_HWCACHE_ALIGN | SLAB_ACCOUNT);
if (!mrt_cachep)
return -ENOMEM;
diff --git a/net/ipv6/mcast.c b/net/ipv6/mcast.c
index 4d2b9377ba2d..fad53a5cdfff 100644
--- a/net/ipv6/mcast.c
+++ b/net/ipv6/mcast.c
@@ -240,7 +240,8 @@ static int __ipv6_sock_mc_join(struct sock *sk, int ifindex,
return err;
}
- mc_lst->next = np->ipv6_mc_list;
+ rcu_assign_pointer(mc_lst->next,
+ sock_dereference(np->ipv6_mc_list, sk));
rcu_assign_pointer(np->ipv6_mc_list, mc_lst);
return 0;
@@ -300,7 +301,8 @@ int ipv6_sock_mc_drop(struct sock *sk, int ifindex, const struct in6_addr *addr)
lnk = &mc_lst->next) {
if ((ifindex == 0 || mc_lst->ifindex == ifindex) &&
ipv6_addr_equal(&mc_lst->addr, addr)) {
- *lnk = mc_lst->next;
+ rcu_assign_pointer(*lnk,
+ sock_dereference(mc_lst->next, sk));
__ipv6_sock_mc_drop(sk, mc_lst);
return 0;
}
@@ -333,7 +335,8 @@ void __ipv6_sock_mc_close(struct sock *sk)
struct ipv6_mc_socklist *mc_lst;
while ((mc_lst = sock_dereference(np->ipv6_mc_list, sk)) != NULL) {
- np->ipv6_mc_list = mc_lst->next;
+ rcu_assign_pointer(np->ipv6_mc_list,
+ sock_dereference(mc_lst->next, sk));
__ipv6_sock_mc_drop(sk, mc_lst);
}
}
@@ -355,12 +358,12 @@ int ip6_mc_source(int add, int omode, struct sock *sk,
{
struct ipv6_pinfo *inet6 = inet6_sk(sk);
struct in6_addr *source, *group;
+ struct ip6_sf_socklist *newpsl, *psl;
struct net *net = sock_net(sk);
struct ipv6_mc_socklist *pmc;
- struct ip6_sf_socklist *psl;
struct inet6_dev *idev;
int leavegroup = 0;
- int i, j, rv;
+ int i, j;
int err;
source = &((struct sockaddr_in6 *)&pgsr->gsr_source)->sin6_addr;
@@ -409,13 +412,11 @@ int ip6_mc_source(int add, int omode, struct sock *sk,
if (!add) {
if (!psl)
goto done; /* err = -EADDRNOTAVAIL */
- rv = !0;
for (i = 0; i < psl->sl_count; i++) {
- rv = !ipv6_addr_equal(&psl->sl_addr[i], source);
- if (rv == 0)
+ if (ipv6_addr_equal(&psl->sl_addr[i], source))
break;
}
- if (rv) /* source not found */
+ if (i == psl->sl_count) /* source not found */
goto done; /* err = -EADDRNOTAVAIL */
/* special case - (INCLUDE, empty) == LEAVE_GROUP */
@@ -424,58 +425,74 @@ int ip6_mc_source(int add, int omode, struct sock *sk,
goto done;
}
+ atomic_sub(struct_size(psl, sl_addr, psl->sl_max),
+ &sk->sk_omem_alloc);
+
+ if (psl->sl_count == 1) {
+ newpsl = NULL;
+ } else {
+ newpsl = sock_kmalloc(sk, struct_size(newpsl, sl_addr,
+ psl->sl_count - 1),
+ GFP_KERNEL);
+ if (!newpsl) {
+ atomic_add(struct_size(psl, sl_addr, psl->sl_max),
+ &sk->sk_omem_alloc);
+ err = -ENOBUFS;
+ goto done;
+ }
+ newpsl->sl_max = psl->sl_count - 1;
+ newpsl->sl_count = psl->sl_count - 1;
+ for (j = 0; j < i; j++)
+ newpsl->sl_addr[j] = psl->sl_addr[j];
+ for (j = i + 1; j < psl->sl_count; j++)
+ newpsl->sl_addr[j - 1] = psl->sl_addr[j];
+ }
+
/* update the interface filter */
ip6_mc_del_src(idev, group, omode, 1, source, 1);
- for (j = i+1; j < psl->sl_count; j++)
- psl->sl_addr[j-1] = psl->sl_addr[j];
- psl->sl_count--;
+ rcu_assign_pointer(pmc->sflist, newpsl);
+ kfree_rcu(psl, rcu);
err = 0;
goto done;
}
/* else, add a new source to the filter */
- if (psl && psl->sl_count >= sysctl_mld_max_msf) {
+ if (psl && psl->sl_count >= READ_ONCE(sysctl_mld_max_msf)) {
err = -ENOBUFS;
goto done;
}
- if (!psl || psl->sl_count == psl->sl_max) {
- struct ip6_sf_socklist *newpsl;
- int count = IP6_SFBLOCK;
-
- if (psl)
- count += psl->sl_max;
- newpsl = sock_kmalloc(sk, struct_size(newpsl, sl_addr, count),
- GFP_KERNEL);
- if (!newpsl) {
- err = -ENOBUFS;
- goto done;
- }
- newpsl->sl_max = count;
- newpsl->sl_count = count - IP6_SFBLOCK;
- if (psl) {
- for (i = 0; i < psl->sl_count; i++)
- newpsl->sl_addr[i] = psl->sl_addr[i];
- atomic_sub(struct_size(psl, sl_addr, psl->sl_max),
- &sk->sk_omem_alloc);
+ if (psl) {
+ for (i = 0; i < psl->sl_count; i++) {
+ if (ipv6_addr_equal(&psl->sl_addr[i], source))
+ goto done; /* err = -EADDRNOTAVAIL */
}
- rcu_assign_pointer(pmc->sflist, newpsl);
- kfree_rcu(psl, rcu);
- psl = newpsl;
}
- rv = 1; /* > 0 for insert logic below if sl_count is 0 */
- for (i = 0; i < psl->sl_count; i++) {
- rv = !ipv6_addr_equal(&psl->sl_addr[i], source);
- if (rv == 0) /* There is an error in the address. */
- goto done;
+
+ i = psl ? psl->sl_count + 1 : 1;
+ newpsl = sock_kmalloc(sk, struct_size(newpsl, sl_addr, i),
+ GFP_KERNEL);
+ if (!newpsl) {
+ err = -ENOBUFS;
+ goto done;
}
- for (j = psl->sl_count-1; j >= i; j--)
- psl->sl_addr[j+1] = psl->sl_addr[j];
- psl->sl_addr[i] = *source;
- psl->sl_count++;
- err = 0;
+ newpsl->sl_max = i;
+ newpsl->sl_count = i;
+ if (psl) {
+ for (j = 0; j < psl->sl_count; j++)
+ newpsl->sl_addr[j] = psl->sl_addr[j];
+ }
+ newpsl->sl_addr[i - 1] = *source;
+
/* update the interface list */
ip6_mc_add_src(idev, group, omode, 1, source, 1);
+
+ if (psl)
+ atomic_sub(struct_size(psl, sl_addr, psl->sl_max),
+ &sk->sk_omem_alloc);
+ rcu_assign_pointer(pmc->sflist, newpsl);
+ kfree_rcu(psl, rcu);
+ err = 0;
done:
mutex_unlock(&idev->mc_lock);
in6_dev_put(idev);
@@ -784,9 +801,11 @@ static void mld_del_delrec(struct inet6_dev *idev, struct ifmcaddr6 *im)
if (!pmc)
return;
if (pmc_prev)
- rcu_assign_pointer(pmc_prev->next, pmc->next);
+ rcu_assign_pointer(pmc_prev->next,
+ mc_dereference(pmc->next, idev));
else
- rcu_assign_pointer(idev->mc_tomb, pmc->next);
+ rcu_assign_pointer(idev->mc_tomb,
+ mc_dereference(pmc->next, idev));
im->idev = pmc->idev;
if (im->mca_sfmode == MCAST_INCLUDE) {
@@ -965,7 +984,7 @@ static int __ipv6_dev_mc_inc(struct net_device *dev,
return -ENOMEM;
}
- rcu_assign_pointer(mc->next, idev->mc_list);
+ rcu_assign_pointer(mc->next, mc_dereference(idev->mc_list, idev));
rcu_assign_pointer(idev->mc_list, mc);
mld_del_delrec(idev, mc);
@@ -999,7 +1018,8 @@ int __ipv6_dev_mc_dec(struct inet6_dev *idev, const struct in6_addr *addr)
WRITE_ONCE(ma->mca_users, new_users);
if (new_users == 0) {
- *map = ma->next;
+ rcu_assign_pointer(*map,
+ mc_dereference(ma->next, idev));
igmp6_group_dropped(ma);
inet6_ifmcaddr_notify(idev->dev, ma,
@@ -2350,14 +2370,18 @@ static int ip6_mc_del1_src(struct ifmcaddr6 *pmc, int sfmode,
if (psf->sf_oldin && !(pmc->mca_flags & MAF_NOREPORT) &&
!mld_in_v1_mode(idev)) {
- psf->sf_crcount = idev->mc_qrv;
- rcu_assign_pointer(psf->sf_next,
- mc_dereference(pmc->mca_tomb, idev));
- rcu_assign_pointer(pmc->mca_tomb, psf);
- rv = 1;
- } else {
- kfree_rcu(psf, rcu);
+ struct ip6_sf_list *dpsf = kmalloc_obj(*dpsf);
+
+ if (dpsf) {
+ *dpsf = *psf;
+ dpsf->sf_crcount = idev->mc_qrv;
+ rcu_assign_pointer(dpsf->sf_next,
+ mc_dereference(pmc->mca_tomb, idev));
+ rcu_assign_pointer(pmc->mca_tomb, dpsf);
+ rv = 1;
+ }
}
+ kfree_rcu(psf, rcu);
}
return rv;
}
@@ -2620,7 +2644,7 @@ static void ip6_mc_clear_src(struct ifmcaddr6 *pmc)
static void igmp6_join_group(struct ifmcaddr6 *ma)
{
- unsigned long delay;
+ unsigned long delay, interval;
mc_assert_locked(ma->idev);
@@ -2629,13 +2653,17 @@ static void igmp6_join_group(struct ifmcaddr6 *ma)
igmp6_send(&ma->mca_addr, ma->idev->dev, ICMPV6_MGM_REPORT);
- delay = get_random_u32_below(unsolicited_report_interval(ma->idev));
+ interval = unsolicited_report_interval(ma->idev);
+ delay = interval;
if (cancel_delayed_work(&ma->mca_work)) {
refcount_dec(&ma->mca_refcnt);
delay = ma->mca_work.timer.expires - jiffies;
}
+ if (delay >= interval)
+ delay = get_random_u32_below(interval);
+
if (!mod_delayed_work(mld_wq, &ma->mca_work, delay))
refcount_inc(&ma->mca_refcnt);
WRITE_ONCE(ma->mca_flags, ma->mca_flags |
@@ -3000,7 +3028,7 @@ static int igmp6_mc_seq_show(struct seq_file *seq, void *v)
struct ifmcaddr6 *im = (struct ifmcaddr6 *)v;
struct igmp6_mc_iter_state *state = igmp6_mc_seq_private(seq);
unsigned int mca_flags = READ_ONCE(im->mca_flags);
- unsigned long expires = READ_ONCE(im->mca_work.timer.expires);
+ long delta = READ_ONCE(im->mca_work.timer.expires) - jiffies;
seq_printf(seq,
"%-4d %-15s %pi6 %5d %08X %ld\n",
@@ -3008,7 +3036,7 @@ static int igmp6_mc_seq_show(struct seq_file *seq, void *v)
&im->mca_addr,
READ_ONCE(im->mca_users), mca_flags,
(mca_flags & MAF_TIMER_RUNNING) ?
- jiffies_to_clock_t(expires - jiffies) : 0);
+ jiffies_delta_to_clock_t(delta) : 0);
return 0;
}
diff --git a/net/ipv6/netfilter/ip6_tables.c b/net/ipv6/netfilter/ip6_tables.c
index f42fb96ef64b..313c4aac377a 100644
--- a/net/ipv6/netfilter/ip6_tables.c
+++ b/net/ipv6/netfilter/ip6_tables.c
@@ -647,6 +647,11 @@ check_entry_size_and_hooks(struct ip6t_entry *e,
/* Clear counters and comefrom */
e->counters = ((struct xt_counters) { 0, 0 });
e->comefrom = 0;
+
+ /* set F_PROTO, else ip6_packet_match won't do the right thing. */
+ if (e->ipv6.proto)
+ e->ipv6.flags |= IP6T_F_PROTO;
+
return 0;
}
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index dc9060af5bb1..ee707e48f4ef 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -4010,6 +4010,7 @@ static int __ip6_del_rt_siblings(struct fib6_info *rt, struct fib6_config *cfg)
struct net *net = info->nl_net;
struct sk_buff *skb = NULL;
struct fib6_table *table;
+ struct fib6_node *fn;
int err = -ENOENT;
if (rt == net->ipv6.fib6_null_entry)
@@ -4017,9 +4018,13 @@ static int __ip6_del_rt_siblings(struct fib6_info *rt, struct fib6_config *cfg)
table = rt->fib6_table;
spin_lock_bh(&table->tb6_lock);
+ fn = rcu_dereference_protected(rt->fib6_node,
+ lockdep_is_held(&table->tb6_lock));
+ if (!fn)
+ goto out_unlock;
+
if (rt->fib6_nsiblings && cfg->fc_delete_all_nh) {
struct fib6_info *sibling, *next_sibling;
- struct fib6_node *fn;
/* prefer to send a single notification with all hops */
skb = nlmsg_new(rt6_nlmsg_size(rt), GFP_ATOMIC);
@@ -4041,8 +4046,6 @@ static int __ip6_del_rt_siblings(struct fib6_info *rt, struct fib6_config *cfg)
* and emit a replace or delete notification, respectively.
*/
info->skip_notify_kernel = 1;
- fn = rcu_dereference_protected(rt->fib6_node,
- lockdep_is_held(&table->tb6_lock));
if (rcu_access_pointer(fn->leaf) == rt) {
struct fib6_info *last_sibling, *replace_rt;
diff --git a/net/ipv6/tcpv6_offload.c b/net/ipv6/tcpv6_offload.c
index f2a659cd6183..eec3778855eb 100644
--- a/net/ipv6/tcpv6_offload.c
+++ b/net/ipv6/tcpv6_offload.c
@@ -26,7 +26,18 @@ static void tcp6_check_fraglist_gro(struct list_head *head, struct sk_buff *skb,
p = tcp_gro_lookup(head, th);
if (p) {
- NAPI_GRO_CB(skb)->is_flist = NAPI_GRO_CB(p)->is_flist;
+ /* flist GRO applies to consecutive non-GSO skbs */
+ if (!skb_is_gso(skb) || !NAPI_GRO_CB(p)->is_flist) {
+ NAPI_GRO_CB(skb)->is_flist = NAPI_GRO_CB(p)->is_flist;
+ return;
+ }
+
+ /* Fall back to the regular GRO path */
+ if (NAPI_GRO_CB(p)->count == 1)
+ NAPI_GRO_CB(p)->is_flist = 0;
+
+ NAPI_GRO_CB(skb)->is_flist = 0;
+
return;
}
@@ -36,7 +47,7 @@ static void tcp6_check_fraglist_gro(struct list_head *head, struct sk_buff *skb,
sk = __inet6_lookup_established(net, &hdr->saddr, th->source,
&hdr->daddr, ntohs(th->dest),
iif, sdif);
- NAPI_GRO_CB(skb)->is_flist = !sk;
+ NAPI_GRO_CB(skb)->is_flist = !sk && !skb_is_gso(skb);
if (sk)
sock_gen_put(sk);
#endif /* IS_ENABLED(CONFIG_IPV6) */
diff --git a/net/mac802154/ieee802154_i.h b/net/mac802154/ieee802154_i.h
index 8f2bff268392..279ff4e9b11e 100644
--- a/net/mac802154/ieee802154_i.h
+++ b/net/mac802154/ieee802154_i.h
@@ -74,6 +74,10 @@ struct ieee802154_local {
struct work_struct rx_beacon_work;
struct list_head rx_mac_cmd_list;
struct work_struct rx_mac_cmd_work;
+ /* Serializes rx_beacon_list and rx_mac_cmd_list against the RX
+ * softirq producer, the mac_wq workers and the teardown flush.
+ */
+ spinlock_t rx_lock;
/* Association */
struct ieee802154_pan_device *assoc_dev;
@@ -300,6 +304,10 @@ static inline bool mac802154_is_beaconing(struct ieee802154_local *local)
}
void mac802154_rx_mac_cmd_worker(struct work_struct *work);
+void mac802154_flush_list(struct list_head *list,
+ struct ieee802154_sub_if_data *sdata);
+void mac802154_flush_queued_pkts(struct ieee802154_local *local,
+ struct ieee802154_sub_if_data *sdata);
int mac802154_perform_association(struct ieee802154_sub_if_data *sdata,
struct ieee802154_pan_device *coord,
diff --git a/net/mac802154/iface.c b/net/mac802154/iface.c
index b823720630e7..31353795fa24 100644
--- a/net/mac802154/iface.c
+++ b/net/mac802154/iface.c
@@ -694,6 +694,7 @@ void ieee802154_if_remove(struct ieee802154_sub_if_data *sdata)
mutex_unlock(&sdata->local->iflist_mtx);
synchronize_rcu();
+ mac802154_flush_queued_pkts(sdata->local, sdata);
unregister_netdevice(sdata->dev);
}
@@ -705,6 +706,11 @@ void ieee802154_remove_interfaces(struct ieee802154_local *local)
list_for_each_entry_safe(sdata, tmp, &local->interfaces, list) {
list_del_rcu(&sdata->list);
+ /* Best-effort: a frame the RX softirq queues for this sdata
+ * after the flush still pins the netdev, so the
+ * unregister_netdevice() below waits it out.
+ */
+ mac802154_flush_queued_pkts(local, sdata);
unregister_netdevice(sdata->dev);
}
mutex_unlock(&local->iflist_mtx);
diff --git a/net/mac802154/main.c b/net/mac802154/main.c
index ea1efef3572a..386c086c79d1 100644
--- a/net/mac802154/main.c
+++ b/net/mac802154/main.c
@@ -91,6 +91,7 @@ ieee802154_alloc_hw(size_t priv_data_len, const struct ieee802154_ops *ops)
INIT_LIST_HEAD(&local->interfaces);
INIT_LIST_HEAD(&local->rx_beacon_list);
INIT_LIST_HEAD(&local->rx_mac_cmd_list);
+ spin_lock_init(&local->rx_lock);
mutex_init(&local->iflist_mtx);
tasklet_setup(&local->tasklet, ieee802154_tasklet_handler);
diff --git a/net/mac802154/rx.c b/net/mac802154/rx.c
index cd8f2a11920d..19b5382e85a8 100644
--- a/net/mac802154/rx.c
+++ b/net/mac802154/rx.c
@@ -35,16 +35,23 @@ void mac802154_rx_beacon_worker(struct work_struct *work)
container_of(work, struct ieee802154_local, rx_beacon_work);
struct cfg802154_mac_pkt *mac_pkt;
- mac_pkt = list_first_entry_or_null(&local->rx_beacon_list,
- struct cfg802154_mac_pkt, node);
- if (!mac_pkt)
- return;
+ for (;;) {
+ spin_lock_bh(&local->rx_lock);
+ mac_pkt = list_first_entry_or_null(&local->rx_beacon_list,
+ struct cfg802154_mac_pkt, node);
+ if (mac_pkt)
+ list_del(&mac_pkt->node);
+ spin_unlock_bh(&local->rx_lock);
+ if (!mac_pkt)
+ break;
- mac802154_process_beacon(local, mac_pkt->skb, mac_pkt->page, mac_pkt->channel);
+ mac802154_process_beacon(local, mac_pkt->skb,
+ mac_pkt->page, mac_pkt->channel);
- list_del(&mac_pkt->node);
- kfree_skb(mac_pkt->skb);
- kfree(mac_pkt);
+ netdev_put(mac_pkt->sdata->dev, &mac_pkt->dev_tracker);
+ kfree_skb(mac_pkt->skb);
+ kfree(mac_pkt);
+ }
}
static bool mac802154_should_answer_beacon_req(struct ieee802154_local *local)
@@ -68,22 +75,15 @@ static bool mac802154_should_answer_beacon_req(struct ieee802154_local *local)
return interval == IEEE802154_ACTIVE_SCAN_DURATION;
}
-void mac802154_rx_mac_cmd_worker(struct work_struct *work)
+static void mac802154_rx_mac_cmd(struct ieee802154_local *local,
+ struct cfg802154_mac_pkt *mac_pkt)
{
- struct ieee802154_local *local =
- container_of(work, struct ieee802154_local, rx_mac_cmd_work);
- struct cfg802154_mac_pkt *mac_pkt;
u8 mac_cmd;
int rc;
- mac_pkt = list_first_entry_or_null(&local->rx_mac_cmd_list,
- struct cfg802154_mac_pkt, node);
- if (!mac_pkt)
- return;
-
rc = ieee802154_get_mac_cmd(mac_pkt->skb, &mac_cmd);
if (rc)
- goto out;
+ return;
switch (mac_cmd) {
case IEEE802154_CMD_BEACON_REQ:
@@ -121,11 +121,81 @@ void mac802154_rx_mac_cmd_worker(struct work_struct *work)
default:
break;
}
+}
-out:
- list_del(&mac_pkt->node);
- kfree_skb(mac_pkt->skb);
- kfree(mac_pkt);
+void mac802154_rx_mac_cmd_worker(struct work_struct *work)
+{
+ struct ieee802154_local *local =
+ container_of(work, struct ieee802154_local, rx_mac_cmd_work);
+ struct cfg802154_mac_pkt *mac_pkt;
+
+ for (;;) {
+ spin_lock_bh(&local->rx_lock);
+ mac_pkt = list_first_entry_or_null(&local->rx_mac_cmd_list,
+ struct cfg802154_mac_pkt, node);
+ if (mac_pkt)
+ list_del(&mac_pkt->node);
+ spin_unlock_bh(&local->rx_lock);
+ if (!mac_pkt)
+ break;
+
+ /* A stopped interface cannot transmit; skipping avoids a
+ * needless association response (and the !netif_running()
+ * warning it would trip) during teardown. The beacon worker
+ * needs no such check as it never transmits.
+ */
+ if (ieee802154_sdata_running(mac_pkt->sdata))
+ mac802154_rx_mac_cmd(local, mac_pkt);
+
+ netdev_put(mac_pkt->sdata->dev, &mac_pkt->dev_tracker);
+ kfree_skb(mac_pkt->skb);
+ kfree(mac_pkt);
+ }
+}
+
+/**
+ * mac802154_flush_list - free queued RX frames on @list
+ * @list: rx_beacon_list or rx_mac_cmd_list
+ * @sdata: only free frames received on this interface, or %NULL for all
+ *
+ * Each frame pins the net_device it was received on (via netdev_hold()),
+ * so release that reference as the frame is dropped. Caller must hold
+ * local->rx_lock.
+ */
+void mac802154_flush_list(struct list_head *list,
+ struct ieee802154_sub_if_data *sdata)
+{
+ struct cfg802154_mac_pkt *mac_pkt, *tmp;
+
+ list_for_each_entry_safe(mac_pkt, tmp, list, node) {
+ if (sdata && mac_pkt->sdata != sdata)
+ continue;
+ list_del(&mac_pkt->node);
+ netdev_put(mac_pkt->sdata->dev, &mac_pkt->dev_tracker);
+ kfree_skb(mac_pkt->skb);
+ kfree(mac_pkt);
+ }
+}
+
+/**
+ * mac802154_flush_queued_pkts - drop queued RX work referencing @sdata
+ * @local: the mac802154 device
+ * @sdata: interface being removed
+ *
+ * The workers dereference the queued frame's interface directly
+ * (mac_pkt->sdata) or through skb->dev in mac802154_process_beacon(). Drop
+ * the not-yet-started entries belonging to @sdata before it is unregistered
+ * so their netdev reference is released; an entry already dequeued by a
+ * running worker keeps its own reference until the worker completes, which
+ * unregister_netdevice() then waits out.
+ */
+void mac802154_flush_queued_pkts(struct ieee802154_local *local,
+ struct ieee802154_sub_if_data *sdata)
+{
+ spin_lock_bh(&local->rx_lock);
+ mac802154_flush_list(&local->rx_beacon_list, sdata);
+ mac802154_flush_list(&local->rx_mac_cmd_list, sdata);
+ spin_unlock_bh(&local->rx_lock);
}
static int
@@ -221,7 +291,10 @@ ieee802154_subif_frame(struct ieee802154_sub_if_data *sdata,
mac_pkt->sdata = sdata;
mac_pkt->page = sdata->local->scan_page;
mac_pkt->channel = sdata->local->scan_channel;
+ netdev_hold(sdata->dev, &mac_pkt->dev_tracker, GFP_ATOMIC);
+ spin_lock(&sdata->local->rx_lock);
list_add_tail(&mac_pkt->node, &sdata->local->rx_beacon_list);
+ spin_unlock(&sdata->local->rx_lock);
queue_work(sdata->local->mac_wq, &sdata->local->rx_beacon_work);
return NET_RX_SUCCESS;
@@ -233,7 +306,10 @@ ieee802154_subif_frame(struct ieee802154_sub_if_data *sdata,
mac_pkt->skb = skb_get(skb);
mac_pkt->sdata = sdata;
+ netdev_hold(sdata->dev, &mac_pkt->dev_tracker, GFP_ATOMIC);
+ spin_lock(&sdata->local->rx_lock);
list_add_tail(&mac_pkt->node, &sdata->local->rx_mac_cmd_list);
+ spin_unlock(&sdata->local->rx_lock);
queue_work(sdata->local->mac_wq, &sdata->local->rx_mac_cmd_work);
return NET_RX_SUCCESS;
diff --git a/net/mac802154/scan.c b/net/mac802154/scan.c
index 005338f89b75..ec36eb5716f5 100644
--- a/net/mac802154/scan.c
+++ b/net/mac802154/scan.c
@@ -104,13 +104,9 @@ static unsigned int mac802154_scan_get_channel_time(u8 duration_order,
static void mac802154_flush_queued_beacons(struct ieee802154_local *local)
{
- struct cfg802154_mac_pkt *mac_pkt, *tmp;
-
- list_for_each_entry_safe(mac_pkt, tmp, &local->rx_beacon_list, node) {
- list_del(&mac_pkt->node);
- kfree_skb(mac_pkt->skb);
- kfree(mac_pkt);
- }
+ spin_lock_bh(&local->rx_lock);
+ mac802154_flush_list(&local->rx_beacon_list, NULL);
+ spin_unlock_bh(&local->rx_lock);
}
static void
diff --git a/net/mptcp/options.c b/net/mptcp/options.c
index 1057d500577b..bf8ea4cccb2f 100644
--- a/net/mptcp/options.c
+++ b/net/mptcp/options.c
@@ -93,7 +93,8 @@ static void mptcp_parse_option(const struct sk_buff *skb,
* In other words, the only way for checksums not to be used
* is if both hosts in their SYNs set A=0."
*/
- if (flags & MPTCP_CAP_CHECKSUM_REQD)
+ if ((flags & MPTCP_CAP_CHECKSUM_REQD) &&
+ opsize < TCPOLEN_MPTCP_MPC_ACK_DATA)
mp_opt->suboptions |= OPTION_MPTCP_CSUMREQD;
mp_opt->deny_join_id0 = !!(flags & MPTCP_CAP_DENY_JOIN_ID0);
@@ -529,7 +530,7 @@ static bool mptcp_established_options_mp(struct sock *sk, struct sk_buff *skb,
return false;
/* MPC/MPJ needed only on 3rd ack packet, DATA_FIN and TCP shutdown take precedence */
- if (READ_ONCE(subflow->fully_established) || snd_data_fin_enable ||
+ if (subflow->fully_established || snd_data_fin_enable ||
subflow->snd_isn != TCP_SKB_CB(skb)->seq ||
sk->sk_state != TCP_ESTABLISHED)
return false;
@@ -611,6 +612,7 @@ static void mptcp_write_data_fin(struct mptcp_subflow_context *subflow,
ext->data_seq = data_fin_tx_seq;
ext->subflow_seq = 0;
ext->data_len = 1;
+ ext->csum = 0;
} else if (ext->data_seq + ext->data_len == data_fin_tx_seq) {
/* If there's an existing DSS mapping and it is the
* final mapping, DATA_FIN consumes 1 additional byte of
@@ -980,7 +982,7 @@ static bool check_fully_established(struct mptcp_sock *msk, struct sock *ssk,
/* here we can process OoO, in-window pkts, only in-sequence 4th ack
* will make the subflow fully established
*/
- if (likely(READ_ONCE(subflow->fully_established))) {
+ if (likely(subflow->fully_established)) {
/* on passive sockets, check for 3rd ack retransmission
* note that msk is always set by subflow_syn_recv_sock()
* for mp_join subflows
diff --git a/net/mptcp/pm.c b/net/mptcp/pm.c
index 64a1236aabee..79bf88ca033b 100644
--- a/net/mptcp/pm.c
+++ b/net/mptcp/pm.c
@@ -462,10 +462,10 @@ bool mptcp_pm_announced_alloc(struct mptcp_sock *msk,
add_entry->addr = *addr;
add_entry->sock = msk;
- add_entry->retrans_times = 0;
timer_setup(&add_entry->timer, mptcp_pm_add_addr_timer, 0);
reset_timer:
+ add_entry->retrans_times = 0;
add_entry->timer_done = false;
timeout = mptcp_adjust_add_addr_timeout(msk);
if (timeout)
@@ -1065,7 +1065,8 @@ bool mptcp_pm_is_backup(struct mptcp_sock *msk, struct sock_common *skc)
return mptcp_pm_nl_is_backup(msk, &skc_local);
}
-static void mptcp_pm_subflows_chk_stale(const struct mptcp_sock *msk, struct sock *ssk)
+static void
+mptcp_pm_subflow_chk_stale(const struct mptcp_sock *msk, struct sock *ssk)
{
struct mptcp_subflow_context *iter, *subflow = mptcp_subflow_ctx(ssk);
struct sock *sk = (struct sock *)msk;
@@ -1102,22 +1103,34 @@ static void mptcp_pm_subflows_chk_stale(const struct mptcp_sock *msk, struct soc
}
}
-void mptcp_pm_subflow_chk_stale(const struct mptcp_sock *msk, struct sock *ssk)
+void mptcp_pm_chk_stale(const struct mptcp_sock *msk)
{
- struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk);
- u32 rcv_tstamp = READ_ONCE(tcp_sk(ssk)->rcv_tstamp);
-
- /* keep track of rtx periods with no progress */
- if (!subflow->stale_count) {
- subflow->stale_rcv_tstamp = rcv_tstamp;
- subflow->stale_count++;
- } else if (subflow->stale_rcv_tstamp == rcv_tstamp) {
- if (subflow->stale_count < U8_MAX)
+ struct mptcp_subflow_context *subflow;
+
+ mptcp_for_each_subflow(msk, subflow) {
+ struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+ u32 rcv_tstamp;
+
+ if (!__mptcp_subflow_active(subflow))
+ continue;
+
+ /* No data outstanding at TCP level? not stale */
+ if (tcp_rtx_and_write_queues_empty(ssk))
+ continue;
+
+ /* keep track of rtx periods with no progress */
+ rcv_tstamp = READ_ONCE(tcp_sk(ssk)->rcv_tstamp);
+ if (!subflow->stale_count) {
+ subflow->stale_rcv_tstamp = rcv_tstamp;
subflow->stale_count++;
- mptcp_pm_subflows_chk_stale(msk, ssk);
- } else {
- subflow->stale_count = 0;
- mptcp_subflow_set_active(subflow);
+ } else if (subflow->stale_rcv_tstamp == rcv_tstamp) {
+ if (subflow->stale_count < U8_MAX)
+ subflow->stale_count++;
+ mptcp_pm_subflow_chk_stale(msk, ssk);
+ } else {
+ subflow->stale_count = 0;
+ mptcp_subflow_set_active(subflow);
+ }
}
}
diff --git a/net/mptcp/pm_kernel.c b/net/mptcp/pm_kernel.c
index 424f1a7f9248..1a7750813235 100644
--- a/net/mptcp/pm_kernel.c
+++ b/net/mptcp/pm_kernel.c
@@ -1137,6 +1137,8 @@ static int mptcp_nl_remove_id_zero_address(struct net *net,
while ((msk = mptcp_token_iter_next(net, &s_slot, &s_num)) != NULL) {
struct sock *sk = (struct sock *)msk;
struct mptcp_addr_info msk_local;
+ struct mptcp_addr_info anno_addr;
+ bool announced;
if (list_empty(&msk->conn_list) || mptcp_pm_is_userspace(msk))
goto next;
@@ -1146,7 +1148,13 @@ static int mptcp_nl_remove_id_zero_address(struct net *net,
goto next;
lock_sock(sk);
+ /* Drop a possibly pending ADD_ADDR for this address. */
+ anno_addr = msk_local;
+ anno_addr.port = 0;
+ announced = mptcp_pm_announced_remove(msk, &anno_addr);
spin_lock_bh(&msk->pm.lock);
+ if (announced)
+ msk->pm.add_addr_signaled--;
mptcp_pm_remove_addr(msk, &list);
mptcp_pm_rm_subflow(msk, &list);
__mark_subflow_endp_available(msk, 0);
diff --git a/net/mptcp/pm_userspace.c b/net/mptcp/pm_userspace.c
index 2203cc2d2748..57c13293a5f8 100644
--- a/net/mptcp/pm_userspace.c
+++ b/net/mptcp/pm_userspace.c
@@ -69,6 +69,19 @@ static int mptcp_userspace_pm_append_new_local_addr(struct mptcp_sock *msk,
}
if (!addr_match && !id_match) {
+ unsigned int id;
+
+ if (!entry->addr.id && needs_id) {
+ id = find_next_zero_bit(id_bitmap,
+ MPTCP_PM_MAX_ADDR_ID + 1, 1);
+ if (id > MPTCP_PM_MAX_ADDR_ID) {
+ ret = -ENOSPC;
+ goto append_err;
+ }
+ } else {
+ id = entry->addr.id;
+ }
+
/* Memory for the entry is allocated from the
* sock option buffer.
*/
@@ -78,10 +91,7 @@ static int mptcp_userspace_pm_append_new_local_addr(struct mptcp_sock *msk,
goto append_err;
}
- if (!e->addr.id && needs_id)
- e->addr.id = find_next_zero_bit(id_bitmap,
- MPTCP_PM_MAX_ADDR_ID + 1,
- 1);
+ e->addr.id = id;
list_add_tail_rcu(&e->list, &msk->pm.userspace_pm_local_addr_list);
msk->pm.local_addr_used++;
ret = e->addr.id;
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 7c8180d8d5ef..49c46498af96 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -95,6 +95,7 @@ bool __mptcp_try_fallback(struct mptcp_sock *msk, int fb_mib)
msk->allow_subflows = false;
set_bit(MPTCP_FALLBACK_DONE, &msk->flags);
+ clear_bit(MPTCP_RTX_ENABLED, &msk->flags);
__MPTCP_INC_STATS(net, fb_mib);
spin_unlock_bh(&msk->fallback_lock);
return true;
@@ -772,12 +773,12 @@ static bool __mptcp_move_skbs_from_subflow(struct mptcp_sock *msk,
mptcp_dss_corruption(msk, ssk);
}
} else {
+ sk_eat_skb(ssk, skb);
+
if (unlikely(!fin)) {
DEBUG_NET_WARN_ON_ONCE(1);
mptcp_dss_corruption(msk, ssk);
}
-
- sk_eat_skb(ssk, skb);
}
WRITE_ONCE(tp->copied_seq, seq);
@@ -1001,13 +1002,14 @@ static bool mptcp_rtx_timer_pending(struct sock *sk)
static void mptcp_reset_rtx_timer(struct sock *sk)
{
+ struct mptcp_sock *msk = mptcp_sk(sk);
unsigned long tout;
- /* prevent rescheduling on close */
- if (unlikely(inet_sk_state_load(sk) == TCP_CLOSE))
+ /* Prevent rescheduling on close and in case of fallback. */
+ if (!test_bit(MPTCP_RTX_ENABLED, &msk->flags))
return;
- tout = mptcp_sk(sk)->timer_ival;
+ tout = msk->timer_ival;
sk_reset_timer(sk, &sk->mptcp_retransmit_timer, jiffies + tout);
}
@@ -1586,7 +1588,9 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
static void mptcp_push_release(struct sock *ssk, struct mptcp_sendmsg_info *info)
{
- tcp_push(ssk, 0, info->mss_now, tcp_sk(ssk)->nonagle, info->size_goal);
+ if (info->mss_now)
+ tcp_push(ssk, 0, info->mss_now, tcp_sk(ssk)->nonagle,
+ info->size_goal);
release_sock(ssk);
}
@@ -1774,7 +1778,8 @@ static void __mptcp_subflow_push_pending(struct sock *sk, struct sock *ssk, bool
ret = __subflow_push_pending(sk, ssk, &info);
if (ret <= 0)
keep_pushing = false;
- copied += ret;
+ else
+ copied += ret;
}
mptcp_for_each_subflow(msk, subflow) {
@@ -2469,7 +2474,6 @@ struct sock *mptcp_subflow_get_retrans(struct mptcp_sock *msk)
/* still data outstanding at TCP level? skip this */
if (!tcp_rtx_and_write_queues_empty(ssk)) {
- mptcp_pm_subflow_chk_stale(msk, ssk);
min_stale_count = min_t(int, min_stale_count, subflow->stale_count);
continue;
}
@@ -2801,9 +2805,10 @@ static void __mptcp_retrans(struct sock *sk)
int ret, err;
u16 len = 0;
+ mptcp_pm_chk_stale(msk);
+
mptcp_clean_una_wakeup(sk);
- /* first check ssk: need to kick "stale" logic */
err = mptcp_sched_get_retrans(msk);
dfrag = mptcp_rtx_head(sk);
if (!dfrag) {
@@ -2985,8 +2990,7 @@ static void mptcp_do_fastclose(struct sock *sk)
*/
inet_csk(ssk)->icsk_ack.rcv_mss = TCP_MIN_MSS;
- tcp_send_active_reset(ssk, ssk->sk_allocation,
- SK_RST_REASON_TCP_ABORT_ON_CLOSE);
+ tcp_send_active_reset(ssk, SK_RST_REASON_TCP_ABORT_ON_CLOSE);
unlock:
release_sock(ssk);
}
@@ -3192,6 +3196,9 @@ void mptcp_set_state(struct sock *sk, int state)
* transition from TCP_SYN_RECV to TCP_CLOSE_WAIT.
*/
break;
+ case TCP_CLOSE:
+ clear_bit(MPTCP_RTX_ENABLED, &mptcp_sk(sk)->flags);
+ fallthrough;
default:
if (oldstate == TCP_ESTABLISHED || oldstate == TCP_CLOSE_WAIT)
MPTCP_DEC_STATS(sock_net(sk), MPTCP_MIB_CURRESTAB);
@@ -3452,6 +3459,7 @@ static void mptcp_destroy_common(struct mptcp_sock *msk)
static int mptcp_disconnect(struct sock *sk, int flags)
{
+ struct inet_connection_sock *icsk = inet_csk(sk);
struct mptcp_sock *msk = mptcp_sk(sk);
/* We are on the fastopen error path. We can't call straight into the
@@ -3464,8 +3472,13 @@ static int mptcp_disconnect(struct sock *sk, int flags)
mptcp_check_listen_stop(sk);
mptcp_set_state(sk, TCP_CLOSE);
- mptcp_stop_rtx_timer(sk);
- mptcp_stop_tout_timer(sk);
+ /* The later subflow close can not kick again the tout timer,
+ * as the msk is already in closed status.
+ */
+ msk->timer_ival = icsk->icsk_rto_min;
+ sk_stop_timer_sync(sk, &sk->mptcp_retransmit_timer);
+ icsk->icsk_mtup.probe_timestamp = 0;
+ sk_stop_timer_sync(sk, &icsk->mptcp_tout_timer);
mptcp_pm_connection_closed(msk);
@@ -3744,7 +3757,7 @@ static void schedule_3rdack_retransmission(struct sock *ssk)
struct tcp_sock *tp = tcp_sk(ssk);
unsigned long timeout;
- if (READ_ONCE(mptcp_subflow_ctx(ssk)->fully_established))
+ if (mptcp_subflow_ctx(ssk)->fully_established)
return;
/* reschedule with a timeout above RTT, as we must look only for drop */
@@ -4009,6 +4022,7 @@ static int mptcp_connect(struct sock *sk, struct sockaddr_unsized *uaddr,
if (IS_ERR(ssk))
return PTR_ERR(ssk);
+ set_bit(MPTCP_RTX_ENABLED, &msk->flags);
mptcp_set_state(sk, TCP_SYN_SENT);
subflow = mptcp_subflow_ctx(ssk);
#ifdef CONFIG_TCP_MD5SIG
@@ -4156,6 +4170,7 @@ static int mptcp_listen(struct socket *sock, int backlog)
goto unlock;
}
+ set_bit(MPTCP_RTX_ENABLED, &msk->flags);
mptcp_set_state(sk, TCP_LISTEN);
sock_set_flag(sk, SOCK_RCU_FREE);
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index 1b80f2d6ec5a..d44aaa783c22 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -116,6 +116,7 @@
#define MPTCP_WORK_RTX 1
#define MPTCP_FALLBACK_DONE 2
#define MPTCP_WORK_CLOSE_SUBFLOW 3
+#define MPTCP_RTX_ENABLED 4
/* MPTCP socket release cb flags */
#define MPTCP_PUSH_PENDING 1
@@ -580,11 +581,11 @@ struct mptcp_subflow_context {
remote_key_valid : 1, /* received the peer key from */
disposable : 1, /* ctx can be free at ulp release time */
closing : 1, /* must not pass rx data to msk anymore */
- stale : 1, /* unable to snd/rcv data, do not use for xmit */
valid_csum_seen : 1, /* at least one csum validated */
is_mptfo : 1, /* subflow is doing TFO */
close_event_done : 1, /* has done the post-closed part */
mpc_drop : 1, /* the MPC option has been dropped in a rtx */
+ resetting : 1, /* subflow is resetting */
__unused : 8;
bool data_avail;
bool scheduled;
@@ -604,7 +605,11 @@ struct mptcp_subflow_context {
u8 reset_seen:1;
u8 reset_transient:1;
u8 reset_reason:4;
- u8 stale_count;
+ u8 stale_count; /* Protected by the msk socket lock */
+ u8 stale; /* Protected by the msk socket lock,
+ * if set the subflow is unable to snd/rcv
+ * data, the schedule should skip it
+ */
u32 subflow_id;
@@ -687,7 +692,7 @@ mptcp_send_active_reset_reason(struct sock *sk)
enum sk_rst_reason reason;
reason = sk_rst_convert_mptcp_reason(subflow->reset_reason);
- tcp_send_active_reset(sk, GFP_ATOMIC, reason);
+ tcp_send_active_reset(sk, reason);
}
/* Made the fwd mem carried by the given skb available to the msk,
@@ -1103,7 +1108,7 @@ int mptcp_pm_parse_entry(struct nlattr *attr, struct genl_info *info,
bool mptcp_pm_addr_families_match(const struct sock *sk,
const struct mptcp_addr_info *loc,
const struct mptcp_addr_info *rem);
-void mptcp_pm_subflow_chk_stale(const struct mptcp_sock *msk, struct sock *ssk);
+void mptcp_pm_chk_stale(const struct mptcp_sock *msk);
void mptcp_pm_new_connection(struct mptcp_sock *msk, const struct sock *ssk, int server_side);
void mptcp_pm_fully_established(struct mptcp_sock *msk, const struct sock *ssk);
bool mptcp_pm_allow_new_subflow(struct mptcp_sock *msk);
diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c
index e1f20ff8fdb4..edd33dc9367f 100644
--- a/net/mptcp/subflow.c
+++ b/net/mptcp/subflow.c
@@ -436,6 +436,10 @@ void mptcp_subflow_reset(struct sock *ssk)
/* must hold: tcp_done() could drop last reference on parent */
sock_hold(sk);
+ subflow->resetting = 1;
+
+ /* No need to delay the actual close for to-be discarded data. */
+ __skb_queue_purge(&ssk->sk_receive_queue);
mptcp_send_active_reset_reason(ssk);
tcp_done(ssk);
if (!test_and_set_bit(MPTCP_WORK_CLOSE_SUBFLOW, &mptcp_sk(sk)->flags))
@@ -1873,6 +1877,13 @@ static void subflow_state_change(struct sock *sk)
__subflow_state_change(sk);
+ /* Rx queue processing is unneeded, error reporting will take place at
+ * __mptcp_close_ssk() time and subflow reset can't happen in case of
+ * fallback: subflow_sched_work_if_closed() would be a no-op.
+ */
+ if (subflow->resetting)
+ return;
+
/* as recvmsg() does not acquire the subflow socket for ssk selection
* a fin packet carrying a DSS can be unnoticed if we don't trigger
* the data available machinery here.
@@ -2074,7 +2085,6 @@ static void subflow_ulp_clone(const struct request_sock *req,
new_ctx->request_bkup = subflow_req->request_bkup;
WRITE_ONCE(new_ctx->remote_id, subflow_req->remote_id);
new_ctx->token = subflow_req->token;
- new_ctx->thmac = subflow_req->thmac;
/* the subflow req id is valid, fetched via subflow_check_req()
* and subflow_token_join_request()
diff --git a/net/mptcp/syncookies.c b/net/mptcp/syncookies.c
index b5cac5701122..9474706641c1 100644
--- a/net/mptcp/syncookies.c
+++ b/net/mptcp/syncookies.c
@@ -26,7 +26,8 @@ struct join_entry {
u32 local_nonce;
u8 join_id;
u8 local_id;
- u8 backup;
+ u8 backup:1,
+ request_bkup:1;
u8 valid;
};
@@ -63,6 +64,7 @@ static void mptcp_join_store_state(struct join_entry *entry,
entry->remote_nonce = subflow_req->remote_nonce;
entry->local_nonce = subflow_req->local_nonce;
entry->backup = subflow_req->backup;
+ entry->request_bkup = subflow_req->request_bkup;
entry->join_id = subflow_req->remote_id;
entry->local_id = subflow_req->local_id;
entry->valid = 1;
@@ -117,6 +119,7 @@ bool mptcp_token_join_cookie_init_state(struct mptcp_subflow_request_sock *subfl
subflow_req->remote_nonce = e->remote_nonce;
subflow_req->local_nonce = e->local_nonce;
subflow_req->backup = e->backup;
+ subflow_req->request_bkup = e->request_bkup;
subflow_req->remote_id = e->join_id;
subflow_req->local_id = e->local_id;
subflow_req->token = e->token;
diff --git a/net/netfilter/ipvs/ip_vs_sync.c b/net/netfilter/ipvs/ip_vs_sync.c
index ea5fdd4f4ce7..5383aeafb0ae 100644
--- a/net/netfilter/ipvs/ip_vs_sync.c
+++ b/net/netfilter/ipvs/ip_vs_sync.c
@@ -747,9 +747,9 @@ void ip_vs_sync_conn(struct netns_ipvs *ipvs, struct ip_vs_conn *cp, int pkts)
if (cp->flags & IP_VS_CONN_F_SEQ_MASK) {
*(p++) = IPVS_OPT_SEQ_DATA;
*(p++) = sizeof(struct ip_vs_sync_conn_options);
- hton_seq((struct ip_vs_seq *)p, &cp->in_seq);
+ hton_seq(&cp->in_seq, (struct ip_vs_seq *)p);
p += sizeof(struct ip_vs_seq);
- hton_seq((struct ip_vs_seq *)p, &cp->out_seq);
+ hton_seq(&cp->out_seq, (struct ip_vs_seq *)p);
p += sizeof(struct ip_vs_seq);
}
/* Handle pe data */
@@ -999,10 +999,10 @@ static void ip_vs_process_message_v0(struct netns_ipvs *ipvs, const char *buffer
pp->name, state);
continue;
}
- } else {
- if (state >= IP_VS_CTPL_S_LAST)
- IP_VS_DBG(7, "BACKUP v0, Invalid tpl state %u\n",
- state);
+ } else if (state >= IP_VS_CTPL_S_LAST) {
+ IP_VS_DBG(7, "BACKUP v0, Invalid tpl state %u\n",
+ state);
+ continue;
}
ip_vs_conn_fill_param(ipvs, AF_INET, s->protocol,
@@ -1159,10 +1159,10 @@ static inline int ip_vs_proc_sync_conn(struct netns_ipvs *ipvs, __u8 *p, __u8 *m
retc = 40;
goto out;
}
- } else {
- if (state >= IP_VS_CTPL_S_LAST)
- IP_VS_DBG(7, "BACKUP, Invalid tpl state %u\n",
- state);
+ } else if (state >= IP_VS_CTPL_S_LAST) {
+ IP_VS_DBG(7, "BACKUP, Invalid tpl state %u\n", state);
+ retc = 40;
+ goto out;
}
if (ip_vs_conn_fill_param_sync(ipvs, af, s, ¶m, pe_data,
pe_data_len, pe_name, pe_name_len)) {
diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c
index 11e873a0bd14..92c3bb77d27e 100644
--- a/net/netfilter/nf_conntrack_netlink.c
+++ b/net/netfilter/nf_conntrack_netlink.c
@@ -1077,6 +1077,8 @@ static int ctnetlink_start(struct netlink_callback *cb)
}
cb->data = filter;
+ if (filter)
+ cb->answer_flags = NLM_F_DUMP_FILTERED;
return 0;
}
diff --git a/net/netfilter/nf_conntrack_sip.c b/net/netfilter/nf_conntrack_sip.c
index e4a70d1d77b0..4fb33b5e9a85 100644
--- a/net/netfilter/nf_conntrack_sip.c
+++ b/net/netfilter/nf_conntrack_sip.c
@@ -429,7 +429,7 @@ static const char *sip_skip_whitespace(const char *dptr, const char *limit)
dptr = sip_follow_continuation(dptr, limit);
break;
}
- return dptr;
+ return dptr < limit ? dptr : NULL;
}
/* Search within a SIP header value, dealing with continuation lines */
diff --git a/net/netfilter/nf_log_syslog.c b/net/netfilter/nf_log_syslog.c
index 5ffde27d450a..ae0f9a47556c 100644
--- a/net/netfilter/nf_log_syslog.c
+++ b/net/netfilter/nf_log_syslog.c
@@ -1073,12 +1073,12 @@ static int __init nf_log_syslog_init(void)
static void __exit nf_log_syslog_exit(void)
{
- unregister_pernet_subsys(&nf_log_syslog_net_ops);
nf_log_unregister(&nf_ip_logger);
nf_log_unregister(&nf_arp_logger);
nf_log_unregister(&nf_ip6_logger);
nf_log_unregister(&nf_netdev_logger);
nf_log_unregister(&nf_bridge_logger);
+ unregister_pernet_subsys(&nf_log_syslog_net_ops);
}
module_init(nf_log_syslog_init);
diff --git a/net/netfilter/nfnetlink_cttimeout.c b/net/netfilter/nfnetlink_cttimeout.c
index 170d3db860c5..659bafcacc85 100644
--- a/net/netfilter/nfnetlink_cttimeout.c
+++ b/net/netfilter/nfnetlink_cttimeout.c
@@ -652,9 +652,9 @@ static void __exit cttimeout_exit(void)
{
nfnetlink_subsys_unregister(&cttimeout_subsys);
- unregister_pernet_subsys(&cttimeout_ops);
RCU_INIT_POINTER(nf_ct_timeout_hook, NULL);
synchronize_net();
+ unregister_pernet_subsys(&cttimeout_ops);
}
module_init(cttimeout_init);
diff --git a/net/netfilter/nfnetlink_log.c b/net/netfilter/nfnetlink_log.c
index 6c7fa2ed34f5..d923f2cb1398 100644
--- a/net/netfilter/nfnetlink_log.c
+++ b/net/netfilter/nfnetlink_log.c
@@ -228,13 +228,18 @@ static void __nfulnl_flush(struct nfulnl_instance *inst);
static void
__instance_destroy(struct nfulnl_instance *inst)
{
+ spin_lock(&inst->lock);
+ if (inst->copy_mode == NFULNL_COPY_DISABLED) {
+ /* attempt to UNBIND a queue already pending
+ * destruction via netlink close event. Ignore.
+ */
+ spin_unlock(&inst->lock);
+ return;
+ }
+
/* first pull it out of the global list */
hlist_del_rcu(&inst->hlist);
- /* then flush all pending packets from skb */
-
- spin_lock(&inst->lock);
-
/* lockless readers wont be able to use us */
inst->copy_mode = NFULNL_COPY_DISABLED;
@@ -1233,8 +1238,8 @@ static void __exit nfnetlink_log_fini(void)
{
nfnetlink_subsys_unregister(&nfulnl_subsys);
netlink_unregister_notifier(&nfulnl_rtnl_notifier);
- unregister_pernet_subsys(&nfnl_log_net_ops);
nf_log_unregister(&nfulnl_logger);
+ unregister_pernet_subsys(&nfnl_log_net_ops);
}
MODULE_DESCRIPTION("netfilter userspace logging");
diff --git a/net/netlink/genetlink.c b/net/netlink/genetlink.c
index 0da39eaed255..41d37442f186 100644
--- a/net/netlink/genetlink.c
+++ b/net/netlink/genetlink.c
@@ -1513,6 +1513,7 @@ struct ctrl_dump_policy_ctx {
struct netlink_policy_dump_state *state;
const struct genl_family *rt;
struct genl_op_iter *op_iter;
+ struct module *owner;
u32 op;
u16 fam_id;
u8 dump_map:1,
@@ -1555,6 +1556,9 @@ static int ctrl_dumppolicy_start(struct netlink_callback *cb)
return -ENOENT;
ctx->rt = rt;
+ ctx->owner = rt->module;
+ if (!try_module_get(ctx->owner))
+ return -ENOENT;
if (tb[CTRL_ATTR_OP]) {
struct genl_split_ops doit, dump;
@@ -1565,7 +1569,7 @@ static int ctrl_dumppolicy_start(struct netlink_callback *cb)
err = genl_get_cmd_both(ctx->op, rt, &doit, &dump);
if (err) {
NL_SET_BAD_ATTR(cb->extack, tb[CTRL_ATTR_OP]);
- return err;
+ goto err_put_owner;
}
if (doit.policy) {
@@ -1583,16 +1587,20 @@ static int ctrl_dumppolicy_start(struct netlink_callback *cb)
goto err_free_state;
}
- if (!ctx->state)
- return -ENODATA;
+ if (!ctx->state) {
+ err = -ENODATA;
+ goto err_put_owner;
+ }
ctx->dump_map = 1;
return 0;
}
ctx->op_iter = kmalloc_obj(*ctx->op_iter);
- if (!ctx->op_iter)
- return -ENOMEM;
+ if (!ctx->op_iter) {
+ err = -ENOMEM;
+ goto err_put_owner;
+ }
genl_op_iter_init(rt, ctx->op_iter);
ctx->dump_map = genl_op_iter_next(ctx->op_iter);
@@ -1624,6 +1632,8 @@ static int ctrl_dumppolicy_start(struct netlink_callback *cb)
netlink_policy_dump_free(ctx->state);
err_free_op_iter:
kfree(ctx->op_iter);
+err_put_owner:
+ module_put(ctx->owner);
return err;
}
@@ -1760,6 +1770,7 @@ static int ctrl_dumppolicy_done(struct netlink_callback *cb)
kfree(ctx->op_iter);
netlink_policy_dump_free(ctx->state);
+ module_put(ctx->owner);
return 0;
}
diff --git a/net/openvswitch/flow.c b/net/openvswitch/flow.c
index 46c1d66aad8c..868d9fdf3afd 100644
--- a/net/openvswitch/flow.c
+++ b/net/openvswitch/flow.c
@@ -288,7 +288,7 @@ static void get_ipv6_ext_hdrs(struct sk_buff *skb, struct ipv6hdr *nh,
if (*ext_hdrs & OFPIEH12_ESP)
*ext_hdrs |= OFPIEH12_UNREP;
if ((*ext_hdrs & ~(OFPIEH12_HOP | OFPIEH12_DEST |
- OFPIEH12_ROUTER | IPPROTO_FRAGMENT |
+ OFPIEH12_ROUTER | OFPIEH12_FRAG |
OFPIEH12_AUTH | OFPIEH12_UNREP)) ||
dest_options_header_count >= 2) {
*ext_hdrs |= OFPIEH12_UNSEQ;
@@ -301,7 +301,7 @@ static void get_ipv6_ext_hdrs(struct sk_buff *skb, struct ipv6hdr *nh,
*ext_hdrs |= OFPIEH12_UNREP;
if ((*ext_hdrs &
~(OFPIEH12_HOP | OFPIEH12_DEST | OFPIEH12_ROUTER |
- IPPROTO_FRAGMENT | OFPIEH12_UNREP)) ||
+ OFPIEH12_FRAG | OFPIEH12_UNREP)) ||
dest_options_header_count >= 2) {
*ext_hdrs |= OFPIEH12_UNSEQ;
}
diff --git a/net/openvswitch/flow_table.c b/net/openvswitch/flow_table.c
index 67d5b8c0fe79..1e0f9d193eb0 100644
--- a/net/openvswitch/flow_table.c
+++ b/net/openvswitch/flow_table.c
@@ -257,11 +257,13 @@ static int tbl_mask_array_realloc(struct flow_table *tbl, int size)
if (ovsl_dereference(old->masks[i]))
new->masks[new->count++] = old->masks[i];
}
- call_rcu(&old->rcu, mask_array_rcu_cb);
}
rcu_assign_pointer(tbl->mask_array, new);
+ if (old)
+ call_rcu(&old->rcu, mask_array_rcu_cb);
+
return 0;
}
diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
index b22cda322136..76bde7906d49 100644
--- a/net/packet/af_packet.c
+++ b/net/packet/af_packet.c
@@ -2675,7 +2675,8 @@ static int tpacket_parse_header(struct packet_sock *po, void *frame,
int size_max, void **data)
{
union tpacket_uhdr ph;
- int tp_len, off;
+ u32 tp_len;
+ int off;
ph.raw = frame;
@@ -2695,7 +2696,7 @@ static int tpacket_parse_header(struct packet_sock *po, void *frame,
break;
}
if (unlikely(tp_len > size_max)) {
- pr_err("packet size is too long (%d > %d)\n", tp_len, size_max);
+ pr_err("packet size is too long (%u > %d)\n", tp_len, size_max);
return -EMSGSIZE;
}
diff --git a/net/rds/connection.c b/net/rds/connection.c
index 7c8ab8e973e1..b6c4beb50eaf 100644
--- a/net/rds/connection.c
+++ b/net/rds/connection.c
@@ -106,10 +106,12 @@ static struct rds_connection *rds_conn_lookup(struct net *net,
}
/*
- * This is called by transports as they're bringing down a connection.
- * It clears partial message state so that the transport can start sending
- * and receiving over this connection again in the future. It is up to
- * the transport to have serialized this call with its send and recv.
+ * This is called by rds_conn_shutdown() once the transport has brought
+ * a path down. It clears partial message state so that the transport
+ * can start sending and receiving over this path again in the future.
+ * The caller owns RDS_IN_XMIT and RDS_RECV_REFILL across this call,
+ * which is what serializes it against the send and receive-refill
+ * paths.
*/
static void rds_conn_path_reset(struct rds_conn_path *cp)
{
@@ -120,7 +122,16 @@ static void rds_conn_path_reset(struct rds_conn_path *cp)
rds_stats_inc(s_conn_reset);
rds_send_path_reset(cp);
- cp->cp_flags = 0;
+
+ /* Clear the bits the reset is responsible for individually: a
+ * blanket cp_flags = 0 is a plain store that can clobber a
+ * concurrent atomic read-modify-write on the same word.
+ * RDS_IN_XMIT and RDS_RECV_REFILL are held as locks by the
+ * caller, rds_conn_shutdown(), which releases them once the
+ * teardown is complete.
+ */
+ clear_bit(RDS_LL_SEND_FULL, &cp->cp_flags);
+ clear_bit(RDS_RECONNECT_PENDING, &cp->cp_flags);
/* Do not clear next_rx_seq here, else we cannot distinguish
* retransmitted packets from new packets, and will hand all
@@ -406,28 +417,70 @@ void rds_conn_shutdown(struct rds_conn_path *cp)
}
mutex_unlock(&cp->cp_cm_lock);
+ /* Quiesce the transmit and receive-refill paths by
+ * acquiring their bit locks, not merely waiting for
+ * them to be released: with a plain wait, either path
+ * can re-take its lock the instant after we sample it
+ * clear and then run concurrently with the transport
+ * shutdown and the path reset below. Holding both
+ * locks across the teardown makes that structurally
+ * impossible.
+ */
wait_event(cp->cp_waitq,
- !test_bit(RDS_IN_XMIT, &cp->cp_flags));
+ !test_and_set_bit_lock(RDS_IN_XMIT, &cp->cp_flags));
wait_event(cp->cp_waitq,
- !test_bit(RDS_RECV_REFILL, &cp->cp_flags));
+ !test_and_set_bit(RDS_RECV_REFILL, &cp->cp_flags));
conn->c_trans->conn_path_shutdown(cp);
rds_conn_path_reset(cp);
+ /* Release the two locks and wake any waiter (e.g.
+ * rds_tcp_reset_callbacks()) that blocked on them while
+ * we held them. The unlock orders the transport's ring
+ * re-initialization and the path reset above before
+ * either bit is seen clear. rds_conn_path_reset() leaves
+ * both bits alone: ownership ends here, not inside the
+ * reset.
+ */
+ clear_bit_unlock(RDS_IN_XMIT, &cp->cp_flags);
+ clear_bit_unlock(RDS_RECV_REFILL, &cp->cp_flags);
+ wake_up_all(&cp->cp_waitq);
+
if (!rds_conn_path_transition(cp, RDS_CONN_DISCONNECTING,
- RDS_CONN_DOWN) &&
- !rds_conn_path_transition(cp, RDS_CONN_ERROR,
RDS_CONN_DOWN)) {
- /* This can happen - eg when we're in the middle of tearing
- * down the connection, and someone unloads the rds module.
- * Quite reproducible with loopback connections.
- * Mostly harmless.
+ /* The path was dropped again while we tore it
+ * down: by a socket state-change callback in
+ * irq context on receipt of a FIN, or by an
+ * accept that claimed the path just before a
+ * drop put it back to RDS_CONN_ERROR and then
+ * installed a fresh socket on it. Unless a
+ * pending destroy suppressed it, the drop also
+ * queued another shutdown pass, and that pass
+ * must run, because it is what tears down
+ * whatever attached to the path after the
+ * transport shutdown above sampled its state.
+ * Consuming the RDS_CONN_ERROR here would turn
+ * that pass into a no-op: leave the state
+ * alone, and let the pass finish the job.
*
- * Note that this also happens with rds-tcp because
- * we could have triggered rds_conn_path_drop in irq
- * mode from rds_tcp_state change on the receipt of
- * a FIN, thus we need to recheck for RDS_CONN_ERROR
- * here.
+ * Quiesce the reconnect timer before bailing
+ * out, though. When a pending destroy did
+ * suppress the queue, no later pass runs, and
+ * rds_conn_path_destroy() is about to flush
+ * cp_down_w and free the path: it must not
+ * find cp_conn_w still armed. A successor
+ * pass, when there is one, re-arms the
+ * reconnect from its own tail.
+ */
+ cancel_delayed_work_sync(&cp->cp_conn_w);
+ clear_bit(RDS_RECONNECT_PENDING, &cp->cp_flags);
+
+ if (rds_conn_path_state(cp) == RDS_CONN_ERROR)
+ return;
+ /* No current cp_state writer leaves a
+ * DISCONNECTING path in any state but
+ * RDS_CONN_ERROR; report loudly if one ever
+ * does.
*/
rds_conn_path_error(cp, "%s: failed to transition "
"to state DOWN, current state "
diff --git a/net/rds/ib_recv.c b/net/rds/ib_recv.c
index 357128d34a54..bd6cb3ffaa57 100644
--- a/net/rds/ib_recv.c
+++ b/net/rds/ib_recv.c
@@ -363,15 +363,14 @@ static int acquire_refill(struct rds_connection *conn)
static void release_refill(struct rds_connection *conn)
{
- clear_bit(RDS_RECV_REFILL, &conn->c_flags);
- smp_mb__after_atomic();
+ clear_bit_unlock(RDS_RECV_REFILL, &conn->c_flags);
/* We don't use wait_on_bit()/wake_up_bit() because our waking is in a
* hot path and finding waiters is very rare. We don't want to walk
* the system-wide hashed waitqueue buckets in the fast path only to
* almost never find waiters.
*/
- if (waitqueue_active(&conn->c_waitq))
+ if (wq_has_sleeper(&conn->c_waitq))
wake_up_all(&conn->c_waitq);
}
@@ -392,7 +391,9 @@ void rds_ib_recv_refill(struct rds_connection *conn, int prefill, gfp_t gfp)
/* the goal here is to just make sure that someone, somewhere
* is posting buffers. If we can't get the refill lock,
- * let them do their thing
+ * let them do their thing. The holder may also be
+ * rds_conn_shutdown() tearing the path down, in which case
+ * there is nothing to post.
*/
if (!acquire_refill(conn))
return;
diff --git a/net/rds/message.c b/net/rds/message.c
index 7feb0eb6537d..9cbf045fe003 100644
--- a/net/rds/message.c
+++ b/net/rds/message.c
@@ -405,7 +405,9 @@ struct rds_message *rds_message_map_pages(unsigned long *page_addrs, unsigned in
for (i = 0; i < rm->data.op_nents; ++i) {
sg_set_page(&rm->data.op_sg[i],
virt_to_page((void *)page_addrs[i]),
- PAGE_SIZE, 0);
+ i == rm->data.op_nents - 1
+ ? total_len - (i * PAGE_SIZE)
+ : PAGE_SIZE, 0);
}
return rm;
diff --git a/net/rds/send.c b/net/rds/send.c
index 68be1bf0e0ad..ffb3dd999838 100644
--- a/net/rds/send.c
+++ b/net/rds/send.c
@@ -114,8 +114,13 @@ static void release_in_xmit(struct rds_conn_path *cp)
* hot path and finding waiters is very rare. We don't want to walk
* the system-wide hashed waitqueue buckets in the fast path only to
* almost never find waiters.
+ *
+ * wq_has_sleeper() supplies the full barrier that orders the wait
+ * queue read after the bit clear; clear_bit_unlock() alone is only
+ * a release and would let this check read a stale empty queue,
+ * losing the wake-up.
*/
- if (waitqueue_active(&cp->cp_waitq))
+ if (wq_has_sleeper(&cp->cp_waitq))
wake_up_all(&cp->cp_waitq);
}
@@ -231,8 +236,11 @@ int rds_send_xmit(struct rds_conn_path *cp)
WRITE_ONCE(cp->cp_send_gen, send_gen);
/*
- * rds_conn_shutdown() sets the conn state and then tests RDS_IN_XMIT,
- * we do the opposite to avoid races.
+ * rds_conn_shutdown() sets the conn state and then acquires
+ * RDS_IN_XMIT; we take the lock first and then check the state.
+ * Ownership is decided by the atomic RMW on the cp_flags word:
+ * if the teardown won the bit we back off here, and if we won
+ * it the teardown waits until we release it.
*/
if (!rds_conn_path_up(cp)) {
release_in_xmit(cp);
diff --git a/net/rds/tcp.c b/net/rds/tcp.c
index b263634ac750..774a71f88d37 100644
--- a/net/rds/tcp.c
+++ b/net/rds/tcp.c
@@ -115,46 +115,90 @@ void rds_tcp_restore_callbacks(struct socket *sock,
}
/*
- * rds_tcp_reset_callbacks() switches the to the new sock and
- * returns the existing tc->t_sock.
+ * rds_tcp_reset_callbacks() switches a path to a new socket and
+ * releases the old one it finds in tc->t_sock, resolving a duelling
+ * SYN.
*
- * The only functions that set tc->t_sock are rds_tcp_set_callbacks
- * and rds_tcp_reset_callbacks. Send and receive trust that
- * it is set. The absence of RDS_CONN_UP bit protects those paths
- * from being called while it isn't set.
+ * tc->t_sock is set by rds_tcp_set_callbacks() and cleared by
+ * rds_tcp_restore_callbacks(). Four paths write it: the active
+ * connect in rds_tcp_conn_path_connect(), which sets it and clears it
+ * again on failure; the accept path in rds_tcp_accept_one(), which
+ * sets it for a path with no socket yet; the teardown in
+ * rds_tcp_conn_path_shutdown(), which clears it; and the swap done
+ * here, which does both. The connect and accept paths are serialized
+ * against each other by t_conn_path_lock. Send and receive trust
+ * that it is set: the absence of RDS_CONN_UP protects those paths
+ * from being called while it isn't, and the swap done here runs under
+ * RDS_IN_XMIT so that it cannot interleave with a sender already
+ * inside rds_send_xmit().
*/
void rds_tcp_reset_callbacks(struct socket *sock,
struct rds_conn_path *cp)
{
struct rds_tcp_connection *tc = cp->cp_transport_data;
- struct socket *osock = tc->t_sock;
-
- if (!osock)
- goto newsock;
+ struct socket *osock;
/* Need to resolve a duelling SYN between peers.
* We have an outstanding SYN to this peer, which may
* potentially have transitioned to the RDS_CONN_UP state,
* so we must quiesce any send threads before resetting
- * cp_transport_data. We quiesce these threads by setting
- * cp_state to something other than RDS_CONN_UP, and then
- * waiting for any existing threads in rds_send_xmit to
- * complete release_in_xmit(). (Subsequent threads entering
- * rds_send_xmit() will bail on !rds_conn_up().
+ * cp_transport_data. Setting cp_state to something other
+ * than RDS_CONN_UP stops new senders, and owning RDS_IN_XMIT
+ * excludes any thread already inside rds_send_xmit() - or a
+ * teardown in rds_conn_shutdown(), which holds the same lock
+ * for the duration of the transport shutdown - for the whole
+ * socket swap and the rds_send_path_reset() below.
*
- * However an incoming syn-ack at this point would end up
- * marking the conn as RDS_CONN_UP, and would again permit
- * rds_send_xmi() threads through, so ideally we would
- * synchronize on RDS_CONN_UP after lock_sock(), but cannot
- * do that: waiting on !RDS_IN_XMIT after lock_sock() may
- * end up deadlocking with tcp_sendmsg(), and the RDS_IN_XMIT
- * would not get set. As a result, we set c_state to
- * RDS_CONN_RESETTTING, to ensure that rds_tcp_state_change
- * cannot mark rds_conn_path_up() in the window before lock_sock()
+ * An incoming syn-ack at this point would end up marking the
+ * conn as RDS_CONN_UP, and would again permit rds_send_xmit()
+ * threads through, so ideally we would synchronize on
+ * RDS_CONN_UP after lock_sock(), but cannot do that: acquiring
+ * RDS_IN_XMIT after lock_sock() may end up deadlocking with
+ * tcp_sendmsg(), which takes the socket lock while holding
+ * RDS_IN_XMIT. As a result, we set c_state to
+ * RDS_CONN_RESETTING, to ensure that rds_tcp_state_change
+ * cannot mark rds_conn_path_up() in the window before
+ * lock_sock().
+ *
+ * Only make that transition if the path is still connecting
+ * (or already resetting from an earlier duel). A path in any
+ * other state - typically RDS_CONN_DISCONNECTING or
+ * RDS_CONN_ERROR with a shutdown in flight - is dropped
+ * instead. That still replaces its state, with RDS_CONN_ERROR,
+ * and, unless a pending destroy is about to reap the whole
+ * connection anyway, queues one more shutdown pass. A shutdown
+ * already in flight leaves that RDS_CONN_ERROR alone when it
+ * finishes; the queued pass then completes the transition to
+ * RDS_CONN_DOWN and tears down anything that attached to the
+ * path in the meantime.
+ */
+ if (!rds_conn_path_transition(cp, RDS_CONN_CONNECTING,
+ RDS_CONN_RESETTING) &&
+ !rds_conn_path_transition(cp, RDS_CONN_RESETTING,
+ RDS_CONN_RESETTING))
+ rds_conn_path_drop(cp, 0);
+ wait_event(cp->cp_waitq,
+ !test_and_set_bit_lock(RDS_IN_XMIT, &cp->cp_flags));
+
+ /* Read t_sock only while owning RDS_IN_XMIT, never before the
+ * wait: the teardown in rds_conn_shutdown() releases the old
+ * socket and clears t_sock, so a pointer sampled earlier can
+ * be stale by the time we wake up. The teardown holds the
+ * same lock while it does so, so what we read here cannot
+ * change under us until we release it.
+ */
+ osock = tc->t_sock;
+ if (!osock)
+ goto newsock;
+
+ /* reset receive side state for rds_tcp_data_recv() for osock.
+ *
+ * The sync cancels while owning RDS_IN_XMIT rely on cp_wq
+ * being ordered: a teardown blocked on the bit occupies
+ * cp_wq's only execution slot, so cp_send_w and cp_recv_w are
+ * pending at most and the cancels never flush. Nothing here
+ * may flush or wait on cp_wq itself.
*/
- atomic_set(&cp->cp_state, RDS_CONN_RESETTING);
- wait_event(cp->cp_waitq, !test_bit(RDS_IN_XMIT, &cp->cp_flags));
- /* reset receive side state for rds_tcp_data_recv() for osock */
cancel_delayed_work_sync(&cp->cp_send_w);
cancel_delayed_work_sync(&cp->cp_recv_w);
lock_sock(osock->sk);
@@ -172,6 +216,9 @@ void rds_tcp_reset_callbacks(struct socket *sock,
lock_sock(sock->sk);
rds_tcp_set_callbacks(sock, cp);
release_sock(sock->sk);
+
+ clear_bit_unlock(RDS_IN_XMIT, &cp->cp_flags);
+ wake_up_all(&cp->cp_waitq);
}
/* Add tc to rds_tcp_tc_list and set tc->t_sock. See comments
diff --git a/net/rds/tcp_listen.c b/net/rds/tcp_listen.c
index a3db9b057084..13fa60c1985b 100644
--- a/net/rds/tcp_listen.c
+++ b/net/rds/tcp_listen.c
@@ -295,7 +295,11 @@ int rds_tcp_accept_one(struct rds_tcp_net *rtn)
if (rs_tcp->t_sock) {
/* Duelling SYN has been handled in rds_tcp_accept_one() */
rds_tcp_reset_callbacks(new_sock, cp);
- /* rds_connect_path_complete() marks RDS_CONN_UP */
+ /* rds_connect_path_complete() marks RDS_CONN_UP, or,
+ * if a concurrent shutdown won the duel, drops the
+ * path again and the pass that drop queues reaps the
+ * socket installed above.
+ */
rds_connect_path_complete(cp, RDS_CONN_RESETTING);
} else {
rds_tcp_set_callbacks(new_sock, cp);
diff --git a/net/sched/act_api.c b/net/sched/act_api.c
index b4415d358c91..19501dc99464 100644
--- a/net/sched/act_api.c
+++ b/net/sched/act_api.c
@@ -443,12 +443,21 @@ static size_t tcf_action_shared_attrs_size(const struct tc_action *act)
+ nla_total_size(IFNAMSIZ) /* TCA_ACT_KIND */
+ cookie_len /* TCA_ACT_COOKIE */
+ nla_total_size(sizeof(struct nla_bitfield32)) /* TCA_ACT_HW_STATS */
+ /* TCA_ACT_USED_HW_STATS */
+ + nla_total_size(sizeof(struct nla_bitfield32))
+ + nla_total_size(sizeof(u32)) /* TCA_ACT_IN_HW_COUNT */
+ nla_total_size(0) /* TCA_ACT_STATS nested */
+ nla_total_size(sizeof(struct nla_bitfield32)) /* TCA_ACT_FLAGS */
/* TCA_STATS_BASIC */
+ nla_total_size_64bit(sizeof(struct gnet_stats_basic))
- /* TCA_STATS_PKT64 */
- + nla_total_size_64bit(sizeof(u64))
+ /* TCA_STATS_BASIC_HW */
+ + nla_total_size_64bit(sizeof(struct gnet_stats_basic))
+ /* TCA_STATS_PKT64, emitted by both of the basic copies above */
+ + 2 * nla_total_size_64bit(sizeof(u64))
+ /* TCA_STATS_RATE_EST */
+ + nla_total_size_64bit(sizeof(struct gnet_stats_rate_est))
+ /* TCA_STATS_RATE_EST64 */
+ + nla_total_size_64bit(sizeof(struct gnet_stats_rate_est64))
/* TCA_STATS_QUEUE */
+ nla_total_size_64bit(sizeof(struct gnet_stats_queue))
+ nla_total_size(0) /* TCA_ACT_OPTIONS nested */
@@ -1191,18 +1200,13 @@ EXPORT_SYMBOL(tcf_action_exec);
int tcf_action_destroy(struct tc_action *actions[], int bind)
{
- const struct tc_action_ops *ops;
struct tc_action *a;
int ret = 0, i;
tcf_act_for_each_action(i, a, actions) {
actions[i] = NULL;
- ops = a->ops;
- ret = __tcf_idr_release(a, bind, true);
- if (ret == ACT_P_DELETED)
- module_put(ops->owner);
- else if (ret < 0)
- return ret;
+ /* Drop our reference even if the action is still bound to a filter. */
+ ret = tcf_idr_release(a, bind);
}
return ret;
}
@@ -1688,12 +1692,12 @@ static int tca_get_fill(struct sk_buff *skb, struct tc_action *actions[],
static int
tcf_get_notify(struct net *net, u32 portid, struct nlmsghdr *n,
- struct tc_action *actions[], int event,
+ struct tc_action *actions[], size_t attr_size, int event,
struct netlink_ext_ack *extack)
{
struct sk_buff *skb;
- skb = alloc_skb(NLMSG_GOODSIZE, GFP_KERNEL);
+ skb = alloc_skb(max(attr_size, NLMSG_GOODSIZE), GFP_KERNEL);
if (!skb)
return -ENOBUFS;
if (tca_get_fill(skb, actions, portid, n->nlmsg_seq, 0, event,
@@ -1858,11 +1862,13 @@ static int tcf_action_delete(struct net *net, struct tc_action *actions[])
static struct sk_buff *tcf_reoffload_del_notify_msg(struct net *net,
struct tc_action *action)
{
- size_t attr_size = tcf_action_fill_size(action);
struct tc_action *actions[TCA_ACT_MAX_PRIO] = {
[0] = action,
};
struct sk_buff *skb;
+ size_t attr_size;
+
+ attr_size = tcf_action_full_attrs_size(tcf_action_fill_size(action));
skb = alloc_skb(max(attr_size, NLMSG_GOODSIZE), GFP_KERNEL);
if (!skb)
@@ -1879,15 +1885,18 @@ static struct sk_buff *tcf_reoffload_del_notify_msg(struct net *net,
static int tcf_reoffload_del_notify(struct net *net, struct tc_action *action)
{
const struct tc_action_ops *ops = action->ops;
- struct sk_buff *skb;
+ struct sk_buff *skb = NULL;
int ret;
- if (!rtnl_notify_needed(net, 0, RTNLGRP_TC)) {
- skb = NULL;
- } else {
+ if (rtnl_notify_needed(net, 0, RTNLGRP_TC)) {
skb = tcf_reoffload_del_notify_msg(net, action);
+ /* The action has already lost its hardware instance and is
+ * skip_sw, so it must be released whether or not the
+ * notification can be built. Drop the notification rather
+ * than leave an action behind that processes no packets.
+ */
if (IS_ERR(skb))
- return PTR_ERR(skb);
+ skb = NULL;
}
ret = tcf_idr_release_unsafe(action);
@@ -2044,7 +2053,8 @@ tca_action_gd(struct net *net, struct nlattr *nla, struct nlmsghdr *n,
attr_size = tcf_action_full_attrs_size(attr_size);
if (event == RTM_GETACTION)
- ret = tcf_get_notify(net, portid, n, actions, event, extack);
+ ret = tcf_get_notify(net, portid, n, actions, attr_size, event,
+ extack);
else { /* delete */
ret = tcf_del_notify(net, n, actions, portid, attr_size, extack);
if (ret)
diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c
index 9966766661d5..c47d2ee13641 100644
--- a/net/sched/cls_api.c
+++ b/net/sched/cls_api.c
@@ -3254,7 +3254,7 @@ static int tc_ctl_chain(struct sk_buff *skb, struct nlmsghdr *n,
tcf_chain_put(chain);
errout_block:
tcf_block_release(q, block, true);
- if (err == -EAGAIN)
+ if (err == -EAGAIN && n->nlmsg_type == RTM_NEWCHAIN)
/* Replay the request. */
goto replay;
return err;
diff --git a/net/sched/cls_route.c b/net/sched/cls_route.c
index 0d1324c90583..0f211f030fd9 100644
--- a/net/sched/cls_route.c
+++ b/net/sched/cls_route.c
@@ -11,6 +11,7 @@
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/errno.h>
+#include <linux/refcount.h>
#include <linux/skbuff.h>
#include <net/dst.h>
#include <net/route.h>
@@ -41,6 +42,7 @@ struct route4_head {
struct route4_bucket {
/* 16 FROM buckets + 16 IIF buckets + 1 wildcard bucket */
struct route4_filter __rcu *ht[16 + 16 + 1];
+ refcount_t filters_ref;
struct rcu_head rcu;
};
@@ -336,7 +338,7 @@ static int route4_delete(struct tcf_proto *tp, void *arg, bool *last,
struct route4_filter *nf;
struct route4_bucket *b;
unsigned int h = 0;
- int i, h1;
+ int h1;
if (!head || !f)
return -EINVAL;
@@ -362,23 +364,14 @@ static int route4_delete(struct tcf_proto *tp, void *arg, bool *last,
tcf_exts_get_net(&f->exts);
tcf_queue_work(&f->rwork, route4_delete_filter_work);
- /* Strip RTNL protected tree */
- for (i = 0; i <= 32; i++) {
- struct route4_filter *rt;
-
- rt = rtnl_dereference(b->ht[i]);
- if (rt)
- goto out;
+ if (refcount_dec_and_test(&b->filters_ref)) {
+ RCU_INIT_POINTER(head->table[to_hash(h)], NULL);
+ kfree_rcu(b, rcu);
}
-
- /* OK, session has no flows */
- RCU_INIT_POINTER(head->table[to_hash(h)], NULL);
- kfree_rcu(b, rcu);
break;
}
}
-out:
*last = true;
for (h1 = 0; h1 <= 256; h1++) {
if (rcu_access_pointer(head->table[h1])) {
@@ -400,8 +393,9 @@ static const struct nla_policy route4_policy[TCA_ROUTE4_MAX + 1] = {
static int route4_set_parms(struct net *net, struct tcf_proto *tp,
unsigned long base, struct route4_filter *f,
u32 handle, struct route4_head *head,
- struct nlattr **tb, struct nlattr *est, int new,
- u32 flags, struct netlink_ext_ack *extack)
+ struct nlattr **tb, struct nlattr *est,
+ struct route4_filter *fold, u32 flags,
+ struct netlink_ext_ack *extack)
{
u32 id = 0, to = 0, nhandle = 0x8000;
struct route4_filter *fp;
@@ -414,7 +408,7 @@ static int route4_set_parms(struct net *net, struct tcf_proto *tp,
return err;
if (tb[TCA_ROUTE4_TO]) {
- if (new && handle & 0x8000) {
+ if (!fold && handle & 0x8000) {
NL_SET_ERR_MSG(extack, "Invalid handle");
return -EINVAL;
}
@@ -437,14 +431,14 @@ static int route4_set_parms(struct net *net, struct tcf_proto *tp,
} else
nhandle |= 0xFFFF << 16;
- if (handle && new) {
+ if (handle && (!fold || nhandle == (handle & ~0x7F00)))
nhandle |= handle & 0x7F00;
- if (nhandle != handle) {
- NL_SET_ERR_MSG_FMT(extack,
- "Handle mismatch constructed: %x (expected: %x)",
- handle, nhandle);
- return -EINVAL;
- }
+
+ if (handle && !fold && nhandle != handle) {
+ NL_SET_ERR_MSG_FMT(extack,
+ "Handle mismatch constructed: %x (expected: %x)",
+ handle, nhandle);
+ return -EINVAL;
}
if (!nhandle) {
@@ -459,6 +453,7 @@ static int route4_set_parms(struct net *net, struct tcf_proto *tp,
if (b == NULL)
return -ENOBUFS;
+ refcount_set(&b->filters_ref, 1);
rcu_assign_pointer(head->table[h1], b);
} else {
unsigned int h2 = from_hash(nhandle >> 16);
@@ -466,8 +461,14 @@ static int route4_set_parms(struct net *net, struct tcf_proto *tp,
for (fp = rtnl_dereference(b->ht[h2]);
fp;
fp = rtnl_dereference(fp->next))
- if (fp->handle == f->handle)
+ if (fp != fold && fp->handle == nhandle) {
+ NL_SET_ERR_MSG_FMT(extack,
+ "Handle %x is already in use",
+ nhandle);
return -EEXIST;
+ }
+
+ refcount_inc(&b->filters_ref);
}
if (tb[TCA_ROUTE4_TO])
@@ -500,9 +501,8 @@ static int route4_change(struct net *net, struct sk_buff *in_skb,
struct route4_filter *fold, *f1, *pfp, *f = NULL;
struct route4_bucket *b;
struct nlattr *tb[TCA_ROUTE4_MAX + 1];
- unsigned int h, th;
+ unsigned int h;
int err;
- bool new = true;
if (!handle) {
NL_SET_ERR_MSG(extack, "Creating with handle of 0 is invalid");
@@ -539,11 +539,10 @@ static int route4_change(struct net *net, struct sk_buff *in_skb,
f->tp = fold->tp;
f->bkt = fold->bkt;
- new = false;
}
err = route4_set_parms(net, tp, base, f, handle, head, tb,
- tca[TCA_RATE], new, flags, extack);
+ tca[TCA_RATE], fold, flags, extack);
if (err < 0)
goto errout;
@@ -560,17 +559,20 @@ static int route4_change(struct net *net, struct sk_buff *in_skb,
rcu_assign_pointer(*fp, f);
if (fold) {
- th = to_hash(fold->handle);
+ b = fold->bkt;
h = from_hash(fold->handle >> 16);
- b = rtnl_dereference(head->table[th]);
- if (b) {
- fp = &b->ht[h];
- for (pfp = rtnl_dereference(*fp); pfp;
- fp = &pfp->next, pfp = rtnl_dereference(*fp)) {
- if (pfp == fold) {
- rcu_assign_pointer(*fp, fold->next);
- break;
+ fp = &b->ht[h];
+ for (pfp = rtnl_dereference(*fp); pfp;
+ fp = &pfp->next, pfp = rtnl_dereference(*fp)) {
+ if (pfp == fold) {
+ rcu_assign_pointer(*fp, fold->next);
+ if (refcount_dec_and_test(&b->filters_ref)) {
+ unsigned int th = to_hash(fold->handle);
+
+ RCU_INIT_POINTER(head->table[th], NULL);
+ kfree_rcu(b, rcu);
}
+ break;
}
}
}
diff --git a/net/sched/cls_u32.c b/net/sched/cls_u32.c
index ac6d0fa5a40e..a3e65c8cf29e 100644
--- a/net/sched/cls_u32.c
+++ b/net/sched/cls_u32.c
@@ -370,6 +370,10 @@ static int u32_init(struct tcf_proto *tp)
refcount_set(&root_ht->refcnt, 1);
root_ht->handle = tp_c ? gen_new_htid(tp_c, root_ht) : id2handle(0);
+ if (root_ht->handle == 0) {
+ kfree(root_ht);
+ return -ENOMEM;
+ }
root_ht->prio = tp->prio;
root_ht->is_root = true;
idr_init(&root_ht->handle_idr);
@@ -695,21 +699,33 @@ static int u32_delete(struct tcf_proto *tp, void *arg, bool *last,
return ret;
}
-static u32 gen_new_kid(struct tc_u_hnode *ht, u32 htid)
+static u32 gen_new_kid(struct tc_u_hnode *ht, u32 htid, int *err)
{
u32 index = htid | 0x800;
u32 max = htid | 0xFFF;
+ *err = 0;
+
if (idr_alloc_u32(&ht->handle_idr, NULL, &index, max, GFP_KERNEL)) {
index = htid + 1;
- if (idr_alloc_u32(&ht->handle_idr, NULL, &index, max,
- GFP_KERNEL))
- index = max;
+ *err = idr_alloc_u32(&ht->handle_idr, NULL, &index, max,
+ GFP_KERNEL);
+ if (*err)
+ return 0;
}
return index;
}
+static int u32_kid_extack(int err, struct netlink_ext_ack *extack)
+{
+ if (err == -ENOSPC)
+ NL_SET_ERR_MSG_MOD(extack, "Hash table node ID pool exhausted");
+ else
+ NL_SET_ERR_MSG_MOD(extack, "Failed to allocate node ID");
+ return err;
+}
+
static const struct nla_policy u32_policy[TCA_U32_MAX + 1] = {
[TCA_U32_CLASSID] = { .type = NLA_U32 },
[TCA_U32_HASH] = { .type = NLA_U32 },
@@ -1079,7 +1095,9 @@ static int u32_change(struct net *net, struct sk_buff *in_skb,
* handle which is used to uniquely identify the match entry.
*/
if (!TC_U32_NODE(handle)) {
- handle = gen_new_kid(ht, htid);
+ handle = gen_new_kid(ht, htid, &err);
+ if (err)
+ return u32_kid_extack(err, extack);
} else {
handle = htid | TC_U32_NODE(handle);
err = idr_alloc_u32(&ht->handle_idr, NULL, &handle,
@@ -1091,7 +1109,9 @@ static int u32_change(struct net *net, struct sk_buff *in_skb,
/* The user did not give us a handle; lets just generate one
* from the table's pool of nodeids.
*/
- handle = gen_new_kid(ht, htid);
+ handle = gen_new_kid(ht, htid, &err);
+ if (err)
+ return u32_kid_extack(err, extack);
}
if (tb[TCA_U32_SEL] == NULL) {
diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c
index 90503e59e6e3..463ededcdcfe 100644
--- a/net/sched/sch_api.c
+++ b/net/sched/sch_api.c
@@ -1385,7 +1385,7 @@ static struct Qdisc *qdisc_create(struct net_device *dev,
err_out3:
qdisc_lock_uninit(sch, ops);
netdev_put(dev, &sch->dev_tracker);
- qdisc_free(sch);
+ qdisc_free_rcu(sch);
err_out2:
bpf_module_put(ops, ops->owner);
err_out:
diff --git a/net/sched/sch_drr.c b/net/sched/sch_drr.c
index 91b1ef824afa..8621d057edd9 100644
--- a/net/sched/sch_drr.c
+++ b/net/sched/sch_drr.c
@@ -82,8 +82,9 @@ static int drr_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
NL_SET_ERR_MSG(extack, "Specified DRR quantum cannot be zero");
return -EINVAL;
}
+ quantum = clamp_t(u32, quantum, 256, 1 << 20);
} else
- quantum = psched_mtu(qdisc_dev(sch));
+ quantum = clamp_t(u32, (u32)psched_mtu(qdisc_dev(sch)), 256, 1 << 20);
if (cl != NULL) {
if (tca[TCA_RATE]) {
diff --git a/net/sched/sch_dualpi2.c b/net/sched/sch_dualpi2.c
index 4f678d4ff10e..4947def7c49e 100644
--- a/net/sched/sch_dualpi2.c
+++ b/net/sched/sch_dualpi2.c
@@ -208,9 +208,11 @@ static void dualpi2_reset_c_protection(struct dualpi2_sched_data *q)
static void dualpi2_calculate_c_protection(struct Qdisc *sch,
struct dualpi2_sched_data *q, u32 wc)
{
+ u32 mtu = clamp_t(u32, psched_mtu(qdisc_dev(sch)), 1, 1 << 20);
+
q->c_protection_wc = wc;
q->c_protection_wl = MAX_WC - wc;
- q->c_protection_init = (s32)psched_mtu(qdisc_dev(sch)) *
+ q->c_protection_init = (s32)mtu *
((int)q->c_protection_wc - (int)q->c_protection_wl);
dualpi2_reset_c_protection(q);
}
@@ -285,8 +287,9 @@ static bool must_drop(struct Qdisc *sch, struct dualpi2_sched_data *q,
u64 local_l_prob;
bool overload;
u32 prob;
+ u32 mtu = clamp_t(u32, psched_mtu(qdisc_dev(sch)), 1, 1 << 20);
- if (sch->qstats.backlog < 2 * psched_mtu(qdisc_dev(sch)))
+ if (sch->qstats.backlog < 2 * mtu)
return false;
prob = READ_ONCE(q->pi2_prob);
@@ -712,7 +715,8 @@ static u32 get_memory_limit(struct Qdisc *sch, u32 limit)
/* Apply rule of thumb, i.e., doubling the packet length,
* to further include per packet overhead in memory_limit.
*/
- u64 memlim = mul_u32_u32(limit, 2 * psched_mtu(qdisc_dev(sch)));
+ u64 memlim = mul_u32_u32(limit, 2 * clamp_t(u32, psched_mtu(qdisc_dev(sch)),
+ 1, 1 << 20));
if (upper_32_bits(memlim))
return U32_MAX;
diff --git a/net/sched/sch_ets.c b/net/sched/sch_ets.c
index 25fcf4079fec..6cc902a03838 100644
--- a/net/sched/sch_ets.c
+++ b/net/sched/sch_ets.c
@@ -83,11 +83,7 @@ static int ets_quantum_parse(struct Qdisc *sch, const struct nlattr *attr,
unsigned int *quantum,
struct netlink_ext_ack *extack)
{
- *quantum = nla_get_u32(attr);
- if (!*quantum) {
- NL_SET_ERR_MSG(extack, "ETS quantum cannot be zero");
- return -EINVAL;
- }
+ *quantum = clamp_t(u32, nla_get_u32(attr), 256, 1 << 20);
return 0;
}
@@ -632,11 +628,13 @@ static int ets_qdisc_change(struct Qdisc *sch, struct nlattr *opt,
return err;
}
/* If there are more bands than strict + quanta provided, the remaining
- * ones are ETS with quantum of MTU. Initialize the missing values here.
+ * ones are ETS with quantum of max(MTU, 256). Initialize the missing
+ * values here.
*/
for (i = nstrict; i < nbands; i++) {
if (!quanta[i])
- quanta[i] = psched_mtu(qdisc_dev(sch));
+ quanta[i] = clamp_t(u32, (u32)psched_mtu(qdisc_dev(sch)),
+ 256, 1 << 20);
}
/* Before commit, make sure we can allocate all new qdiscs */
diff --git a/net/sched/sch_fq_pie.c b/net/sched/sch_fq_pie.c
index b27d95418707..5982847df8f8 100644
--- a/net/sched/sch_fq_pie.c
+++ b/net/sched/sch_fq_pie.c
@@ -341,7 +341,8 @@ static int fq_pie_change(struct Qdisc *sch, struct nlattr *opt,
nla_get_u32(tb[TCA_FQ_PIE_BETA]));
if (tb[TCA_FQ_PIE_QUANTUM])
- WRITE_ONCE(q->quantum, nla_get_u32(tb[TCA_FQ_PIE_QUANTUM]));
+ WRITE_ONCE(q->quantum,
+ max(256U, nla_get_u32(tb[TCA_FQ_PIE_QUANTUM])));
if (tb[TCA_FQ_PIE_MEMORY_LIMIT])
WRITE_ONCE(q->memory_limit,
diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
index 4539dc2c6d38..6f6a6f0d5eb0 100644
--- a/net/sched/sch_generic.c
+++ b/net/sched/sch_generic.c
@@ -1086,21 +1086,21 @@ void qdisc_reset(struct Qdisc *qdisc)
}
EXPORT_SYMBOL(qdisc_reset);
-void qdisc_free(struct Qdisc *qdisc)
+static void qdisc_free_cb(struct rcu_head *head)
{
- if (qdisc_is_percpu_stats(qdisc)) {
- free_percpu(qdisc->cpu_bstats);
- free_percpu(qdisc->cpu_qstats);
+ struct Qdisc *q = container_of(head, struct Qdisc, rcu);
+
+ if (qdisc_is_percpu_stats(q)) {
+ free_percpu(q->cpu_bstats);
+ free_percpu(q->cpu_qstats);
}
- kfree(qdisc);
+ kfree(q);
}
-static void qdisc_free_cb(struct rcu_head *head)
+void qdisc_free_rcu(struct Qdisc *qdisc)
{
- struct Qdisc *q = container_of(head, struct Qdisc, rcu);
-
- qdisc_free(q);
+ call_rcu(&qdisc->rcu, qdisc_free_cb);
}
static void __qdisc_destroy(struct Qdisc *qdisc)
@@ -1127,7 +1127,7 @@ static void __qdisc_destroy(struct Qdisc *qdisc)
trace_qdisc_destroy(qdisc);
- call_rcu(&qdisc->rcu, qdisc_free_cb);
+ qdisc_free_rcu(qdisc);
}
void qdisc_destroy(struct Qdisc *qdisc)
diff --git a/net/sched/sch_hhf.c b/net/sched/sch_hhf.c
index 96acab6a8da0..fc72f825fbd9 100644
--- a/net/sched/sch_hhf.c
+++ b/net/sched/sch_hhf.c
@@ -551,7 +551,7 @@ static int hhf_change(struct Qdisc *sch, struct nlattr *opt,
return err;
if (tb[TCA_HHF_QUANTUM])
- new_quantum = nla_get_u32(tb[TCA_HHF_QUANTUM]);
+ new_quantum = max(256U, nla_get_u32(tb[TCA_HHF_QUANTUM]));
if (tb[TCA_HHF_NON_HH_WEIGHT])
new_hhf_non_hh_weight = nla_get_u32(tb[TCA_HHF_NON_HH_WEIGHT]);
@@ -613,7 +613,7 @@ static int hhf_init(struct Qdisc *sch, struct nlattr *opt,
int i;
sch->limit = 1000;
- q->quantum = psched_mtu(qdisc_dev(sch));
+ q->quantum = clamp_t(u32, psched_mtu(qdisc_dev(sch)), 256, 1 << 20);
get_random_bytes(&q->perturbation, sizeof(q->perturbation));
INIT_LIST_HEAD(&q->new_buckets);
INIT_LIST_HEAD(&q->old_buckets);
@@ -624,10 +624,6 @@ static int hhf_init(struct Qdisc *sch, struct nlattr *opt,
q->hhf_evict_timeout = HZ; /* 1 sec */
q->hhf_non_hh_weight = 2;
- if ((int)q->quantum <= 0 ||
- (u64)q->quantum * q->hhf_non_hh_weight > INT_MAX)
- q->quantum = 256;
-
if (opt) {
int err = hhf_change(sch, opt, extack);
diff --git a/net/sched/sch_pie.c b/net/sched/sch_pie.c
index b41f2def2e2c..3b7863ffd284 100644
--- a/net/sched/sch_pie.c
+++ b/net/sched/sch_pie.c
@@ -35,7 +35,7 @@ bool pie_drop_early(struct Qdisc *sch, struct pie_params *params,
{
u64 rnd;
u64 local_prob = vars->prob;
- u32 mtu = psched_mtu(qdisc_dev(sch));
+ u32 mtu = clamp_t(u32, psched_mtu(qdisc_dev(sch)), 1, 1 << 20);
/* If there is still burst allowance left skip random early drop */
if (vars->burst_time > 0)
diff --git a/net/sched/sch_sfq.c b/net/sched/sch_sfq.c
index 187d3ed578f2..8bbcfc9e85d9 100644
--- a/net/sched/sch_sfq.c
+++ b/net/sched/sch_sfq.c
@@ -660,6 +660,11 @@ static int sfq_change(struct Qdisc *sch, struct nlattr *opt,
return -EINVAL;
}
+ if (ctl->quantum > 1 << 20) {
+ NL_SET_ERR_MSG_MOD(extack, "quantum too large");
+ return -EINVAL;
+ }
+
if (ctl->perturb_period < 0 ||
ctl->perturb_period > INT_MAX / HZ) {
NL_SET_ERR_MSG_MOD(extack, "invalid perturb period");
@@ -688,7 +693,7 @@ static int sfq_change(struct Qdisc *sch, struct nlattr *opt,
/* update and validate configuration */
if (ctl->quantum)
- quantum = ctl->quantum;
+ quantum = max(256U, ctl->quantum);
if (ctl->flows)
maxflows = min_t(u32, ctl->flows, SFQ_MAX_FLOWS);
if (ctl->divisor) {
diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c
index 236e25abc7a4..84a4c97d0f75 100644
--- a/net/sctp/sm_make_chunk.c
+++ b/net/sctp/sm_make_chunk.c
@@ -3215,6 +3215,9 @@ bool sctp_verify_asconf(const struct sctp_association *asoc,
*errp = param.p;
switch (param.p->type) {
case SCTP_PARAM_ERR_CAUSE:
+ if (length < sizeof(struct sctp_addip_param) +
+ sizeof(struct sctp_errhdr))
+ return false;
break;
case SCTP_PARAM_IPV4_ADDRESS:
if (length != sizeof(struct sctp_ipv4addr_param))
@@ -3448,20 +3451,15 @@ static __be16 sctp_get_asconf_response(struct sctp_chunk *asconf_ack,
case SCTP_PARAM_ERR_CAUSE:
length = sizeof(*asconf_ack_param);
err_param = (void *)asconf_ack_param + length;
- asconf_ack_len -= length;
- if (asconf_ack_len > 0)
- return err_param->cause;
- else
- return SCTP_ERROR_INV_PARAM;
- break;
+ return err_param->cause;
default:
return SCTP_ERROR_INV_PARAM;
}
}
length = ntohs(asconf_ack_param->param_hdr.length);
- asconf_ack_param = (void *)asconf_ack_param + length;
- asconf_ack_len -= length;
+ asconf_ack_param = (void *)asconf_ack_param + SCTP_PAD4(length);
+ asconf_ack_len -= SCTP_PAD4(length);
}
return err_code;
diff --git a/net/sctp/sm_sideeffect.c b/net/sctp/sm_sideeffect.c
index 94716406d602..0d99b7e8c082 100644
--- a/net/sctp/sm_sideeffect.c
+++ b/net/sctp/sm_sideeffect.c
@@ -1545,17 +1545,8 @@ static int sctp_cmd_interpreter(enum sctp_event_type event_type,
timeout = asoc->timeouts[cmd->obj.to];
BUG_ON(!timeout);
- /*
- * SCTP has a hard time with timer starts. Because we process
- * timer starts as side effects, it can be hard to tell if we
- * have already started a timer or not, which leads to BUG
- * halts when we call add_timer. So here, instead of just starting
- * a timer, if the timer is already started, and just mod
- * the timer with the shorter of the two expiration times
- */
- if (!timer_pending(timer))
+ if (!timer_reduce(timer, jiffies + timeout))
sctp_association_hold(asoc);
- timer_reduce(timer, jiffies + timeout);
break;
case SCTP_CMD_TIMER_RESTART:
diff --git a/net/tipc/link.c b/net/tipc/link.c
index 49dfc098d89b..6427c69f8929 100644
--- a/net/tipc/link.c
+++ b/net/tipc/link.c
@@ -504,7 +504,7 @@ bool tipc_link_create(struct net *net, char *if_name, int bearer_id,
snprintf(l->name, sizeof(l->name), "%s:%s-%s:unknown",
self_str, if_name, peer_str);
- strcpy(l->if_name, if_name);
+ strscpy(l->if_name, if_name);
l->addr = peer;
l->peer_caps = peer_caps;
l->net = net;
@@ -574,7 +574,7 @@ bool tipc_link_bc_create(struct net *net, u32 ownnode, u32 peer, u8 *peer_id,
snprintf(l->name, sizeof(l->name), "%s:%s", tipc_bclink_name,
peer_str);
} else {
- strcpy(l->name, tipc_bclink_name);
+ strscpy(l->name, tipc_bclink_name);
}
trace_tipc_link_reset(l, TIPC_DUMP_ALL, "bclink created!");
tipc_link_reset(l);
@@ -1898,7 +1898,7 @@ static void tipc_link_build_proto_msg(struct tipc_link *l, int mtyp, bool probe,
msg_set_dest_session(hdr, l->peer_session);
}
msg_set_max_pkt(hdr, l->advertised_mtu);
- strcpy(data, l->if_name);
+ memcpy(data, l->if_name, TIPC_MAX_IF_NAME);
msg_set_size(hdr, INT_H_SIZE + TIPC_MAX_IF_NAME);
skb_trim(skb, INT_H_SIZE + TIPC_MAX_IF_NAME);
}
diff --git a/net/tipc/name_table.c b/net/tipc/name_table.c
index 253c72d1366e..6fda36ab1766 100644
--- a/net/tipc/name_table.c
+++ b/net/tipc/name_table.c
@@ -763,21 +763,40 @@ struct publication *tipc_nametbl_publish(struct net *net, struct tipc_uaddr *ua,
struct tipc_socket_addr *sk, u32 key)
{
struct name_table *nt = tipc_name_table(net);
+ u32 max_user_pub = TIPC_MAX_PUBL - 1;
struct tipc_net *tn = tipc_net(net);
struct publication *p = NULL;
struct sk_buff *skb = NULL;
+ bool protocol_type = false;
u32 rc_dests;
+ if (ua->sr.type == TIPC_NODE_STATE || ua->sr.type == TIPC_LINK_STATE ||
+ ua->sr.type == TIPC_TOP_SRV)
+ protocol_type = true;
+
spin_lock_bh(&tn->nametbl_lock);
+ if (protocol_type)
+ goto insert;
- if (nt->local_publ_count >= TIPC_MAX_PUBL) {
- pr_warn("Bind failed, max limit %u reached\n", TIPC_MAX_PUBL);
+ /* Reserve one entry for node state service type because it has cluster
+ * scope and it is distributed in bulk. So, the maximum number of user's
+ * publications is (TIPC_MAX_PUBL - 1).
+ */
+ if (nt->local_publ_count >= max_user_pub) {
+ pr_warn("Bind failed, max limit %u reached\n", max_user_pub);
goto exit;
}
+insert:
p = tipc_nametbl_insert_publ(net, ua, sk, key);
if (p) {
- nt->local_publ_count++;
+ /* Not count node state, link state and topology server types
+ * so that maximum nt->local_publ_count does not prevent
+ * protocol service types from being inserted into the name
+ * table.
+ */
+ if (!protocol_type)
+ nt->local_publ_count++;
skb = tipc_named_publish(net, p);
}
rc_dests = nt->rc_dests;
@@ -810,7 +829,10 @@ void tipc_nametbl_withdraw(struct net *net, struct tipc_uaddr *ua,
p = tipc_nametbl_remove_publ(net, ua, sk, key);
if (p) {
- nt->local_publ_count--;
+ if (p->sr.type != TIPC_NODE_STATE &&
+ p->sr.type != TIPC_LINK_STATE &&
+ p->sr.type != TIPC_TOP_SRV)
+ nt->local_publ_count--;
skb = tipc_named_withdraw(net, p);
list_del_init(&p->binding_sock);
kfree_rcu(p, rcu);
diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c
index 10ed9421e43a..c7a03b01119d 100644
--- a/net/unix/af_unix.c
+++ b/net/unix/af_unix.c
@@ -2805,8 +2805,8 @@ static int unix_stream_recv_urg(struct unix_stream_read_state *state)
return 1;
}
-static struct sk_buff *manage_oob(struct sk_buff *skb, struct sock *sk,
- int flags, int copied)
+static struct sk_buff *manage_oob(struct sk_buff *skb, struct sk_buff **last,
+ struct sock *sk, int flags, int copied)
{
struct sk_buff *read_skb = NULL, *unread_skb = NULL;
struct unix_sock *u = unix_sk(sk);
@@ -2820,11 +2820,13 @@ static struct sk_buff *manage_oob(struct sk_buff *skb, struct sock *sk,
if (copied && (!u->oob_skb || skb == u->oob_skb)) {
skb = NULL;
} else if (flags & MSG_PEEK) {
+ *last = skb;
skb = skb_peek_next(skb, &sk->sk_receive_queue);
} else {
read_skb = skb;
skb = skb_peek_next(skb, &sk->sk_receive_queue);
__skb_unlink(read_skb, &sk->sk_receive_queue);
+ *last = skb;
}
if (!skb)
@@ -2843,8 +2845,10 @@ static struct sk_buff *manage_oob(struct sk_buff *skb, struct sock *sk,
__skb_unlink(skb, &sk->sk_receive_queue);
unread_skb = skb;
skb = skb_peek(&sk->sk_receive_queue);
+ *last = skb;
}
} else if (!sock_flag(sk, SOCK_URGINLINE)) {
+ *last = skb;
skb = skb_peek_next(skb, &sk->sk_receive_queue);
}
@@ -2964,8 +2968,8 @@ static int unix_stream_read_generic(struct unix_stream_read_state *state,
again:
#if IS_ENABLED(CONFIG_AF_UNIX_OOB)
if (skb) {
- skb = manage_oob(skb, sk, flags, copied);
- if (!skb && copied) {
+ skb = manage_oob(skb, &last, sk, flags, copied);
+ if (!skb && (copied || !state->size)) {
unix_state_unlock(sk);
break;
}
diff --git a/rust/bindings/lib.rs b/rust/bindings/lib.rs
index 812f8e5a08d5..439ab88a5da1 100644
--- a/rust/bindings/lib.rs
+++ b/rust/bindings/lib.rs
@@ -22,11 +22,13 @@
#![feature(cfi_encoding)]
#[allow(dead_code)]
+#[allow(clippy::as_underscore)]
#[allow(clippy::cast_lossless)]
#[allow(clippy::ptr_as_ptr)]
#[allow(clippy::ref_as_ptr)]
#[allow(clippy::undocumented_unsafe_blocks)]
-#[cfg_attr(CONFIG_RUSTC_HAS_UNNECESSARY_TRANSMUTES, allow(unnecessary_transmutes))]
+#[cfg_attr(not(CONFIG_RUSTC_HAS_UNNECESSARY_TRANSMUTES), allow(unknown_lints))]
+#[allow(unnecessary_transmutes)]
#[cfg_attr(
CONFIG_RUSTC_HAS_SUSPICIOUS_RUNTIME_SYMBOL_DEFINITIONS,
allow(suspicious_runtime_symbol_definitions)
diff --git a/rust/kernel/num.rs b/rust/kernel/num.rs
index 8532b511384c..ba1eb58aee02 100644
--- a/rust/kernel/num.rs
+++ b/rust/kernel/num.rs
@@ -13,9 +13,14 @@ pub enum Unsigned {}
/// Designates signed primitive types.
pub enum Signed {}
+mod private {
+ pub trait Sealed {}
+}
+
/// Describes core properties of integer types.
pub trait Integer:
- Sized
+ private::Sealed
+ + Sized
+ Copy
+ Clone
+ PartialEq
@@ -54,6 +59,8 @@ pub trait Integer:
macro_rules! impl_integer {
($($type:ty: $signedness:ty), *) => {
$(
+ impl private::Sealed for $type {}
+
impl Integer for $type {
type Signedness = $signedness;
diff --git a/rust/pin-init/src/lib.rs b/rust/pin-init/src/lib.rs
index fd40c8f244a1..27a6969fa29b 100644
--- a/rust/pin-init/src/lib.rs
+++ b/rust/pin-init/src/lib.rs
@@ -493,13 +493,7 @@ macro_rules! stack_pin_init {
(let $var:ident $(: $t:ty)? = $val:expr) => {
let val = $val;
let mut $var = ::core::pin::pin!($crate::__internal::StackInit$(::<$t>)?::uninit());
- let mut $var = match $crate::__internal::StackInit::init($var, val) {
- Ok(res) => res,
- Err(x) => {
- let x: ::core::convert::Infallible = x;
- match x {}
- }
- };
+ let Ok(mut $var) = $crate::__internal::StackInit::init($var, val);
};
}
diff --git a/rust/uapi/lib.rs b/rust/uapi/lib.rs
index 797ead5b5626..003e6d4f7c4b 100644
--- a/rust/uapi/lib.rs
+++ b/rust/uapi/lib.rs
@@ -10,6 +10,7 @@
#![no_std]
#![allow(
clippy::all,
+ clippy::as_underscore,
clippy::cast_lossless,
clippy::ptr_as_ptr,
clippy::ref_as_ptr,
@@ -23,7 +24,8 @@
unreachable_pub,
unsafe_op_in_unsafe_fn
)]
-#![cfg_attr(CONFIG_RUSTC_HAS_UNNECESSARY_TRANSMUTES, allow(unnecessary_transmutes))]
+#![cfg_attr(not(CONFIG_RUSTC_HAS_UNNECESSARY_TRANSMUTES), allow(unknown_lints))]
+#![allow(unnecessary_transmutes)]
#![cfg_attr(
CONFIG_RUSTC_HAS_SUSPICIOUS_RUNTIME_SYMBOL_DEFINITIONS,
allow(suspicious_runtime_symbol_definitions)
diff --git a/samples/rust/rust_print_main.rs b/samples/rust/rust_print_main.rs
index 682207c81fc2..01729e87d6b5 100644
--- a/samples/rust/rust_print_main.rs
+++ b/samples/rust/rust_print_main.rs
@@ -23,10 +23,10 @@ fn arc_print() -> Result {
let b = UniqueArc::new("hello, world", GFP_KERNEL)?;
// Prints the value of data in `a`.
- pr_info!("{}", a);
+ pr_info!("{}\n", a);
// Uses ":?" to print debug fmt of `b`.
- pr_info!("{:?}", b);
+ pr_info!("{:?}\n", b);
let a: Arc<&str> = b.into();
let c = a.clone();
@@ -42,7 +42,7 @@ fn arc_print() -> Result {
use kernel::fmt::Display;
fn arc_dyn_print(arc: &Arc<dyn Display>) {
- pr_info!("Arc<dyn Display> says {arc}");
+ pr_info!("Arc<dyn Display> says {arc}\n");
}
let a_i32_display: Arc<dyn Display> = Arc::new(42i32, GFP_KERNEL)?;
@@ -53,7 +53,7 @@ fn arc_dyn_print(arc: &Arc<dyn Display>) {
}
// Pretty-prints the debug formatting with lower-case hexadecimal integers.
- pr_info!("{:#x?}", a);
+ pr_info!("{:#x?}\n", a);
Ok(())
}
diff --git a/scripts/livepatch/init.c b/scripts/livepatch/init.c
index f14d8c8fb35f..16aff8f736eb 100644
--- a/scripts/livepatch/init.c
+++ b/scripts/livepatch/init.c
@@ -51,7 +51,7 @@ static int __init livepatch_mod_init(void)
if (!funcs) {
ret = -ENOMEM;
for (int j = 0; j < i; j++)
- kfree(objs[i].funcs);
+ kfree(objs[j].funcs);
goto err_free_objs;
}
diff --git a/scripts/mksysmap b/scripts/mksysmap
index c4531eacde20..856b26ba2ac0 100755
--- a/scripts/mksysmap
+++ b/scripts/mksysmap
@@ -35,7 +35,7 @@
/ __efistub_/d
# arm64 local symbols in PIE namespace
-/ __pi_\\$/d
+/ __pi_\$/d
/ __pi_\.L/d
# arm64 local symbols in non-VHE KVM namespace
@@ -83,7 +83,7 @@
/ _SDA2_BASE_$/d
# MODULE_INFO()
-/ __UNIQUE_ID_modinfo[0-9]*$/d
+/ __UNIQUE_ID_modinfo_[0-9]*$/d
# ---------------------------------------------------------------------------
# Ignored patterns
diff --git a/scripts/sorttable.c b/scripts/sorttable.c
index d8dc2a1b7c31..d7b50581c732 100644
--- a/scripts/sorttable.c
+++ b/scripts/sorttable.c
@@ -116,7 +116,7 @@ static inline void *get_index(void *start, int entsize, int index)
}
static int extable_ent_size;
-static int long_size;
+static int long_size __maybe_unused;
#define ERRSTR_MAXSZ 256
diff --git a/security/landlock/fs.c b/security/landlock/fs.c
index 8fc7f82a374a..f1c9effc93fe 100644
--- a/security/landlock/fs.c
+++ b/security/landlock/fs.c
@@ -1222,11 +1222,12 @@ static int current_check_refer_path(struct dentry *const old_dentry,
/*
* old_dentry may be the root of the common mount point and
* !IS_ROOT(old_dentry) at the same time (e.g. with open_tree() and
- * OPEN_TREE_CLONE). We do not need to call dget(old_parent) because
- * we keep a reference to old_dentry.
+ * OPEN_TREE_CLONE). Pin the dentry used as old_parent in either case.
+ * Otherwise, dget_parent() safely fetches and pins the current parent
+ * against a concurrent rename(2).
*/
- old_parent = (old_dentry == mnt_dir.dentry) ? old_dentry :
- old_dentry->d_parent;
+ old_parent = (old_dentry == mnt_dir.dentry) ? dget(old_dentry) :
+ dget_parent(old_dentry);
/* new_dir->dentry is equal to new_dentry->d_parent */
allow_parent1 = collect_domain_accesses(subject->domain, mnt_dir.dentry,
@@ -1235,8 +1236,10 @@ static int current_check_refer_path(struct dentry *const old_dentry,
allow_parent2 = collect_domain_accesses(subject->domain, mnt_dir.dentry,
new_dir->dentry,
&layer_masks_parent2);
- if (allow_parent1 && allow_parent2)
+ if (allow_parent1 && allow_parent2) {
+ dput(old_parent);
return 0;
+ }
/*
* To be able to compare source and destination domain access rights,
@@ -1248,8 +1251,10 @@ static int current_check_refer_path(struct dentry *const old_dentry,
subject->domain, &mnt_dir, access_request_parent1,
&layer_masks_parent1, &request1, old_dentry,
access_request_parent2, &layer_masks_parent2, &request2,
- exchange ? new_dentry : NULL))
+ exchange ? new_dentry : NULL)) {
+ dput(old_parent);
return 0;
+ }
if (request1.access) {
request1.audit.u.path.dentry = old_parent;
@@ -1259,6 +1264,7 @@ static int current_check_refer_path(struct dentry *const old_dentry,
request2.audit.u.path.dentry = new_dir->dentry;
landlock_log_denial(subject, &request2);
}
+ dput(old_parent);
/*
* This prioritizes EACCES over EXDEV for all actions, including
diff --git a/sound/core/pcm_native.c b/sound/core/pcm_native.c
index fa6723a6e827..2fcdde09a030 100644
--- a/sound/core/pcm_native.c
+++ b/sound/core/pcm_native.c
@@ -4019,20 +4019,33 @@ int snd_pcm_mmap_data(struct snd_pcm_substream *substream, struct file *file,
return -EINVAL;
}
runtime = substream->runtime;
- if (runtime->state == SNDRV_PCM_STATE_OPEN)
- return -EBADFD;
- if (!(runtime->info & SNDRV_PCM_INFO_MMAP))
- return -ENXIO;
+ /* don't race with buffer reallocation in hw_params/hw_free */
+ if (!atomic_inc_unless_negative(&runtime->buffer_accessing))
+ return -EBUSY;
+ if (runtime->state == SNDRV_PCM_STATE_OPEN) {
+ err = -EBADFD;
+ goto out;
+ }
+ if (!(runtime->info & SNDRV_PCM_INFO_MMAP)) {
+ err = -ENXIO;
+ goto out;
+ }
if (runtime->access == SNDRV_PCM_ACCESS_RW_INTERLEAVED ||
- runtime->access == SNDRV_PCM_ACCESS_RW_NONINTERLEAVED)
- return -EINVAL;
+ runtime->access == SNDRV_PCM_ACCESS_RW_NONINTERLEAVED) {
+ err = -EINVAL;
+ goto out;
+ }
size = area->vm_end - area->vm_start;
offset = area->vm_pgoff << PAGE_SHIFT;
dma_bytes = PAGE_ALIGN(runtime->dma_bytes);
- if ((size_t)size > dma_bytes)
- return -EINVAL;
- if (offset > dma_bytes - size)
- return -EINVAL;
+ if ((size_t)size > dma_bytes) {
+ err = -EINVAL;
+ goto out;
+ }
+ if (offset > dma_bytes - size) {
+ err = -EINVAL;
+ goto out;
+ }
area->vm_ops = &snd_pcm_vm_ops_data;
area->vm_private_data = substream;
@@ -4042,6 +4055,8 @@ int snd_pcm_mmap_data(struct snd_pcm_substream *substream, struct file *file,
err = snd_pcm_lib_default_mmap(substream, area);
if (!err)
atomic_inc(&substream->mmap_count);
+out:
+ atomic_dec(&runtime->buffer_accessing);
return err;
}
EXPORT_SYMBOL(snd_pcm_mmap_data);
diff --git a/sound/core/ump.c b/sound/core/ump.c
index 632c13baf21e..a85d1006e7a5 100644
--- a/sound/core/ump.c
+++ b/sound/core/ump.c
@@ -1333,6 +1333,8 @@ static void update_legacy_names(struct snd_ump_endpoint *ump)
{
struct snd_rawmidi *rmidi = ump->legacy_rmidi;
+ if (!rmidi)
+ return;
update_legacy_substreams(ump, rmidi, SNDRV_RAWMIDI_STREAM_INPUT);
update_legacy_substreams(ump, rmidi, SNDRV_RAWMIDI_STREAM_OUTPUT);
}
@@ -1341,6 +1343,8 @@ static void ump_legacy_set_rawmidi_name(struct snd_ump_endpoint *ump)
{
struct snd_rawmidi *rmidi = ump->legacy_rmidi;
+ if (!rmidi)
+ return;
snprintf(rmidi->name, sizeof(rmidi->name), "%.68s (MIDI 1.0)",
ump->core.name);
}
diff --git a/sound/drivers/dummy.c b/sound/drivers/dummy.c
index 41ceeafce824..21eb0d76d6f8 100644
--- a/sound/drivers/dummy.c
+++ b/sound/drivers/dummy.c
@@ -782,7 +782,7 @@ static int snd_dummy_capsrc_put(struct snd_kcontrol *kcontrol, struct snd_ctl_el
left = ucontrol->value.integer.value[0] & 1;
right = ucontrol->value.integer.value[1] & 1;
guard(spinlock_irq)(&dummy->mixer_lock);
- change = dummy->capture_source[addr][0] != left &&
+ change = dummy->capture_source[addr][0] != left ||
dummy->capture_source[addr][1] != right;
dummy->capture_source[addr][0] = left;
dummy->capture_source[addr][1] = right;
diff --git a/sound/hda/common/codec.c b/sound/hda/common/codec.c
index ef533770179b..33dbe21fa165 100644
--- a/sound/hda/common/codec.c
+++ b/sound/hda/common/codec.c
@@ -2271,6 +2271,7 @@ static int snd_hda_spdif_default_put(struct snd_kcontrol *kcontrol,
int idx = kcontrol->private_value;
struct hda_spdif_out *spdif;
hda_nid_t nid;
+ unsigned int old_status;
unsigned short val;
int change;
@@ -2279,6 +2280,7 @@ static int snd_hda_spdif_default_put(struct snd_kcontrol *kcontrol,
guard(mutex)(&codec->spdif_mutex);
spdif = snd_array_elem(&codec->spdif_out, idx);
nid = spdif->nid;
+ old_status = spdif->status;
spdif->status = ucontrol->value.iec958.status[0] |
((unsigned int)ucontrol->value.iec958.status[1] << 8) |
((unsigned int)ucontrol->value.iec958.status[2] << 16) |
@@ -2289,7 +2291,7 @@ static int snd_hda_spdif_default_put(struct snd_kcontrol *kcontrol,
spdif->ctls = val;
if (change && nid != (u16)-1)
set_dig_out_convert(codec, nid, val & 0xff, (val >> 8) & 0xff);
- return change;
+ return change || spdif->status != old_status;
}
#define snd_hda_spdif_out_switch_info snd_ctl_boolean_mono_info
diff --git a/sound/hda/core/device.c b/sound/hda/core/device.c
index 160c8d0453b0..ebaa106d0f01 100644
--- a/sound/hda/core/device.c
+++ b/sound/hda/core/device.c
@@ -404,6 +404,7 @@ static void setup_fg_nodes(struct hdac_device *codec)
*/
int snd_hdac_refresh_widgets(struct hdac_device *codec)
{
+ hda_nid_t fg = codec->afg ? codec->afg : codec->mfg;
hda_nid_t start_nid;
int nums, err = 0;
@@ -412,10 +413,10 @@ int snd_hdac_refresh_widgets(struct hdac_device *codec)
* widgets array.
*/
guard(mutex)(&codec->widget_lock);
- nums = snd_hdac_get_sub_nodes(codec, codec->afg, &start_nid);
+ nums = snd_hdac_get_sub_nodes(codec, fg, &start_nid);
if (!start_nid || nums <= 0 || nums >= 0xff) {
dev_err(&codec->dev, "cannot read sub nodes for FG 0x%02x\n",
- codec->afg);
+ fg);
return -EINVAL;
}
diff --git a/sound/pci/ctxfi/cthw20k2.c b/sound/pci/ctxfi/cthw20k2.c
index 07e1490a6d17..36066ffe8784 100644
--- a/sound/pci/ctxfi/cthw20k2.c
+++ b/sound/pci/ctxfi/cthw20k2.c
@@ -994,6 +994,7 @@ static int daio_mgr_dao_init(struct hw *hw, void *blk, unsigned int idx, unsigne
/* S/PDIF output */
switch ((conf & 0xf)) {
case 1:
+ case 9:
set_field(&ctl->txctl[idx], ATXCTL_NUC, 0);
break;
case 2:
diff --git a/sound/soc/amd/acp-da7219-max98357a.c b/sound/soc/amd/acp-da7219-max98357a.c
index af559653e625..1ac729a58bb4 100644
--- a/sound/soc/amd/acp-da7219-max98357a.c
+++ b/sound/soc/amd/acp-da7219-max98357a.c
@@ -17,6 +17,7 @@
#include <linux/i2c.h>
#include <linux/input.h>
#include <linux/acpi.h>
+#include <linux/pci.h>
#include "acp.h"
#include "../codecs/da7219.h"
@@ -742,6 +743,18 @@ static const struct regulator_desc acp_da7219_desc = {
.n_voltages = 1,
};
+/*
+ * The ACP3.x+ (Raven/Picasso and later) audio coprocessor is a dedicated PCI
+ * function. Carrizo/Stoney - the only platforms handled by this driver - reach
+ * the ACP through the GPU driver and have no such device.
+ */
+#define ACP3X_PCI_DEV_ID 0x15e2
+
+static const struct pci_device_id acp3x_pci_ids[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_AMD, ACP3X_PCI_DEV_ID) },
+ { 0, },
+};
+
static int cz_probe(struct platform_device *pdev)
{
int ret;
@@ -750,6 +763,16 @@ static int cz_probe(struct platform_device *pdev)
struct regulator_dev *rdev;
struct device *dev = &pdev->dev;
+ /*
+ * AMDI5682 is also matched by acp3x-alc5682-max98357 (Raven/Picasso).
+ * If the ACP3.x PCI function is present this is such a board; return
+ * -ENODEV so that driver binds instead.
+ */
+ if (pci_dev_present(acp3x_pci_ids)) {
+ dev_info(dev, "ACP3.x PCI device present, deferring to acp3x-alc5682-max98357\n");
+ return -ENODEV;
+ }
+
card = (struct snd_soc_card *)acp_soc_is_rltk_max(dev);
if (!card)
return -ENODEV;
diff --git a/sound/soc/amd/renoir/acp3x-pdm-dma.c b/sound/soc/amd/renoir/acp3x-pdm-dma.c
index e60e3821703c..3f59b753243d 100644
--- a/sound/soc/amd/renoir/acp3x-pdm-dma.c
+++ b/sound/soc/amd/renoir/acp3x-pdm-dma.c
@@ -104,7 +104,7 @@ static void disable_pdm_interrupts(void __iomem *acp_base)
u32 ext_int_ctrl;
ext_int_ctrl = rn_readl(acp_base + ACP_EXTERNAL_INTR_CNTL);
- ext_int_ctrl |= ~PDM_DMA_INTR_MASK;
+ ext_int_ctrl &= ~PDM_DMA_INTR_MASK;
rn_writel(ext_int_ctrl, acp_base + ACP_EXTERNAL_INTR_CNTL);
}
diff --git a/sound/soc/amd/yc/acp6x-pdm-dma.c b/sound/soc/amd/yc/acp6x-pdm-dma.c
index 710db721ffa4..40c4d833f4ed 100644
--- a/sound/soc/amd/yc/acp6x-pdm-dma.c
+++ b/sound/soc/amd/yc/acp6x-pdm-dma.c
@@ -275,9 +275,11 @@ static int acp6x_pdm_dma_close(struct snd_soc_component *component,
struct snd_pcm_substream *substream)
{
struct pdm_dev_data *adata = dev_get_drvdata(component->dev);
+ struct snd_pcm_runtime *runtime = substream->runtime;
acp6x_disable_pdm_interrupts(adata->acp6x_base);
adata->capture_stream = NULL;
+ kfree(runtime->private_data);
return 0;
}
diff --git a/sound/soc/bcm/bcm63xx-i2s-whistler.c b/sound/soc/bcm/bcm63xx-i2s-whistler.c
index c47ed1e6ea2b..14d111fe29d8 100644
--- a/sound/soc/bcm/bcm63xx-i2s-whistler.c
+++ b/sound/soc/bcm/bcm63xx-i2s-whistler.c
@@ -285,6 +285,7 @@ static const struct of_device_id snd_soc_bcm_audio_match[] = {
{.compatible = "brcm,bcm63xx-i2s"},
{ }
};
+MODULE_DEVICE_TABLE(of, snd_soc_bcm_audio_match);
#endif
static struct platform_driver bcm63xx_i2s_driver = {
diff --git a/sound/soc/codecs/ab8500-codec.c b/sound/soc/codecs/ab8500-codec.c
index 6e8ef9cd1b31..e1a0e35836e6 100644
--- a/sound/soc/codecs/ab8500-codec.c
+++ b/sound/soc/codecs/ab8500-codec.c
@@ -20,10 +20,8 @@
#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/init.h>
-#include <linux/delay.h>
#include <linux/pm.h>
#include <linux/platform_device.h>
-#include <linux/mutex.h>
#include <linux/mfd/abx500/ab8500.h>
#include <linux/mfd/abx500.h>
#include <linux/mfd/abx500/ab8500-sysctrl.h>
@@ -53,32 +51,9 @@
/* Macrocell register definitions */
#define AB8500_GPIO_DIR4_REG 0x13 /* Bank AB8500_MISC */
-/* Nr of FIR/IIR-coeff banks in ANC-block */
-#define AB8500_NR_OF_ANC_COEFF_BANKS 2
-
-/* Minimum duration to keep ANC IIR Init bit high or
-low before proceeding with the configuration sequence */
-#define AB8500_ANC_SM_DELAY 2000
-
-/* Sidetone states */
-static const char * const enum_sid_state[] = {
- "Unconfigured",
- "Apply FIR",
- "FIR is configured",
-};
-enum sid_state {
- SID_UNCONFIGURED = 0,
- SID_APPLY_FIR = 1,
- SID_FIR_CONFIGURED = 2,
-};
-
/* Private data for AB8500 device-driver */
struct ab8500_codec_drvdata {
struct regmap *regmap;
- struct mutex ctrl_lock;
-
- /* Sidetone */
- enum sid_state sid_status;
};
static inline const char *amic_micbias_str(enum amic_micbias micbias)
@@ -258,7 +233,7 @@ static const struct snd_kcontrol_new dapm_anc_in_select[] = {
/* ANC - Enable/Disable */
static const struct snd_kcontrol_new dapm_anc_enable[] = {
SOC_DAPM_SINGLE("Switch", AB8500_ANCCONF1,
- AB8500_ANCCONF1_ENANC, 0, 0),
+ AB8500_ANCCONF1_ENANC, 1, 0),
};
/* ANC to Earpiece - Mute */
@@ -340,12 +315,6 @@ static const struct snd_soc_dapm_widget ab8500_dapm_widgets[] = {
/* DA/AD */
- SND_SOC_DAPM_INPUT("ADC Input"),
- SND_SOC_DAPM_ADC("ADC", "ab8500_0c", SND_SOC_NOPM, 0, 0),
-
- SND_SOC_DAPM_DAC("DAC", NULL, SND_SOC_NOPM, 0, 0),
- SND_SOC_DAPM_OUTPUT("DAC Output"),
-
SND_SOC_DAPM_AIF_IN("DA_IN1", NULL, 0, SND_SOC_NOPM, 0, 0),
SND_SOC_DAPM_AIF_IN("DA_IN2", NULL, 0, SND_SOC_NOPM, 0, 0),
SND_SOC_DAPM_AIF_IN("DA_IN3", NULL, 0, SND_SOC_NOPM, 0, 0),
@@ -537,9 +506,8 @@ static const struct snd_soc_dapm_widget ab8500_dapm_widgets[] = {
SND_SOC_DAPM_MIXER("AD3 Channel Volume",
SND_SOC_NOPM, 0, 0,
NULL, 0),
- SND_SOC_DAPM_MIXER("AD3 Enable",
- AB8500_ADPATHENA, AB8500_ADPATHENA_ENAD34, 0,
- NULL, 0),
+ SND_SOC_DAPM_SUPPLY("AD34 Enable", AB8500_ADPATHENA,
+ AB8500_ADPATHENA_ENAD34, 0, NULL, 0),
/* Mic 2 */
@@ -598,9 +566,8 @@ static const struct snd_soc_dapm_widget ab8500_dapm_widgets[] = {
SND_SOC_NOPM, 0, 0,
NULL, 0),
- SND_SOC_DAPM_MIXER("AD12 Enable",
- AB8500_ADPATHENA, AB8500_ADPATHENA_ENAD12, 0,
- NULL, 0),
+ SND_SOC_DAPM_SUPPLY("AD12 Enable", AB8500_ADPATHENA,
+ AB8500_ADPATHENA_ENAD12, 0, NULL, 0),
/* HD Capture path */
@@ -614,12 +581,8 @@ static const struct snd_soc_dapm_widget ab8500_dapm_widgets[] = {
SND_SOC_DAPM_MIXER("AD6 Channel Volume",
SND_SOC_NOPM, 0, 0,
NULL, 0),
- SND_SOC_DAPM_MIXER("AD57 Enable",
- AB8500_ADPATHENA, AB8500_ADPATHENA_ENAD5768, 0,
- NULL, 0),
- SND_SOC_DAPM_MIXER("AD68 Enable",
- AB8500_ADPATHENA, AB8500_ADPATHENA_ENAD5768, 0,
- NULL, 0),
+ SND_SOC_DAPM_SUPPLY("AD5768 Enable", AB8500_ADPATHENA,
+ AB8500_ADPATHENA_ENAD5768, 0, NULL, 0),
/* Digital Microphone path */
@@ -651,15 +614,8 @@ static const struct snd_soc_dapm_widget ab8500_dapm_widgets[] = {
SND_SOC_DAPM_MIXER("AD4 Channel Volume",
SND_SOC_NOPM, 0, 0,
NULL, 0),
- SND_SOC_DAPM_MIXER("AD4 Enable",
- AB8500_ADPATHENA, AB8500_ADPATHENA_ENAD34,
- 0, NULL, 0),
-
/* Acoustical Noise Cancellation path */
- SND_SOC_DAPM_INPUT("ANC Configure Input"),
- SND_SOC_DAPM_OUTPUT("ANC Configure Output"),
-
SND_SOC_DAPM_MUX("ANC Source",
SND_SOC_NOPM, 0, 0,
dapm_anc_in_select),
@@ -702,24 +658,13 @@ static const struct snd_soc_dapm_route ab8500_dapm_routes[] = {
{"Main Supply", NULL, "Audio Power"},
{"Main Supply", NULL, "Audio Analog Power"},
- {"DAC", NULL, "ab8500_0p"},
- {"DAC", NULL, "Main Supply"},
- {"ADC", NULL, "ab8500_0c"},
- {"ADC", NULL, "Main Supply"},
-
- /* ANC Configure */
- {"ANC Configure Input", NULL, "Main Supply"},
- {"ANC Configure Output", NULL, "ANC Configure Input"},
-
- /* AD/DA */
- {"ADC", NULL, "ADC Input"},
- {"DAC Output", NULL, "DAC"},
-
/* Powerup charge pump if DA1/2 is in use */
{"DA_IN1", NULL, "ab8500_0p"},
+ {"DA_IN1", NULL, "Main Supply"},
{"DA_IN1", NULL, "Charge Pump"},
{"DA_IN2", NULL, "ab8500_0p"},
+ {"DA_IN2", NULL, "Main Supply"},
{"DA_IN2", NULL, "Charge Pump"},
/* Headset path */
@@ -754,8 +699,10 @@ static const struct snd_soc_dapm_route ab8500_dapm_routes[] = {
/* HF or LineOut path */
{"DA_IN3", NULL, "ab8500_0p"},
+ {"DA_IN3", NULL, "Main Supply"},
{"DA3 Channel Volume", NULL, "DA_IN3"},
{"DA_IN4", NULL, "ab8500_0p"},
+ {"DA_IN4", NULL, "Main Supply"},
{"DA4 Channel Volume", NULL, "DA_IN4"},
{"Speaker Left Source", "Audio Path", "DA3 Channel Volume"},
@@ -813,8 +760,10 @@ static const struct snd_soc_dapm_route ab8500_dapm_routes[] = {
/* Vibrator path */
{"DA_IN5", NULL, "ab8500_0p"},
+ {"DA_IN5", NULL, "Main Supply"},
{"DA5 Channel Volume", NULL, "DA_IN5"},
{"DA_IN6", NULL, "ab8500_0p"},
+ {"DA_IN6", NULL, "Main Supply"},
{"DA6 Channel Volume", NULL, "DA_IN6"},
{"VIB1 DAC", NULL, "DA5 Channel Volume"},
@@ -856,13 +805,15 @@ static const struct snd_soc_dapm_route ab8500_dapm_routes[] = {
{"AD1 Channel Volume", NULL, "AD1 Source Select"},
{"AD2 Channel Volume", NULL, "AD2 Source Select"},
- {"AD12 Enable", NULL, "AD1 Channel Volume"},
- {"AD12 Enable", NULL, "AD2 Channel Volume"},
+ {"AD1 Channel Volume", NULL, "AD12 Enable"},
+ {"AD2 Channel Volume", NULL, "AD12 Enable"},
- {"AD_OUT1", NULL, "ab8500_0c"},
- {"AD_OUT1", NULL, "AD12 Enable"},
- {"AD_OUT2", NULL, "ab8500_0c"},
- {"AD_OUT2", NULL, "AD12 Enable"},
+ {"ab8500_0c", NULL, "AD_OUT1"},
+ {"AD_OUT1", NULL, "Main Supply"},
+ {"AD_OUT1", NULL, "AD1 Channel Volume"},
+ {"ab8500_0c", NULL, "AD_OUT2"},
+ {"AD_OUT2", NULL, "Main Supply"},
+ {"AD_OUT2", NULL, "AD2 Channel Volume"},
/* Mic 1 */
@@ -879,11 +830,11 @@ static const struct snd_soc_dapm_route ab8500_dapm_routes[] = {
{"AD3 Source Select", "Mic 1", "MIC1 ADC"},
{"AD3 Channel Volume", NULL, "AD3 Source Select"},
+ {"AD3 Channel Volume", NULL, "AD34 Enable"},
- {"AD3 Enable", NULL, "AD3 Channel Volume"},
-
- {"AD_OUT3", NULL, "ab8500_0c"},
- {"AD_OUT3", NULL, "AD3 Enable"},
+ {"ab8500_0c", NULL, "AD_OUT3"},
+ {"AD_OUT3", NULL, "Main Supply"},
+ {"AD_OUT3", NULL, "AD3 Channel Volume"},
/* HD Capture path */
@@ -892,14 +843,15 @@ static const struct snd_soc_dapm_route ab8500_dapm_routes[] = {
{"AD5 Channel Volume", NULL, "AD5 Source Select"},
{"AD6 Channel Volume", NULL, "AD6 Source Select"},
+ {"AD5 Channel Volume", NULL, "AD5768 Enable"},
+ {"AD6 Channel Volume", NULL, "AD5768 Enable"},
- {"AD57 Enable", NULL, "AD5 Channel Volume"},
- {"AD68 Enable", NULL, "AD6 Channel Volume"},
-
- {"AD_OUT57", NULL, "ab8500_0c"},
- {"AD_OUT57", NULL, "AD57 Enable"},
- {"AD_OUT68", NULL, "ab8500_0c"},
- {"AD_OUT68", NULL, "AD68 Enable"},
+ {"ab8500_0c", NULL, "AD_OUT57"},
+ {"AD_OUT57", NULL, "Main Supply"},
+ {"AD_OUT57", NULL, "AD5 Channel Volume"},
+ {"ab8500_0c", NULL, "AD_OUT68"},
+ {"AD_OUT68", NULL, "Main Supply"},
+ {"AD_OUT68", NULL, "AD6 Channel Volume"},
/* Digital Microphone path */
@@ -910,17 +862,25 @@ static const struct snd_soc_dapm_route ab8500_dapm_routes[] = {
{"DMic 5", NULL, "V-DMIC"},
{"DMic 6", NULL, "V-DMIC"},
- {"AD1 Source Select", NULL, "DMic 1"},
- {"AD2 Source Select", NULL, "DMic 2"},
- {"AD3 Source Select", NULL, "DMic 3"},
- {"AD5 Source Select", NULL, "DMic 5"},
- {"AD6 Source Select", NULL, "DMic 6"},
+ {"DMIC1", NULL, "DMic 1"},
+ {"DMIC2", NULL, "DMic 2"},
+ {"DMIC3", NULL, "DMic 3"},
+ {"DMIC4", NULL, "DMic 4"},
+ {"DMIC5", NULL, "DMic 5"},
+ {"DMIC6", NULL, "DMic 6"},
- {"AD4 Channel Volume", NULL, "DMic 4"},
- {"AD4 Enable", NULL, "AD4 Channel Volume"},
+ {"AD1 Source Select", "DMic 1", "DMIC1"},
+ {"AD2 Source Select", "DMic 2", "DMIC2"},
+ {"AD3 Source Select", "DMic 3", "DMIC3"},
+ {"AD5 Source Select", "DMic 5", "DMIC5"},
+ {"AD6 Source Select", "DMic 6", "DMIC6"},
- {"AD_OUT4", NULL, "ab8500_0c"},
- {"AD_OUT4", NULL, "AD4 Enable"},
+ {"AD4 Channel Volume", NULL, "DMIC4"},
+ {"AD4 Channel Volume", NULL, "AD34 Enable"},
+
+ {"ab8500_0c", NULL, "AD_OUT4"},
+ {"AD_OUT4", NULL, "Main Supply"},
+ {"AD_OUT4", NULL, "AD4 Channel Volume"},
/* LineIn Bypass path */
@@ -945,13 +905,13 @@ static const struct snd_soc_dapm_route ab8500_dapm_routes[] = {
/* Sidetone Filter path */
- {"Sidetone Left Source", "LineIn Left", "AD12 Enable"},
- {"Sidetone Left Source", "LineIn Right", "AD12 Enable"},
- {"Sidetone Left Source", "Mic 1", "AD3 Enable"},
+ {"Sidetone Left Source", "LineIn Left", "AD1 Channel Volume"},
+ {"Sidetone Left Source", "LineIn Right", "AD2 Channel Volume"},
+ {"Sidetone Left Source", "Mic 1", "AD3 Channel Volume"},
{"Sidetone Left Source", "Headset Left", "DA_IN1"},
- {"Sidetone Right Source", "LineIn Right", "AD12 Enable"},
- {"Sidetone Right Source", "Mic 1", "AD3 Enable"},
- {"Sidetone Right Source", "DMic 4", "AD4 Enable"},
+ {"Sidetone Right Source", "LineIn Right", "AD2 Channel Volume"},
+ {"Sidetone Right Source", "Mic 1", "AD3 Channel Volume"},
+ {"Sidetone Right Source", "DMic 4", "AD4 Channel Volume"},
{"Sidetone Right Source", "Headset Right", "DA_IN2"},
{"STFIR1 Control", NULL, "Sidetone Left Source"},
@@ -979,78 +939,6 @@ static const struct snd_soc_dapm_route ab8500_dapm_routes_mic2_vamicx[] = {
{"MIC2 V-AMICx Enable", NULL, "V-AMIC2"},
};
-/*
- * Control-events
- */
-
-static int sid_status_control_get(struct snd_kcontrol *kcontrol,
- struct snd_ctl_elem_value *ucontrol)
-{
- struct snd_soc_component *component = snd_kcontrol_chip(kcontrol);
- struct ab8500_codec_drvdata *drvdata = dev_get_drvdata(component->dev);
-
- mutex_lock(&drvdata->ctrl_lock);
- ucontrol->value.enumerated.item[0] = drvdata->sid_status;
- mutex_unlock(&drvdata->ctrl_lock);
-
- return 0;
-}
-
-/* Write sidetone FIR-coefficients configuration sequence */
-static int sid_status_control_put(struct snd_kcontrol *kcontrol,
- struct snd_ctl_elem_value *ucontrol)
-{
- struct snd_soc_component *component = snd_kcontrol_chip(kcontrol);
- struct ab8500_codec_drvdata *drvdata = dev_get_drvdata(component->dev);
- unsigned int param, sidconf;
- int status = 1;
-
- dev_dbg(component->dev, "%s: Enter\n", __func__);
-
- if (ucontrol->value.enumerated.item[0] != SID_APPLY_FIR) {
- dev_err(component->dev,
- "%s: ERROR: This control supports '%s' only!\n",
- __func__, enum_sid_state[SID_APPLY_FIR]);
- return -EIO;
- }
-
- mutex_lock(&drvdata->ctrl_lock);
-
- sidconf = snd_soc_component_read(component, AB8500_SIDFIRCONF);
- if (((sidconf & BIT(AB8500_SIDFIRCONF_FIRSIDBUSY)) != 0)) {
- if ((sidconf & BIT(AB8500_SIDFIRCONF_ENFIRSIDS)) == 0) {
- dev_err(component->dev, "%s: Sidetone busy while off!\n",
- __func__);
- status = -EPERM;
- } else {
- status = -EBUSY;
- }
- goto out;
- }
-
- snd_soc_component_write(component, AB8500_SIDFIRADR, 0);
-
- for (param = 0; param < AB8500_SID_FIR_COEFFS; param++) {
- snd_soc_component_write(component, AB8500_SIDFIRCOEF1, 0);
- snd_soc_component_write(component, AB8500_SIDFIRCOEF2, 0);
- }
-
- snd_soc_component_update_bits(component, AB8500_SIDFIRADR,
- BIT(AB8500_SIDFIRADR_FIRSIDSET),
- BIT(AB8500_SIDFIRADR_FIRSIDSET));
- snd_soc_component_update_bits(component, AB8500_SIDFIRADR,
- BIT(AB8500_SIDFIRADR_FIRSIDSET), 0);
-
- drvdata->sid_status = SID_FIR_CONFIGURED;
-
-out:
- mutex_unlock(&drvdata->ctrl_lock);
-
- dev_dbg(component->dev, "%s: Exit\n", __func__);
-
- return status;
-}
-
/*
* Controls - Non-DAPM ASoC
*/
@@ -1334,9 +1222,6 @@ static SOC_ENUM_SINGLE_DECL(soc_enum_bfifomast,
AB8500_FIFOCONF3, AB8500_FIFOCONF3_BFIFOMAST_SHIFT,
enum_slavemaster);
-/* Sidetone */
-static SOC_ENUM_SINGLE_EXT_DECL(soc_enum_sidstate, enum_sid_state);
-
/* ANC */
static struct snd_kcontrol_new ab8500_ctrls[] = {
@@ -1627,8 +1512,6 @@ static struct snd_kcontrol_new ab8500_ctrls[] = {
AB8500_ANC_WARP_DELAY_MIN, AB8500_ANC_WARP_DELAY_MAX, 0),
/* Sidetone */
- SOC_ENUM_EXT("Sidetone Status", soc_enum_sidstate,
- sid_status_control_get, sid_status_control_put),
SOC_SINGLE_STROBE("Sidetone Reset",
AB8500_SIDFIRADR, AB8500_SIDFIRADR_FIRSIDSET, 0),
};
@@ -1640,23 +1523,25 @@ static struct snd_kcontrol_new ab8500_ctrls[] = {
static int ab8500_audio_init_audioblock(struct snd_soc_component *component)
{
int status;
+ u8 mask = AB8500_STW4500CTRL3_CLK32KOUT2DIS |
+ AB8500_STW4500CTRL3_RESETAUDN;
dev_dbg(component->dev, "%s: Enter.\n", __func__);
- /* Reset audio-registers and disable 32kHz-clock output 2 */
- status = ab8500_sysctrl_write(AB8500_STW4500CTRL3,
- AB8500_STW4500CTRL3_CLK32KOUT2DIS |
- AB8500_STW4500CTRL3_RESETAUDN,
- AB8500_STW4500CTRL3_RESETAUDN);
+ /* Reset the audio registers and disable the unused 32 kHz output. */
+ status = ab8500_sysctrl_write(AB8500_STW4500CTRL3, mask,
+ AB8500_STW4500CTRL3_CLK32KOUT2DIS);
if (status < 0)
return status;
- return 0;
+ return ab8500_sysctrl_write(AB8500_STW4500CTRL3, mask, mask);
}
static int ab8500_audio_setup_mics(struct snd_soc_component *component,
struct amic_settings *amics)
{
+ struct device *dev = component->dev;
+ struct ab8500 *ab8500 = dev_get_drvdata(dev->parent);
struct snd_soc_dapm_context *dapm = snd_soc_component_to_dapm(component);
u8 value8;
unsigned int value;
@@ -1665,20 +1550,21 @@ static int ab8500_audio_setup_mics(struct snd_soc_component *component,
dev_dbg(component->dev, "%s: Enter.\n", __func__);
- /* Set DMic-clocks to outputs */
- status = abx500_get_register_interruptible(component->dev, AB8500_MISC,
- AB8500_GPIO_DIR4_REG,
- &value8);
- if (status < 0)
- return status;
- value = value8 | GPIO27_DIR_OUTPUT | GPIO29_DIR_OUTPUT |
- GPIO31_DIR_OUTPUT;
- status = abx500_set_register_interruptible(component->dev,
- AB8500_MISC,
- AB8500_GPIO_DIR4_REG,
- value);
- if (status < 0)
- return status;
+ /* Set DMic-clocks to outputs; these GPIOs do not exist on AB8505. */
+ if (!is_ab8505(ab8500)) {
+ status = abx500_get_register_interruptible(dev, AB8500_MISC,
+ AB8500_GPIO_DIR4_REG,
+ &value8);
+ if (status < 0)
+ return status;
+ value = value8 | GPIO27_DIR_OUTPUT | GPIO29_DIR_OUTPUT |
+ GPIO31_DIR_OUTPUT;
+ status = abx500_set_register_interruptible(dev, AB8500_MISC,
+ AB8500_GPIO_DIR4_REG,
+ value);
+ if (status < 0)
+ return status;
+ }
/* Attach regulators to AMic DAPM-paths */
dev_dbg(component->dev, "%s: Mic 1a regulator: %s\n", __func__,
@@ -1747,149 +1633,91 @@ static int ab8500_audio_set_ear_cmv(struct snd_soc_component *component,
return 0;
}
-static int ab8500_audio_set_bit_delay(struct snd_soc_dai *dai,
- unsigned int delay)
-{
- unsigned int mask, val;
- struct snd_soc_component *component = dai->component;
-
- mask = BIT(AB8500_DIGIFCONF2_IF0DEL);
- val = 0;
-
- switch (delay) {
- case 0:
- break;
- case 1:
- val |= BIT(AB8500_DIGIFCONF2_IF0DEL);
- break;
- default:
- dev_err(dai->component->dev,
- "%s: ERROR: Unsupported bit-delay (0x%x)!\n",
- __func__, delay);
- return -EINVAL;
- }
-
- dev_dbg(dai->component->dev, "%s: IF0 Bit-delay: %d bits.\n",
- __func__, delay);
- snd_soc_component_update_bits(component, AB8500_DIGIFCONF2, mask, val);
-
- return 0;
-}
-
-/* Gates clocking according format mask */
-static int ab8500_codec_set_dai_clock_gate(struct snd_soc_component *component,
- unsigned int fmt)
-{
- unsigned int mask;
- unsigned int val;
-
- mask = BIT(AB8500_DIGIFCONF1_ENMASTGEN) |
- BIT(AB8500_DIGIFCONF1_ENFSBITCLK0);
-
- val = BIT(AB8500_DIGIFCONF1_ENMASTGEN);
-
- switch (fmt & SND_SOC_DAIFMT_CLOCK_MASK) {
- case SND_SOC_DAIFMT_CONT: /* continuous clock */
- dev_dbg(component->dev, "%s: IF0 Clock is continuous.\n",
- __func__);
- val |= BIT(AB8500_DIGIFCONF1_ENFSBITCLK0);
- break;
- case SND_SOC_DAIFMT_GATED: /* clock is gated */
- dev_dbg(component->dev, "%s: IF0 Clock is gated.\n",
- __func__);
- break;
- default:
- dev_err(component->dev,
- "%s: ERROR: Unsupported clock mask (0x%x)!\n",
- __func__, fmt & SND_SOC_DAIFMT_CLOCK_MASK);
- return -EINVAL;
- }
-
- snd_soc_component_update_bits(component, AB8500_DIGIFCONF1, mask, val);
-
- return 0;
-}
-
static int ab8500_codec_set_dai_fmt(struct snd_soc_dai *dai, unsigned int fmt)
{
- unsigned int mask;
- unsigned int val;
struct snd_soc_component *component = dai->component;
- int status;
+ unsigned int conf1_mask, conf1_val = 0;
+ unsigned int conf2_mask, conf2_val = 0;
+ unsigned int conf3_mask, conf3_val = 0;
+ bool provider = false;
+ int ret;
dev_dbg(component->dev, "%s: Enter (fmt = 0x%x)\n", __func__, fmt);
- mask = BIT(AB8500_DIGIFCONF3_IF1DATOIF0AD) |
+ conf3_mask = BIT(AB8500_DIGIFCONF3_IF1DATOIF0AD) |
BIT(AB8500_DIGIFCONF3_IF1CLKTOIF0CLK) |
BIT(AB8500_DIGIFCONF3_IF0BFIFOEN) |
BIT(AB8500_DIGIFCONF3_IF0MASTER);
- val = 0;
switch (fmt & SND_SOC_DAIFMT_CLOCK_PROVIDER_MASK) {
case SND_SOC_DAIFMT_CBP_CFP:
- dev_dbg(dai->component->dev,
+ dev_dbg(component->dev,
"%s: IF0 Master-mode: AB8500 provider.\n", __func__);
- val |= BIT(AB8500_DIGIFCONF3_IF0MASTER);
+ conf3_val |= BIT(AB8500_DIGIFCONF3_IF0MASTER);
+ provider = true;
break;
case SND_SOC_DAIFMT_CBC_CFC:
- dev_dbg(dai->component->dev,
+ dev_dbg(component->dev,
"%s: IF0 Master-mode: AB8500 consumer.\n", __func__);
break;
case SND_SOC_DAIFMT_CBC_CFP:
case SND_SOC_DAIFMT_CBP_CFC:
- dev_err(dai->component->dev,
+ dev_err(component->dev,
"%s: ERROR: The device is either a provider or a consumer.\n",
__func__);
fallthrough;
default:
- dev_err(dai->component->dev,
- "%s: ERROR: Unsupporter clocking mask 0x%x\n",
+ dev_err(component->dev,
+ "%s: ERROR: Unsupported clocking mask 0x%x\n",
__func__, fmt & SND_SOC_DAIFMT_CLOCK_PROVIDER_MASK);
return -EINVAL;
}
- snd_soc_component_update_bits(component, AB8500_DIGIFCONF3, mask, val);
-
- /* Set clock gating */
- status = ab8500_codec_set_dai_clock_gate(component, fmt);
- if (status) {
- dev_err(dai->component->dev,
- "%s: ERROR: Failed to set clock gate (%d).\n",
- __func__, status);
- return status;
+ conf1_mask = BIT(AB8500_DIGIFCONF1_ENMASTGEN) |
+ BIT(AB8500_DIGIFCONF1_ENFSBITCLK0);
+ switch (fmt & SND_SOC_DAIFMT_CLOCK_MASK) {
+ case SND_SOC_DAIFMT_CONT:
+ if (provider)
+ conf1_val = conf1_mask;
+ break;
+ case SND_SOC_DAIFMT_GATED:
+ if (provider)
+ conf1_val = BIT(AB8500_DIGIFCONF1_ENMASTGEN);
+ break;
+ default:
+ dev_err(component->dev, "%s: Unsupported clock mask 0x%x\n",
+ __func__, fmt & SND_SOC_DAIFMT_CLOCK_MASK);
+ return -EINVAL;
}
- /* Setting data transfer format */
-
- mask = BIT(AB8500_DIGIFCONF2_IF0FORMAT0) |
- BIT(AB8500_DIGIFCONF2_IF0FORMAT1) |
- BIT(AB8500_DIGIFCONF2_FSYNC0P) |
- BIT(AB8500_DIGIFCONF2_BITCLK0P);
- val = 0;
+ conf2_mask = BIT(AB8500_DIGIFCONF2_IF0FORMAT0) |
+ BIT(AB8500_DIGIFCONF2_IF0FORMAT1) |
+ BIT(AB8500_DIGIFCONF2_IF0DEL) |
+ BIT(AB8500_DIGIFCONF2_FSYNC0P) |
+ BIT(AB8500_DIGIFCONF2_BITCLK0P);
switch (fmt & SND_SOC_DAIFMT_FORMAT_MASK) {
case SND_SOC_DAIFMT_I2S: /* I2S mode */
- dev_dbg(dai->component->dev, "%s: IF0 Protocol: I2S\n", __func__);
- val |= BIT(AB8500_DIGIFCONF2_IF0FORMAT1);
- ab8500_audio_set_bit_delay(dai, 0);
+ dev_dbg(component->dev, "%s: IF0 Protocol: I2S\n", __func__);
+ conf2_val |= BIT(AB8500_DIGIFCONF2_IF0FORMAT1) |
+ BIT(AB8500_DIGIFCONF2_IF0DEL);
break;
case SND_SOC_DAIFMT_DSP_A: /* L data MSB after FRM LRC */
- dev_dbg(dai->component->dev,
+ dev_dbg(component->dev,
"%s: IF0 Protocol: DSP A (TDM)\n", __func__);
- val |= BIT(AB8500_DIGIFCONF2_IF0FORMAT0);
- ab8500_audio_set_bit_delay(dai, 1);
+ conf2_val |= BIT(AB8500_DIGIFCONF2_IF0FORMAT0) |
+ BIT(AB8500_DIGIFCONF2_IF0DEL);
break;
case SND_SOC_DAIFMT_DSP_B: /* L data MSB during FRM LRC */
- dev_dbg(dai->component->dev,
+ dev_dbg(component->dev,
"%s: IF0 Protocol: DSP B (TDM)\n", __func__);
- val |= BIT(AB8500_DIGIFCONF2_IF0FORMAT0);
- ab8500_audio_set_bit_delay(dai, 0);
+ conf2_val |= BIT(AB8500_DIGIFCONF2_IF0FORMAT0);
break;
default:
- dev_err(dai->component->dev,
+ dev_err(component->dev,
"%s: ERROR: Unsupported format (0x%x)!\n",
__func__, fmt & SND_SOC_DAIFMT_FORMAT_MASK);
return -EINVAL;
@@ -1897,39 +1725,50 @@ static int ab8500_codec_set_dai_fmt(struct snd_soc_dai *dai, unsigned int fmt)
switch (fmt & SND_SOC_DAIFMT_INV_MASK) {
case SND_SOC_DAIFMT_NB_NF: /* normal bit clock + frame */
- dev_dbg(dai->component->dev,
+ dev_dbg(component->dev,
"%s: IF0: Normal bit clock, normal frame\n",
__func__);
break;
case SND_SOC_DAIFMT_NB_IF: /* normal BCLK + inv FRM */
- dev_dbg(dai->component->dev,
+ dev_dbg(component->dev,
"%s: IF0: Normal bit clock, inverted frame\n",
__func__);
- val |= BIT(AB8500_DIGIFCONF2_FSYNC0P);
+ conf2_val |= BIT(AB8500_DIGIFCONF2_FSYNC0P);
break;
case SND_SOC_DAIFMT_IB_NF: /* invert BCLK + nor FRM */
- dev_dbg(dai->component->dev,
+ dev_dbg(component->dev,
"%s: IF0: Inverted bit clock, normal frame\n",
__func__);
- val |= BIT(AB8500_DIGIFCONF2_BITCLK0P);
+ conf2_val |= BIT(AB8500_DIGIFCONF2_BITCLK0P);
break;
case SND_SOC_DAIFMT_IB_IF: /* invert BCLK + FRM */
- dev_dbg(dai->component->dev,
+ dev_dbg(component->dev,
"%s: IF0: Inverted bit clock, inverted frame\n",
__func__);
- val |= BIT(AB8500_DIGIFCONF2_FSYNC0P);
- val |= BIT(AB8500_DIGIFCONF2_BITCLK0P);
+ conf2_val |= BIT(AB8500_DIGIFCONF2_FSYNC0P) |
+ BIT(AB8500_DIGIFCONF2_BITCLK0P);
break;
default:
- dev_err(dai->component->dev,
+ dev_err(component->dev,
"%s: ERROR: Unsupported INV mask 0x%x\n",
__func__, fmt & SND_SOC_DAIFMT_INV_MASK);
return -EINVAL;
}
- snd_soc_component_update_bits(component, AB8500_DIGIFCONF2, mask, val);
+ ret = snd_soc_component_update_bits(component, AB8500_DIGIFCONF3,
+ conf3_mask, conf3_val);
+ if (ret < 0)
+ return ret;
- return 0;
+ ret = snd_soc_component_update_bits(component, AB8500_DIGIFCONF1,
+ conf1_mask, conf1_val);
+ if (ret < 0)
+ return ret;
+
+ ret = snd_soc_component_update_bits(component, AB8500_DIGIFCONF2,
+ conf2_mask, conf2_val);
+
+ return ret < 0 ? ret : 0;
}
static int ab8500_codec_set_dai_tdm_slot(struct snd_soc_dai *dai,
@@ -1937,23 +1776,27 @@ static int ab8500_codec_set_dai_tdm_slot(struct snd_soc_dai *dai,
int slots, int slot_width)
{
struct snd_soc_component *component = dai->component;
- unsigned int val, mask, slot, slots_active;
+ unsigned int active_mask, clock_ratio, slot, value, ad_out, reg;
+ unsigned int tx_active, rx_active;
+ unsigned int conf1_val, conf2_val;
+ unsigned int mask;
+ int channel, ret;
mask = BIT(AB8500_DIGIFCONF2_IF0WL0) |
BIT(AB8500_DIGIFCONF2_IF0WL1);
- val = 0;
+ conf2_val = 0;
switch (slot_width) {
case 16:
break;
case 20:
- val |= BIT(AB8500_DIGIFCONF2_IF0WL0);
+ conf2_val |= BIT(AB8500_DIGIFCONF2_IF0WL0);
break;
case 24:
- val |= BIT(AB8500_DIGIFCONF2_IF0WL1);
+ conf2_val |= BIT(AB8500_DIGIFCONF2_IF0WL1);
break;
case 32:
- val |= BIT(AB8500_DIGIFCONF2_IF0WL1) |
+ conf2_val |= BIT(AB8500_DIGIFCONF2_IF0WL1) |
BIT(AB8500_DIGIFCONF2_IF0WL0);
break;
default:
@@ -1962,27 +1805,11 @@ static int ab8500_codec_set_dai_tdm_slot(struct snd_soc_dai *dai,
return -EINVAL;
}
- dev_dbg(dai->component->dev, "%s: IF0 slot-width: %d bits.\n",
- __func__, slot_width);
- snd_soc_component_update_bits(component, AB8500_DIGIFCONF2, mask, val);
-
- /* Setup TDM clocking according to slot count */
- dev_dbg(dai->component->dev, "%s: Slots, total: %d\n", __func__, slots);
- mask = BIT(AB8500_DIGIFCONF1_IF0BITCLKOS0) |
- BIT(AB8500_DIGIFCONF1_IF0BITCLKOS1);
switch (slots) {
case 2:
- val = AB8500_MASK_NONE;
- break;
case 4:
- val = BIT(AB8500_DIGIFCONF1_IF0BITCLKOS0);
- break;
case 8:
- val = BIT(AB8500_DIGIFCONF1_IF0BITCLKOS1);
- break;
case 16:
- val = BIT(AB8500_DIGIFCONF1_IF0BITCLKOS0) |
- BIT(AB8500_DIGIFCONF1_IF0BITCLKOS1);
break;
default:
dev_err(dai->component->dev,
@@ -1990,92 +1817,134 @@ static int ab8500_codec_set_dai_tdm_slot(struct snd_soc_dai *dai,
__func__, slots);
return -EINVAL;
}
- snd_soc_component_update_bits(component, AB8500_DIGIFCONF1, mask, val);
-
- /* Setup TDM DA according to active tx slots */
- if (tx_mask & ~0xff)
- return -EINVAL;
-
- mask = AB8500_DASLOTCONFX_SLTODAX_MASK;
- tx_mask = tx_mask << AB8500_DA_DATA0_OFFSET;
- slots_active = hweight32(tx_mask);
-
- dev_dbg(dai->component->dev, "%s: Slots, active, TX: %d\n", __func__,
- slots_active);
-
- switch (slots_active) {
- case 0:
+ clock_ratio = slots * slot_width;
+ switch (clock_ratio) {
+ case 32:
+ conf1_val = 0;
break;
- case 1:
- slot = ffs(tx_mask);
- snd_soc_component_update_bits(component, AB8500_DASLOTCONF1, mask, slot);
- snd_soc_component_update_bits(component, AB8500_DASLOTCONF3, mask, slot);
- snd_soc_component_update_bits(component, AB8500_DASLOTCONF2, mask, slot);
- snd_soc_component_update_bits(component, AB8500_DASLOTCONF4, mask, slot);
+ case 64:
+ conf1_val = BIT(AB8500_DIGIFCONF1_IF0BITCLKOS0);
break;
- case 2:
- slot = ffs(tx_mask);
- snd_soc_component_update_bits(component, AB8500_DASLOTCONF1, mask, slot);
- snd_soc_component_update_bits(component, AB8500_DASLOTCONF3, mask, slot);
- slot = fls(tx_mask);
- snd_soc_component_update_bits(component, AB8500_DASLOTCONF2, mask, slot);
- snd_soc_component_update_bits(component, AB8500_DASLOTCONF4, mask, slot);
+ case 128:
+ conf1_val = BIT(AB8500_DIGIFCONF1_IF0BITCLKOS1);
break;
- case 8:
- dev_dbg(dai->component->dev,
- "%s: In 8-channel mode DA-from-slot mapping is set manually.",
- __func__);
+ case 256:
+ conf1_val = BIT(AB8500_DIGIFCONF1_IF0BITCLKOS0) |
+ BIT(AB8500_DIGIFCONF1_IF0BITCLKOS1);
break;
default:
- dev_err(dai->component->dev,
- "%s: Unsupported number of active TX-slots (%d)!\n",
- __func__, slots_active);
+ dev_err(component->dev, "%s: Unsupported BCLK ratio (%u)!\n",
+ __func__, clock_ratio);
return -EINVAL;
}
- /* Setup TDM AD according to active RX-slots */
+ active_mask = GENMASK(min(slots, 8) - 1, 0);
+ if ((tx_mask | rx_mask) & ~active_mask) {
+ dev_err(component->dev, "%s: Slot mask exceeds slot count\n",
+ __func__);
+ return -EINVAL;
+ }
- if (rx_mask & ~0xff)
+ tx_active = hweight32(tx_mask);
+ rx_active = hweight32(rx_mask);
+ if (tx_active != 0 && tx_active != 1 && tx_active != 2 &&
+ tx_active != 8) {
+ dev_err(component->dev, "%s: Unsupported active TX slots (%u)!\n",
+ __func__, tx_active);
return -EINVAL;
+ }
+ if (rx_active != 0 && rx_active != 1 && rx_active != 2 &&
+ rx_active != 8) {
+ dev_err(component->dev, "%s: Unsupported active RX slots (%u)!\n",
+ __func__, rx_active);
+ return -EINVAL;
+ }
- rx_mask = rx_mask << AB8500_AD_DATA0_OFFSET;
- slots_active = hweight32(rx_mask);
+ dev_dbg(component->dev,
+ "%s: %d slots of %d bits, TX active: %u, RX active: %u\n",
+ __func__, slots, slot_width, tx_active, rx_active);
- dev_dbg(dai->component->dev, "%s: Slots, active, RX: %d\n", __func__,
- slots_active);
+ ret = snd_soc_component_update_bits(component, AB8500_DIGIFCONF2,
+ mask, conf2_val);
+ if (ret < 0)
+ return ret;
- switch (slots_active) {
- case 0:
- break;
- case 1:
- slot = ffs(rx_mask);
- snd_soc_component_update_bits(component, AB8500_ADSLOTSEL(slot),
- AB8500_MASK_SLOT(slot),
- AB8500_ADSLOTSELX_AD_OUT_TO_SLOT(AB8500_AD_OUT3, slot));
- break;
- case 2:
- slot = ffs(rx_mask);
- snd_soc_component_update_bits(component,
- AB8500_ADSLOTSEL(slot),
- AB8500_MASK_SLOT(slot),
- AB8500_ADSLOTSELX_AD_OUT_TO_SLOT(AB8500_AD_OUT3, slot));
- slot = fls(rx_mask);
- snd_soc_component_update_bits(component,
- AB8500_ADSLOTSEL(slot),
- AB8500_MASK_SLOT(slot),
- AB8500_ADSLOTSELX_AD_OUT_TO_SLOT(AB8500_AD_OUT2, slot));
- break;
- case 8:
- dev_dbg(dai->component->dev,
- "%s: In 8-channel mode AD-to-slot mapping is set manually.",
- __func__);
- break;
- default:
- dev_err(dai->component->dev,
- "%s: Unsupported number of active RX-slots (%d)!\n",
- __func__, slots_active);
- return -EINVAL;
+ mask = BIT(AB8500_DIGIFCONF1_IF0BITCLKOS0) |
+ BIT(AB8500_DIGIFCONF1_IF0BITCLKOS1);
+ ret = snd_soc_component_update_bits(component, AB8500_DIGIFCONF1,
+ mask, conf1_val);
+ if (ret < 0)
+ return ret;
+
+ mask = AB8500_DASLOTCONFX_SLTODAX_MASK;
+ if (tx_active == 1 || tx_active == 2) {
+ slot = __ffs(tx_mask) + AB8500_DA_DATA0_OFFSET;
+ reg = AB8500_DASLOTCONF1;
+ ret = snd_soc_component_update_bits(component, reg, mask, slot);
+ if (ret < 0)
+ return ret;
+ reg = AB8500_DASLOTCONF3;
+ ret = snd_soc_component_update_bits(component, reg, mask, slot);
+ if (ret < 0)
+ return ret;
+
+ if (tx_active == 2)
+ slot = __fls(tx_mask) + AB8500_DA_DATA0_OFFSET;
+ reg = AB8500_DASLOTCONF2;
+ ret = snd_soc_component_update_bits(component, reg, mask, slot);
+ if (ret < 0)
+ return ret;
+ reg = AB8500_DASLOTCONF4;
+ ret = snd_soc_component_update_bits(component, reg, mask, slot);
+ if (ret < 0)
+ return ret;
+ } else if (tx_active == 8) {
+ channel = 0;
+ for (slot = 0; slot < 8; slot++) {
+ if (!(tx_mask & BIT(slot)))
+ continue;
+ reg = AB8500_DASLOTCONF1 + channel++;
+ value = slot + AB8500_DA_DATA0_OFFSET;
+ ret = snd_soc_component_update_bits(component, reg, mask, value);
+ if (ret < 0)
+ return ret;
+ }
+ }
+
+ if (rx_active == 1 || rx_active == 2) {
+ slot = __ffs(rx_mask) + AB8500_AD_DATA0_OFFSET;
+ value = AB8500_ADSLOTSELX_AD_OUT_TO_SLOT(AB8500_AD_OUT3,
+ slot);
+ reg = AB8500_ADSLOTSEL(slot);
+ mask = AB8500_MASK_SLOT(slot);
+ ret = snd_soc_component_update_bits(component, reg, mask, value);
+ if (ret < 0)
+ return ret;
+
+ if (rx_active == 2) {
+ slot = __fls(rx_mask) + AB8500_AD_DATA0_OFFSET;
+ value = AB8500_ADSLOTSELX_AD_OUT_TO_SLOT(AB8500_AD_OUT2,
+ slot);
+ reg = AB8500_ADSLOTSEL(slot);
+ mask = AB8500_MASK_SLOT(slot);
+ ret = snd_soc_component_update_bits(component, reg, mask, value);
+ if (ret < 0)
+ return ret;
+ }
+ } else if (rx_active == 8) {
+ channel = 0;
+ for (slot = 0; slot < 8; slot++) {
+ if (!(rx_mask & BIT(slot)))
+ continue;
+ ad_out = AB8500_AD_OUT1 + channel++;
+ value = AB8500_ADSLOTSELX_AD_OUT_TO_SLOT(ad_out, slot);
+ reg = AB8500_ADSLOTSEL(slot);
+ mask = AB8500_MASK_SLOT(slot);
+ ret = snd_soc_component_update_bits(component, reg, mask, value);
+ if (ret < 0)
+ return ret;
+ }
}
return 0;
@@ -2172,10 +2041,8 @@ static void ab8500_codec_of_probe(struct device *dev, struct device_node *np,
static int ab8500_codec_probe(struct snd_soc_component *component)
{
- struct snd_soc_dapm_context *dapm = snd_soc_component_to_dapm(component);
struct device *dev = component->dev;
struct device_node *np = dev->of_node;
- struct ab8500_codec_drvdata *drvdata = dev_get_drvdata(dev);
struct ab8500_codec_platform_data codec_pdata;
int status;
@@ -2183,6 +2050,13 @@ static int ab8500_codec_probe(struct snd_soc_component *component)
ab8500_codec_of_probe(dev, np, &codec_pdata);
+ status = ab8500_audio_init_audioblock(component);
+ if (status < 0) {
+ dev_err(dev, "%s: failed to init audio-block (%d)!\n",
+ __func__, status);
+ return status;
+ }
+
status = ab8500_audio_setup_mics(component, &codec_pdata.amics);
if (status < 0) {
pr_err("%s: Failed to setup mics (%d)!\n", __func__, status);
@@ -2195,23 +2069,12 @@ static int ab8500_codec_probe(struct snd_soc_component *component)
return status;
}
- status = ab8500_audio_init_audioblock(component);
- if (status < 0) {
- dev_err(dev, "%s: failed to init audio-block (%d)!\n",
- __func__, status);
- return status;
- }
-
/* Override HW-defaults */
snd_soc_component_write(component, AB8500_ANACONF5,
BIT(AB8500_ANACONF5_HSAUTOEN));
snd_soc_component_write(component, AB8500_SHORTCIRCONF,
BIT(AB8500_SHORTCIRCONF_HSZCDDIS));
- snd_soc_dapm_disable_pin(dapm, "ANC Configure Input");
-
- mutex_init(&drvdata->ctrl_lock);
-
return status;
}
@@ -2240,7 +2103,6 @@ static int ab8500_codec_driver_probe(struct platform_device *pdev)
GFP_KERNEL);
if (!drvdata)
return -ENOMEM;
- drvdata->sid_status = SID_UNCONFIGURED;
dev_set_drvdata(&pdev->dev, drvdata);
drvdata->regmap = devm_regmap_init(&pdev->dev, NULL, &pdev->dev,
diff --git a/sound/soc/codecs/cs35l56-i2c.c b/sound/soc/codecs/cs35l56-i2c.c
index 4f6ddf1c5a3f..5e69ddbe342a 100644
--- a/sound/soc/codecs/cs35l56-i2c.c
+++ b/sound/soc/codecs/cs35l56-i2c.c
@@ -51,15 +51,7 @@ static int cs35l56_i2c_probe(struct i2c_client *client)
return dev_err_probe(cs35l56->base.dev, ret, "Failed to allocate register map\n");
}
- ret = cs35l56_common_probe(cs35l56);
- if (ret != 0)
- return ret;
-
- ret = cs35l56_irq_request(&cs35l56->base, client->irq);
- if (ret < 0)
- cs35l56_remove(cs35l56);
-
- return ret;
+ return cs35l56_common_probe(cs35l56, client->irq);
}
static void cs35l56_i2c_remove(struct i2c_client *client)
diff --git a/sound/soc/codecs/cs35l56-sdw.c b/sound/soc/codecs/cs35l56-sdw.c
index 0a55b93b96f9..a3812ab4c022 100644
--- a/sound/soc/codecs/cs35l56-sdw.c
+++ b/sound/soc/codecs/cs35l56-sdw.c
@@ -483,11 +483,7 @@ static int cs35l56_sdw_probe(struct sdw_slave *peripheral, const struct sdw_devi
/* Start in cache-only until device is enumerated */
regcache_cache_only(cs35l56->base.regmap, true);
- ret = cs35l56_common_probe(cs35l56);
- if (ret != 0)
- return ret;
-
- return 0;
+ return cs35l56_common_probe(cs35l56, -EINVAL);
}
static void cs35l56_sdw_remove(struct sdw_slave *peripheral)
diff --git a/sound/soc/codecs/cs35l56-spi.c b/sound/soc/codecs/cs35l56-spi.c
index b1eb924a5b6c..21b18da9e73d 100644
--- a/sound/soc/codecs/cs35l56-spi.c
+++ b/sound/soc/codecs/cs35l56-spi.c
@@ -40,15 +40,7 @@ static int cs35l56_spi_probe(struct spi_device *spi)
if (ret)
return ret;
- ret = cs35l56_common_probe(cs35l56);
- if (ret != 0)
- return ret;
-
- ret = cs35l56_irq_request(&cs35l56->base, spi->irq);
- if (ret < 0)
- cs35l56_remove(cs35l56);
-
- return ret;
+ return cs35l56_common_probe(cs35l56, spi->irq);
}
static void cs35l56_spi_remove(struct spi_device *spi)
diff --git a/sound/soc/codecs/cs35l56.c b/sound/soc/codecs/cs35l56.c
index 063ef7a70de0..8beef93ac467 100644
--- a/sound/soc/codecs/cs35l56.c
+++ b/sound/soc/codecs/cs35l56.c
@@ -18,9 +18,11 @@
#include <linux/interrupt.h>
#include <linux/math.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/pm.h>
#include <linux/pm_runtime.h>
#include <linux/property.h>
+#include <linux/reboot.h>
#include <linux/regmap.h>
#include <linux/regulator/consumer.h>
#include <linux/slab.h>
@@ -37,6 +39,13 @@
#include "wm_adsp.h"
#include "cs35l56.h"
+/*
+ * snd_soc_register_component() can call component_probe() on all instances
+ * in a card, so deferred registration must be protected across all instances.
+ */
+static DEFINE_MUTEX(cs35l56_component_register_lock);
+static bool cs35l56_shutting_down;
+
void cs35l56_mask_soundwire_interrupts(struct sdw_slave *peripheral)
{
/*
@@ -1368,12 +1377,6 @@ static int _cs35l56_component_probe(struct snd_soc_component *component)
BUILD_BUG_ON(ARRAY_SIZE(cs35l56_tx_input_texts) != ARRAY_SIZE(cs35l56_tx_input_values));
- if (!wait_for_completion_timeout(&cs35l56->init_completion,
- msecs_to_jiffies(5000))) {
- dev_err(cs35l56->base.dev, "%s: init_completion timed out\n", __func__);
- return -ENODEV;
- }
-
cs35l56->dsp.part = kasprintf(GFP_KERNEL, "cs35l%02x", cs35l56->base.type);
if (!cs35l56->dsp.part)
return -ENOMEM;
@@ -1941,7 +1944,46 @@ static int cs35l56_try_get_broken_sdca_spkid_gpio(struct cs35l56_private *cs35l5
return ret;
}
-int cs35l56_common_probe(struct cs35l56_private *cs35l56)
+static int cs35l56_component_register(struct cs35l56_private *cs35l56)
+{
+ int ret;
+
+ ret = snd_soc_register_component(cs35l56->base.dev,
+ &soc_component_dev_cs35l56,
+ cs35l56_dai, ARRAY_SIZE(cs35l56_dai));
+ if (ret < 0) {
+ dev_err(cs35l56->base.dev, "Register codec failed: %d\n", ret);
+ return ret;
+ }
+
+ cs35l56->component_registered = true;
+
+ return 0;
+}
+
+static void cs35l56_component_register_work(struct work_struct *work)
+{
+ struct cs35l56_private *cs35l56 = container_of(work,
+ struct cs35l56_private,
+ component_register_work);
+ int ret;
+
+ guard(mutex)(&cs35l56_component_register_lock);
+
+ if (cs35l56_shutting_down)
+ return;
+
+ PM_RUNTIME_ACQUIRE_AUTOSUSPEND(cs35l56->base.dev, pm_err);
+ ret = PM_RUNTIME_ACQUIRE_ERR(&pm_err);
+ if (ret) {
+ dev_err(cs35l56->base.dev, "register_work failed to get pm_runtime: %d\n", ret);
+ return;
+ }
+
+ cs35l56_component_register(cs35l56);
+}
+
+int cs35l56_common_probe(struct cs35l56_private *cs35l56, int irq)
{
int ret;
@@ -1949,6 +1991,7 @@ int cs35l56_common_probe(struct cs35l56_private *cs35l56)
mutex_init(&cs35l56->base.irq_lock);
cs35l56->base.cal_index = -1;
cs35l56->speaker_id = -ENOENT;
+ INIT_WORK(&cs35l56->component_register_work, cs35l56_component_register_work);
dev_set_drvdata(cs35l56->base.dev, cs35l56);
@@ -2018,16 +2061,29 @@ int cs35l56_common_probe(struct cs35l56_private *cs35l56)
goto err_remove_wm_adsp;
}
- ret = snd_soc_register_component(cs35l56->base.dev,
- &soc_component_dev_cs35l56,
- cs35l56_dai, ARRAY_SIZE(cs35l56_dai));
- if (ret < 0) {
- dev_err_probe(cs35l56->base.dev, ret, "Register codec failed\n");
+ ret = cs35l56_irq_request(&cs35l56->base, irq);
+ if (ret)
goto err_remove_wm_adsp;
+
+ /*
+ * Defer calling snd_soc_register_component() on SoundWire to prevent
+ * a deadlock where it calls our component_probe(), which requires the
+ * SoundWire enumeration to complete, but because we are still in probe()
+ * the SoundWire core will not call the update_status() callback. At time
+ * of writing snd_soc_register_component() never returns EPROBE_DEFER.
+ */
+ if (!cs35l56->sdw_peripheral) {
+ ret = cs35l56_component_register(cs35l56);
+ if (ret < 0)
+ goto err_free_irq;
}
return 0;
+err_free_irq:
+ if (cs35l56->base.irq)
+ devm_free_irq(cs35l56->base.dev, cs35l56->base.irq, &cs35l56->base);
+
err_remove_wm_adsp:
wm_adsp2_remove(&cs35l56->dsp);
@@ -2049,6 +2105,7 @@ EXPORT_SYMBOL_NS_GPL(cs35l56_common_probe, "SND_SOC_CS35L56_CORE");
int cs35l56_init(struct cs35l56_private *cs35l56)
{
+ bool first_time_init = !cs35l56->base.init_done;
int ret;
/*
@@ -2125,13 +2182,23 @@ int cs35l56_init(struct cs35l56_private *cs35l56)
cs35l56->base.init_done = true;
complete_all(&cs35l56->init_completion);
+ if (cs35l56->sdw_peripheral && first_time_init) {
+ /*
+ * Hardware now accessible, queue work to call
+ * snd_soc_register_component().
+ */
+ queue_work(system_freezable_wq, &cs35l56->component_register_work);
+ }
+
return 0;
}
EXPORT_SYMBOL_NS_GPL(cs35l56_init, "SND_SOC_CS35L56_CORE");
void cs35l56_remove(struct cs35l56_private *cs35l56)
{
- snd_soc_unregister_component(cs35l56->base.dev);
+ cancel_work_sync(&cs35l56->component_register_work);
+ if (cs35l56->component_registered)
+ snd_soc_unregister_component(cs35l56->base.dev);
cs35l56->base.init_done = false;
@@ -2166,6 +2233,37 @@ EXPORT_NS_GPL_DEV_PM_OPS(cs35l56_pm_ops_i2c_spi, SND_SOC_CS35L56_CORE) = {
};
#endif
+static int cs35l56_reboot_notify(struct notifier_block *nb,
+ unsigned long action, void *data)
+{
+ guard(mutex)(&cs35l56_component_register_lock);
+ cs35l56_shutting_down = true;
+
+ return NOTIFY_DONE;
+}
+
+static struct notifier_block cs35l56_reboot_notifier = {
+ .notifier_call = cs35l56_reboot_notify,
+};
+
+static int __init cs35l56_modinit(void)
+{
+ /*
+ * Use reboot notifier to prevent race between shutdown and
+ * snd_soc_register_component(). Driver shutdown() callback would
+ * run too late, after device_shutdown() is already walking the
+ * device list that component registration can modify.
+ */
+ return register_reboot_notifier(&cs35l56_reboot_notifier);
+}
+module_init(cs35l56_modinit);
+
+static void __exit cs35l56_modexit(void)
+{
+ unregister_reboot_notifier(&cs35l56_reboot_notifier);
+}
+module_exit(cs35l56_modexit);
+
MODULE_DESCRIPTION("ASoC CS35L56 driver");
MODULE_IMPORT_NS("SND_SOC_CS35L56_SHARED");
MODULE_IMPORT_NS("SND_SOC_CS_AMP_LIB");
diff --git a/sound/soc/codecs/cs35l56.h b/sound/soc/codecs/cs35l56.h
index 9acd2e7e17c9..6adba4d2da1f 100644
--- a/sound/soc/codecs/cs35l56.h
+++ b/sound/soc/codecs/cs35l56.h
@@ -32,6 +32,7 @@ struct sdw_slave;
struct cs35l56_private {
struct wm_adsp dsp; /* must be first member */
struct cs35l56_base base;
+ struct work_struct component_register_work;
struct work_struct dsp_work;
struct workqueue_struct *dsp_wq;
struct snd_soc_component *component;
@@ -43,6 +44,7 @@ struct cs35l56_private {
bool sdw_irq_no_unmask;
bool soft_resetting;
bool sdw_attached;
+ bool component_registered;
struct completion init_completion;
int speaker_id;
@@ -78,7 +80,7 @@ int cs35l56_system_resume_early(struct device *dev);
int cs35l56_system_resume(struct device *dev);
irqreturn_t cs35l56_irq(int irq, void *data);
int cs35l56_irq_request(struct cs35l56_base *cs35l56_base, int irq);
-int cs35l56_common_probe(struct cs35l56_private *cs35l56);
+int cs35l56_common_probe(struct cs35l56_private *cs35l56, int irq);
int cs35l56_init(struct cs35l56_private *cs35l56);
void cs35l56_remove(struct cs35l56_private *cs35l56);
diff --git a/sound/soc/codecs/mt6351.c b/sound/soc/codecs/mt6351.c
index 1768c249650d..3be0491a2588 100644
--- a/sound/soc/codecs/mt6351.c
+++ b/sound/soc/codecs/mt6351.c
@@ -1478,6 +1478,7 @@ static const struct of_device_id mt6351_of_match[] = {
{.compatible = "mediatek,mt6351-sound",},
{}
};
+MODULE_DEVICE_TABLE(of, mt6351_of_match);
static struct platform_driver mt6351_codec_driver = {
.driver = {
diff --git a/sound/soc/fsl/fsl_micfil.c b/sound/soc/fsl/fsl_micfil.c
index 60ac8eabab9d..5d8f0f76ab46 100644
--- a/sound/soc/fsl/fsl_micfil.c
+++ b/sound/soc/fsl/fsl_micfil.c
@@ -953,12 +953,17 @@ static int fsl_micfil_reparent_rootclk(struct fsl_micfil *micfil, unsigned int s
/* Get root clock */
clk = micfil->mclk;
- /* Disable clock first, for it was enabled by pm_runtime */
+ /* Reparent root clock to the PLL matching this sample rate */
fsl_asoc_reparent_pll_clocks(dev, clk, micfil->pll8k_clk,
micfil->pll11k_clk, ratio);
- ret = clk_prepare_enable(clk);
- if (ret)
- return ret;
+
+ /* Enable only once; hw_params can be called multiple times */
+ if (!micfil->mclk_flag) {
+ ret = clk_prepare_enable(clk);
+ if (ret)
+ return ret;
+ micfil->mclk_flag = true;
+ }
return 0;
}
@@ -991,8 +996,6 @@ static int fsl_micfil_hw_params(struct snd_pcm_substream *substream,
if (ret)
return ret;
- micfil->mclk_flag = true;
-
/* floor(K * CLKDIV) */
switch (micfil->quality) {
case QUALITY_HIGH:
@@ -1068,8 +1071,10 @@ static int fsl_micfil_hw_free(struct snd_pcm_substream *substream,
{
struct fsl_micfil *micfil = snd_soc_dai_get_drvdata(dai);
- clk_disable_unprepare(micfil->mclk);
- micfil->mclk_flag = false;
+ if (micfil->mclk_flag) {
+ clk_disable_unprepare(micfil->mclk);
+ micfil->mclk_flag = false;
+ }
return 0;
}
diff --git a/sound/soc/intel/atom/sst/sst_pci.c b/sound/soc/intel/atom/sst/sst_pci.c
index 44bb11c69490..1a53c993b57f 100644
--- a/sound/soc/intel/atom/sst/sst_pci.c
+++ b/sound/soc/intel/atom/sst/sst_pci.c
@@ -167,6 +167,7 @@ static const struct pci_device_id intel_sst_ids[] = {
{ PCI_DEVICE_DATA(INTEL, SST_TNG, 0) },
{ 0, }
};
+MODULE_DEVICE_TABLE(pci, intel_sst_ids);
static struct pci_driver sst_driver = {
.name = SST_DRV_NAME,
diff --git a/sound/soc/intel/avs/core.c b/sound/soc/intel/avs/core.c
index 1a53856c2ffb..a6b877f41e8b 100644
--- a/sound/soc/intel/avs/core.c
+++ b/sound/soc/intel/avs/core.c
@@ -473,8 +473,13 @@ static int avs_pci_probe(struct pci_dev *pci, const struct pci_device_id *id)
}
snd_hdac_bus_parse_capabilities(bus);
- if (bus->mlcap)
- snd_hdac_ext_bus_get_ml_capabilities(bus);
+ if (bus->mlcap) {
+ ret = snd_hdac_ext_bus_get_ml_capabilities(bus);
+ if (ret < 0) {
+ dev_err(dev, "failed to get ml capabilities: %d\n", ret);
+ goto err_ml_cap;
+ }
+ }
if (dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64)))
dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
@@ -516,6 +521,8 @@ static int avs_pci_probe(struct pci_dev *pci, const struct pci_device_id *id)
snd_hdac_bus_free_stream_pages(bus);
snd_hdac_ext_stream_free_all(bus);
err_init_streams:
+ snd_hdac_ext_link_free_all(bus);
+err_ml_cap:
iounmap(adev->dsp_ba);
err_remap_bar4:
iounmap(bus->remap_addr);
diff --git a/sound/soc/intel/avs/debugfs.c b/sound/soc/intel/avs/debugfs.c
index 701c247227bf..fc321f61fc92 100644
--- a/sound/soc/intel/avs/debugfs.c
+++ b/sound/soc/intel/avs/debugfs.c
@@ -8,6 +8,7 @@
#include <linux/debugfs.h>
#include <linux/kfifo.h>
+#include <linux/module.h>
#include <linux/wait.h>
#include <linux/sched/signal.h>
#include <linux/string_helpers.h>
@@ -235,15 +236,20 @@ static int strace_open(struct inode *inode, struct file *file)
if (!try_module_get(adev->dev->driver->owner))
return -ENODEV;
- if (kfifo_initialized(&adev->trace_fifo))
- return -EBUSY;
+ if (kfifo_initialized(&adev->trace_fifo)) {
+ ret = -EBUSY;
+ goto err;
+ }
ret = kfifo_alloc(&adev->trace_fifo, PAGE_SIZE, GFP_KERNEL);
if (ret < 0)
- return ret;
+ goto err;
file->private_data = adev;
return 0;
+err:
+ module_put(adev->dev->driver->owner);
+ return ret;
}
static int strace_release(struct inode *inode, struct file *file)
diff --git a/sound/soc/intel/avs/path.c b/sound/soc/intel/avs/path.c
index 2291f9728a54..de699e23c9e2 100644
--- a/sound/soc/intel/avs/path.c
+++ b/sound/soc/intel/avs/path.c
@@ -838,15 +838,10 @@ static int avs_path_module_type_create(struct avs_dev *adev, struct avs_path_mod
static int avs_path_module_send_init_configs(struct avs_dev *adev, struct avs_path_module *mod)
{
- struct avs_soc_component *acomp;
-
- acomp = to_avs_soc_component(mod->template->owner->owner->owner->owner->comp);
-
- u32 num_ids = mod->template->num_config_ids;
- u32 *ids = mod->template->config_ids;
+ struct avs_tplg_module *template = mod->template;
- for (int i = 0; i < num_ids; i++) {
- struct avs_tplg_init_config *config = &acomp->tplg->init_configs[ids[i]];
+ for (int i = 0; i < template->num_init_configs; i++) {
+ struct avs_tplg_init_config *config = template->init_configs[i];
size_t len = config->length;
void *data = config->data;
u32 param = config->param;
diff --git a/sound/soc/intel/avs/pcm.c b/sound/soc/intel/avs/pcm.c
index 797b9c9163b4..094ea1366ea8 100644
--- a/sound/soc/intel/avs/pcm.c
+++ b/sound/soc/intel/avs/pcm.c
@@ -6,6 +6,7 @@
// Amadeusz Slawinski <amadeuszx.slawinski@linux.intel.com>
//
+#include <linux/cleanup.h>
#include <linux/debugfs.h>
#include <linux/device.h>
#include <sound/hda_register.h>
@@ -987,13 +988,25 @@ static int avs_component_load_libraries(struct avs_soc_component *acomp)
return ret;
}
+static int avs_request_topology(struct snd_soc_component *component, const char *name,
+ const struct firmware **fw)
+{
+ char *fullname __free(kfree) = NULL;
+
+ fullname = kasprintf(GFP_KERNEL, "%s/%s", component->driver->topology_name_prefix, name);
+ if (!fullname)
+ return -ENOMEM;
+
+ return request_firmware(fw, fullname, component->dev);
+}
+
static int avs_component_probe(struct snd_soc_component *component)
{
struct snd_soc_card *card = component->card;
struct snd_soc_acpi_mach *mach;
struct avs_soc_component *acomp;
+ const struct firmware *fw;
struct avs_dev *adev;
- char *filename;
int ret;
dev_dbg(card->dev, "probing %s card %s\n", component->name, card->name);
@@ -1009,13 +1022,7 @@ static int avs_component_probe(struct snd_soc_component *component)
goto finalize;
/* Load specified topology and create debugfs for it. */
- filename = kasprintf(GFP_KERNEL, "%s/%s", component->driver->topology_name_prefix,
- mach->tplg_filename);
- if (!filename)
- return -ENOMEM;
-
- ret = avs_load_topology(component, filename);
- kfree(filename);
+ ret = avs_request_topology(component, mach->tplg_filename, &fw);
if (ret == -ENOENT && !strncmp(mach->tplg_filename, "hda-", 4)) {
unsigned int vendor_id;
@@ -1030,18 +1037,17 @@ static int avs_component_probe(struct snd_soc_component *component)
"hda-generic-tplg.bin");
if (!mach->tplg_filename)
return -ENOMEM;
- filename = kasprintf(GFP_KERNEL, "%s/%s", component->driver->topology_name_prefix,
- mach->tplg_filename);
- if (!filename)
- return -ENOMEM;
dev_info(card->dev, "trying to load fallback topology %s\n", mach->tplg_filename);
- ret = avs_load_topology(component, filename);
- kfree(filename);
+ ret = avs_request_topology(component, mach->tplg_filename, &fw);
}
if (ret < 0)
return ret;
+ ret = snd_soc_tplg_component_load(component, &avs_tplg_ops, fw);
+ if (ret)
+ return ret;
+
ret = avs_component_load_libraries(acomp);
if (ret < 0) {
dev_err(card->dev, "libraries loading failed: %d\n", ret);
diff --git a/sound/soc/intel/avs/topology.c b/sound/soc/intel/avs/topology.c
index 9033f683393c..c9837d78e623 100644
--- a/sound/soc/intel/avs/topology.c
+++ b/sound/soc/intel/avs/topology.c
@@ -350,6 +350,7 @@ AVS_DEFINE_PTR_PARSER(modcfg_base, struct avs_tplg_modcfg_base, modcfgs_base);
AVS_DEFINE_PTR_PARSER(modcfg_ext, struct avs_tplg_modcfg_ext, modcfgs_ext);
AVS_DEFINE_PTR_PARSER(pplcfg, struct avs_tplg_pplcfg, pplcfgs);
AVS_DEFINE_PTR_PARSER(binding, struct avs_tplg_binding, bindings);
+AVS_DEFINE_PTR_PARSER(init_config, struct avs_tplg_init_config, init_configs);
AVS_DEFINE_PTR_PARSER(nhlt_config, struct avs_tplg_nhlt_config, nhlt_configs);
static int
@@ -1198,7 +1199,7 @@ static const struct avs_tplg_token_parser module_parsers[] = {
{
.token = AVS_TKN_MOD_INIT_CONFIG_NUM_IDS_U32,
.type = SND_SOC_TPLG_TUPLE_TYPE_WORD,
- .offset = offsetof(struct avs_tplg_module, num_config_ids),
+ .offset = offsetof(struct avs_tplg_module, num_init_configs),
.parse = avs_parse_byte_token,
},
{
@@ -1214,10 +1215,32 @@ static const struct avs_tplg_token_parser init_config_parsers[] = {
.token = AVS_TKN_MOD_INIT_CONFIG_ID_U32,
.type = SND_SOC_TPLG_TUPLE_TYPE_WORD,
.offset = 0,
- .parse = avs_parse_word_token,
+ .parse = avs_parse_init_config_ptr,
},
};
+static int avs_tplg_module_init_configs(struct snd_soc_component *comp,
+ struct avs_tplg_module *module,
+ struct snd_soc_tplg_vendor_array *tuples, u32 block_size)
+{
+ struct avs_tplg_init_config **cfgs;
+ int ret;
+
+ if (!module->num_init_configs)
+ return -EINVAL;
+
+ cfgs = devm_kcalloc(comp->card->dev, module->num_init_configs, sizeof(*cfgs), GFP_KERNEL);
+ if (!cfgs)
+ return -ENOMEM;
+
+ ret = parse_dictionary_entries(comp, tuples, block_size, cfgs, module->num_init_configs,
+ sizeof(*cfgs), AVS_TKN_MOD_INIT_CONFIG_ID_U32,
+ init_config_parsers, ARRAY_SIZE(init_config_parsers));
+ if (!ret)
+ module->init_configs = cfgs;
+ return ret;
+}
+
static struct avs_tplg_module *
avs_tplg_module_create(struct snd_soc_component *comp, struct avs_tplg_pipeline *owner,
struct snd_soc_tplg_vendor_array *tuples, u32 block_size)
@@ -1244,27 +1267,11 @@ avs_tplg_module_create(struct snd_soc_component *comp, struct avs_tplg_pipeline
block_size -= esize;
/* Parse trailing config ids if any. */
if (block_size) {
- u32 num_config_ids = module->num_config_ids;
- u32 *config_ids;
-
- if (!num_config_ids)
- return ERR_PTR(-EINVAL);
-
- config_ids = devm_kcalloc(comp->card->dev, num_config_ids, sizeof(*config_ids),
- GFP_KERNEL);
- if (!config_ids)
- return ERR_PTR(-ENOMEM);
-
tuples = avs_tplg_vendor_array_at(tuples, esize);
- ret = parse_dictionary_entries(comp, tuples, block_size,
- config_ids, num_config_ids, sizeof(*config_ids),
- AVS_TKN_MOD_INIT_CONFIG_ID_U32,
- init_config_parsers,
- ARRAY_SIZE(init_config_parsers));
+
+ ret = avs_tplg_module_init_configs(comp, module, tuples, block_size);
if (ret)
return ERR_PTR(ret);
-
- module->config_ids = config_ids;
}
module->owner = owner;
@@ -2194,7 +2201,7 @@ avs_control_load(struct snd_soc_component *comp, int index, struct snd_kcontrol_
return 0;
}
-static const struct snd_soc_tplg_ops avs_tplg_ops = {
+const struct snd_soc_tplg_ops avs_tplg_ops = {
.io_ops = avs_control_ops,
.io_ops_count = ARRAY_SIZE(avs_control_ops),
.control_load = avs_control_load,
diff --git a/sound/soc/intel/avs/topology.h b/sound/soc/intel/avs/topology.h
index 1cf7455b6c01..189984ce7b51 100644
--- a/sound/soc/intel/avs/topology.h
+++ b/sound/soc/intel/avs/topology.h
@@ -221,8 +221,8 @@ struct avs_tplg_module {
u8 domain;
struct avs_tplg_modcfg_ext *cfg_ext;
u32 ctl_id;
- u32 num_config_ids;
- u32 *config_ids;
+ u32 num_init_configs;
+ struct avs_tplg_init_config **init_configs;
struct avs_tplg_nhlt_config *nhlt_config;
struct avs_tplg_pipeline *owner;
@@ -230,6 +230,7 @@ struct avs_tplg_module {
struct list_head node;
};
+extern const struct snd_soc_tplg_ops avs_tplg_ops;
struct avs_tplg *avs_tplg_new(struct snd_soc_component *comp);
int avs_load_topology(struct snd_soc_component *comp, const char *filename);
diff --git a/sound/soc/sprd/sprd-pcm-compress.c b/sound/soc/sprd/sprd-pcm-compress.c
index a7d437b49fbf..e5249924b54d 100644
--- a/sound/soc/sprd/sprd-pcm-compress.c
+++ b/sound/soc/sprd/sprd-pcm-compress.c
@@ -17,7 +17,7 @@
/* Default values if userspace does not set */
#define SPRD_COMPR_MIN_FRAGMENT_SIZE SZ_8K
-#define SPRD_COMPR_MAX_FRAGMENT_SIZE SZ_128K
+#define SPRD_COMPR_MAX_FRAGMENT_SIZE SZ_32K
#define SPRD_COMPR_MIN_NUM_FRAGMENTS 4
#define SPRD_COMPR_MAX_NUM_FRAGMENTS 64
@@ -271,6 +271,19 @@ static int sprd_platform_compr_set_params(struct snd_soc_component *component,
struct sprd_compr_params compr_params = { };
int ret;
+ /*
+ * The stage 0 IRAM buffer and the stage 1 DDR buffer are allocated
+ * with fixed sizes at open time, so the requested fragment size and
+ * fragments must fit into them, otherwise sprd_platform_compr_copy()
+ * would overflow the buffers. Note the compress core only checks the
+ * fragment size and fragments against an u32 overflow, not against
+ * the buffer sizes advertised by get_caps.
+ */
+ if (params->buffer.fragment_size > SPRD_COMPR_IRAM_BUF_SIZE ||
+ (u64)params->buffer.fragment_size * params->buffer.fragments >
+ SPRD_COMPR_AREA_BUF_SIZE)
+ return -EINVAL;
+
/*
* Configure the DMA engine 2-stage transfer mode. Channel 1 set as the
* destination channel, and channel 0 set as the source channel, that
diff --git a/sound/soc/sti/uniperif_reader.c b/sound/soc/sti/uniperif_reader.c
index 45d7613f595c..5347f9620f25 100644
--- a/sound/soc/sti/uniperif_reader.c
+++ b/sound/soc/sti/uniperif_reader.c
@@ -416,6 +416,8 @@ int uni_reader_init(struct platform_device *pdev,
else
reader->hw = &uni_reader_pcm_hw;
+ spin_lock_init(&reader->irq_lock);
+
ret = devm_request_irq(&pdev->dev, reader->irq,
uni_reader_irq_handler, IRQF_SHARED,
dev_name(&pdev->dev), reader);
@@ -424,8 +426,6 @@ int uni_reader_init(struct platform_device *pdev,
return -EBUSY;
}
- spin_lock_init(&reader->irq_lock);
-
return 0;
}
EXPORT_SYMBOL_GPL(uni_reader_init);
diff --git a/sound/soc/ux500/ux500_msp_dai.c b/sound/soc/ux500/ux500_msp_dai.c
index 7798957c6504..37c48cc70394 100644
--- a/sound/soc/ux500/ux500_msp_dai.c
+++ b/sound/soc/ux500/ux500_msp_dai.c
@@ -14,7 +14,7 @@
#include <linux/clk.h>
#include <linux/of.h>
#include <linux/regulator/consumer.h>
-#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/reset.h>
#include <sound/soc.h>
#include <sound/soc-dai.h>
@@ -34,8 +34,10 @@ static int setup_pcm_multichan(struct snd_soc_dai *dai,
if (drvdata->slots > 1) {
msp_config->multichannel_configured = 1;
- multi->tx_multichannel_enable = true;
- multi->rx_multichannel_enable = true;
+ multi->tx_multichannel_enable =
+ msp_config->direction & MSP_DIR_TX;
+ multi->rx_multichannel_enable =
+ msp_config->direction & MSP_DIR_RX;
multi->rx_comparison_enable_mode = MSP_COMPARISON_DISABLED;
multi->tx_channel_0_enable = drvdata->tx_mask;
@@ -57,72 +59,21 @@ static int setup_pcm_multichan(struct snd_soc_dai *dai,
return 0;
}
-static int setup_frameper(struct snd_soc_dai *dai, unsigned int rate,
- struct msp_protdesc *prot_desc)
+static void setup_frameper(struct snd_soc_dai *dai,
+ struct msp_protdesc *prot_desc)
{
struct ux500_msp_i2s_drvdata *drvdata = dev_get_drvdata(dai->dev);
- switch (drvdata->slots) {
- case 1:
- switch (rate) {
- case 8000:
- prot_desc->frame_period =
- FRAME_PER_SINGLE_SLOT_8_KHZ;
- break;
-
- case 16000:
- prot_desc->frame_period =
- FRAME_PER_SINGLE_SLOT_16_KHZ;
- break;
-
- case 44100:
- prot_desc->frame_period =
- FRAME_PER_SINGLE_SLOT_44_1_KHZ;
- break;
-
- case 48000:
- prot_desc->frame_period =
- FRAME_PER_SINGLE_SLOT_48_KHZ;
- break;
-
- default:
- dev_err(dai->dev,
- "%s: Error: Unsupported sample-rate (freq = %d)!\n",
- __func__, rate);
- return -EINVAL;
- }
- break;
-
- case 2:
- prot_desc->frame_period = FRAME_PER_2_SLOTS;
- break;
-
- case 8:
- prot_desc->frame_period = FRAME_PER_8_SLOTS;
- break;
-
- case 16:
- prot_desc->frame_period = FRAME_PER_16_SLOTS;
- break;
- default:
- dev_err(dai->dev,
- "%s: Error: Unsupported slot-count (slots = %d)!\n",
- __func__, drvdata->slots);
- return -EINVAL;
- }
-
- prot_desc->clocks_per_frame =
- prot_desc->frame_period+1;
+ prot_desc->clocks_per_frame = drvdata->slots * drvdata->slot_width;
+ prot_desc->frame_period = prot_desc->clocks_per_frame - 1;
dev_dbg(dai->dev, "%s: Clocks per frame: %u\n",
__func__,
prot_desc->clocks_per_frame);
-
- return 0;
}
-static int setup_pcm_framing(struct snd_soc_dai *dai, unsigned int rate,
- struct msp_protdesc *prot_desc)
+static int setup_pcm_framing(struct snd_soc_dai *dai,
+ struct msp_protdesc *prot_desc)
{
struct ux500_msp_i2s_drvdata *drvdata = dev_get_drvdata(dai->dev);
@@ -163,7 +114,9 @@ static int setup_pcm_framing(struct snd_soc_dai *dai, unsigned int rate,
prot_desc->tx_elem_len_2 = MSP_ELEM_LEN_16;
prot_desc->rx_elem_len_2 = MSP_ELEM_LEN_16;
- return setup_frameper(dai, rate, prot_desc);
+ setup_frameper(dai, prot_desc);
+
+ return 0;
}
static int setup_clocking(struct snd_soc_dai *dai,
@@ -177,7 +130,16 @@ static int setup_clocking(struct snd_soc_dai *dai,
case SND_SOC_DAIFMT_NB_IF:
msp_config->tx_fsync_pol ^= 1 << TFSPOL_SHIFT;
msp_config->rx_fsync_pol ^= 1 << RFSPOL_SHIFT;
+ break;
+
+ case SND_SOC_DAIFMT_IB_NF:
+ msp_config->bclk_inverted = true;
+ break;
+ case SND_SOC_DAIFMT_IB_IF:
+ msp_config->bclk_inverted = true;
+ msp_config->tx_fsync_pol ^= 1 << TFSPOL_SHIFT;
+ msp_config->rx_fsync_pol ^= 1 << RFSPOL_SHIFT;
break;
default:
@@ -192,6 +154,7 @@ static int setup_clocking(struct snd_soc_dai *dai,
case SND_SOC_DAIFMT_BC_FC:
dev_dbg(dai->dev, "%s: Codec is master.\n", __func__);
+ msp_config->clock_provider = false;
msp_config->iodelay = 0x20;
msp_config->rx_fsync_sel = 0;
msp_config->tx_fsync_sel = 1 << TFSSEL_SHIFT;
@@ -204,6 +167,7 @@ static int setup_clocking(struct snd_soc_dai *dai,
case SND_SOC_DAIFMT_BP_FP:
dev_dbg(dai->dev, "%s: Codec is slave.\n", __func__);
+ msp_config->clock_provider = true;
msp_config->tx_clk_sel = TX_CLK_SEL_SRG;
msp_config->tx_fsync_sel = TX_SYNC_SRG_PROG;
msp_config->rx_clk_sel = RX_CLK_SEL_SRG;
@@ -362,7 +326,7 @@ static int setup_msp_config(struct snd_pcm_substream *substream,
if (ret < 0)
return ret;
- ret = setup_pcm_framing(dai, runtime->rate, prot_desc);
+ ret = setup_pcm_framing(dai, prot_desc);
if (ret < 0)
return ret;
@@ -424,21 +388,21 @@ static void ux500_msp_dai_shutdown(struct snd_pcm_substream *substream,
int ret;
struct ux500_msp_i2s_drvdata *drvdata = dev_get_drvdata(dai->dev);
bool is_playback = (substream->stream == SNDRV_PCM_STREAM_PLAYBACK);
+ unsigned int configured = is_playback ? PLAYBACK_CONFIGURED :
+ CAPTURE_CONFIGURED;
+ unsigned int dir = is_playback ? MSP_DIR_TX : MSP_DIR_RX;
dev_dbg(dai->dev, "%s: MSP %d (%s): Enter.\n", __func__, dai->id,
snd_pcm_stream_str(substream));
- if (drvdata->vape_opp_constraint == 1) {
- prcmu_qos_update_requirement(PRCMU_QOS_APE_OPP,
- "ux500_msp_i2s", 50);
- drvdata->vape_opp_constraint = 0;
- }
-
- if (ux500_msp_i2s_close(drvdata->msp,
- is_playback ? MSP_DIR_TX : MSP_DIR_RX)) {
- dev_err(dai->dev,
- "%s: Error: MSP %d (%s): Unable to close i2s.\n",
- __func__, dai->id, snd_pcm_stream_str(substream));
+ if (drvdata->configured & configured) {
+ if (ux500_msp_i2s_close(drvdata->msp, dir)) {
+ dev_err(dai->dev,
+ "%s: Error: MSP %d (%s): Unable to close i2s.\n",
+ __func__, dai->id,
+ snd_pcm_stream_str(substream));
+ }
+ drvdata->configured &= ~configured;
}
/* Disable and unprepare clocks */
@@ -456,15 +420,23 @@ static void ux500_msp_dai_shutdown(struct snd_pcm_substream *substream,
static int ux500_msp_dai_prepare(struct snd_pcm_substream *substream,
struct snd_soc_dai *dai)
{
- int ret = 0;
struct ux500_msp_i2s_drvdata *drvdata = dev_get_drvdata(dai->dev);
struct snd_pcm_runtime *runtime = substream->runtime;
struct ux500_msp_config msp_config;
+ bool is_playback = substream->stream == SNDRV_PCM_STREAM_PLAYBACK;
+ unsigned int configured = is_playback ? PLAYBACK_CONFIGURED :
+ CAPTURE_CONFIGURED;
+ int ret;
dev_dbg(dai->dev, "%s: MSP %d (%s): Enter (rate = %d).\n", __func__,
dai->id, snd_pcm_stream_str(substream), runtime->rate);
- setup_msp_config(substream, dai, &msp_config);
+ if (drvdata->configured & configured)
+ return 0;
+
+ ret = setup_msp_config(substream, dai, &msp_config);
+ if (ret)
+ return ret;
ret = ux500_msp_i2s_open(drvdata->msp, &msp_config);
if (ret < 0) {
@@ -473,22 +445,9 @@ static int ux500_msp_dai_prepare(struct snd_pcm_substream *substream,
return ret;
}
- /* Set OPP-level */
- if ((drvdata->fmt & SND_SOC_DAIFMT_CLOCK_PROVIDER_MASK) &&
- (drvdata->msp->f_bitclk > 19200000)) {
- /* If the bit-clock is higher than 19.2MHz, Vape should be
- * run in 100% OPP. Only when bit-clock is used (MSP master)
- */
- prcmu_qos_update_requirement(PRCMU_QOS_APE_OPP,
- "ux500-msp-i2s", 100);
- drvdata->vape_opp_constraint = 1;
- } else {
- prcmu_qos_update_requirement(PRCMU_QOS_APE_OPP,
- "ux500-msp-i2s", 50);
- drvdata->vape_opp_constraint = 0;
- }
+ drvdata->configured |= configured;
- return ret;
+ return 0;
}
static int ux500_msp_dai_hw_params(struct snd_pcm_substream *substream,
@@ -496,7 +455,6 @@ static int ux500_msp_dai_hw_params(struct snd_pcm_substream *substream,
struct snd_soc_dai *dai)
{
unsigned int mask, slots_active;
- struct snd_pcm_runtime *runtime = substream->runtime;
struct ux500_msp_i2s_drvdata *drvdata = dev_get_drvdata(dai->dev);
dev_dbg(dai->dev, "%s: MSP %d (%s): Enter.\n",
@@ -504,9 +462,8 @@ static int ux500_msp_dai_hw_params(struct snd_pcm_substream *substream,
switch (drvdata->fmt & SND_SOC_DAIFMT_FORMAT_MASK) {
case SND_SOC_DAIFMT_I2S:
- snd_pcm_hw_constraint_minmax(runtime,
- SNDRV_PCM_HW_PARAM_CHANNELS,
- 1, 2);
+ if (params_channels(params) < 1 || params_channels(params) > 2)
+ return -EINVAL;
break;
case SND_SOC_DAIFMT_DSP_B:
@@ -518,9 +475,8 @@ static int ux500_msp_dai_hw_params(struct snd_pcm_substream *substream,
slots_active = hweight32(mask);
dev_dbg(dai->dev, "TDM-slots active: %d", slots_active);
- snd_pcm_hw_constraint_single(runtime,
- SNDRV_PCM_HW_PARAM_CHANNELS,
- slots_active);
+ if (!slots_active || params_channels(params) != slots_active)
+ return -EINVAL;
break;
default:
@@ -553,20 +509,21 @@ static int ux500_msp_dai_set_dai_fmt(struct snd_soc_dai *dai,
default:
dev_err(dai->dev,
"%s: Error: Unsupported protocol/master (fmt = 0x%x)!\n",
- __func__, drvdata->fmt);
+ __func__, fmt);
return -EINVAL;
}
switch (fmt & SND_SOC_DAIFMT_INV_MASK) {
case SND_SOC_DAIFMT_NB_NF:
case SND_SOC_DAIFMT_NB_IF:
+ case SND_SOC_DAIFMT_IB_NF:
case SND_SOC_DAIFMT_IB_IF:
break;
default:
dev_err(dai->dev,
"%s: Error: Unsupported inversion (fmt = 0x%x)!\n",
- __func__, drvdata->fmt);
+ __func__, fmt);
return -EINVAL;
}
@@ -600,17 +557,23 @@ static int ux500_msp_dai_set_tdm_slot(struct snd_soc_dai *dai,
__func__, slots);
return -EINVAL;
}
- drvdata->slots = slots;
- if (!(slot_width == 16)) {
+ if (slot_width != 16) {
dev_err(dai->dev, "%s: Error: Unsupported slot-width (%d)!\n",
__func__, slot_width);
return -EINVAL;
}
- drvdata->slot_width = slot_width;
- drvdata->tx_mask = tx_mask & cap;
- drvdata->rx_mask = rx_mask & cap;
+ if ((tx_mask | rx_mask) & ~cap) {
+ dev_err(dai->dev, "%s: Slot mask exceeds %d slots\n",
+ __func__, slots);
+ return -EINVAL;
+ }
+
+ drvdata->slots = slots;
+ drvdata->slot_width = slot_width;
+ drvdata->tx_mask = tx_mask;
+ drvdata->rx_mask = rx_mask;
return 0;
}
@@ -716,6 +679,7 @@ static const struct snd_soc_component_driver ux500_msp_component = {
static int ux500_msp_drv_probe(struct platform_device *pdev)
{
struct ux500_msp_i2s_drvdata *drvdata;
+ struct reset_control *reset;
int ret = 0;
drvdata = devm_kzalloc(&pdev->dev,
@@ -729,7 +693,6 @@ static int ux500_msp_drv_probe(struct platform_device *pdev)
drvdata->tx_mask = 0x01;
drvdata->rx_mask = 0x01;
drvdata->slot_width = 16;
- drvdata->master_clk = MSP_INPUT_FREQ_APB;
drvdata->reg_vape = devm_regulator_get(&pdev->dev, "v-ape");
if (IS_ERR(drvdata->reg_vape)) {
@@ -739,8 +702,6 @@ static int ux500_msp_drv_probe(struct platform_device *pdev)
__func__, ret);
return ret;
}
- prcmu_qos_add_requirement(PRCMU_QOS_APE_OPP, (char *)pdev->name, 50);
-
drvdata->pclk = devm_clk_get(&pdev->dev, "apb_pclk");
if (IS_ERR(drvdata->pclk)) {
ret = PTR_ERR(drvdata->pclk);
@@ -758,9 +719,19 @@ static int ux500_msp_drv_probe(struct platform_device *pdev)
__func__, ret);
return ret;
}
+ drvdata->master_clk = clk_get_rate(drvdata->clk);
+ if (!drvdata->master_clk) {
+ dev_err(&pdev->dev, "MSP clock has no rate\n");
+ return -EINVAL;
+ }
+
+ reset = devm_reset_control_get_exclusive_deasserted(&pdev->dev, NULL);
+ if (IS_ERR(reset))
+ return dev_err_probe(&pdev->dev, PTR_ERR(reset),
+ "Failed to deassert MSP reset\n");
ret = ux500_msp_i2s_init_msp(pdev, &drvdata->msp);
- if (!drvdata->msp) {
+ if (ret) {
dev_err(&pdev->dev,
"%s: ERROR: Failed to init MSP-struct (%d)!",
__func__, ret);
@@ -799,8 +770,6 @@ static void ux500_msp_drv_remove(struct platform_device *pdev)
snd_soc_unregister_component(&pdev->dev);
- prcmu_qos_remove_requirement(PRCMU_QOS_APE_OPP, "ux500_msp_i2s");
-
ux500_msp_i2s_cleanup_msp(pdev, drvdata->msp);
}
diff --git a/sound/soc/ux500/ux500_msp_dai.h b/sound/soc/ux500/ux500_msp_dai.h
index 30bf70838196..aae582030d95 100644
--- a/sound/soc/ux500/ux500_msp_dai.h
+++ b/sound/soc/ux500/ux500_msp_dai.h
@@ -22,17 +22,6 @@
#define UX500_I2S_FORMATS (SNDRV_PCM_FMTBIT_S16_LE)
-#define FRAME_PER_SINGLE_SLOT_8_KHZ 31
-#define FRAME_PER_SINGLE_SLOT_16_KHZ 124
-#define FRAME_PER_SINGLE_SLOT_44_1_KHZ 63
-#define FRAME_PER_SINGLE_SLOT_48_KHZ 49
-#define FRAME_PER_2_SLOTS 31
-#define FRAME_PER_8_SLOTS 138
-#define FRAME_PER_16_SLOTS 277
-
-#define UX500_MSP_INTERNAL_CLOCK_FREQ 40000000
-#define UX500_MSP1_INTERNAL_CLOCK_FREQ UX500_MSP_INTERNAL_CLOCK_FREQ
-
#define UX500_MSP_MIN_CHANNELS 1
#define UX500_MSP_MAX_CHANNELS 8
@@ -47,6 +36,7 @@ struct ux500_msp_i2s_drvdata {
struct ux500_msp *msp;
struct regulator *reg_vape;
unsigned int fmt;
+ unsigned int configured;
unsigned int tx_mask;
unsigned int rx_mask;
int slots;
@@ -57,8 +47,6 @@ struct ux500_msp_i2s_drvdata {
struct clk *clk;
struct clk *pclk;
- /* Regulators */
- int vape_opp_constraint;
};
int ux500_msp_dai_set_data_delay(struct snd_soc_dai *dai, int delay);
diff --git a/sound/soc/ux500/ux500_msp_i2s.c b/sound/soc/ux500/ux500_msp_i2s.c
index fbfeefa418ca..683b485fb570 100644
--- a/sound/soc/ux500/ux500_msp_i2s.c
+++ b/sound/soc/ux500/ux500_msp_i2s.c
@@ -201,10 +201,12 @@ static int configure_protocol(struct ux500_msp *msp,
/* The code below should not be separated. */
temp_reg = readl(msp->registers + MSP_GCR) & ~TX_CLK_POL_RISING;
- temp_reg |= MSP_TX_CLKPOL_BIT(~protdesc->tx_clk_pol);
+ temp_reg |= MSP_TX_CLKPOL_BIT(!protdesc->tx_clk_pol ^
+ config->bclk_inverted);
writel(temp_reg, msp->registers + MSP_GCR);
temp_reg = readl(msp->registers + MSP_GCR) & ~RX_CLK_POL_RISING;
- temp_reg |= MSP_RX_CLKPOL_BIT(protdesc->rx_clk_pol);
+ temp_reg |= MSP_RX_CLKPOL_BIT(protdesc->rx_clk_pol ^
+ config->bclk_inverted);
writel(temp_reg, msp->registers + MSP_GCR);
return 0;
@@ -212,35 +214,20 @@ static int configure_protocol(struct ux500_msp *msp,
static int setup_bitclk(struct ux500_msp *msp, struct ux500_msp_config *config)
{
+ struct msp_protdesc *protdesc;
+ u64 desired_bitclk;
+ unsigned int bitclk;
u32 reg_val_GCR;
- u32 frame_per = 0;
- u32 sck_div = 0;
- u32 frame_width = 0;
- u32 temp_reg = 0;
- struct msp_protdesc *protdesc = NULL;
+ u32 sck_div;
+ u32 temp_reg;
reg_val_GCR = readl(msp->registers + MSP_GCR);
writel(reg_val_GCR & ~SRG_ENABLE, msp->registers + MSP_GCR);
- if (config->default_protdesc)
- protdesc =
- (struct msp_protdesc *)&prot_descs[config->protocol];
- else
- protdesc = (struct msp_protdesc *)&config->protdesc;
-
switch (config->protocol) {
case MSP_PCM_PROTOCOL:
case MSP_PCM_COMPAND_PROTOCOL:
- frame_width = protdesc->frame_width;
- sck_div = config->f_inputclk / (config->frame_freq *
- (protdesc->clocks_per_frame));
- frame_per = protdesc->frame_period;
- break;
case MSP_I2S_PROTOCOL:
- frame_width = protdesc->frame_width;
- sck_div = config->f_inputclk / (config->frame_freq *
- (protdesc->clocks_per_frame));
- frame_per = protdesc->frame_period;
break;
default:
dev_err(msp->dev, "%s: ERROR: Unknown protocol (%d)!\n",
@@ -249,12 +236,35 @@ static int setup_bitclk(struct ux500_msp *msp, struct ux500_msp_config *config)
return -EINVAL;
}
+ if (config->default_protdesc)
+ protdesc = (struct msp_protdesc *)&prot_descs[config->protocol];
+ else
+ protdesc = &config->protdesc;
+
+ if (!config->frame_freq || !protdesc->clocks_per_frame)
+ return -EINVAL;
+
+ desired_bitclk = (u64)config->frame_freq * protdesc->clocks_per_frame;
+ if (desired_bitclk > config->f_inputclk)
+ return -EINVAL;
+ bitclk = desired_bitclk;
+ if (config->f_inputclk % bitclk) {
+ dev_err(msp->dev,
+ "Input clock %u cannot generate bit clock %u\n",
+ config->f_inputclk, bitclk);
+ return -EINVAL;
+ }
+
+ sck_div = config->f_inputclk / bitclk;
+ if (!sck_div || sck_div > SCK_DIV_MASK + 1)
+ return -EINVAL;
+
temp_reg = (sck_div - 1) & SCK_DIV_MASK;
- temp_reg |= FRAME_WIDTH_BITS(frame_width);
- temp_reg |= FRAME_PERIOD_BITS(frame_per);
+ temp_reg |= FRAME_WIDTH_BITS(protdesc->frame_width);
+ temp_reg |= FRAME_PERIOD_BITS(protdesc->frame_period);
writel(temp_reg, msp->registers + MSP_SRG);
- msp->f_bitclk = (config->f_inputclk)/(sck_div + 1);
+ msp->f_bitclk = config->f_inputclk / sck_div;
/* Enable bit-clock */
udelay(100);
@@ -344,20 +354,27 @@ static int configure_multichannel(struct ux500_msp *msp,
return 0;
}
-static int enable_msp(struct ux500_msp *msp, struct ux500_msp_config *config)
+static int enable_msp(struct ux500_msp *msp, struct ux500_msp_config *config,
+ bool first)
{
- int status = 0;
- u32 reg_val_DMACR, reg_val_GCR;
+ int status;
+ u32 reg_val_DMACR;
/* Configure msp with protocol dependent settings */
- configure_protocol(msp, config);
- setup_bitclk(msp, config);
+ status = configure_protocol(msp, config);
+ if (status)
+ return status;
+
+ if (first && config->clock_provider) {
+ status = setup_bitclk(msp, config);
+ if (status)
+ return status;
+ }
+
if (config->multichannel_configured == 1) {
status = configure_multichannel(msp, config);
if (status)
- dev_warn(msp->dev,
- "%s: WARN: configure_multichannel failed (%d)!\n",
- __func__, status);
+ return status;
}
reg_val_DMACR = readl(msp->registers + MSP_DMACR);
@@ -369,11 +386,7 @@ static int enable_msp(struct ux500_msp *msp, struct ux500_msp_config *config)
writel(config->iodelay, msp->registers + MSP_IODLY);
- /* Enable frame generation logic */
- reg_val_GCR = readl(msp->registers + MSP_GCR);
- writel(reg_val_GCR | FRAME_GEN_ENABLE, msp->registers + MSP_GCR);
-
- return status;
+ return 0;
}
static void flush_fifo_rx(struct ux500_msp *msp)
@@ -411,12 +424,37 @@ static void flush_fifo_tx(struct ux500_msp *msp)
writel(reg_val_GCR, msp->registers + MSP_GCR);
}
+static bool ux500_msp_config_compatible(struct ux500_msp *msp,
+ struct ux500_msp_config *config)
+{
+ struct ux500_msp_config *active = &msp->config;
+
+ return active->f_inputclk == config->f_inputclk &&
+ active->tx_clk_sel == config->tx_clk_sel &&
+ active->rx_clk_sel == config->rx_clk_sel &&
+ active->srg_clk_sel == config->srg_clk_sel &&
+ active->rx_fsync_pol == config->rx_fsync_pol &&
+ active->tx_fsync_pol == config->tx_fsync_pol &&
+ active->rx_fsync_sel == config->rx_fsync_sel &&
+ active->tx_fsync_sel == config->tx_fsync_sel &&
+ active->default_protdesc == config->default_protdesc &&
+ active->protocol == config->protocol &&
+ active->frame_freq == config->frame_freq &&
+ active->data_size == config->data_size &&
+ active->def_elem_len == config->def_elem_len &&
+ active->clock_provider == config->clock_provider &&
+ active->bclk_inverted == config->bclk_inverted &&
+ !memcmp(&active->protdesc, &config->protdesc,
+ sizeof(active->protdesc));
+}
+
int ux500_msp_i2s_open(struct ux500_msp *msp,
struct ux500_msp_config *config)
{
u32 old_reg, new_reg, mask;
int res;
unsigned int tx_sel, rx_sel, tx_busy, rx_busy;
+ bool first;
if (in_interrupt()) {
dev_err(msp->dev,
@@ -444,40 +482,68 @@ int ux500_msp_i2s_open(struct ux500_msp *msp,
return -EBUSY;
}
- msp->dir_busy |= (tx_sel ? MSP_DIR_TX : 0) | (rx_sel ? MSP_DIR_RX : 0);
-
- /* First do the global config register */
- mask = RX_CLK_SEL_MASK | TX_CLK_SEL_MASK | RX_FSYNC_MASK |
- TX_FSYNC_MASK | RX_SYNC_SEL_MASK | TX_SYNC_SEL_MASK |
- RX_FIFO_ENABLE_MASK | TX_FIFO_ENABLE_MASK | SRG_CLK_SEL_MASK |
- LOOPBACK_MASK | TX_EXTRA_DELAY_MASK;
-
- new_reg = (config->tx_clk_sel | config->rx_clk_sel |
- config->rx_fsync_pol | config->tx_fsync_pol |
- config->rx_fsync_sel | config->tx_fsync_sel |
- config->rx_fifo_config | config->tx_fifo_config |
- config->srg_clk_sel | config->loopback_enable |
- config->tx_data_enable);
+ first = !msp->dir_busy;
+ if (!first && !ux500_msp_config_compatible(msp, config)) {
+ dev_err(msp->dev, "%s: Incompatible duplex configuration\n",
+ __func__);
+ return -EBUSY;
+ }
- old_reg = readl(msp->registers + MSP_GCR);
- old_reg &= ~mask;
- new_reg |= old_reg;
- writel(new_reg, msp->registers + MSP_GCR);
+ if (first) {
+ /* First do the global config register */
+ mask = RX_CLK_SEL_MASK | TX_CLK_SEL_MASK | RX_FSYNC_MASK |
+ TX_FSYNC_MASK | RX_SYNC_SEL_MASK | TX_SYNC_SEL_MASK |
+ RX_FIFO_ENABLE_MASK | TX_FIFO_ENABLE_MASK |
+ SRG_CLK_SEL_MASK | LOOPBACK_MASK | TX_EXTRA_DELAY_MASK;
+
+ new_reg = config->tx_clk_sel | config->rx_clk_sel |
+ config->rx_fsync_pol | config->tx_fsync_pol |
+ config->rx_fsync_sel | config->tx_fsync_sel |
+ config->rx_fifo_config | config->tx_fifo_config |
+ config->srg_clk_sel | config->loopback_enable |
+ config->tx_data_enable;
+
+ old_reg = readl(msp->registers + MSP_GCR);
+ old_reg &= ~mask;
+ new_reg |= old_reg;
+ writel(new_reg, msp->registers + MSP_GCR);
+ writel(MSP_WMRK_TX_4_ELEMENTS | MSP_WMRK_RX_4_ELEMENTS,
+ msp->registers + MSP_WMRK);
+ }
- res = enable_msp(msp, config);
+ res = enable_msp(msp, config, first);
if (res < 0) {
dev_err(msp->dev, "%s: ERROR: enable_msp failed (%d)!\n",
__func__, res);
- return -EBUSY;
+ if (tx_sel)
+ writel(0, msp->registers + MSP_TCF);
+ if (rx_sel)
+ writel(0, msp->registers + MSP_RCF);
+ if (first) {
+ writel(0, msp->registers + MSP_GCR);
+ writel(0, msp->registers + MSP_DMACR);
+ writel(0, msp->registers + MSP_SRG);
+ writel(0, msp->registers + MSP_MCR);
+ }
+ return res;
+ }
+
+ msp->dir_busy |= config->direction;
+ if (first) {
+ msp->config = *config;
+ msp->clock_provider = config->clock_provider;
}
if (config->loopback_enable & 0x80)
msp->loopback_enable = 1;
/* Flush FIFOs */
- flush_fifo_tx(msp);
- flush_fifo_rx(msp);
+ if (tx_sel)
+ flush_fifo_tx(msp);
+ if (rx_sel)
+ flush_fifo_rx(msp);
- msp->msp_state = MSP_STATE_CONFIGURED;
+ if (!msp->dir_running)
+ msp->msp_state = MSP_STATE_CONFIGURED;
return 0;
}
@@ -494,7 +560,6 @@ static void disable_msp_rx(struct ux500_msp *msp)
~(RX_SERVICE_INT | RX_OVERRUN_ERROR_INT),
msp->registers + MSP_IMSC);
- msp->dir_busy &= ~MSP_DIR_RX;
}
static void disable_msp_tx(struct ux500_msp *msp)
@@ -510,7 +575,6 @@ static void disable_msp_tx(struct ux500_msp *msp)
~(TX_SERVICE_INT | TX_UNDERRUN_ERR_INT),
msp->registers + MSP_IMSC);
- msp->dir_busy &= ~MSP_DIR_TX;
}
static int disable_msp(struct ux500_msp *msp, unsigned int dir)
@@ -520,7 +584,7 @@ static int disable_msp(struct ux500_msp *msp, unsigned int dir)
reg_val_GCR = readl(msp->registers + MSP_GCR);
disable_tx = dir & MSP_DIR_TX;
- disable_rx = dir & MSP_DIR_TX;
+ disable_rx = dir & MSP_DIR_RX;
if (disable_tx && disable_rx) {
reg_val_GCR = readl(msp->registers + MSP_GCR);
writel(reg_val_GCR | LOOPBACK_MASK,
@@ -553,7 +617,15 @@ static int disable_msp(struct ux500_msp *msp, unsigned int dir)
int ux500_msp_i2s_trigger(struct ux500_msp *msp, int cmd, int direction)
{
- u32 reg_val_GCR, enable_bit;
+ u32 reg_val_DMACR, reg_val_GCR, dma_enable_bit, enable_bit;
+ unsigned int dir;
+
+ if (direction == SNDRV_PCM_STREAM_PLAYBACK)
+ dir = MSP_DIR_TX;
+ else if (direction == SNDRV_PCM_STREAM_CAPTURE)
+ dir = MSP_DIR_RX;
+ else
+ return -EINVAL;
if (msp->msp_state == MSP_STATE_IDLE) {
dev_err(msp->dev, "%s: ERROR: MSP is not configured!\n",
@@ -565,21 +637,44 @@ int ux500_msp_i2s_trigger(struct ux500_msp *msp, int cmd, int direction)
case SNDRV_PCM_TRIGGER_START:
case SNDRV_PCM_TRIGGER_RESUME:
case SNDRV_PCM_TRIGGER_PAUSE_RELEASE:
- if (direction == SNDRV_PCM_STREAM_PLAYBACK)
+ if (direction == SNDRV_PCM_STREAM_PLAYBACK) {
enable_bit = TX_ENABLE;
- else
+ dma_enable_bit = TX_DMA_ENABLE;
+ } else {
enable_bit = RX_ENABLE;
+ dma_enable_bit = RX_DMA_ENABLE;
+ }
+ if (!(msp->dir_busy & dir))
+ return -EINVAL;
+ reg_val_DMACR = readl(msp->registers + MSP_DMACR);
+ writel(reg_val_DMACR | dma_enable_bit,
+ msp->registers + MSP_DMACR);
reg_val_GCR = readl(msp->registers + MSP_GCR);
+ if (msp->clock_provider)
+ enable_bit |= FRAME_GEN_ENABLE;
writel(reg_val_GCR | enable_bit, msp->registers + MSP_GCR);
+ msp->dir_running |= dir;
+ msp->msp_state = MSP_STATE_RUNNING;
break;
case SNDRV_PCM_TRIGGER_STOP:
case SNDRV_PCM_TRIGGER_SUSPEND:
case SNDRV_PCM_TRIGGER_PAUSE_PUSH:
- if (direction == SNDRV_PCM_STREAM_PLAYBACK)
+ if (!(msp->dir_busy & dir))
+ return -EINVAL;
+ if (direction == SNDRV_PCM_STREAM_PLAYBACK) {
disable_msp_tx(msp);
- else
+ msp->dir_running &= ~MSP_DIR_TX;
+ } else {
disable_msp_rx(msp);
+ msp->dir_running &= ~MSP_DIR_RX;
+ }
+ if (!msp->dir_running) {
+ reg_val_GCR = readl(msp->registers + MSP_GCR);
+ writel(reg_val_GCR & ~FRAME_GEN_ENABLE,
+ msp->registers + MSP_GCR);
+ msp->msp_state = MSP_STATE_CONFIGURED;
+ }
break;
default:
return -EINVAL;
@@ -594,7 +689,18 @@ int ux500_msp_i2s_close(struct ux500_msp *msp, unsigned int dir)
dev_dbg(msp->dev, "%s: Enter (dir = 0x%01x).\n", __func__, dir);
+ if (!dir || dir & ~(MSP_DIR_TX | MSP_DIR_RX) ||
+ (msp->dir_busy & dir) != dir)
+ return -EINVAL;
+
status = disable_msp(msp, dir);
+ msp->dir_busy &= ~dir;
+ msp->dir_running &= ~dir;
+ if (msp->dir_busy && !msp->dir_running) {
+ writel(readl(msp->registers + MSP_GCR) & ~FRAME_GEN_ENABLE,
+ msp->registers + MSP_GCR);
+ msp->msp_state = MSP_STATE_CONFIGURED;
+ }
if (msp->dir_busy == 0) {
/* disable sample rate and frame generators */
msp->msp_state = MSP_STATE_IDLE;
@@ -618,6 +724,8 @@ int ux500_msp_i2s_close(struct ux500_msp *msp, unsigned int dir)
writel(0, msp->registers + MSP_RCE1);
writel(0, msp->registers + MSP_RCE2);
writel(0, msp->registers + MSP_RCE3);
+ memset(&msp->config, 0, sizeof(msp->config));
+ msp->clock_provider = false;
}
return status;
@@ -627,7 +735,7 @@ int ux500_msp_i2s_close(struct ux500_msp *msp, unsigned int dir)
int ux500_msp_i2s_init_msp(struct platform_device *pdev,
struct ux500_msp **msp_p)
{
- struct resource *res = NULL;
+ struct resource *res;
struct ux500_msp *msp;
*msp_p = devm_kzalloc(&pdev->dev, sizeof(struct ux500_msp), GFP_KERNEL);
@@ -637,20 +745,10 @@ int ux500_msp_i2s_init_msp(struct platform_device *pdev,
msp->dev = &pdev->dev;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (res == NULL) {
- dev_err(&pdev->dev, "%s: ERROR: Unable to get resource!\n",
- __func__);
- return -ENOMEM;
- }
-
+ msp->registers = devm_platform_get_and_ioremap_resource(pdev, 0, &res);
+ if (IS_ERR(msp->registers))
+ return PTR_ERR(msp->registers);
msp->tx_rx_addr = res->start + MSP_DR;
- msp->registers = devm_ioremap(&pdev->dev, res->start,
- resource_size(res));
- if (msp->registers == NULL) {
- dev_err(&pdev->dev, "%s: ERROR: ioremap failed!\n", __func__);
- return -ENOMEM;
- }
msp->msp_state = MSP_STATE_IDLE;
msp->loopback_enable = 0;
diff --git a/sound/soc/ux500/ux500_msp_i2s.h b/sound/soc/ux500/ux500_msp_i2s.h
index 69d4ebc409fc..2bf2699bdc49 100644
--- a/sound/soc/ux500/ux500_msp_i2s.h
+++ b/sound/soc/ux500/ux500_msp_i2s.h
@@ -12,8 +12,6 @@
#include <linux/platform_device.h>
-#define MSP_INPUT_FREQ_APB 48000000
-
/*** Stereo mode. Used for APB data accesses as 16 bits accesses (mono),
* 32 bits accesses (stereo).
***/
@@ -64,6 +62,7 @@ enum msp_direction {
#define MSP_SRG 0x10
#define MSP_FLR 0x14
#define MSP_DMACR 0x18
+#define MSP_WMRK 0x1c
#define MSP_IMSC 0x20
#define MSP_RIS 0x24
@@ -230,6 +229,10 @@ enum msp_direction {
#define RDMAE_SHIFT 0
#define TDMAE_SHIFT 1
+/* FIFO watermark register */
+#define MSP_WMRK_RX_4_ELEMENTS BIT(0)
+#define MSP_WMRK_TX_4_ELEMENTS BIT(3)
+
/* Interrupt Register */
#define RX_SERVICE_INT BIT(0)
#define RX_OVERRUN_ERROR_INT BIT(1)
@@ -460,6 +463,8 @@ struct ux500_msp_config {
enum msp_data_size data_size;
unsigned int def_elem_len;
unsigned int iodelay;
+ bool clock_provider;
+ bool bclk_inverted;
};
struct ux500_msp {
@@ -470,8 +475,11 @@ struct ux500_msp {
enum msp_state msp_state;
int def_elem_len;
unsigned int dir_busy;
+ unsigned int dir_running;
int loopback_enable;
unsigned int f_bitclk;
+ bool clock_provider;
+ struct ux500_msp_config config;
};
int ux500_msp_i2s_init_msp(struct platform_device *pdev,
diff --git a/sound/usb/caiaq/audio.c b/sound/usb/caiaq/audio.c
index ba3f73455ebe..bb6280aa3533 100644
--- a/sound/usb/caiaq/audio.c
+++ b/sound/usb/caiaq/audio.c
@@ -828,16 +828,13 @@ int snd_usb_caiaq_audio_init(struct snd_usb_caiaqdev *cdev)
cdev->data_urbs_in = alloc_urbs(cdev, SNDRV_PCM_STREAM_CAPTURE, &ret);
if (ret < 0) {
- kfree(cdev->data_cb_info);
- free_urbs(cdev->data_urbs_in);
+ snd_usb_caiaq_audio_free(cdev);
return ret;
}
cdev->data_urbs_out = alloc_urbs(cdev, SNDRV_PCM_STREAM_PLAYBACK, &ret);
if (ret < 0) {
- kfree(cdev->data_cb_info);
- free_urbs(cdev->data_urbs_in);
- free_urbs(cdev->data_urbs_out);
+ snd_usb_caiaq_audio_free(cdev);
return ret;
}
@@ -858,6 +855,9 @@ void snd_usb_caiaq_audio_free(struct snd_usb_caiaqdev *cdev)
dev_dbg(dev, "%s(%p)\n", __func__, cdev);
free_urbs(cdev->data_urbs_in);
+ cdev->data_urbs_in = NULL;
free_urbs(cdev->data_urbs_out);
+ cdev->data_urbs_out = NULL;
kfree(cdev->data_cb_info);
+ cdev->data_cb_info = NULL;
}
diff --git a/sound/usb/caiaq/device.c b/sound/usb/caiaq/device.c
index a16e59248480..3d821fde4582 100644
--- a/sound/usb/caiaq/device.c
+++ b/sound/usb/caiaq/device.c
@@ -192,8 +192,8 @@ static void usb_ep1_command_reply_dispatch (struct urb* urb)
break;
}
- cdev->ep1_in_urb.actual_length = 0;
- ret = usb_submit_urb(&cdev->ep1_in_urb, GFP_ATOMIC);
+ cdev->ep1_in_urb->actual_length = 0;
+ ret = usb_submit_urb(cdev->ep1_in_urb, GFP_ATOMIC);
if (ret < 0)
dev_err(dev, "unable to submit urb. OOM!?\n");
}
@@ -408,6 +408,10 @@ static void card_free(struct snd_card *card)
#endif
snd_usb_caiaq_audio_free(cdev);
usb_put_dev(cdev->chip.dev);
+ usb_free_urb(cdev->ep1_in_urb);
+ cdev->ep1_in_urb = NULL;
+ usb_free_urb(cdev->midi_out_urb);
+ cdev->midi_out_urb = NULL;
}
static int create_card(struct usb_device *usb_dev,
@@ -457,22 +461,30 @@ static int init_card(struct snd_usb_caiaqdev *cdev)
return -EIO;
}
- usb_init_urb(&cdev->ep1_in_urb);
- usb_init_urb(&cdev->midi_out_urb);
+ cdev->ep1_in_urb = usb_alloc_urb(0, GFP_KERNEL);
+ if (!cdev->ep1_in_urb)
+ return -ENOMEM;
- usb_fill_bulk_urb(&cdev->ep1_in_urb, usb_dev,
+ cdev->midi_out_urb = usb_alloc_urb(0, GFP_KERNEL);
+ if (!cdev->midi_out_urb) {
+ usb_free_urb(cdev->ep1_in_urb);
+ cdev->ep1_in_urb = NULL;
+ return -ENOMEM;
+ }
+
+ usb_fill_bulk_urb(cdev->ep1_in_urb, usb_dev,
usb_rcvbulkpipe(usb_dev, 0x1),
cdev->ep1_in_buf, EP1_BUFSIZE,
usb_ep1_command_reply_dispatch, cdev);
- usb_fill_bulk_urb(&cdev->midi_out_urb, usb_dev,
+ usb_fill_bulk_urb(cdev->midi_out_urb, usb_dev,
usb_sndbulkpipe(usb_dev, 0x1),
cdev->midi_out_buf, EP1_BUFSIZE,
snd_usb_caiaq_midi_output_done, cdev);
/* sanity checks of EPs before actually submitting */
- if (usb_urb_ep_type_check(&cdev->ep1_in_urb) ||
- usb_urb_ep_type_check(&cdev->midi_out_urb)) {
+ if (usb_urb_ep_type_check(cdev->ep1_in_urb) ||
+ usb_urb_ep_type_check(cdev->midi_out_urb)) {
dev_err(dev, "invalid EPs\n");
return -EINVAL;
}
@@ -480,7 +492,7 @@ static int init_card(struct snd_usb_caiaqdev *cdev)
init_waitqueue_head(&cdev->ep1_wait_queue);
init_waitqueue_head(&cdev->prepare_wait_queue);
- if (usb_submit_urb(&cdev->ep1_in_urb, GFP_KERNEL) != 0)
+ if (usb_submit_urb(cdev->ep1_in_urb, GFP_KERNEL) != 0)
return -EIO;
err = snd_usb_caiaq_send_command(cdev, EP1_CMD_GET_DEVICE_INFO, NULL, 0);
@@ -530,7 +542,7 @@ static int init_card(struct snd_usb_caiaqdev *cdev)
return 0;
err_kill_urb:
- usb_kill_urb(&cdev->ep1_in_urb);
+ usb_kill_urb(cdev->ep1_in_urb);
return err;
}
@@ -576,8 +588,8 @@ static void snd_disconnect(struct usb_interface *intf)
#endif
snd_usb_caiaq_audio_disconnect(cdev);
- usb_kill_urb(&cdev->ep1_in_urb);
- usb_kill_urb(&cdev->midi_out_urb);
+ usb_kill_urb(cdev->ep1_in_urb);
+ usb_kill_urb(cdev->midi_out_urb);
snd_card_free_when_closed(card);
}
diff --git a/sound/usb/caiaq/device.h b/sound/usb/caiaq/device.h
index 743eb0387b5f..1c6f34693fa8 100644
--- a/sound/usb/caiaq/device.h
+++ b/sound/usb/caiaq/device.h
@@ -60,8 +60,8 @@ struct snd_usb_caiaq_cb_info;
struct snd_usb_caiaqdev {
struct snd_usb_audio chip;
- struct urb ep1_in_urb;
- struct urb midi_out_urb;
+ struct urb *ep1_in_urb;
+ struct urb *midi_out_urb;
struct urb **data_urbs_in;
struct urb **data_urbs_out;
struct snd_usb_caiaq_cb_info *data_cb_info;
diff --git a/sound/usb/caiaq/midi.c b/sound/usb/caiaq/midi.c
index c656d0162432..18529484c8dc 100644
--- a/sound/usb/caiaq/midi.c
+++ b/sound/usb/caiaq/midi.c
@@ -43,7 +43,7 @@ static int snd_usb_caiaq_midi_output_close(struct snd_rawmidi_substream *substre
{
struct snd_usb_caiaqdev *cdev = substream->rmidi->private_data;
if (cdev->midi_out_active) {
- usb_kill_urb(&cdev->midi_out_urb);
+ usb_kill_urb(cdev->midi_out_urb);
cdev->midi_out_active = 0;
}
return 0;
@@ -64,9 +64,9 @@ static void snd_usb_caiaq_midi_send(struct snd_usb_caiaqdev *cdev,
return;
cdev->midi_out_buf[2] = len;
- cdev->midi_out_urb.transfer_buffer_length = len+3;
+ cdev->midi_out_urb->transfer_buffer_length = len+3;
- ret = usb_submit_urb(&cdev->midi_out_urb, GFP_ATOMIC);
+ ret = usb_submit_urb(cdev->midi_out_urb, GFP_ATOMIC);
if (ret < 0)
dev_err(dev,
"snd_usb_caiaq_midi_send(%p): usb_submit_urb() failed,"
diff --git a/sound/usb/usx2y/us122l.c b/sound/usb/usx2y/us122l.c
index f00b53346abd..a5db0d044ef9 100644
--- a/sound/usb/usx2y/us122l.c
+++ b/sound/usb/usx2y/us122l.c
@@ -180,8 +180,11 @@ static int usb_stream_hwdep_mmap(struct snd_hwdep *hw,
guard(mutex)(&us122l->mutex);
s = us122l->sk.s;
read = offset < s->read_size;
- if (read && area->vm_flags & VM_WRITE)
- return -EPERM;
+ if (read) {
+ if (area->vm_flags & VM_WRITE)
+ return -EPERM;
+ vm_flags_clear(area, VM_MAYWRITE);
+ }
/* if userspace tries to mmap beyond end of our buffer, fail */
if (size > PAGE_ALIGN(read ? s->read_size : s->write_size)) {
dev_warn(hw->card->dev, "%s: size %lu > %u\n", __func__,
diff --git a/sound/usb/usx2y/usbusx2y.c b/sound/usb/usx2y/usbusx2y.c
index 4190227c5a2a..108f9bddf9aa 100644
--- a/sound/usb/usx2y/usbusx2y.c
+++ b/sound/usb/usx2y/usbusx2y.c
@@ -189,6 +189,9 @@ static void i_usx2y_in04_int(struct urb *urb)
return;
}
+ if (urb->actual_length < USX2Y_IN04_SIZE)
+ goto resubmit;
+
if (us428ctls) {
diff = -1;
if (us428ctls->ctl_snapshot_last == -2) {
@@ -196,7 +199,7 @@ static void i_usx2y_in04_int(struct urb *urb)
memcpy(usx2y->in04_last, usx2y->in04_buf, sizeof(usx2y->in04_last));
us428ctls->ctl_snapshot_last = -1;
} else {
- for (i = 0; i < 21; i++) {
+ for (i = 0; i < USX2Y_IN04_SIZE; i++) {
if (usx2y->in04_last[i] != ((char *)usx2y->in04_buf)[i]) {
if (diff < 0)
diff = i;
@@ -253,6 +256,7 @@ static void i_usx2y_in04_int(struct urb *urb)
if (err)
dev_err(&urb->dev->dev, "in04_int() usb_submit_urb err=%i\n", err);
+resubmit:
urb->dev = usx2y->dev;
usb_submit_urb(urb, GFP_ATOMIC);
}
@@ -305,7 +309,7 @@ int usx2y_in04_init(struct usx2ydev *usx2y)
goto error;
}
- usx2y->in04_buf = kmalloc(21, GFP_KERNEL);
+ usx2y->in04_buf = kzalloc(USX2Y_IN04_SIZE, GFP_KERNEL);
if (!usx2y->in04_buf) {
err = -ENOMEM;
goto error;
@@ -313,7 +317,7 @@ int usx2y_in04_init(struct usx2ydev *usx2y)
init_waitqueue_head(&usx2y->in04_wait_queue);
usb_fill_int_urb(usx2y->in04_urb, usx2y->dev, usb_rcvintpipe(usx2y->dev, 0x4),
- usx2y->in04_buf, 21,
+ usx2y->in04_buf, USX2Y_IN04_SIZE,
i_usx2y_in04_int, usx2y,
10);
if (usb_urb_ep_type_check(usx2y->in04_urb)) {
diff --git a/sound/usb/usx2y/usbusx2y.h b/sound/usb/usx2y/usbusx2y.h
index 6a76d04bf1c7..7b6deed17d4b 100644
--- a/sound/usb/usx2y/usbusx2y.h
+++ b/sound/usb/usx2y/usbusx2y.h
@@ -5,6 +5,8 @@
#include "../midi.h"
#include "usbus428ctldefs.h"
+#define USX2Y_IN04_SIZE sizeof(struct us428_ctls)
+
#define NRURBS 2
/* Default value used for nr of packs per urb.
@@ -55,7 +57,7 @@ struct usx2ydev {
int stride;
struct urb *in04_urb;
void *in04_buf;
- char in04_last[24];
+ char in04_last[USX2Y_IN04_SIZE];
unsigned int in04_int_calls;
struct snd_usx2y_urb_seq *us04;
wait_queue_head_t in04_wait_queue;
diff --git a/tools/bootconfig/main.c b/tools/bootconfig/main.c
index 7dc9fff9b637..17d971d47f87 100644
--- a/tools/bootconfig/main.c
+++ b/tools/bootconfig/main.c
@@ -140,6 +140,9 @@ static int load_xbc_fd(int fd, char **buf, int size)
{
int ret;
+ if (size < 0 || size > XBC_DATA_MAX)
+ return -EINVAL;
+
*buf = malloc(size + 1);
if (!*buf)
return -ENOMEM;
@@ -168,6 +171,13 @@ static int load_xbc_file(const char *path, char **buf)
return ret;
}
+ if (stat.st_size > XBC_DATA_MAX) {
+ pr_err("%s size is too big\n", path);
+ ret = -E2BIG;
+ close(fd);
+ return ret;
+ }
+
ret = load_xbc_fd(fd, buf, stat.st_size);
close(fd);
@@ -218,7 +228,8 @@ static int load_xbc_from_initrd(int fd, char **buf)
csum = le32toh(csum);
/* Wrong size error */
- if (stat.st_size < size + BOOTCONFIG_FOOTER_SIZE) {
+ if (size > XBC_DATA_MAX ||
+ size > stat.st_size - BOOTCONFIG_FOOTER_SIZE) {
pr_err("bootconfig size is too big\n");
return -E2BIG;
}
diff --git a/tools/objtool/klp-checksum.c b/tools/objtool/klp-checksum.c
index b8e47f28997e..ebe25f9c5260 100644
--- a/tools/objtool/klp-checksum.c
+++ b/tools/objtool/klp-checksum.c
@@ -54,6 +54,19 @@ static int checksum_debug_init(struct objtool_file *file)
return 0;
}
+/*
+ * Detect a reference to anonymous constant pool data which the compiler places
+ * in .rodata.cst<num> and which either has an .LC<num> symbol associated with
+ * it or (with Clang) no symbol at all. These are typically initializers for
+ * local function stack data, so they're considered part of the function rather
+ * than data per se.
+ */
+static bool is_anonymous_const_data(struct symbol *sym)
+{
+ return strstarts(sym->sec->name, ".rodata.cst") &&
+ (is_sec_sym(sym) || strstarts(sym->name, ".LC"));
+}
+
static void checksum_update_insn(struct objtool_file *file, struct symbol *func,
struct instruction *insn)
{
@@ -129,6 +142,14 @@ static void checksum_update_insn(struct objtool_file *file, struct symbol *func,
goto alts;
}
+ if (is_anonymous_const_data(sym)) {
+ void *cst;
+
+ cst = sym->sec->data->d_buf + sym->offset + offset;
+ __checksum_update_insn(func, insn, cst, sym->sec->sh.sh_entsize);
+ goto alts;
+ }
+
if (is_sec_sym(sym)) {
sym = find_symbol_containing(reloc->sym->sec, offset);
if (!sym)
diff --git a/tools/perf/util/powerpc-vpadtl.c b/tools/perf/util/powerpc-vpadtl.c
index 710f3093f3f9..af6783cfdb53 100644
--- a/tools/perf/util/powerpc-vpadtl.c
+++ b/tools/perf/util/powerpc-vpadtl.c
@@ -196,7 +196,7 @@ static int powerpc_vpadtl_sample(struct powerpc_vpadtl_entry *record,
sample.cpumode = PERF_RECORD_MISC_KERNEL;
sample.time = save;
sample.raw_data = record;
- sample.raw_size = sizeof(record);
+ sample.raw_size = sizeof(*record);
event.sample.header.type = PERF_RECORD_SAMPLE;
event.sample.header.misc = sample.cpumode;
event.sample.header.size = sizeof(struct perf_event_header);
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index 35104a56d8e3..1de5796fc4ff 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -1947,7 +1947,16 @@ int dso__load(struct dso *dso, struct map *map)
if (next_slot) {
ss_pos++;
- if (dso__binary_type(dso) == DSO_BINARY_TYPE__NOT_FOUND)
+ /*
+ * The binary type is used to find the file containing
+ * the executed instructions, so prefer the types that
+ * refer to the actual object over debug-only files such
+ * as DSO_BINARY_TYPE__DEBUGLINK.
+ */
+ if (dso__binary_type(dso) == DSO_BINARY_TYPE__NOT_FOUND ||
+ symtab_type == DSO_BINARY_TYPE__BUILD_ID_CACHE ||
+ (symtab_type == DSO_BINARY_TYPE__SYSTEM_PATH_DSO &&
+ dso__binary_type(dso) != DSO_BINARY_TYPE__BUILD_ID_CACHE))
dso__set_binary_type(dso, symtab_type);
if (syms_ss && runtime_ss)
diff --git a/tools/sched_ext/scx_central.bpf.c b/tools/sched_ext/scx_central.bpf.c
index 64dd60b3e922..65dae9e45400 100644
--- a/tools/sched_ext/scx_central.bpf.c
+++ b/tools/sched_ext/scx_central.bpf.c
@@ -299,6 +299,7 @@ static int central_timerfn(void *map, int *key, struct bpf_timer *timer)
u64 now = scx_bpf_now();
u64 nr_to_kick = nr_queued;
s32 i, curr_cpu;
+ int ret;
curr_cpu = bpf_get_smp_processor_id();
if (timer_pinned && (curr_cpu != central_cpu)) {
@@ -332,7 +333,10 @@ static int central_timerfn(void *map, int *key, struct bpf_timer *timer)
scx_bpf_kick_cpu(cpu, SCX_KICK_PREEMPT);
}
- bpf_timer_start(timer, TIMER_INTERVAL_NS, BPF_F_TIMER_CPU_PIN);
+ ret = bpf_timer_start(timer, TIMER_INTERVAL_NS,
+ timer_pinned ? BPF_F_TIMER_CPU_PIN : 0);
+ if (ret)
+ scx_bpf_error("bpf_timer_start failed (%d)", ret);
__sync_fetch_and_add(&nr_timers, 1);
return 0;
}
diff --git a/tools/sched_ext/scx_flatcg.bpf.c b/tools/sched_ext/scx_flatcg.bpf.c
index ddcf6bc03b11..98fdd4b04033 100644
--- a/tools/sched_ext/scx_flatcg.bpf.c
+++ b/tools/sched_ext/scx_flatcg.bpf.c
@@ -934,7 +934,7 @@ void BPF_STRUCT_OPS(fcg_cgroup_move, struct task_struct *p,
if (!(from_cgc = find_cgrp_ctx(from)) || !(to_cgc = find_cgrp_ctx(to)))
return;
- delta = time_delta(p->scx.dsq_vtime, from_cgc->tvtime_now);
+ delta = (s64)(p->scx.dsq_vtime - from_cgc->tvtime_now);
scx_bpf_task_set_dsq_vtime(p, to_cgc->tvtime_now + delta);
}
diff --git a/tools/testing/selftests/alsa/mixer-test.c b/tools/testing/selftests/alsa/mixer-test.c
index a329f901c5ed..0857d64c322a 100644
--- a/tools/testing/selftests/alsa/mixer-test.c
+++ b/tools/testing/selftests/alsa/mixer-test.c
@@ -319,8 +319,8 @@ static bool ctl_value_index_valid(struct ctl_data *ctl,
/* Only check step size if there is one and we're in bounds */
if (snd_ctl_elem_info_get_step(ctl->info) &&
- (int_val - snd_ctl_elem_info_get_min(ctl->info) %
- snd_ctl_elem_info_get_step(ctl->info))) {
+ (int_val - snd_ctl_elem_info_get_min(ctl->info)) %
+ snd_ctl_elem_info_get_step(ctl->info)) {
ksft_print_msg("%s.%d value %ld invalid for step %ld minimum %ld\n",
ctl->name, index, int_val,
snd_ctl_elem_info_get_step(ctl->info),
diff --git a/tools/testing/selftests/bpf/progs/test_bpf_nf.c b/tools/testing/selftests/bpf/progs/test_bpf_nf.c
index df43649ecb78..eda9b7bbab75 100644
--- a/tools/testing/selftests/bpf/progs/test_bpf_nf.c
+++ b/tools/testing/selftests/bpf/progs/test_bpf_nf.c
@@ -190,8 +190,8 @@ nf_ct_test(struct nf_conn *(*lookup_fn)(void *, struct bpf_sock_tuple *, u32,
ct = alloc_fn(ctx, &bpf_tuple, sizeof(bpf_tuple.ipv4), &opts_def,
sizeof(opts_def));
if (ct) {
- __u16 sport = bpf_get_prandom_u32();
- __u16 dport = bpf_get_prandom_u32();
+ __u16 sport = bpf_get_prandom_u32() % 65535 + 1;
+ __u16 dport = bpf_get_prandom_u32() % 65535 + 1;
union nf_inet_addr saddr = {};
union nf_inet_addr daddr = {};
struct nf_conn *ct_ins;
@@ -293,8 +293,8 @@ nf_ct_opts_new_test(struct nf_conn *(*lookup_fn)(void *, struct bpf_sock_tuple *
ct = alloc_fn(ctx, &bpf_tuple, sizeof(bpf_tuple.ipv4), &opts_def,
sizeof(opts_def));
if (ct) {
- __u16 sport = bpf_get_prandom_u32();
- __u16 dport = bpf_get_prandom_u32();
+ __u16 sport = bpf_get_prandom_u32() % 65535 + 1;
+ __u16 dport = bpf_get_prandom_u32() % 65535 + 1;
union nf_inet_addr saddr = {};
union nf_inet_addr daddr = {};
struct nf_conn *ct_ins;
diff --git a/tools/testing/selftests/bpf/progs/verifier_subprog_precision.c b/tools/testing/selftests/bpf/progs/verifier_subprog_precision.c
index d21d32f6a676..d990532e88b6 100644
--- a/tools/testing/selftests/bpf/progs/verifier_subprog_precision.c
+++ b/tools/testing/selftests/bpf/progs/verifier_subprog_precision.c
@@ -287,9 +287,9 @@ __msg("17: (b7) r0 = 0")
__msg("18: (95) exit")
__msg("returning from callee:")
__msg("to caller at 9:")
-__msg("frame 0: propagating r1,r4")
+__msg("frame 0: propagating r1,r3,r4")
__msg("mark_precise: frame0: last_idx 9 first_idx 9 subseq_idx -1")
-__msg("mark_precise: frame0: regs=r1,r4 stack= before 18: (95) exit")
+__msg("mark_precise: frame0: regs=r1,r3,r4 stack= before 18: (95) exit")
__msg("from 18 to 9: safe")
__naked int callback_result_precise(void)
{
@@ -419,9 +419,9 @@ __msg("to caller at 9:")
/* r1, r4 are always precise for bpf_loop(),
* r6 was marked before backtracking to callback body.
*/
-__msg("frame 0: propagating r1,r4,r6")
+__msg("frame 0: propagating r1,r3,r4,r6")
__msg("mark_precise: frame0: last_idx 9 first_idx 9 subseq_idx -1")
-__msg("mark_precise: frame0: regs=r1,r4,r6 stack= before 16: (95) exit")
+__msg("mark_precise: frame0: regs=r1,r3,r4,r6 stack= before 16: (95) exit")
__msg("mark_precise: frame1: regs= stack= before 15: (b7) r0 = 0")
__msg("mark_precise: frame1: regs= stack= before 9: (85) call bpf_loop")
__msg("mark_precise: frame0: parent state regs= stack=:")
@@ -575,9 +575,9 @@ __msg("to caller at 10:")
/* r1, r4 are always precise for bpf_loop(),
* fp-8 was marked before backtracking to callback body.
*/
-__msg("frame 0: propagating r1,r4,fp-8")
+__msg("frame 0: propagating r1,r3,r4,fp-8")
__msg("mark_precise: frame0: last_idx 10 first_idx 10 subseq_idx -1")
-__msg("mark_precise: frame0: regs=r1,r4 stack=-8 before 18: (95) exit")
+__msg("mark_precise: frame0: regs=r1,r3,r4 stack=-8 before 18: (95) exit")
__msg("mark_precise: frame1: regs= stack= before 17: (b7) r0 = 0")
__msg("mark_precise: frame1: regs= stack= before 10: (85) call bpf_loop#181")
__msg("mark_precise: frame0: parent state regs= stack=:")
diff --git a/tools/testing/selftests/cgroup/test_core.c b/tools/testing/selftests/cgroup/test_core.c
index 88ca832d4fc1..64f4962a4659 100644
--- a/tools/testing/selftests/cgroup/test_core.c
+++ b/tools/testing/selftests/cgroup/test_core.c
@@ -927,7 +927,6 @@ int main(int argc, char *argv[])
int i;
ksft_print_header();
- ksft_set_plan(ARRAY_SIZE(tests));
if (cg_find_unified_root(root, sizeof(root), &nsdelegate)) {
if (setup_named_v1_root(root, sizeof(root), CG_NAMED_NAME))
ksft_exit_skip("cgroup v2 isn't mounted and could not setup named v1 hierarchy\n");
@@ -940,6 +939,7 @@ int main(int argc, char *argv[])
ksft_exit_skip("Failed to set memory controller\n");
post_v2_setup:
+ ksft_set_plan(ARRAY_SIZE(tests));
for (i = 0; i < ARRAY_SIZE(tests); i++) {
switch (tests[i].fn(root)) {
case KSFT_PASS:
diff --git a/tools/testing/selftests/cgroup/test_cpu.c b/tools/testing/selftests/cgroup/test_cpu.c
index 7a40d76b9548..8f96bfd19ab9 100644
--- a/tools/testing/selftests/cgroup/test_cpu.c
+++ b/tools/testing/selftests/cgroup/test_cpu.c
@@ -799,7 +799,6 @@ int main(int argc, char *argv[])
int i;
ksft_print_header();
- ksft_set_plan(ARRAY_SIZE(tests));
if (cg_find_unified_root(root, sizeof(root), NULL))
ksft_exit_skip("cgroup v2 isn't mounted\n");
@@ -807,6 +806,7 @@ int main(int argc, char *argv[])
if (cg_write(root, "cgroup.subtree_control", "+cpu"))
ksft_exit_skip("Failed to set cpu controller\n");
+ ksft_set_plan(ARRAY_SIZE(tests));
for (i = 0; i < ARRAY_SIZE(tests); i++) {
switch (tests[i].fn(root)) {
case KSFT_PASS:
diff --git a/tools/testing/selftests/cgroup/test_cpuset.c b/tools/testing/selftests/cgroup/test_cpuset.c
index c5cf8b56ceb8..52ed8fba685a 100644
--- a/tools/testing/selftests/cgroup/test_cpuset.c
+++ b/tools/testing/selftests/cgroup/test_cpuset.c
@@ -250,7 +250,6 @@ int main(int argc, char *argv[])
int i;
ksft_print_header();
- ksft_set_plan(ARRAY_SIZE(tests));
if (cg_find_unified_root(root, sizeof(root), NULL))
ksft_exit_skip("cgroup v2 isn't mounted\n");
@@ -258,6 +257,7 @@ int main(int argc, char *argv[])
if (cg_write(root, "cgroup.subtree_control", "+cpuset"))
ksft_exit_skip("Failed to set cpuset controller\n");
+ ksft_set_plan(ARRAY_SIZE(tests));
for (i = 0; i < ARRAY_SIZE(tests); i++) {
switch (tests[i].fn(root)) {
case KSFT_PASS:
diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
index b2e60671273e..ebbc5b4def24 100755
--- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
+++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
@@ -791,7 +791,6 @@ check_isolcpus()
EXPECTED_ISOLCPUS=$1
ISCPUS=${CGROUP2}/cpuset.cpus.isolated
ISOLCPUS=$(cat $ISCPUS)
- HKICPUS=$(cat /sys/devices/system/cpu/isolated)
LASTISOLCPU=
SCHED_DOMAINS=/sys/kernel/debug/sched/domains
if [[ $EXPECTED_ISOLCPUS = . ]]
@@ -829,11 +828,6 @@ check_isolcpus()
ISOLCPUS=
EXPECTED_ISOLCPUS=$EXPECTED_SDOMAIN
- #
- # The inverse of HK_TYPE_DOMAIN cpumask in $HKICPUS should match $ISOLCPUS
- #
- [[ "$ISOLCPUS" != "$HKICPUS" ]] && return 1
-
#
# Use the sched domain in debugfs to check isolated CPUs, if available
#
diff --git a/tools/testing/selftests/cgroup/test_freezer.c b/tools/testing/selftests/cgroup/test_freezer.c
index 0569e93fa6b0..f28bb02e9783 100644
--- a/tools/testing/selftests/cgroup/test_freezer.c
+++ b/tools/testing/selftests/cgroup/test_freezer.c
@@ -1491,9 +1491,9 @@ int main(int argc, char *argv[])
int i;
ksft_print_header();
- ksft_set_plan(ARRAY_SIZE(tests));
if (cg_find_unified_root(root, sizeof(root), NULL))
ksft_exit_skip("cgroup v2 isn't mounted\n");
+ ksft_set_plan(ARRAY_SIZE(tests));
for (i = 0; i < ARRAY_SIZE(tests); i++) {
switch (tests[i].fn(root)) {
case KSFT_PASS:
diff --git a/tools/testing/selftests/cgroup/test_kill.c b/tools/testing/selftests/cgroup/test_kill.c
index f6cd23a8ecc7..99cafd9dc013 100644
--- a/tools/testing/selftests/cgroup/test_kill.c
+++ b/tools/testing/selftests/cgroup/test_kill.c
@@ -278,9 +278,9 @@ int main(int argc, char *argv[])
int i;
ksft_print_header();
- ksft_set_plan(ARRAY_SIZE(tests));
if (cg_find_unified_root(root, sizeof(root), NULL))
ksft_exit_skip("cgroup v2 isn't mounted\n");
+ ksft_set_plan(ARRAY_SIZE(tests));
for (i = 0; i < ARRAY_SIZE(tests); i++) {
switch (tests[i].fn(root)) {
case KSFT_PASS:
diff --git a/tools/testing/selftests/cgroup/test_kmem.c b/tools/testing/selftests/cgroup/test_kmem.c
index 1db0ba1226b9..437f2d35f205 100644
--- a/tools/testing/selftests/cgroup/test_kmem.c
+++ b/tools/testing/selftests/cgroup/test_kmem.c
@@ -145,7 +145,7 @@ static int cg_run_in_subcgroups(const char *parent,
return -1;
}
- if (cg_run(child, fn, NULL)) {
+ if (cg_run(child, fn, arg)) {
cg_destroy(child);
free(child);
return -1;
@@ -426,7 +426,6 @@ int main(int argc, char **argv)
int i;
ksft_print_header();
- ksft_set_plan(ARRAY_SIZE(tests));
if (cg_find_unified_root(root, sizeof(root), NULL))
ksft_exit_skip("cgroup v2 isn't mounted\n");
@@ -441,6 +440,7 @@ int main(int argc, char **argv)
if (cg_write(root, "cgroup.subtree_control", "+memory"))
ksft_exit_skip("Failed to set memory controller\n");
+ ksft_set_plan(ARRAY_SIZE(tests));
for (i = 0; i < ARRAY_SIZE(tests); i++) {
switch (tests[i].fn(root)) {
case KSFT_PASS:
diff --git a/tools/testing/selftests/cgroup/test_memcontrol.c b/tools/testing/selftests/cgroup/test_memcontrol.c
index 0ebf796f3cff..3a84d068fbf3 100644
--- a/tools/testing/selftests/cgroup/test_memcontrol.c
+++ b/tools/testing/selftests/cgroup/test_memcontrol.c
@@ -1798,7 +1798,6 @@ int main(int argc, char **argv)
page_size = BUF_SIZE;
ksft_print_header();
- ksft_set_plan(ARRAY_SIZE(tests));
if (cg_find_unified_root(root, sizeof(root), NULL))
ksft_exit_skip("cgroup v2 isn't mounted\n");
@@ -1823,6 +1822,7 @@ int main(int argc, char **argv)
ksft_exit_skip("Failed to query cgroup mount option\n");
has_localevents = proc_status;
+ ksft_set_plan(ARRAY_SIZE(tests));
for (i = 0; i < ARRAY_SIZE(tests); i++) {
switch (tests[i].fn(root)) {
case KSFT_PASS:
diff --git a/tools/testing/selftests/cgroup/test_pids.c b/tools/testing/selftests/cgroup/test_pids.c
index 9a387c815d2c..710109b53dfe 100644
--- a/tools/testing/selftests/cgroup/test_pids.c
+++ b/tools/testing/selftests/cgroup/test_pids.c
@@ -148,7 +148,6 @@ int main(int argc, char **argv)
char root[PATH_MAX];
ksft_print_header();
- ksft_set_plan(ARRAY_SIZE(tests));
if (cg_find_unified_root(root, sizeof(root), NULL))
ksft_exit_skip("cgroup v2 isn't mounted\n");
@@ -163,6 +162,7 @@ int main(int argc, char **argv)
if (cg_write(root, "cgroup.subtree_control", "+pids"))
ksft_exit_skip("Failed to set pids controller\n");
+ ksft_set_plan(ARRAY_SIZE(tests));
for (int i = 0; i < ARRAY_SIZE(tests); i++) {
switch (tests[i].fn(root)) {
case KSFT_PASS:
diff --git a/tools/testing/selftests/cgroup/test_zswap.c b/tools/testing/selftests/cgroup/test_zswap.c
index 49b36ee79160..6e7b89315bbf 100644
--- a/tools/testing/selftests/cgroup/test_zswap.c
+++ b/tools/testing/selftests/cgroup/test_zswap.c
@@ -810,7 +810,6 @@ int main(int argc, char **argv)
page_size = BUF_SIZE;
ksft_print_header();
- ksft_set_plan(ARRAY_SIZE(tests));
if (cg_find_unified_root(root, sizeof(root), NULL))
ksft_exit_skip("cgroup v2 isn't mounted\n");
@@ -827,6 +826,7 @@ int main(int argc, char **argv)
if (cg_write(root, "cgroup.subtree_control", "+memory"))
ksft_exit_skip("Failed to set memory controller\n");
+ ksft_set_plan(ARRAY_SIZE(tests));
for (i = 0; i < ARRAY_SIZE(tests); i++) {
switch (tests[i].fn(root)) {
case KSFT_PASS:
diff --git a/tools/testing/selftests/net/fib_nexthops.sh b/tools/testing/selftests/net/fib_nexthops.sh
index ac868a731694..daceb1ec592a 100755
--- a/tools/testing/selftests/net/fib_nexthops.sh
+++ b/tools/testing/selftests/net/fib_nexthops.sh
@@ -522,6 +522,20 @@ ipv6_fdb_grp_fcnal()
run_cmd "$BRIDGE fdb add 02:02:00:00:00:14 dev vx10 nhid 61 self"
log_test $? 255 "Fdb mac add with nexthop"
+ # fdb entries with a nexthop group cannot be aged out
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:15 dev vx10 nhid 102 self static"
+ log_test $? 0 "Fdb mac add with nexthop group and static state"
+
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic"
+ log_test $? 255 "Fdb mac add with nexthop group and dynamic state"
+
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self"
+ run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic"
+ log_test $? 255 "Fdb mac replace with nexthop group and dynamic state"
+
+ run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic"
+ log_test $? 255 "Fdb mac append with nexthop group and dynamic state"
+
run_cmd "$IP -6 ro add 2001:db8:101::1/128 nhid 66"
log_test $? 2 "Route add with fdb nexthop"
@@ -622,6 +636,20 @@ ipv4_fdb_grp_fcnal()
run_cmd "$BRIDGE fdb add 02:02:00:00:00:14 dev vx10 nhid 12 self"
log_test $? 255 "Fdb mac add with nexthop"
+ # fdb entries with a nexthop group cannot be aged out
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:15 dev vx10 nhid 102 self static"
+ log_test $? 0 "Fdb mac add with nexthop group and static state"
+
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic"
+ log_test $? 255 "Fdb mac add with nexthop group and dynamic state"
+
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self"
+ run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic"
+ log_test $? 255 "Fdb mac replace with nexthop group and dynamic state"
+
+ run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic"
+ log_test $? 255 "Fdb mac append with nexthop group and dynamic state"
+
run_cmd "$IP ro add 172.16.0.0/22 nhid 16"
log_test $? 2 "Route add with fdb nexthop"
diff --git a/tools/testing/selftests/net/mptcp/mptcp_connect.c b/tools/testing/selftests/net/mptcp/mptcp_connect.c
index cbe573c4ab3a..4c339ad449a7 100644
--- a/tools/testing/selftests/net/mptcp/mptcp_connect.c
+++ b/tools/testing/selftests/net/mptcp/mptcp_connect.c
@@ -381,6 +381,9 @@ static int sock_connect_mptcp(const char * const remoteaddr,
hints.ai_family = pf;
+ /* Keep the resolved address alive for the whole execution: it is
+ * used again when reconnecting, and will be released at exit time.
+ */
xgetaddrinfo(remoteaddr, port, &hints, &addr);
for (a = addr; a; a = a->ai_next) {
sock = socket(a->ai_family, a->ai_socktype, proto);
@@ -421,7 +424,6 @@ static int sock_connect_mptcp(const char * const remoteaddr,
sock = -1;
}
- freeaddrinfo(addr);
if (sock != -1)
SOCK_TEST_TCPULP(sock, proto);
return sock;
diff --git a/tools/testing/selftests/net/mptcp/mptcp_lib.sh b/tools/testing/selftests/net/mptcp/mptcp_lib.sh
index 5ef6033775c8..b9d14647f401 100644
--- a/tools/testing/selftests/net/mptcp/mptcp_lib.sh
+++ b/tools/testing/selftests/net/mptcp/mptcp_lib.sh
@@ -108,12 +108,14 @@ mptcp_lib_pr_info() {
mptcp_lib_pr_nstat() {
local ns="${1}"
- local hist="/tmp/${ns}.out"
+ local cache="/tmp/${ns}.out"
+ local hist="/tmp/${ns}.nstat"
- if [ -f "${hist}" ]; then
- awk '$2 != 0 { print " "$0 }' "${hist}"
+ if [ -f "${cache}" ]; then
+ awk '$2 != 0 { print " "$0 }' "${cache}"
else
- ip netns exec "${ns}" nstat -as | grep Tcp
+ NSTAT_HISTORY="${hist}" ip netns exec "${ns}" nstat -s |
+ grep Tcp
fi
}
@@ -414,19 +416,21 @@ mptcp_lib_nstat_get() {
}
# $1: ns, $2: MIB counter
-# Get the counter from the history (mptcp_lib_nstat_{init,get}()) if available.
-# If not, get the counter from nstat ignoring any history.
+# Get the counter from the cache (mptcp_lib_nstat_{init,get}()) if available.
+# If not, get the counter from nstat ignoring any cache, but using the history.
mptcp_lib_get_counter() {
local ns="${1}"
local counter="${2}"
- local hist="/tmp/${ns}.out"
+ local cache="/tmp/${ns}.out"
+ local hist="/tmp/${ns}.nstat"
local count
- if [[ -s "${hist}" && "${counter}" == *"Tcp"* ]]; then
- count=$(awk "/^${counter} / {print \$2; exit}" "${hist}")
+ if [[ -s "${cache}" && "${counter}" == *"Tcp"* ]]; then
+ count=$(awk "/^${counter} / {print \$2; exit}" "${cache}")
else
- count=$(ip netns exec "${ns}" nstat -asz "${counter}" |
- awk 'NR==1 {next} {print $2}')
+ count=$(NSTAT_HISTORY="${hist}" ip netns exec "${ns}" \
+ nstat -sz "${counter}" |
+ awk 'NR==1 {next} {print $2}')
fi
if [ -z "${count}" ]; then
mptcp_lib_fail_if_expected_feature "${counter} counter"
diff --git a/tools/testing/selftests/net/pmtu.sh b/tools/testing/selftests/net/pmtu.sh
index a3323c21f001..c7cd271714ef 100755
--- a/tools/testing/selftests/net/pmtu.sh
+++ b/tools/testing/selftests/net/pmtu.sh
@@ -1457,7 +1457,7 @@ test_pmtu_ipvX_over_bridged_vxlanY_or_geneveY_exception() {
mtu "${ns_b}" ${type}_b $((${ll_mtu} + 1000))
run_cmd ${ns_c} ${ping} -q -M want -i 0.1 -c 10 -s $((${ll_mtu} + 500)) ${dst} || return 1
- run_cmd ${ns_a} ${ping} -q -M want -i 0.1 -w 1 -s $((${ll_mtu} + 500)) ${dst} || return 1
+ run_cmd ${ns_a} ${ping} -q -M want -i 0.1 -c 10 -s $((${ll_mtu} + 500)) ${dst} || return 1
# Check that exceptions were created
pmtu="$(route_get_dst_pmtu_from_exception "${ns_c}" ${dst})"
diff --git a/tools/testing/selftests/net/test_vxlan_mdb.sh b/tools/testing/selftests/net/test_vxlan_mdb.sh
index 58da5de99ac4..f9600aabd4a2 100755
--- a/tools/testing/selftests/net/test_vxlan_mdb.sh
+++ b/tools/testing/selftests/net/test_vxlan_mdb.sh
@@ -685,6 +685,9 @@ star_g_common()
run_cmd "bridge -n $ns1 mdb add dev vx0 port vx0 grp $grp permanent filter_mode exclude source_list $grp dst $vtep_ip src_vni 10010"
log_test $? 255 "Invalid source in source list"
+ run_cmd "bridge -n $ns1 mdb add dev vx0 port vx0 grp $grp permanent filter_mode exclude source_list $all_zeros_grp dst $vtep_ip src_vni 10010"
+ log_test $? 255 "All-zeros source in source list"
+
run_cmd "bridge -n $ns1 mdb add dev vx0 port vx0 grp $grp permanent source_list $src1 dst $vtep_ip src_vni 10010"
log_test $? 255 "Source list without filter mode"
}
@@ -784,6 +787,9 @@ sg_common()
run_cmd "bridge -n $ns1 mdb add dev vx0 port vx0 grp $grp src $grp permanent dst $vtep_ip src_vni 10010"
log_test $? 255 "(S, G) with an invalid source list"
+ run_cmd "bridge -n $ns1 mdb add dev vx0 port vx0 grp $grp src $all_zeros_grp permanent dst $vtep_ip src_vni 10010"
+ log_test $? 255 "(S, G) with an all-zeros source"
+
run_cmd "bridge -n $ns1 mdb add dev vx0 port vx0 grp $all_zeros_grp src $src permanent dst $vtep_ip src_vni 10010"
log_test $? 255 "All-zeros group with source"
}
diff --git a/tools/testing/selftests/powerpc/tm/tm.h b/tools/testing/selftests/powerpc/tm/tm.h
index c03c6e778876..6024ce4ba6ff 100644
--- a/tools/testing/selftests/powerpc/tm/tm.h
+++ b/tools/testing/selftests/powerpc/tm/tm.h
@@ -105,8 +105,12 @@ static inline bool failure_is_nesting(void)
static inline int tcheck(void)
{
long cr;
- asm volatile ("tcheck 0" : "=r"(cr) : : "cr0");
- return (cr >> 28) & 4;
+ asm volatile("tcheck 0;"
+ "mfcr %0;"
+ : "=r"(cr)
+ :
+ : "cr0");
+ return (cr >> 28) & 0xf;
}
static inline bool tcheck_doomed(void)
diff --git a/tools/testing/selftests/ublk/Makefile b/tools/testing/selftests/ublk/Makefile
index 6e4fe8d1fed1..6dbae1391a06 100644
--- a/tools/testing/selftests/ublk/Makefile
+++ b/tools/testing/selftests/ublk/Makefile
@@ -69,6 +69,8 @@ TEST_PROGS += test_stress_08.sh
TEST_PROGS += test_stress_09.sh
TEST_FILES := settings
+TEST_FILES += test_common.sh
+TEST_FILES += trace
TEST_GEN_PROGS_EXTENDED = kublk metadata_size
STANDALONE_UTILS := metadata_size.c
^ permalink raw reply [flat|nested] 2+ messages in thread