* Re: Linux 6.18.55
2026-10-03 10:54 Linux 6.18.55 Greg Kroah-Hartman
@ 2026-10-03 10:54 ` Greg Kroah-Hartman
0 siblings, 0 replies; 2+ messages in thread
From: Greg Kroah-Hartman @ 2026-10-03 10:54 UTC (permalink / raw)
To: linux-kernel, akpm, torvalds, stable; +Cc: lwn, jslaby, Greg Kroah-Hartman
diff --git a/Documentation/arch/arm64/booting.rst b/Documentation/arch/arm64/booting.rst
index e4f953839f71..ace9d8a490eb 100644
--- a/Documentation/arch/arm64/booting.rst
+++ b/Documentation/arch/arm64/booting.rst
@@ -465,6 +465,7 @@ Before jumping into the kernel, the following conditions must be met:
- HDFGWTR2_EL2.nPMICNTR_EL0 (bit 2) must be initialised to 0b1.
- HDFGWTR2_EL2.nPMICFILTR_EL0 (bit 3) must be initialised to 0b1.
- HDFGWTR2_EL2.nPMUACR_EL1 (bit 4) must be initialised to 0b1.
+ - HDFGWTR2_EL2.nPMZR_EL0 (bit 21) must be initialised to 0b1.
For CPUs with SPE data source filtering (FEAT_SPE_FDS):
diff --git a/Documentation/arch/s390/pci.rst b/Documentation/arch/s390/pci.rst
index d5755484d8e7..8dc09634163b 100644
--- a/Documentation/arch/s390/pci.rst
+++ b/Documentation/arch/s390/pci.rst
@@ -6,6 +6,7 @@ S/390 PCI
Authors:
- Pierre Morel
+ - Niklas Schnelle
Copyright, IBM Corp. 2020
@@ -27,14 +28,16 @@ Command line parameters
debugfs entries
---------------
-The S/390 debug feature (s390dbf) generates views to hold various debug results in sysfs directories of the form:
+The S/390 debug feature (s390dbf) generates views to hold various debug results
+in sysfs directories of the form:
* /sys/kernel/debug/s390dbf/pci_*/
For example:
- /sys/kernel/debug/s390dbf/pci_msg/sprintf
- Holds messages from the processing of PCI events, like machine check handling
+
+ holds messages from the processing of PCI events, like machine check handling
and setting of global functionality, like UID checking.
Change the level of logging to be more or less verbose by piping
@@ -47,87 +50,134 @@ Sysfs entries
Entries specific to zPCI functions and entries that hold zPCI information.
-* /sys/bus/pci/slots/XXXXXXXX
+* /sys/bus/pci/slots/XXXXXXXX:
- The slot entries are set up using the function identifier (FID) of the
- PCI function. The format depicted as XXXXXXXX above is 8 hexadecimal digits
- with 0 padding and lower case hexadecimal digits.
+ The slot entries are set up using the function identifier (FID) of the PCI
+ function as slot name. The format depicted as XXXXXXXX above is 8 hexadecimal
+ digits with 0 padding and lower case hexadecimal digits.
- /sys/bus/pci/slots/XXXXXXXX/power
A physical function that currently supports a virtual function cannot be
powered off until all virtual functions are removed with:
- echo 0 > /sys/bus/pci/devices/XXXX:XX:XX.X/sriov_numvf
+ echo 0 > /sys/bus/pci/devices/DDDD:BB:dd.f/sriov_numvfs
-* /sys/bus/pci/devices/XXXX:XX:XX.X/
+* /sys/bus/pci/devices/DDDD:BB:dd.f/:
- - function_id
- A zPCI function identifier that uniquely identifies the function in the Z server.
+ - function_id:
+ The zPCI function identifier (FID) is a 32-bit hexadecimal value that
+ uniquely identifies the PCI function. Unless the hypervisor provides
+ a virtual FID e.g. on KVM this identifier is unique across the machine even
+ between different partitions.
- - function_handle
- Low-level identifier used for a configured PCI function.
- It might be useful for debugging.
+ - function_handle:
+ This 32-bit hexadecimal value is a low-level identifier used for a PCI
+ function. Note that the function handle may be changed and become invalid
+ on PCI events and when enabling/disabling the PCI function.
- - pchid
- Model-dependent location of the I/O adapter.
+ - pchid:
+ This 16-bit hexadecimal value encodes a model-dependent location for
+ the PCI function.
- - pfgid
- PCI function group ID, functions that share identical functionality
+ - pfgid:
+ PCI function group ID; functions that share identical functionality
use a common identifier.
A PCI group defines interrupts, IOMMU, IOTLB, and DMA specifics.
- - vfn
+ - vfn:
The virtual function number, from 1 to N for virtual functions,
0 for physical functions.
- - pft
- The PCI function type
-
- - port
- The port corresponds to the physical port the function is attached to.
- It also gives an indication of the physical function a virtual function
- is attached to.
-
- - uid
- The user identifier (UID) may be defined as part of the machine
- configuration or the z/VM or KVM guest configuration. If the accompanying
- uid_is_unique attribute is 1 the platform guarantees that the UID is unique
- within that instance and no devices with the same UID can be attached
- during the lifetime of the system.
-
- - uid_is_unique
- Indicates whether the user identifier (UID) is guaranteed to be and remain
- unique within this Linux instance.
-
- - pfip/segmentX
+ - pft:
+ The PCI function type is an s390-specific type attribute. It indicates
+ a more general, usage oriented, type than PCI Specification
+ class/vendor/device identifiers. That is PCI functions with the same pft
+ value may be backed by different hardware implementations. At the same time
+ apart from unclassified functions (pft is 0x00) the same pft value
+ generally implies a similar usage model. At the same time the same
+ PCI hardware device may appear with different pft values when in a
+ different usage model. For example NETD and NETH VFs may be implemented
+ by the same PCI hardware device but in NETD the parent Physical Function
+ is user managed while with NETH it is platform managed.
+
+ Currently the following PFT values are defined:
+
+ - 0x00 (UNC): Unclassified
+ - 0x02 (ROCE): RoCE Express
+ - 0x05 (ISM): Internal Shared Memory
+ - 0x0a (ROC2): RoCE Express 2
+ - 0x0b (NVMe): NVMe
+ - 0x0c (NETH): Network Express hybrid
+ - 0x0d (CNW): Cloud Network Adapter
+ - 0x0f (NETD): Network Express direct
+
+ - port:
+ The port is a decimal value corresponding to the physical port the function
+ is attached to. Virtual Functions (VFs) share the port with their parent
+ Physical Function (PF). A value of 0 indicates that the port attribute is
+ not applicable for that PCI function type.
+
+ - uid:
+ The user-defined identifier (UID) for a PCI function is a 32-bit
+ hexadecimal value. It is defined on a per instance basis as part of the
+ partition, KVM guest, or z/VM guest configuration. If UID Checking is
+ enabled the platform ensures that the UID is unique within that instance
+ and no two PCI functions with the same UID will be visible to the instance.
+
+ Independent of this guarantee and unlike the function ID (FID) the UID may
+ be the same in different partitions within the same machine. This allows to
+ create PCI configurations in multiple partitions to be identical in the
+ UID-namespace.
+
+ - uid_is_unique:
+ A 0 or 1 flag indicating whether the user-defined identifier (UID) is
+ guaranteed to be and remain unique within this Linux instance. This
+ platform feature is called UID Checking.
+
+ - pfip/segmentX:
The segments determine the isolation of a function.
They correspond to the physical path to the function.
The more the segments are different, the more the functions are isolated.
+ - fidparm:
+ Contains an 8-bit-per-PCI function parameter field in hexadecimal provided
+ by the platform. The meaning of this field is PCI function type specific.
+ For NETH VFs a value of 0x01 indicates that the function supports
+ promiscuous mode.
+
+* /sys/firmware/clp/uid_checking:
+
+ In addition to the per-device uid_is_unique attribute this presents a
+ global indication of whether UID Checking is enabled. This allows users
+ to check for UID Checking even when no PCI functions are configured.
+
Enumeration and hotplug
=======================
The PCI address consists of four parts: domain, bus, device and function,
-and is of this form: DDDD:BB:dd.f
+and is of this form: DDDD:BB:dd.f.
-* When not using multi-functions (norid is set, or the firmware does not
- support multi-functions):
+* For a PCI function for which the platform does not expose the RID, the
+ pci=norid kernel parameter is used, or a so-called isolated Virtual Function
+ which does have RID information but is used without its parent Physical
+ Function being part of the same PCI configuration:
- There is only one function per domain.
- - The domain is set from the zPCI function's UID as defined during the
- LPAR creation.
+ - The domain is set from the zPCI function's UID if UID Checking is on;
+ otherwise the domain ID is generated dynamically and is not stable
+ across reboots or hot plug.
-* When using multi-functions (norid parameter is not set),
- zPCI functions are addressed differently:
+* For a PCI function for which the platform exposes the RID and which
+ is not an Isolated Virtual Function:
- There is still only one bus per domain.
- - There can be up to 256 functions per bus.
+ - There can be up to 256 PCI functions per bus.
- - The domain part of the address of all functions for
- a multi-Function device is set from the zPCI function's UID as defined
- in the LPAR creation for the function zero.
+ - The domain part of the address of all functions within the same topology is
+ that of the configured PCI function with the lowest devfn within that
+ topology.
- - New functions will only be ready for use after the function zero
- (the function with devfn 0) has been enumerated.
+ - Virtual Functions generated by an SR-IOV capable Physical Function only
+ become visible once SR-IOV is enabled.
diff --git a/Makefile b/Makefile
index 56bb2168e656..84df9673f9b6 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
VERSION = 6
PATCHLEVEL = 18
-SUBLEVEL = 54
+SUBLEVEL = 55
EXTRAVERSION =
NAME = Baby Opossum Posse
diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h
index 99a7c0235e6d..48f686e4bba2 100644
--- a/arch/arm64/include/asm/el2_setup.h
+++ b/arch/arm64/include/asm/el2_setup.h
@@ -409,6 +409,7 @@
b.lt .Lskip_fgt2_\@
mov x0, xzr
+ mov x2, xzr
mrs x1, id_aa64dfr0_el1
ubfx x1, x1, #ID_AA64DFR0_EL1_PMUVer_SHIFT, #4
cmp x1, #ID_AA64DFR0_EL1_PMUVer_V3P9
@@ -417,6 +418,11 @@
orr x0, x0, #HDFGRTR2_EL2_nPMICNTR_EL0
orr x0, x0, #HDFGRTR2_EL2_nPMICFILTR_EL0
orr x0, x0, #HDFGRTR2_EL2_nPMUACR_EL1
+ orr x2, x2, #HDFGWTR2_EL2_nPMICNTR_EL0
+ orr x2, x2, #HDFGWTR2_EL2_nPMICFILTR_EL0
+ orr x2, x2, #HDFGWTR2_EL2_nPMUACR_EL1
+ /* PMZR_EL0 is write-only, so it has no read trap to disable */
+ orr x2, x2, #HDFGWTR2_EL2_nPMZR_EL0
.Lskip_pmuv3p9_\@:
/* If SPE is implemented, */
__spe_vers_imp .Lskip_spefds_\@, ID_AA64DFR0_EL1_PMSVer_IMP, x1
@@ -427,10 +433,11 @@
cbz x1, .Lskip_spefds_\@
/* disable traps of PMSDSFR to EL2. */
orr x0, x0, #HDFGRTR2_EL2_nPMSDSFR_EL1
+ orr x2, x2, #HDFGWTR2_EL2_nPMSDSFR_EL1
.Lskip_spefds_\@:
msr_s SYS_HDFGRTR2_EL2, x0
- msr_s SYS_HDFGWTR2_EL2, x0
+ msr_s SYS_HDFGWTR2_EL2, x2
msr_s SYS_HFGRTR2_EL2, xzr
msr_s SYS_HFGWTR2_EL2, xzr
msr_s SYS_HFGITR2_EL2, xzr
diff --git a/arch/arm64/include/asm/io.h b/arch/arm64/include/asm/io.h
index 8cbd1e96fd50..a8507de4f84c 100644
--- a/arch/arm64/include/asm/io.h
+++ b/arch/arm64/include/asm/io.h
@@ -272,7 +272,8 @@ static inline void __iomem *ioremap_prot(phys_addr_t phys, size_t size,
pgprot_t prot;
ptdesc_t user_prot_val = pgprot_val(user_prot);
- if (WARN_ON_ONCE(!(user_prot_val & PTE_USER)))
+ /* Reject PROT_NONE and exec-only */
+ if (!(user_prot_val & PTE_USER))
return NULL;
prot = __pgprot_modify(PAGE_KERNEL, PTE_ATTRINDX_MASK,
diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h
index 08be89c95466..7195be508d99 100644
--- a/arch/arm64/include/asm/kvm_pkvm.h
+++ b/arch/arm64/include/asm/kvm_pkvm.h
@@ -23,10 +23,12 @@ void pkvm_destroy_hyp_vm(struct kvm *kvm);
int pkvm_create_hyp_vcpu(struct kvm_vcpu *vcpu);
/*
- * This functions as an allow-list of protected VM capabilities.
- * Features not explicitly allowed by this function are denied.
+ * Check whether the specific capability is allowed in pKVM.
+ *
+ * Certain features are allowed only for non-protected VMs in pKVM, which is why
+ * this takes the VM (kvm) as a parameter.
*/
-static inline bool kvm_pvm_ext_allowed(long ext)
+static inline bool kvm_pkvm_ext_allowed(struct kvm *kvm, long ext)
{
switch (ext) {
case KVM_CAP_IRQCHIP:
@@ -43,7 +45,7 @@ static inline bool kvm_pvm_ext_allowed(long ext)
case KVM_CAP_ARM_PTRAUTH_GENERIC:
return true;
default:
- return false;
+ return !kvm || !kvm_vm_is_protected(kvm);
}
}
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index c00ec6c904c7..1a7425375d0e 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -28,18 +28,21 @@ bool cpu_errata_set_target_impl(u64 num, void *impl_cpus)
return true;
}
+static inline bool __is_midr_in_range(u32 midr, struct midr_range const *range)
+{
+ return midr_is_cpu_model_range(midr, range->model,
+ range->rv_min, range->rv_max);
+}
+
static inline bool is_midr_in_range(struct midr_range const *range)
{
int i;
if (!target_impl_cpu_num)
- return midr_is_cpu_model_range(read_cpuid_id(), range->model,
- range->rv_min, range->rv_max);
+ return __is_midr_in_range(read_cpuid_id(), range);
for (i = 0; i < target_impl_cpu_num; i++) {
- if (midr_is_cpu_model_range(target_impl_cpus[i].midr,
- range->model,
- range->rv_min, range->rv_max))
+ if (__is_midr_in_range(target_impl_cpus[i].midr, range))
return true;
}
return false;
@@ -59,7 +62,7 @@ __is_affected_midr_range(const struct arm64_cpu_capabilities *entry,
u32 midr, u32 revidr)
{
const struct arm64_midr_revidr *fix;
- if (!is_midr_in_range(&entry->midr_range))
+ if (!__is_midr_in_range(midr, &entry->midr_range))
return false;
midr &= MIDR_REVISION_MASK | MIDR_VARIANT_MASK;
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 5d2435dd76f3..52549bebb782 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -87,7 +87,7 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
if (cap->flags)
return -EINVAL;
- if (kvm_vm_is_protected(kvm) && !kvm_pvm_ext_allowed(cap->cap))
+ if (is_protected_kvm_enabled() && !kvm_pkvm_ext_allowed(kvm, cap->cap))
return -EINVAL;
switch (cap->cap) {
@@ -312,7 +312,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
{
int r;
- if (kvm && kvm_vm_is_protected(kvm) && !kvm_pvm_ext_allowed(ext))
+ if (is_protected_kvm_enabled() && !kvm_pkvm_ext_allowed(kvm, ext))
return 0;
switch (ext) {
diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c
index 877818840704..fbb37edfbaba 100644
--- a/arch/arm64/kvm/emulate-nested.c
+++ b/arch/arm64/kvm/emulate-nested.c
@@ -1423,7 +1423,7 @@ static const struct encoding_to_trap_config encoding_to_fgt[] __initconst = {
SR_FGT(OP_AT_S1E1A, HFGITR, ATS1E1A, 1),
SR_FGT(OP_COSP_RCTX, HFGITR, COSPRCTX, 1),
SR_FGT(OP_GCSPUSHX, HFGITR, nGCSEPP, 0),
- SR_FGT(OP_GCSPOPX, HFGITR, nGCSEPP, 0),
+ SR_FGT(OP_GCSPOPCX, HFGITR, nGCSEPP, 0),
SR_FGT(OP_GCSPUSHM, HFGITR, nGCSPUSHM_EL1, 0),
SR_FGT(OP_BRB_IALL, HFGITR, nBRBIALL, 0),
SR_FGT(OP_BRB_INJ, HFGITR, nBRBINJ, 0),
diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c
index cafd09565f83..6eade6babbbd 100644
--- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
+++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
@@ -338,8 +338,8 @@ static void pkvm_init_features_from_host(struct pkvm_hyp_vm *hyp_vm, const struc
/* CTR_EL0 is always under host control, even for protected VMs. */
hyp_vm->kvm.arch.ctr_el0 = host_kvm->arch.ctr_el0;
- if (test_bit(KVM_ARCH_FLAG_MTE_ENABLED, &host_kvm->arch.flags))
- set_bit(KVM_ARCH_FLAG_MTE_ENABLED, &kvm->arch.flags);
+ /* Preserve the vgic model so that GICv3 emulation works */
+ hyp_vm->kvm.arch.vgic.vgic_model = host_kvm->arch.vgic.vgic_model;
/* No restrictions for non-protected VMs. */
if (!kvm_vm_is_protected(kvm)) {
@@ -353,29 +353,33 @@ static void pkvm_init_features_from_host(struct pkvm_hyp_vm *hyp_vm, const struc
if (test_bit(KVM_ARCH_FLAG_WRITABLE_IMP_ID_REGS, &host_arch_flags))
hyp_vm->kvm.arch.midr_el1 = host_kvm->arch.midr_el1;
- return;
+ goto out;
}
+ if (kvm_pkvm_ext_allowed(kvm, KVM_CAP_ARM_MTE))
+ kvm->arch.flags |= host_arch_flags & BIT(KVM_ARCH_FLAG_MTE_ENABLED);
+
bitmap_zero(allowed_features, KVM_VCPU_MAX_FEATURES);
set_bit(KVM_ARM_VCPU_PSCI_0_2, allowed_features);
- if (kvm_pvm_ext_allowed(KVM_CAP_ARM_PMU_V3))
+ if (kvm_pkvm_ext_allowed(kvm, KVM_CAP_ARM_PMU_V3))
set_bit(KVM_ARM_VCPU_PMU_V3, allowed_features);
- if (kvm_pvm_ext_allowed(KVM_CAP_ARM_PTRAUTH_ADDRESS))
+ if (kvm_pkvm_ext_allowed(kvm, KVM_CAP_ARM_PTRAUTH_ADDRESS))
set_bit(KVM_ARM_VCPU_PTRAUTH_ADDRESS, allowed_features);
- if (kvm_pvm_ext_allowed(KVM_CAP_ARM_PTRAUTH_GENERIC))
+ if (kvm_pkvm_ext_allowed(kvm, KVM_CAP_ARM_PTRAUTH_GENERIC))
set_bit(KVM_ARM_VCPU_PTRAUTH_GENERIC, allowed_features);
- if (kvm_pvm_ext_allowed(KVM_CAP_ARM_SVE)) {
+ if (kvm_pkvm_ext_allowed(kvm, KVM_CAP_ARM_SVE))
set_bit(KVM_ARM_VCPU_SVE, allowed_features);
- kvm->arch.flags |= host_arch_flags & BIT(KVM_ARCH_FLAG_GUEST_HAS_SVE);
- }
bitmap_and(kvm->arch.vcpu_features, host_kvm->arch.vcpu_features,
allowed_features, KVM_VCPU_MAX_FEATURES);
+out:
+ __assign_bit(KVM_ARCH_FLAG_GUEST_HAS_SVE, &kvm->arch.flags,
+ kvm_vcpu_has_feature(kvm, KVM_ARM_VCPU_SVE));
}
static void unpin_host_vcpu(struct kvm_vcpu *host_vcpu)
@@ -441,7 +445,7 @@ static int pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *hyp_vcpu, struct kvm_vcpu *h
unsigned int sve_max_vl;
size_t sve_state_size;
void *sve_state;
- int ret = 0;
+ int ret;
if (!vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) {
vcpu_clear_flag(vcpu, VCPU_SVE_FINALIZED);
@@ -450,25 +454,21 @@ static int pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *hyp_vcpu, struct kvm_vcpu *h
/* Limit guest vector length to the maximum supported by the host. */
sve_max_vl = min(READ_ONCE(host_vcpu->arch.sve_max_vl), kvm_host_sve_max_vl);
- sve_state_size = sve_state_size_from_vl(sve_max_vl);
sve_state = kern_hyp_va(READ_ONCE(host_vcpu->arch.sve_state));
- if (!sve_state || !sve_state_size) {
- ret = -EINVAL;
- goto err;
- }
+ if (!sve_vl_valid(sve_max_vl) || !sve_state)
+ return -EINVAL;
+
+ sve_state_size = sve_state_size_from_vl(sve_max_vl);
ret = hyp_pin_shared_mem(sve_state, sve_state + sve_state_size);
if (ret)
- goto err;
+ return ret;
vcpu->arch.sve_state = sve_state;
vcpu->arch.sve_max_vl = sve_max_vl;
return 0;
-err:
- clear_bit(KVM_ARM_VCPU_SVE, vcpu->kvm->arch.vcpu_features);
- return ret;
}
static int vm_copy_id_regs(struct pkvm_hyp_vcpu *hyp_vcpu)
diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c
index 97643fc02d92..15b25086447a 100644
--- a/arch/arm64/kvm/hyp/nvhe/setup.c
+++ b/arch/arm64/kvm/hyp/nvhe/setup.c
@@ -269,6 +269,16 @@ static int fix_host_ownership(void)
return ret;
}
+ /* The stacks sit in the private VA range, not the linear map. */
+ for (i = 0; i < hyp_nr_cpus; i++) {
+ struct kvm_nvhe_init_params *params = per_cpu_ptr(&kvm_init_params, i);
+ u64 start = params->stack_hyp_va - NVHE_STACK_SIZE;
+
+ ret = kvm_pgtable_walk(&pkvm_pgtable, start, NVHE_STACK_SIZE, &walker);
+ if (ret)
+ return ret;
+ }
+
return 0;
}
diff --git a/arch/arm64/kvm/hyp/nvhe/sys_regs.c b/arch/arm64/kvm/hyp/nvhe/sys_regs.c
index 82da9b03692d..3108b5185c20 100644
--- a/arch/arm64/kvm/hyp/nvhe/sys_regs.c
+++ b/arch/arm64/kvm/hyp/nvhe/sys_regs.c
@@ -444,6 +444,8 @@ static const struct sys_reg_desc pvm_sys_reg_descs[] = {
/* Scalable Vector Registers are restricted. */
+ HOST_HANDLED(SYS_ICC_PMR_EL1),
+
RAZ_WI(SYS_ERRIDR_EL1),
RAZ_WI(SYS_ERRSELR_EL1),
RAZ_WI(SYS_ERXFR_EL1),
@@ -457,9 +459,12 @@ static const struct sys_reg_desc pvm_sys_reg_descs[] = {
/* Limited Ordering Regions Registers are restricted. */
+ HOST_HANDLED(SYS_ICC_DIR_EL1),
+ HOST_HANDLED(SYS_ICC_RPR_EL1),
HOST_HANDLED(SYS_ICC_SGI1R_EL1),
HOST_HANDLED(SYS_ICC_ASGI1R_EL1),
HOST_HANDLED(SYS_ICC_SGI0R_EL1),
+ HOST_HANDLED(SYS_ICC_CTLR_EL1),
{ SYS_DESC(SYS_ICC_SRE_EL1), .access = pvm_gic_read_sre, },
HOST_HANDLED(SYS_CCSIDR_EL1),
diff --git a/arch/arm64/kvm/hypercalls.c b/arch/arm64/kvm/hypercalls.c
index 58c5fe7d7572..ccbfe847f78b 100644
--- a/arch/arm64/kvm/hypercalls.c
+++ b/arch/arm64/kvm/hypercalls.c
@@ -185,7 +185,8 @@ static int kvm_smccc_set_filter(struct kvm *kvm, struct kvm_smccc_filter __user
start = filter.base;
end = start + filter.nr_functions - 1;
- if (end < start || filter.action >= NR_SMCCC_FILTER_ACTIONS)
+ if (!filter.nr_functions || end < start ||
+ filter.action >= NR_SMCCC_FILTER_ACTIONS)
return -EINVAL;
mutex_lock(&kvm->arch.config_lock);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index b4237699705b..2dae3fcb7a7f 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -59,27 +59,36 @@ static phys_addr_t stage2_range_addr_end(phys_addr_t addr, phys_addr_t end)
* long will also starve other vCPUs. We have to also make sure that the page
* tables are not freed while we released the lock.
*/
-static int stage2_apply_range(struct kvm_s2_mmu *mmu, phys_addr_t addr,
+static int stage2_apply_range(struct kvm_s2_mmu *mmu, phys_addr_t start,
phys_addr_t end,
int (*fn)(struct kvm_pgtable *, u64, u64),
bool resched)
{
struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
+ bool lock_dropped = false;
+ phys_addr_t addr = start;
int ret;
u64 next;
do {
struct kvm_pgtable *pgt = mmu->pgt;
+ /*
+ * We may be raced on PGT teardown when we release the
+ * kvm->mmu_lock. That's fine as the PGT is legitimately no
+ * longer present.
+ */
if (!pgt)
- return -EINVAL;
+ return lock_dropped ? 0 : -EINVAL;
next = stage2_range_addr_end(addr, end);
ret = fn(pgt, addr, next - addr);
if (ret)
break;
- if (resched && next != end)
+ if (resched && next != end) {
cond_resched_rwlock_write(&kvm->mmu_lock);
+ lock_dropped = true;
+ }
} while (addr = next, addr != end);
return ret;
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index 869209e8bd1c..9be5346140ce 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -1129,6 +1129,17 @@ void kvm_handle_s1e2_tlbi(struct kvm_vcpu *vcpu, u32 inst, u64 val)
invalidate_vncr_va(vcpu->kvm, &scope);
}
+static void kvm_invalidate_vncr_ipa_all(struct kvm *kvm)
+{
+ struct kvm_pgtable *pgt = kvm->arch.mmu.pgt;
+
+ lockdep_assert_held_write(&kvm->mmu_lock);
+
+ /* if the mmu lock was dropped, pgt teardown may have raced. */
+ if (pgt)
+ kvm_invalidate_vncr_ipa(kvm, 0, BIT(pgt->ia_bits));
+}
+
void kvm_nested_s2_wp(struct kvm *kvm)
{
int i;
@@ -1142,7 +1153,7 @@ void kvm_nested_s2_wp(struct kvm *kvm)
kvm_stage2_wp_range(mmu, 0, kvm_phys_size(mmu));
}
- kvm_invalidate_vncr_ipa(kvm, 0, BIT(kvm->arch.mmu.pgt->ia_bits));
+ kvm_invalidate_vncr_ipa_all(kvm);
}
void kvm_nested_s2_unmap(struct kvm *kvm, bool may_block)
@@ -1158,7 +1169,7 @@ void kvm_nested_s2_unmap(struct kvm *kvm, bool may_block)
kvm_stage2_unmap_range(mmu, 0, kvm_phys_size(mmu), may_block);
}
- kvm_invalidate_vncr_ipa(kvm, 0, BIT(kvm->arch.mmu.pgt->ia_bits));
+ kvm_invalidate_vncr_ipa_all(kvm);
}
void kvm_nested_s2_flush(struct kvm *kvm)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 6cb126585e71..308ba3dba809 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -792,6 +792,7 @@ static u64 *demux_wb_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
struct kvm_guest_debug_arch *dbg = &vcpu->arch.vcpu_debug_state;
switch (rd->Op2) {
+ case 0b001:
case 0b100:
return &dbg->dbg_bvr[rd->CRm];
case 0b101:
diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c
index cdaf2af5e765..763fa5dff167 100644
--- a/arch/arm64/kvm/vgic/vgic-its.c
+++ b/arch/arm64/kvm/vgic/vgic-its.c
@@ -1648,7 +1648,7 @@ static void vgic_mmio_write_its_baser(struct kvm *kvm,
unsigned long val)
{
const struct vgic_its_abi *abi = vgic_its_get_abi(its);
- u64 entry_size, table_type;
+ u64 old, entry_size, table_type;
u64 reg, *regptr, clearbits = 0;
/* When GITS_CTLR.Enable is 1, we ignore write accesses. */
@@ -1671,7 +1671,9 @@ static void vgic_mmio_write_its_baser(struct kvm *kvm,
return;
}
- reg = update_64bit_reg(*regptr, addr & 7, len, val);
+ old = *regptr;
+
+ reg = update_64bit_reg(old, addr & 7, len, val);
reg &= ~GITS_BASER_RO_MASK;
reg &= ~clearbits;
@@ -1681,7 +1683,8 @@ static void vgic_mmio_write_its_baser(struct kvm *kvm,
*regptr = reg;
- if (!(reg & GITS_BASER_VALID)) {
+ /* The ITS driver rewrites an unchanged GITS_BASER<n> on resume. */
+ if (reg != old) {
/* Take the its_lock to prevent a race with a save/restore */
mutex_lock(&its->its_lock);
switch (table_type) {
@@ -1692,6 +1695,8 @@ static void vgic_mmio_write_its_baser(struct kvm *kvm,
vgic_its_free_collection_list(kvm, its);
break;
}
+ /* A concurrent injection may have cached a translation. */
+ vgic_its_invalidate_cache(its);
mutex_unlock(&its->its_lock);
}
}
@@ -2012,18 +2017,22 @@ static int vgic_its_attr_regs_access(struct kvm_device *dev,
return ret;
}
-static u32 compute_next_devid_offset(struct list_head *h,
+static u32 compute_next_devid_offset(struct vgic_its *its, u64 baser,
struct its_device *dev)
{
- struct its_device *next;
- u32 next_offset;
+ struct its_device *next = dev;
- if (list_is_last(&dev->dev_list, h))
- return 0;
- next = list_next_entry(dev, dev_list);
- next_offset = next->device_id - dev->device_id;
+ /*
+ * Point at the next device vgic_its_save_device_tables() saves. It
+ * sorts device_list first, so the subtraction cannot underflow.
+ */
+ list_for_each_entry_continue(next, &its->device_list, dev_list) {
+ if (vgic_its_check_id(its, baser, next->device_id, NULL))
+ return min_t(u32, next->device_id - dev->device_id,
+ VITS_DTE_MAX_DEVID_OFFSET);
+ }
- return min_t(u32, next_offset, VITS_DTE_MAX_DEVID_OFFSET);
+ return 0;
}
static u32 compute_next_eventid_offset(struct list_head *h, struct its_ite *ite)
@@ -2263,17 +2272,18 @@ static int vgic_its_restore_itt(struct vgic_its *its, struct its_device *dev)
* vgic_its_save_dte - Save a device table entry at a given GPA
*
* @its: ITS handle
+ * @baser: GITS_BASER<dev> the caller is saving against
* @dev: ITS device
* @ptr: GPA
*/
-static int vgic_its_save_dte(struct vgic_its *its, struct its_device *dev,
- gpa_t ptr)
+static int vgic_its_save_dte(struct vgic_its *its, u64 baser,
+ struct its_device *dev, gpa_t ptr)
{
u64 val, itt_addr_field;
u32 next_offset;
itt_addr_field = dev->itt_addr >> 8;
- next_offset = compute_next_devid_offset(&its->device_list, dev);
+ next_offset = compute_next_devid_offset(its, baser, dev);
val = (1ULL << KVM_ITS_DTE_VALID_SHIFT |
((u64)next_offset << KVM_ITS_DTE_NEXT_SHIFT) |
(itt_addr_field << KVM_ITS_DTE_ITTADDR_SHIFT) |
@@ -2372,15 +2382,16 @@ static int vgic_its_save_device_tables(struct vgic_its *its)
int ret;
gpa_t eaddr;
+ /* Don't fail a save that userspace must be able to issue. */
if (!vgic_its_check_id(its, baser,
dev->device_id, &eaddr))
- return -EINVAL;
+ continue;
ret = vgic_its_save_itt(its, dev);
if (ret)
return ret;
- ret = vgic_its_save_dte(its, dev, eaddr);
+ ret = vgic_its_save_dte(its, baser, dev, eaddr);
if (ret)
return ret;
}
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 7be20cb32a27..10b9079c1212 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -592,6 +592,8 @@ static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf)
* 12 registers are on the stack
*/
emit(A64_SUB_I(1, A64_SP, A64_FP, 96), ctx);
+ /* The callback may use its own BPF stack, set up fp for it. */
+ ctx->fp_used = true;
}
/* Stack must be multiples of 16B */
diff --git a/arch/mips/net/bpf_jit_comp32.c b/arch/mips/net/bpf_jit_comp32.c
index 40a878b672f5..15a2a153dc87 100644
--- a/arch/mips/net/bpf_jit_comp32.c
+++ b/arch/mips/net/bpf_jit_comp32.c
@@ -1111,7 +1111,7 @@ static void emit_jmp_i64(struct jit_context *ctx,
emit(ctx, xor, tmp, lo(dst), tmp);
}
if (imm < 0) { /* Compare sign extension */
- emit(ctx, addu, MIPS_R_T9, hi(dst), 1);
+ emit(ctx, addiu, MIPS_R_T9, hi(dst), 1);
emit(ctx, or, tmp, tmp, MIPS_R_T9);
} else { /* Compare zero extension */
emit(ctx, or, tmp, tmp, hi(dst));
diff --git a/arch/mips/net/bpf_jit_comp64.c b/arch/mips/net/bpf_jit_comp64.c
index fa7e9aa37f49..6681ccac9dd9 100644
--- a/arch/mips/net/bpf_jit_comp64.c
+++ b/arch/mips/net/bpf_jit_comp64.c
@@ -305,8 +305,7 @@ static void emit_bswap_r64(struct jit_context *ctx, u8 dst, u32 width)
case 16:
emit_sext(ctx, dst, dst);
emit_bswap_r(ctx, dst, width);
- if (cpu_has_mips64r2 || cpu_has_mips64r6)
- emit_zext(ctx, dst);
+ emit_zext(ctx, dst);
break;
}
clobber_reg(ctx, dst);
diff --git a/arch/parisc/include/asm/thread_info.h b/arch/parisc/include/asm/thread_info.h
index b283738bb6da..249370706242 100644
--- a/arch/parisc/include/asm/thread_info.h
+++ b/arch/parisc/include/asm/thread_info.h
@@ -24,7 +24,7 @@ struct thread_info {
/* thread information allocation */
-#ifdef CONFIG_IRQSTACKS
+#if defined(CONFIG_IRQSTACKS) && !defined(CONFIG_64BIT)
#define THREAD_SIZE_ORDER 2 /* PA-RISC requires at least 16k stack */
#else
#define THREAD_SIZE_ORDER 3 /* PA-RISC requires at least 32k stack */
diff --git a/arch/riscv/kvm/aia_imsic.c b/arch/riscv/kvm/aia_imsic.c
index 11422cb95a64..a357ea73516f 100644
--- a/arch/riscv/kvm/aia_imsic.c
+++ b/arch/riscv/kvm/aia_imsic.c
@@ -952,8 +952,15 @@ int kvm_riscv_aia_imsic_rw_attr(struct kvm *kvm, unsigned long type,
if (!vcpu)
return -ENODEV;
- isel = KVM_DEV_RISCV_AIA_IMSIC_GET_ISEL(type);
+ if (mutex_lock_killable(&vcpu->mutex))
+ return -EINTR;
+
imsic = vcpu->arch.aia_context.imsic_state;
+ if (!imsic) {
+ rc = -ENODEV;
+ goto out_unlock;
+ }
+ isel = KVM_DEV_RISCV_AIA_IMSIC_GET_ISEL(type);
read_lock_irqsave(&imsic->vsfile_lock, flags);
@@ -976,6 +983,8 @@ int kvm_riscv_aia_imsic_rw_attr(struct kvm *kvm, unsigned long type,
rc = imsic_vsfile_rw(vsfile_hgei, vsfile_cpu, imsic->nr_eix,
isel, write, val);
+out_unlock:
+ mutex_unlock(&vcpu->mutex);
return rc;
}
diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
index 58f5f3536ffd..b5a809a091d6 100644
--- a/arch/riscv/kvm/mmu.c
+++ b/arch/riscv/kvm/mmu.c
@@ -380,6 +380,8 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
vma_pageshift, current);
return 0;
}
+ if (is_sigpending_pfn(hfn))
+ return -EINTR;
if (is_error_noslot_pfn(hfn))
return -EFAULT;
diff --git a/arch/riscv/kvm/nacl.c b/arch/riscv/kvm/nacl.c
index 08a95ad9ada2..f922e9f62fee 100644
--- a/arch/riscv/kvm/nacl.c
+++ b/arch/riscv/kvm/nacl.c
@@ -42,12 +42,24 @@ void __kvm_riscv_nacl_hfence(void *shmem,
}
}
- entp = shmem + SBI_NACL_SHMEM_HFENCE_ENTRY_CONFIG(i);
- *entp = cpu_to_lelong(control);
+ /*
+ * Per SBI v3.0 section 15.1.2, the Page_Number and Page_Count
+ * words must be updated before the Config word with its Pending
+ * bit set. WRITE_ONCE() stops the compiler from reordering the
+ * stores and smp_wmb() makes the parameter words globally
+ * visible to the SBI implementation (or NACL hardware) before
+ * the Pending bit is set.
+ */
entp = shmem + SBI_NACL_SHMEM_HFENCE_ENTRY_PNUM(i);
- *entp = cpu_to_lelong(page_num);
+ WRITE_ONCE(*entp, cpu_to_lelong(page_num));
entp = shmem + SBI_NACL_SHMEM_HFENCE_ENTRY_PCOUNT(i);
- *entp = cpu_to_lelong(page_count);
+ WRITE_ONCE(*entp, cpu_to_lelong(page_count));
+
+ /* Ensure the parameter words are visible before the Pending bit */
+ smp_wmb();
+
+ entp = shmem + SBI_NACL_SHMEM_HFENCE_ENTRY_CONFIG(i);
+ WRITE_ONCE(*entp, cpu_to_lelong(control));
}
int kvm_riscv_nacl_enable(void)
diff --git a/arch/riscv/kvm/vcpu_exit.c b/arch/riscv/kvm/vcpu_exit.c
index 0bb0c51e3c89..219f553cc2b7 100644
--- a/arch/riscv/kvm/vcpu_exit.c
+++ b/arch/riscv/kvm/vcpu_exit.c
@@ -248,7 +248,7 @@ int kvm_riscv_vcpu_exit(struct kvm_vcpu *vcpu, struct kvm_run *run,
}
/* Print details in-case of error */
- if (ret < 0) {
+ if (ret < 0 && ret != -EINTR) {
kvm_err("VCPU exit error %d\n", ret);
kvm_err("SEPC=0x%lx SSTATUS=0x%lx HSTATUS=0x%lx\n",
vcpu->arch.guest_context.sepc,
diff --git a/arch/riscv/kvm/vcpu_pmu.c b/arch/riscv/kvm/vcpu_pmu.c
index ef55089c0c1a..b374d92c9aa6 100644
--- a/arch/riscv/kvm/vcpu_pmu.c
+++ b/arch/riscv/kvm/vcpu_pmu.c
@@ -249,12 +249,13 @@ static int pmu_ctr_read(struct kvm_vcpu *vcpu, unsigned long cidx,
if (pmc->cinfo.type == SBI_PMU_CTR_TYPE_FW) {
fevent_code = get_event_code(pmc->event_idx);
pmc->counter_val = kvpmu->fw_event[fevent_code].value;
+ *out_val = pmc->counter_val;
} else if (pmc->perf_event) {
- pmc->counter_val += perf_event_read_value(pmc->perf_event, &enabled, &running);
+ *out_val = pmc->counter_val +
+ perf_event_read_value(pmc->perf_event, &enabled, &running);
} else {
return -EINVAL;
}
- *out_val = pmc->counter_val;
return 0;
}
@@ -619,7 +620,6 @@ int kvm_riscv_vcpu_pmu_ctr_stop(struct kvm_vcpu *vcpu, unsigned long ctr_base,
{
struct kvm_pmu *kvpmu = vcpu_to_pmu(vcpu);
int i, pmc_index, sbiret = 0;
- u64 enabled, running;
struct kvm_pmc *pmc;
int fevent_code;
bool snap_flag_set = flags & SBI_PMU_STOP_FLAG_TAKE_SNAPSHOT;
@@ -648,14 +648,19 @@ int kvm_riscv_vcpu_pmu_ctr_stop(struct kvm_vcpu *vcpu, unsigned long ctr_base,
goto out;
}
- if (!kvpmu->fw_event[fevent_code].started)
+ if (!kvpmu->fw_event[fevent_code].started) {
sbiret = SBI_ERR_ALREADY_STOPPED;
-
- kvpmu->fw_event[fevent_code].started = false;
+ } else {
+ kvpmu->fw_event[fevent_code].started = false;
+ pmc->counter_val = kvpmu->fw_event[fevent_code].value;
+ }
} else if (pmc->perf_event) {
if (pmc->started) {
- /* Stop counting the counter */
- perf_event_disable(pmc->perf_event);
+ /*
+ * Stop the counter and fold the live count into counter_val.
+ * Reset the event value to avoid redundant accumulation.
+ */
+ pmc->counter_val += perf_event_pause(pmc->perf_event, true);
pmc->started = false;
} else {
sbiret = SBI_ERR_ALREADY_STOPPED;
@@ -669,11 +674,6 @@ int kvm_riscv_vcpu_pmu_ctr_stop(struct kvm_vcpu *vcpu, unsigned long ctr_base,
}
if (snap_flag_set && !sbiret) {
- if (pmc->cinfo.type == SBI_PMU_CTR_TYPE_FW)
- pmc->counter_val = kvpmu->fw_event[fevent_code].value;
- else if (pmc->perf_event)
- pmc->counter_val += perf_event_read_value(pmc->perf_event,
- &enabled, &running);
/*
* The counter and overflow indicies in the snapshot region are w.r.to
* cbase. Modify the set bit in the counter mask instead of the pmc_index
@@ -700,9 +700,10 @@ int kvm_riscv_vcpu_pmu_ctr_stop(struct kvm_vcpu *vcpu, unsigned long ctr_base,
}
}
- if (shmem_needs_update)
- kvm_vcpu_write_guest(vcpu, kvpmu->snapshot_addr, kvpmu->sdata,
- sizeof(struct riscv_pmu_snapshot_data));
+ if (shmem_needs_update &&
+ kvm_vcpu_write_guest(vcpu, kvpmu->snapshot_addr, kvpmu->sdata,
+ sizeof(struct riscv_pmu_snapshot_data)))
+ sbiret = SBI_ERR_FAILURE;
out:
retdata->err_val = sbiret;
diff --git a/arch/riscv/kvm/vcpu_sbi_hsm.c b/arch/riscv/kvm/vcpu_sbi_hsm.c
index f26207f84bab..06a15629c26b 100644
--- a/arch/riscv/kvm/vcpu_sbi_hsm.c
+++ b/arch/riscv/kvm/vcpu_sbi_hsm.c
@@ -95,9 +95,9 @@ static int kvm_sbi_ext_hsm_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
ret = kvm_sbi_hsm_vcpu_get_status(vcpu);
if (ret >= 0) {
retdata->out_val = ret;
- retdata->err_val = 0;
+ ret = 0;
}
- return 0;
+ break;
case SBI_EXT_HSM_HART_SUSPEND:
switch (lower_32_bits(cp->a0)) {
case SBI_HSM_SUSPEND_RET_DEFAULT:
diff --git a/arch/riscv/kvm/vcpu_timer.c b/arch/riscv/kvm/vcpu_timer.c
index 85a7262115e1..ea579ae0c213 100644
--- a/arch/riscv/kvm/vcpu_timer.c
+++ b/arch/riscv/kvm/vcpu_timer.c
@@ -60,10 +60,13 @@ static enum hrtimer_restart kvm_riscv_vcpu_hrtimer_expired(struct hrtimer *h)
static int kvm_riscv_vcpu_timer_cancel(struct kvm_vcpu_timer *t)
{
- if (!t->init_done || !t->next_set)
+ if (!t->init_done)
return -EINVAL;
hrtimer_cancel(&t->hrt);
+
+ if (!t->next_set)
+ return -EINVAL;
t->next_set = false;
return 0;
diff --git a/arch/s390/crypto/hmac_s390.c b/arch/s390/crypto/hmac_s390.c
index 58444da9b004..04f3330ad06f 100644
--- a/arch/s390/crypto/hmac_s390.c
+++ b/arch/s390/crypto/hmac_s390.c
@@ -151,7 +151,10 @@ static int hash_data(const u8 *in, unsigned int inlen,
#undef PARAM_INIT
- cpacf_klmd(func, ¶m, in, inlen);
+ if (final)
+ cpacf_klmd(func, ¶m, in, inlen);
+ else
+ cpacf_kimd(func, ¶m, in, inlen);
memcpy(digest, ¶m, digestsize);
diff --git a/arch/s390/include/asm/debug.h b/arch/s390/include/asm/debug.h
index 6375276d94ea..293f19420165 100644
--- a/arch/s390/include/asm/debug.h
+++ b/arch/s390/include/asm/debug.h
@@ -460,7 +460,11 @@ static int VNAME(var, active_entries)[EARLY_AREAS] __initdata
#define __REGISTER_STATIC_DEBUG_INFO(var, name, pages, areas, view) \
static int __init VNAME(var, reg)(void) \
{ \
- debug_register_static(&var, (pages), (areas)); \
+ int rc; \
+ \
+ rc = debug_register_static(&var, (pages), (areas)); \
+ if (rc) \
+ return rc; \
debug_register_view(&var, (view)); \
return 0; \
} \
@@ -492,7 +496,7 @@ static debug_info_t __refdata var = \
__DEBUG_INFO_INIT(var, (name), (buf_size)); \
__REGISTER_STATIC_DEBUG_INFO(var, name, pages, nr_areas, view)
-void debug_register_static(debug_info_t *id, int pages_per_area, int nr_areas);
+int debug_register_static(debug_info_t *id, int pages_per_area, int nr_areas);
#endif /* MODULE */
diff --git a/arch/s390/kernel/debug.c b/arch/s390/kernel/debug.c
index f5359b010051..c4db346688ed 100644
--- a/arch/s390/kernel/debug.c
+++ b/arch/s390/kernel/debug.c
@@ -346,7 +346,8 @@ static debug_info_t *debug_info_copy(debug_info_t *in, int mode)
debug_info_free(rc);
} while (1);
- if (mode == NO_AREAS)
+ /* debug_register_static() failure leaves areas NULL, bounds intact */
+ if (mode == NO_AREAS || !in->areas)
goto out;
for (i = 0; i < in->nr_areas; i++) {
@@ -862,8 +863,12 @@ EXPORT_SYMBOL(debug_register);
*
* Note: This function is called automatically via an initcall generated by
* DEFINE_STATIC_DEBUG_INFO.
+ *
+ * Return:
+ * - 0 on success
+ * - negative error code on failure
*/
-void debug_register_static(debug_info_t *id, int pages_per_area, int nr_areas)
+int debug_register_static(debug_info_t *id, int pages_per_area, int nr_areas)
{
unsigned long flags;
debug_info_t *copy;
@@ -871,7 +876,7 @@ void debug_register_static(debug_info_t *id, int pages_per_area, int nr_areas)
if (!initialized) {
pr_err("Tried to register debug feature %s too early\n",
id->name);
- return;
+ return -EINVAL;
}
copy = debug_info_alloc("", pages_per_area, nr_areas, id->buf_size,
@@ -886,7 +891,7 @@ void debug_register_static(debug_info_t *id, int pages_per_area, int nr_areas)
id->active_entries = NULL;
spin_unlock_irqrestore(&id->lock, flags);
- return;
+ return -ENOMEM;
}
/* Replace static trace area with dynamic copy. */
@@ -904,6 +909,8 @@ void debug_register_static(debug_info_t *id, int pages_per_area, int nr_areas)
mutex_lock(&debug_mutex);
_debug_register(id);
mutex_unlock(&debug_mutex);
+
+ return 0;
}
/* Remove debugfs entries. */
diff --git a/arch/s390/kernel/uv.c b/arch/s390/kernel/uv.c
index 5d17609bcfe1..d49206bf7415 100644
--- a/arch/s390/kernel/uv.c
+++ b/arch/s390/kernel/uv.c
@@ -857,7 +857,7 @@ static int find_secret_in_page(const u8 secret_id[UV_SECRET_ID_LEN],
{
u16 i;
- for (i = 0; i < list->total_num_secrets; i++) {
+ for (i = 0; i < list->num_secr_stored; i++) {
if (memcmp(secret_id, list->secrets[i].id, UV_SECRET_ID_LEN) == 0) {
*secret = list->secrets[i].hdr;
return 0;
@@ -878,11 +878,14 @@ int uv_find_secret(const u8 secret_id[UV_SECRET_ID_LEN],
struct uv_secret_list *list,
struct uv_secret_list_item_hdr *secret)
{
- u16 start_idx = 0;
+ u16 start_idx;
u16 list_rc;
int ret;
+ list->next_secret_idx = 0;
+
do {
+ start_idx = list->next_secret_idx;
uv_list_secrets(list, start_idx, &list_rc, NULL);
if (list_rc != UVC_RC_EXECUTED && list_rc != UVC_RC_MORE_DATA) {
if (list_rc == UVC_RC_INV_CMD)
@@ -893,7 +896,6 @@ int uv_find_secret(const u8 secret_id[UV_SECRET_ID_LEN],
ret = find_secret_in_page(secret_id, list, secret);
if (ret == 0)
return ret;
- start_idx = list->next_secret_idx;
} while (list_rc == UVC_RC_MORE_DATA && start_idx < list->next_secret_idx);
return -ENOENT;
diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
index 1c83962baf5f..d39884e63fa1 100644
--- a/arch/s390/kvm/interrupt.c
+++ b/arch/s390/kvm/interrupt.c
@@ -1579,23 +1579,21 @@ static int __inject_set_prefix(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq)
}
#define KVM_S390_STOP_SUPP_FLAGS (KVM_S390_STOP_FLAG_STORE_STATUS)
-static int __inject_sigp_stop(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq)
+static int __inject_sigp_stop(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq, bool *storestatus)
{
struct kvm_s390_local_interrupt *li = &vcpu->arch.local_int;
struct kvm_s390_stop_info *stop = &li->irq.stop;
- int rc = 0;
vcpu->stat.inject_stop_signal++;
trace_kvm_s390_inject_vcpu(vcpu->vcpu_id, KVM_S390_SIGP_STOP, 0, 0);
if (irq->u.stop.flags & ~KVM_S390_STOP_SUPP_FLAGS)
return -EINVAL;
-
if (is_vcpu_stopped(vcpu)) {
- if (irq->u.stop.flags & KVM_S390_STOP_FLAG_STORE_STATUS)
- rc = kvm_s390_store_status_unloaded(vcpu,
- KVM_S390_STORE_STATUS_NOADDR);
- return rc;
+ if (!(irq->u.stop.flags & KVM_S390_STOP_FLAG_STORE_STATUS))
+ return 0;
+ *storestatus = true;
+ return -EWOULDBLOCK;
}
if (test_and_set_bit(IRQ_PEND_SIGP_STOP, &li->pending_irqs))
@@ -2132,7 +2130,7 @@ void kvm_s390_clear_stop_irq(struct kvm_vcpu *vcpu)
spin_unlock(&li->lock);
}
-static int do_inject_vcpu(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq)
+static int do_inject_vcpu(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq, bool *storestatus)
{
int rc;
@@ -2144,7 +2142,7 @@ static int do_inject_vcpu(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq)
rc = __inject_set_prefix(vcpu, irq);
break;
case KVM_S390_SIGP_STOP:
- rc = __inject_sigp_stop(vcpu, irq);
+ rc = __inject_sigp_stop(vcpu, irq, storestatus);
break;
case KVM_S390_RESTART:
rc = __inject_sigp_restart(vcpu);
@@ -2180,11 +2178,16 @@ static int do_inject_vcpu(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq)
int kvm_s390_inject_vcpu(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq)
{
struct kvm_s390_local_interrupt *li = &vcpu->arch.local_int;
+ bool storestatus = false;
int rc;
spin_lock(&li->lock);
- rc = do_inject_vcpu(vcpu, irq);
+ rc = do_inject_vcpu(vcpu, irq, &storestatus);
spin_unlock(&li->lock);
+
+ if (rc == -EWOULDBLOCK && storestatus)
+ rc = kvm_s390_store_status_unloaded(vcpu, KVM_S390_STORE_STATUS_NOADDR);
+
if (!rc)
kvm_s390_vcpu_wakeup(vcpu);
return rc;
@@ -2928,7 +2931,8 @@ int kvm_set_msi(struct kvm_kernel_irq_routing_entry *e, struct kvm *kvm,
int kvm_s390_set_irq_state(struct kvm_vcpu *vcpu, void __user *irqstate, int len)
{
struct kvm_s390_local_interrupt *li = &vcpu->arch.local_int;
- struct kvm_s390_irq *buf;
+ struct kvm_s390_irq *buf __free(kvfree) = NULL;
+ bool tmp, storestatus = false;
int r = 0;
int n;
@@ -2936,31 +2940,33 @@ int kvm_s390_set_irq_state(struct kvm_vcpu *vcpu, void __user *irqstate, int len
if (!buf)
return -ENOMEM;
- if (copy_from_user((void *) buf, irqstate, len)) {
- r = -EFAULT;
- goto out_free;
- }
+ if (copy_from_user((void *)buf, irqstate, len))
+ return -EFAULT;
- /*
- * Don't allow setting the interrupt state
- * when there are already interrupts pending
- */
- spin_lock(&li->lock);
- if (li->pending_irqs) {
- r = -EBUSY;
- goto out_unlock;
- }
+ scoped_guard(spinlock, &li->lock) {
+ /*
+ * Don't allow setting the interrupt state
+ * when there are already interrupts pending
+ */
+ if (li->pending_irqs)
+ return -EBUSY;
- for (n = 0; n < len / sizeof(*buf); n++) {
- r = do_inject_vcpu(vcpu, &buf[n]);
- if (r)
- break;
+ for (n = 0; n < len / sizeof(*buf); n++) {
+ tmp = false;
+ r = do_inject_vcpu(vcpu, &buf[n], &tmp);
+ if (r == -EWOULDBLOCK && tmp) {
+ storestatus = true;
+ r = 0;
+ }
+ if (r)
+ break;
+ }
}
-out_unlock:
- spin_unlock(&li->lock);
-out_free:
- vfree(buf);
+ if (storestatus) {
+ n = kvm_s390_store_status_unloaded(vcpu, KVM_S390_STORE_STATUS_NOADDR);
+ return r ? r : n;
+ }
return r;
}
diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c
index 27db1e72c623..27cac30b4b75 100644
--- a/arch/s390/pci/pci_event.c
+++ b/arch/s390/pci/pci_event.c
@@ -191,6 +191,7 @@ static pci_ers_result_t zpci_event_attempt_error_recovery(struct pci_dev *pdev)
device_lock(&pdev->dev);
if (pdev->error_state == pci_channel_io_perm_failure) {
ers_res = PCI_ERS_RESULT_DISCONNECT;
+ status_str = "skipped (permanent failure)";
goto out_unlock;
}
pdev->error_state = pci_channel_io_frozen;
@@ -257,8 +258,8 @@ static pci_ers_result_t zpci_event_attempt_error_recovery(struct pci_dev *pdev)
driver->err_handler->resume(pdev);
pci_uevent_ers(pdev, PCI_ERS_RESULT_RECOVERED);
out_unlock:
+ zpci_report_status(zdev, pdev, "recovery", status_str);
device_unlock(&pdev->dev);
- zpci_report_status(zdev, "recovery", status_str);
return ers_res;
}
@@ -321,8 +322,10 @@ static void __zpci_event_error(struct zpci_ccdf_err *ccdf)
pr_err("%s: Event 0x%x reports an error for PCI function 0x%x\n",
pdev ? pci_name(pdev) : "n/a", ccdf->pec, ccdf->fid);
- if (!pdev)
+ if (!pdev) {
+ zpci_report_status(zdev, NULL, "error event", "no pdev bound");
goto no_pdev;
+ }
switch (ccdf->pec) {
case 0x002a: /* Error event concerns FMB */
diff --git a/arch/s390/pci/pci_report.c b/arch/s390/pci/pci_report.c
index 1b494e5ecc4d..af6d86d8e23e 100644
--- a/arch/s390/pci/pci_report.c
+++ b/arch/s390/pci/pci_report.c
@@ -88,9 +88,23 @@ static struct debug_view debug_log_view = {
NULL
};
+static ssize_t zpci_report_pdev(struct pci_dev *pdev, char *buf, size_t size)
+{
+ struct pci_driver *driver;
+ const char *start = buf;
+ char *end = buf + size;
+
+ device_lock_assert(&pdev->dev);
+ buf += scnprintf(buf, end - buf, "state: %s\n", zpci_state_str(pdev->error_state));
+ driver = to_pci_driver(pdev->dev.driver);
+ buf += scnprintf(buf, end - buf, "driver: %s\n", (driver) ? driver->name : "n/a");
+ return buf - start;
+}
+
/**
* zpci_report_status - Report the status of operations on a PCI device
- * @zdev: The PCI device for which to report status
+ * @zdev: The zPCI device for which to report status
+ * @pdev: The PCI device associated with the zdev if any, NULL otherwise
* @operation: A string representing the operation reported
* @status: A string representing the status of the operation
*
@@ -104,15 +118,14 @@ static struct debug_view debug_log_view = {
*
* Return: 0 on success an error code < 0 otherwise.
*/
-int zpci_report_status(struct zpci_dev *zdev, const char *operation, const char *status)
+int zpci_report_status(struct zpci_dev *zdev, struct pci_dev *pdev,
+ const char *operation, const char *status)
{
struct zpci_report_error *report;
- struct pci_driver *driver = NULL;
- struct pci_dev *pdev = NULL;
char *buf, *end;
int ret;
- if (!zdev || !zdev->zbus)
+ if (!zdev)
return -ENODEV;
/* Protected virtualization hosts get nothing from us */
@@ -122,18 +135,13 @@ int zpci_report_status(struct zpci_dev *zdev, const char *operation, const char
report = (void *)get_zeroed_page(GFP_KERNEL);
if (!report)
return -ENOMEM;
- if (zdev->zbus->bus)
- pdev = pci_get_slot(zdev->zbus->bus, zdev->devfn);
- if (pdev)
- driver = to_pci_driver(pdev->dev.driver);
buf = report->data.log_data;
end = report->data.log_data + ZPCI_REPORT_DATA_SIZE;
buf += scnprintf(buf, end - buf, "report: %s\n", operation);
buf += scnprintf(buf, end - buf, "status: %s\n", status);
- buf += scnprintf(buf, end - buf, "state: %s\n",
- (pdev) ? zpci_state_str(pdev->error_state) : "n/a");
- buf += scnprintf(buf, end - buf, "driver: %s\n", (driver) ? driver->name : "n/a");
+ if (pdev)
+ buf += zpci_report_pdev(pdev, buf, end - buf);
ret = debug_dump(pci_debug_msg_id, &debug_log_view, buf, end - buf, true);
if (ret < 0)
pr_err("Reading PCI debug messages failed with code %d\n", ret);
diff --git a/arch/s390/pci/pci_report.h b/arch/s390/pci/pci_report.h
index e08003d51a97..dd7b0b05001c 100644
--- a/arch/s390/pci/pci_report.h
+++ b/arch/s390/pci/pci_report.h
@@ -8,9 +8,11 @@
*/
#ifndef __S390_PCI_REPORT_H
#define __S390_PCI_REPORT_H
+#include <linux/pci.h>
struct zpci_dev;
-int zpci_report_status(struct zpci_dev *zdev, const char *operation, const char *status);
+int zpci_report_status(struct zpci_dev *zdev, struct pci_dev *pdev,
+ const char *operation, const char *status);
#endif /* __S390_PCI_REPORT_H */
diff --git a/arch/x86/boot/startup/sev-startup.c b/arch/x86/boot/startup/sev-startup.c
index 09725428d3e6..789e99d38d17 100644
--- a/arch/x86/boot/startup/sev-startup.c
+++ b/arch/x86/boot/startup/sev-startup.c
@@ -27,7 +27,6 @@
#include <asm/cpu_entry_area.h>
#include <asm/stacktrace.h>
#include <asm/sev.h>
-#include <asm/sev-internal.h>
#include <asm/insn-eval.h>
#include <asm/fpu/xcr.h>
#include <asm/processor.h>
@@ -41,6 +40,8 @@
#include <asm/cpuid/api.h>
#include <asm/cmdline.h>
+#include "../../coco/sev/internal.h"
+
/* Include code shared with pre-decompression boot stage */
#include "sev-shared.c"
diff --git a/arch/x86/coco/sev/Makefile b/arch/x86/coco/sev/Makefile
index b2e9ec2f6901..3111a2ee6722 100644
--- a/arch/x86/coco/sev/Makefile
+++ b/arch/x86/coco/sev/Makefile
@@ -1,6 +1,6 @@
# SPDX-License-Identifier: GPL-2.0
-obj-y += core.o noinstr.o vc-handle.o
+obj-y += core.o noinstr.o vc-handle.o svsm.o
# Clang 14 and older may fail to respect __no_sanitize_undefined when inlining
UBSAN_SANITIZE_noinstr.o := n
diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index d20e9cc065a8..fefd7031be02 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -31,7 +31,6 @@
#include <asm/cpu_entry_area.h>
#include <asm/stacktrace.h>
#include <asm/sev.h>
-#include <asm/sev-internal.h>
#include <asm/insn-eval.h>
#include <asm/fpu/xcr.h>
#include <asm/processor.h>
@@ -46,6 +45,8 @@
#include <asm/cmdline.h>
#include <asm/msr.h>
+#include "internal.h"
+
/* Bitmap of SEV features supported by the hypervisor */
u64 sev_hv_features __ro_after_init;
SYM_PIC_ALIAS(sev_hv_features);
@@ -54,40 +55,6 @@ SYM_PIC_ALIAS(sev_hv_features);
u64 sev_secrets_pa __ro_after_init;
SYM_PIC_ALIAS(sev_secrets_pa);
-/* For early boot SVSM communication */
-struct svsm_ca boot_svsm_ca_page __aligned(PAGE_SIZE);
-SYM_PIC_ALIAS(boot_svsm_ca_page);
-
-/*
- * SVSM related information:
- * During boot, the page tables are set up as identity mapped and later
- * changed to use kernel virtual addresses. Maintain separate virtual and
- * physical addresses for the CAA to allow SVSM functions to be used during
- * early boot, both with identity mapped virtual addresses and proper kernel
- * virtual addresses.
- */
-u64 boot_svsm_caa_pa __ro_after_init;
-SYM_PIC_ALIAS(boot_svsm_caa_pa);
-
-DEFINE_PER_CPU(struct svsm_ca *, svsm_caa);
-DEFINE_PER_CPU(u64, svsm_caa_pa);
-
-static inline struct svsm_ca *svsm_get_caa(void)
-{
- if (sev_cfg.use_cas)
- return this_cpu_read(svsm_caa);
- else
- return rip_rel_ptr(&boot_svsm_ca_page);
-}
-
-static inline u64 svsm_get_caa_pa(void)
-{
- if (sev_cfg.use_cas)
- return this_cpu_read(svsm_caa_pa);
- else
- return boot_svsm_caa_pa;
-}
-
/* AP INIT values as documented in the APM2 section "Processor Initialization State" */
#define AP_INIT_CS_LIMIT 0xffff
#define AP_INIT_DS_LIMIT 0xffff
@@ -218,95 +185,6 @@ static u64 __init get_jump_table_addr(void)
return ret;
}
-static int svsm_perform_ghcb_protocol(struct ghcb *ghcb, struct svsm_call *call)
-{
- struct es_em_ctxt ctxt;
- u8 pending = 0;
-
- vc_ghcb_invalidate(ghcb);
-
- /*
- * Fill in protocol and format specifiers. This can be called very early
- * in the boot, so use rip-relative references as needed.
- */
- ghcb->protocol_version = ghcb_version;
- ghcb->ghcb_usage = GHCB_DEFAULT_USAGE;
-
- ghcb_set_sw_exit_code(ghcb, SVM_VMGEXIT_SNP_RUN_VMPL);
- ghcb_set_sw_exit_info_1(ghcb, 0);
- ghcb_set_sw_exit_info_2(ghcb, 0);
-
- sev_es_wr_ghcb_msr(__pa(ghcb));
-
- svsm_issue_call(call, &pending);
-
- if (pending)
- return -EINVAL;
-
- switch (verify_exception_info(ghcb, &ctxt)) {
- case ES_OK:
- break;
- case ES_EXCEPTION:
- vc_forward_exception(&ctxt);
- fallthrough;
- default:
- return -EINVAL;
- }
-
- return svsm_process_result_codes(call);
-}
-
-static int svsm_perform_call_protocol(struct svsm_call *call)
-{
- struct ghcb_state state;
- unsigned long flags;
- struct ghcb *ghcb;
- int ret;
-
- flags = native_local_irq_save();
-
- if (sev_cfg.ghcbs_initialized)
- ghcb = __sev_get_ghcb(&state);
- else if (boot_ghcb)
- ghcb = boot_ghcb;
- else
- ghcb = NULL;
-
- do {
- ret = ghcb ? svsm_perform_ghcb_protocol(ghcb, call)
- : __pi_svsm_perform_msr_protocol(call);
- } while (ret == -EAGAIN);
-
- if (sev_cfg.ghcbs_initialized)
- __sev_put_ghcb(&state);
-
- native_local_irq_restore(flags);
-
- return ret;
-}
-
-static inline void __pval_terminate(u64 pfn, bool action, unsigned int page_size,
- int ret, u64 svsm_ret)
-{
- WARN(1, "PVALIDATE failure: pfn: 0x%llx, action: %u, size: %u, ret: %d, svsm_ret: 0x%llx\n",
- pfn, action, page_size, ret, svsm_ret);
-
- sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PVALIDATE);
-}
-
-static void svsm_pval_terminate(struct svsm_pvalidate_call *pc, int ret, u64 svsm_ret)
-{
- unsigned int page_size;
- bool action;
- u64 pfn;
-
- pfn = pc->entry[pc->cur_index].pfn;
- action = pc->entry[pc->cur_index].action;
- page_size = pc->entry[pc->cur_index].page_size;
-
- __pval_terminate(pfn, action, page_size, ret, svsm_ret);
-}
-
static void pval_pages(struct snp_psc_desc *desc)
{
struct psc_entry *e;
@@ -343,152 +221,6 @@ static void pval_pages(struct snp_psc_desc *desc)
}
}
-static u64 svsm_build_ca_from_pfn_range(u64 pfn, u64 pfn_end, bool action,
- struct svsm_pvalidate_call *pc)
-{
- struct svsm_pvalidate_entry *pe;
-
- /* Nothing in the CA yet */
- pc->num_entries = 0;
- pc->cur_index = 0;
-
- pe = &pc->entry[0];
-
- while (pfn < pfn_end) {
- pe->page_size = RMP_PG_SIZE_4K;
- pe->action = action;
- pe->ignore_cf = 0;
- pe->rsvd = 0;
- pe->pfn = pfn;
-
- pe++;
- pfn++;
-
- pc->num_entries++;
- if (pc->num_entries == SVSM_PVALIDATE_MAX_COUNT)
- break;
- }
-
- return pfn;
-}
-
-static int svsm_build_ca_from_psc_desc(struct snp_psc_desc *desc, unsigned int desc_entry,
- struct svsm_pvalidate_call *pc)
-{
- struct svsm_pvalidate_entry *pe;
- struct psc_entry *e;
-
- /* Nothing in the CA yet */
- pc->num_entries = 0;
- pc->cur_index = 0;
-
- pe = &pc->entry[0];
- e = &desc->entries[desc_entry];
-
- while (desc_entry <= desc->hdr.end_entry) {
- pe->page_size = e->pagesize ? RMP_PG_SIZE_2M : RMP_PG_SIZE_4K;
- pe->action = e->operation == SNP_PAGE_STATE_PRIVATE;
- pe->ignore_cf = 0;
- pe->rsvd = 0;
- pe->pfn = e->gfn;
-
- pe++;
- e++;
-
- desc_entry++;
- pc->num_entries++;
- if (pc->num_entries == SVSM_PVALIDATE_MAX_COUNT)
- break;
- }
-
- return desc_entry;
-}
-
-static void svsm_pval_pages(struct snp_psc_desc *desc)
-{
- struct svsm_pvalidate_entry pv_4k[VMGEXIT_PSC_MAX_ENTRY];
- unsigned int i, pv_4k_count = 0;
- struct svsm_pvalidate_call *pc;
- struct svsm_call call = {};
- unsigned long flags;
- bool action;
- u64 pc_pa;
- int ret;
-
- /*
- * This can be called very early in the boot, use native functions in
- * order to avoid paravirt issues.
- */
- flags = native_local_irq_save();
-
- /*
- * The SVSM calling area (CA) can support processing 510 entries at a
- * time. Loop through the Page State Change descriptor until the CA is
- * full or the last entry in the descriptor is reached, at which time
- * the SVSM is invoked. This repeats until all entries in the descriptor
- * are processed.
- */
- call.caa = svsm_get_caa();
-
- pc = (struct svsm_pvalidate_call *)call.caa->svsm_buffer;
- pc_pa = svsm_get_caa_pa() + offsetof(struct svsm_ca, svsm_buffer);
-
- /* Protocol 0, Call ID 1 */
- call.rax = SVSM_CORE_CALL(SVSM_CORE_PVALIDATE);
- call.rcx = pc_pa;
-
- for (i = 0; i <= desc->hdr.end_entry;) {
- i = svsm_build_ca_from_psc_desc(desc, i, pc);
-
- do {
- ret = svsm_perform_call_protocol(&call);
- if (!ret)
- continue;
-
- /*
- * Check if the entry failed because of an RMP mismatch (a
- * PVALIDATE at 2M was requested, but the page is mapped in
- * the RMP as 4K).
- */
-
- if (call.rax_out == SVSM_PVALIDATE_FAIL_SIZEMISMATCH &&
- pc->entry[pc->cur_index].page_size == RMP_PG_SIZE_2M) {
- /* Save this entry for post-processing at 4K */
- pv_4k[pv_4k_count++] = pc->entry[pc->cur_index];
-
- /* Skip to the next one unless at the end of the list */
- pc->cur_index++;
- if (pc->cur_index < pc->num_entries)
- ret = -EAGAIN;
- else
- ret = 0;
- }
- } while (ret == -EAGAIN);
-
- if (ret)
- svsm_pval_terminate(pc, ret, call.rax_out);
- }
-
- /* Process any entries that failed to be validated at 2M and validate them at 4K */
- for (i = 0; i < pv_4k_count; i++) {
- u64 pfn, pfn_end;
-
- action = pv_4k[i].action;
- pfn = pv_4k[i].pfn;
- pfn_end = pfn + 512;
-
- while (pfn < pfn_end) {
- pfn = svsm_build_ca_from_pfn_range(pfn, pfn_end, action, pc);
-
- ret = svsm_perform_call_protocol(&call);
- if (ret)
- svsm_pval_terminate(pc, ret, call.rax_out);
- }
- }
-
- native_local_irq_restore(flags);
-}
-
static void pvalidate_pages(struct snp_psc_desc *desc)
{
struct psc_entry *e;
@@ -635,17 +367,13 @@ static unsigned long __set_pages_state(struct snp_psc_desc *data, unsigned long
local_irq_save(flags);
- if (sev_cfg.ghcbs_initialized)
- ghcb = __sev_get_ghcb(&state);
- else
- ghcb = boot_ghcb;
+ ghcb = __sev_get_ghcb(&state);
/* Invoke the hypervisor to perform the page state changes */
if (!ghcb || vmgexit_psc(ghcb, data))
sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PSC);
- if (sev_cfg.ghcbs_initialized)
- __sev_put_ghcb(&state);
+ __sev_put_ghcb(&state);
local_irq_restore(flags);
@@ -1589,56 +1317,6 @@ static int __init report_snp_info(void)
}
arch_initcall(report_snp_info);
-static void update_attest_input(struct svsm_call *call, struct svsm_attest_call *input)
-{
- /* If (new) lengths have been returned, propagate them up */
- if (call->rcx_out != call->rcx)
- input->manifest_buf.len = call->rcx_out;
-
- if (call->rdx_out != call->rdx)
- input->certificates_buf.len = call->rdx_out;
-
- if (call->r8_out != call->r8)
- input->report_buf.len = call->r8_out;
-}
-
-int snp_issue_svsm_attest_req(u64 call_id, struct svsm_call *call,
- struct svsm_attest_call *input)
-{
- struct svsm_attest_call *ac;
- unsigned long flags;
- u64 attest_call_pa;
- int ret;
-
- if (!snp_vmpl)
- return -EINVAL;
-
- local_irq_save(flags);
-
- call->caa = svsm_get_caa();
-
- ac = (struct svsm_attest_call *)call->caa->svsm_buffer;
- attest_call_pa = svsm_get_caa_pa() + offsetof(struct svsm_ca, svsm_buffer);
-
- *ac = *input;
-
- /*
- * Set input registers for the request and set RDX and R8 to known
- * values in order to detect length values being returned in them.
- */
- call->rax = call_id;
- call->rcx = attest_call_pa;
- call->rdx = -1;
- call->r8 = -1;
- ret = svsm_perform_call_protocol(call);
- update_attest_input(call, input);
-
- local_irq_restore(flags);
-
- return ret;
-}
-EXPORT_SYMBOL_GPL(snp_issue_svsm_attest_req);
-
static int snp_issue_guest_request(struct snp_guest_req *req)
{
struct snp_req_data *input = &req->input;
@@ -1703,64 +1381,6 @@ static int snp_issue_guest_request(struct snp_guest_req *req)
return ret;
}
-/**
- * snp_svsm_vtpm_probe() - Probe if SVSM provides a vTPM device
- *
- * Check that there is SVSM and that it supports at least TPM_SEND_COMMAND
- * which is the only request used so far.
- *
- * Return: true if the platform provides a vTPM SVSM device, false otherwise.
- */
-static bool snp_svsm_vtpm_probe(void)
-{
- struct svsm_call call = {};
-
- /* The vTPM device is available only if a SVSM is present */
- if (!snp_vmpl)
- return false;
-
- call.caa = svsm_get_caa();
- call.rax = SVSM_VTPM_CALL(SVSM_VTPM_QUERY);
-
- if (svsm_perform_call_protocol(&call))
- return false;
-
- /* Check platform commands contains TPM_SEND_COMMAND - platform command 8 */
- return call.rcx_out & BIT_ULL(8);
-}
-
-/**
- * snp_svsm_vtpm_send_command() - Execute a vTPM operation on SVSM
- * @buffer: A buffer used to both send the command and receive the response.
- *
- * Execute a SVSM_VTPM_CMD call as defined by
- * "Secure VM Service Module for SEV-SNP Guests" Publication # 58019 Revision: 1.00
- *
- * All command request/response buffers have a common structure as specified by
- * the following table:
- * Byte Size In/Out Description
- * Offset (Bytes)
- * 0x000 4 In Platform command
- * Out Platform command response size
- *
- * Each command can build upon this common request/response structure to create
- * a structure specific to the command. See include/linux/tpm_svsm.h for more
- * details.
- *
- * Return: 0 on success, -errno on failure
- */
-int snp_svsm_vtpm_send_command(u8 *buffer)
-{
- struct svsm_call call = {};
-
- call.caa = svsm_get_caa();
- call.rax = SVSM_VTPM_CALL(SVSM_VTPM_CMD);
- call.rcx = __pa(buffer);
-
- return svsm_perform_call_protocol(&call);
-}
-EXPORT_SYMBOL_GPL(snp_svsm_vtpm_send_command);
-
static struct platform_device sev_guest_device = {
.name = "sev-guest",
.id = -1,
diff --git a/arch/x86/coco/sev/internal.h b/arch/x86/coco/sev/internal.h
new file mode 100644
index 000000000000..039326b5c799
--- /dev/null
+++ b/arch/x86/coco/sev/internal.h
@@ -0,0 +1,119 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __X86_COCO_SEV_INTERNAL_H__
+#define __X86_COCO_SEV_INTERNAL_H__
+
+#define DR7_RESET_VALUE 0x400
+
+extern u64 sev_hv_features;
+extern u64 sev_secrets_pa;
+
+/* #VC handler runtime per-CPU data */
+struct sev_es_runtime_data {
+ struct ghcb ghcb_page;
+
+ /*
+ * Reserve one page per CPU as backup storage for the unencrypted GHCB.
+ * It is needed when an NMI happens while the #VC handler uses the real
+ * GHCB, and the NMI handler itself is causing another #VC exception. In
+ * that case the GHCB content of the first handler needs to be backed up
+ * and restored.
+ */
+ struct ghcb backup_ghcb;
+
+ /*
+ * Mark the per-cpu GHCBs as in-use to detect nested #VC exceptions.
+ * There is no need for it to be atomic, because nothing is written to
+ * the GHCB between the read and the write of ghcb_active. So it is safe
+ * to use it when a nested #VC exception happens before the write.
+ *
+ * This is necessary for example in the #VC->NMI->#VC case when the NMI
+ * happens while the first #VC handler uses the GHCB. When the NMI code
+ * raises a second #VC handler it might overwrite the contents of the
+ * GHCB written by the first handler. To avoid this the content of the
+ * GHCB is saved and restored when the GHCB is detected to be in use
+ * already.
+ */
+ bool ghcb_active;
+ bool backup_ghcb_active;
+
+ /*
+ * Cached DR7 value - write it on DR7 writes and return it on reads.
+ * That value will never make it to the real hardware DR7 as debugging
+ * is currently unsupported in SEV-ES guests.
+ */
+ unsigned long dr7;
+};
+
+struct ghcb_state {
+ struct ghcb *ghcb;
+};
+
+extern struct svsm_ca boot_svsm_ca_page;
+
+struct ghcb *__sev_get_ghcb(struct ghcb_state *state);
+void __sev_put_ghcb(struct ghcb_state *state);
+
+DECLARE_PER_CPU(struct sev_es_runtime_data*, runtime_data);
+DECLARE_PER_CPU(struct sev_es_save_area *, sev_vmsa);
+
+void early_set_pages_state(unsigned long vaddr, unsigned long paddr,
+ unsigned long npages, const struct psc_desc *desc);
+
+DECLARE_PER_CPU(struct svsm_ca *, svsm_caa);
+DECLARE_PER_CPU(u64, svsm_caa_pa);
+
+extern u64 boot_svsm_caa_pa;
+
+enum es_result verify_exception_info(struct ghcb *ghcb, struct es_em_ctxt *ctxt);
+void vc_forward_exception(struct es_em_ctxt *ctxt);
+void svsm_pval_pages(struct snp_psc_desc *desc);
+int svsm_perform_call_protocol(struct svsm_call *call);
+bool snp_svsm_vtpm_probe(void);
+
+static inline u64 sev_es_rd_ghcb_msr(void)
+{
+ return native_rdmsrq(MSR_AMD64_SEV_ES_GHCB);
+}
+
+static __always_inline void sev_es_wr_ghcb_msr(u64 val)
+{
+ u32 low, high;
+
+ low = (u32)(val);
+ high = (u32)(val >> 32);
+
+ native_wrmsr(MSR_AMD64_SEV_ES_GHCB, low, high);
+}
+
+enum es_result sev_es_ghcb_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt, bool write);
+
+u64 get_hv_features(void);
+
+const struct snp_cpuid_table *snp_cpuid_get_table(void);
+
+static inline struct svsm_ca *svsm_get_caa(void)
+{
+ if (sev_cfg.use_cas)
+ return this_cpu_read(svsm_caa);
+ else
+ return rip_rel_ptr(&boot_svsm_ca_page);
+}
+
+static inline u64 svsm_get_caa_pa(void)
+{
+ if (sev_cfg.use_cas)
+ return this_cpu_read(svsm_caa_pa);
+ else
+ return boot_svsm_caa_pa;
+}
+
+static inline void __pval_terminate(u64 pfn, bool action, unsigned int page_size,
+ int ret, u64 svsm_ret)
+{
+ WARN(1, "PVALIDATE failure: pfn: 0x%llx, action: %u, size: %u, ret: %d, svsm_ret: 0x%llx\n",
+ pfn, action, page_size, ret, svsm_ret);
+
+ sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PVALIDATE);
+}
+
+#endif /* __X86_COCO_SEV_INTERNAL_H__ */
diff --git a/arch/x86/coco/sev/noinstr.c b/arch/x86/coco/sev/noinstr.c
index 422ce72f51d5..e1e03f12fc7b 100644
--- a/arch/x86/coco/sev/noinstr.c
+++ b/arch/x86/coco/sev/noinstr.c
@@ -16,7 +16,8 @@
#include <asm/msr.h>
#include <asm/ptrace.h>
#include <asm/sev.h>
-#include <asm/sev-internal.h>
+
+#include "internal.h"
static __always_inline bool on_vc_stack(struct pt_regs *regs)
{
@@ -120,8 +121,10 @@ noinstr struct ghcb *__sev_get_ghcb(struct ghcb_state *state)
WARN_ON(!irqs_disabled());
- if (!sev_cfg.ghcbs_initialized)
+ if (!sev_cfg.ghcbs_initialized) {
+ state->ghcb = NULL;
return boot_ghcb;
+ }
data = this_cpu_read(runtime_data);
ghcb = &data->ghcb_page;
diff --git a/arch/x86/coco/sev/svsm.c b/arch/x86/coco/sev/svsm.c
new file mode 100644
index 000000000000..2d493d55ff2e
--- /dev/null
+++ b/arch/x86/coco/sev/svsm.c
@@ -0,0 +1,362 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * SVSM support code
+ */
+
+#include <linux/types.h>
+
+#include <asm/sev.h>
+
+#include "internal.h"
+
+/* For early boot SVSM communication */
+struct svsm_ca boot_svsm_ca_page __aligned(PAGE_SIZE);
+SYM_PIC_ALIAS(boot_svsm_ca_page);
+
+/*
+ * SVSM related information:
+ * During boot, the page tables are set up as identity mapped and later
+ * changed to use kernel virtual addresses. Maintain separate virtual and
+ * physical addresses for the CAA to allow SVSM functions to be used during
+ * early boot, both with identity mapped virtual addresses and proper kernel
+ * virtual addresses.
+ */
+u64 boot_svsm_caa_pa __ro_after_init;
+SYM_PIC_ALIAS(boot_svsm_caa_pa);
+
+DEFINE_PER_CPU(struct svsm_ca *, svsm_caa);
+DEFINE_PER_CPU(u64, svsm_caa_pa);
+
+static int svsm_perform_ghcb_protocol(struct ghcb *ghcb, struct svsm_call *call)
+{
+ struct es_em_ctxt ctxt;
+ u8 pending = 0;
+
+ vc_ghcb_invalidate(ghcb);
+
+ /*
+ * Fill in protocol and format specifiers. This can be called very early
+ * in the boot, so use rip-relative references as needed.
+ */
+ ghcb->protocol_version = ghcb_version;
+ ghcb->ghcb_usage = GHCB_DEFAULT_USAGE;
+
+ ghcb_set_sw_exit_code(ghcb, SVM_VMGEXIT_SNP_RUN_VMPL);
+ ghcb_set_sw_exit_info_1(ghcb, 0);
+ ghcb_set_sw_exit_info_2(ghcb, 0);
+
+ sev_es_wr_ghcb_msr(__pa(ghcb));
+
+ svsm_issue_call(call, &pending);
+
+ if (pending)
+ return -EINVAL;
+
+ switch (verify_exception_info(ghcb, &ctxt)) {
+ case ES_OK:
+ break;
+ case ES_EXCEPTION:
+ vc_forward_exception(&ctxt);
+ fallthrough;
+ default:
+ return -EINVAL;
+ }
+
+ return svsm_process_result_codes(call);
+}
+
+int svsm_perform_call_protocol(struct svsm_call *call)
+{
+ struct ghcb_state state;
+ unsigned long flags;
+ struct ghcb *ghcb;
+ int ret;
+
+ flags = native_local_irq_save();
+
+ /*
+ * 'caa' is a per-CPU variable. To avoid using a stale or incorrect
+ * 'caa' if the task is preempted or migrated to another CPU after it
+ * is fetched, always fetch 'caa' and then issue the SVSM call with
+ * interrupts disabled. This ensures the correct 'caa' is used.
+ */
+ call->caa = svsm_get_caa();
+
+ ghcb = __sev_get_ghcb(&state);
+
+ do {
+ ret = ghcb ? svsm_perform_ghcb_protocol(ghcb, call)
+ : __pi_svsm_perform_msr_protocol(call);
+ } while (ret == -EAGAIN);
+
+ __sev_put_ghcb(&state);
+
+ native_local_irq_restore(flags);
+
+ return ret;
+}
+
+static u64 svsm_build_ca_from_pfn_range(u64 pfn, u64 pfn_end, bool action,
+ struct svsm_pvalidate_call *pc)
+{
+ struct svsm_pvalidate_entry *pe;
+
+ /* Nothing in the CA yet */
+ pc->num_entries = 0;
+ pc->cur_index = 0;
+
+ pe = &pc->entry[0];
+
+ while (pfn < pfn_end) {
+ pe->page_size = RMP_PG_SIZE_4K;
+ pe->action = action;
+ pe->ignore_cf = 0;
+ pe->rsvd = 0;
+ pe->pfn = pfn;
+
+ pe++;
+ pfn++;
+
+ pc->num_entries++;
+ if (pc->num_entries == SVSM_PVALIDATE_MAX_COUNT)
+ break;
+ }
+
+ return pfn;
+}
+
+static int svsm_build_ca_from_psc_desc(struct snp_psc_desc *desc, unsigned int desc_entry,
+ struct svsm_pvalidate_call *pc)
+{
+ struct svsm_pvalidate_entry *pe;
+ struct psc_entry *e;
+
+ /* Nothing in the CA yet */
+ pc->num_entries = 0;
+ pc->cur_index = 0;
+
+ pe = &pc->entry[0];
+ e = &desc->entries[desc_entry];
+
+ while (desc_entry <= desc->hdr.end_entry) {
+ pe->page_size = e->pagesize ? RMP_PG_SIZE_2M : RMP_PG_SIZE_4K;
+ pe->action = e->operation == SNP_PAGE_STATE_PRIVATE;
+ pe->ignore_cf = 0;
+ pe->rsvd = 0;
+ pe->pfn = e->gfn;
+
+ pe++;
+ e++;
+
+ desc_entry++;
+ pc->num_entries++;
+ if (pc->num_entries == SVSM_PVALIDATE_MAX_COUNT)
+ break;
+ }
+
+ return desc_entry;
+}
+
+static void svsm_pval_terminate(struct svsm_pvalidate_call *pc, int ret, u64 svsm_ret)
+{
+ unsigned int page_size;
+ bool action;
+ u64 pfn;
+
+ pfn = pc->entry[pc->cur_index].pfn;
+ action = pc->entry[pc->cur_index].action;
+ page_size = pc->entry[pc->cur_index].page_size;
+
+ __pval_terminate(pfn, action, page_size, ret, svsm_ret);
+}
+
+void svsm_pval_pages(struct snp_psc_desc *desc)
+{
+ struct svsm_pvalidate_entry pv_4k[VMGEXIT_PSC_MAX_ENTRY];
+ unsigned int i, pv_4k_count = 0;
+ struct svsm_pvalidate_call *pc;
+ struct svsm_call call = {};
+ unsigned long flags;
+ bool action;
+ u64 pc_pa;
+ int ret;
+
+ /*
+ * This can be called very early in the boot, use native functions in
+ * order to avoid paravirt issues.
+ */
+ flags = native_local_irq_save();
+
+ /*
+ * The SVSM calling area (CA) can support processing 510 entries at a
+ * time. Loop through the Page State Change descriptor until the CA is
+ * full or the last entry in the descriptor is reached, at which time
+ * the SVSM is invoked. This repeats until all entries in the descriptor
+ * are processed.
+ */
+ call.caa = svsm_get_caa();
+
+ pc = (struct svsm_pvalidate_call *)call.caa->svsm_buffer;
+ pc_pa = svsm_get_caa_pa() + offsetof(struct svsm_ca, svsm_buffer);
+
+ /* Protocol 0, Call ID 1 */
+ call.rax = SVSM_CORE_CALL(SVSM_CORE_PVALIDATE);
+ call.rcx = pc_pa;
+
+ for (i = 0; i <= desc->hdr.end_entry;) {
+ i = svsm_build_ca_from_psc_desc(desc, i, pc);
+
+ do {
+ ret = svsm_perform_call_protocol(&call);
+ if (!ret)
+ continue;
+
+ /*
+ * Check if the entry failed because of an RMP mismatch (a
+ * PVALIDATE at 2M was requested, but the page is mapped in
+ * the RMP as 4K).
+ */
+
+ if (call.rax_out == SVSM_PVALIDATE_FAIL_SIZEMISMATCH &&
+ pc->entry[pc->cur_index].page_size == RMP_PG_SIZE_2M) {
+ /* Save this entry for post-processing at 4K */
+ pv_4k[pv_4k_count++] = pc->entry[pc->cur_index];
+
+ /* Skip to the next one unless at the end of the list */
+ pc->cur_index++;
+ if (pc->cur_index < pc->num_entries)
+ ret = -EAGAIN;
+ else
+ ret = 0;
+ }
+ } while (ret == -EAGAIN);
+
+ if (ret)
+ svsm_pval_terminate(pc, ret, call.rax_out);
+ }
+
+ /* Process any entries that failed to be validated at 2M and validate them at 4K */
+ for (i = 0; i < pv_4k_count; i++) {
+ u64 pfn, pfn_end;
+
+ action = pv_4k[i].action;
+ pfn = pv_4k[i].pfn;
+ pfn_end = pfn + 512;
+
+ while (pfn < pfn_end) {
+ pfn = svsm_build_ca_from_pfn_range(pfn, pfn_end, action, pc);
+
+ ret = svsm_perform_call_protocol(&call);
+ if (ret)
+ svsm_pval_terminate(pc, ret, call.rax_out);
+ }
+ }
+
+ native_local_irq_restore(flags);
+}
+
+static void update_attest_input(struct svsm_call *call, struct svsm_attest_call *input)
+{
+ /* If (new) lengths have been returned, propagate them up */
+ if (call->rcx_out != call->rcx)
+ input->manifest_buf.len = call->rcx_out;
+
+ if (call->rdx_out != call->rdx)
+ input->certificates_buf.len = call->rdx_out;
+
+ if (call->r8_out != call->r8)
+ input->report_buf.len = call->r8_out;
+}
+
+int snp_issue_svsm_attest_req(u64 call_id, struct svsm_call *call,
+ struct svsm_attest_call *input)
+{
+ struct svsm_attest_call *ac;
+ unsigned long flags;
+ u64 attest_call_pa;
+ int ret;
+
+ if (!snp_vmpl)
+ return -EINVAL;
+
+ local_irq_save(flags);
+
+ call->caa = svsm_get_caa();
+
+ ac = (struct svsm_attest_call *)call->caa->svsm_buffer;
+ attest_call_pa = svsm_get_caa_pa() + offsetof(struct svsm_ca, svsm_buffer);
+
+ *ac = *input;
+
+ /*
+ * Set input registers for the request and set RDX and R8 to known
+ * values in order to detect length values being returned in them.
+ */
+ call->rax = call_id;
+ call->rcx = attest_call_pa;
+ call->rdx = -1;
+ call->r8 = -1;
+ ret = svsm_perform_call_protocol(call);
+ update_attest_input(call, input);
+
+ local_irq_restore(flags);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(snp_issue_svsm_attest_req);
+
+/**
+ * snp_svsm_vtpm_send_command() - Execute a vTPM operation on SVSM
+ * @buffer: A buffer used to both send the command and receive the response.
+ *
+ * Execute a SVSM_VTPM_CMD call as defined by
+ * "Secure VM Service Module for SEV-SNP Guests" Publication # 58019 Revision: 1.00
+ *
+ * All command request/response buffers have a common structure as specified by
+ * the following table:
+ * Byte Size In/Out Description
+ * Offset (Bytes)
+ * 0x000 4 In Platform command
+ * Out Platform command response size
+ *
+ * Each command can build upon this common request/response structure to create
+ * a structure specific to the command. See include/linux/tpm_svsm.h for more
+ * details.
+ *
+ * Return: 0 on success, -errno on failure
+ */
+int snp_svsm_vtpm_send_command(u8 *buffer)
+{
+ struct svsm_call call = {};
+
+ call.rax = SVSM_VTPM_CALL(SVSM_VTPM_CMD);
+ call.rcx = __pa(buffer);
+
+ return svsm_perform_call_protocol(&call);
+}
+EXPORT_SYMBOL_GPL(snp_svsm_vtpm_send_command);
+
+/**
+ * snp_svsm_vtpm_probe() - Probe if SVSM provides a vTPM device
+ *
+ * Check that there is SVSM and that it supports at least TPM_SEND_COMMAND
+ * which is the only request used so far.
+ *
+ * Return: true if the platform provides a vTPM SVSM device, false otherwise.
+ */
+bool snp_svsm_vtpm_probe(void)
+{
+ struct svsm_call call = {};
+
+ /* The vTPM device is available only if a SVSM is present */
+ if (!snp_vmpl)
+ return false;
+
+ call.rax = SVSM_VTPM_CALL(SVSM_VTPM_QUERY);
+
+ if (svsm_perform_call_protocol(&call))
+ return false;
+
+ /* Check platform commands contains TPM_SEND_COMMAND - platform command 8 */
+ return call.rcx_out & BIT_ULL(8);
+}
diff --git a/arch/x86/coco/sev/vc-handle.c b/arch/x86/coco/sev/vc-handle.c
index 7fc136a35334..c54f2a032e39 100644
--- a/arch/x86/coco/sev/vc-handle.c
+++ b/arch/x86/coco/sev/vc-handle.c
@@ -23,7 +23,6 @@
#include <asm/init.h>
#include <asm/stacktrace.h>
#include <asm/sev.h>
-#include <asm/sev-internal.h>
#include <asm/insn-eval.h>
#include <asm/fpu/xcr.h>
#include <asm/processor.h>
@@ -35,6 +34,8 @@
#include <asm/apic.h>
#include <asm/cpuid/api.h>
+#include "internal.h"
+
static enum es_result vc_slow_virt_to_phys(struct ghcb *ghcb, struct es_em_ctxt *ctxt,
unsigned long vaddr, phys_addr_t *paddr)
{
diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 4dacbf664b30..0d5afdaa9164 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -412,11 +412,12 @@ static struct event_constraint intel_lnc_event_constraints[] = {
INTEL_UEVENT_CONSTRAINT(0x0ca3, 0x4),
INTEL_UEVENT_CONSTRAINT(0x04a4, 0x1),
INTEL_UEVENT_CONSTRAINT(0x08a4, 0x1),
- INTEL_UEVENT_CONSTRAINT(0x10a4, 0x1),
+ INTEL_UEVENT_CONSTRAINT(0x10a4, 0x8),
INTEL_UEVENT_CONSTRAINT(0x01b1, 0x8),
INTEL_UEVENT_CONSTRAINT(0x01cd, 0x3fc),
INTEL_UEVENT_CONSTRAINT(0x02cd, 0x3),
+ INTEL_UEVENT_CONSTRAINT(0x87d0, 0x3ff),
INTEL_EVENT_CONSTRAINT_RANGE(0xd0, 0xdf, 0xf),
INTEL_UEVENT_CONSTRAINT(0x00e0, 0xf),
@@ -4627,12 +4628,15 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
struct kvm_pmu *kvm_pmu = (struct kvm_pmu *)data;
u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl);
u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable;
- int global_ctrl, pebs_enable;
+ u64 guest_pebs_mask;
+ int global_ctrl;
/*
* In addition to obeying exclude_guest/exclude_host, remove bits being
* used for PEBS when running a guest, because PEBS writes to virtual
- * addresses (not physical addresses).
+ * addresses (not physical addresses). If the guest wants to utilize
+ * PEBS, and PEBS can be safely enabled in the guest, bits for the guest's
+ * PEBS-enabled counters will be OR'd back in as appropriate.
*/
*nr = 0;
global_ctrl = (*nr)++;
@@ -4662,41 +4666,68 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
return arr;
}
- if (!kvm_pmu || !x86_pmu.pebs_ept)
+ /*
+ * If the CPU doesn't support PEBS in the guest, then there's nothing
+ * more to do as disabling PMCs via PERF_GLOBAL_CTRL is sufficient on
+ * CPUs with guest/host isolation.
+ */
+ if (!x86_pmu.pebs_ept)
return arr;
+ /*
+ * Restrict guest PEBS events to counters that (a) perf supports, (b)
+ * the guest wants to use for PEBS, (c) are not excluded from counting
+ * in the guest, and (d) _are_ excluded from counting in the host.
+ */
+ guest_pebs_mask = pebs_mask & intel_ctrl & kvm_pmu->pebs_enable &
+ ~cpuc->intel_ctrl_host_mask &
+ cpuc->intel_ctrl_guest_mask;
+
+ /*
+ * Disable counters where the guest PMC is different than the host PMC
+ * being used on behalf of the guest, as the PEBS record includes
+ * PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the
+ * wrong counter(s).
+ */
+ guest_pebs_mask &= ~kvm_pmu->host_cross_mapped_mask;
+
+ /*
+ * FIXME: Allow guest and host usage of PEBS events to co-exist instead
+ * of disabling guest PEBS entirely if the host is using PEBS.
+ * What exactly goes wrong if guest and host are using PEBS is
+ * unknown.
+ */
+ if (pebs_mask & ~cpuc->intel_ctrl_guest_mask)
+ guest_pebs_mask = 0;
+
+ /*
+ * Context switch DS_AREA and PEBS_DATA_CFG if and only if PEBS will be
+ * active in the guest; if no records will be generated while the guest
+ * is running, then simply keep the host values resident in hardware.
+ */
arr[(*nr)++] = (struct perf_guest_switch_msr){
.msr = MSR_IA32_DS_AREA,
.host = (unsigned long)cpuc->ds,
- .guest = kvm_pmu->ds_area,
+ .guest = guest_pebs_mask ? kvm_pmu->ds_area : (unsigned long)cpuc->ds,
};
if (x86_pmu.intel_cap.pebs_baseline) {
arr[(*nr)++] = (struct perf_guest_switch_msr){
.msr = MSR_PEBS_DATA_CFG,
.host = cpuc->active_pebs_data_cfg,
- .guest = kvm_pmu->pebs_data_cfg,
+ .guest = guest_pebs_mask ? kvm_pmu->pebs_data_cfg :
+ cpuc->active_pebs_data_cfg,
};
}
- pebs_enable = (*nr)++;
- arr[pebs_enable] = (struct perf_guest_switch_msr){
- .msr = MSR_IA32_PEBS_ENABLE,
- .host = cpuc->pebs_enabled & ~cpuc->intel_ctrl_guest_mask,
- .guest = pebs_mask & ~cpuc->intel_ctrl_host_mask & kvm_pmu->pebs_enable,
- };
-
- if (arr[pebs_enable].host) {
- /* Disable guest PEBS if host PEBS is enabled. */
- arr[pebs_enable].guest = 0;
- } else {
- /* Disable guest PEBS thoroughly for cross-mapped PEBS counters. */
- arr[pebs_enable].guest &= ~kvm_pmu->host_cross_mapped_mask;
- arr[global_ctrl].guest &= ~kvm_pmu->host_cross_mapped_mask;
- /* Set hw GLOBAL_CTRL bits for PEBS counter when it runs for guest */
- arr[global_ctrl].guest |= arr[pebs_enable].guest;
- }
-
+ /*
+ * Do NOT mess with PEBS_ENABLED. As above, disabling counters via
+ * PERF_GLOBAL_CTRL is sufficient, and loading a stale PEBS_ENABLED,
+ * e.g. on VM-Exit, can put the system in a bad state. Simply enable
+ * counters in PERF_GLOBAL_CTRL, as perf load PEBS_ENABLED with the
+ * full value, i.e. perf *also* relies on PERF_GLOBAL_CTRL.
+ */
+ arr[global_ctrl].guest |= guest_pebs_mask;
return arr;
}
diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c
index 711e8a6f46a3..81f6a673d745 100644
--- a/arch/x86/events/intel/ds.c
+++ b/arch/x86/events/intel/ds.c
@@ -315,6 +315,7 @@ static inline void pebs_set_tlb_lock(u64 *val, bool tlb, bool lock)
static u64 __grt_latency_data(struct perf_event *event, u64 status,
u8 dse, bool tlb, bool lock, bool blk)
{
+ union perf_mem_data_src src;
u64 val;
WARN_ON_ONCE(is_hybrid() &&
@@ -330,7 +331,16 @@ static u64 __grt_latency_data(struct perf_event *event, u64 status,
else
val |= P(BLK, NA);
- return val;
+ src.val = val;
+
+ if (event->hw.flags &
+ (PERF_X86_EVENT_PEBS_LDLAT | PERF_X86_EVENT_PEBS_LD_HSW))
+ src.mem_op = P(OP, LOAD);
+ if (event->hw.flags &
+ (PERF_X86_EVENT_PEBS_STLAT | PERF_X86_EVENT_PEBS_ST_HSW))
+ src.mem_op = P(OP, STORE);
+
+ return src.val;
}
u64 grt_latency_data(struct perf_event *event, u64 status)
@@ -385,7 +395,11 @@ static u64 lnc_latency_data(struct perf_event *event, u64 status)
val |= P(BLK, NA);
src.val = val;
- if (event->hw.flags & PERF_X86_EVENT_PEBS_ST_HSW)
+ if (event->hw.flags &
+ (PERF_X86_EVENT_PEBS_LDLAT | PERF_X86_EVENT_PEBS_LD_HSW))
+ src.mem_op = P(OP, LOAD);
+ if (event->hw.flags &
+ (PERF_X86_EVENT_PEBS_STLAT | PERF_X86_EVENT_PEBS_ST_HSW))
src.mem_op = P(OP, STORE);
return src.val;
@@ -1010,8 +1024,8 @@ struct event_constraint intel_glm_pebs_event_constraints[] = {
struct event_constraint intel_grt_pebs_event_constraints[] = {
/* Allow all events as PEBS with no flags */
- INTEL_HYBRID_LAT_CONSTRAINT(0x5d0, 0x3),
- INTEL_HYBRID_LAT_CONSTRAINT(0x6d0, 0xf),
+ INTEL_HYBRID_LDLAT_CONSTRAINT(0x5d0, 0x3),
+ INTEL_HYBRID_STLAT_CONSTRAINT(0x6d0, 0x3f),
EVENT_CONSTRAINT_END
};
@@ -1208,19 +1222,11 @@ struct event_constraint intel_lnc_pebs_event_constraints[] = {
INTEL_FLAGS_UEVENT_CONSTRAINT(0x100, 0x100000000ULL), /* INST_RETIRED.PREC_DIST */
INTEL_FLAGS_UEVENT_CONSTRAINT(0x0400, 0x800000000ULL),
+ INTEL_FLAGS_UEVENT_CONSTRAINT(0x012a, 0x1), /* OCR.* events */
+ INTEL_FLAGS_UEVENT_CONSTRAINT(0x012b, 0x1), /* OCR.* events */
+
INTEL_HYBRID_LDLAT_CONSTRAINT(0x1cd, 0x3fc),
INTEL_HYBRID_STLAT_CONSTRAINT(0x2cd, 0x3),
- INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD(0x11d0, 0xf), /* MEM_INST_RETIRED.STLB_MISS_LOADS */
- INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST(0x12d0, 0xf), /* MEM_INST_RETIRED.STLB_MISS_STORES */
- INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD(0x21d0, 0xf), /* MEM_INST_RETIRED.LOCK_LOADS */
- INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD(0x41d0, 0xf), /* MEM_INST_RETIRED.SPLIT_LOADS */
- INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST(0x42d0, 0xf), /* MEM_INST_RETIRED.SPLIT_STORES */
- INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD(0x81d0, 0xf), /* MEM_INST_RETIRED.ALL_LOADS */
- INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST(0x82d0, 0xf), /* MEM_INST_RETIRED.ALL_STORES */
-
- INTEL_FLAGS_EVENT_CONSTRAINT_DATALA_LD_RANGE(0xd1, 0xd4, 0xf),
-
- INTEL_FLAGS_EVENT_CONSTRAINT(0xd0, 0xf),
/*
* Everything else is handled by PMU_FL_PEBS_ALL, because we
diff --git a/arch/x86/include/asm/sev-internal.h b/arch/x86/include/asm/sev-internal.h
deleted file mode 100644
index c58c47c68ab6..000000000000
--- a/arch/x86/include/asm/sev-internal.h
+++ /dev/null
@@ -1,87 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-
-#define DR7_RESET_VALUE 0x400
-
-extern u64 sev_hv_features;
-extern u64 sev_secrets_pa;
-
-/* #VC handler runtime per-CPU data */
-struct sev_es_runtime_data {
- struct ghcb ghcb_page;
-
- /*
- * Reserve one page per CPU as backup storage for the unencrypted GHCB.
- * It is needed when an NMI happens while the #VC handler uses the real
- * GHCB, and the NMI handler itself is causing another #VC exception. In
- * that case the GHCB content of the first handler needs to be backed up
- * and restored.
- */
- struct ghcb backup_ghcb;
-
- /*
- * Mark the per-cpu GHCBs as in-use to detect nested #VC exceptions.
- * There is no need for it to be atomic, because nothing is written to
- * the GHCB between the read and the write of ghcb_active. So it is safe
- * to use it when a nested #VC exception happens before the write.
- *
- * This is necessary for example in the #VC->NMI->#VC case when the NMI
- * happens while the first #VC handler uses the GHCB. When the NMI code
- * raises a second #VC handler it might overwrite the contents of the
- * GHCB written by the first handler. To avoid this the content of the
- * GHCB is saved and restored when the GHCB is detected to be in use
- * already.
- */
- bool ghcb_active;
- bool backup_ghcb_active;
-
- /*
- * Cached DR7 value - write it on DR7 writes and return it on reads.
- * That value will never make it to the real hardware DR7 as debugging
- * is currently unsupported in SEV-ES guests.
- */
- unsigned long dr7;
-};
-
-struct ghcb_state {
- struct ghcb *ghcb;
-};
-
-extern struct svsm_ca boot_svsm_ca_page;
-
-struct ghcb *__sev_get_ghcb(struct ghcb_state *state);
-void __sev_put_ghcb(struct ghcb_state *state);
-
-DECLARE_PER_CPU(struct sev_es_runtime_data*, runtime_data);
-DECLARE_PER_CPU(struct sev_es_save_area *, sev_vmsa);
-
-void early_set_pages_state(unsigned long vaddr, unsigned long paddr,
- unsigned long npages, const struct psc_desc *desc);
-
-DECLARE_PER_CPU(struct svsm_ca *, svsm_caa);
-DECLARE_PER_CPU(u64, svsm_caa_pa);
-
-extern u64 boot_svsm_caa_pa;
-
-enum es_result verify_exception_info(struct ghcb *ghcb, struct es_em_ctxt *ctxt);
-void vc_forward_exception(struct es_em_ctxt *ctxt);
-
-static inline u64 sev_es_rd_ghcb_msr(void)
-{
- return native_rdmsrq(MSR_AMD64_SEV_ES_GHCB);
-}
-
-static __always_inline void sev_es_wr_ghcb_msr(u64 val)
-{
- u32 low, high;
-
- low = (u32)(val);
- high = (u32)(val >> 32);
-
- native_wrmsr(MSR_AMD64_SEV_ES_GHCB, low, high);
-}
-
-enum es_result sev_es_ghcb_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt, bool write);
-
-u64 get_hv_features(void);
-
-const struct snp_cpuid_table *snp_cpuid_get_table(void);
diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c
index 877dd7d3bd75..2c87622dc501 100644
--- a/arch/x86/kernel/cpu/mce/core.c
+++ b/arch/x86/kernel/cpu/mce/core.c
@@ -2078,6 +2078,9 @@ bool filter_mce(struct mce *m)
static __always_inline void exc_machine_check_kernel(struct pt_regs *regs)
{
irqentry_state_t irq_state;
+ unsigned long dr7;
+
+ dr7 = local_db_save();
WARN_ON_ONCE(user_mode(regs));
@@ -2086,20 +2089,26 @@ static __always_inline void exc_machine_check_kernel(struct pt_regs *regs)
* mce_check_crashing_cpu() for details.
*/
if (mca_cfg.initialized && mce_check_crashing_cpu())
- return;
+ goto out;
irq_state = irqentry_nmi_enter(regs);
do_machine_check(regs);
irqentry_nmi_exit(regs, irq_state);
+out:
+ local_db_restore(dr7);
}
static __always_inline void exc_machine_check_user(struct pt_regs *regs)
{
+ unsigned long dr7;
+
irqentry_enter_from_user_mode(regs);
+ dr7 = local_db_save();
do_machine_check(regs);
+ local_db_restore(dr7);
irqentry_exit_to_user_mode(regs);
}
@@ -2108,21 +2117,13 @@ static __always_inline void exc_machine_check_user(struct pt_regs *regs)
/* MCE hit kernel mode */
DEFINE_IDTENTRY_MCE(exc_machine_check)
{
- unsigned long dr7;
-
- dr7 = local_db_save();
exc_machine_check_kernel(regs);
- local_db_restore(dr7);
}
/* The user mode variant. */
DEFINE_IDTENTRY_MCE_USER(exc_machine_check)
{
- unsigned long dr7;
-
- dr7 = local_db_save();
exc_machine_check_user(regs);
- local_db_restore(dr7);
}
#ifdef CONFIG_X86_FRED
@@ -2139,28 +2140,20 @@ DEFINE_IDTENTRY_MCE_USER(exc_machine_check)
*/
DEFINE_FREDENTRY_MCE(exc_machine_check)
{
- unsigned long dr7;
-
- dr7 = local_db_save();
if (user_mode(regs))
exc_machine_check_user(regs);
else
exc_machine_check_kernel(regs);
- local_db_restore(dr7);
}
#endif
#else
/* 32bit unified entry point */
DEFINE_IDTENTRY_RAW(exc_machine_check)
{
- unsigned long dr7;
-
- dr7 = local_db_save();
if (user_mode(regs))
exc_machine_check_user(regs);
else
exc_machine_check_kernel(regs);
- local_db_restore(dr7);
}
#endif
diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index 487ad19a236e..31b2b75156db 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -710,14 +710,6 @@ void kvm_pmu_deliver_pmi(struct kvm_vcpu *vcpu)
bool kvm_pmu_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)
{
- switch (msr) {
- case MSR_CORE_PERF_GLOBAL_STATUS:
- case MSR_CORE_PERF_GLOBAL_CTRL:
- case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
- return kvm_pmu_has_perf_global_ctrl(vcpu_to_pmu(vcpu));
- default:
- break;
- }
return kvm_pmu_call(msr_idx_to_pmc)(vcpu, msr) ||
kvm_pmu_call(is_valid_msr)(vcpu, msr);
}
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index c32c38520a34..3d2f0006a087 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -1986,6 +1986,12 @@ static void sev_migrate_from(struct kvm *dst_kvm, struct kvm *src_kvm)
src->pages_locked = 0;
src->es_active = false;
+ /*
+ * Do cache maintenance on the source VM as it is no longer an SEV VM,
+ * i.e. memory reclaim flows won't trigger cache maintenance on the VM.
+ */
+ sev_writeback_caches(src_kvm);
+
list_cut_before(&dst->regions_list, &src->regions_list, &src->regions_list);
mutex_lock(&sev_mirror_lock);
@@ -2125,6 +2131,10 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)
* the set of CPUs from the source. If a CPU was used to run a vCPU in
* the source VM but is never used for the destination VM, then the CPU
* can only have cached memory that was accessible to the source VM.
+ * Furthermore, KVM *must* perform cache maintenance on the source VM,
+ * as the source VM may have access to memory that the destination VM
+ * does not, i.e. KVM could skip flushes if memory is reclaimed from
+ * the old VM but not the new VM.
*/
if (!zalloc_cpumask_var(&dst_sev->have_run_cpus, GFP_KERNEL_ACCOUNT)) {
ret = -ENOMEM;
@@ -2920,13 +2930,17 @@ void sev_vm_destroy(struct kvm *kvm)
struct list_head *head = &sev->regions_list;
struct list_head *pos, *q;
+ /*
+ * Free the mask even if the VM is not *currently* an SEV VM, as it may
+ * have been an SEV VM prior to intra-host migration.
+ */
+ free_cpumask_var(sev->have_run_cpus);
+
if (!sev_guest(kvm))
return;
WARN_ON(!list_empty(&sev->mirror_vms));
- free_cpumask_var(sev->have_run_cpus);
-
/*
* If this is a mirror VM, remove it from the owner's list of a mirrors
* and skip ASID cleanup (the ASID is tied to the lifetime of the owner).
diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
index 333d7c573d40..a9d21a97c962 100644
--- a/arch/x86/kvm/vmx/pmu_intel.c
+++ b/arch/x86/kvm/vmx/pmu_intel.c
@@ -190,6 +190,9 @@ static bool intel_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)
int ret;
switch (msr) {
+ case MSR_CORE_PERF_GLOBAL_STATUS:
+ case MSR_CORE_PERF_GLOBAL_CTRL:
+ case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
case MSR_CORE_PERF_FIXED_CTR_CTRL:
return kvm_pmu_has_perf_global_ctrl(pmu);
case MSR_IA32_PEBS_ENABLE:
diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c
index 25076a5acd96..37f3b786143e 100644
--- a/arch/x86/pci/fixup.c
+++ b/arch/x86/pci/fixup.c
@@ -886,6 +886,105 @@ static void quirk_clear_strap_no_soft_reset_dev2_f0(struct pci_dev *dev)
}
}
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD, 0x15b8, quirk_clear_strap_no_soft_reset_dev2_f0);
+
+/*
+ * Enhanced atomic operations can cause corruption with 64-bit DMA
+ * on these devices.
+ */
+#define RX_ENH_ATOMIC_EN BIT(8)
+
+static const u32 nbio_7_7_pcie_smn_addrs[] = {
+ 0x111401d0,
+ 0x111411d0,
+ 0x111421d0,
+ 0x111431d0,
+ 0x111441d0,
+ 0x112401d0,
+ 0x112411d0,
+ 0x112421d0,
+ 0x112431d0,
+ 0x112441d0,
+ 0x112451d0,
+ 0x113401d0,
+ 0x114401d0,
+};
+
+static const u32 nbio_7_11_pcie_smn_addrs[] = {
+ 0x112401d0,
+ 0x112411d0,
+ 0x112421d0,
+ 0x112431d0,
+ 0x112441d0,
+ 0x112451d0,
+ 0x113401d0,
+ 0x113411d0,
+ 0x113421d0,
+ 0x113431d0,
+ 0x113441d0,
+ 0x113451d0,
+};
+
+static void quirk_amd_nbio_enhanced_atomic(struct pci_dev *host_bridge,
+ const u32 *smn_addrs,
+ size_t nr_smn_addrs)
+{
+ bool changed = false;
+ size_t i;
+ u32 data;
+ int ret;
+
+ for (i = 0; i < nr_smn_addrs; i++) {
+ ret = amd_smn_read(0, smn_addrs[i], &data);
+ if (ret)
+ continue;
+ if (!(data & RX_ENH_ATOMIC_EN))
+ continue;
+ data = data & ~RX_ENH_ATOMIC_EN;
+ ret = amd_smn_write(0, smn_addrs[i], data);
+ if (ret)
+ continue;
+ if (changed)
+ continue;
+ ret = amd_smn_read(0, smn_addrs[i], &data);
+ if (ret)
+ continue;
+ if (data & RX_ENH_ATOMIC_EN)
+ continue;
+ changed = true;
+ }
+
+ if (changed)
+ pci_info(host_bridge, "enhanced atomics disabled\n");
+}
+
+static void quirk_amd_nbio_7_7_disable_enhanced_atomic(struct pci_dev *dev)
+{
+ quirk_amd_nbio_enhanced_atomic(dev, nbio_7_7_pcie_smn_addrs,
+ ARRAY_SIZE(nbio_7_7_pcie_smn_addrs));
+}
+
+static void quirk_amd_nbio_7_11_disable_enhanced_atomic(struct pci_dev *dev)
+{
+ quirk_amd_nbio_enhanced_atomic(dev, nbio_7_11_pcie_smn_addrs,
+ ARRAY_SIZE(nbio_7_11_pcie_smn_addrs));
+}
+
+/* Phoenix, Hawk Point (NBIO 7.7) */
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD, 0x14E8,
+ quirk_amd_nbio_7_7_disable_enhanced_atomic);
+DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x14E8,
+ quirk_amd_nbio_7_7_disable_enhanced_atomic);
+
+/* Strix, Krackan, Strix Halo (NBIO 7.11) */
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD, 0x1507,
+ quirk_amd_nbio_7_11_disable_enhanced_atomic);
+DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1507,
+ quirk_amd_nbio_7_11_disable_enhanced_atomic);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD, 0x1122,
+ quirk_amd_nbio_7_11_disable_enhanced_atomic);
+DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1122,
+ quirk_amd_nbio_7_11_disable_enhanced_atomic);
+
#endif
/*
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 8df501fa8def..14e68ca9784b 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -261,12 +261,18 @@ static void ata_scsi_set_passthru_sense_fields(struct ata_queued_cmd *qc)
/* descriptor format */
len = sb[7];
- desc = (char *)scsi_sense_desc_find(sb, len + 8, 9);
+ desc = (char *)scsi_sense_desc_find(sb, SCSI_SENSE_BUFFERSIZE, 9);
if (!desc) {
- if (SCSI_SENSE_BUFFERSIZE < len + 14)
+ /*
+ * The descriptor is written at sb[8 + len] and is 14
+ * bytes long, so it needs len + 22 bytes of buffer.
+ */
+ if (len + 22 > SCSI_SENSE_BUFFERSIZE)
return;
sb[7] = len + 14;
desc = sb + 8 + len;
+ } else if (desc - sb > SCSI_SENSE_BUFFERSIZE - 14) {
+ return;
}
desc[0] = 9;
desc[1] = 12;
diff --git a/drivers/bluetooth/btintel_pcie.c b/drivers/bluetooth/btintel_pcie.c
index 8d3129aa2a1a..3d1ed8568877 100644
--- a/drivers/bluetooth/btintel_pcie.c
+++ b/drivers/bluetooth/btintel_pcie.c
@@ -1018,6 +1018,11 @@ static void btintel_pcie_msix_tx_handle(struct btintel_pcie_data *data)
txq = &data->txq;
+ if (cr_hia >= txq->count) {
+ bt_dev_err(data->hdev, "TXQ: invalid cr_hia %u", cr_hia);
+ return;
+ }
+
while (cr_tia != cr_hia) {
data->tx_wait_done = true;
wake_up(&data->tx_wait_q);
@@ -1416,6 +1421,11 @@ static void btintel_pcie_msix_rx_handle(struct btintel_pcie_data *data)
rxq = &data->rxq;
+ if (cr_hia >= rxq->count) {
+ bt_dev_err(hdev, "RXQ: invalid cr_hia %u", cr_hia);
+ return;
+ }
+
/* The firmware sends multiple CD in a single MSI-X and it needs to
* process all received CDs in this interrupt.
*/
@@ -1423,6 +1433,12 @@ static void btintel_pcie_msix_rx_handle(struct btintel_pcie_data *data)
urbd1 = &rxq->urbd1s[cr_tia];
ipc_print_urbd1(data->hdev, urbd1, cr_tia);
+ if (urbd1->frbd_tag >= rxq->count) {
+ bt_dev_err(hdev, "RXQ: invalid frbd_tag %u",
+ urbd1->frbd_tag);
+ return;
+ }
+
buf = &rxq->bufs[urbd1->frbd_tag];
if (!buf) {
bt_dev_err(hdev, "RXQ: failed to get the DMA buffer for %d",
diff --git a/drivers/bluetooth/btnxpuart.c b/drivers/bluetooth/btnxpuart.c
index 509d837e353e..c7acf6c7efcf 100644
--- a/drivers/bluetooth/btnxpuart.c
+++ b/drivers/bluetooth/btnxpuart.c
@@ -1397,9 +1397,11 @@ static int nxp_process_fw_dump(struct hci_dev *hdev, struct sk_buff *skb)
msecs_to_jiffies(20000));
}
- err = hci_devcd_append(hdev, skb_clone(skb, GFP_ATOMIC));
- if (err < 0)
- goto free_skb;
+ if (IS_ENABLED(CONFIG_DEV_COREDUMP)) {
+ err = hci_devcd_append(hdev, skb_clone(skb, GFP_ATOMIC));
+ if (err < 0)
+ goto free_skb;
+ }
if (buf_len == 0) {
bt_dev_warn(hdev, "==== FW dump complete ===");
diff --git a/drivers/dpll/dpll_netlink.c b/drivers/dpll/dpll_netlink.c
index eb23ee401aae..d8917150477e 100644
--- a/drivers/dpll/dpll_netlink.c
+++ b/drivers/dpll/dpll_netlink.c
@@ -1059,8 +1059,7 @@ dpll_pin_ref_sync_state_set(struct dpll_pin *pin,
unsigned long i;
int ret;
- ref_sync_pin = xa_find(&pin->ref_sync_pins, &ref_sync_pin_idx,
- ULONG_MAX, XA_PRESENT);
+ ref_sync_pin = xa_load(&pin->ref_sync_pins, ref_sync_pin_idx);
if (!ref_sync_pin) {
NL_SET_ERR_MSG(extack, "reference sync pin not found");
return -EINVAL;
diff --git a/drivers/gpio/gpio-arizona.c b/drivers/gpio/gpio-arizona.c
index a7e98d395d8e..88cf013f0d90 100644
--- a/drivers/gpio/gpio-arizona.c
+++ b/drivers/gpio/gpio-arizona.c
@@ -115,8 +115,12 @@ static int arizona_gpio_direction_out(struct gpio_chip *chip,
if (value)
value = ARIZONA_GPN_LVL;
- return regmap_update_bits(arizona->regmap, ARIZONA_GPIO1_CTRL + offset,
- ARIZONA_GPN_DIR | ARIZONA_GPN_LVL, value);
+ ret = regmap_update_bits(arizona->regmap, ARIZONA_GPIO1_CTRL + offset,
+ ARIZONA_GPN_DIR | ARIZONA_GPN_LVL, value);
+ if (ret < 0 && (val & ARIZONA_GPN_DIR) && persistent)
+ pm_runtime_put_autosuspend(chip->parent);
+
+ return ret;
}
static int arizona_gpio_set(struct gpio_chip *chip, unsigned int offset,
diff --git a/drivers/gpio/gpio-tps65219.c b/drivers/gpio/gpio-tps65219.c
index 158f63bcf10c..8e00c3bc1492 100644
--- a/drivers/gpio/gpio-tps65219.c
+++ b/drivers/gpio/gpio-tps65219.c
@@ -79,7 +79,7 @@ static int tps65219_gpio_get(struct gpio_chip *gc, unsigned int offset)
if (ret)
return ret;
- ret = !!(val & BIT(TPS65219_MFP_GPIO_STATUS_MASK));
+ ret = !!(val & TPS65219_MFP_GPIO_STATUS_MASK);
dev_warn(dev, "GPIO%d = %d, MULTI_DEVICE_ENABLE, not a standard GPIO\n", offset, ret);
/*
@@ -87,7 +87,7 @@ static int tps65219_gpio_get(struct gpio_chip *gc, unsigned int offset)
* status bit.
*/
- if (tps65219_gpio_get_direction(gc, offset) == GPIO_LINE_DIRECTION_OUT)
+ if (gc->get_direction(gc, offset) == GPIO_LINE_DIRECTION_OUT)
return -ENOTSUPP;
return ret;
@@ -158,8 +158,10 @@ static int tps65214_gpio_change_direction(struct gpio_chip *gc, unsigned int off
if (ret)
dev_err(dev, "GPIO%d configured as VSEL, not GPIO\n", offset);
+ val = direction == GPIO_LINE_DIRECTION_OUT ?
+ TPS65214_GPIO0_DIR_MASK : 0;
ret = regmap_update_bits(gpio->tps->regmap, TPS65219_REG_GENERAL_CONFIG,
- TPS65214_GPIO0_DIR_MASK, direction);
+ TPS65214_GPIO0_DIR_MASK, val);
if (ret)
dev_err(dev, "Fail to change direction to %u for GPIO%d.\n", direction, offset);
@@ -176,7 +178,7 @@ static int tps65219_gpio_direction_input(struct gpio_chip *gc, unsigned int offs
return -ENOTSUPP;
}
- if (tps65219_gpio_get_direction(gc, offset) == GPIO_LINE_DIRECTION_IN)
+ if (gc->get_direction(gc, offset) == GPIO_LINE_DIRECTION_IN)
return 0;
return gpio->change_dir(gc, offset, GPIO_LINE_DIRECTION_IN);
@@ -190,7 +192,7 @@ static int tps65219_gpio_direction_output(struct gpio_chip *gc, unsigned int off
if (offset != TPS6521X_GPIO0_IDX)
return 0;
- if (tps65219_gpio_get_direction(gc, offset) == GPIO_LINE_DIRECTION_OUT)
+ if (gc->get_direction(gc, offset) == GPIO_LINE_DIRECTION_OUT)
return 0;
return gpio->change_dir(gc, offset, GPIO_LINE_DIRECTION_OUT);
diff --git a/drivers/gpio/gpio-zynq.c b/drivers/gpio/gpio-zynq.c
index c2e4a79cd6d8..32a9dadb9aad 100644
--- a/drivers/gpio/gpio-zynq.c
+++ b/drivers/gpio/gpio-zynq.c
@@ -797,15 +797,7 @@ static int __maybe_unused zynq_gpio_runtime_resume(struct device *dev)
static int zynq_gpio_request(struct gpio_chip *chip, unsigned int offset)
{
- int ret;
-
- ret = pm_runtime_get_sync(chip->parent);
-
- /*
- * If the device is already active pm_runtime_get() will return 1 on
- * success, but gpio_request still needs to return 0.
- */
- return ret < 0 ? ret : 0;
+ return pm_runtime_resume_and_get(chip->parent);
}
static void zynq_gpio_free(struct gpio_chip *chip, unsigned int offset)
diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
index 89abf7a238d8..d67d361cc456 100644
--- a/drivers/gpio/gpiolib-cdev.c
+++ b/drivers/gpio/gpiolib-cdev.c
@@ -2245,18 +2245,19 @@ static void gpio_v2_line_info_changed_to_v1(
#endif /* CONFIG_GPIO_CDEV_V1 */
-static void gpio_desc_to_lineinfo(struct gpio_desc *desc,
- struct gpio_v2_line_info *info, bool atomic)
+static int gpio_desc_to_lineinfo(struct gpio_desc *desc,
+ struct gpio_v2_line_info *info, bool atomic)
{
u32 debounce_period_us;
unsigned long dflags;
const char *label;
+ memset(info, 0, sizeof(*info));
+
CLASS(gpio_chip_guard, guard)(desc);
if (!guard.gc)
- return;
+ return -ENODEV;
- memset(info, 0, sizeof(*info));
info->offset = gpiod_hwgpio(desc);
if (desc->name)
@@ -2331,6 +2332,8 @@ static void gpio_desc_to_lineinfo(struct gpio_desc *desc,
debounce_period_us;
info->num_attrs++;
}
+
+ return 0;
}
struct gpio_chardev_data {
@@ -2382,6 +2385,7 @@ static int lineinfo_get_v1(struct gpio_chardev_data *cdev, void __user *ip,
struct gpio_desc *desc;
struct gpioline_info lineinfo;
struct gpio_v2_line_info lineinfo_v2;
+ int ret;
if (copy_from_user(&lineinfo, ip, sizeof(lineinfo)))
return -EFAULT;
@@ -2399,7 +2403,10 @@ static int lineinfo_get_v1(struct gpio_chardev_data *cdev, void __user *ip,
return -EBUSY;
}
- gpio_desc_to_lineinfo(desc, &lineinfo_v2, false);
+ ret = gpio_desc_to_lineinfo(desc, &lineinfo_v2, false);
+ if (ret)
+ return ret;
+
gpio_v2_line_info_to_v1(&lineinfo_v2, &lineinfo);
if (copy_to_user(ip, &lineinfo, sizeof(lineinfo))) {
@@ -2417,6 +2424,7 @@ static int lineinfo_get(struct gpio_chardev_data *cdev, void __user *ip,
{
struct gpio_desc *desc;
struct gpio_v2_line_info lineinfo;
+ int ret;
if (copy_from_user(&lineinfo, ip, sizeof(lineinfo)))
return -EFAULT;
@@ -2436,7 +2444,10 @@ static int lineinfo_get(struct gpio_chardev_data *cdev, void __user *ip,
if (test_and_set_bit(lineinfo.offset, cdev->watched_lines))
return -EBUSY;
}
- gpio_desc_to_lineinfo(desc, &lineinfo, false);
+
+ ret = gpio_desc_to_lineinfo(desc, &lineinfo, false);
+ if (ret)
+ return ret;
if (copy_to_user(ip, &lineinfo, sizeof(lineinfo))) {
if (watch)
@@ -2562,6 +2573,7 @@ static int lineinfo_changed_notify(struct notifier_block *nb,
struct lineinfo_changed_ctx *ctx;
struct gpio_desc *desc = data;
struct file *fp;
+ int ret;
if (!test_bit(gpiod_hwgpio(desc), cdev->watched_lines))
return NOTIFY_DONE;
@@ -2592,7 +2604,13 @@ static int lineinfo_changed_notify(struct notifier_block *nb,
ctx->chg.event_type = action;
ctx->chg.timestamp_ns = ktime_get_ns();
- gpio_desc_to_lineinfo(desc, &ctx->chg.info, true);
+
+ ret = gpio_desc_to_lineinfo(desc, &ctx->chg.info, true);
+ if (ret) {
+ fput(fp);
+ return NOTIFY_DONE;
+ }
+
/* Keep the GPIO device alive until we emit the event. */
ctx->gdev = gpio_device_get(desc->gdev);
ctx->cdev = cdev;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
index 67db986eda3f..36f429e7f202 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
@@ -1118,6 +1118,7 @@ static int amdgpu_acpi_enumerate_xcc(void)
GFP_KERNEL);
if (!xcc_info) {
DRM_ERROR("Failed to allocate memory for xcc info\n");
+ acpi_dev_put(acpi_dev);
return -ENOMEM;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
index d6f58128bc74..3b3f404295b4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
@@ -1671,8 +1671,10 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused)
/* Avoid accidently unparking the sched thread during GPU reset */
r = down_write_killable(&adev->reset_domain->sem);
- if (r)
+ if (r) {
+ pm_runtime_put_autosuspend(dev->dev);
return r;
+ }
/* hold on the scheduler */
for (i = 0; i < AMDGPU_MAX_RINGS; i++) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 6377501c70a8..b47989692bba 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -5315,7 +5315,7 @@ int amdgpu_device_suspend(struct drm_device *dev, bool notify_clients)
dev_warn(adev->dev, "smart shift update failed\n");
if (notify_clients)
- drm_client_dev_suspend(adev_to_drm(adev), false);
+ drm_client_dev_suspend(adev_to_drm(adev));
cancel_delayed_work_sync(&adev->delayed_init_work);
@@ -5445,7 +5445,7 @@ int amdgpu_device_resume(struct drm_device *dev, bool notify_clients)
flush_delayed_work(&adev->delayed_init_work);
if (notify_clients)
- drm_client_dev_resume(adev_to_drm(adev), false);
+ drm_client_dev_resume(adev_to_drm(adev));
amdgpu_ras_resume(adev);
@@ -6060,7 +6060,7 @@ int amdgpu_device_reinit_after_reset(struct amdgpu_reset_context *reset_context)
if (r)
goto out;
- drm_client_dev_resume(adev_to_drm(tmp_adev), false);
+ drm_client_dev_resume(adev_to_drm(tmp_adev));
/*
* The GPU enters bad state once faulty pages
@@ -6395,7 +6395,7 @@ static void amdgpu_device_halt_activities(struct amdgpu_device *adev,
*/
amdgpu_unregister_gpu_instance(tmp_adev);
- drm_client_dev_suspend(adev_to_drm(tmp_adev), false);
+ drm_client_dev_suspend(adev_to_drm(tmp_adev));
/* disable ras on ALL IPs */
if (!need_emergency_restart && !amdgpu_reset_in_dpc(adev) &&
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
index 431cc39ea017..79313f1f243b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
@@ -260,7 +260,6 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,
ring->adev = adev;
ring->num_hw_submission = sched_hw_submission;
ring->sched_score = sched_score;
- ring->vmid_wait = dma_fence_get_stub();
ring->idx = adev->num_rings++;
adev->rings[ring->idx] = ring;
@@ -380,6 +379,7 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,
ring->max_dw = max_dw;
ring->hw_prio = hw_prio;
+ ring->vmid_wait = dma_fence_get_stub();
if (!ring->no_scheduler && ring->funcs->type < AMDGPU_HW_IP_NUM) {
hw_ip = ring->funcs->type;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 6aae94ca7dcb..f78ac6777ee5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -2673,6 +2673,7 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm,
amdgpu_bo_unref(&root_bo);
error_free_delayed:
+ dma_fence_put(vm->last_update);
dma_fence_put(vm->last_tlb_flush);
dma_fence_put(vm->last_unlocked);
ttm_lru_bulk_move_fini(&adev->mman.bdev, &vm->lru_bulk_move);
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index 89686abcaf92..c2479e2e964b 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -3100,7 +3100,7 @@ static void dm_gpureset_toggle_interrupts(struct amdgpu_device *adev,
if (acrtc && state->stream_status[i].plane_count != 0 &&
amdgpu_ip_version(adev, DCE_HWIP, 0) == 0) {
irq_source = IRQ_TYPE_PFLIP + acrtc->otg_inst;
- rc = dc_interrupt_set(adev->dm.dc, irq_source, enable) ? 0 : -EBUSY;
+ rc = amdgpu_dm_irq_set(adev, irq_source, enable) ? 0 : -EBUSY;
if (rc)
drm_warn(adev_to_drm(adev), "Failed to %s pflip interrupts\n",
enable ? "enable" : "disable");
@@ -3124,7 +3124,7 @@ static void dm_gpureset_toggle_interrupts(struct amdgpu_device *adev,
/* During gpu-reset we disable and then enable vblank irq, so
* don't use amdgpu_irq_get/put() to avoid refcount change.
*/
- if (!dc_interrupt_set(adev->dm.dc, irq_source, enable))
+ if (!amdgpu_dm_irq_set(adev, irq_source, enable))
drm_warn(adev_to_drm(adev), "Failed to %sable vblank interrupt\n", enable ? "en" : "dis");
} else if (acrtc && state->stream_status[i].plane_count != 0) {
@@ -11639,8 +11639,10 @@ static int dm_update_crtc_state(struct amdgpu_display_manager *dm,
skip_modeset:
/* Release extra reference */
- if (new_stream)
+ if (new_stream) {
dc_stream_release(new_stream);
+ new_stream = NULL;
+ }
new_stream = NULL;
/*
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
index caf6f7387a20..a83fdd7b644a 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
@@ -527,6 +527,18 @@ struct amdgpu_display_manager {
struct common_irq_params
vupdate_params[DC_IRQ_SOURCE_VUPDATE6 - DC_IRQ_SOURCE_VUPDATE1 + 1];
+ /**
+ * @irq_reg_lock:
+ *
+ * Serializes the read-modify-writes of the HW interrupt control
+ * registers. Several interrupt sources share one register - e.g. the
+ * enable and clear bits of both VSTARTUP (vblank) and VUPDATE_NO_LOCK
+ * live in OTG_GLOBAL_SYNC_STATUS. Therefore, enabling one source must
+ * not race with acking another. Held only across amdgpu_dm_irq_set()
+ * and amdgpu_dm_irq_ack().
+ */
+ spinlock_t irq_reg_lock;
+
/**
* @dmub_trace_params:
*
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c
index 788068303152..9082b862e7aa 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c
@@ -31,6 +31,7 @@
#include "amdgpu_dm_psr.h"
#include "amdgpu_dm_replay.h"
#include "amdgpu_dm_crtc.h"
+#include "amdgpu_dm_irq.h"
#include "amdgpu_dm_plane.h"
#include "amdgpu_dm_trace.h"
#include "amdgpu_dm_debugfs.h"
@@ -86,7 +87,7 @@ int amdgpu_dm_crtc_set_vupdate_irq(struct drm_crtc *crtc, bool enable)
irq_source = IRQ_TYPE_VUPDATE + acrtc->otg_inst;
- rc = dc_interrupt_set(adev->dm.dc, irq_source, enable) ? 0 : -EBUSY;
+ rc = amdgpu_dm_irq_set(adev, irq_source, enable) ? 0 : -EBUSY;
DRM_DEBUG_VBL("crtc %d - vupdate irq %sabling: r=%d\n",
acrtc->crtc_id, enable ? "en" : "dis", rc);
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
index ddda474d5878..7633af242919 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
@@ -1189,12 +1189,13 @@ void dm_helpers_free_gpu_mem(
bool dm_helpers_dmub_outbox_interrupt_control(struct dc_context *ctx, bool enable)
{
+ struct amdgpu_device *adev = ctx->driver_context;
enum dc_irq_source irq_source;
bool ret;
irq_source = DC_IRQ_SOURCE_DMCUB_OUTBOX;
- ret = dc_interrupt_set(ctx->dc, irq_source, enable);
+ ret = amdgpu_dm_irq_set(adev, irq_source, enable);
DRM_DEBUG_DRIVER("Dmub trace irq %sabling: r=%d\n",
enable ? "en" : "dis", ret);
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c
index a1c722112c22..59c7efc60c90 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c
@@ -424,6 +424,7 @@ int amdgpu_dm_irq_init(struct amdgpu_device *adev)
DRM_DEBUG_KMS("DM_IRQ\n");
spin_lock_init(&adev->dm.irq_handler_list_table_lock);
+ spin_lock_init(&adev->dm.irq_reg_lock);
for (src = 0; src < DAL_IRQ_SOURCES_NUMBER; src++) {
/* low context handler list init */
@@ -495,7 +496,7 @@ void amdgpu_dm_irq_suspend(struct amdgpu_device *adev)
hnd_list_l = &adev->dm.irq_handler_list_low_tab[src];
hnd_list_h = &adev->dm.irq_handler_list_high_tab[src];
if (!list_empty(hnd_list_l) || !list_empty(hnd_list_h))
- dc_interrupt_set(adev->dm.dc, src, false);
+ amdgpu_dm_irq_set(adev, src, false);
DM_IRQ_TABLE_UNLOCK(adev, irq_table_flags);
@@ -529,7 +530,7 @@ void amdgpu_dm_irq_resume_early(struct amdgpu_device *adev)
hnd_list_l = &adev->dm.irq_handler_list_low_tab[src];
hnd_list_h = &adev->dm.irq_handler_list_high_tab[src];
if (!list_empty(hnd_list_l) || !list_empty(hnd_list_h))
- dc_interrupt_set(adev->dm.dc, src, true);
+ amdgpu_dm_irq_set(adev, src, true);
}
DM_IRQ_TABLE_UNLOCK(adev, irq_table_flags);
@@ -553,7 +554,7 @@ void amdgpu_dm_irq_resume_late(struct amdgpu_device *adev)
hnd_list_l = &adev->dm.irq_handler_list_low_tab[src];
hnd_list_h = &adev->dm.irq_handler_list_high_tab[src];
if (!list_empty(hnd_list_l) || !list_empty(hnd_list_h))
- dc_interrupt_set(adev->dm.dc, src, true);
+ amdgpu_dm_irq_set(adev, src, true);
}
DM_IRQ_TABLE_UNLOCK(adev, irq_table_flags);
@@ -637,6 +638,21 @@ static void amdgpu_dm_irq_immediate_work(struct amdgpu_device *adev,
DM_IRQ_TABLE_UNLOCK(adev, irq_table_flags);
}
+bool amdgpu_dm_irq_set(struct amdgpu_device *adev, enum dc_irq_source src,
+ bool enable)
+{
+ guard(spinlock_irqsave)(&adev->dm.irq_reg_lock);
+
+ return dc_interrupt_set(adev->dm.dc, src, enable);
+}
+
+void amdgpu_dm_irq_ack(struct amdgpu_device *adev, enum dc_irq_source src)
+{
+ guard(spinlock_irqsave)(&adev->dm.irq_reg_lock);
+
+ dc_interrupt_ack(adev->dm.dc, src);
+}
+
/**
* amdgpu_dm_irq_handler - Generic DM IRQ handler
* @adev: amdgpu base driver device containing the DM device
@@ -657,7 +673,7 @@ static int amdgpu_dm_irq_handler(struct amdgpu_device *adev,
entry->src_id,
entry->src_data[0]);
- dc_interrupt_ack(adev->dm.dc, src);
+ amdgpu_dm_irq_ack(adev, src);
/* Call high irq work immediately */
amdgpu_dm_irq_immediate_work(adev, src);
@@ -695,7 +711,7 @@ static int amdgpu_dm_set_hpd_irq_state(struct amdgpu_device *adev,
enum dc_irq_source src = amdgpu_dm_hpd_to_dal_irq_source(type);
bool st = (state == AMDGPU_IRQ_STATE_ENABLE);
- dc_interrupt_set(adev->dm.dc, src, st);
+ amdgpu_dm_irq_set(adev, src, st);
return 0;
}
@@ -729,7 +745,7 @@ static inline int dm_irq_state(struct amdgpu_device *adev,
if (dc && dc->caps.ips_support && dc->idle_optimizations_allowed)
dc_allow_idle_optimizations(dc, false);
- dc_interrupt_set(adev->dm.dc, irq_source, st);
+ amdgpu_dm_irq_set(adev, irq_source, st);
return 0;
}
@@ -783,7 +799,7 @@ static int amdgpu_dm_set_dmub_outbox_irq_state(struct amdgpu_device *adev,
enum dc_irq_source irq_source = DC_IRQ_SOURCE_DMCUB_OUTBOX;
bool st = (state == AMDGPU_IRQ_STATE_ENABLE);
- dc_interrupt_set(adev->dm.dc, irq_source, st);
+ amdgpu_dm_irq_set(adev, irq_source, st);
return 0;
}
@@ -809,7 +825,7 @@ static int amdgpu_dm_set_dmub_trace_irq_state(struct amdgpu_device *adev,
enum dc_irq_source irq_source = DC_IRQ_SOURCE_DMCUB_OUTBOX0;
bool st = (state == AMDGPU_IRQ_STATE_ENABLE);
- dc_interrupt_set(adev->dm.dc, irq_source, st);
+ amdgpu_dm_irq_set(adev, irq_source, st);
return 0;
}
@@ -873,9 +889,7 @@ void amdgpu_dm_set_irq_funcs(struct amdgpu_device *adev)
}
void amdgpu_dm_outbox_init(struct amdgpu_device *adev)
{
- dc_interrupt_set(adev->dm.dc,
- DC_IRQ_SOURCE_DMCUB_OUTBOX,
- true);
+ amdgpu_dm_irq_set(adev, DC_IRQ_SOURCE_DMCUB_OUTBOX, true);
}
/**
@@ -896,7 +910,7 @@ void amdgpu_dm_hpd_init(struct amdgpu_device *adev)
/* First, clear all hpd and hpdrx interrupts */
for (i = DC_IRQ_SOURCE_HPD1; i <= DC_IRQ_SOURCE_HPD6RX; i++) {
- if (!dc_interrupt_set(adev->dm.dc, i, false))
+ if (!amdgpu_dm_irq_set(adev, i, false))
drm_err(dev, "Failed to clear hpd(rx) source=%d on init\n",
i);
}
@@ -918,7 +932,7 @@ void amdgpu_dm_hpd_init(struct amdgpu_device *adev)
* of dm. Note that only hpd interrupt types are registered with
* base driver; hpd_rx types aren't. IOW, amdgpu_irq_get/put on
* hpd_rx isn't available. DM currently controls hpd_rx
- * explicitly with dc_interrupt_set()
+ * explicitly with amdgpu_dm_irq_set()
*/
if (dc_link->irq_source_hpd != DC_IRQ_SOURCE_INVALID) {
irq_type = dc_link->irq_source_hpd - DC_IRQ_SOURCE_HPD1;
@@ -927,23 +941,21 @@ void amdgpu_dm_hpd_init(struct amdgpu_device *adev)
* and what bios reports as the # of connectors with hpd
* sources. Since the # of hpd source types registered
* with base driver == mode_info.num_hpd, we have to
- * fallback to dc_interrupt_set for the remaining types.
+ * fallback to amdgpu_dm_irq_set for the remaining types.
*/
if (irq_type < adev->mode_info.num_hpd) {
if (amdgpu_irq_get(adev, &adev->hpd_irq, irq_type))
drm_err(dev, "DM_IRQ: Failed get HPD for source=%d)!\n",
dc_link->irq_source_hpd);
} else {
- dc_interrupt_set(adev->dm.dc,
- dc_link->irq_source_hpd,
- true);
+ amdgpu_dm_irq_set(adev, dc_link->irq_source_hpd,
+ true);
}
}
if (dc_link->irq_source_hpd_rx != DC_IRQ_SOURCE_INVALID) {
- dc_interrupt_set(adev->dm.dc,
- dc_link->irq_source_hpd_rx,
- true);
+ amdgpu_dm_irq_set(adev, dc_link->irq_source_hpd_rx,
+ true);
}
}
drm_connector_list_iter_end(&iter);
@@ -984,16 +996,14 @@ void amdgpu_dm_hpd_fini(struct amdgpu_device *adev)
drm_err(dev, "DM_IRQ: Failed put HPD for source=%d!\n",
dc_link->irq_source_hpd);
} else {
- dc_interrupt_set(adev->dm.dc,
- dc_link->irq_source_hpd,
- false);
+ amdgpu_dm_irq_set(adev, dc_link->irq_source_hpd,
+ false);
}
}
if (dc_link->irq_source_hpd_rx != DC_IRQ_SOURCE_INVALID) {
- dc_interrupt_set(adev->dm.dc,
- dc_link->irq_source_hpd_rx,
- false);
+ amdgpu_dm_irq_set(adev, dc_link->irq_source_hpd_rx,
+ false);
}
}
drm_connector_list_iter_end(&iter);
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.h
index 4f6b58f4f90d..f0a4577848f3 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.h
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.h
@@ -81,6 +81,34 @@ void amdgpu_dm_irq_unregister_interrupt(struct amdgpu_device *adev,
enum dc_irq_source irq_source,
void *ih_index);
+/**
+ * amdgpu_dm_irq_set - enable or disable a DC interrupt source.
+ *
+ * @adev: AMD DRM device
+ * @src: DC interrupt source to toggle
+ * @enable: true to enable the source, false to disable it
+ *
+ * DM-wide replacement for dc_interrupt_set(). As locking is DM's
+ * responsibility, this is a thin wrapper serializes the underlying
+ * read-modify-write against the other interrupt sources sharing HW control
+ * registers with @src, so DM must never call dc_interrupt_set() directly.
+ *
+ * Returns: true if the source was toggled.
+ */
+bool amdgpu_dm_irq_set(struct amdgpu_device *adev, enum dc_irq_source src,
+ bool enable);
+
+/**
+ * amdgpu_dm_irq_ack - acknowledge a DC interrupt source.
+ *
+ * @adev: AMD DRM device
+ * @src: DC interrupt source to acknowledge
+ *
+ * DM-wide replacement for dc_interrupt_ack(), serialized the same way as
+ * amdgpu_dm_irq_set().
+ */
+void amdgpu_dm_irq_ack(struct amdgpu_device *adev, enum dc_irq_source src);
+
void amdgpu_dm_set_irq_funcs(struct amdgpu_device *adev);
void amdgpu_dm_outbox_init(struct amdgpu_device *adev);
diff --git a/drivers/gpu/drm/amd/display/dc/dml/Makefile b/drivers/gpu/drm/amd/display/dc/dml/Makefile
index 268b5fbdb48b..1b5da6fc345c 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/Makefile
+++ b/drivers/gpu/drm/amd/display/dc/dml/Makefile
@@ -36,7 +36,11 @@ ifneq ($(CONFIG_FRAME_WARN),0)
frame_warn_limit := 3072
endif
else
- frame_warn_limit := 2048
+ ifeq ($(CONFIG_CC_IS_CLANG),y)
+ frame_warn_limit := 3072
+ else
+ frame_warn_limit := 2048
+ endif
endif
ifeq ($(call test-lt, $(CONFIG_FRAME_WARN), $(frame_warn_limit)),y)
diff --git a/drivers/gpu/drm/bridge/samsung-dsim.c b/drivers/gpu/drm/bridge/samsung-dsim.c
index ad8c6aa49d48..272e6255c732 100644
--- a/drivers/gpu/drm/bridge/samsung-dsim.c
+++ b/drivers/gpu/drm/bridge/samsung-dsim.c
@@ -1862,7 +1862,7 @@ static int samsung_dsim_register_te_irq(struct samsung_dsim *dsi, struct device
int te_gpio_irq;
int ret;
- dsi->te_gpio = devm_gpiod_get_optional(dev, "te", GPIOD_IN);
+ dsi->te_gpio = gpiod_get_optional(dev, "te", GPIOD_IN);
if (!dsi->te_gpio)
return 0;
else if (IS_ERR(dsi->te_gpio))
diff --git a/drivers/gpu/drm/clients/drm_fbdev_client.c b/drivers/gpu/drm/clients/drm_fbdev_client.c
index f894ba52bdb5..935cd8503222 100644
--- a/drivers/gpu/drm/clients/drm_fbdev_client.c
+++ b/drivers/gpu/drm/clients/drm_fbdev_client.c
@@ -26,9 +26,19 @@ static void drm_fbdev_client_unregister(struct drm_client_dev *client)
}
}
-static int drm_fbdev_client_restore(struct drm_client_dev *client)
+static int drm_fbdev_client_restore(struct drm_client_dev *client, bool force)
{
- drm_fb_helper_lastclose(client->dev);
+ struct drm_fb_helper *fb_helper = drm_fb_helper_from_client(client);
+
+ /*
+ * The client is registered before the initial fbdev probe.
+ * If probing failed, the client remains registered but there
+ * is no valid fbdev framebuffer to restore.
+ */
+ if (!fb_helper->info || !fb_helper->fb)
+ return 0;
+
+ drm_fb_helper_restore_fbdev_mode_unlocked(fb_helper, force);
return 0;
}
@@ -62,26 +72,20 @@ static int drm_fbdev_client_hotplug(struct drm_client_dev *client)
return ret;
}
-static int drm_fbdev_client_suspend(struct drm_client_dev *client, bool holds_console_lock)
+static int drm_fbdev_client_suspend(struct drm_client_dev *client)
{
struct drm_fb_helper *fb_helper = drm_fb_helper_from_client(client);
- if (holds_console_lock)
- drm_fb_helper_set_suspend(fb_helper, true);
- else
- drm_fb_helper_set_suspend_unlocked(fb_helper, true);
+ drm_fb_helper_set_suspend_unlocked(fb_helper, true);
return 0;
}
-static int drm_fbdev_client_resume(struct drm_client_dev *client, bool holds_console_lock)
+static int drm_fbdev_client_resume(struct drm_client_dev *client)
{
struct drm_fb_helper *fb_helper = drm_fb_helper_from_client(client);
- if (holds_console_lock)
- drm_fb_helper_set_suspend(fb_helper, false);
- else
- drm_fb_helper_set_suspend_unlocked(fb_helper, false);
+ drm_fb_helper_set_suspend_unlocked(fb_helper, false);
return 0;
}
diff --git a/drivers/gpu/drm/clients/drm_log.c b/drivers/gpu/drm/clients/drm_log.c
index 1e137786b36a..36cc569fd75d 100644
--- a/drivers/gpu/drm/clients/drm_log.c
+++ b/drivers/gpu/drm/clients/drm_log.c
@@ -328,7 +328,7 @@ static int drm_log_client_hotplug(struct drm_client_dev *client)
return 0;
}
-static int drm_log_client_suspend(struct drm_client_dev *client, bool _console_lock)
+static int drm_log_client_suspend(struct drm_client_dev *client)
{
struct drm_log *dlog = client_to_drm_log(client);
@@ -337,7 +337,7 @@ static int drm_log_client_suspend(struct drm_client_dev *client, bool _console_l
return 0;
}
-static int drm_log_client_resume(struct drm_client_dev *client, bool _console_lock)
+static int drm_log_client_resume(struct drm_client_dev *client)
{
struct drm_log *dlog = client_to_drm_log(client);
diff --git a/drivers/gpu/drm/drm_client_event.c b/drivers/gpu/drm/drm_client_event.c
index c83196ad8b59..cf8799d3734d 100644
--- a/drivers/gpu/drm/drm_client_event.c
+++ b/drivers/gpu/drm/drm_client_event.c
@@ -101,7 +101,7 @@ void drm_client_dev_hotplug(struct drm_device *dev)
}
EXPORT_SYMBOL(drm_client_dev_hotplug);
-void drm_client_dev_restore(struct drm_device *dev)
+void drm_client_dev_restore(struct drm_device *dev, bool force)
{
struct drm_client_dev *client;
int ret;
@@ -114,7 +114,7 @@ void drm_client_dev_restore(struct drm_device *dev)
if (!client->funcs || !client->funcs->restore)
continue;
- ret = client->funcs->restore(client);
+ ret = client->funcs->restore(client, force);
drm_dbg_kms(dev, "%s: ret=%d\n", client->name, ret);
if (!ret) /* The first one to return zero gets the privilege to restore */
break;
@@ -122,7 +122,7 @@ void drm_client_dev_restore(struct drm_device *dev)
mutex_unlock(&dev->clientlist_mutex);
}
-static int drm_client_suspend(struct drm_client_dev *client, bool holds_console_lock)
+static int drm_client_suspend(struct drm_client_dev *client)
{
struct drm_device *dev = client->dev;
int ret = 0;
@@ -131,7 +131,7 @@ static int drm_client_suspend(struct drm_client_dev *client, bool holds_console_
return 0;
if (client->funcs && client->funcs->suspend)
- ret = client->funcs->suspend(client, holds_console_lock);
+ ret = client->funcs->suspend(client);
drm_dbg_kms(dev, "%s: ret=%d\n", client->name, ret);
client->suspended = true;
@@ -139,20 +139,20 @@ static int drm_client_suspend(struct drm_client_dev *client, bool holds_console_
return ret;
}
-void drm_client_dev_suspend(struct drm_device *dev, bool holds_console_lock)
+void drm_client_dev_suspend(struct drm_device *dev)
{
struct drm_client_dev *client;
mutex_lock(&dev->clientlist_mutex);
list_for_each_entry(client, &dev->clientlist, list) {
if (!client->suspended)
- drm_client_suspend(client, holds_console_lock);
+ drm_client_suspend(client);
}
mutex_unlock(&dev->clientlist_mutex);
}
EXPORT_SYMBOL(drm_client_dev_suspend);
-static int drm_client_resume(struct drm_client_dev *client, bool holds_console_lock)
+static int drm_client_resume(struct drm_client_dev *client)
{
struct drm_device *dev = client->dev;
int ret = 0;
@@ -161,7 +161,7 @@ static int drm_client_resume(struct drm_client_dev *client, bool holds_console_l
return 0;
if (client->funcs && client->funcs->resume)
- ret = client->funcs->resume(client, holds_console_lock);
+ ret = client->funcs->resume(client);
drm_dbg_kms(dev, "%s: ret=%d\n", client->name, ret);
client->suspended = false;
@@ -172,14 +172,14 @@ static int drm_client_resume(struct drm_client_dev *client, bool holds_console_l
return ret;
}
-void drm_client_dev_resume(struct drm_device *dev, bool holds_console_lock)
+void drm_client_dev_resume(struct drm_device *dev)
{
struct drm_client_dev *client;
mutex_lock(&dev->clientlist_mutex);
list_for_each_entry(client, &dev->clientlist, list) {
if (client->suspended)
- drm_client_resume(client, holds_console_lock);
+ drm_client_resume(client);
}
mutex_unlock(&dev->clientlist_mutex);
}
diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c
index df73433b4652..71889daef3fc 100644
--- a/drivers/gpu/drm/drm_fb_helper.c
+++ b/drivers/gpu/drm/drm_fb_helper.c
@@ -253,6 +253,7 @@ __drm_fb_helper_restore_fbdev_mode_unlocked(struct drm_fb_helper *fb_helper,
/**
* drm_fb_helper_restore_fbdev_mode_unlocked - restore fbdev configuration
* @fb_helper: driver-allocated fbdev helper, can be NULL
+ * @force: ignore present DRM master
*
* This helper should be called from fbdev emulation's &drm_client_funcs.restore
* callback. It ensures that the user isn't greeted with a black screen when the
@@ -261,9 +262,9 @@ __drm_fb_helper_restore_fbdev_mode_unlocked(struct drm_fb_helper *fb_helper,
* Returns:
* 0 on success, or a negative errno code otherwise.
*/
-int drm_fb_helper_restore_fbdev_mode_unlocked(struct drm_fb_helper *fb_helper)
+int drm_fb_helper_restore_fbdev_mode_unlocked(struct drm_fb_helper *fb_helper, bool force)
{
- return __drm_fb_helper_restore_fbdev_mode_unlocked(fb_helper, false);
+ return __drm_fb_helper_restore_fbdev_mode_unlocked(fb_helper, force);
}
EXPORT_SYMBOL(drm_fb_helper_restore_fbdev_mode_unlocked);
@@ -1316,9 +1317,9 @@ int drm_fb_helper_set_par(struct fb_info *info)
* the KDSET IOCTL with KD_TEXT, and only after that drops the master
* status when exiting.
*
- * In the past this was caught by drm_fb_helper_lastclose(), but on
- * modern systems where logind always keeps a drm fd open to orchestrate
- * the vt switching, this doesn't work.
+ * In the past this was caught by drm_fb_helper_restore_fbdev_mode_unlocked(),
+ * but on modern systems where logind always keeps a drm fd open to
+ * orchestrate the vt switching, this doesn't work.
*
* To not break the userspace ABI we have this special case here, which
* is only used for the above case. Everything else uses the normal
@@ -1946,16 +1947,3 @@ int drm_fb_helper_hotplug_event(struct drm_fb_helper *fb_helper)
return 0;
}
EXPORT_SYMBOL(drm_fb_helper_hotplug_event);
-
-/**
- * drm_fb_helper_lastclose - DRM driver lastclose helper for fbdev emulation
- * @dev: DRM device
- *
- * This function is obsolete. Call drm_fb_helper_restore_fbdev_mode_unlocked()
- * instead.
- */
-void drm_fb_helper_lastclose(struct drm_device *dev)
-{
- drm_fb_helper_restore_fbdev_mode_unlocked(dev->fb_helper);
-}
-EXPORT_SYMBOL(drm_fb_helper_lastclose);
diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c
index eebd1a05ee97..be5e617ceb9f 100644
--- a/drivers/gpu/drm/drm_file.c
+++ b/drivers/gpu/drm/drm_file.c
@@ -405,7 +405,7 @@ EXPORT_SYMBOL(drm_open);
static void drm_lastclose(struct drm_device *dev)
{
- drm_client_dev_restore(dev);
+ drm_client_dev_restore(dev, false);
if (dev_is_pci(dev->dev))
vga_switcheroo_process_delayed_switch();
diff --git a/drivers/gpu/drm/drm_modeset_helper.c b/drivers/gpu/drm/drm_modeset_helper.c
index 988735560570..a57f6a10ada4 100644
--- a/drivers/gpu/drm/drm_modeset_helper.c
+++ b/drivers/gpu/drm/drm_modeset_helper.c
@@ -203,10 +203,10 @@ int drm_mode_config_helper_suspend(struct drm_device *dev)
if (dev->mode_config.poll_enabled)
drm_kms_helper_poll_disable(dev);
- drm_client_dev_suspend(dev, false);
+ drm_client_dev_suspend(dev);
state = drm_atomic_helper_suspend(dev);
if (IS_ERR(state)) {
- drm_client_dev_resume(dev, false);
+ drm_client_dev_resume(dev);
/*
* Don't enable polling if it was never initialized
@@ -252,7 +252,7 @@ int drm_mode_config_helper_resume(struct drm_device *dev)
DRM_ERROR("Failed to resume (%d)\n", ret);
dev->mode_config.suspend_state = NULL;
- drm_client_dev_resume(dev, false);
+ drm_client_dev_resume(dev);
/*
* Don't enable polling if it is not initialized
diff --git a/drivers/gpu/drm/i915/display/intel_cursor.c b/drivers/gpu/drm/i915/display/intel_cursor.c
index d4d181f9dca5..c83cfb6b763a 100644
--- a/drivers/gpu/drm/i915/display/intel_cursor.c
+++ b/drivers/gpu/drm/i915/display/intel_cursor.c
@@ -536,7 +536,8 @@ static void i9xx_cursor_disable_sel_fetch_arm(struct intel_dsb *dsb,
struct intel_display *display = to_intel_display(plane);
enum pipe pipe = plane->pipe;
- if (!crtc_state->enable_psr2_sel_fetch)
+ if (!crtc_state->enable_psr2_sel_fetch &&
+ !crtc_state->clear_psr2_sel_fetch)
return;
intel_de_write_dsb(display, dsb, SEL_FETCH_CUR_CTL(pipe), 0);
@@ -569,8 +570,10 @@ static void i9xx_cursor_update_sel_fetch_arm(struct intel_dsb *dsb,
struct intel_display *display = to_intel_display(plane);
enum pipe pipe = plane->pipe;
- if (!crtc_state->enable_psr2_sel_fetch)
+ if (!crtc_state->enable_psr2_sel_fetch) {
+ i9xx_cursor_disable_sel_fetch_arm(dsb, plane, crtc_state);
return;
+ }
if (drm_rect_height(&plane_state->psr2_sel_fetch_area) > 0) {
if (crtc_state->enable_psr2_su_region_et) {
diff --git a/drivers/gpu/drm/i915/display/intel_display_types.h b/drivers/gpu/drm/i915/display/intel_display_types.h
index 0496b3263ed1..9af08181bda3 100644
--- a/drivers/gpu/drm/i915/display/intel_display_types.h
+++ b/drivers/gpu/drm/i915/display/intel_display_types.h
@@ -1120,6 +1120,8 @@ struct intel_crtc_state {
bool has_sel_update;
bool enable_psr2_sel_fetch;
bool enable_psr2_su_region_et;
+ /* Drop the stale selective fetch enable bits as selective fetch is turned off */
+ bool clear_psr2_sel_fetch;
bool req_psr2_sdp_prior_scanline;
bool has_panel_replay;
bool wm_level_disabled;
diff --git a/drivers/gpu/drm/i915/display/intel_dp_mst.c b/drivers/gpu/drm/i915/display/intel_dp_mst.c
index 352f7ef29c28..cc1fe1465985 100644
--- a/drivers/gpu/drm/i915/display/intel_dp_mst.c
+++ b/drivers/gpu/drm/i915/display/intel_dp_mst.c
@@ -799,7 +799,8 @@ static u8 get_pipes_downstream_of_mst_port(struct intel_atomic_state *state,
if (&connector->mst.dp->mst.mgr != mst_mgr)
continue;
- if (connector->mst.port != parent_port &&
+ if (parent_port &&
+ connector->mst.port != parent_port &&
!drm_dp_mst_port_downstream_of_parent(mst_mgr,
connector->mst.port,
parent_port))
@@ -2069,6 +2070,27 @@ bool intel_dp_mst_crtc_needs_modeset(struct intel_atomic_state *state,
return false;
}
+bool intel_dp_mst_stream_disconnected(struct intel_atomic_state *state,
+ const struct intel_crtc *crtc)
+{
+ struct intel_connector *connector;
+
+ connector = get_connector_in_state_for_crtc(state, crtc);
+ if (!connector)
+ return false;
+
+ if (!connector->mst.dp)
+ return false;
+
+ if (!connector->mst.dp->mst.mgr.mst_state)
+ return true;
+
+ if (drm_connector_is_unregistered(&connector->base))
+ return true;
+
+ return false;
+}
+
/**
* intel_dp_mst_prepare_probe - Prepare an MST link for topology probing
* @intel_dp: DP port object
diff --git a/drivers/gpu/drm/i915/display/intel_dp_mst.h b/drivers/gpu/drm/i915/display/intel_dp_mst.h
index ab09b487c6bb..8ce89242c05c 100644
--- a/drivers/gpu/drm/i915/display/intel_dp_mst.h
+++ b/drivers/gpu/drm/i915/display/intel_dp_mst.h
@@ -28,6 +28,8 @@ int intel_dp_mst_atomic_check_link(struct intel_atomic_state *state,
struct intel_link_bw_limits *limits);
bool intel_dp_mst_crtc_needs_modeset(struct intel_atomic_state *state,
struct intel_crtc *crtc);
+bool intel_dp_mst_stream_disconnected(struct intel_atomic_state *state,
+ const struct intel_crtc *crtc);
void intel_dp_mst_prepare_probe(struct intel_dp *intel_dp);
bool intel_dp_mst_verify_dpcd_state(struct intel_dp *intel_dp);
diff --git a/drivers/gpu/drm/i915/display/intel_link_bw.c b/drivers/gpu/drm/i915/display/intel_link_bw.c
index f52dee0ea412..94de53a6e619 100644
--- a/drivers/gpu/drm/i915/display/intel_link_bw.c
+++ b/drivers/gpu/drm/i915/display/intel_link_bw.c
@@ -63,7 +63,8 @@ void intel_link_bw_init_limits(struct intel_atomic_state *state,
intel_atomic_get_new_crtc_state(state, crtc);
int forced_bpp_x16 = get_forced_link_bpp_x16(state, crtc);
- if (state->base.duplicated && crtc_state) {
+ if ((state->base.duplicated && crtc_state) ||
+ intel_dp_mst_stream_disconnected(state, crtc)) {
limits->max_bpp_x16[pipe] = crtc_state->max_link_bpp_x16;
if (crtc_state->fec_enable)
limits->force_fec_pipes |= BIT(pipe);
diff --git a/drivers/gpu/drm/i915/display/intel_psr.c b/drivers/gpu/drm/i915/display/intel_psr.c
index 1eeae3243bd5..5f6ccd953c1e 100644
--- a/drivers/gpu/drm/i915/display/intel_psr.c
+++ b/drivers/gpu/drm/i915/display/intel_psr.c
@@ -49,6 +49,7 @@
#include "intel_hdmi.h"
#include "intel_psr.h"
#include "intel_psr_regs.h"
+#include "intel_quirks.h"
#include "intel_snps_phy.h"
#include "intel_step.h"
#include "intel_vblank.h"
@@ -592,6 +593,13 @@ static void _panel_replay_init_dpcd(struct intel_dp *intel_dp)
if (intel_dp->mst_detect == DRM_DP_MST)
return;
+ if (intel_dp_is_edp(intel_dp) &&
+ intel_has_dpcd_quirk(intel_dp, QUIRK_DISABLE_EDP_PANEL_REPLAY)) {
+ drm_dbg_kms(display->drm,
+ "Panel Replay support not currently available for this setup\n");
+ return;
+ }
+
ret = drm_dp_dpcd_read_data(&intel_dp->aux, DP_PANEL_REPLAY_CAP_SUPPORT,
&intel_dp->pr_dpcd, sizeof(intel_dp->pr_dpcd));
if (ret < 0)
@@ -660,6 +668,9 @@ static void _psr_init_dpcd(struct intel_dp *intel_dp)
intel_dp->psr.sink_sync_latency =
intel_dp_get_sink_sync_latency(intel_dp);
+ if (intel_has_quirk(display, QUIRK_DISABLE_PSR2))
+ return;
+
if (DISPLAY_VER(display) >= 9 &&
intel_dp->psr_dpcd[0] >= DP_PSR2_WITH_Y_COORD_IS_SUPPORTED) {
bool y_req = intel_dp->psr_dpcd[1] &
@@ -2756,6 +2767,8 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
struct intel_crtc *crtc)
{
struct intel_display *display = to_intel_display(state);
+ const struct intel_crtc_state *old_crtc_state =
+ intel_atomic_get_old_crtc_state(state, crtc);
struct intel_crtc_state *crtc_state = intel_atomic_get_new_crtc_state(state, crtc);
struct intel_plane_state *new_plane_state, *old_plane_state;
struct intel_plane *plane;
@@ -2768,6 +2781,19 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
bool full_update = false, su_area_changed;
int i, ret;
+ /*
+ * Selective fetch is not always usable, for instance it is dropped
+ * while pipe CRC is active. The planes keep their selective fetch
+ * enable bit set in hardware over that, and a plane disabled while
+ * selective fetch is off never gets the bit cleared. Once selective
+ * fetch comes back the hardware would resume fetching for a plane that
+ * is no longer enabled and keep its DDB range reserved, so have the
+ * plane update drop the bit for every plane of the pipe as selective
+ * fetch is turned off.
+ */
+ crtc_state->clear_psr2_sel_fetch = old_crtc_state->enable_psr2_sel_fetch &&
+ !crtc_state->enable_psr2_sel_fetch;
+
if (!crtc_state->enable_psr2_sel_fetch)
return 0;
diff --git a/drivers/gpu/drm/i915/display/intel_quirks.c b/drivers/gpu/drm/i915/display/intel_quirks.c
index 1abbdd426e58..988d0de8645c 100644
--- a/drivers/gpu/drm/i915/display/intel_quirks.c
+++ b/drivers/gpu/drm/i915/display/intel_quirks.c
@@ -86,6 +86,20 @@ static void quirk_edp_limit_rate_hbr2(struct intel_display *display)
drm_info(display->drm, "Applying eDP Limit rate to HBR2 quirk\n");
}
+static void quirk_disable_edp_panel_replay(struct intel_dp *intel_dp)
+{
+ struct intel_display *display = to_intel_display(intel_dp);
+
+ intel_set_dpcd_quirk(intel_dp, QUIRK_DISABLE_EDP_PANEL_REPLAY);
+ drm_info(display->drm, "Applying disable Panel Replay quirk\n");
+}
+
+static void quirk_disable_psr2(struct intel_display *display)
+{
+ intel_set_quirk(display, QUIRK_DISABLE_PSR2);
+ drm_info(display->drm, "PSR2 support not currently available for this setup, applying disable PSR2 quirk\n");
+}
+
struct intel_quirk {
int device;
int subsystem_vendor;
@@ -108,6 +122,8 @@ struct intel_dpcd_quirk {
#define SINK_DEVICE_ID_ANY SINK_DEVICE_ID(0, 0, 0, 0, 0, 0)
+#define DEVICE_ID_ANY 0
+
/* For systems that don't have a meaningful PCI subdevice/subvendor ID */
struct intel_dmi_quirk {
void (*hook)(struct intel_display *display);
@@ -240,6 +256,12 @@ static struct intel_quirk intel_quirks[] = {
/* Dell XPS 13 7390 2-in-1 */
{ 0x8a52, 0x1028, 0x08b0, quirk_edp_limit_rate_hbr2 },
+
+ /* HP Pavilion Plus Laptop 14-ew1xxx */
+ { 0x7d55, 0x103c, 0x8c31, quirk_edp_limit_rate_hbr2 },
+
+ /* Xiaomi Book Pro 14 2026 */
+ { 0xb081, 0x1d72, 0x2424, quirk_disable_psr2 },
};
static const struct intel_dpcd_quirk intel_dpcd_quirks[] = {
@@ -251,7 +273,14 @@ static const struct intel_dpcd_quirk intel_dpcd_quirks[] = {
.sink_oui = SINK_OUI(0x38, 0xec, 0x11),
.hook = quirk_fw_sync_len,
},
-
+ /* Dell XPS 14 DA14260 */
+ {
+ .device = DEVICE_ID_ANY,
+ .subsystem_vendor = 0x1028,
+ .subsystem_device = 0x0db9,
+ .sink_oui = SINK_OUI(0x00, 0x22, 0xb9),
+ .hook = quirk_disable_edp_panel_replay,
+ },
};
void intel_init_quirks(struct intel_display *display)
@@ -262,7 +291,8 @@ void intel_init_quirks(struct intel_display *display)
for (i = 0; i < ARRAY_SIZE(intel_quirks); i++) {
struct intel_quirk *q = &intel_quirks[i];
- if (d->device == q->device &&
+ if ((d->device == q->device ||
+ q->device == DEVICE_ID_ANY) &&
(d->subsystem_vendor == q->subsystem_vendor ||
q->subsystem_vendor == PCI_ANY_ID) &&
(d->subsystem_device == q->subsystem_device ||
@@ -285,7 +315,8 @@ void intel_init_dpcd_quirks(struct intel_dp *intel_dp,
for (i = 0; i < ARRAY_SIZE(intel_dpcd_quirks); i++) {
const struct intel_dpcd_quirk *q = &intel_dpcd_quirks[i];
- if (d->device == q->device &&
+ if ((d->device == q->device ||
+ q->device == DEVICE_ID_ANY) &&
(d->subsystem_vendor == q->subsystem_vendor ||
q->subsystem_vendor == PCI_ANY_ID) &&
(d->subsystem_device == q->subsystem_device ||
diff --git a/drivers/gpu/drm/i915/display/intel_quirks.h b/drivers/gpu/drm/i915/display/intel_quirks.h
index 06da0e286c67..970a4fe52faf 100644
--- a/drivers/gpu/drm/i915/display/intel_quirks.h
+++ b/drivers/gpu/drm/i915/display/intel_quirks.h
@@ -21,6 +21,8 @@ enum intel_quirk_id {
QUIRK_NO_PPS_BACKLIGHT_POWER_HOOK,
QUIRK_FW_SYNC_LEN,
QUIRK_EDP_LIMIT_RATE_HBR2,
+ QUIRK_DISABLE_EDP_PANEL_REPLAY,
+ QUIRK_DISABLE_PSR2,
};
void intel_init_quirks(struct intel_display *display);
diff --git a/drivers/gpu/drm/i915/display/skl_universal_plane.c b/drivers/gpu/drm/i915/display/skl_universal_plane.c
index e13fb781e7b2..62bca23e5132 100644
--- a/drivers/gpu/drm/i915/display/skl_universal_plane.c
+++ b/drivers/gpu/drm/i915/display/skl_universal_plane.c
@@ -892,7 +892,8 @@ static void icl_plane_disable_sel_fetch_arm(struct intel_dsb *dsb,
struct intel_display *display = to_intel_display(plane);
enum pipe pipe = plane->pipe;
- if (!crtc_state->enable_psr2_sel_fetch)
+ if (!crtc_state->enable_psr2_sel_fetch &&
+ !crtc_state->clear_psr2_sel_fetch)
return;
intel_de_write_dsb(display, dsb, SEL_FETCH_PLANE_CTL(pipe, plane->id), 0);
@@ -1606,10 +1607,8 @@ static void icl_plane_update_sel_fetch_arm(struct intel_dsb *dsb,
struct intel_display *display = to_intel_display(plane);
enum pipe pipe = plane->pipe;
- if (!crtc_state->enable_psr2_sel_fetch)
- return;
-
- if (drm_rect_height(&plane_state->psr2_sel_fetch_area) > 0)
+ if (crtc_state->enable_psr2_sel_fetch &&
+ drm_rect_height(&plane_state->psr2_sel_fetch_area) > 0)
intel_de_write_dsb(display, dsb, SEL_FETCH_PLANE_CTL(pipe, plane->id),
SEL_FETCH_PLANE_CTL_ENABLE);
else
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.c b/drivers/gpu/drm/i915/gem/i915_gem_object.c
index 478011e5ecb3..54c62a149a15 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_object.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_object.c
@@ -88,6 +88,7 @@ struct drm_i915_gem_object *i915_gem_object_alloc(void)
void i915_gem_object_free(struct drm_i915_gem_object *obj)
{
+ dma_resv_fini(&obj->base._resv);
return kmem_cache_free(slab_objects, obj);
}
@@ -143,7 +144,6 @@ void __i915_gem_object_fini(struct drm_i915_gem_object *obj)
{
mutex_destroy(&obj->mm.get_page.lock);
mutex_destroy(&obj->mm.get_dma_page.lock);
- dma_resv_fini(&obj->base._resv);
}
/**
diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c
index a28c3710c4d5..89be8da79d3b 100644
--- a/drivers/gpu/drm/i915/i915_driver.c
+++ b/drivers/gpu/drm/i915/i915_driver.c
@@ -978,7 +978,7 @@ void i915_driver_shutdown(struct drm_i915_private *i915)
intel_runtime_pm_disable(&i915->runtime_pm);
intel_power_domains_disable(display);
- drm_client_dev_suspend(&i915->drm, false);
+ drm_client_dev_suspend(&i915->drm);
if (intel_display_device_present(display)) {
drm_kms_helper_poll_disable(&i915->drm);
intel_display_driver_disable_user_access(display);
@@ -1061,7 +1061,7 @@ static int i915_drm_suspend(struct drm_device *dev)
/* We do a lot of poking in a lot of registers, make sure they work
* properly. */
intel_power_domains_disable(display);
- drm_client_dev_suspend(dev, false);
+ drm_client_dev_suspend(dev);
if (intel_display_device_present(display)) {
drm_kms_helper_poll_disable(dev);
intel_display_driver_disable_user_access(display);
@@ -1245,7 +1245,7 @@ static int i915_drm_resume(struct drm_device *dev)
intel_opregion_resume(display);
- drm_client_dev_resume(dev, false);
+ drm_client_dev_resume(dev);
intel_power_domains_enable(display);
diff --git a/drivers/gpu/drm/imagination/pvr_free_list.c b/drivers/gpu/drm/imagination/pvr_free_list.c
index 5228e214491c..2ced452e9ce3 100644
--- a/drivers/gpu/drm/imagination/pvr_free_list.c
+++ b/drivers/gpu/drm/imagination/pvr_free_list.c
@@ -8,6 +8,7 @@
#include "pvr_vm.h"
#include <drm/drm_gem.h>
+#include <drm/drm_print.h>
#include <linux/slab.h>
#include <linux/xarray.h>
#include <uapi/drm/pvr_drm.h>
@@ -612,13 +613,21 @@ pvr_free_list_process_reconstruct_req(struct pvr_device *pvr_dev,
};
struct rogue_fwif_freelists_reconstruction_data *resp =
&resp_cmd.cmd_data.free_lists_reconstruction_data;
+ u32 count = min_t(u32, req->freelist_count,
+ ARRAY_SIZE(req->freelist_ids));
- for (u32 i = 0; i < req->freelist_count; i++)
+ if (count != req->freelist_count) {
+ drm_warn_once(from_pvr_device(pvr_dev),
+ "Requested reconstruction of %u freelists, limiting to %u\n",
+ req->freelist_count, count);
+ }
+
+ for (u32 i = 0; i < count; i++)
pvr_free_list_reconstruct(pvr_dev, req->freelist_ids[i]);
- resp->freelist_count = req->freelist_count;
+ resp->freelist_count = count;
memcpy(resp->freelist_ids, req->freelist_ids,
- req->freelist_count * sizeof(resp->freelist_ids[0]));
+ count * sizeof(resp->freelist_ids[0]));
WARN_ON(pvr_kccb_send_cmd(pvr_dev, &resp_cmd, NULL));
}
diff --git a/drivers/gpu/drm/imagination/pvr_mmu.c b/drivers/gpu/drm/imagination/pvr_mmu.c
index 450d476d183f..146eb003d0ab 100644
--- a/drivers/gpu/drm/imagination/pvr_mmu.c
+++ b/drivers/gpu/drm/imagination/pvr_mmu.c
@@ -12,6 +12,7 @@
#include "pvr_rogue_mmu_defs.h"
#include <drm/drm_drv.h>
+#include <drm/drm_print.h>
#include <linux/atomic.h>
#include <linux/bitops.h>
#include <linux/dma-mapping.h>
@@ -2337,6 +2338,7 @@ void pvr_mmu_op_context_destroy(struct pvr_mmu_op_context *op_ctx)
* pvr_mmu_op_context_create() - Create an MMU op context.
* @ctx: MMU context associated with owning VM context.
* @sgt: Scatter gather table containing pages pinned for use by this context.
+ * @device_addr: Virtual device address at the start of the requested mapping.
* @sgt_offset: Start offset of the requested device-virtual memory mapping.
* @size: Size in bytes of the requested device-virtual memory mapping. For an
* unmapping, this should be zero so that no page tables are allocated.
@@ -2348,8 +2350,9 @@ void pvr_mmu_op_context_destroy(struct pvr_mmu_op_context *op_ctx)
*/
struct pvr_mmu_op_context *
pvr_mmu_op_context_create(struct pvr_mmu_context *ctx, struct sg_table *sgt,
- u64 sgt_offset, u64 size)
+ u64 device_addr, u64 sgt_offset, u64 size)
{
+ u64 start_addr = device_addr + sgt_offset;
int err;
struct pvr_mmu_op_context *op_ctx =
@@ -2366,16 +2369,16 @@ pvr_mmu_op_context_create(struct pvr_mmu_context *ctx, struct sg_table *sgt,
if (size) {
/*
* The number of page table objects we need to prealloc is
- * indicated by the mapping size, start offset and the sizes
+ * indicated by the mapping size, start address and the sizes
* of the areas mapped per PT or PD. The range calculation is
* identical to that for the index into a table for a device
* address, so we reuse those functions here.
*/
- const u32 l1_start_idx = pvr_page_table_l2_idx(sgt_offset);
- const u32 l1_end_idx = pvr_page_table_l2_idx(sgt_offset + size);
+ const u32 l1_start_idx = pvr_page_table_l2_idx(start_addr);
+ const u32 l1_end_idx = pvr_page_table_l2_idx(start_addr + size);
const u32 l1_count = l1_end_idx - l1_start_idx + 1;
- const u32 l0_start_idx = pvr_page_table_l1_idx(sgt_offset);
- const u32 l0_end_idx = pvr_page_table_l1_idx(sgt_offset + size);
+ const u32 l0_start_idx = pvr_page_table_l1_idx(start_addr);
+ const u32 l0_end_idx = pvr_page_table_l1_idx(start_addr + size);
const u32 l0_count = l0_end_idx - l0_start_idx + 1;
/*
@@ -2556,7 +2559,9 @@ pvr_mmu_map_sgl(struct pvr_mmu_op_context *op_ctx, struct scatterlist *sgl,
err_destroy_pages:
memcpy(&op_ctx->curr_page, &ptr_copy, sizeof(op_ctx->curr_page));
- err = pvr_mmu_op_context_unmap_curr_page(op_ctx, page);
+ if (pvr_mmu_op_context_unmap_curr_page(op_ctx, page))
+ drm_err(from_pvr_device(op_ctx->mmu_ctx->pvr_dev),
+ "%s : Failure in unmapping pages\n", __func__);
return err;
}
diff --git a/drivers/gpu/drm/imagination/pvr_mmu.h b/drivers/gpu/drm/imagination/pvr_mmu.h
index a8ecd460168d..2c02d61ba0a2 100644
--- a/drivers/gpu/drm/imagination/pvr_mmu.h
+++ b/drivers/gpu/drm/imagination/pvr_mmu.h
@@ -99,7 +99,7 @@ dma_addr_t pvr_mmu_get_root_table_dma_addr(struct pvr_mmu_context *ctx);
void pvr_mmu_op_context_destroy(struct pvr_mmu_op_context *op_ctx);
struct pvr_mmu_op_context *
pvr_mmu_op_context_create(struct pvr_mmu_context *ctx,
- struct sg_table *sgt, u64 sgt_offset, u64 size);
+ struct sg_table *sgt, u64 device_addr, u64 sgt_offset, u64 size);
int pvr_mmu_map(struct pvr_mmu_op_context *op_ctx, u64 size, u64 flags,
u64 device_addr);
diff --git a/drivers/gpu/drm/imagination/pvr_vm.c b/drivers/gpu/drm/imagination/pvr_vm.c
index b2f8e89dafb7..e59b93c1a6e1 100644
--- a/drivers/gpu/drm/imagination/pvr_vm.c
+++ b/drivers/gpu/drm/imagination/pvr_vm.c
@@ -275,7 +275,7 @@ pvr_vm_bind_op_map_init(struct pvr_vm_bind_op *bind_op,
goto err_bind_op_fini;
bind_op->mmu_op_ctx =
- pvr_mmu_op_context_create(vm_ctx->mmu_ctx, sgt, offset, size);
+ pvr_mmu_op_context_create(vm_ctx->mmu_ctx, sgt, device_addr, offset, size);
err = PTR_ERR_OR_ZERO(bind_op->mmu_op_ctx);
if (err) {
bind_op->mmu_op_ctx = NULL;
@@ -317,7 +317,7 @@ pvr_vm_bind_op_unmap_init(struct pvr_vm_bind_op *bind_op,
}
bind_op->mmu_op_ctx =
- pvr_mmu_op_context_create(vm_ctx->mmu_ctx, NULL, 0, 0);
+ pvr_mmu_op_context_create(vm_ctx->mmu_ctx, NULL, device_addr, 0, 0);
err = PTR_ERR_OR_ZERO(bind_op->mmu_op_ctx);
if (err) {
bind_op->mmu_op_ctx = NULL;
diff --git a/drivers/gpu/drm/nouveau/nouveau_bo.c b/drivers/gpu/drm/nouveau/nouveau_bo.c
index f26562eafffc..902de271cc42 100644
--- a/drivers/gpu/drm/nouveau/nouveau_bo.c
+++ b/drivers/gpu/drm/nouveau/nouveau_bo.c
@@ -578,8 +578,9 @@ int nouveau_bo_pin_locked(struct nouveau_bo *nvbo, uint32_t domain, bool contig)
"0x%08x vs 0x%08x\n", bo,
bo->resource->mem_type, domain);
ret = -EBUSY;
+ } else {
+ ttm_bo_pin(&nvbo->bo);
}
- ttm_bo_pin(&nvbo->bo);
goto out;
}
diff --git a/drivers/gpu/drm/nouveau/nouveau_connector.c b/drivers/gpu/drm/nouveau/nouveau_connector.c
index 902e0e93e968..7c6d91ee2722 100644
--- a/drivers/gpu/drm/nouveau/nouveau_connector.c
+++ b/drivers/gpu/drm/nouveau/nouveau_connector.c
@@ -600,8 +600,11 @@ nouveau_connector_detect(struct drm_connector *connector, bool force)
new_edid = drm_get_edid(connector, nv_encoder->i2c);
} else {
ret = nvif_outp_edid_get(&nv_encoder->outp, (u8 **)&new_edid);
- if (ret < 0)
+ if (ret < 0) {
+ pm_runtime_mark_last_busy(dev->dev);
+ pm_runtime_put_autosuspend(dev->dev);
return connector_status_disconnected;
+ }
}
nouveau_connector_set_edid(nv_connector, new_edid);
diff --git a/drivers/gpu/drm/nouveau/nouveau_display.c b/drivers/gpu/drm/nouveau/nouveau_display.c
index 805d0a87aa54..d03364f3c597 100644
--- a/drivers/gpu/drm/nouveau/nouveau_display.c
+++ b/drivers/gpu/drm/nouveau/nouveau_display.c
@@ -764,7 +764,7 @@ nouveau_display_suspend(struct drm_device *dev, bool runtime)
{
struct nouveau_display *disp = nouveau_display(dev);
- drm_client_dev_suspend(dev, false);
+ drm_client_dev_suspend(dev);
if (drm_drv_uses_atomic_modeset(dev)) {
if (!runtime) {
@@ -795,7 +795,7 @@ nouveau_display_resume(struct drm_device *dev, bool runtime)
}
}
- drm_client_dev_resume(dev, false);
+ drm_client_dev_resume(dev);
}
int
diff --git a/drivers/gpu/drm/nouveau/nouveau_drm.c b/drivers/gpu/drm/nouveau/nouveau_drm.c
index b760586b1727..d05da2db94a4 100644
--- a/drivers/gpu/drm/nouveau/nouveau_drm.c
+++ b/drivers/gpu/drm/nouveau/nouveau_drm.c
@@ -584,6 +584,7 @@ nouveau_drm_device_fini(struct nouveau_drm *drm)
if (nouveau_pmops_runtime()) {
pm_runtime_get_sync(dev->dev);
pm_runtime_forbid(dev->dev);
+ pm_runtime_dont_use_autosuspend(dev->dev);
}
nouveau_led_fini(dev);
@@ -1219,10 +1220,8 @@ nouveau_drm_open(struct drm_device *dev, struct drm_file *fpriv)
mutex_unlock(&drm->clients_lock);
done:
- if (ret && cli) {
- nouveau_cli_fini(cli);
+ if (ret && cli)
kfree(cli);
- }
pm_runtime_mark_last_busy(dev->dev);
pm_runtime_put_autosuspend(dev->dev);
diff --git a/drivers/gpu/drm/nouveau/nouveau_gem.c b/drivers/gpu/drm/nouveau/nouveau_gem.c
index 80e2bde9915f..c497497af82d 100644
--- a/drivers/gpu/drm/nouveau/nouveau_gem.c
+++ b/drivers/gpu/drm/nouveau/nouveau_gem.c
@@ -522,6 +522,7 @@ validate_init(struct nouveau_channel *chan, struct drm_file *file_priv,
if (unlikely(ret)) {
if (ret != -ERESTARTSYS)
NV_PRINTK(err, cli, "fail reserve\n");
+ drm_gem_object_put(gem);
break;
}
}
@@ -531,6 +532,7 @@ validate_init(struct nouveau_channel *chan, struct drm_file *file_priv,
struct nouveau_vma *vma = nouveau_vma_find(nvbo, vmm);
if (!vma) {
NV_PRINTK(err, cli, "vma not found!\n");
+ drm_gem_object_put(gem);
ret = -EINVAL;
break;
}
diff --git a/drivers/gpu/drm/nouveau/nouveau_sched.c b/drivers/gpu/drm/nouveau/nouveau_sched.c
index a7bf539e5d86..16158b1acb56 100644
--- a/drivers/gpu/drm/nouveau/nouveau_sched.c
+++ b/drivers/gpu/drm/nouveau/nouveau_sched.c
@@ -518,7 +518,7 @@ nouveau_sched_destroy(struct nouveau_sched **psched)
struct nouveau_sched *sched = *psched;
nouveau_sched_fini(sched);
- kfree(sched);
+ kfree_rcu(sched, rcu);
*psched = NULL;
}
diff --git a/drivers/gpu/drm/nouveau/nouveau_sched.h b/drivers/gpu/drm/nouveau/nouveau_sched.h
index 20cd1da8db73..51ce8dcf6285 100644
--- a/drivers/gpu/drm/nouveau/nouveau_sched.h
+++ b/drivers/gpu/drm/nouveau/nouveau_sched.h
@@ -98,6 +98,7 @@ void nouveau_job_free(struct nouveau_job *job);
struct nouveau_sched {
struct drm_gpu_scheduler base;
+ struct rcu_head rcu;
struct drm_sched_entity entity;
struct workqueue_struct *wq;
struct mutex mutex;
diff --git a/drivers/gpu/drm/nouveau/nouveau_uvmm.c b/drivers/gpu/drm/nouveau/nouveau_uvmm.c
index be3b50e17ff2..2ee1c3811993 100644
--- a/drivers/gpu/drm/nouveau/nouveau_uvmm.c
+++ b/drivers/gpu/drm/nouveau/nouveau_uvmm.c
@@ -780,6 +780,9 @@ op_map(struct nouveau_uvma *uvma)
{
struct nouveau_bo *nvbo = nouveau_gem_object(uvma->va.gem.obj);
+ if (drm_gpuva_invalidated(&uvma->va))
+ return;
+
nouveau_uvma_map(uvma, nouveau_mem(nvbo->bo.resource));
}
@@ -1164,6 +1167,7 @@ bind_lock_validate(struct nouveau_job *job, struct drm_exec *exec,
drm_gpuva_for_each_op(va_op, op->ops) {
struct drm_gem_object *obj = op_gem_obj(va_op);
+ struct nouveau_bo *nvbo;
if (unlikely(!obj))
continue;
@@ -1178,8 +1182,13 @@ bind_lock_validate(struct nouveau_job *job, struct drm_exec *exec,
if (va_op->op == DRM_GPUVA_OP_UNMAP)
continue;
- ret = nouveau_bo_validate(nouveau_gem_object(obj),
- true, false);
+ nvbo = nouveau_gem_object(obj);
+ if (!(nvbo->valid_domains &
+ (NOUVEAU_GEM_DOMAIN_VRAM | NOUVEAU_GEM_DOMAIN_GART)))
+ return -EINVAL;
+
+ nouveau_bo_placement_set(nvbo, nvbo->valid_domains, 0);
+ ret = nouveau_bo_validate(nvbo, true, false);
if (ret)
return ret;
}
diff --git a/drivers/gpu/drm/nouveau/nvif/vmm.c b/drivers/gpu/drm/nouveau/nvif/vmm.c
index 07c1ebc2a941..d9671807e722 100644
--- a/drivers/gpu/drm/nouveau/nvif/vmm.c
+++ b/drivers/gpu/drm/nouveau/nvif/vmm.c
@@ -192,6 +192,7 @@ void
nvif_vmm_dtor(struct nvif_vmm *vmm)
{
kfree(vmm->page);
+ vmm->page = NULL;
nvif_object_dtor(&vmm->object);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/uoutp.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/uoutp.c
index 377d0e0cef84..9887b3898505 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/uoutp.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/uoutp.c
@@ -253,8 +253,7 @@ nvkm_uoutp_mthd_hdmi(struct nvkm_outp *outp, void *argv, u32 argc)
if (!ior->func->hdmi ||
args->v0.max_ac_packet > 0x1f ||
- args->v0.rekey > 0x7f ||
- (args->v0.scdc && !ior->func->hdmi->scdc))
+ args->v0.rekey > 0x7f)
return -EINVAL;
if (!args->v0.enable) {
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/clk/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/clk/base.c
index 71420f81714b..ecde72a4f2ee 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/clk/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/clk/base.c
@@ -199,16 +199,18 @@ nvkm_cstate_prog(struct nvkm_clk *clk, struct nvkm_pstate *pstate, int cstatei)
}
if (volt) {
- ret = nvkm_volt_set_id(volt, cstate->voltage,
- pstate->base.voltage, clk->temp, -1);
- if (ret && ret != -ENODEV)
- nvkm_error(subdev, "failed to lower voltage: %d\n", ret);
+ int err = nvkm_volt_set_id(volt, cstate->voltage,
+ pstate->base.voltage, clk->temp, -1);
+
+ if (err && err != -ENODEV)
+ nvkm_error(subdev, "failed to lower voltage: %d\n", err);
}
if (therm) {
- ret = nvkm_therm_cstate(therm, pstate->fanspeed, -1);
- if (ret && ret != -ENODEV)
- nvkm_error(subdev, "failed to lower fan speed: %d\n", ret);
+ int err = nvkm_therm_cstate(therm, pstate->fanspeed, -1);
+
+ if (err && err != -ENODEV)
+ nvkm_error(subdev, "failed to lower fan speed: %d\n", err);
}
return ret;
@@ -473,6 +475,7 @@ static int
nvkm_clk_ustate_update(struct nvkm_clk *clk, int req)
{
struct nvkm_pstate *pstate;
+ bool found = false;
int i = 0;
if (!clk->allow_reclock)
@@ -480,12 +483,14 @@ nvkm_clk_ustate_update(struct nvkm_clk *clk, int req)
if (req != -1 && req != -2) {
list_for_each_entry(pstate, &clk->states, head) {
- if (pstate->pstate == req)
+ if (pstate->pstate == req) {
+ found = true;
break;
+ }
i++;
}
- if (pstate->pstate != req)
+ if (!found)
return -EINVAL;
req = i;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramnv1a.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramnv1a.c
index 18241c6ba5fa..4d52a158f320 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramnv1a.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramnv1a.c
@@ -51,6 +51,8 @@ nv1a_ram_new(struct nvkm_fb *fb, struct nvkm_ram **pram)
mib = ((mem >> 4) & 127) + 1;
}
+ pci_dev_put(bridge);
+
return nvkm_ram_new_(&nv04_ram_func, fb, NVKM_RAM_TYPE_STOLEN,
mib * 1024 * 1024, pram);
}
diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c
index 9e35b14e2bf0..60afaa8e56b4 100644
--- a/drivers/gpu/drm/radeon/radeon_device.c
+++ b/drivers/gpu/drm/radeon/radeon_device.c
@@ -1635,7 +1635,7 @@ int radeon_suspend_kms(struct drm_device *dev, bool suspend,
}
if (notify_clients)
- drm_client_dev_suspend(dev, false);
+ drm_client_dev_suspend(dev);
return 0;
}
@@ -1739,7 +1739,7 @@ int radeon_resume_kms(struct drm_device *dev, bool resume, bool notify_clients)
radeon_pm_compute_clocks(rdev);
if (notify_clients)
- drm_client_dev_resume(dev, false);
+ drm_client_dev_resume(dev);
return 0;
}
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 00c609c3b53a..8575c7787fc3 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -113,6 +113,8 @@ struct virtio_gpu_object {
bool dumb;
bool created;
bool attached;
+ /* a guest-bound transfer is queued and its mapping not yet synced */
+ bool from_host_pending;
bool host3d_blob, guest_blob;
uint32_t blob_mem, blob_flags;
@@ -191,6 +193,9 @@ struct virtio_gpu_vbuffer {
struct list_head list;
uint32_t seqno;
+
+ /* guest-bound transfer whose shmem backing needs a CPU sync */
+ bool sync_for_cpu;
};
struct virtio_gpu_output {
diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c b/drivers/gpu/drm/virtio/virtgpu_gem.c
index 7174086119c5..aea5e1b60e7f 100644
--- a/drivers/gpu/drm/virtio/virtgpu_gem.c
+++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
@@ -45,7 +45,7 @@ static int virtio_gpu_gem_create(struct drm_file *file,
ret = drm_gem_handle_create(file, &obj->base.base, &handle);
if (ret) {
- drm_gem_object_release(&obj->base.base);
+ drm_gem_object_put(&obj->base.base);
return ret;
}
diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
index c33c057365f8..86777a7d41d8 100644
--- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
+++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
@@ -180,7 +180,7 @@ static int virtio_gpu_resource_create_ioctl(struct drm_device *dev, void *data,
ret = drm_gem_handle_create(file, obj, &handle);
if (ret) {
- drm_gem_object_release(obj);
+ drm_gem_object_put(obj);
return ret;
}
@@ -256,6 +256,27 @@ static int virtio_gpu_transfer_from_host_ioctl(struct drm_device *dev,
if (ret != 0)
goto err_put_free;
+ if (virtio_gpu_is_shmem(bo) && virtio_gpu_use_dma_api(vgdev->vdev)) {
+ /*
+ * The sync on completion restores the whole mapping, so an
+ * earlier transfer has to be done before this one snapshots it.
+ * Otherwise the snapshot predates anything the CPU wrote once
+ * that transfer's fence signalled, and the later sync would
+ * discard it. Nothing can add a fence behind our back here,
+ * since doing so takes the reservation we already hold.
+ * This writes the pages, so it waits as a writer does. READ
+ * usage covers existing readers.
+ */
+ long wait = dma_resv_wait_timeout(objs->objs[0]->resv,
+ DMA_RESV_USAGE_READ, true,
+ MAX_SCHEDULE_TIMEOUT);
+
+ if (wait < 0) {
+ ret = wait;
+ goto err_unlock;
+ }
+ }
+
fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context, 0);
if (!fence) {
ret = -ENOMEM;
@@ -315,6 +336,28 @@ static int virtio_gpu_transfer_to_host_ioctl(struct drm_device *dev, void *data,
if (ret != 0)
goto err_put_free;
+ /*
+ * A transfer the other way may have queued without yet syncing
+ * its mapping. Pushing the guest pages into it now would
+ * discard what the device wrote there, so wait for that sync:
+ * it runs before the fence it belongs to is signalled. The
+ * flag is only set under this reservation, so it cannot appear
+ * behind our back, and the acquire pairs with the release in
+ * that sync, so finding it clear means the pages it wrote are
+ * visible here too.
+ */
+ if (smp_load_acquire(&bo->from_host_pending)) {
+ long wait = dma_resv_wait_timeout(objs->objs[0]->resv,
+ DMA_RESV_USAGE_WRITE,
+ true,
+ MAX_SCHEDULE_TIMEOUT);
+
+ if (wait < 0) {
+ ret = wait;
+ goto err_unlock;
+ }
+ }
+
ret = -ENOMEM;
fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context,
0);
@@ -546,14 +589,14 @@ static int virtio_gpu_resource_create_blob_ioctl(struct drm_device *dev,
if (params.blob_flags & VIRTGPU_BLOB_FLAG_USE_CROSS_DEVICE) {
ret = virtio_gpu_resource_assign_uuid(vgdev, bo);
if (ret) {
- drm_gem_object_release(obj);
+ drm_gem_object_put(obj);
return ret;
}
}
ret = drm_gem_handle_create(file, obj, &handle);
if (ret) {
- drm_gem_object_release(obj);
+ drm_gem_object_put(obj);
return ret;
}
diff --git a/drivers/gpu/drm/virtio/virtgpu_prime.c b/drivers/gpu/drm/virtio/virtgpu_prime.c
index 2fedd5d3bd62..ce49282198cb 100644
--- a/drivers/gpu/drm/virtio/virtgpu_prime.c
+++ b/drivers/gpu/drm/virtio/virtgpu_prime.c
@@ -312,7 +312,7 @@ struct drm_gem_object *virtgpu_gem_prime_import(struct drm_device *dev,
}
}
- if (!vgdev->has_resource_blob)
+ if (!vgdev->has_resource_blob || vgdev->has_virgl_3d)
return drm_gem_prime_import(dev, buf);
bo = kzalloc(sizeof(*bo), GFP_KERNEL);
diff --git a/drivers/gpu/drm/virtio/virtgpu_submit.c b/drivers/gpu/drm/virtio/virtgpu_submit.c
index 409ecdb0d680..3ee876cb004b 100644
--- a/drivers/gpu/drm/virtio/virtgpu_submit.c
+++ b/drivers/gpu/drm/virtio/virtgpu_submit.c
@@ -389,10 +389,13 @@ static int virtio_gpu_init_submit(struct virtio_gpu_submit *submit,
if ((exbuf->flags & VIRTGPU_EXECBUF_FENCE_FD_OUT) ||
exbuf->num_out_syncobjs ||
exbuf->num_bo_handles ||
- drm_fence_event)
+ drm_fence_event) {
out_fence = virtio_gpu_fence_alloc(vgdev, fence_ctx, ring_idx);
- else
+ if (!out_fence)
+ return -ENOMEM;
+ } else {
out_fence = NULL;
+ }
if (drm_fence_event) {
err = virtio_gpu_fence_event_create(dev, file, out_fence, ring_idx);
@@ -538,6 +541,10 @@ int virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data,
virtio_gpu_process_post_deps(&submit);
virtio_gpu_complete_submit(&submit);
cleanup:
+ if (ret && submit.out_fence && submit.out_fence->e) {
+ drm_event_cancel_free(dev, &submit.out_fence->e->base);
+ submit.out_fence->e = NULL;
+ }
virtio_gpu_cleanup_submit(&submit);
return ret;
diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c
index 4a72c69fe28e..592ad4f9b100 100644
--- a/drivers/gpu/drm/virtio/virtgpu_vq.c
+++ b/drivers/gpu/drm/virtio/virtgpu_vq.c
@@ -240,6 +240,33 @@ void virtio_gpu_dequeue_ctrl_func(struct work_struct *work)
} while (!virtqueue_enable_cb(vgdev->ctrlq.vq));
spin_unlock(&vgdev->ctrlq.qlock);
+ /*
+ * Sync guest-bound transfers before signalling anything, so that a
+ * waiter cannot read the backing pages while what the device wrote is
+ * still in a bounce buffer. This cannot be folded into the loop below:
+ * virtio_gpu_fence_event_process() also signals every earlier fence in
+ * the same context, so any entry there may signal this entry's fence.
+ */
+ list_for_each_entry(entry, &reclaim_list, list) {
+ if (entry->sync_for_cpu) {
+ struct virtio_gpu_object *bo =
+ gem_to_virtio_gpu_obj(entry->objs->objs[0]);
+
+ dma_sync_sgtable_for_cpu(vgdev->vdev->dev.parent,
+ bo->base.sgt, DMA_FROM_DEVICE);
+ /*
+ * Release, so a transfer the other way that skips its
+ * wait on the strength of this cannot go on to read
+ * the backing pages before the sync above is visible.
+ * Nothing orders the two otherwise: where the mapping
+ * bounces on a coherent device the sync is a plain
+ * copy, and dma_direct_sync_sg_for_cpu() emits its
+ * barrier only for the non-coherent case.
+ */
+ smp_store_release(&bo->from_host_pending, false);
+ }
+ }
+
list_for_each_entry(entry, &reclaim_list, list) {
resp = (struct virtio_gpu_ctrl_hdr *)entry->resp_buf;
@@ -1225,12 +1252,31 @@ void virtio_gpu_cmd_transfer_from_host_3d(struct virtio_gpu_device *vgdev,
struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(objs->objs[0]);
struct virtio_gpu_transfer_host_3d *cmd_p;
struct virtio_gpu_vbuffer *vbuf;
+ bool use_dma_api = virtio_gpu_use_dma_api(vgdev->vdev);
cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
memset(cmd_p, 0, sizeof(*cmd_p));
vbuf->objs = objs;
+ if (virtio_gpu_is_shmem(bo) && use_dma_api) {
+ /*
+ * The device writes only the requested box, so prime the
+ * mapping with the current contents: otherwise the sync on
+ * completion would hand back whatever a bounce buffer held for
+ * the regions the device does not touch.
+ */
+ dma_sync_sgtable_for_device(vgdev->vdev->dev.parent,
+ bo->base.sgt, DMA_TO_DEVICE);
+ vbuf->sync_for_cpu = true;
+ /*
+ * Set under the reservation the caller holds, so a transfer
+ * the other way cannot miss it and push the guest pages into
+ * the mapping while the device still owns it.
+ */
+ WRITE_ONCE(bo->from_host_pending, true);
+ }
+
cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_TRANSFER_FROM_HOST_3D);
cmd_p->hdr.ctx_id = cpu_to_le32(ctx_id);
cmd_p->resource_id = cpu_to_le32(bo->hw_res_handle);
diff --git a/drivers/gpu/drm/virtio/virtgpu_vram.c b/drivers/gpu/drm/virtio/virtgpu_vram.c
index 5ad3b7c6f73c..f955779d5ea0 100644
--- a/drivers/gpu/drm/virtio/virtgpu_vram.c
+++ b/drivers/gpu/drm/virtio/virtgpu_vram.c
@@ -205,16 +205,12 @@ int virtio_gpu_vram_create(struct virtio_gpu_device *vgdev,
/* Create fake offset */
ret = drm_gem_create_mmap_offset(obj);
- if (ret) {
- kfree(vram);
- return ret;
- }
+ if (ret)
+ goto err_release_obj;
ret = virtio_gpu_resource_id_get(vgdev, &vram->base.hw_res_handle);
- if (ret) {
- kfree(vram);
- return ret;
- }
+ if (ret)
+ goto err_release_obj;
virtio_gpu_cmd_resource_create_blob(vgdev, &vram->base, params, NULL,
0);
@@ -228,4 +224,9 @@ int virtio_gpu_vram_create(struct virtio_gpu_device *vgdev,
*bo_ptr = &vram->base;
return 0;
+
+err_release_obj:
+ drm_gem_object_release(obj);
+ kfree(vram);
+ return ret;
}
diff --git a/drivers/gpu/drm/xe/display/xe_display.c b/drivers/gpu/drm/xe/display/xe_display.c
index 2ca957294b85..fbdf1bad97d8 100644
--- a/drivers/gpu/drm/xe/display/xe_display.c
+++ b/drivers/gpu/drm/xe/display/xe_display.c
@@ -331,7 +331,7 @@ void xe_display_pm_suspend(struct xe_device *xe)
* properly.
*/
intel_power_domains_disable(display);
- drm_client_dev_suspend(&xe->drm, false);
+ drm_client_dev_suspend(&xe->drm);
if (intel_display_device_present(display)) {
drm_kms_helper_poll_disable(&xe->drm);
@@ -363,7 +363,7 @@ void xe_display_pm_shutdown(struct xe_device *xe)
return;
intel_power_domains_disable(display);
- drm_client_dev_suspend(&xe->drm, false);
+ drm_client_dev_suspend(&xe->drm);
if (intel_display_device_present(display)) {
drm_kms_helper_poll_disable(&xe->drm);
@@ -488,7 +488,7 @@ void xe_display_pm_resume(struct xe_device *xe)
intel_opregion_resume(display);
- drm_client_dev_resume(&xe->drm, false);
+ drm_client_dev_resume(&xe->drm);
intel_power_domains_enable(display);
}
diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c
index 5c531b8d9211..5eec1253c9f2 100644
--- a/drivers/gpu/drm/xe/xe_bo.c
+++ b/drivers/gpu/drm/xe/xe_bo.c
@@ -853,6 +853,13 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict,
} else {
drm_dbg(&xe->drm, "Evict system allocator BO failed=%pe\n",
ERR_PTR(ret));
+ /*
+ * The semantic we want upon SVM eviction failure
+ * because of racing access is keep walking for
+ * eviction, which is -ENOSPC.
+ */
+ if (ret == -EBUSY)
+ ret = -ENOSPC;
}
goto out;
diff --git a/drivers/gpu/drm/xe/xe_bo.h b/drivers/gpu/drm/xe/xe_bo.h
index 71169b57fbda..949b248c5adf 100644
--- a/drivers/gpu/drm/xe/xe_bo.h
+++ b/drivers/gpu/drm/xe/xe_bo.h
@@ -9,6 +9,8 @@
#include <drm/drm_prime.h>
#include <drm/ttm/ttm_tt.h>
+#include <xen/xen.h>
+
#include "xe_bo_types.h"
#include "xe_macros.h"
#include "xe_validation.h"
@@ -481,6 +483,23 @@ static inline unsigned int xe_sg_segment_size(struct device *dev)
struct scatterlist __maybe_unused sg;
size_t max = BIT_ULL(sizeof(sg.length) * 8) - 1;
+ /*
+ * For Xen PV guests pages aren't contiguous in DMA (machine) address
+ * space. The DMA API takes care of that both in dma_alloc_* (by
+ * calling into the hypervisor to make the pages contiguous) and in
+ * dma_map_* (by bounce buffering). But xe (like i915, see commit
+ * 78a07fe777c4) ignores the coherency aspects of the DMA API and thus
+ * can't cope with bounce buffering actually happening, so add a hack
+ * here to force small allocations and mappings when running in PV
+ * mode on Xen.
+ *
+ * Note this will still break if bounce buffering is required for other
+ * reasons, like confidential computing hypervisors or PCIe root ports
+ * with addressing limitations.
+ */
+ if (xen_pv_domain())
+ return PAGE_SIZE;
+
max = min_t(size_t, max, dma_max_mapping_size(dev));
/*
diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c
index 66a738f17a3b..2cc8fe677dec 100644
--- a/drivers/gpu/drm/xe/xe_hw_engine.c
+++ b/drivers/gpu/drm/xe/xe_hw_engine.c
@@ -577,22 +577,37 @@ static void adjust_idledly(struct xe_hw_engine *hwe)
u32 idledly_units_ps = 8 * gt->info.timestamp_base;
u32 maxcnt_units_ns = 640;
bool inhibit_switch = 0;
+ bool wa_applied = false;
+
+ if (!IS_SRIOV_VF(gt_to_xe(gt)) && XE_GT_WA(gt, 16023105232)) {
+ /* xe_gt_clock_init() warns and zeroes timestamp_base on unknown crystal clock. */
+ if (!idledly_units_ps)
+ return;
- if (!IS_SRIOV_VF(gt_to_xe(hwe->gt)) && XE_GT_WA(gt, 16023105232)) {
idledly = xe_mmio_read32(>->mmio, RING_IDLEDLY(hwe->mmio_base));
maxcnt = xe_mmio_read32(>->mmio, RING_PWRCTX_MAXCNT(hwe->mmio_base));
inhibit_switch = idledly & INHIBIT_SWITCH_UNTIL_PREEMPTED;
idledly = REG_FIELD_GET(IDLE_DELAY, idledly);
- idledly = DIV_ROUND_CLOSEST(idledly * idledly_units_ps, 1000);
+ idledly = DIV_ROUND_CLOSEST_ULL((u64)idledly * idledly_units_ps, 1000);
maxcnt = REG_FIELD_GET(IDLE_WAIT_TIME, maxcnt);
maxcnt *= maxcnt_units_ns;
- if (xe_gt_WARN_ON(gt, idledly >= maxcnt || inhibit_switch)) {
- idledly = DIV_ROUND_CLOSEST(((maxcnt - 1) * 1000),
- idledly_units_ps);
- xe_mmio_write32(>->mmio, RING_IDLEDLY(hwe->mmio_base), idledly);
+ /* Clear the inhibit switch without disturbing a valid delay. */
+ if (inhibit_switch)
+ wa_applied = true;
+
+ if (xe_gt_WARN_ON(gt, idledly >= maxcnt)) {
+ /* Floor below maxcnt; write 0 to still clear the inhibit bit. */
+ idledly = maxcnt ?
+ DIV_ROUND_DOWN_ULL((u64)(maxcnt - 1) * 1000,
+ idledly_units_ps) : 0;
+ wa_applied = true;
}
+
+ if (wa_applied)
+ xe_mmio_write32(>->mmio, RING_IDLEDLY(hwe->mmio_base),
+ REG_FIELD_PREP(IDLE_DELAY, idledly));
}
}
diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index 423705bcd94c..a4272bc248ec 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -1744,21 +1744,13 @@ void xe_vm_close_and_put(struct xe_vm *vm)
vma->gpuva.flags |= XE_VMA_DESTROYED;
}
- /*
- * All vm operations will add shared fences to resv.
- * The only exception is eviction for a shared object,
- * but even so, the unbind when evicted would still
- * install a fence to resv. Hence it's safe to
- * destroy the pagetables immediately.
- */
- xe_vm_free_scratch(vm);
- xe_vm_pt_destroy(vm);
xe_vm_unlock(vm);
/*
- * VM is now dead, cannot re-add nodes to vm->vmas if it's NULL
- * Since we hold a refcount to the bo, we can remove and free
- * the members safely without locking.
+ * Unlink and destroy all contested external-BO VMAs before destroying
+ * the page tables. Otherwise, concurrent eviction holding only bo->resv
+ * can walk the BO's VMAs and attempt to invalidate/zap page tables that
+ * have already been freed.
*/
list_for_each_entry_safe(vma, next_vma, &contested,
combined_links.destroy) {
@@ -1766,6 +1758,11 @@ void xe_vm_close_and_put(struct xe_vm *vm)
xe_vma_destroy_unlocked(vma);
}
+ xe_vm_lock(vm, false);
+ xe_vm_free_scratch(vm);
+ xe_vm_pt_destroy(vm);
+ xe_vm_unlock(vm);
+
xe_svm_fini(vm);
up_write(&vm->lock);
diff --git a/drivers/hid/amd-sfh-hid/amd_sfh_common.h b/drivers/hid/amd-sfh-hid/amd_sfh_common.h
index 78f830c133e5..bd8dc16feb61 100644
--- a/drivers/hid/amd-sfh-hid/amd_sfh_common.h
+++ b/drivers/hid/amd-sfh-hid/amd_sfh_common.h
@@ -12,11 +12,15 @@
#include <linux/mutex.h>
#include <linux/pci.h>
+#include <linux/sizes.h>
#include "amd_sfh_hid.h"
#define PCI_DEVICE_ID_AMD_MP2 0x15E4
#define PCI_DEVICE_ID_AMD_MP2_1_1 0x164A
+/* The BAR 2 size must cover the highest register offset (0x10958) */
+#define AMD_SFH_MIN_BAR_SIZE SZ_128K
+
#define AMD_C2P_MSG(regno) (0x10500 + ((regno) * 4))
#define AMD_P2C_MSG(regno) (0x10680 + ((regno) * 4))
diff --git a/drivers/hid/amd-sfh-hid/amd_sfh_pcie.c b/drivers/hid/amd-sfh-hid/amd_sfh_pcie.c
index 4b81cebdc335..039b6ac327d3 100644
--- a/drivers/hid/amd-sfh-hid/amd_sfh_pcie.c
+++ b/drivers/hid/amd-sfh-hid/amd_sfh_pcie.c
@@ -451,6 +451,16 @@ static int amd_mp2_pci_probe(struct pci_dev *pdev, const struct pci_device_id *i
if (rc)
return rc;
+ if (!(pci_resource_flags(pdev, 2) & IORESOURCE_MEM)) {
+ dev_err(&pdev->dev, "BAR 2 is not IORESOURCE_MEM\n");
+ return -ENODEV;
+ }
+
+ if (pci_resource_len(pdev, 2) < AMD_SFH_MIN_BAR_SIZE) {
+ dev_err(&pdev->dev, "BAR 2 is too small\n");
+ return -EINVAL;
+ }
+
rc = pcim_iomap_regions(pdev, BIT(2), DRIVER_NAME);
if (rc)
return rc;
diff --git a/drivers/hid/bpf/hid_bpf_dispatch.c b/drivers/hid/bpf/hid_bpf_dispatch.c
index e6cf551b41e8..9fbbf59cefa1 100644
--- a/drivers/hid/bpf/hid_bpf_dispatch.c
+++ b/drivers/hid/bpf/hid_bpf_dispatch.c
@@ -362,7 +362,7 @@ hid_bpf_release_context(struct hid_bpf_ctx *ctx)
static int
__hid_bpf_hw_check_params(struct hid_bpf_ctx *ctx, __u8 *buf, size_t *buf__sz,
- enum hid_report_type rtype)
+ enum hid_report_type rtype, bool hw_request)
{
struct hid_report_enum *report_enum;
struct hid_report *report;
@@ -391,6 +391,10 @@ __hid_bpf_hw_check_params(struct hid_bpf_ctx *ctx, __u8 *buf, size_t *buf__sz,
report_len = hid_report_len(report);
+ /* unnumbered reports need to have a report ID reserved in the first byte */
+ if (hw_request && report_enum->numbered == 0)
+ report_len += 1;
+
if (*buf__sz > report_len)
*buf__sz = report_len;
@@ -423,7 +427,7 @@ hid_bpf_hw_request(struct hid_bpf_ctx *ctx, __u8 *buf, size_t buf__sz,
return -EDEADLOCK;
/* check arguments */
- ret = __hid_bpf_hw_check_params(ctx, buf, &size, rtype);
+ ret = __hid_bpf_hw_check_params(ctx, buf, &size, rtype, true);
if (ret)
return ret;
@@ -483,7 +487,7 @@ hid_bpf_hw_output_report(struct hid_bpf_ctx *ctx, __u8 *buf, size_t buf__sz)
return -EDEADLOCK;
/* check arguments */
- ret = __hid_bpf_hw_check_params(ctx, buf, &size, HID_OUTPUT_REPORT);
+ ret = __hid_bpf_hw_check_params(ctx, buf, &size, HID_OUTPUT_REPORT, true);
if (ret)
return ret;
@@ -509,7 +513,7 @@ __hid_bpf_input_report(struct hid_bpf_ctx *ctx, enum hid_report_type type, u8 *b
return -EDEADLOCK;
/* check arguments */
- ret = __hid_bpf_hw_check_params(ctx, buf, &size, type);
+ ret = __hid_bpf_hw_check_params(ctx, buf, &size, type, false);
if (ret)
return ret;
diff --git a/drivers/hid/hid-alps.c b/drivers/hid/hid-alps.c
index 8c3f83532ce9..0556cb5645eb 100644
--- a/drivers/hid/hid-alps.c
+++ b/drivers/hid/hid-alps.c
@@ -407,6 +407,8 @@ static int u1_raw_event(struct alps_dev *hdata, u8 *data, int size)
return 1;
case U1_SP_ABSOLUTE_REPORT_ID:
+ if (!hdata->input2)
+ return 0;
sp_x = get_unaligned_le16(data+2);
sp_y = get_unaligned_le16(data+4);
@@ -738,7 +740,6 @@ static int alps_input_configured(struct hid_device *hdev, struct hid_input *hi)
goto exit;
}
- data->input2 = input2;
input2->phys = input->phys;
input2->name = "DualPoint Stick";
input2->id.bustype = BUS_I2C;
@@ -762,11 +763,12 @@ static int alps_input_configured(struct hid_device *hdev, struct hid_input *hi)
__set_bit(INPUT_PROP_POINTER, input2->propbit);
__set_bit(INPUT_PROP_POINTING_STICK, input2->propbit);
- if (input_register_device(data->input2)) {
+ if (input_register_device(input2)) {
input_free_device(input2);
ret = -ENOENT;
goto exit;
}
+ data->input2 = input2;
}
exit:
@@ -823,6 +825,24 @@ static int alps_probe(struct hid_device *hdev, const struct hid_device_id *id)
return 0;
}
+static void alps_remove(struct hid_device *hdev)
+{
+ struct alps_dev *data = hid_get_drvdata(hdev);
+
+ /*
+ * input2 ("DualPoint Stick") is allocated separately and is not
+ * tracked in hdev->inputs, so the default remove path
+ * (hid_hw_stop -> hidinput_disconnect) does not unregister it.
+ *
+ * Stop the device first so that no URB callback can touch input2
+ * while it is being unregistered, then drop it explicitly.
+ */
+ hid_hw_stop(hdev);
+
+ if (data->input2)
+ input_unregister_device(data->input2);
+}
+
static const struct hid_device_id alps_id[] = {
{ HID_DEVICE(HID_BUS_ANY, HID_GROUP_ANY,
USB_VENDOR_ID_ALPS_JP, HID_DEVICE_ID_ALPS_U1_DUAL) },
@@ -843,10 +863,9 @@ static struct hid_driver alps_driver = {
.raw_event = alps_raw_event,
.input_mapping = alps_input_mapping,
.input_configured = alps_input_configured,
-#ifdef CONFIG_PM
- .resume = alps_post_resume,
- .reset_resume = alps_post_reset,
-#endif
+ .resume = pm_ptr(alps_post_resume),
+ .reset_resume = pm_ptr(alps_post_reset),
+ .remove = alps_remove,
};
module_hid_driver(alps_driver);
diff --git a/drivers/hid/hid-ids.h b/drivers/hid/hid-ids.h
index 96a9fa53e4ad..a9db2b30be9b 100644
--- a/drivers/hid/hid-ids.h
+++ b/drivers/hid/hid-ids.h
@@ -1232,6 +1232,9 @@
#define USB_DEVICE_ID_SAMSUNG_WIRELESS_UNIVERSAL_KBD 0xa006
#define USB_DEVICE_ID_SAMSUNG_WIRELESS_MULTI_HOGP_KBD 0xa064
+#define USB_VENDOR_ID_SDINNOVATION 0x36ae
+#define USB_DEVICE_ID_SDINNOVATION_GAMING_KBD 0xfeab
+
#define USB_VENDOR_ID_SEMICO 0x1a2c
#define USB_DEVICE_ID_SEMICO_USB_KEYKOARD 0x0023
#define USB_DEVICE_ID_SEMICO_USB_KEYKOARD2 0x0027
diff --git a/drivers/hid/hid-multitouch.c b/drivers/hid/hid-multitouch.c
index 35f9f4f76a55..4b736cd012db 100644
--- a/drivers/hid/hid-multitouch.c
+++ b/drivers/hid/hid-multitouch.c
@@ -79,6 +79,7 @@ MODULE_LICENSE("GPL");
#define MT_QUIRK_APPLE_TOUCHBAR BIT(23)
#define MT_QUIRK_YOGABOOK9I BIT(24)
#define MT_QUIRK_KEEP_LATENCY_ON_CLOSE BIT(25)
+#define MT_QUIRK_IGNORE_FEATURE_ID_MISMATCH BIT(26)
#define MT_INPUTMODE_TOUCHSCREEN 0x02
#define MT_INPUTMODE_TOUCHPAD 0x03
@@ -233,6 +234,7 @@ static void mt_post_parse(struct mt_device *td, struct mt_application *app);
#define MT_CLS_APPLE_TOUCHBAR 0x0114
#define MT_CLS_YOGABOOK9I 0x0115
#define MT_CLS_EGALAX_P80H84 0x0116
+#define MT_CLS_ASUS_ROG_Z13_FOLIO 0x0117
#define MT_CLS_SIS 0x0457
#define MT_DEFAULT_MAXCONTACT 10
@@ -403,6 +405,16 @@ static const struct mt_class mt_classes[] = {
.quirks = MT_QUIRK_ALWAYS_VALID |
MT_QUIRK_CONTACT_CNT_ACCURATE |
MT_QUIRK_ASUS_CUSTOM_UP },
+ { .name = MT_CLS_ASUS_ROG_Z13_FOLIO,
+ .quirks = MT_QUIRK_ALWAYS_VALID |
+ MT_QUIRK_IGNORE_DUPLICATES |
+ MT_QUIRK_HOVERING |
+ MT_QUIRK_CONTACT_CNT_ACCURATE |
+ MT_QUIRK_STICKY_FINGERS |
+ MT_QUIRK_WIN8_PTP_BUTTONS |
+ MT_QUIRK_CONFIDENCE |
+ MT_QUIRK_IGNORE_FEATURE_ID_MISMATCH,
+ .export_all_inputs = true },
{ .name = MT_CLS_VTL,
.quirks = MT_QUIRK_ALWAYS_VALID |
MT_QUIRK_CONTACT_CNT_ACCURATE |
@@ -505,6 +517,7 @@ static const struct attribute_group mt_attribute_group = {
static void mt_get_feature(struct hid_device *hdev, struct hid_report *report)
{
+ struct mt_device *td = hid_get_drvdata(hdev);
int ret;
u32 size = hid_report_len(report);
u8 *buf;
@@ -526,8 +539,14 @@ static void mt_get_feature(struct hid_device *hdev, struct hid_report *report)
dev_warn(&hdev->dev, "failed to fetch feature %d\n",
report->id);
} else {
- /* The report ID in the request and the response should match */
- if (report->id != buf[0]) {
+ /*
+ * The report ID in the request and the response should match.
+ * Some firmware (e.g. the ASUS ROG Z13 Folio
+ * touchpad) returns a mismatched ID on this specific fetch;
+ * tolerate it only for devices explicitly flagged as such.
+ */
+ if (report->id != buf[0] &&
+ !(td->mtclass.quirks & MT_QUIRK_IGNORE_FEATURE_ID_MISMATCH)) {
hid_err(hdev, "Returned feature report did not match the request\n");
goto free;
}
@@ -2716,6 +2735,12 @@ static const struct hid_device_id mt_devices[] = {
HID_DEVICE(BUS_I2C, HID_GROUP_MULTITOUCH_WIN_8,
I2C_VENDOR_ID_HANTICK, I2C_PRODUCT_ID_HANTICK_5288) },
+ /* Asus ROG Flow Z13 (2025) GZ302EA keyboard-cover touchpad */
+ { .driver_data = MT_CLS_ASUS_ROG_Z13_FOLIO,
+ HID_DEVICE(BUS_USB, HID_GROUP_MULTITOUCH_WIN_8,
+ USB_VENDOR_ID_ASUSTEK,
+ USB_DEVICE_ID_ASUSTEK_ROG_Z13_FOLIO) },
+
/* Generic MT device */
{ HID_DEVICE(HID_BUS_ANY, HID_GROUP_MULTITOUCH, HID_ANY_ID, HID_ANY_ID) },
diff --git a/drivers/hid/hid-quirks.c b/drivers/hid/hid-quirks.c
index ebf865c2dbb3..9d18b962e0ea 100644
--- a/drivers/hid/hid-quirks.c
+++ b/drivers/hid/hid-quirks.c
@@ -182,6 +182,7 @@ static const struct hid_device_id hid_quirks[] = {
{ HID_USB_DEVICE(USB_VENDOR_ID_SAITEK, USB_DEVICE_ID_SAITEK_X52_2), HID_QUIRK_INCREMENT_USAGE_ON_DUPLICATE },
{ HID_USB_DEVICE(USB_VENDOR_ID_SAITEK, USB_DEVICE_ID_SAITEK_X52_PRO), HID_QUIRK_INCREMENT_USAGE_ON_DUPLICATE },
{ HID_USB_DEVICE(USB_VENDOR_ID_SAITEK, USB_DEVICE_ID_SAITEK_X65), HID_QUIRK_INCREMENT_USAGE_ON_DUPLICATE },
+ { HID_USB_DEVICE(USB_VENDOR_ID_SDINNOVATION, USB_DEVICE_ID_SDINNOVATION_GAMING_KBD), HID_QUIRK_ALWAYS_POLL },
{ HID_USB_DEVICE(USB_VENDOR_ID_SEMICO, USB_DEVICE_ID_SEMICO_USB_KEYKOARD2), HID_QUIRK_NO_INIT_REPORTS },
{ HID_USB_DEVICE(USB_VENDOR_ID_SEMICO, USB_DEVICE_ID_SEMICO_USB_KEYKOARD), HID_QUIRK_NO_INIT_REPORTS },
{ HID_USB_DEVICE(USB_VENDOR_ID_SENNHEISER, USB_DEVICE_ID_SENNHEISER_BTD500USB), HID_QUIRK_NOGET },
@@ -421,7 +422,7 @@ static const struct hid_device_id hid_have_special_driver[] = {
#endif
#if IS_ENABLED(CONFIG_HID_ELECOM)
{ HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_ELECOM, USB_DEVICE_ID_ELECOM_BM084) },
- { HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_ELECOM, USB_DEVICE_ID_ELECOM_M_XGL20DLBK) },
+ { HID_USB_DEVICE(USB_VENDOR_ID_ELECOM, USB_DEVICE_ID_ELECOM_M_XGL20DLBK) },
{ HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_ELECOM, USB_DEVICE_ID_ELECOM_M_HT1MRBK_01AC) },
{ HID_USB_DEVICE(USB_VENDOR_ID_ELECOM, USB_DEVICE_ID_ELECOM_M_XT3URBK_00FB) },
{ HID_USB_DEVICE(USB_VENDOR_ID_ELECOM, USB_DEVICE_ID_ELECOM_M_XT3URBK_018F) },
diff --git a/drivers/hid/wacom_sys.c b/drivers/hid/wacom_sys.c
index e298ed9b502a..25dcd1bf8db0 100644
--- a/drivers/hid/wacom_sys.c
+++ b/drivers/hid/wacom_sys.c
@@ -117,8 +117,9 @@ static int wacom_wac_pen_serial_enforce(struct hid_device *hdev,
/* Queue events which have invalid tool type or serial number */
for (i = 0; i < report->maxfield; i++) {
- for (j = 0; j < report->field[i]->maxusage; j++) {
- struct hid_field *field = report->field[i];
+ struct hid_field *field = report->field[i];
+
+ for (j = 0; j < field->report_count; j++) {
struct hid_usage *usage = &field->usage[j];
unsigned int equivalent_usage = wacom_equivalent_usage(usage->hid);
unsigned int offset;
diff --git a/drivers/i2c/busses/i2c-qcom-geni.c b/drivers/i2c/busses/i2c-qcom-geni.c
index 6b2caaaa9437..307e2e4030af 100644
--- a/drivers/i2c/busses/i2c-qcom-geni.c
+++ b/drivers/i2c/busses/i2c-qcom-geni.c
@@ -77,6 +77,9 @@ enum geni_i2c_err_code {
#define XFER_TIMEOUT HZ
#define RST_TIMEOUT HZ
+#define GENI_SE_CLK_32MHZ (32 * HZ_PER_MHZ)
+#define GENI_SE_CLK_19P2MHZ 19200000UL
+
struct geni_i2c_dev {
struct geni_se se;
u32 tx_wm;
@@ -91,6 +94,7 @@ struct geni_i2c_dev {
struct clk *core_clk;
u32 clk_freq_out;
const struct geni_i2c_clk_fld *clk_fld;
+ u32 clk_idx;
int suspended;
void *dma_buf;
size_t xfer_len;
@@ -165,19 +169,44 @@ static const struct geni_i2c_clk_fld geni_i2c_clk_map_32mhz[] = {
static int geni_i2c_clk_map_idx(struct geni_i2c_dev *gi2c)
{
const struct geni_i2c_clk_fld *itr;
+ unsigned long res_freq;
- if (clk_get_rate(gi2c->se.clk) == 32 * HZ_PER_MHZ)
+ /*
+ * Frequency counter tables are calibrated for a specific source
+ * clock frequency and are not valid for any multiple of it
+ * (e.g. 64 MHz, 128 MHz).
+ * Use exact=true and verify res_freq matches req_freq literally
+ * to reject harmonics: a 64 MHz clock that divides evenly to
+ * 32 MHz would pass exact matching but produce double the intended
+ * I2C frequency with these counter values.
+ */
+ if (!geni_se_clk_freq_match(&gi2c->se, GENI_SE_CLK_32MHZ,
+ &gi2c->clk_idx, &res_freq, true) &&
+ res_freq == GENI_SE_CLK_32MHZ) {
itr = geni_i2c_clk_map_32mhz;
- else
+ } else if (!geni_se_clk_freq_match(&gi2c->se, GENI_SE_CLK_19P2MHZ,
+ &gi2c->clk_idx, &res_freq, true) &&
+ res_freq == GENI_SE_CLK_19P2MHZ) {
itr = geni_i2c_clk_map_19p2mhz;
+ } else {
+ dev_err(gi2c->se.dev,
+ "Unsupported SE source clock: must be exactly 32 MHz or 19.2 MHz\n");
+ return -EINVAL;
+ }
while (itr->clk_freq_out != 0) {
if (itr->clk_freq_out == gi2c->clk_freq_out) {
gi2c->clk_fld = itr;
+ dev_dbg(gi2c->se.dev,
+ "I2C clk selected: freq: %u Hz, clk_idx: %u\n",
+ gi2c->clk_freq_out, gi2c->clk_idx);
return 0;
}
itr++;
}
+
+ dev_err(gi2c->se.dev, "Unsupported I2C output frequency %u Hz\n", gi2c->clk_freq_out);
+
return -EINVAL;
}
@@ -186,7 +215,7 @@ static void qcom_geni_i2c_conf(struct geni_i2c_dev *gi2c)
const struct geni_i2c_clk_fld *itr = gi2c->clk_fld;
u32 val;
- writel_relaxed(0, gi2c->se.base + SE_GENI_CLK_SEL);
+ writel_relaxed(gi2c->clk_idx, gi2c->se.base + SE_GENI_CLK_SEL);
val = (itr->clk_div << CLK_DIV_SHFT) | SER_CLK_EN;
writel_relaxed(val, gi2c->se.base + GENI_SER_M_CLK_CFG);
@@ -823,8 +852,7 @@ static int geni_i2c_probe(struct platform_device *pdev)
ret = geni_i2c_clk_map_idx(gi2c);
if (ret)
- return dev_err_probe(dev, ret, "Invalid clk frequency %d Hz\n",
- gi2c->clk_freq_out);
+ return ret;
gi2c->adap.algo = &geni_i2c_algo;
init_completion(&gi2c->done);
diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
index 6b9509e93928..e555c5ce4bdf 100644
--- a/drivers/net/bonding/bond_main.c
+++ b/drivers/net/bonding/bond_main.c
@@ -492,7 +492,7 @@ static int bond_ipsec_add_sa(struct net_device *bond_dev,
!real_dev->xfrmdev_ops->xdo_dev_state_add ||
netif_is_bond_master(real_dev)) {
NL_SET_ERR_MSG_MOD(extack, "Slave does not support ipsec offload");
- err = -EINVAL;
+ err = -EOPNOTSUPP;
goto out;
}
diff --git a/drivers/net/dsa/mt7530-mdio.c b/drivers/net/dsa/mt7530-mdio.c
index 6cc2c8975af6..119fdbea5716 100644
--- a/drivers/net/dsa/mt7530-mdio.c
+++ b/drivers/net/dsa/mt7530-mdio.c
@@ -227,15 +227,17 @@ mt7530_remove(struct mdio_device *mdiodev)
if (!priv)
return;
- ret = regulator_disable(priv->core_pwr);
- if (ret < 0)
- dev_err(priv->dev,
- "Failed to disable core power: %d\n", ret);
-
- ret = regulator_disable(priv->io_pwr);
- if (ret < 0)
- dev_err(priv->dev, "Failed to disable io pwr: %d\n",
- ret);
+ if (priv->id == ID_MT7530) {
+ ret = regulator_disable(priv->core_pwr);
+ if (ret < 0)
+ dev_err(priv->dev,
+ "Failed to disable core power: %d\n", ret);
+
+ ret = regulator_disable(priv->io_pwr);
+ if (ret < 0)
+ dev_err(priv->dev, "Failed to disable io pwr: %d\n",
+ ret);
+ }
mt7530_remove_common(priv);
diff --git a/drivers/net/dsa/mt7530.c b/drivers/net/dsa/mt7530.c
index d7ebb7b8ee72..4001779a9159 100644
--- a/drivers/net/dsa/mt7530.c
+++ b/drivers/net/dsa/mt7530.c
@@ -3446,9 +3446,6 @@ EXPORT_SYMBOL_GPL(mt7530_probe_common);
void
mt7530_remove_common(struct mt7530_priv *priv)
{
- if (priv->irq_domain)
- mt7530_free_mdio_irq(priv);
-
dsa_unregister_switch(priv->ds);
mutex_destroy(&priv->reg_mutex);
diff --git a/drivers/net/ethernet/airoha/airoha_npu.c b/drivers/net/ethernet/airoha/airoha_npu.c
index ddc5a6a7681b..6319d31d4033 100644
--- a/drivers/net/ethernet/airoha/airoha_npu.c
+++ b/drivers/net/ethernet/airoha/airoha_npu.c
@@ -5,6 +5,7 @@
*/
#include <linux/devcoredump.h>
+#include <linux/devm-helpers.h>
#include <linux/firmware.h>
#include <linux/platform_device.h>
#include <linux/of_net.h>
@@ -669,12 +670,15 @@ static int airoha_npu_probe(struct platform_device *pdev)
if (irq < 0)
return irq;
+ err = devm_work_autocancel(dev, &core->wdt_work,
+ airoha_npu_wdt_work);
+ if (err)
+ return err;
+
err = devm_request_irq(dev, irq, airoha_npu_wdt_handler,
IRQF_SHARED, "airoha-npu-wdt", core);
if (err)
return err;
-
- INIT_WORK(&core->wdt_work, airoha_npu_wdt_work);
}
/* wlan IRQ lines */
@@ -716,18 +720,8 @@ static int airoha_npu_probe(struct platform_device *pdev)
return 0;
}
-static void airoha_npu_remove(struct platform_device *pdev)
-{
- struct airoha_npu *npu = platform_get_drvdata(pdev);
- int i;
-
- for (i = 0; i < ARRAY_SIZE(npu->cores); i++)
- cancel_work_sync(&npu->cores[i].wdt_work);
-}
-
static struct platform_driver airoha_npu_driver = {
.probe = airoha_npu_probe,
- .remove = airoha_npu_remove,
.driver = {
.name = "airoha-npu",
.of_match_table = of_airoha_npu_match,
diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
index 5d05020a6d05..e35eef9f6ce1 100644
--- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
+++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
@@ -4126,6 +4126,8 @@ static int ena_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
err_device_destroy:
ena_com_delete_host_info(ena_dev);
ena_com_admin_destroy(ena_dev);
+ ena_phc_destroy(adapter);
+ ena_com_mmio_reg_read_request_destroy(ena_dev);
ena_devlink_destroy:
ena_devlink_free(devlink);
err_metrics_destroy:
diff --git a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
index 7efa3fc257b3..e58f1d2c26bd 100644
--- a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
+++ b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
@@ -1602,6 +1602,9 @@ static int atl1c_clean_tx(struct napi_struct *napi, int budget)
AT_READ_REGW(&adapter->hw, atl1c_qregs[tpd_ring->num].tpd_cons,
&hw_next_to_clean);
+ if (unlikely(hw_next_to_clean >= tpd_ring->count))
+ hw_next_to_clean = next_to_clean;
+
while (next_to_clean != hw_next_to_clean) {
buffer_info = &tpd_ring->buffer_info[next_to_clean];
if (buffer_info->skb) {
diff --git a/drivers/net/ethernet/atheros/atl1e/atl1e_main.c b/drivers/net/ethernet/atheros/atl1e/atl1e_main.c
index 40290028580b..437989edb780 100644
--- a/drivers/net/ethernet/atheros/atl1e/atl1e_main.c
+++ b/drivers/net/ethernet/atheros/atl1e/atl1e_main.c
@@ -1234,6 +1234,9 @@ static bool atl1e_clean_tx_irq(struct atl1e_adapter *adapter)
u16 hw_next_to_clean = AT_READ_REGW(&adapter->hw, REG_TPD_CONS_IDX);
u16 next_to_clean = atomic_read(&tx_ring->next_to_clean);
+ if (unlikely(hw_next_to_clean >= tx_ring->count))
+ hw_next_to_clean = next_to_clean;
+
while (next_to_clean != hw_next_to_clean) {
tx_buffer = &tx_ring->tx_buffer[next_to_clean];
if (tx_buffer->dma) {
diff --git a/drivers/net/ethernet/atheros/atlx/atl1.c b/drivers/net/ethernet/atheros/atlx/atl1.c
index 98a4d089270e..957d5598dda5 100644
--- a/drivers/net/ethernet/atheros/atlx/atl1.c
+++ b/drivers/net/ethernet/atheros/atlx/atl1.c
@@ -2066,6 +2066,9 @@ static int atl1_intr_tx(struct atl1_adapter *adapter)
sw_tpd_next_to_clean = atomic_read(&tpd_ring->next_to_clean);
cmb_tpd_next_to_clean = le16_to_cpu(adapter->cmb.cmb->tpd_cons_idx);
+ if (unlikely(cmb_tpd_next_to_clean >= tpd_ring->count))
+ cmb_tpd_next_to_clean = sw_tpd_next_to_clean;
+
while (cmb_tpd_next_to_clean != sw_tpd_next_to_clean) {
buffer_info = &tpd_ring->buffer_info[sw_tpd_next_to_clean];
if (buffer_info->dma) {
diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.c b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
index 41539f9f8061..a72e136fd9c3 100644
--- a/drivers/net/ethernet/broadcom/genet/bcmgenet.c
+++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
@@ -853,7 +853,8 @@ static int bcmgenet_get_coalesce(struct net_device *dev,
ec->rx_max_coalesced_frames =
bcmgenet_rdma_ring_readl(priv, 0, DMA_MBUF_DONE_THRESH);
ec->rx_coalesce_usecs =
- bcmgenet_rdma_readl(priv, DMA_RING0_TIMEOUT) * 8192 / 1000;
+ (bcmgenet_rdma_readl(priv, DMA_RING0_TIMEOUT) &
+ DMA_TIMEOUT_MASK) * 8192 / 1000;
for (i = 0; i <= priv->hw_params->rx_queues; i++) {
ring = &priv->rx_rings[i];
@@ -1347,9 +1348,8 @@ static void bcmgenet_get_ethtool_stats(struct net_device *dev,
p = (char *)&stats64;
p += s->stat_offset;
- if (sizeof(unsigned long) != sizeof(u32) &&
- s->stat_sizeof == sizeof(unsigned long))
- data[i] = *(unsigned long *)p;
+ if (s->stat_sizeof == sizeof(u64))
+ data[i] = *(u64 *)p;
else
data[i] = *(u32 *)p;
}
@@ -1759,13 +1759,12 @@ static int bcmgenet_power_up(struct bcmgenet_priv *priv,
int ret = 0;
u32 reg;
- if (!bcmgenet_has_ext(priv))
- return ret;
-
- reg = bcmgenet_ext_readl(priv, EXT_EXT_PWR_MGMT);
-
switch (mode) {
case GENET_POWER_PASSIVE:
+ if (!bcmgenet_has_ext(priv))
+ break;
+
+ reg = bcmgenet_ext_readl(priv, EXT_EXT_PWR_MGMT);
reg &= ~(EXT_PWR_DOWN_DLL | EXT_PWR_DOWN_BIAS |
EXT_ENERGY_DET_MASK);
if (GENET_IS_V5(priv) && !bcmgenet_has_ephy_16nm(priv)) {
@@ -1789,8 +1788,12 @@ static int bcmgenet_power_up(struct bcmgenet_priv *priv,
break;
case GENET_POWER_CABLE_SENSE:
+ if (!bcmgenet_has_ext(priv))
+ break;
+
/* enable APD */
if (!GENET_IS_V5(priv)) {
+ reg = bcmgenet_ext_readl(priv, EXT_EXT_PWR_MGMT);
reg |= EXT_PWR_DN_EN_LD;
bcmgenet_ext_writel(priv, reg, EXT_EXT_PWR_MGMT);
}
@@ -3439,6 +3442,8 @@ static void bcmgenet_netif_stop(struct net_device *dev, bool stop_phy)
{
struct bcmgenet_priv *priv = netdev_priv(dev);
+ /* Stop completion polling before it can wake a stopped queue */
+ bcmgenet_disable_tx_napi(priv);
netif_tx_disable(dev);
/* Disable MAC receive */
@@ -3453,7 +3458,6 @@ static void bcmgenet_netif_stop(struct net_device *dev, bool stop_phy)
/* Disable MAC transmit. TX DMA disabled must be done before this */
umac_enable_set(priv, CMD_TX_EN, false);
- bcmgenet_disable_tx_napi(priv);
bcmgenet_disable_rx_napi(priv);
bcmgenet_intr_disable(priv);
@@ -3630,6 +3634,9 @@ static int bcmgenet_set_mac_addr(struct net_device *dev, void *p)
if (netif_running(dev))
return -EBUSY;
+ if (!is_valid_ether_addr(addr->sa_data))
+ return -EADDRNOTAVAIL;
+
eth_hw_addr_set(dev, addr->sa_data);
return 0;
@@ -4140,10 +4147,10 @@ static int bcmgenet_probe(struct platform_device *pdev)
priv->rx_rings[i].rx_max_coalesced_frames = 1;
/* Initialize u64 stats seq counter for 32bit machines */
- for (i = 0; i <= priv->hw_params->rx_queues; i++)
+ for (i = 0; i <= GENET_MAX_MQ_CNT; i++) {
u64_stats_init(&priv->rx_rings[i].stats64.syncp);
- for (i = 0; i <= priv->hw_params->tx_queues; i++)
u64_stats_init(&priv->tx_rings[i].stats64.syncp);
+ }
/* libphy will determine the link state */
netif_carrier_off(dev);
@@ -4325,6 +4332,8 @@ static int bcmgenet_suspend(struct device *d)
netif_device_detach(dev);
if (device_may_wakeup(d) && priv->wolopts) {
+ /* Stop completion polling before it can wake a stopped queue */
+ bcmgenet_disable_tx_napi(priv);
netif_tx_disable(dev);
/* Suspend non-wake Rx data flows */
@@ -4353,7 +4362,6 @@ static int bcmgenet_suspend(struct device *d)
netdev_warn(priv->dev,
"Timed out while disabling TX DMA\n");
- bcmgenet_disable_tx_napi(priv);
bcmgenet_disable_rx_napi(priv);
disable_irq(priv->irq1);
bcmgenet_tx_reclaim_all(dev);
diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index af21952373ea..50eb40de7b22 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -17928,11 +17928,14 @@ static int tg3_init_one(struct pci_dev *pdev,
err = tg3_get_device_address(tp, addr);
if (err) {
- dev_err(&pdev->dev,
- "Could not obtain valid ethernet address, aborting\n");
- goto err_out_apeunmap;
+ dev_warn_probe(&pdev->dev, err,
+ "Could not obtain a valid ethernet address\n");
+ if (err == -EPROBE_DEFER)
+ goto err_out_apeunmap;
+ eth_hw_addr_random(dev);
+ } else {
+ eth_hw_addr_set(dev, addr);
}
- eth_hw_addr_set(dev, addr);
intmbx = MAILBOX_INTERRUPT_0 + TG3_64BIT_REG_LOW;
rcvmbx = MAILBOX_RCVRET_CON_IDX_0 + TG3_64BIT_REG_LOW;
@@ -18060,6 +18063,10 @@ static int tg3_init_one(struct pci_dev *pdev,
return 0;
err_out_apeunmap:
+ if (tg3_flag(tp, USE_PHYLIB))
+ tg3_phy_fini(tp);
+ tg3_mdio_fini(tp);
+
if (tp->aperegs) {
iounmap(tp->aperegs);
tp->aperegs = NULL;
diff --git a/drivers/net/ethernet/brocade/bna/bnad.c b/drivers/net/ethernet/brocade/bna/bnad.c
index 9bed33295839..85868754d75a 100644
--- a/drivers/net/ethernet/brocade/bna/bnad.c
+++ b/drivers/net/ethernet/brocade/bna/bnad.c
@@ -2574,6 +2574,22 @@ bnad_ioceth_disable(struct bnad *bnad)
return err;
}
+/*
+ * The IOC timers rearm one another, so deleting one cannot stop a
+ * sibling callback from arming it again. Shut them down so a later
+ * mod_timer() is ignored.
+ */
+static void
+bnad_ioc_timers_shutdown(struct bnad *bnad)
+{
+ struct bfa_ioc *ioc = &bnad->bna.ioceth.ioc;
+
+ timer_shutdown_sync(&ioc->ioc_timer);
+ timer_shutdown_sync(&ioc->sem_timer);
+ timer_shutdown_sync(&ioc->hb_timer);
+ timer_shutdown_sync(&ioc->iocpf_timer);
+}
+
static int
bnad_ioceth_enable(struct bnad *bnad)
{
@@ -3730,9 +3746,7 @@ bnad_pci_probe(struct pci_dev *pdev,
bnad_res_free(bnad, &bnad->mod_res_info[0], BNA_MOD_RES_T_MAX);
disable_ioceth:
bnad_ioceth_disable(bnad);
- timer_delete_sync(&bnad->bna.ioceth.ioc.ioc_timer);
- timer_delete_sync(&bnad->bna.ioceth.ioc.sem_timer);
- timer_delete_sync(&bnad->bna.ioceth.ioc.hb_timer);
+ bnad_ioc_timers_shutdown(bnad);
spin_lock_irqsave(&bnad->bna_lock, flags);
bna_uninit(bna);
spin_unlock_irqrestore(&bnad->bna_lock, flags);
@@ -3773,9 +3787,7 @@ bnad_pci_remove(struct pci_dev *pdev)
mutex_lock(&bnad->conf_mutex);
bnad_ioceth_disable(bnad);
- timer_delete_sync(&bnad->bna.ioceth.ioc.ioc_timer);
- timer_delete_sync(&bnad->bna.ioceth.ioc.sem_timer);
- timer_delete_sync(&bnad->bna.ioceth.ioc.hb_timer);
+ bnad_ioc_timers_shutdown(bnad);
spin_lock_irqsave(&bnad->bna_lock, flags);
bna_uninit(bna);
spin_unlock_irqrestore(&bnad->bna_lock, flags);
diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
index 02fd4bd04487..1eaf9be48315 100644
--- a/drivers/net/ethernet/cadence/macb_main.c
+++ b/drivers/net/ethernet/cadence/macb_main.c
@@ -2663,14 +2663,24 @@ static int macb_alloc_consistent(struct macb *bp)
size = bp->num_queues * macb_tx_ring_size_per_queue(bp);
tx = dma_alloc_coherent(dev, size, &tx_dma, GFP_KERNEL);
- if (!tx || upper_32_bits(tx_dma) != upper_32_bits(tx_dma + size - 1))
+ if (!tx)
+ goto out_err;
+ /* Record the buffer so that the error path frees it. */
+ bp->queues[0].tx_ring = tx;
+ bp->queues[0].tx_ring_dma = tx_dma;
+ if (upper_32_bits(tx_dma) != upper_32_bits(tx_dma + size - 1))
goto out_err;
netdev_dbg(bp->dev, "Allocated %zu bytes for %u TX rings at %08lx (mapped %p)\n",
size, bp->num_queues, (unsigned long)tx_dma, tx);
size = bp->num_queues * macb_rx_ring_size_per_queue(bp);
rx = dma_alloc_coherent(dev, size, &rx_dma, GFP_KERNEL);
- if (!rx || upper_32_bits(rx_dma) != upper_32_bits(rx_dma + size - 1))
+ if (!rx)
+ goto out_err;
+ /* Record the buffer so that the error path frees it. */
+ bp->queues[0].rx_ring = rx;
+ bp->queues[0].rx_ring_dma = rx_dma;
+ if (upper_32_bits(rx_dma) != upper_32_bits(rx_dma + size - 1))
goto out_err;
netdev_dbg(bp->dev, "Allocated %zu bytes for %u RX rings at %08lx (mapped %p)\n",
size, bp->num_queues, (unsigned long)rx_dma, rx);
diff --git a/drivers/net/ethernet/freescale/fman/fman.c b/drivers/net/ethernet/freescale/fman/fman.c
index b96d4e03f169..cb05f8cc6043 100644
--- a/drivers/net/ethernet/freescale/fman/fman.c
+++ b/drivers/net/ethernet/freescale/fman/fman.c
@@ -2734,6 +2734,7 @@ static struct fman *read_dts_node(struct platform_device *of_dev)
}
clk_rate = clk_get_rate(clk);
+ clk_put(clk);
if (!clk_rate) {
err = -EINVAL;
dev_err(&of_dev->dev, "%s: Failed to determine FM%d clock rate\n",
diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h
index 9e7d6860b83f..f0bc910dd6d4 100644
--- a/drivers/net/ethernet/google/gve/gve.h
+++ b/drivers/net/ethernet/google/gve/gve.h
@@ -1172,6 +1172,12 @@ static inline bool gve_is_gqi(struct gve_priv *priv)
priv->queue_format == GVE_GQI_QPL_FORMAT;
}
+static inline bool gve_is_dqo(struct gve_priv *priv)
+{
+ return priv->queue_format == GVE_DQO_RDA_FORMAT ||
+ priv->queue_format == GVE_DQO_QPL_FORMAT;
+}
+
static inline u32 gve_num_tx_queues(struct gve_priv *priv)
{
return priv->tx_cfg.num_queues + priv->tx_cfg.num_xdp_queues;
@@ -1254,7 +1260,11 @@ void gve_rx_start_ring_gqi(struct gve_priv *priv, int idx);
void gve_rx_stop_ring_gqi(struct gve_priv *priv, int idx);
u16 gve_get_pkt_buf_size(const struct gve_priv *priv, bool enable_hplit);
bool gve_header_split_supported(const struct gve_priv *priv);
-int gve_set_hsplit_config(struct gve_priv *priv, u8 tcp_data_split);
+int gve_set_rx_buf_len_config(struct gve_priv *priv, u32 rx_buf_len,
+ struct netlink_ext_ack *extack,
+ struct gve_rx_alloc_rings_cfg *rx_alloc_cfg);
+int gve_set_hsplit_config(struct gve_priv *priv, u8 tcp_data_split,
+ struct gve_rx_alloc_rings_cfg *rx_alloc_cfg);
/* rx buffer handling */
int gve_buf_ref_cnt(struct gve_rx_buf_state_dqo *bs);
void gve_free_page_dqo(struct gve_priv *priv, struct gve_rx_buf_state_dqo *bs,
diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/ethernet/google/gve/gve_adminq.c
index 4f33d094a2ef..367a9f0e89b5 100644
--- a/drivers/net/ethernet/google/gve/gve_adminq.c
+++ b/drivers/net/ethernet/google/gve/gve_adminq.c
@@ -791,7 +791,7 @@ static void gve_adminq_get_create_rx_queue_cmd(struct gve_priv *priv,
cmd->create_rx_queue.rx_buff_ring_size =
cpu_to_be16(priv->rx_desc_cnt);
cmd->create_rx_queue.enable_rsc =
- !!(priv->dev->features & NETIF_F_LRO);
+ !!(priv->dev->features & NETIF_F_GRO_HW);
if (priv->header_split_enabled)
cmd->create_rx_queue.header_buffer_size =
cpu_to_be16(priv->header_buf_size);
@@ -1123,9 +1123,13 @@ int gve_adminq_describe_device(struct gve_priv *priv)
gve_set_default_rss_sizes(priv);
- /* DQO supports LRO. */
- if (!gve_is_gqi(priv))
- priv->dev->hw_features |= NETIF_F_LRO;
+ /* DQO supports HW-GRO and UDP_GSO */
+ if (gve_is_dqo(priv)) {
+ u64 additional_features = NETIF_F_GRO_HW | NETIF_F_GSO_UDP_L4;
+
+ priv->dev->hw_features |= additional_features;
+ priv->dev->features |= additional_features;
+ }
priv->max_registered_pages =
be64_to_cpu(descriptor->max_registered_pages);
diff --git a/drivers/net/ethernet/google/gve/gve_desc_dqo.h b/drivers/net/ethernet/google/gve/gve_desc_dqo.h
index f7786b03c744..d2c86c8eeae2 100644
--- a/drivers/net/ethernet/google/gve/gve_desc_dqo.h
+++ b/drivers/net/ethernet/google/gve/gve_desc_dqo.h
@@ -14,6 +14,11 @@
#define GVE_TX_MAX_HDR_SIZE_DQO 255
#define GVE_TX_MIN_TSO_MSS_DQO 88
+/* HW limit. This also has to fit in the 14 bits of the mss field of
+ * struct gve_tx_tso_context_desc_dqo.
+ */
+#define GVE_TX_MAX_TSO_MSS_DQO 9728
+
#ifndef __LITTLE_ENDIAN_BITFIELD
#error "Only little endian supported"
#endif
diff --git a/drivers/net/ethernet/google/gve/gve_ethtool.c b/drivers/net/ethernet/google/gve/gve_ethtool.c
index edc0c718f71e..7135112eb870 100644
--- a/drivers/net/ethernet/google/gve/gve_ethtool.c
+++ b/drivers/net/ethernet/google/gve/gve_ethtool.c
@@ -554,6 +554,8 @@ static void gve_get_ringparam(struct net_device *netdev,
cmd->rx_pending = priv->rx_desc_cnt;
cmd->tx_pending = priv->tx_desc_cnt;
+ kernel_cmd->rx_buf_len = priv->rx_cfg.packet_buffer_size;
+
if (!gve_header_split_supported(priv))
kernel_cmd->tcp_data_split = ETHTOOL_TCP_DATA_SPLIT_UNKNOWN;
else if (priv->header_split_enabled)
@@ -562,34 +564,6 @@ static void gve_get_ringparam(struct net_device *netdev,
kernel_cmd->tcp_data_split = ETHTOOL_TCP_DATA_SPLIT_DISABLED;
}
-static int gve_adjust_ring_sizes(struct gve_priv *priv,
- u16 new_tx_desc_cnt,
- u16 new_rx_desc_cnt)
-{
- struct gve_tx_alloc_rings_cfg tx_alloc_cfg = {0};
- struct gve_rx_alloc_rings_cfg rx_alloc_cfg = {0};
- int err;
-
- /* get current queue configuration */
- gve_get_curr_alloc_cfgs(priv, &tx_alloc_cfg, &rx_alloc_cfg);
-
- /* copy over the new ring_size from ethtool */
- tx_alloc_cfg.ring_size = new_tx_desc_cnt;
- rx_alloc_cfg.ring_size = new_rx_desc_cnt;
-
- if (netif_running(priv->dev)) {
- err = gve_adjust_config(priv, &tx_alloc_cfg, &rx_alloc_cfg);
- if (err)
- return err;
- }
-
- /* Set new ring_size for the next up */
- priv->tx_desc_cnt = new_tx_desc_cnt;
- priv->rx_desc_cnt = new_rx_desc_cnt;
-
- return 0;
-}
-
static int gve_validate_req_ring_size(struct gve_priv *priv, u16 new_tx_desc_cnt,
u16 new_rx_desc_cnt)
{
@@ -609,34 +583,70 @@ static int gve_validate_req_ring_size(struct gve_priv *priv, u16 new_tx_desc_cnt
return 0;
}
+static int gve_set_ring_sizes_config(struct gve_priv *priv, u16 new_tx_desc_cnt,
+ u16 new_rx_desc_cnt,
+ struct gve_tx_alloc_rings_cfg *tx_alloc_cfg,
+ struct gve_rx_alloc_rings_cfg *rx_alloc_cfg)
+{
+ if (new_tx_desc_cnt == priv->tx_desc_cnt &&
+ new_rx_desc_cnt == priv->rx_desc_cnt)
+ return 0;
+
+ if (!priv->modify_ring_size_enabled) {
+ dev_err(&priv->pdev->dev, "Modify ring size is not supported.\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (gve_validate_req_ring_size(priv, new_tx_desc_cnt, new_rx_desc_cnt))
+ return -EINVAL;
+
+ tx_alloc_cfg->ring_size = new_tx_desc_cnt;
+ rx_alloc_cfg->ring_size = new_rx_desc_cnt;
+ return 0;
+}
+
static int gve_set_ringparam(struct net_device *netdev,
struct ethtool_ringparam *cmd,
struct kernel_ethtool_ringparam *kernel_cmd,
struct netlink_ext_ack *extack)
{
+ struct gve_tx_alloc_rings_cfg tx_alloc_cfg = {0};
+ struct gve_rx_alloc_rings_cfg rx_alloc_cfg = {0};
struct gve_priv *priv = netdev_priv(netdev);
- u16 new_tx_cnt, new_rx_cnt;
int err;
- err = gve_set_hsplit_config(priv, kernel_cmd->tcp_data_split);
+ gve_get_curr_alloc_cfgs(priv, &tx_alloc_cfg, &rx_alloc_cfg);
+
+ err = gve_set_rx_buf_len_config(priv, kernel_cmd->rx_buf_len, extack,
+ &rx_alloc_cfg);
if (err)
return err;
- if (cmd->tx_pending == priv->tx_desc_cnt && cmd->rx_pending == priv->rx_desc_cnt)
- return 0;
-
- if (!priv->modify_ring_size_enabled) {
- dev_err(&priv->pdev->dev, "Modify ring size is not supported.\n");
- return -EOPNOTSUPP;
- }
-
- new_tx_cnt = cmd->tx_pending;
- new_rx_cnt = cmd->rx_pending;
+ err = gve_set_hsplit_config(priv, kernel_cmd->tcp_data_split,
+ &rx_alloc_cfg);
+ if (err)
+ return err;
- if (gve_validate_req_ring_size(priv, new_tx_cnt, new_rx_cnt))
- return -EINVAL;
+ err = gve_set_ring_sizes_config(priv, cmd->tx_pending, cmd->rx_pending,
+ &tx_alloc_cfg, &rx_alloc_cfg);
+ if (err)
+ return err;
- return gve_adjust_ring_sizes(priv, new_tx_cnt, new_rx_cnt);
+ if (netif_running(priv->dev)) {
+ err = gve_adjust_config(priv, &tx_alloc_cfg, &rx_alloc_cfg);
+ if (err)
+ return err;
+ } else {
+ /* Set ring params for the next up */
+ priv->rx_cfg.packet_buffer_size =
+ rx_alloc_cfg.packet_buffer_size;
+ priv->header_split_enabled = rx_alloc_cfg.enable_header_split;
+ priv->rx_cfg.packet_buffer_size =
+ rx_alloc_cfg.packet_buffer_size;
+ priv->tx_desc_cnt = tx_alloc_cfg.ring_size;
+ priv->rx_desc_cnt = rx_alloc_cfg.ring_size;
+ }
+ return 0;
}
static int gve_user_reset(struct net_device *netdev, u32 *flags)
@@ -971,7 +981,8 @@ static int gve_get_ts_info(struct net_device *netdev,
const struct ethtool_ops gve_ethtool_ops = {
.supported_coalesce_params = ETHTOOL_COALESCE_USECS,
- .supported_ring_params = ETHTOOL_RING_USE_TCP_DATA_SPLIT,
+ .supported_ring_params = ETHTOOL_RING_USE_TCP_DATA_SPLIT |
+ ETHTOOL_RING_USE_RX_BUF_LEN,
.get_drvinfo = gve_get_drvinfo,
.get_strings = gve_get_strings,
.get_sset_count = gve_get_sset_count,
diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c
index 975104984943..5fbb02d14dc3 100644
--- a/drivers/net/ethernet/google/gve/gve_main.c
+++ b/drivers/net/ethernet/google/gve/gve_main.c
@@ -1707,18 +1707,28 @@ static int gve_xsk_wakeup(struct net_device *dev, u32 queue_id, u32 flags)
return 0;
}
-static int verify_xdp_configuration(struct net_device *dev)
+static int gve_verify_xdp_configuration(struct net_device *dev,
+ struct netlink_ext_ack *extack)
{
struct gve_priv *priv = netdev_priv(dev);
u16 max_xdp_mtu;
- if (dev->features & NETIF_F_LRO) {
- netdev_warn(dev, "XDP is not supported when LRO is on.\n");
+ if (dev->features & NETIF_F_GRO_HW) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "XDP is not supported when HW-GRO is on.");
return -EOPNOTSUPP;
}
if (priv->header_split_enabled) {
- netdev_warn(dev, "XDP is not supported when header-data split is enabled.\n");
+ NL_SET_ERR_MSG_MOD(extack,
+ "XDP is not supported when header-data split is enabled.");
+ return -EOPNOTSUPP;
+ }
+
+ if (priv->rx_cfg.packet_buffer_size != SZ_2K) {
+ NL_SET_ERR_MSG_FMT_MOD(extack,
+ "XDP is not supported for Rx buf len %d, only %d supported.",
+ priv->rx_cfg.packet_buffer_size, SZ_2K);
return -EOPNOTSUPP;
}
@@ -1727,17 +1737,20 @@ static int verify_xdp_configuration(struct net_device *dev)
max_xdp_mtu -= GVE_RX_PAD;
if (dev->mtu > max_xdp_mtu) {
- netdev_warn(dev, "XDP is not supported for mtu %d.\n",
- dev->mtu);
+ NL_SET_ERR_MSG_FMT_MOD(extack,
+ "XDP is not supported for mtu %d.",
+ dev->mtu);
return -EOPNOTSUPP;
}
if (priv->rx_cfg.num_queues != priv->tx_cfg.num_queues ||
(2 * priv->tx_cfg.num_queues > priv->tx_cfg.max_queues)) {
- netdev_warn(dev, "XDP load failed: The number of configured RX queues %d should be equal to the number of configured TX queues %d and the number of configured RX/TX queues should be less than or equal to half the maximum number of RX/TX queues %d",
- priv->rx_cfg.num_queues,
- priv->tx_cfg.num_queues,
+ netdev_warn(dev,
+ "XDP load failed: The number of configured RX queues %d should be equal to the number of configured TX queues %d and the number of configured RX/TX queues should be less than or equal to half the maximum number of RX/TX queues %d.",
+ priv->rx_cfg.num_queues, priv->tx_cfg.num_queues,
priv->tx_cfg.max_queues);
+ NL_SET_ERR_MSG_MOD(extack,
+ "XDP load failed: The number of configured RX queues should be equal to the number of configured TX queues and the number of configured RX/TX queues should be less than or equal to half the maximum number of RX/TX queues");
return -EINVAL;
}
return 0;
@@ -1748,7 +1761,7 @@ static int gve_xdp(struct net_device *dev, struct netdev_bpf *xdp)
struct gve_priv *priv = netdev_priv(dev);
int err;
- err = verify_xdp_configuration(dev);
+ err = gve_verify_xdp_configuration(dev, xdp->extack);
if (err)
return err;
switch (xdp->command) {
@@ -2058,12 +2071,42 @@ bool gve_header_split_supported(const struct gve_priv *priv)
priv->queue_format == GVE_DQO_RDA_FORMAT && !priv->xdp_prog;
}
-int gve_set_hsplit_config(struct gve_priv *priv, u8 tcp_data_split)
+int gve_set_rx_buf_len_config(struct gve_priv *priv, u32 rx_buf_len,
+ struct netlink_ext_ack *extack,
+ struct gve_rx_alloc_rings_cfg *rx_alloc_cfg)
+{
+ u32 old_rx_buf_len = rx_alloc_cfg->packet_buffer_size;
+
+ if (rx_buf_len == old_rx_buf_len)
+ return 0;
+
+ /* device options may not always contain support for 4K buffers */
+ if (!gve_is_dqo(priv) || priv->max_rx_buffer_size < SZ_4K) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Modifying Rx buf len is not supported");
+ return -EOPNOTSUPP;
+ }
+
+ if (priv->xdp_prog && rx_buf_len != SZ_2K) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Rx buf len can only be 2048 when XDP is on");
+ return -EINVAL;
+ }
+
+ if (rx_buf_len != SZ_2K && rx_buf_len != SZ_4K) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Rx buf len can only be 2048 or 4096");
+ return -EINVAL;
+ }
+ rx_alloc_cfg->packet_buffer_size = rx_buf_len;
+
+ return 0;
+}
+
+int gve_set_hsplit_config(struct gve_priv *priv, u8 tcp_data_split,
+ struct gve_rx_alloc_rings_cfg *rx_alloc_cfg)
{
- struct gve_tx_alloc_rings_cfg tx_alloc_cfg = {0};
- struct gve_rx_alloc_rings_cfg rx_alloc_cfg = {0};
bool enable_hdr_split;
- int err = 0;
if (tcp_data_split == ETHTOOL_TCP_DATA_SPLIT_UNKNOWN)
return 0;
@@ -2081,14 +2124,11 @@ int gve_set_hsplit_config(struct gve_priv *priv, u8 tcp_data_split)
if (enable_hdr_split == priv->header_split_enabled)
return 0;
- gve_get_curr_alloc_cfgs(priv, &tx_alloc_cfg, &rx_alloc_cfg);
-
- rx_alloc_cfg.enable_header_split = enable_hdr_split;
- rx_alloc_cfg.packet_buffer_size = gve_get_pkt_buf_size(priv, enable_hdr_split);
+ rx_alloc_cfg->enable_header_split = enable_hdr_split;
+ rx_alloc_cfg->packet_buffer_size =
+ gve_get_pkt_buf_size(priv, enable_hdr_split);
- if (netif_running(priv->dev))
- err = gve_adjust_config(priv, &tx_alloc_cfg, &rx_alloc_cfg);
- return err;
+ return 0;
}
static int gve_set_features(struct net_device *netdev,
@@ -2102,12 +2142,13 @@ static int gve_set_features(struct net_device *netdev,
gve_get_curr_alloc_cfgs(priv, &tx_alloc_cfg, &rx_alloc_cfg);
- if ((netdev->features & NETIF_F_LRO) != (features & NETIF_F_LRO)) {
- netdev->features ^= NETIF_F_LRO;
- if (priv->xdp_prog && (netdev->features & NETIF_F_LRO)) {
+ if ((netdev->features & NETIF_F_GRO_HW) !=
+ (features & NETIF_F_GRO_HW)) {
+ netdev->features ^= NETIF_F_GRO_HW;
+ if (priv->xdp_prog && (netdev->features & NETIF_F_GRO_HW)) {
netdev_warn(netdev,
- "XDP is not supported when LRO is on.\n");
- err = -EOPNOTSUPP;
+ "HW-GRO is not supported when XDP is on.");
+ err = -EOPNOTSUPP;
goto revert_features;
}
if (netif_running(netdev)) {
diff --git a/drivers/net/ethernet/google/gve/gve_tx_dqo.c b/drivers/net/ethernet/google/gve/gve_tx_dqo.c
index 5fbee184f0e7..b5565d0b197f 100644
--- a/drivers/net/ethernet/google/gve/gve_tx_dqo.c
+++ b/drivers/net/ethernet/google/gve/gve_tx_dqo.c
@@ -575,34 +575,52 @@ static void gve_tx_fill_pkt_desc_dqo(struct gve_tx_ring *tx, u32 *desc_idx,
*/
static int gve_prep_tso(struct sk_buff *skb)
{
+ struct skb_shared_info *shinfo = skb_shinfo(skb);
+ u32 paylen, l4_start;
struct tcphdr *tcp;
+ struct udphdr *udp;
int header_len;
- u32 paylen;
int err;
- /* Note: HW requires MSS (gso_size) to be <= 9728 and the total length
- * of the TSO to be <= 262143.
+ /* Note: HW requires the total length of the TSO to be <= 262143,
+ * this is enforced by netif_set_tso_max_size().
*
- * However, we don't validate these because:
- * - Hypervisor enforces a limit of 9K MTU
- * - Kernel will not produce a TSO larger than 64k
+ * MSS (gso_size) can not be trusted: packets forwarded from a tap or
+ * injected by a packet socket can carry an arbitrary value, while the
+ * mss field of the TSO context descriptor is only 14 bits wide.
+ *
+ * A too big MSS is dropped here instead of being rejected from
+ * gve_features_check_dqo(), because software segmentation would
+ * produce packets larger than the device can send.
*/
-
- if (unlikely(skb_shinfo(skb)->gso_size < GVE_TX_MIN_TSO_MSS_DQO))
+ if (unlikely(shinfo->gso_size > GVE_TX_MAX_TSO_MSS_DQO))
return -1;
- if (!(skb_shinfo(skb)->gso_type & (SKB_GSO_TCPV4 | SKB_GSO_TCPV6)))
- return -EINVAL;
-
/* Needed because we will modify header. */
err = skb_cow_head(skb, 0);
if (err < 0)
return err;
- tcp = tcp_hdr(skb);
- paylen = skb->len - skb_transport_offset(skb);
- csum_replace_by_diff(&tcp->check, (__force __wsum)htonl(paylen));
- header_len = skb_tcp_all_headers(skb);
+ l4_start = skb_transport_offset(skb);
+ paylen = skb->len - l4_start;
+
+ switch (shinfo->gso_type) {
+ case SKB_GSO_TCPV4:
+ case SKB_GSO_TCPV6:
+ tcp = tcp_hdr(skb);
+ csum_replace_by_diff(&tcp->check,
+ (__force __wsum)htonl(paylen));
+ header_len = skb_tcp_all_headers(skb);
+ break;
+ case SKB_GSO_UDP_L4:
+ udp = udp_hdr(skb);
+ csum_replace_by_diff(&udp->check,
+ (__force __wsum)htonl(paylen));
+ header_len = sizeof(struct udphdr) + l4_start;
+ break;
+ default:
+ return -EINVAL;
+ }
if (unlikely(header_len > GVE_TX_MAX_HDR_SIZE_DQO))
return -EINVAL;
@@ -908,13 +926,22 @@ static bool gve_can_send_tso(const struct sk_buff *skb)
{
const int max_bufs_per_seg = GVE_TX_MAX_DATA_DESCS - 1;
const struct skb_shared_info *shinfo = skb_shinfo(skb);
- const int header_len = skb_tcp_all_headers(skb);
const int gso_size = shinfo->gso_size;
int cur_seg_num_bufs;
int prev_frag_size;
int cur_seg_size;
+ int header_len;
int i;
+ if (unlikely(gso_size < GVE_TX_MIN_TSO_MSS_DQO))
+ return false;
+
+ /* Must match the header length programmed by gve_prep_tso(). */
+ if (skb_is_gso_tcp(skb))
+ header_len = skb_tcp_all_headers(skb);
+ else
+ header_len = skb_transport_offset(skb) + sizeof(struct udphdr);
+
cur_seg_size = skb_headlen(skb) - header_len;
prev_frag_size = skb_headlen(skb);
cur_seg_num_bufs = cur_seg_size > 0;
@@ -956,7 +983,17 @@ netdev_features_t gve_features_check_dqo(struct sk_buff *skb,
struct net_device *dev,
netdev_features_t features)
{
- if (skb_is_gso(skb) && !gve_can_send_tso(skb))
+ if (!skb_is_gso(skb))
+ return features;
+
+ /* Keep the GSO bits for a too big MSS, so that gve_prep_tso() drops
+ * the packet: software segmentation would give packets larger than
+ * the device can send.
+ */
+ if (skb_shinfo(skb)->gso_size > GVE_TX_MAX_TSO_MSS_DQO)
+ return features;
+
+ if (!gve_can_send_tso(skb))
return features & ~NETIF_F_GSO_MASK;
return features;
diff --git a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_mac.c b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_mac.c
index bc6b269be299..f1cb6d56e4b9 100644
--- a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_mac.c
+++ b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_mac.c
@@ -793,6 +793,7 @@ static int hns_mac_register_phy(struct hns_mac_cb *mac_cb)
dev_err(mac_cb->dev,
"mac%d mdio is NULL, dsaf will probe again later\n",
mac_cb->mac_id);
+ put_device(&pdev->dev);
return -EPROBE_DEFER;
}
@@ -801,6 +802,8 @@ static int hns_mac_register_phy(struct hns_mac_cb *mac_cb)
dev_dbg(mac_cb->dev, "mac%d register phy addr:%d\n",
mac_cb->mac_id, addr);
+ put_device(&pdev->dev);
+
return rc;
}
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/common.h b/drivers/net/ethernet/marvell/octeontx2/af/common.h
index 8a08bebf08c2..e6c6e03c3f29 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/common.h
+++ b/drivers/net/ethernet/marvell/octeontx2/af/common.h
@@ -7,6 +7,10 @@
#ifndef COMMON_H
#define COMMON_H
+#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
+#include <linux/mm.h>
+
#include "rvu_struct.h"
#define OTX2_ALIGN 128 /* Align to cacheline */
@@ -44,6 +48,33 @@ struct qmem {
u32 qsize;
};
+static inline void *otx2_dma_alloc_coherent(struct device *dev, size_t size,
+ dma_addr_t *dma_handle)
+{
+ dma_addr_t dma_addr;
+ void *vaddr;
+
+ vaddr = kzalloc(size, GFP_KERNEL);
+ if (!vaddr)
+ return NULL;
+
+ dma_addr = dma_map_single(dev, vaddr, size, DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(dev, dma_addr)) {
+ kfree(vaddr);
+ return NULL;
+ }
+
+ *dma_handle = dma_addr;
+ return vaddr;
+}
+
+static inline void otx2_dma_free_coherent(struct device *dev, size_t size,
+ void *vaddr, dma_addr_t dma_handle)
+{
+ dma_unmap_single(dev, dma_handle, size, DMA_BIDIRECTIONAL);
+ kfree(vaddr);
+}
+
static inline int qmem_alloc(struct device *dev, struct qmem **q,
int qsize, int entry_sz)
{
@@ -60,8 +91,11 @@ static inline int qmem_alloc(struct device *dev, struct qmem **q,
qmem->entry_sz = entry_sz;
qmem->alloc_sz = (qsize * entry_sz) + OTX2_ALIGN;
- qmem->base = dma_alloc_attrs(dev, qmem->alloc_sz, &qmem->iova,
- GFP_KERNEL, DMA_ATTR_FORCE_CONTIGUOUS);
+
+ if (get_order(PAGE_ALIGN(qmem->alloc_sz)) > MAX_PAGE_ORDER)
+ return -ENOMEM;
+
+ qmem->base = otx2_dma_alloc_coherent(dev, qmem->alloc_sz, &qmem->iova);
if (!qmem->base)
return -ENOMEM;
@@ -80,10 +114,9 @@ static inline void qmem_free(struct device *dev, struct qmem *qmem)
return;
if (qmem->base)
- dma_free_attrs(dev, qmem->alloc_sz,
- qmem->base - qmem->align,
- qmem->iova - qmem->align,
- DMA_ATTR_FORCE_CONTIGUOUS);
+ otx2_dma_free_coherent(dev, qmem->alloc_sz,
+ qmem->base - qmem->align,
+ qmem->iova - qmem->align);
devm_kfree(dev, qmem);
}
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
index 0a1c778d14b6..f7033765ed3f 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
@@ -710,110 +710,72 @@ static int get_max_column_width(struct rvu *rvu)
}
/* Dumps current provisioning status of all RVU block LFs */
-static ssize_t rvu_dbg_rsrc_attach_status(struct file *filp,
- char __user *buffer,
- size_t count, loff_t *ppos)
+static int rvu_dbg_rsrc_attach_status(struct seq_file *filp, void *unused)
{
- int index, off = 0, flag = 0, len = 0, i = 0;
- struct rvu *rvu = filp->private_data;
- int bytes_not_copied = 0;
+ struct rvu *rvu = filp->private;
+ int index, pf, vf, pcifunc;
struct rvu_block block;
- int pf, vf, pcifunc;
- int buf_size = 2048;
int lf_str_size;
char *lfs;
- char *buf;
-
- /* don't allow partial reads */
- if (*ppos != 0)
- return 0;
-
- buf = kzalloc(buf_size, GFP_KERNEL);
- if (!buf)
- return -ENOMEM;
- /* Get the maximum width of a column */
lf_str_size = get_max_column_width(rvu);
+ if (lf_str_size < 0)
+ return lf_str_size;
lfs = kzalloc(lf_str_size, GFP_KERNEL);
- if (!lfs) {
- kfree(buf);
+ if (!lfs)
return -ENOMEM;
- }
- off += scnprintf(&buf[off], buf_size - 1 - off, "%-*s", lf_str_size,
- "pcifunc");
- for (index = 0; index < BLK_COUNT; index++)
- if (strlen(rvu->hw->block[index].name)) {
- off += scnprintf(&buf[off], buf_size - 1 - off,
- "%-*s", lf_str_size,
- rvu->hw->block[index].name);
- }
- off += scnprintf(&buf[off], buf_size - 1 - off, "\n");
- bytes_not_copied = copy_to_user(buffer + (i * off), buf, off);
- if (bytes_not_copied)
- goto out;
+ seq_printf(filp, "%-*s", lf_str_size, "pcifunc");
+ for (index = 0; index < BLK_COUNT; index++)
+ if (strlen(rvu->hw->block[index].name))
+ seq_printf(filp, "%-*s", lf_str_size,
+ rvu->hw->block[index].name);
- i++;
- *ppos += off;
+ seq_putc(filp, '\n');
for (pf = 0; pf < rvu->hw->total_pfs; pf++) {
for (vf = 0; vf <= rvu->hw->total_vfs; vf++) {
- off = 0;
- flag = 0;
pcifunc = rvu_make_pcifunc(rvu->pdev, pf, vf);
if (!pcifunc)
continue;
- if (vf) {
+ for (index = 0; index < BLK_COUNT; index++) {
+ block = rvu->hw->block[index];
+ if (!strlen(block.name))
+ continue;
+ lfs[0] = '\0';
+ get_lf_str_list(&block, pcifunc, lfs);
+ if (strlen(lfs))
+ break;
+ }
+ if (index == BLK_COUNT)
+ continue;
+
+ if (vf)
sprintf(lfs, "PF%d:VF%d", pf, vf - 1);
- off = scnprintf(&buf[off],
- buf_size - 1 - off,
- "%-*s", lf_str_size, lfs);
- } else {
+ else
sprintf(lfs, "PF%d", pf);
- off = scnprintf(&buf[off],
- buf_size - 1 - off,
- "%-*s", lf_str_size, lfs);
- }
+ seq_printf(filp, "%-*s", lf_str_size, lfs);
for (index = 0; index < BLK_COUNT; index++) {
block = rvu->hw->block[index];
if (!strlen(block.name))
continue;
- len = 0;
- lfs[len] = '\0';
- get_lf_str_list(&block, pcifunc, lfs);
- if (strlen(lfs))
- flag = 1;
- off += scnprintf(&buf[off], buf_size - 1 - off,
- "%-*s", lf_str_size, lfs);
- }
- if (flag) {
- off += scnprintf(&buf[off],
- buf_size - 1 - off, "\n");
- bytes_not_copied = copy_to_user(buffer +
- (i * off),
- buf, off);
- if (bytes_not_copied)
- goto out;
-
- i++;
- *ppos += off;
+ lfs[0] = '\0';
+ get_lf_str_list(&block, pcifunc, lfs);
+ seq_printf(filp, "%-*s", lf_str_size, lfs);
}
+ seq_putc(filp, '\n');
}
}
-out:
kfree(lfs);
- kfree(buf);
- if (bytes_not_copied)
- return -EFAULT;
- return *ppos;
+ return 0;
}
-RVU_DEBUG_FOPS(rsrc_status, rsrc_attach_status, NULL);
+RVU_DEBUG_SEQ_FOPS(rsrc_status, rsrc_attach_status, NULL);
static int rvu_dbg_rvu_pf_cgx_map_display(struct seq_file *filp, void *unused)
{
diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.c b/drivers/net/ethernet/mediatek/mtk_eth_soc.c
index 31eee8739c57..3f43e487d887 100644
--- a/drivers/net/ethernet/mediatek/mtk_eth_soc.c
+++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.c
@@ -4509,6 +4509,10 @@ static int mtk_unreg_dev(struct mtk_eth *eth)
mac = netdev_priv(eth->netdev[i]);
if (MTK_HAS_CAPS(eth->soc->caps, MTK_QDMA))
unregister_netdevice_notifier(&mac->device_notifier);
+
+ if (eth->netdev[i]->reg_state != NETREG_REGISTERED)
+ continue;
+
unregister_netdev(eth->netdev[i]);
}
@@ -5340,7 +5344,7 @@ static int mtk_probe(struct platform_device *pdev)
err = register_netdev(eth->netdev[i]);
if (err) {
dev_err(eth->dev, "error bringing up device\n");
- goto err_deinit_ppe;
+ goto err_unreg_netdev;
} else
netif_info(eth, probe, eth->netdev[i],
"mediatek frame engine at 0x%08lx, irq %d\n",
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c b/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c
index 9d1c677814e0..5eadc66037c4 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c
@@ -89,9 +89,16 @@ mlx5_esw_bridge_lower_rep_vport_num_vhca_id_get(struct net_device *dev, struct m
struct net_device *lower_dev;
struct list_head *iter;
- if (netif_is_lag_master(dev) || mlx5e_eswitch_rep(dev))
- return mlx5_esw_bridge_rep_vport_num_vhca_id_get(dev, esw, vport_num,
- esw_owner_vhca_id);
+ if (netif_is_lag_master(dev) || mlx5e_eswitch_rep(dev)) {
+ struct net_device *rep;
+
+ rep = mlx5_esw_bridge_rep_vport_num_vhca_id_get(dev, esw, vport_num,
+ esw_owner_vhca_id);
+ if (rep && !mlx5_esw_bridge_port_exists(*vport_num, *esw_owner_vhca_id,
+ esw->br_offloads))
+ return NULL;
+ return rep;
+ }
netdev_for_each_lower_dev(dev, lower_dev, iter) {
struct net_device *rep;
@@ -108,6 +115,28 @@ mlx5_esw_bridge_lower_rep_vport_num_vhca_id_get(struct net_device *dev, struct m
return NULL;
}
+static bool mlx5_esw_bridge_rep_port_lookup(struct net_device *dev,
+ struct mlx5_esw_bridge_offloads *br_offloads,
+ u16 *vport_num, u16 *esw_owner_vhca_id)
+{
+ if (!mlx5_esw_bridge_rep_vport_num_vhca_id_get(dev, br_offloads->esw, vport_num,
+ esw_owner_vhca_id))
+ return false;
+
+ return mlx5_esw_bridge_port_exists(*vport_num, *esw_owner_vhca_id, br_offloads);
+}
+
+static bool mlx5_esw_bridge_lower_rep_port_lookup(struct net_device *dev,
+ struct mlx5_esw_bridge_offloads *br_offloads,
+ u16 *vport_num, u16 *esw_owner_vhca_id)
+{
+ if (!mlx5_esw_bridge_lower_rep_vport_num_vhca_id_get(dev, br_offloads->esw, vport_num,
+ esw_owner_vhca_id))
+ return false;
+
+ return mlx5_esw_bridge_port_exists(*vport_num, *esw_owner_vhca_id, br_offloads);
+}
+
static bool mlx5_esw_bridge_is_local(struct net_device *dev, struct net_device *rep,
struct mlx5_eswitch *esw)
{
@@ -222,8 +251,7 @@ mlx5_esw_bridge_port_obj_add(struct net_device *dev,
u16 vport_num, esw_owner_vhca_id;
int err;
- if (!mlx5_esw_bridge_rep_vport_num_vhca_id_get(dev, br_offloads->esw, &vport_num,
- &esw_owner_vhca_id))
+ if (!mlx5_esw_bridge_rep_port_lookup(dev, br_offloads, &vport_num, &esw_owner_vhca_id))
return 0;
port_obj_info->handled = true;
@@ -255,8 +283,7 @@ mlx5_esw_bridge_port_obj_del(struct net_device *dev,
const struct switchdev_obj_port_mdb *mdb;
u16 vport_num, esw_owner_vhca_id;
- if (!mlx5_esw_bridge_rep_vport_num_vhca_id_get(dev, br_offloads->esw, &vport_num,
- &esw_owner_vhca_id))
+ if (!mlx5_esw_bridge_rep_port_lookup(dev, br_offloads, &vport_num, &esw_owner_vhca_id))
return 0;
port_obj_info->handled = true;
@@ -287,8 +314,8 @@ mlx5_esw_bridge_port_obj_attr_set(struct net_device *dev,
u16 vport_num, esw_owner_vhca_id;
int err = 0;
- if (!mlx5_esw_bridge_lower_rep_vport_num_vhca_id_get(dev, br_offloads->esw, &vport_num,
- &esw_owner_vhca_id))
+ if (!mlx5_esw_bridge_lower_rep_port_lookup(dev, br_offloads, &vport_num,
+ &esw_owner_vhca_id))
return 0;
port_attr_info->handled = true;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c
index 870d12364f99..0461599b31fd 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c
@@ -1082,6 +1082,9 @@ mlx5_tc_ct_shared_counter_get(struct mlx5_tc_ct_priv *ct_priv,
spin_unlock_bh(&ct_priv->ht_lock);
+ if (rev_entry)
+ mlx5_tc_ct_entry_put(rev_entry);
+
create_counter:
shared_counter = mlx5_tc_ct_counter_create(ct_priv);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c
index 91cfabc45032..d8325c0f97c1 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c
@@ -1564,14 +1564,14 @@ static void setup_fte_addr6(struct mlx5_flow_spec *spec,
memcpy(MLX5_ADDR_OF(fte_match_param, spec->match_value,
outer_headers.src_ipv4_src_ipv6.ipv6_layout.ipv6), saddr, 16);
memcpy(MLX5_ADDR_OF(fte_match_param, spec->match_criteria,
- outer_headers.src_ipv4_src_ipv6.ipv6_layout.ipv6), dmask, 16);
+ outer_headers.src_ipv4_src_ipv6.ipv6_layout.ipv6), smask, 16);
}
if (!addr6_all_zero(daddr)) {
memcpy(MLX5_ADDR_OF(fte_match_param, spec->match_value,
outer_headers.dst_ipv4_dst_ipv6.ipv6_layout.ipv6), daddr, 16);
memcpy(MLX5_ADDR_OF(fte_match_param, spec->match_criteria,
- outer_headers.dst_ipv4_dst_ipv6.ipv6_layout.ipv6), smask, 16);
+ outer_headers.dst_ipv4_dst_ipv6.ipv6_layout.ipv6), dmask, 16);
}
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c
index a7255b42043b..9fbafc8a1bba 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c
@@ -1727,6 +1727,8 @@ void mlx5e_macsec_build_netdev(struct mlx5e_priv *priv)
mlx5_core_dbg(priv->mdev, "mlx5e: MACsec acceleration enabled\n");
netdev->macsec_ops = &macsec_offload_ops;
netdev->features |= NETIF_F_HW_MACSEC;
+ netdev->hw_features |= NETIF_F_HW_MACSEC;
+ netdev->vlan_features |= NETIF_F_HW_MACSEC;
netif_keep_dst(netdev);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index a7d84b540488..fa87f36f0b88 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -5685,7 +5685,6 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev)
netdev->vlan_features |= NETIF_F_SG;
netdev->vlan_features |= NETIF_F_HW_CSUM;
- netdev->vlan_features |= NETIF_F_HW_MACSEC;
netdev->vlan_features |= NETIF_F_GRO;
netdev->vlan_features |= NETIF_F_TSO;
netdev->vlan_features |= NETIF_F_TSO6;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.c
index 60e10047770f..88487e1a5887 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.c
@@ -1649,10 +1649,8 @@ int mlx5_esw_bridge_vport_unlink(struct net_device *br_netdev, u16 vport_num,
int err;
port = mlx5_esw_bridge_port_lookup(vport_num, esw_owner_vhca_id, br_offloads);
- if (!port) {
- NL_SET_ERR_MSG_MOD(extack, "Port is not attached to any bridge");
- return -EINVAL;
- }
+ if (!port)
+ return 0;
if (port->bridge->ifindex != br_netdev->ifindex) {
NL_SET_ERR_MSG_MOD(extack, "Port is attached to another bridge");
return -EINVAL;
@@ -1682,10 +1680,19 @@ int mlx5_esw_bridge_vport_peer_unlink(struct net_device *br_netdev, u16 vport_nu
struct mlx5_esw_bridge_offloads *br_offloads,
struct netlink_ext_ack *extack)
{
+ if (!MLX5_CAP_ESW(br_offloads->esw->dev, merged_eswitch))
+ return 0;
+
return mlx5_esw_bridge_vport_unlink(br_netdev, vport_num, esw_owner_vhca_id, br_offloads,
extack);
}
+bool mlx5_esw_bridge_port_exists(u16 vport_num, u16 esw_owner_vhca_id,
+ struct mlx5_esw_bridge_offloads *br_offloads)
+{
+ return mlx5_esw_bridge_port_lookup(vport_num, esw_owner_vhca_id, br_offloads);
+}
+
int mlx5_esw_bridge_port_vlan_add(u16 vport_num, u16 esw_owner_vhca_id, u16 vid, u16 flags,
struct mlx5_esw_bridge_offloads *br_offloads,
struct netlink_ext_ack *extack)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.h b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.h
index d6f539161993..a4e59cc21089 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.h
@@ -80,6 +80,8 @@ int mlx5_esw_bridge_vlan_proto_set(u16 vport_num, u16 esw_owner_vhca_id, u16 pro
struct mlx5_esw_bridge_offloads *br_offloads);
int mlx5_esw_bridge_mcast_set(u16 vport_num, u16 esw_owner_vhca_id, bool enable,
struct mlx5_esw_bridge_offloads *br_offloads);
+bool mlx5_esw_bridge_port_exists(u16 vport_num, u16 esw_owner_vhca_id,
+ struct mlx5_esw_bridge_offloads *br_offloads);
int mlx5_esw_bridge_port_vlan_add(u16 vport_num, u16 esw_owner_vhca_id, u16 vid, u16 flags,
struct mlx5_esw_bridge_offloads *br_offloads,
struct netlink_ext_ack *extack);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.c
index e749618229bc..8946ba5e30bf 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.c
@@ -36,6 +36,7 @@ struct mlx5_devcom_comp {
struct mlx5_devcom_key key;
mlx5_devcom_event_handler_t handler;
struct kref ref;
+ int nr_devs;
bool ready;
struct rw_semaphore sem;
struct lock_class_key lock_key;
@@ -169,6 +170,7 @@ devcom_alloc_comp_dev(struct mlx5_devcom_dev *devc,
down_write(&comp->sem);
list_add_tail(&devcom->list, &comp->comp_dev_list_head);
+ WRITE_ONCE(comp->nr_devs, comp->nr_devs + 1);
up_write(&comp->sem);
return devcom;
@@ -181,6 +183,7 @@ devcom_free_comp_dev(struct mlx5_devcom_comp_dev *devcom)
down_write(&comp->sem);
list_del(&devcom->list);
+ WRITE_ONCE(comp->nr_devs, comp->nr_devs - 1);
up_write(&comp->sem);
kref_put(&devcom->devc->ref, mlx5_devcom_dev_release);
@@ -283,7 +286,7 @@ int mlx5_devcom_comp_get_size(struct mlx5_devcom_comp_dev *devcom)
{
struct mlx5_devcom_comp *comp = devcom->comp;
- return kref_read(&comp->ref);
+ return READ_ONCE(comp->nr_devs);
}
int mlx5_devcom_send_event(struct mlx5_devcom_comp_dev *devcom,
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h
index d3a7ad921f18..00c76708f233 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_csr.h
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_csr.h
@@ -958,6 +958,7 @@ enum {
/* PUL User Registers */
#define FBNIC_CSR_START_PUL_USER 0x31000 /* CSR section delimiter */
#define FBNIC_PUL_OB_TLP_HDR_AW_CFG 0x3103d /* 0xc40f4 */
+#define FBNIC_PUL_OB_TLP_HDR_AW_CFG_FLUSH_MODE CSR_BIT(20)
#define FBNIC_PUL_OB_TLP_HDR_AW_CFG_FLUSH CSR_BIT(19)
#define FBNIC_PUL_OB_TLP_HDR_AW_CFG_BME CSR_BIT(18)
#define FBNIC_PUL_OB_TLP_HDR_AR_CFG 0x3103e /* 0xc40f8 */
@@ -1175,6 +1176,10 @@ enum {
#define FBNIC_IPC_MBX_DESC_LEN_MASK DESC_GENMASK(63, 48)
#define FBNIC_IPC_MBX_DESC_EOM DESC_BIT(46)
#define FBNIC_IPC_MBX_DESC_ADDR_MASK DESC_GENMASK(45, 3)
+/* Set with FW_CMPL when the FW completed a descriptor without successfully
+ * processing it (e.g. a mailbox DMA error); the completion has no valid data.
+ */
+#define FBNIC_IPC_MBX_DESC_FW_ERR DESC_BIT(2)
#define FBNIC_IPC_MBX_DESC_FW_CMPL DESC_BIT(1)
#define FBNIC_IPC_MBX_DESC_HOST_CMPL DESC_BIT(0)
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c b/drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c
index b7238dd967fe..3a98b28380d8 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c
@@ -170,6 +170,52 @@ static int fbnic_dbg_ipo_dst_show(struct seq_file *s, void *v)
}
DEFINE_SHOW_ATTRIBUTE(fbnic_dbg_ipo_dst);
+static void fbnic_dbg_fw_mbx_display(struct seq_file *s,
+ struct fbnic_dev *fbd, int mbx_idx)
+{
+ struct fbnic_fw_mbx *mbx = &fbd->mbx[mbx_idx];
+ char hdr[80];
+ int i;
+
+ /* Generate header */
+ seq_puts(s, mbx_idx == FBNIC_IPC_MBX_RX_IDX ? "Rx\n" : "Tx\n");
+
+ seq_printf(s, "Rdy: %d Head: %d Tail: %d resp_error: %llu\n",
+ mbx->ready, mbx->head, mbx->tail, mbx->resp_error);
+
+ snprintf(hdr, sizeof(hdr), "%3s %-4s %s %-12s %s %-3s %-16s\n",
+ "Idx", "Len", "E", "Addr", "F", "H", "Raw");
+ seq_puts(s, hdr);
+ fbnic_dbg_desc_break(s, strnlen(hdr, sizeof(hdr)));
+
+ for (i = 0; i < FBNIC_IPC_MBX_DESC_LEN; i++) {
+ u64 desc = __fbnic_mbx_rd_desc(fbd, mbx_idx, i);
+
+ seq_printf(s, "%-3.2d %04lld %d %012llx %d %-3d %016llx\n",
+ i, FIELD_GET(FBNIC_IPC_MBX_DESC_LEN_MASK, desc),
+ !!(desc & FBNIC_IPC_MBX_DESC_EOM),
+ desc & FBNIC_IPC_MBX_DESC_ADDR_MASK,
+ !!(desc & FBNIC_IPC_MBX_DESC_FW_CMPL),
+ !!(desc & FBNIC_IPC_MBX_DESC_HOST_CMPL),
+ desc);
+ }
+}
+
+static int fbnic_dbg_fw_mbx_show(struct seq_file *s, void *v)
+{
+ struct fbnic_dev *fbd = s->private;
+
+ fbnic_dbg_fw_mbx_display(s, fbd, FBNIC_IPC_MBX_RX_IDX);
+
+ /* Add blank line between Rx and Tx */
+ seq_puts(s, "\n");
+
+ fbnic_dbg_fw_mbx_display(s, fbd, FBNIC_IPC_MBX_TX_IDX);
+
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(fbnic_dbg_fw_mbx);
+
static int fbnic_dbg_fw_log_show(struct seq_file *s, void *v)
{
struct fbnic_dev *fbd = s->private;
@@ -249,6 +295,8 @@ void fbnic_dbg_fbd_init(struct fbnic_dev *fbd)
&fbnic_dbg_ipo_src_fops);
debugfs_create_file("ipo_dst", 0400, fbd->dbg_fbd, fbd,
&fbnic_dbg_ipo_dst_fops);
+ debugfs_create_file("fw_mbx", 0400, fbd->dbg_fbd, fbd,
+ &fbnic_dbg_fw_mbx_fops);
debugfs_create_file("fw_log", 0400, fbd->dbg_fbd, fbd,
&fbnic_dbg_fw_log_fops);
}
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c b/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c
index 08aed4103323..d67be9f1a575 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_ethtool.c
@@ -299,6 +299,11 @@ fbnic_get_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring,
kernel_ring->hds_thresh = fbn->hds_thresh;
}
+static u32 fbnic_ring_size_pow2(u32 size)
+{
+ return size ? roundup_pow_of_two(size) : 0;
+}
+
static void fbnic_set_rings(struct fbnic_net *fbn,
struct ethtool_ringparam *ring,
struct kernel_ethtool_ringparam *kernel_ring)
@@ -320,10 +325,10 @@ fbnic_set_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring,
struct fbnic_net *clone;
int err;
- ring->rx_pending = roundup_pow_of_two(ring->rx_pending);
- ring->rx_mini_pending = roundup_pow_of_two(ring->rx_mini_pending);
- ring->rx_jumbo_pending = roundup_pow_of_two(ring->rx_jumbo_pending);
- ring->tx_pending = roundup_pow_of_two(ring->tx_pending);
+ ring->rx_pending = fbnic_ring_size_pow2(ring->rx_pending);
+ ring->rx_mini_pending = fbnic_ring_size_pow2(ring->rx_mini_pending);
+ ring->rx_jumbo_pending = fbnic_ring_size_pow2(ring->rx_jumbo_pending);
+ ring->tx_pending = fbnic_ring_size_pow2(ring->tx_pending);
/* These are absolute minimums allowing the device and driver to operate
* but not necessarily guarantee reasonable performance. Settings below
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c
index 1204c12ea0fa..645408a04a4c 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_fw.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.c
@@ -40,7 +40,7 @@ static void __fbnic_mbx_invalidate_desc(struct fbnic_dev *fbd, int mbx_idx,
fw_wr32(fbd, desc_offset + 1, 0);
}
-static u64 __fbnic_mbx_rd_desc(struct fbnic_dev *fbd, int mbx_idx, int desc_idx)
+u64 __fbnic_mbx_rd_desc(struct fbnic_dev *fbd, int mbx_idx, int desc_idx)
{
u32 desc_offset = FBNIC_IPC_MBX(mbx_idx, desc_idx);
u64 desc;
@@ -60,8 +60,15 @@ static void fbnic_mbx_reset_desc_ring(struct fbnic_dev *fbd, int mbx_idx)
*/
switch (mbx_idx) {
case FBNIC_IPC_MBX_RX_IDX:
+ /* Clearing BME blocks the device from writing to the host
+ * but leaves the requests parked in the write pipeline. The
+ * write path only clears outstanding requests when both FLUSH
+ * and FLUSH_MODE are set; FLUSH_MODE lets them drain without
+ * landing on the host.
+ */
wr32(fbd, FBNIC_PUL_OB_TLP_HDR_AW_CFG,
- FBNIC_PUL_OB_TLP_HDR_AW_CFG_FLUSH);
+ FBNIC_PUL_OB_TLP_HDR_AW_CFG_FLUSH |
+ FBNIC_PUL_OB_TLP_HDR_AW_CFG_FLUSH_MODE);
break;
case FBNIC_IPC_MBX_TX_IDX:
wr32(fbd, FBNIC_PUL_OB_TLP_HDR_AR_CFG,
@@ -286,6 +293,12 @@ static void fbnic_mbx_process_tx_msgs(struct fbnic_dev *fbd)
if (!(desc & FBNIC_IPC_MBX_DESC_FW_CMPL))
break;
+ if (desc & FBNIC_IPC_MBX_DESC_FW_ERR) {
+ tx_mbx->resp_error++;
+ dev_warn_ratelimited(fbd->dev,
+ "FW completed a Tx mailbox request with an error\n");
+ }
+
fbnic_mbx_unmap_and_free_msg(fbd, FBNIC_IPC_MBX_TX_IDX, head);
head++;
@@ -1591,7 +1604,7 @@ static const struct fbnic_tlv_parser fbnic_fw_tlv_parser[] = {
static void fbnic_mbx_process_rx_msgs(struct fbnic_dev *fbd)
{
struct fbnic_fw_mbx *rx_mbx = &fbd->mbx[FBNIC_IPC_MBX_RX_IDX];
- u8 head = rx_mbx->head;
+ u8 head = rx_mbx->head, tail = rx_mbx->tail;
u64 desc, length;
while (head != rx_mbx->tail) {
@@ -1602,8 +1615,15 @@ static void fbnic_mbx_process_rx_msgs(struct fbnic_dev *fbd)
if (!(desc & FBNIC_IPC_MBX_DESC_FW_CMPL))
break;
- dma_unmap_single(fbd->dev, rx_mbx->buf_info[head].addr,
- PAGE_SIZE, DMA_FROM_DEVICE);
+ if (desc & FBNIC_IPC_MBX_DESC_FW_ERR) {
+ rx_mbx->resp_error++;
+ dev_warn_ratelimited(fbd->dev,
+ "FW reported an error on an Rx mailbox message; dropping\n");
+ goto next_page;
+ }
+
+ dma_sync_single_for_cpu(fbd->dev, rx_mbx->buf_info[head].addr,
+ FBNIC_RX_PAGE_SIZE, DMA_FROM_DEVICE);
msg = rx_mbx->buf_info[head].msg;
@@ -1636,19 +1656,26 @@ static void fbnic_mbx_process_rx_msgs(struct fbnic_dev *fbd)
dev_dbg(fbd->dev, "Parsed msg type %d\n", msg->hdr.type);
next_page:
+ fw_wr32(fbd, FBNIC_IPC_MBX(FBNIC_IPC_MBX_RX_IDX, head), 0);
- free_page((unsigned long)rx_mbx->buf_info[head].msg);
+ rx_mbx->buf_info[tail] = rx_mbx->buf_info[head];
rx_mbx->buf_info[head].msg = NULL;
+ rx_mbx->buf_info[head].addr = 0;
- head++;
- head %= FBNIC_IPC_MBX_DESC_LEN;
+ __fbnic_mbx_wr_desc(fbd, FBNIC_IPC_MBX_RX_IDX, tail,
+ FIELD_PREP(FBNIC_IPC_MBX_DESC_LEN_MASK,
+ FBNIC_RX_PAGE_SIZE) |
+ (rx_mbx->buf_info[tail].addr &
+ FBNIC_IPC_MBX_DESC_ADDR_MASK) |
+ FBNIC_IPC_MBX_DESC_HOST_CMPL);
+
+ head = (head + 1) & (FBNIC_IPC_MBX_DESC_LEN - 1);
+ tail = (tail + 1) & (FBNIC_IPC_MBX_DESC_LEN - 1);
}
/* Record head for next interrupt */
rx_mbx->head = head;
-
- /* Make sure we have at least one page for the FW to write to */
- fbnic_mbx_alloc_rx_msgs(fbd);
+ rx_mbx->tail = tail;
}
void fbnic_mbx_poll(struct fbnic_dev *fbd)
@@ -1662,7 +1689,9 @@ void fbnic_mbx_poll(struct fbnic_dev *fbd)
int fbnic_mbx_poll_tx_ready(struct fbnic_dev *fbd)
{
struct fbnic_fw_mbx *tx_mbx = &fbd->mbx[FBNIC_IPC_MBX_TX_IDX];
+ struct fbnic_fw_mbx *rx_mbx = &fbd->mbx[FBNIC_IPC_MBX_RX_IDX];
unsigned long timeout = jiffies + 10 * HZ + 1;
+ u64 tx_resp_error, rx_resp_error;
int err, i;
do {
@@ -1690,6 +1719,9 @@ int fbnic_mbx_poll_tx_ready(struct fbnic_dev *fbd)
* mgmt.version once we get the actual version from the firmware
* in the capabilities request message.
*/
+send_cap_req:
+ tx_resp_error = tx_mbx->resp_error;
+ rx_resp_error = rx_mbx->resp_error;
err = fbnic_fw_xmit_simple_msg(fbd, FBNIC_TLV_MSG_ID_HOST_CAP_REQ);
if (err)
goto clean_mbx;
@@ -1707,9 +1739,27 @@ int fbnic_mbx_poll_tx_ready(struct fbnic_dev *fbd)
msleep(20);
fbnic_mbx_poll(fbd);
+ /* A valid capabilities response ends the poll. Check it
+ * before the FW_ERR retry below so a response parsed in the
+ * same poll as an unrelated FW_ERR is not discarded.
+ */
+ if (fbd->fw_cap.running.mgmt.version >= MIN_FW_VER_CODE)
+ break;
+
/* set err, but wait till mgmt.version check to report it */
- if (!time_is_after_jiffies(timeout))
+ if (!time_is_after_jiffies(timeout)) {
err = -ETIMEDOUT;
+ continue;
+ }
+
+ /* The FW can flag our capabilities request (Tx) or its
+ * response (Rx) with FW_ERR, in which case it produced no
+ * usable response. The ring is not wedged, so re-issue the
+ * request instead of spinning until the timeout.
+ */
+ if (tx_mbx->resp_error != tx_resp_error ||
+ rx_mbx->resp_error != rx_resp_error)
+ goto send_cap_req;
}
return 0;
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_fw.h b/drivers/net/ethernet/meta/fbnic/fbnic_fw.h
index 1ecd777aaada..57ab3a5450be 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_fw.h
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_fw.h
@@ -12,6 +12,7 @@ struct fbnic_tlv_msg;
struct fbnic_fw_mbx {
u8 ready, head, tail;
+ u64 resp_error;
struct {
struct fbnic_tlv_msg *msg;
dma_addr_t addr;
@@ -92,6 +93,7 @@ struct fbnic_fw_completion {
} u;
};
+u64 __fbnic_mbx_rd_desc(struct fbnic_dev *fbd, int mbx_idx, int desc_idx);
void fbnic_mbx_init(struct fbnic_dev *fbd);
void fbnic_mbx_clean(struct fbnic_dev *fbd);
int fbnic_mbx_set_cmpl(struct fbnic_dev *fbd,
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c
index 99b7c0718e80..cabbc88bb92d 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_pci.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_pci.c
@@ -414,6 +414,7 @@ static int fbnic_pm_suspend(struct device *dev)
{
struct fbnic_dev *fbd = dev_get_drvdata(dev);
struct net_device *netdev = fbd->netdev;
+ struct fbnic_net *fbn;
if (fbnic_init_failure(fbd))
goto null_uc_addr;
@@ -421,11 +422,16 @@ static int fbnic_pm_suspend(struct device *dev)
rtnl_lock();
netdev_lock(netdev);
+ fbn = netdev_priv(netdev);
+
netif_device_detach(netdev);
if (netif_running(netdev))
netdev->netdev_ops->ndo_stop(netdev);
+ /* The IRQs are about to be freed, so drop the napi vector count */
+ fbn->num_napi = 0;
+
netdev_unlock(netdev);
rtnl_unlock();
@@ -488,16 +494,20 @@ static int __fbnic_pm_resume(struct device *dev)
if (fbnic_init_failure(fbd))
return 0;
+ rtnl_lock();
+ netdev_lock(netdev);
+
fbn = netdev_priv(netdev);
/* Reset the queues if needed */
fbnic_reset_queues(fbn, fbn->num_tx_queues, fbn->num_rx_queues);
- rtnl_lock();
- netdev_lock(netdev);
-
- if (netif_running(netdev))
+ if (netif_running(netdev)) {
err = __fbnic_open(fbn);
+ /* On failure the vectors are freed, so drop the count */
+ if (err)
+ fbn->num_napi = 0;
+ }
netdev_unlock(netdev);
rtnl_unlock();
diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
index 0ea861ff40f5..946c2997d212 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
@@ -1620,7 +1620,7 @@ fbnic_alloc_qt_page_pools(struct fbnic_net *fbn, struct fbnic_q_triad *qt,
return 0;
err_destroy_sub0:
- page_pool_destroy(pp);
+ page_pool_destroy(qt->sub0.page_pool);
return PTR_ERR(pp);
}
@@ -1789,7 +1789,7 @@ int fbnic_alloc_napi_vectors(struct fbnic_net *fbn)
int err;
/* Allocate 1 Tx queue per napi vector */
- if (num_napi < FBNIC_MAX_TXQS && num_napi == num_tx + num_rx) {
+ if (num_napi <= FBNIC_MAX_TXQS && num_napi == num_tx + num_rx) {
while (num_tx) {
err = fbnic_alloc_napi_vector(fbd, fbn,
num_napi, v_idx,
diff --git a/drivers/net/ethernet/netronome/nfp/crypto/ipsec.c b/drivers/net/ethernet/netronome/nfp/crypto/ipsec.c
index 9e7c285eaa6b..960d7513aa8d 100644
--- a/drivers/net/ethernet/netronome/nfp/crypto/ipsec.c
+++ b/drivers/net/ethernet/netronome/nfp/crypto/ipsec.c
@@ -625,11 +625,12 @@ int nfp_net_ipsec_rx(struct nfp_meta_parsed *meta, struct sk_buff *skb)
xa_lock(&nn->xa_ipsec);
x = xa_load(&nn->xa_ipsec, saidx);
+ if (x)
+ xfrm_state_hold(x);
xa_unlock(&nn->xa_ipsec);
if (!x)
return -EINVAL;
- xfrm_state_hold(x);
sp->xvec[sp->len++] = x;
sp->olen++;
xo = xfrm_offload(skb);
diff --git a/drivers/net/ethernet/spacemit/k1_emac.c b/drivers/net/ethernet/spacemit/k1_emac.c
index d64ca7bbda9e..170e657e1de0 100644
--- a/drivers/net/ethernet/spacemit/k1_emac.c
+++ b/drivers/net/ethernet/spacemit/k1_emac.c
@@ -823,6 +823,9 @@ static void emac_tx_mem_map(struct emac_priv *priv, struct sk_buff *skb)
while (i != head) {
emac_free_tx_buf(priv, i);
+ tx_desc_addr = &((struct emac_desc *)tx_ring->desc_addr)[i];
+ memset(tx_desc_addr, 0, sizeof(*tx_desc_addr));
+
if (++i == tx_ring->total_cnt)
i = 0;
}
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c
index 643578266dfc..8580175a5f92 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c
@@ -1434,8 +1434,11 @@ static int gmac_clk_enable(struct rk_priv_data *bsp_priv, bool enable)
return ret;
ret = clk_prepare_enable(bsp_priv->clk_phy);
- if (ret)
+ if (ret) {
+ clk_bulk_disable_unprepare(bsp_priv->num_clks,
+ bsp_priv->clks);
return ret;
+ }
if (bsp_priv->ops && bsp_priv->ops->set_clock_selection)
bsp_priv->ops->set_clock_selection(bsp_priv,
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c
index aac68dc28dc1..2c63229bfb1a 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c
@@ -496,11 +496,11 @@ static void dwmac4_set_sarc(struct dma_desc *p, u32 sarc_type)
p->des3 |= cpu_to_le32(sarc_type & TDES3_SA_INSERT_CTRL_MASK);
}
-static int set_16kib_bfsize(int mtu)
+static int set_16kib_bfsize(int len)
{
int ret = 0;
- if (unlikely(mtu >= BUF_SIZE_8KiB))
+ if (unlikely(len > BUF_SIZE_8KiB))
ret = BUF_SIZE_16KiB;
return ret;
}
diff --git a/drivers/net/ethernet/stmicro/stmmac/hwif.h b/drivers/net/ethernet/stmicro/stmmac/hwif.h
index 75e915e86c78..ba36f1aa8c8a 100644
--- a/drivers/net/ethernet/stmicro/stmmac/hwif.h
+++ b/drivers/net/ethernet/stmicro/stmmac/hwif.h
@@ -536,7 +536,7 @@ struct stmmac_mode_ops {
unsigned int (*is_jumbo_frm) (int len, int ehn_desc);
int (*jumbo_frm)(struct stmmac_tx_queue *tx_q, struct sk_buff *skb,
int csum);
- int (*set_16kib_bfsize)(int mtu);
+ int (*set_16kib_bfsize)(int len);
void (*init_desc3)(struct dma_desc *p);
void (*refill_desc3)(struct stmmac_rx_queue *rx_q, struct dma_desc *p);
void (*clean_desc3)(struct stmmac_tx_queue *tx_q, struct dma_desc *p);
diff --git a/drivers/net/ethernet/stmicro/stmmac/ring_mode.c b/drivers/net/ethernet/stmicro/stmmac/ring_mode.c
index 45c14c1bb0ea..88a8d22812f5 100644
--- a/drivers/net/ethernet/stmicro/stmmac/ring_mode.c
+++ b/drivers/net/ethernet/stmicro/stmmac/ring_mode.c
@@ -128,10 +128,10 @@ static void clean_desc3(struct stmmac_tx_queue *tx_q, struct dma_desc *p)
p->des3 = 0;
}
-static int set_16kib_bfsize(int mtu)
+static int set_16kib_bfsize(int len)
{
int ret = 0;
- if (unlikely(mtu > BUF_SIZE_8KiB))
+ if (unlikely(len > BUF_SIZE_8KiB))
ret = BUF_SIZE_16KiB;
return ret;
}
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index 6a6cac88f7ec..b9d4c2afc9a8 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -5672,6 +5672,7 @@ static int stmmac_rx(struct stmmac_priv *priv, int limit, u32 queue)
if (!skb) {
page_pool_recycle_direct(rx_q->page_pool,
buf->page);
+ buf->page = NULL;
rx_dropped++;
count++;
goto drain_data;
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_selftests.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_selftests.c
index 3f215ef3c89b..a2f99e2cb43b 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_selftests.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_selftests.c
@@ -13,6 +13,7 @@
#include <linux/ip.h>
#include <linux/phy.h>
#include <linux/udp.h>
+#include <net/dsa.h>
#include <net/pkt_cls.h>
#include <net/pkt_sched.h>
#include <net/tcp.h>
@@ -238,6 +239,10 @@ struct stmmac_test_priv {
struct stmmac_packet_attrs *packet;
struct packet_type pt;
struct completion comp;
+ __be16 packet_type;
+ int (*func)(struct sk_buff *skb, struct net_device *ndev,
+ struct packet_type *pt, struct net_device *orig_ndev);
+ bool capture_all;
int double_vlan;
int vlan_id;
int ok;
@@ -317,6 +322,52 @@ static int stmmac_test_loopback_validate(struct sk_buff *skb,
return 0;
}
+static int stmmac_sft_filter(struct sk_buff *skb, struct net_device *ndev,
+ struct packet_type *pt,
+ struct net_device *orig_ndev)
+{
+ struct stmmac_test_priv *tpriv = pt->af_packet_priv;
+ struct ethhdr *hdr = eth_hdr(skb);
+ int ret = 0;
+
+ if (hdr->h_proto == tpriv->packet_type) {
+ struct sk_buff *nskb = skb_clone(skb, GFP_ATOMIC);
+
+ if (nskb)
+ ret = tpriv->func(nskb, ndev, pt, orig_ndev);
+ }
+
+ kfree_skb(skb);
+ return ret;
+}
+
+static void stmmac_sft_add_pack(struct packet_type *pt)
+{
+ struct stmmac_test_priv *tpriv = pt->af_packet_priv;
+
+ if (netdev_uses_dsa(tpriv->pt.dev) || tpriv->capture_all) {
+ tpriv->packet_type = tpriv->pt.type;
+ tpriv->func = tpriv->pt.func;
+
+ /* DSA conduit will report ETH_P_XDSA, so our packet handler
+ * won't match. Let's register a ETH_P_ALL match and filter
+ * manually in stmmac_sft_filter. This is also useful for
+ * VLAN tests, to capture packets otherwise marked as
+ * OTHERHOST.
+ */
+ tpriv->pt.type = htons(ETH_P_ALL);
+ tpriv->pt.func = stmmac_sft_filter;
+ tpriv->pt.ignore_outgoing = true;
+ }
+
+ dev_add_pack(pt);
+}
+
+static void stmmac_sft_remove_pack(struct packet_type *pt)
+{
+ dev_remove_pack(pt);
+}
+
static int __stmmac_test_loopback(struct stmmac_priv *priv,
struct stmmac_packet_attrs *attr)
{
@@ -338,7 +389,7 @@ static int __stmmac_test_loopback(struct stmmac_priv *priv,
tpriv->packet = attr;
if (!attr->dont_wait)
- dev_add_pack(&tpriv->pt);
+ stmmac_sft_add_pack(&tpriv->pt);
skb = stmmac_test_get_udp_skb(priv, attr);
if (!skb) {
@@ -361,7 +412,7 @@ static int __stmmac_test_loopback(struct stmmac_priv *priv,
cleanup:
if (!attr->dont_wait)
- dev_remove_pack(&tpriv->pt);
+ stmmac_sft_remove_pack(&tpriv->pt);
kfree(tpriv);
return ret;
}
@@ -775,7 +826,7 @@ static int stmmac_test_flowctrl(struct stmmac_priv *priv)
tpriv->pt.func = stmmac_test_flowctrl_validate;
tpriv->pt.dev = priv->dev;
tpriv->pt.af_packet_priv = tpriv;
- dev_add_pack(&tpriv->pt);
+ stmmac_sft_add_pack(&tpriv->pt);
/* Compute minimum number of packets to make FIFO full */
pkt_count = priv->plat->rx_fifo_size;
@@ -833,7 +884,7 @@ static int stmmac_test_flowctrl(struct stmmac_priv *priv)
cleanup:
dev_mc_del(priv->dev, paddr);
dev_set_promiscuity(priv->dev, -1);
- dev_remove_pack(&tpriv->pt);
+ stmmac_sft_remove_pack(&tpriv->pt);
kfree(tpriv);
return ret;
}
@@ -875,6 +926,11 @@ static int stmmac_test_vlan_validate(struct sk_buff *skb,
goto out;
if (skb_headlen(skb) < (STMMAC_TEST_PKT_SIZE - ETH_HLEN))
goto out;
+
+ ehdr = (struct ethhdr *)skb_mac_header(skb);
+ if (!ether_addr_equal_unaligned(ehdr->h_dest, tpriv->packet->dst))
+ goto out;
+
if (tpriv->vlan_id) {
if (skb->vlan_proto != htons(proto))
goto out;
@@ -886,10 +942,6 @@ static int stmmac_test_vlan_validate(struct sk_buff *skb,
}
}
- ehdr = (struct ethhdr *)skb_mac_header(skb);
- if (!ether_addr_equal_unaligned(ehdr->h_dest, tpriv->packet->dst))
- goto out;
-
ihdr = ip_hdr(skb);
if (tpriv->double_vlan)
ihdr = (struct iphdr *)(skb_network_header(skb) + 4);
@@ -931,6 +983,7 @@ static int __stmmac_test_vlanfilt(struct stmmac_priv *priv)
tpriv->pt.dev = priv->dev;
tpriv->pt.af_packet_priv = tpriv;
tpriv->packet = &attr;
+ tpriv->capture_all = true;
/*
* As we use HASH filtering, false positives may appear. This is a
@@ -938,18 +991,20 @@ static int __stmmac_test_vlanfilt(struct stmmac_priv *priv)
* HASH values.
*/
tpriv->vlan_id = 0x123;
- dev_add_pack(&tpriv->pt);
ret = vlan_vid_add(priv->dev, htons(ETH_P_8021Q), tpriv->vlan_id);
if (ret)
goto cleanup;
+ attr.vlan = 1;
+ attr.dst = priv->dev->dev_addr;
+ attr.sport = 9;
+ attr.dport = 9;
+
+ stmmac_sft_add_pack(&tpriv->pt);
+
for (i = 0; i < 4; i++) {
- attr.vlan = 1;
attr.vlan_id_out = tpriv->vlan_id + i;
- attr.dst = priv->dev->dev_addr;
- attr.sport = 9;
- attr.dport = 9;
skb = stmmac_test_get_udp_skb(priv, &attr);
if (!skb) {
@@ -976,9 +1031,9 @@ static int __stmmac_test_vlanfilt(struct stmmac_priv *priv)
}
vlan_del:
+ stmmac_sft_remove_pack(&tpriv->pt);
vlan_vid_del(priv->dev, htons(ETH_P_8021Q), tpriv->vlan_id);
cleanup:
- dev_remove_pack(&tpriv->pt);
kfree(tpriv);
return ret;
}
@@ -1025,6 +1080,7 @@ static int __stmmac_test_dvlanfilt(struct stmmac_priv *priv)
tpriv->pt.dev = priv->dev;
tpriv->pt.af_packet_priv = tpriv;
tpriv->packet = &attr;
+ tpriv->capture_all = true;
/*
* As we use HASH filtering, false positives may appear. This is a
@@ -1032,18 +1088,20 @@ static int __stmmac_test_dvlanfilt(struct stmmac_priv *priv)
* HASH values.
*/
tpriv->vlan_id = 0x123;
- dev_add_pack(&tpriv->pt);
ret = vlan_vid_add(priv->dev, htons(ETH_P_8021AD), tpriv->vlan_id);
if (ret)
goto cleanup;
+ attr.vlan = 2;
+ attr.dst = priv->dev->dev_addr;
+ attr.sport = 9;
+ attr.dport = 9;
+
+ stmmac_sft_add_pack(&tpriv->pt);
+
for (i = 0; i < 4; i++) {
- attr.vlan = 2;
attr.vlan_id_out = tpriv->vlan_id + i;
- attr.dst = priv->dev->dev_addr;
- attr.sport = 9;
- attr.dport = 9;
skb = stmmac_test_get_udp_skb(priv, &attr);
if (!skb) {
@@ -1070,9 +1128,9 @@ static int __stmmac_test_dvlanfilt(struct stmmac_priv *priv)
}
vlan_del:
+ stmmac_sft_remove_pack(&tpriv->pt);
vlan_vid_del(priv->dev, htons(ETH_P_8021AD), tpriv->vlan_id);
cleanup:
- dev_remove_pack(&tpriv->pt);
kfree(tpriv);
return ret;
}
@@ -1303,7 +1361,7 @@ static int stmmac_test_vlanoff_common(struct stmmac_priv *priv, bool svlan)
tpriv->pt.af_packet_priv = tpriv;
tpriv->packet = &attr;
tpriv->vlan_id = 0x123;
- dev_add_pack(&tpriv->pt);
+ tpriv->capture_all = true;
ret = vlan_vid_add(priv->dev, htons(proto), tpriv->vlan_id);
if (ret)
@@ -1311,6 +1369,8 @@ static int stmmac_test_vlanoff_common(struct stmmac_priv *priv, bool svlan)
attr.dst = priv->dev->dev_addr;
+ stmmac_sft_add_pack(&tpriv->pt);
+
skb = stmmac_test_get_udp_skb(priv, &attr);
if (!skb) {
ret = -ENOMEM;
@@ -1328,9 +1388,9 @@ static int stmmac_test_vlanoff_common(struct stmmac_priv *priv, bool svlan)
ret = tpriv->ok ? 0 : -ETIMEDOUT;
vlan_del:
+ stmmac_sft_remove_pack(&tpriv->pt);
vlan_vid_del(priv->dev, htons(proto), tpriv->vlan_id);
cleanup:
- dev_remove_pack(&tpriv->pt);
kfree(tpriv);
return ret;
}
@@ -1342,7 +1402,7 @@ static int stmmac_test_vlanoff(struct stmmac_priv *priv)
static int stmmac_test_svlanoff(struct stmmac_priv *priv)
{
- if (!priv->dma_cap.dvlan)
+ if (!(priv->dev->features & NETIF_F_HW_VLAN_STAG_TX))
return -EOPNOTSUPP;
return stmmac_test_vlanoff_common(priv, true);
}
@@ -1709,6 +1769,9 @@ static int __stmmac_test_jumbo(struct stmmac_priv *priv, u16 queue)
struct stmmac_packet_attrs attr = { };
int size = priv->dma_conf.dma_buf_sz;
+ if (!dwmac_is_xmac(priv->plat->core_type))
+ size -= NET_IP_ALIGN;
+
attr.dst = priv->dev->dev_addr;
attr.max_size = size - ETH_FCS_LEN;
attr.queue_mapping = queue;
diff --git a/drivers/net/ethernet/ti/netcp_core.c b/drivers/net/ethernet/ti/netcp_core.c
index 5ee13db568f0..269d81192089 100644
--- a/drivers/net/ethernet/ti/netcp_core.c
+++ b/drivers/net/ethernet/ti/netcp_core.c
@@ -2167,7 +2167,7 @@ static int netcp_probe(struct platform_device *pdev)
return -ENOMEM;
pm_runtime_enable(&pdev->dev);
- ret = pm_runtime_get_sync(&pdev->dev);
+ ret = pm_runtime_resume_and_get(&pdev->dev);
if (ret < 0) {
dev_err(dev, "Failed to enable NETCP power-domain\n");
pm_runtime_disable(&pdev->dev);
diff --git a/drivers/net/ethernet/wangxun/txgbe/txgbe_fdir.c b/drivers/net/ethernet/wangxun/txgbe/txgbe_fdir.c
index a84010828551..59a47532618c 100644
--- a/drivers/net/ethernet/wangxun/txgbe/txgbe_fdir.c
+++ b/drivers/net/ethernet/wangxun/txgbe/txgbe_fdir.c
@@ -591,15 +591,24 @@ static void txgbe_fdir_filter_restore(struct wx *wx)
queue = TXGBE_RDB_FDIR_DROP_QUEUE;
} else {
u32 ring = ethtool_get_flow_spec_ring(filter->action);
+ u8 vf = ethtool_get_flow_spec_ring_vf(filter->action);
- if (ring >= wx->num_rx_queues) {
+ if (!vf && ring >= wx->num_rx_queues) {
wx_err(wx, "FDIR restore failed, ring:%u\n",
ring);
continue;
+ } else if (vf && (vf > wx->num_vfs ||
+ ring >= wx->num_rx_queues_per_pool)) {
+ wx_err(wx, "FDIR restore failed, vf:%u, ring:%u\n",
+ vf, ring);
+ continue;
}
/* Map the ring onto the absolute queue index */
- queue = wx->rx_ring[ring]->reg_idx;
+ if (!vf)
+ queue = wx->rx_ring[ring]->reg_idx;
+ else
+ queue = ((vf - 1) * wx->num_rx_queues_per_pool) + ring;
}
ret = txgbe_fdir_write_perfect_filter(wx,
diff --git a/drivers/net/macsec.c b/drivers/net/macsec.c
index e20538eda200..ef1d65d98a34 100644
--- a/drivers/net/macsec.c
+++ b/drivers/net/macsec.c
@@ -3533,6 +3533,22 @@ static int macsec_dev_init(struct net_device *dev)
if (err)
return err;
+ err = -ENOMEM;
+ macsec->stats = netdev_alloc_pcpu_stats(struct pcpu_secy_stats);
+ if (!macsec->stats)
+ goto destroy_gro_cells;
+
+ macsec->secy.tx_sc.stats =
+ netdev_alloc_pcpu_stats(struct pcpu_tx_sc_stats);
+ if (!macsec->secy.tx_sc.stats)
+ goto free_secy_stats;
+
+ macsec->secy.tx_sc.md_dst = metadata_dst_alloc(0, METADATA_MACSEC,
+ GFP_KERNEL);
+ if (!macsec->secy.tx_sc.md_dst)
+ goto free_tx_sc_stats;
+ macsec->secy.tx_sc.md_dst->u.macsec_info.sci = macsec->secy.sci;
+
macsec_inherit_tso_max(dev);
dev->hw_features = real_dev->hw_features & MACSEC_OFFLOAD_FEATURES;
@@ -3545,8 +3561,6 @@ static int macsec_dev_init(struct net_device *dev)
macsec_set_head_tail_room(dev);
- if (is_zero_ether_addr(dev->dev_addr))
- eth_hw_addr_inherit(dev, real_dev);
if (is_zero_ether_addr(dev->broadcast))
memcpy(dev->broadcast, real_dev->broadcast, dev->addr_len);
@@ -3554,6 +3568,14 @@ static int macsec_dev_init(struct net_device *dev)
netdev_hold(real_dev, &macsec->dev_tracker, GFP_KERNEL);
return 0;
+
+free_tx_sc_stats:
+ free_percpu(macsec->secy.tx_sc.stats);
+free_secy_stats:
+ free_percpu(macsec->stats);
+destroy_gro_cells:
+ gro_cells_destroy(&macsec->gro_cells);
+ return err;
}
static void macsec_dev_uninit(struct net_device *dev)
@@ -4108,26 +4130,11 @@ static sci_t dev_to_sci(struct net_device *dev, __be16 port)
return make_sci(dev->dev_addr, port);
}
-static int macsec_add_dev(struct net_device *dev, sci_t sci, u8 icv_len)
+static void macsec_init_secy(struct net_device *dev, sci_t sci, u8 icv_len)
{
struct macsec_dev *macsec = macsec_priv(dev);
struct macsec_secy *secy = &macsec->secy;
- macsec->stats = netdev_alloc_pcpu_stats(struct pcpu_secy_stats);
- if (!macsec->stats)
- return -ENOMEM;
-
- secy->tx_sc.stats = netdev_alloc_pcpu_stats(struct pcpu_tx_sc_stats);
- if (!secy->tx_sc.stats)
- return -ENOMEM;
-
- secy->tx_sc.md_dst = metadata_dst_alloc(0, METADATA_MACSEC, GFP_KERNEL);
- if (!secy->tx_sc.md_dst)
- /* macsec and secy percpu stats will be freed when unregistering
- * net_device in macsec_free_netdev()
- */
- return -ENOMEM;
-
if (sci == MACSEC_UNDEF_SCI)
sci = dev_to_sci(dev, MACSEC_PORT_ES);
@@ -4141,15 +4148,12 @@ static int macsec_add_dev(struct net_device *dev, sci_t sci, u8 icv_len)
secy->xpn = DEFAULT_XPN;
secy->sci = sci;
- secy->tx_sc.md_dst->u.macsec_info.sci = sci;
secy->tx_sc.active = true;
secy->tx_sc.encoding_sa = DEFAULT_ENCODING_SA;
secy->tx_sc.encrypt = DEFAULT_ENCRYPT;
secy->tx_sc.send_sci = DEFAULT_SEND_SCI;
secy->tx_sc.end_station = false;
secy->tx_sc.scb = false;
-
- return 0;
}
static struct lock_class_key macsec_netdev_addr_lock_key;
@@ -4212,6 +4216,24 @@ static int macsec_newlink(struct net_device *dev,
if (rx_handler && rx_handler != macsec_handle_frame)
return -EBUSY;
+ if (is_zero_ether_addr(dev->dev_addr))
+ eth_hw_addr_inherit(dev, real_dev);
+
+ if (data && data[IFLA_MACSEC_SCI])
+ sci = nla_get_sci(data[IFLA_MACSEC_SCI]);
+ else if (data && data[IFLA_MACSEC_PORT])
+ sci = dev_to_sci(dev, nla_get_be16(data[IFLA_MACSEC_PORT]));
+ else
+ sci = dev_to_sci(dev, MACSEC_PORT_ES);
+
+ /* Registration can notify listeners before returning. */
+ macsec_init_secy(dev, sci, icv_len);
+ if (data) {
+ err = macsec_changelink_common(dev, data);
+ if (err)
+ return err;
+ }
+
err = register_netdevice(dev);
if (err < 0)
return err;
@@ -4224,31 +4246,11 @@ static int macsec_newlink(struct net_device *dev,
if (err < 0)
goto unregister;
- /* need to be already registered so that ->init has run and
- * the MAC addr is set
- */
- if (data && data[IFLA_MACSEC_SCI])
- sci = nla_get_sci(data[IFLA_MACSEC_SCI]);
- else if (data && data[IFLA_MACSEC_PORT])
- sci = dev_to_sci(dev, nla_get_be16(data[IFLA_MACSEC_PORT]));
- else
- sci = dev_to_sci(dev, MACSEC_PORT_ES);
-
if (rx_handler && sci_exists(real_dev, sci)) {
err = -EBUSY;
goto unlink;
}
- err = macsec_add_dev(dev, sci, icv_len);
- if (err)
- goto unlink;
-
- if (data) {
- err = macsec_changelink_common(dev, data);
- if (err)
- goto del_dev;
- }
-
/* If h/w offloading is available, propagate to the device */
if (macsec_is_offloaded(macsec)) {
const struct macsec_ops *ops;
diff --git a/drivers/net/ovpn/netlink.c b/drivers/net/ovpn/netlink.c
index 0051166c59c7..15664cde85e0 100644
--- a/drivers/net/ovpn/netlink.c
+++ b/drivers/net/ovpn/netlink.c
@@ -100,6 +100,8 @@ static bool ovpn_nl_attr_sockaddr_remote(struct nlattr **attrs,
struct sockaddr_in6 *sin6;
struct sockaddr_in *sin;
struct in6_addr *in6;
+ struct nlattr *scope;
+ u32 scope_id = 0;
__be16 port = 0;
__be32 *in;
@@ -114,6 +116,9 @@ static bool ovpn_nl_attr_sockaddr_remote(struct nlattr **attrs,
} else if (attrs[OVPN_A_PEER_REMOTE_IPV6]) {
ss->ss_family = AF_INET6;
in6 = nla_data(attrs[OVPN_A_PEER_REMOTE_IPV6]);
+ scope = attrs[OVPN_A_PEER_REMOTE_IPV6_SCOPE_ID];
+ if (scope)
+ scope_id = nla_get_u32(scope);
} else {
return false;
}
@@ -126,6 +131,7 @@ static bool ovpn_nl_attr_sockaddr_remote(struct nlattr **attrs,
if (!ipv6_addr_v4mapped(in6)) {
sin6 = (struct sockaddr_in6 *)ss;
sin6->sin6_port = port;
+ sin6->sin6_scope_id = scope_id;
memcpy(&sin6->sin6_addr, in6, sizeof(*in6));
break;
}
@@ -179,6 +185,39 @@ static sa_family_t ovpn_nl_family_get(struct nlattr *addr4,
return AF_UNSPEC;
}
+static int ovpn_nl_peer_check_vpn_addrs(const struct in_addr *addr4,
+ const struct in6_addr *addr6,
+ struct genl_info *info)
+{
+ int addr6_type;
+
+ if (addr4->s_addr == htonl(INADDR_ANY) && ipv6_addr_any(addr6)) {
+ NL_SET_ERR_MSG_MOD(info->extack,
+ "at least one VPN IP must be configured in MP mode");
+ return -EINVAL;
+ }
+
+ if (ipv4_is_multicast(addr4->s_addr) || ipv4_is_lbcast(addr4->s_addr) ||
+ ipv4_is_loopback(addr4->s_addr)) {
+ NL_SET_ERR_MSG_MOD(info->extack,
+ "VPN IPv4 address must be valid unicast or any");
+ return -EADDRNOTAVAIL;
+ }
+
+ if (!ipv6_addr_any(addr6)) {
+ addr6_type = ipv6_addr_type(addr6);
+
+ if (!(addr6_type & IPV6_ADDR_UNICAST) ||
+ (addr6_type & (IPV6_ADDR_LOOPBACK | IPV6_ADDR_COMPATv4))) {
+ NL_SET_ERR_MSG_MOD(info->extack,
+ "VPN IPv6 address must be valid unicast or any");
+ return -EADDRNOTAVAIL;
+ }
+ }
+
+ return 0;
+}
+
static int ovpn_nl_peer_precheck(struct ovpn_priv *ovpn,
struct genl_info *info,
struct nlattr **attrs)
@@ -340,8 +379,10 @@ static int ovpn_nl_peer_modify(struct ovpn_peer *peer, struct genl_info *info,
int ovpn_nl_peer_new_doit(struct sk_buff *skb, struct genl_info *info)
{
- struct nlattr *attrs[OVPN_A_PEER_MAX + 1];
+ struct in_addr vpn_addr4 = { .s_addr = htonl(INADDR_ANY) };
+ struct in6_addr vpn_addr6 = IN6ADDR_ANY_INIT;
struct ovpn_priv *ovpn = info->user_ptr[0];
+ struct nlattr *attrs[OVPN_A_PEER_MAX + 1];
struct ovpn_socket *ovpn_sock;
struct socket *sock = NULL;
struct ovpn_peer *peer;
@@ -365,11 +406,18 @@ int ovpn_nl_peer_new_doit(struct sk_buff *skb, struct genl_info *info)
return -EINVAL;
/* in MP mode VPN IPs are required for selecting the right peer */
- if (ovpn->mode == OVPN_MODE_MP && !attrs[OVPN_A_PEER_VPN_IPV4] &&
- !attrs[OVPN_A_PEER_VPN_IPV6]) {
- NL_SET_ERR_MSG_FMT_MOD(info->extack,
- "VPN IP must be provided in MP mode");
- return -EINVAL;
+ if (ovpn->mode == OVPN_MODE_MP) {
+ if (attrs[OVPN_A_PEER_VPN_IPV4])
+ vpn_addr4.s_addr =
+ nla_get_in_addr(attrs[OVPN_A_PEER_VPN_IPV4]);
+ if (attrs[OVPN_A_PEER_VPN_IPV6])
+ vpn_addr6 =
+ nla_get_in6_addr(attrs[OVPN_A_PEER_VPN_IPV6]);
+
+ ret = ovpn_nl_peer_check_vpn_addrs(&vpn_addr4, &vpn_addr6,
+ info);
+ if (ret < 0)
+ return ret;
}
peer_id = nla_get_u32(attrs[OVPN_A_PEER_ID]);
@@ -467,8 +515,10 @@ int ovpn_nl_peer_new_doit(struct sk_buff *skb, struct genl_info *info)
int ovpn_nl_peer_set_doit(struct sk_buff *skb, struct genl_info *info)
{
- struct nlattr *attrs[OVPN_A_PEER_MAX + 1];
struct ovpn_priv *ovpn = info->user_ptr[0];
+ struct nlattr *attrs[OVPN_A_PEER_MAX + 1];
+ struct in6_addr vpn_addr6;
+ struct in_addr vpn_addr4;
struct ovpn_socket *sock;
struct ovpn_peer *peer;
u32 peer_id;
@@ -515,28 +565,58 @@ int ovpn_nl_peer_set_doit(struct sk_buff *skb, struct genl_info *info)
rcu_read_unlock();
spin_lock_bh(&ovpn->lock);
- ret = ovpn_nl_peer_modify(peer, info, attrs);
- if (ret < 0) {
- spin_unlock_bh(&ovpn->lock);
- ovpn_peer_put(peer);
- return ret;
+
+ vpn_addr4 = peer->vpn_addrs.ipv4;
+ vpn_addr6 = peer->vpn_addrs.ipv6;
+
+ /* reject peer with conflicting VPN address */
+ if (attrs[OVPN_A_PEER_VPN_IPV4]) {
+ vpn_addr4.s_addr = nla_get_in_addr(attrs[OVPN_A_PEER_VPN_IPV4]);
+ if (ovpn_peer_vpn_addr_conflict4(ovpn, peer, &vpn_addr4))
+ goto addr_conflict;
+ }
+ if (attrs[OVPN_A_PEER_VPN_IPV6]) {
+ vpn_addr6 = nla_get_in6_addr(attrs[OVPN_A_PEER_VPN_IPV6]);
+ if (ovpn_peer_vpn_addr_conflict6(ovpn, peer, &vpn_addr6))
+ goto addr_conflict;
}
+ /* in MP mode VPN IPs are required for selecting the right peer */
+ if (ovpn->mode == OVPN_MODE_MP) {
+ ret = ovpn_nl_peer_check_vpn_addrs(&vpn_addr4, &vpn_addr6,
+ info);
+ if (ret < 0)
+ goto unlock;
+ }
+
+ ret = ovpn_nl_peer_modify(peer, info, attrs);
+ if (ret < 0)
+ goto unlock;
+
/* ret == 1 means that VPN IPv4/6 has been modified and rehashing
* is required
*/
- if (ret > 0)
+ if (ret > 0) {
ovpn_peer_hash_vpn_ip(peer);
+ ret = 0;
+ }
/* if the remote endpoint was updated, the by_transp_addr hash bucket
* also needs to be refreshed, otherwise incoming packets from the new
* remote address would fail the lockless lookup
*/
if (attrs[OVPN_A_PEER_REMOTE_IPV4] || attrs[OVPN_A_PEER_REMOTE_IPV6])
ovpn_peer_hash_transp_addr(peer);
+
+unlock:
spin_unlock_bh(&ovpn->lock);
ovpn_peer_put(peer);
- return 0;
+ return ret;
+addr_conflict:
+ NL_SET_ERR_MSG_FMT_MOD(info->extack,
+ "VPN IP is already assigned to another peer");
+ ret = -EADDRINUSE;
+ goto unlock;
}
static int ovpn_nl_send_peer(struct sk_buff *skb, const struct genl_info *info,
diff --git a/drivers/net/ovpn/peer.c b/drivers/net/ovpn/peer.c
index 6b9f5f12fc5a..d07d91e35f5b 100644
--- a/drivers/net/ovpn/peer.c
+++ b/drivers/net/ovpn/peer.c
@@ -109,6 +109,7 @@ struct ovpn_peer *ovpn_peer_new(struct ovpn_priv *ovpn, u32 id)
RCU_INIT_POINTER(peer->bind, NULL);
ovpn_crypto_state_init(&peer->crypto);
spin_lock_init(&peer->lock);
+ seqcount_spinlock_init(&peer->route_key_seq, &peer->lock);
kref_init(&peer->refcount);
ovpn_peer_stats_init(&peer->vpn_stats);
ovpn_peer_stats_init(&peer->link_stats);
@@ -195,13 +196,12 @@ static void __ovpn_peer_hash_transp_addr(struct ovpn_peer *peer,
*/
void ovpn_peer_endpoints_update(struct ovpn_peer *peer, struct sk_buff *skb)
{
+ const void *local_ip = NULL;
struct sockaddr_storage ss;
struct sockaddr_in6 *sa6;
- bool reset_cache = false;
struct sockaddr_in *sa;
struct ovpn_bind *bind;
- const void *local_ip;
- size_t salen = 0;
+ bool floated = false;
spin_lock_bh(&peer->lock);
bind = rcu_dereference_protected(peer->bind,
@@ -228,8 +228,7 @@ void ovpn_peer_endpoints_update(struct ovpn_peer *peer, struct sk_buff *skb)
.sin_addr.s_addr = ip_hdr(skb)->saddr,
.sin_port = udp_hdr(skb)->source,
};
- salen = sizeof(*sa);
- reset_cache = true;
+ floated = true;
break;
}
@@ -241,10 +240,12 @@ void ovpn_peer_endpoints_update(struct ovpn_peer *peer, struct sk_buff *skb)
netdev_name(peer->ovpn->dev),
peer->id, &bind->local.ipv4.s_addr,
&ip_hdr(skb)->daddr);
- bind->local.ipv4.s_addr = ip_hdr(skb)->daddr;
- reset_cache = true;
+ local_ip = &ip_hdr(skb)->daddr;
+ memcpy(&ss, &bind->remote, sizeof(struct sockaddr_in));
+ break;
}
- break;
+ /* nothing changed */
+ goto unlock;
case htons(ETH_P_IPV6):
/* float check */
if (unlikely(!ovpn_bind_skb_src_match(bind, skb))) {
@@ -266,8 +267,7 @@ void ovpn_peer_endpoints_update(struct ovpn_peer *peer, struct sk_buff *skb)
ipv6_iface_scope_id(&ipv6_hdr(skb)->saddr,
skb->skb_iif),
};
- salen = sizeof(*sa6);
- reset_cache = true;
+ floated = true;
break;
}
@@ -280,26 +280,30 @@ void ovpn_peer_endpoints_update(struct ovpn_peer *peer, struct sk_buff *skb)
netdev_name(peer->ovpn->dev),
peer->id, &bind->local.ipv6,
&ipv6_hdr(skb)->daddr);
- bind->local.ipv6 = ipv6_hdr(skb)->daddr;
- reset_cache = true;
+ local_ip = &ipv6_hdr(skb)->daddr;
+ memcpy(&ss, &bind->remote, sizeof(struct sockaddr_in6));
+ break;
}
- break;
+ /* nothing changed */
+ goto unlock;
default:
goto unlock;
}
- if (unlikely(reset_cache))
- dst_cache_reset(&peer->dst_cache);
-
- /* if the peer did not float, we can bail out now */
- if (likely(!salen))
- goto unlock;
-
if (unlikely(ovpn_peer_reset_sockaddr(peer,
(struct sockaddr_storage *)&ss,
local_ip) < 0))
goto unlock;
+ /* reset the cache only after a successful bind update to avoid useless
+ * cache misses on concurrent TX
+ */
+ dst_cache_reset(&peer->dst_cache);
+
+ /* if only the local address changed, bail out now */
+ if (!floated)
+ goto unlock;
+
net_dbg_ratelimited("%s: peer %d floated to %pIScp",
netdev_name(peer->ovpn->dev), peer->id, &ss);
@@ -478,7 +482,7 @@ static struct ovpn_peer *ovpn_peer_get_by_vpn_addr4(struct ovpn_priv *ovpn,
* Return: the peer if found or NULL otherwise
*/
static struct ovpn_peer *ovpn_peer_get_by_vpn_addr6(struct ovpn_priv *ovpn,
- struct in6_addr *addr)
+ const struct in6_addr *addr)
{
struct hlist_nulls_head *nhead;
struct hlist_nulls_node *ntmp;
@@ -503,6 +507,64 @@ static struct ovpn_peer *ovpn_peer_get_by_vpn_addr6(struct ovpn_priv *ovpn,
return NULL;
}
+/**
+ * ovpn_peer_vpn_addr_conflict4 - check if the VPN v4 address is already in use
+ * @ovpn: the openvpn instance to search
+ * @peer: peer being added or updated, or NULL
+ * @addr: VPN IPv4 address to check
+ *
+ * Check whether @addr is already assigned to another peer. @peer is ignored
+ * when found, allowing peer updates that keep an existing address.
+ * Unspecified addresses are ignored.
+ *
+ * Note: the caller must hold @ovpn->lock.
+ *
+ * Return: true on conflict, false otherwise.
+ */
+bool ovpn_peer_vpn_addr_conflict4(struct ovpn_priv *ovpn,
+ const struct ovpn_peer *peer,
+ const struct in_addr *addr)
+{
+ struct ovpn_peer *tmp = NULL;
+
+ lockdep_assert_held(&ovpn->lock);
+
+ /* we don't hash INADDR_ANY, no conflict in that case */
+ if (addr->s_addr != htonl(INADDR_ANY))
+ tmp = ovpn_peer_get_by_vpn_addr4(ovpn, addr->s_addr);
+
+ return tmp && tmp != peer;
+}
+
+/**
+ * ovpn_peer_vpn_addr_conflict6 - check if the VPN v6 address is already in use
+ * @ovpn: the openvpn instance to search
+ * @peer: peer being added or updated, or NULL
+ * @addr: VPN IPv6 address to check
+ *
+ * Check whether @addr is already assigned to another peer. @peer is ignored
+ * when found, allowing peer updates that keep an existing address.
+ * Unspecified addresses are ignored.
+ *
+ * Note: the caller must hold @ovpn->lock.
+ *
+ * Return: true on conflict, false otherwise.
+ */
+bool ovpn_peer_vpn_addr_conflict6(struct ovpn_priv *ovpn,
+ const struct ovpn_peer *peer,
+ const struct in6_addr *addr)
+{
+ struct ovpn_peer *tmp = NULL;
+
+ lockdep_assert_held(&ovpn->lock);
+
+ /* we don't hash ::, no conflict in that case */
+ if (!ipv6_addr_any(addr))
+ tmp = ovpn_peer_get_by_vpn_addr6(ovpn, addr);
+
+ return tmp && tmp != peer;
+}
+
/**
* ovpn_peer_transp_match - check if sockaddr and peer binding match
* @peer: the peer to get the binding from
@@ -985,10 +1047,11 @@ void ovpn_peer_hash_vpn_ip(struct ovpn_peer *peer)
if (hlist_unhashed(&peer->hash_entry_id))
return;
- if (peer->vpn_addrs.ipv4.s_addr != htonl(INADDR_ANY)) {
- /* remove potential old hashing */
- hlist_nulls_del_init_rcu(&peer->hash_entry_addr4);
+ /* remove potential old hashing */
+ hlist_nulls_del_init_rcu(&peer->hash_entry_addr4);
+ hlist_nulls_del_init_rcu(&peer->hash_entry_addr6);
+ if (peer->vpn_addrs.ipv4.s_addr != htonl(INADDR_ANY)) {
nhead = ovpn_get_hash_head(peer->ovpn->peers->by_vpn_addr4,
&peer->vpn_addrs.ipv4,
sizeof(peer->vpn_addrs.ipv4));
@@ -996,9 +1059,6 @@ void ovpn_peer_hash_vpn_ip(struct ovpn_peer *peer)
}
if (!ipv6_addr_any(&peer->vpn_addrs.ipv6)) {
- /* remove potential old hashing */
- hlist_nulls_del_init_rcu(&peer->hash_entry_addr6);
-
nhead = ovpn_get_hash_head(peer->ovpn->peers->by_vpn_addr6,
&peer->vpn_addrs.ipv6,
sizeof(peer->vpn_addrs.ipv6));
@@ -1033,6 +1093,13 @@ static int ovpn_peer_add_mp(struct ovpn_priv *ovpn, struct ovpn_peer *peer)
goto out;
}
+ /* reject peer with conflicting VPN address */
+ if (ovpn_peer_vpn_addr_conflict4(ovpn, NULL, &peer->vpn_addrs.ipv4) ||
+ ovpn_peer_vpn_addr_conflict6(ovpn, NULL, &peer->vpn_addrs.ipv6)) {
+ ret = -EADDRINUSE;
+ goto out;
+ }
+
bind = rcu_dereference_protected(peer->bind, true);
/* peers connected via TCP have bind == NULL */
if (bind) {
diff --git a/drivers/net/ovpn/peer.h b/drivers/net/ovpn/peer.h
index 9a05a278e1cd..2306b84929bb 100644
--- a/drivers/net/ovpn/peer.h
+++ b/drivers/net/ovpn/peer.h
@@ -10,6 +10,7 @@
#ifndef _NET_OVPN_OVPNPEER_H_
#define _NET_OVPN_OVPNPEER_H_
+#include <linux/seqlock.h>
#include <net/dst_cache.h>
#include <net/strparser.h>
@@ -17,6 +18,16 @@
#include "socket.h"
#include "stats.h"
+/**
+ * struct ovpn_route_key - route key used for the peer dst cache
+ * @mark: fwmark used for route lookup
+ * @sport: UDP source port used for route lookup
+ */
+struct ovpn_route_key {
+ u32 mark;
+ __be16 sport;
+};
+
/**
* struct ovpn_peer - the main remote peer object
* @ovpn: main openvpn instance this peer belongs to
@@ -44,6 +55,8 @@
* @tcp.sk_cb.ops: pointer to the original prot_ops object (TCP only)
* @crypto: the crypto configuration (ciphers, keys, etc..)
* @dst_cache: cache for dst_entry used to send to peer
+ * @route_key: route key matching the current dst cache contents
+ * @route_key_seq: seqcount protecting lockless route_key reads
* @bind: remote peer binding
* @keepalive_interval: seconds after which a new keepalive should be sent
* @keepalive_xmit_exp: future timestamp when next keepalive should be sent
@@ -54,7 +67,7 @@
* @vpn_stats: per-peer in-VPN TX/RX stats
* @link_stats: per-peer link/transport TX/RX stats
* @delete_reason: why peer was deleted (i.e. timeout, transport error, ..)
- * @lock: protects binding to peer (bind) and keepalive* fields
+ * @lock: protects binding to peer (bind), route_key and keepalive* fields
* @refcount: reference counter
* @rcu: used to free peer in an RCU safe way
* @release_entry: entry for the socket release list
@@ -97,6 +110,8 @@ struct ovpn_peer {
} tcp;
struct ovpn_crypto_state crypto;
struct dst_cache dst_cache;
+ struct ovpn_route_key route_key;
+ seqcount_spinlock_t route_key_seq;
struct ovpn_bind __rcu *bind;
unsigned long keepalive_interval;
unsigned long keepalive_xmit_exp;
@@ -107,7 +122,7 @@ struct ovpn_peer {
struct ovpn_peer_stats vpn_stats;
struct ovpn_peer_stats link_stats;
enum ovpn_del_peer_reason delete_reason;
- spinlock_t lock; /* protects bind and keepalive* */
+ spinlock_t lock; /* protects bind, route_key and keepalive* */
struct kref refcount;
struct rcu_head rcu;
struct llist_node release_entry;
@@ -147,6 +162,12 @@ struct ovpn_peer *ovpn_peer_get_by_transp_addr(struct ovpn_priv *ovpn,
struct ovpn_peer *ovpn_peer_get_by_id(struct ovpn_priv *ovpn, u32 peer_id);
struct ovpn_peer *ovpn_peer_get_by_dst(struct ovpn_priv *ovpn,
struct sk_buff *skb);
+bool ovpn_peer_vpn_addr_conflict4(struct ovpn_priv *ovpn,
+ const struct ovpn_peer *peer,
+ const struct in_addr *addr);
+bool ovpn_peer_vpn_addr_conflict6(struct ovpn_priv *ovpn,
+ const struct ovpn_peer *peer,
+ const struct in6_addr *addr);
void ovpn_peer_hash_vpn_ip(struct ovpn_peer *peer);
void ovpn_peer_hash_transp_addr(struct ovpn_peer *peer);
bool ovpn_peer_check_by_src(struct ovpn_priv *ovpn, struct sk_buff *skb,
diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c
index 367563d84472..0b620c6cc64d 100644
--- a/drivers/net/ovpn/udp.c
+++ b/drivers/net/ovpn/udp.c
@@ -132,6 +132,77 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb)
return 0;
}
+static bool ovpn_route_key_equal(const struct ovpn_route_key *a,
+ const struct ovpn_route_key *b)
+{
+ return a->mark == b->mark && a->sport == b->sport;
+}
+
+/**
+ * ovpn_dst_cache_check_key - reset peer dst cache after key changes
+ * @peer: the peer owning the dst cache
+ * @cache: the cache that might need to be reset
+ * @key: the route key for the packet being transmitted
+ *
+ * Reset the peer dst cache if it was populated for a different route key.
+ */
+static void ovpn_dst_cache_check_key(struct ovpn_peer *peer,
+ struct dst_cache *cache,
+ const struct ovpn_route_key *key)
+{
+ struct ovpn_route_key old_key;
+ unsigned int seq;
+
+ /* snapshot the saved key before deciding whether the cache matches */
+ do {
+ seq = read_seqcount_begin(&peer->route_key_seq);
+ old_key = peer->route_key;
+ } while (read_seqcount_retry(&peer->route_key_seq, seq));
+
+ /* nothing changed: the current cache can be reused */
+ if (likely(ovpn_route_key_equal(&old_key, key)))
+ return;
+
+ /* recheck under lock because another path may have updated the key */
+ spin_lock_bh(&peer->lock);
+ if (!ovpn_route_key_equal(&peer->route_key, key)) {
+ write_seqcount_begin(&peer->route_key_seq);
+ peer->route_key = *key;
+ dst_cache_reset(cache);
+ write_seqcount_end(&peer->route_key_seq);
+ }
+ spin_unlock_bh(&peer->lock);
+}
+
+/**
+ * ovpn_dst_cache_current - check whether a route lookup matches peer state
+ * @peer: the peer owning the bind and dst cache
+ * @bind: the RCU bind used for the route lookup
+ * @key: the route key used for the route lookup
+ *
+ * Check that @bind is still the current peer bind and that @key still matches
+ * the peer route key. The caller must hold @peer->lock. The TX path keeps
+ * @bind inside an RCU read-side critical section, so pointer identity is enough
+ * to detect whether the bind was replaced while the route lookup was running.
+ *
+ * Return: true if the lookup result still matches the current peer state and
+ * may update the dst cache or replace the bind.
+ */
+static bool ovpn_dst_cache_current(const struct ovpn_peer *peer,
+ const struct ovpn_bind *bind,
+ const struct ovpn_route_key *key)
+{
+ const struct ovpn_bind *curr_bind;
+
+ lockdep_assert_held(&peer->lock);
+
+ curr_bind = rcu_dereference_protected(peer->bind,
+ lockdep_is_held(&peer->lock));
+
+ return curr_bind == bind &&
+ ovpn_route_key_equal(key, &peer->route_key);
+}
+
/**
* ovpn_udp4_output - send IPv4 packet over udp socket
* @peer: the destination peer
@@ -139,21 +210,26 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb)
* @cache: dst cache
* @sk: the socket to send the packet over
* @skb: the packet to send
+ * @key: the route key snapshot used for cache validation and flow lookup
*
* Return: 0 on success or a negative error code otherwise
*/
static int ovpn_udp4_output(struct ovpn_peer *peer, struct ovpn_bind *bind,
struct dst_cache *cache, struct sock *sk,
- struct sk_buff *skb)
+ struct sk_buff *skb,
+ const struct ovpn_route_key *key)
{
+ struct sockaddr_storage remote;
+ struct in_addr local = {};
+ bool reset_local = false;
struct rtable *rt;
struct flowi4 fl = {
.saddr = bind->local.ipv4.s_addr,
.daddr = bind->remote.in4.sin_addr.s_addr,
- .fl4_sport = inet_sk(sk)->inet_sport,
+ .fl4_sport = key->sport,
.fl4_dport = bind->remote.in4.sin_port,
.flowi4_proto = sk->sk_protocol,
- .flowi4_mark = sk->sk_mark,
+ .flowi4_mark = key->mark,
};
int ret;
@@ -162,26 +238,19 @@ static int ovpn_udp4_output(struct ovpn_peer *peer, struct ovpn_bind *bind,
if (rt)
goto transmit;
- if (unlikely(!inet_confirm_addr(sock_net(sk), NULL, 0, fl.saddr,
- RT_SCOPE_HOST))) {
- /* we may end up here when the cached address is not usable
- * anymore. In this case we reset address/cache and perform a
- * new look up
+ if (fl.saddr && unlikely(!inet_confirm_addr(sock_net(sk), NULL, 0,
+ fl.saddr, RT_SCOPE_HOST))) {
+ /* The learned local address is not usable anymore.
+ * Retry with source address autoselection.
*/
fl.saddr = 0;
- spin_lock_bh(&peer->lock);
- bind->local.ipv4.s_addr = 0;
- spin_unlock_bh(&peer->lock);
- dst_cache_reset(cache);
+ reset_local = true;
}
rt = ip_route_output_flow(sock_net(sk), &fl, sk);
if (IS_ERR(rt) && PTR_ERR(rt) == -EINVAL) {
fl.saddr = 0;
- spin_lock_bh(&peer->lock);
- bind->local.ipv4.s_addr = 0;
- spin_unlock_bh(&peer->lock);
- dst_cache_reset(cache);
+ reset_local = true;
rt = ip_route_output_flow(sock_net(sk), &fl, sk);
}
@@ -194,7 +263,30 @@ static int ovpn_udp4_output(struct ovpn_peer *peer, struct ovpn_bind *bind,
ret);
goto err;
}
- dst_cache_set_ip4(cache, &rt->dst, fl.saddr);
+
+ /* avoid storing a stale cache or local address */
+ spin_lock_bh(&peer->lock);
+ if (likely(ovpn_dst_cache_current(peer, bind, key))) {
+ if (!reset_local) {
+ dst_cache_set_ip4(cache, &rt->dst, fl.saddr);
+ spin_unlock_bh(&peer->lock);
+ goto transmit;
+ }
+
+ /* invalidate per-CPU dst entries that may still carry
+ * the stale source
+ */
+ dst_cache_reset(cache);
+
+ /* preserve the current remote */
+ memcpy(&remote, &bind->remote, sizeof(struct sockaddr_in));
+ /* The current packet already has a valid wildcard-source route.
+ * If replacing the bind fails, leave the stale local in place;
+ * a later cache miss will retry the repair.
+ */
+ ovpn_peer_reset_sockaddr(peer, &remote, &local);
+ }
+ spin_unlock_bh(&peer->lock);
transmit:
udp_tunnel_xmit_skb(rt, sk, skb, fl.saddr, fl.daddr, 0,
@@ -214,23 +306,28 @@ static int ovpn_udp4_output(struct ovpn_peer *peer, struct ovpn_bind *bind,
* @cache: dst cache
* @sk: the socket to send the packet over
* @skb: the packet to send
+ * @key: the route key snapshot used for cache validation and flow lookup
*
* Return: 0 on success or a negative error code otherwise
*/
static int ovpn_udp6_output(struct ovpn_peer *peer, struct ovpn_bind *bind,
struct dst_cache *cache, struct sock *sk,
- struct sk_buff *skb)
+ struct sk_buff *skb,
+ const struct ovpn_route_key *key)
{
+ struct in6_addr local = in6addr_any;
+ struct sockaddr_storage remote;
+ bool reset_local = false;
struct dst_entry *dst;
int ret;
struct flowi6 fl = {
.saddr = bind->local.ipv6,
.daddr = bind->remote.in6.sin6_addr,
- .fl6_sport = inet_sk(sk)->inet_sport,
+ .fl6_sport = key->sport,
.fl6_dport = bind->remote.in6.sin6_port,
.flowi6_proto = sk->sk_protocol,
- .flowi6_mark = sk->sk_mark,
+ .flowi6_mark = key->mark,
.flowi6_oif = bind->remote.in6.sin6_scope_id,
};
@@ -239,16 +336,13 @@ static int ovpn_udp6_output(struct ovpn_peer *peer, struct ovpn_bind *bind,
if (dst)
goto transmit;
- if (unlikely(!ipv6_chk_addr(sock_net(sk), &fl.saddr, NULL, 0))) {
- /* we may end up here when the cached address is not usable
- * anymore. In this case we reset address/cache and perform a
- * new look up
+ if (!ipv6_addr_any(&fl.saddr) &&
+ unlikely(!ipv6_chk_addr(sock_net(sk), &fl.saddr, NULL, 0))) {
+ /* The learned local address is not usable anymore.
+ * Retry with source address autoselection.
*/
fl.saddr = in6addr_any;
- spin_lock_bh(&peer->lock);
- bind->local.ipv6 = in6addr_any;
- spin_unlock_bh(&peer->lock);
- dst_cache_reset(cache);
+ reset_local = true;
}
dst = ipv6_stub->ipv6_dst_lookup_flow(sock_net(sk), sk, &fl, NULL);
@@ -259,7 +353,30 @@ static int ovpn_udp6_output(struct ovpn_peer *peer, struct ovpn_bind *bind,
&bind->remote.in6, ret);
goto err;
}
- dst_cache_set_ip6(cache, dst, &fl.saddr);
+
+ /* avoid storing a stale cache or local address */
+ spin_lock_bh(&peer->lock);
+ if (likely(ovpn_dst_cache_current(peer, bind, key))) {
+ if (!reset_local) {
+ dst_cache_set_ip6(cache, dst, &fl.saddr);
+ spin_unlock_bh(&peer->lock);
+ goto transmit;
+ }
+
+ /* invalidate per-CPU dst entries that may still carry
+ * the stale source
+ */
+ dst_cache_reset(cache);
+
+ /* preserve the current remote */
+ memcpy(&remote, &bind->remote, sizeof(struct sockaddr_in6));
+ /* The current packet already has a valid wildcard-source route.
+ * If replacing the bind fails, leave the stale local in place;
+ * a later cache miss will retry the repair.
+ */
+ ovpn_peer_reset_sockaddr(peer, &remote, &local);
+ }
+ spin_unlock_bh(&peer->lock);
transmit:
/* user IPv6 packets may be larger than the transport interface
@@ -288,6 +405,7 @@ static int ovpn_udp6_output(struct ovpn_peer *peer, struct ovpn_bind *bind,
* @cache: dst cache
* @sk: the socket to send the packet over
* @skb: the packet to send
+ * @key: route key snapshot used for cache validation and flow lookup
*
* rcu_read_lock should be held on entry.
* On return, the skb is consumed.
@@ -295,7 +413,8 @@ static int ovpn_udp6_output(struct ovpn_peer *peer, struct ovpn_bind *bind,
* Return: 0 on success or a negative error code otherwise
*/
static int ovpn_udp_output(struct ovpn_peer *peer, struct dst_cache *cache,
- struct sock *sk, struct sk_buff *skb)
+ struct sock *sk, struct sk_buff *skb,
+ struct ovpn_route_key *key)
{
struct ovpn_bind *bind;
int ret;
@@ -315,11 +434,11 @@ static int ovpn_udp_output(struct ovpn_peer *peer, struct dst_cache *cache,
switch (bind->remote.in4.sin_family) {
case AF_INET:
- ret = ovpn_udp4_output(peer, bind, cache, sk, skb);
+ ret = ovpn_udp4_output(peer, bind, cache, sk, skb, key);
break;
#if IS_ENABLED(CONFIG_IPV6)
case AF_INET6:
- ret = ovpn_udp6_output(peer, bind, cache, sk, skb);
+ ret = ovpn_udp6_output(peer, bind, cache, sk, skb, key);
break;
#endif
default:
@@ -341,15 +460,21 @@ static int ovpn_udp_output(struct ovpn_peer *peer, struct dst_cache *cache,
void ovpn_udp_send_skb(struct ovpn_peer *peer, struct sock *sk,
struct sk_buff *skb)
{
+ struct ovpn_route_key key = {
+ .mark = READ_ONCE(sk->sk_mark),
+ .sport = READ_ONCE(inet_sk(sk)->inet_sport),
+ };
int ret;
skb->dev = peer->ovpn->dev;
- skb->mark = READ_ONCE(sk->sk_mark);
+ skb->mark = key.mark;
/* no checksum performed at this layer */
skb->ip_summed = CHECKSUM_NONE;
+ ovpn_dst_cache_check_key(peer, &peer->dst_cache, &key);
+
/* crypto layer -> transport (UDP) */
- ret = ovpn_udp_output(peer, &peer->dst_cache, sk, skb);
+ ret = ovpn_udp_output(peer, &peer->dst_cache, sk, skb, &key);
if (unlikely(ret < 0))
kfree_skb(skb);
}
diff --git a/drivers/net/pcs/pcs-rzn1-miic.c b/drivers/net/pcs/pcs-rzn1-miic.c
index 885f17c32643..cea350745a7b 100644
--- a/drivers/net/pcs/pcs-rzn1-miic.c
+++ b/drivers/net/pcs/pcs-rzn1-miic.c
@@ -594,7 +594,8 @@ static int miic_parse_dt(struct miic *miic, u32 *mode_cfg)
if (!dt_val)
return -ENOMEM;
- memset(dt_val, MIIC_MODCTRL_CONF_NONE, sizeof(*dt_val));
+ memset(dt_val, MIIC_MODCTRL_CONF_NONE,
+ sizeof(*dt_val) * miic->of_data->conf_conv_count);
if (of_property_read_u32(np, "renesas,miic-switch-portin", &conf) == 0)
dt_val[0] = conf;
diff --git a/drivers/net/pcs/pcs-xpcs.c b/drivers/net/pcs/pcs-xpcs.c
index 14c4b10fee74..495068c3642c 100644
--- a/drivers/net/pcs/pcs-xpcs.c
+++ b/drivers/net/pcs/pcs-xpcs.c
@@ -1430,8 +1430,10 @@ static int xpcs_init_clks(struct dw_xpcs *xpcs)
return dev_err_probe(dev, ret, "Failed to get clocks\n");
ret = clk_bulk_prepare_enable(DW_XPCS_NUM_CLKS, xpcs->clks);
- if (ret)
+ if (ret) {
+ clk_bulk_put(DW_XPCS_NUM_CLKS, xpcs->clks);
return dev_err_probe(dev, ret, "Failed to enable clocks\n");
+ }
return 0;
}
diff --git a/drivers/net/phy/micrel.c b/drivers/net/phy/micrel.c
index e6f00aa9a990..d79004dcb768 100644
--- a/drivers/net/phy/micrel.c
+++ b/drivers/net/phy/micrel.c
@@ -6034,6 +6034,7 @@ static int lanphy_write_reg_data(struct phy_device *phydev,
data->val);
if (ret)
break;
+ data++;
}
return ret;
diff --git a/drivers/net/phy/phylink.c b/drivers/net/phy/phylink.c
index 848b6ec9bd85..0b618842d41d 100644
--- a/drivers/net/phy/phylink.c
+++ b/drivers/net/phy/phylink.c
@@ -2115,7 +2115,6 @@ static int phylink_bringup_phy(struct phylink *pl, struct phy_device *phy,
mutex_lock(&pl->phydev_mutex);
mutex_lock(&phy->lock);
mutex_lock(&pl->state_mutex);
- pl->phydev = phy;
pl->phy_state.interface = interface;
pl->phy_state.pause = MLO_PAUSE_NONE;
pl->phy_state.speed = SPEED_UNKNOWN;
@@ -2182,10 +2181,25 @@ static int phylink_bringup_phy(struct phylink *pl, struct phy_device *phy,
ret = 0;
}
- if (ret == 0 && phy_interrupt_is_valid(phy))
+ if (ret)
+ return ret;
+
+ /* Nothing below can fail, so the PHY can be recorded now. Doing it
+ * here rather than above keeps a failed bringup from leaving
+ * pl->phydev pointing at a PHY the caller is about to detach.
+ */
+ mutex_lock(&pl->phydev_mutex);
+ mutex_lock(&phy->lock);
+ mutex_lock(&pl->state_mutex);
+ pl->phydev = phy;
+ mutex_unlock(&pl->state_mutex);
+ mutex_unlock(&phy->lock);
+ mutex_unlock(&pl->phydev_mutex);
+
+ if (phy_interrupt_is_valid(phy))
phy_request_interrupt(phy);
- return ret;
+ return 0;
}
static int phylink_attach_phy(struct phylink *pl, struct phy_device *phy,
diff --git a/drivers/net/usb/catc.c b/drivers/net/usb/catc.c
index 3c824340ffb0..a2fce0843205 100644
--- a/drivers/net/usb/catc.c
+++ b/drivers/net/usb/catc.c
@@ -239,17 +239,26 @@ static void catc_rx_done(struct urb *urb)
}
do {
- if(!catc->is_f5u011) {
- pkt_len = le16_to_cpup((__le16*)pkt_start);
- if (pkt_len > urb->actual_length) {
+ int remaining = urb->actual_length -
+ (pkt_start - (u8 *)urb->transfer_buffer);
+
+ if (!catc->is_f5u011) {
+ if (remaining < pkt_offset) {
catc->netdev->stats.rx_length_errors++;
catc->netdev->stats.rx_errors++;
break;
}
+ pkt_len = le16_to_cpup((__le16 *)pkt_start);
} else {
pkt_len = urb->actual_length;
}
+ if (pkt_len < ETH_HLEN || pkt_len + pkt_offset > remaining) {
+ catc->netdev->stats.rx_length_errors++;
+ catc->netdev->stats.rx_errors++;
+ break;
+ }
+
if (!(skb = dev_alloc_skb(pkt_len)))
return;
diff --git a/drivers/net/usb/cdc_mbim.c b/drivers/net/usb/cdc_mbim.c
index dbf01210b0e7..c587d2d5cbbe 100644
--- a/drivers/net/usb/cdc_mbim.c
+++ b/drivers/net/usb/cdc_mbim.c
@@ -634,6 +634,11 @@ static const struct usb_device_id mbim_devs[] = {
.driver_info = (unsigned long)&cdc_mbim_info,
},
+ /* MeiG Smart SRM821 ZLP conformance */
+ { USB_DEVICE_AND_INTERFACE_INFO(0x2dee, 0x4d53, USB_CLASS_COMM, USB_CDC_SUBCLASS_MBIM, USB_CDC_PROTO_NONE),
+ .driver_info = (unsigned long)&cdc_mbim_info,
+ },
+
/* Some Huawei devices, ME906s-158 (12d1:15c1) and E3372
* (12d1:157d), are known to fail unless the NDP is placed
* after the IP packets. Applying the quirk to all Huawei
diff --git a/drivers/net/usb/lan78xx.c b/drivers/net/usb/lan78xx.c
index 851dbe1090dc..b73490cae3ba 100644
--- a/drivers/net/usb/lan78xx.c
+++ b/drivers/net/usb/lan78xx.c
@@ -5246,10 +5246,12 @@ static bool lan78xx_submit_deferred_urbs(struct lan78xx_net *dev)
!netif_carrier_ok(dev->net) ||
pipe_halted) {
lan78xx_release_tx_buf(dev, skb);
+ usb_put_urb(urb);
continue;
}
ret = usb_submit_urb(urb, GFP_ATOMIC);
+ usb_put_urb(urb);
if (ret == 0) {
netif_trans_update(dev->net);
diff --git a/drivers/net/usb/sr9700.c b/drivers/net/usb/sr9700.c
index a5d364fbc363..f71bd2d5a274 100644
--- a/drivers/net/usb/sr9700.c
+++ b/drivers/net/usb/sr9700.c
@@ -403,7 +403,8 @@ static int sr9700_rx_fixup(struct usbnet *dev, struct sk_buff *skb)
/* ignore the CRC length */
len = (skb->data[1] | (skb->data[2] << 8)) - 4;
- if (len > ETH_FRAME_LEN || len > skb->len || len < 0)
+ if (len > ETH_FRAME_LEN || len < 0 ||
+ len > skb->len - SR_RX_OVERHEAD)
return 0;
/* the last packet of current skb */
diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index 56519b646cac..fea031d491b3 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -1053,6 +1053,7 @@ static int __veth_napi_enable_range(struct net_device *dev, int start, int end)
for (i = start; i < end; i++) {
struct veth_rq *rq = &priv->rq[i];
+ rcu_assign_pointer(rq->xdp_prog, priv->_xdp_prog);
napi_enable(&rq->xdp_napi);
rcu_assign_pointer(priv->rq[i].napi, &priv->rq[i].xdp_napi);
}
@@ -1087,6 +1088,7 @@ static void veth_napi_del_range(struct net_device *dev, int start, int end)
rcu_assign_pointer(priv->rq[i].napi, NULL);
napi_disable(&rq->xdp_napi);
+ rcu_assign_pointer(rq->xdp_prog, NULL);
__netif_napi_del(&rq->xdp_napi);
}
synchronize_net();
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index d2fe26bfaccd..f337fed30c6f 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -3397,6 +3397,14 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
else
virtqueue_disable_cb(sq->vq);
+ if (!use_napi &&
+ unlikely(skb_orphan_frags(skb, GFP_ATOMIC))) {
+ DEV_STATS_INC(dev, tx_dropped);
+ dev_kfree_skb_any(skb);
+ kick = !xmit_more || netif_xmit_stopped(txq);
+ goto kick_vq;
+ }
+
/* timestamp packet in software */
skb_tx_timestamp(skb);
@@ -3429,6 +3437,7 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
kick = use_napi ? __netdev_tx_sent_queue(txq, skb->len, xmit_more) :
!xmit_more || netif_xmit_stopped(txq);
+kick_vq:
if (kick) {
if (virtqueue_kick_prepare(sq->vq) && virtqueue_notify(sq->vq)) {
u64_stats_update_begin(&sq->stats.syncp);
diff --git a/drivers/net/vrf.c b/drivers/net/vrf.c
index 4b62b167650e..7ac320da3ba2 100644
--- a/drivers/net/vrf.c
+++ b/drivers/net/vrf.c
@@ -1225,8 +1225,6 @@ static int vrf_prepare_mac_header(struct sk_buff *skb,
skb->protocol = eth->h_proto;
skb->pkt_type = PACKET_HOST;
- skb_postpush_rcsum(skb, skb->data, ETH_HLEN);
-
skb_pull_inline(skb, ETH_HLEN);
return 0;
diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c
index 1e411a57d1fe..02cc79169945 100644
--- a/drivers/net/vxlan/vxlan_core.c
+++ b/drivers/net/vxlan/vxlan_core.c
@@ -1956,13 +1956,15 @@ static struct sk_buff *vxlan_na_create(struct sk_buff *request,
struct ipv6hdr *pip6;
u8 *daddr;
int na_olen = 8; /* opt hdr + ETH_ALEN for target */
+ int headroom;
int ns_olen;
int i, len;
if (dev == NULL || !pskb_may_pull(request, request->len))
return NULL;
- len = LL_RESERVED_SPACE(dev) + sizeof(struct ipv6hdr) +
+ headroom = LL_RESERVED_SPACE(dev);
+ len = headroom + sizeof(struct ipv6hdr) +
sizeof(*na) + na_olen + dev->needed_tailroom;
reply = alloc_skb(len, GFP_ATOMIC);
if (reply == NULL)
@@ -1970,7 +1972,7 @@ static struct sk_buff *vxlan_na_create(struct sk_buff *request,
reply->protocol = htons(ETH_P_IPV6);
reply->dev = dev;
- skb_reserve(reply, LL_RESERVED_SPACE(request->dev));
+ skb_reserve(reply, headroom);
skb_push(reply, sizeof(struct ethhdr));
skb_reset_mac_header(reply);
diff --git a/drivers/nfc/microread/microread.c b/drivers/nfc/microread/microread.c
index b0e483ee7eec..8746ba29d7ce 100644
--- a/drivers/nfc/microread/microread.c
+++ b/drivers/nfc/microread/microread.c
@@ -251,9 +251,9 @@ static int microread_start_poll(struct nfc_hci_dev *hdev,
param[1] |= (1 << 1);
if ((im_protocols | tm_protocols) & NFC_PROTO_NFC_DEP_MASK) {
- hdev->gb = nfc_get_local_general_bytes(hdev->ndev,
- &hdev->gb_len);
- if (hdev->gb == NULL || hdev->gb_len == 0) {
+ nfc_get_local_general_bytes(hdev->ndev, hdev->gb,
+ sizeof(hdev->gb), &hdev->gb_len);
+ if (hdev->gb_len == 0) {
im_protocols &= ~NFC_PROTO_NFC_DEP_MASK;
tm_protocols &= ~NFC_PROTO_NFC_DEP_MASK;
}
diff --git a/drivers/nfc/nfcmrvl/fw_dnld.c b/drivers/nfc/nfcmrvl/fw_dnld.c
index a9b03dcc4100..2b734c1c1d1f 100644
--- a/drivers/nfc/nfcmrvl/fw_dnld.c
+++ b/drivers/nfc/nfcmrvl/fw_dnld.c
@@ -263,9 +263,14 @@ static int process_state_fw_dnld(struct nfcmrvl_private *priv,
* B8..N: payload
*/
- /* Remove NCI HDR */
- skb_pull(skb, 3);
- if (skb->data[0] != HELPER_CMD_PACKET_FORMAT || skb->len != 5) {
+ if (skb->len != NCI_DATA_HDR_SIZE + 5) {
+ nfc_err(priv->dev, "bad command");
+ return -EINVAL;
+ }
+
+ /* Remove NCI header */
+ skb_pull(skb, NCI_DATA_HDR_SIZE);
+ if (skb->data[0] != HELPER_CMD_PACKET_FORMAT) {
nfc_err(priv->dev, "bad command");
return -EINVAL;
}
diff --git a/drivers/nfc/pn533/pn533.c b/drivers/nfc/pn533/pn533.c
index 9db9e559aeae..a1f6951401cd 100644
--- a/drivers/nfc/pn533/pn533.c
+++ b/drivers/nfc/pn533/pn533.c
@@ -1355,10 +1355,11 @@ static int pn533_poll_dep(struct nfc_dev *nfc_dev)
u8 *next, nfcid3[NFC_NFCID3_MAXSIZE];
u8 passive_data[PASSIVE_DATA_LEN] = {0x00, 0xff, 0xff, 0x00, 0x3};
- if (!dev->gb) {
- dev->gb = nfc_get_local_general_bytes(nfc_dev, &dev->gb_len);
-
- if (!dev->gb || !dev->gb_len) {
+ if (!dev->gb_len) {
+ nfc_get_local_general_bytes(nfc_dev, dev->gb,
+ sizeof(dev->gb),
+ &dev->gb_len);
+ if (!dev->gb_len) {
dev->poll_dep = 0;
queue_work(dev->wq, &dev->rf_work);
}
@@ -1656,8 +1657,9 @@ static int pn533_start_poll(struct nfc_dev *nfc_dev,
}
if (tm_protocols) {
- dev->gb = nfc_get_local_general_bytes(nfc_dev, &dev->gb_len);
- if (dev->gb == NULL)
+ nfc_get_local_general_bytes(nfc_dev, dev->gb,
+ sizeof(dev->gb), &dev->gb_len);
+ if (dev->gb_len == 0)
tm_protocols = 0;
}
diff --git a/drivers/nfc/pn533/pn533.h b/drivers/nfc/pn533/pn533.h
index 09e35b8693f5..5ab668e05121 100644
--- a/drivers/nfc/pn533/pn533.h
+++ b/drivers/nfc/pn533/pn533.h
@@ -6,6 +6,8 @@
* Copyright (C) 2012-2013 Tieto Poland
*/
+#include <net/nfc/nfc.h>
+
#define PN533_DEVICE_STD 0x1
#define PN533_DEVICE_PASORI 0x2
#define PN533_DEVICE_ACR122U 0x3
@@ -166,7 +168,7 @@ struct pn533 {
struct timer_list listen_timer;
int cancel_listen;
- u8 *gb;
+ u8 gb[NFC_MAX_GT_LEN];
size_t gb_len;
u8 tgt_available_prots;
diff --git a/drivers/nfc/pn533/usb.c b/drivers/nfc/pn533/usb.c
index 0f12f86ebb02..35f3350cad32 100644
--- a/drivers/nfc/pn533/usb.c
+++ b/drivers/nfc/pn533/usb.c
@@ -319,7 +319,9 @@ static bool pn533_acr122_is_rx_frame_valid(void *_frame, struct pn533 *dev)
if (frame->ccid.type != 0x83)
return false;
- if (!frame->ccid.datalen)
+ if (frame->ccid.datalen < 2 ||
+ frame->ccid.datalen > PN533_ACR122_FRAME_MAX_PAYLOAD_LEN +
+ PN533_ACR122_RX_FRAME_TAIL_LEN)
return false;
if (frame->data[frame->ccid.datalen - 2] == 0x63)
diff --git a/drivers/nfc/pn544/pn544.c b/drivers/nfc/pn544/pn544.c
index 32a61a185142..6da666169ad2 100644
--- a/drivers/nfc/pn544/pn544.c
+++ b/drivers/nfc/pn544/pn544.c
@@ -377,10 +377,9 @@ static int pn544_hci_start_poll(struct nfc_hci_dev *hdev,
return r;
if ((im_protocols | tm_protocols) & NFC_PROTO_NFC_DEP_MASK) {
- hdev->gb = nfc_get_local_general_bytes(hdev->ndev,
- &hdev->gb_len);
- pr_debug("generate local bytes %p\n", hdev->gb);
- if (hdev->gb == NULL || hdev->gb_len == 0) {
+ nfc_get_local_general_bytes(hdev->ndev, hdev->gb,
+ sizeof(hdev->gb), &hdev->gb_len);
+ if (hdev->gb_len == 0) {
im_protocols &= ~NFC_PROTO_NFC_DEP_MASK;
tm_protocols &= ~NFC_PROTO_NFC_DEP_MASK;
}
diff --git a/drivers/nfc/port100.c b/drivers/nfc/port100.c
index 00d8ea6dcb5d..d5544965481b 100644
--- a/drivers/nfc/port100.c
+++ b/drivers/nfc/port100.c
@@ -636,6 +636,13 @@ static void port100_recv_response(struct urb *urb)
in_frame = dev->in_urb->transfer_buffer;
+ if (urb->actual_length < PORT100_FRAME_HEADER_LEN ||
+ urb->actual_length < port100_rx_frame_size(in_frame)) {
+ nfc_err(&dev->interface->dev, "Received a truncated frame\n");
+ cmd->status = -EIO;
+ goto sched_wq;
+ }
+
if (!port100_rx_frame_is_valid(in_frame)) {
nfc_err(&dev->interface->dev, "Received an invalid frame\n");
cmd->status = -EIO;
diff --git a/drivers/nfc/st21nfca/core.c b/drivers/nfc/st21nfca/core.c
index bec6f607c32c..bfdbdf1e22b0 100644
--- a/drivers/nfc/st21nfca/core.c
+++ b/drivers/nfc/st21nfca/core.c
@@ -351,10 +351,10 @@ static int st21nfca_hci_start_poll(struct nfc_hci_dev *hdev,
if (r < 0)
return r;
} else {
- hdev->gb = nfc_get_local_general_bytes(hdev->ndev,
- &hdev->gb_len);
-
- if (hdev->gb == NULL || hdev->gb_len == 0) {
+ nfc_get_local_general_bytes(hdev->ndev, hdev->gb,
+ sizeof(hdev->gb),
+ &hdev->gb_len);
+ if (hdev->gb_len == 0) {
im_protocols &= ~NFC_PROTO_NFC_DEP_MASK;
tm_protocols &= ~NFC_PROTO_NFC_DEP_MASK;
}
@@ -577,9 +577,7 @@ static int st21nfca_get_iso15693_inventory(struct nfc_hci_dev *hdev,
if (r < 0)
goto exit;
- skb_pull(inventory_skb, 2);
-
- if (inventory_skb->len == 0 ||
+ if (!skb_pull(inventory_skb, 2) || inventory_skb->len < 2 ||
inventory_skb->len > NFC_ISO15693_UID_MAXSIZE) {
r = -EPROTO;
goto exit;
diff --git a/drivers/nfc/st21nfca/i2c.c b/drivers/nfc/st21nfca/i2c.c
index 6d7861383806..58f07725cc5f 100644
--- a/drivers/nfc/st21nfca/i2c.c
+++ b/drivers/nfc/st21nfca/i2c.c
@@ -289,27 +289,36 @@ static int check_crc(u8 *buf, int buflen)
*/
static int st21nfca_hci_i2c_repack(struct sk_buff *skb)
{
- int i, j, r, size;
+ int read, write, r, size;
- if (skb->len < 1 || (skb->len > 1 && skb->data[1] != 0))
+ if (skb->len < ST21NFCA_FRAME_HEADROOM ||
+ !IS_START_OF_FRAME(skb->data))
return -EBADMSG;
size = get_frame_size(skb->data, skb->len);
if (size > 0) {
+ if (size < ST21NFCA_FRAME_HEADROOM + 2)
+ return -EBADMSG;
+
skb_trim(skb, size);
/* remove ST21NFCA byte stuffing for upper layer */
- for (i = 1, j = 0; i < skb->len; i++) {
- if (skb->data[i + j] ==
+ for (read = 1, write = 1; read < skb->len;) {
+ if (skb->data[read] ==
(u8) ST21NFCA_ESCAPE_BYTE_STUFFING) {
- skb->data[i] = skb->data[i + j + 1]
- | ST21NFCA_BYTE_STUFFING_MASK;
- i++;
- j++;
+ if (read + 1 == skb->len)
+ return -EBADMSG;
+
+ skb->data[write++] = skb->data[read + 1]
+ | ST21NFCA_BYTE_STUFFING_MASK;
+ read += 2;
+ } else {
+ skb->data[write++] = skb->data[read++];
}
- skb->data[i] = skb->data[i + j];
}
/* remove byte stuffing useless byte */
- skb_trim(skb, i - j);
+ skb_trim(skb, write);
+ if (skb->len < ST21NFCA_FRAME_HEADROOM + 2)
+ return -EBADMSG;
/* remove ST21NFCA_SOF_EOF from head */
skb_pull(skb, 1);
diff --git a/drivers/nfc/trf7970a.c b/drivers/nfc/trf7970a.c
index 08c27bb438b5..785a2d21a7f1 100644
--- a/drivers/nfc/trf7970a.c
+++ b/drivers/nfc/trf7970a.c
@@ -1997,8 +1997,10 @@ static int trf7970a_startup(struct trf7970a *trf)
return ret;
ret = trf7970a_update_rx_gain_reduction(trf);
- if (ret)
+ if (ret) {
+ trf7970a_power_down(trf);
return ret;
+ }
pm_runtime_set_active(trf->dev);
pm_runtime_enable(trf->dev);
diff --git a/drivers/nfc/virtual_ncidev.c b/drivers/nfc/virtual_ncidev.c
index b957fce83b7c..3eaa63bfee27 100644
--- a/drivers/nfc/virtual_ncidev.c
+++ b/drivers/nfc/virtual_ncidev.c
@@ -195,7 +195,8 @@ static const struct file_operations virtual_ncidev_fops = {
.write = virtual_ncidev_write,
.open = virtual_ncidev_open,
.release = virtual_ncidev_close,
- .unlocked_ioctl = virtual_ncidev_ioctl
+ .unlocked_ioctl = virtual_ncidev_ioctl,
+ .compat_ioctl = compat_ptr_ioctl,
};
static struct miscdevice miscdev = {
diff --git a/drivers/pci/of_property.c b/drivers/pci/of_property.c
index 7aae46f333d9..336aa1d01be8 100644
--- a/drivers/pci/of_property.c
+++ b/drivers/pci/of_property.c
@@ -95,9 +95,13 @@ static int of_pci_prop_bus_range(struct pci_dev *pdev,
struct of_changeset *ocs,
struct device_node *np)
{
- u32 bus_range[] = { pdev->subordinate->busn_res.start,
- pdev->subordinate->busn_res.end };
+ u32 bus_range[2];
+ if (!pdev->subordinate)
+ return 0;
+
+ bus_range[0] = pdev->subordinate->busn_res.start;
+ bus_range[1] = pdev->subordinate->busn_res.end;
return of_changeset_add_prop_u32_array(ocs, np, "bus-range", bus_range,
ARRAY_SIZE(bus_range));
}
@@ -220,6 +224,9 @@ static int of_pci_prop_intr_map(struct pci_dev *pdev, struct of_changeset *ocs,
int ret;
u8 pin;
+ if (!pdev->subordinate)
+ return 0;
+
pnode = pci_device_to_OF_node(pdev->bus->self);
if (!pnode)
pnode = pci_bus_to_OF_node(pdev->bus);
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 7ec2ee3fe909..19fbc7c5625f 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -2449,6 +2449,7 @@ int pci_do_resource_release_and_resize(struct pci_dev *pdev, int resno, int size
struct resource *res = pci_resource_n(pdev, resno);
struct pci_dev_resource *dev_res;
struct pci_bus *bus = pdev->bus;
+ struct pci_dev *bridge = pci_upstream_bridge(pdev);
struct resource *b_win, *r;
LIST_HEAD(saved);
unsigned int i;
@@ -2466,6 +2467,8 @@ int pci_do_resource_release_and_resize(struct pci_dev *pdev, int resno, int size
if (ret)
return ret;
+ down_read(&pci_bus_sem);
+
pci_dev_for_each_resource(pdev, r, i) {
if (i >= PCI_BRIDGE_RESOURCES)
break;
@@ -2484,13 +2487,21 @@ int pci_do_resource_release_and_resize(struct pci_dev *pdev, int resno, int size
pci_resize_resource_set_size(pdev, resno, size);
- if (!bus->self)
- goto out;
+ if (bridge) {
+ ret = pbus_reassign_bridge_resources(bus, res, &saved);
+ if (ret)
+ goto restore;
+ } else {
+ /* No bridge window to adjust; let the core reassign the bus. */
+ pci_bus_assign_resources(bus);
- down_read(&pci_bus_sem);
- ret = pbus_reassign_bridge_resources(bus, res, &saved);
- if (ret)
- goto restore;
+ list_for_each_entry(dev_res, &saved, list) {
+ if (!dev_res->res->parent) {
+ ret = -ENOSPC;
+ goto restore;
+ }
+ }
+ }
out:
up_read(&pci_bus_sem);
diff --git a/drivers/pinctrl/meson/pinctrl-meson-s4.c b/drivers/pinctrl/meson/pinctrl-meson-s4.c
index 872948699e9f..365dafe457a9 100644
--- a/drivers/pinctrl/meson/pinctrl-meson-s4.c
+++ b/drivers/pinctrl/meson/pinctrl-meson-s4.c
@@ -854,7 +854,7 @@ static const char * const i2c1_groups[] = {
static const char * const i2c2_groups[] = {
"i2c2_sda_d", "i2c2_scl_d",
"i2c2_sda_h8", "i2c2_scl_h9",
- "i2c2_sda_h0", "i2c2_scl_h1l,"
+ "i2c2_sda_h0", "i2c2_scl_h1",
};
static const char * const i2c3_groups[] = {
diff --git a/drivers/pinctrl/pinctrl-single.c b/drivers/pinctrl/pinctrl-single.c
index d85e6c1f6321..7d663bed7679 100644
--- a/drivers/pinctrl/pinctrl-single.c
+++ b/drivers/pinctrl/pinctrl-single.c
@@ -1627,7 +1627,8 @@ static int pcs_irq_init_chained_handler(struct pcs_device *pcs,
&pcs_irqdomain_ops,
pcs_soc);
if (!pcs->domain) {
- irq_set_chained_handler(pcs_soc->irq, NULL);
+ pcs_irq_free(pcs);
+ pcs_soc->irq = -1;
return -EINVAL;
}
diff --git a/drivers/pinctrl/sunxi/pinctrl-sun55i-a523-r.c b/drivers/pinctrl/sunxi/pinctrl-sun55i-a523-r.c
index 462aa1c4a5fa..e27e4945def2 100644
--- a/drivers/pinctrl/sunxi/pinctrl-sun55i-a523-r.c
+++ b/drivers/pinctrl/sunxi/pinctrl-sun55i-a523-r.c
@@ -26,7 +26,7 @@ static const u8 a523_r_irq_bank_muxes[SUNXI_PINCTRL_MAX_BANKS] =
static struct sunxi_pinctrl_desc a523_r_pinctrl_data = {
.irq_banks = ARRAY_SIZE(a523_r_irq_bank_map),
.irq_bank_map = a523_r_irq_bank_map,
- .io_bias_cfg_variant = BIAS_VOLTAGE_PIO_POW_MODE_SEL,
+ .io_bias_cfg_variant = BIAS_VOLTAGE_PIO_POW_MODE_CTL_INV,
.pin_base = PL_BASE,
};
diff --git a/drivers/pinctrl/sunxi/pinctrl-sun55i-a523.c b/drivers/pinctrl/sunxi/pinctrl-sun55i-a523.c
index b6f78f1f30ac..88d8acd5bc24 100644
--- a/drivers/pinctrl/sunxi/pinctrl-sun55i-a523.c
+++ b/drivers/pinctrl/sunxi/pinctrl-sun55i-a523.c
@@ -26,7 +26,7 @@ static const u8 a523_irq_bank_muxes[SUNXI_PINCTRL_MAX_BANKS] =
static struct sunxi_pinctrl_desc a523_pinctrl_data = {
.irq_banks = ARRAY_SIZE(a523_irq_bank_map),
.irq_bank_map = a523_irq_bank_map,
- .io_bias_cfg_variant = BIAS_VOLTAGE_PIO_POW_MODE_SEL,
+ .io_bias_cfg_variant = BIAS_VOLTAGE_PIO_POW_MODE_CTL_INV,
};
static int a523_pinctrl_probe(struct platform_device *pdev)
diff --git a/drivers/pinctrl/sunxi/pinctrl-sunxi.c b/drivers/pinctrl/sunxi/pinctrl-sunxi.c
index 93e396041456..95d1dcd6f043 100644
--- a/drivers/pinctrl/sunxi/pinctrl-sunxi.c
+++ b/drivers/pinctrl/sunxi/pinctrl-sunxi.c
@@ -699,6 +699,7 @@ static int sunxi_pinctrl_set_io_bias_cfg(struct sunxi_pinctrl *pctl,
{
unsigned short bank;
unsigned long flags;
+ bool inverted = false;
u32 val, reg;
int uV;
@@ -737,6 +738,9 @@ static int sunxi_pinctrl_set_io_bias_cfg(struct sunxi_pinctrl *pctl,
reg &= ~IO_BIAS_MASK;
writel(reg | val, pctl->membase + sunxi_grp_config_reg(pin));
return 0;
+ case BIAS_VOLTAGE_PIO_POW_MODE_CTL_INV:
+ inverted = true;
+ fallthrough;
case BIAS_VOLTAGE_PIO_POW_MODE_CTL:
val = uV > 1800000 && uV <= 2500000 ? BIT(bank) : 0;
@@ -751,6 +755,8 @@ static int sunxi_pinctrl_set_io_bias_cfg(struct sunxi_pinctrl *pctl,
fallthrough;
case BIAS_VOLTAGE_PIO_POW_MODE_SEL:
val = uV <= 1800000 ? 1 : 0;
+ if (inverted)
+ val = !val;
raw_spin_lock_irqsave(&pctl->lock, flags);
reg = readl(pctl->membase + pctl->pow_mod_sel_offset);
@@ -808,6 +814,21 @@ static void sunxi_pmx_set(struct pinctrl_dev *pctldev,
writel((readl(pctl->membase + reg) & ~mask) | config << shift,
pctl->membase + reg);
+ /*
+ * A pin muxed to gpio_out directly through a pinmux node bypasses
+ * sunxi_pinctrl_gpio_set() and drives whatever its output latch
+ * holds. Now that the pin is in output mode the data register
+ * reads back the latch, so refresh the shadow to keep such pins
+ * driving their pre-existing level.
+ */
+ if (config == SUN4I_FUNC_OUTPUT) {
+ u32 *shadow = &pctl->dat_shadow[pin / PINS_PER_BANK];
+
+ sunxi_data_reg(pctl, pin, ®, &shift, &mask);
+ *shadow = (*shadow & ~mask) |
+ (readl(pctl->membase + reg) & mask);
+ }
+
raw_spin_unlock_irqrestore(&pctl->lock, flags);
}
@@ -964,21 +985,29 @@ static int sunxi_pinctrl_gpio_set(struct gpio_chip *chip, unsigned int offset,
int value)
{
struct sunxi_pinctrl *pctl = gpiochip_get_data(chip);
- u32 reg, shift, mask, val;
+ u32 *shadow = &pctl->dat_shadow[offset / PINS_PER_BANK];
+ u32 reg, shift, mask;
unsigned long flags;
sunxi_data_reg(pctl, offset, ®, &shift, &mask);
raw_spin_lock_irqsave(&pctl->lock, flags);
- val = readl(pctl->membase + reg);
-
+ /*
+ * Reading the data register returns the pin level, not the output
+ * latch, for pins muxed as inputs. A read-modify-write based on
+ * the register would therefore corrupt the latches of input-muxed
+ * pins in the same bank (e.g. an emulated open-drain I2C line
+ * released high), making them drive the wrong level once switched
+ * to output. Base the read-modify-write on a shadow copy of the
+ * latches instead.
+ */
if (value)
- val |= mask;
+ *shadow |= mask;
else
- val &= ~mask;
+ *shadow &= ~mask;
- writel(val, pctl->membase + reg);
+ writel(*shadow, pctl->membase + reg);
raw_spin_unlock_irqrestore(&pctl->lock, flags);
@@ -1504,7 +1533,7 @@ int sunxi_pinctrl_init_with_flags(struct platform_device *pdev,
struct pinctrl_pin_desc *pins;
struct sunxi_pinctrl *pctl;
struct pinmux_ops *pmxops;
- int i, ret, last_pin, pin_idx;
+ int i, ret, last_pin, pin_idx, nbanks;
struct clk *clk;
pctl = devm_kzalloc(&pdev->dev, sizeof(*pctl), GFP_KERNEL);
@@ -1542,6 +1571,37 @@ int sunxi_pinctrl_init_with_flags(struct platform_device *pdev,
if (!pctl->irq_array)
return -ENOMEM;
+ /*
+ * The bus clock has to be enabled before the pinctrl device
+ * registers, as the pin hogs claimed from there access registers.
+ */
+ ret = of_clk_get_parent_count(node);
+ clk = devm_clk_get_enabled(&pdev->dev, ret == 1 ? NULL : "apb");
+ if (IS_ERR(clk))
+ return PTR_ERR(clk);
+
+ /*
+ * Seed the output latch shadow from the hardware so pins the
+ * bootloader left in output mode keep their state; see
+ * sunxi_pinctrl_gpio_set() for why a shadow is needed. This must
+ * happen before the pinctrl device registers, as pin hogs can mux
+ * pins to gpio_out and thereby update the shadow.
+ */
+ last_pin = pctl->desc->pins[pctl->desc->npins - 1].pin.number;
+ nbanks = DIV_ROUND_UP(last_pin + 1 - pctl->desc->pin_base,
+ PINS_PER_BANK);
+ pctl->dat_shadow = devm_kcalloc(&pdev->dev, nbanks,
+ sizeof(*pctl->dat_shadow), GFP_KERNEL);
+ if (!pctl->dat_shadow)
+ return -ENOMEM;
+
+ for (i = 0; i < nbanks; i++) {
+ u32 reg, shift, mask;
+
+ sunxi_data_reg(pctl, i * PINS_PER_BANK, ®, &shift, &mask);
+ pctl->dat_shadow[i] = readl(pctl->membase + reg);
+ }
+
ret = sunxi_pinctrl_build_state(pdev);
if (ret) {
dev_err(&pdev->dev, "dt probe failed: %d\n", ret);
@@ -1596,7 +1656,6 @@ int sunxi_pinctrl_init_with_flags(struct platform_device *pdev,
if (!pctl->chip)
return -ENOMEM;
- last_pin = pctl->desc->pins[pctl->desc->npins - 1].pin.number;
pctl->chip->owner = THIS_MODULE;
pctl->chip->request = gpiochip_generic_request;
pctl->chip->free = gpiochip_generic_free;
@@ -1629,13 +1688,6 @@ int sunxi_pinctrl_init_with_flags(struct platform_device *pdev,
goto gpiochip_error;
}
- ret = of_clk_get_parent_count(node);
- clk = devm_clk_get_enabled(&pdev->dev, ret == 1 ? NULL : "apb");
- if (IS_ERR(clk)) {
- ret = PTR_ERR(clk);
- goto gpiochip_error;
- }
-
pctl->irq = devm_kcalloc(&pdev->dev,
pctl->desc->irq_banks,
sizeof(*pctl->irq),
diff --git a/drivers/pinctrl/sunxi/pinctrl-sunxi.h b/drivers/pinctrl/sunxi/pinctrl-sunxi.h
index ad26e4de16a8..d039d61739fe 100644
--- a/drivers/pinctrl/sunxi/pinctrl-sunxi.h
+++ b/drivers/pinctrl/sunxi/pinctrl-sunxi.h
@@ -85,6 +85,7 @@
#define IO_BIAS_MASK GENMASK(3, 0)
#define SUN4I_FUNC_INPUT 0
+#define SUN4I_FUNC_OUTPUT 1
#define SUN4I_FUNC_IRQ 6
#define SUNXI_PINCTRL_VARIANT_MASK GENMASK(7, 0)
@@ -114,8 +115,10 @@ enum sunxi_desc_bias_voltage {
* Bias voltage is set through PIO_POW_MOD_SEL_REG
* and PIO_POW_MOD_CTL_REG register, as seen on
* A100 and D1 SoC, for example.
+ * Some SoCs invert the encoding for 1.8V vs. 3.3V.
*/
BIAS_VOLTAGE_PIO_POW_MODE_CTL,
+ BIAS_VOLTAGE_PIO_POW_MODE_CTL_INV,
};
struct sunxi_desc_function {
@@ -173,6 +176,12 @@ struct sunxi_pinctrl {
int *irq;
unsigned *irq_array;
raw_spinlock_t lock;
+ /*
+ * Output latch shadow, one word per bank. Seeded lockless at
+ * probe before the pinctrl device registers, protected by @lock
+ * afterwards.
+ */
+ u32 *dat_shadow;
struct pinctrl_dev *pctl_dev;
unsigned long variant;
u32 bank_mem_size;
diff --git a/drivers/remoteproc/qcom_q6v5_adsp.c b/drivers/remoteproc/qcom_q6v5_adsp.c
index b7d472afd5a9..004dab45cf3c 100644
--- a/drivers/remoteproc/qcom_q6v5_adsp.c
+++ b/drivers/remoteproc/qcom_q6v5_adsp.c
@@ -104,6 +104,7 @@ struct qcom_adsp {
struct completion stop_done;
phys_addr_t mem_phys;
+ unsigned long iova;
phys_addr_t mem_reloc;
void *mem_region;
size_t mem_size;
@@ -333,7 +334,7 @@ static void adsp_unmap_carveout(struct rproc *rproc)
struct qcom_adsp *adsp = rproc->priv;
if (adsp->has_iommu)
- iommu_unmap(rproc->domain, adsp->mem_phys, adsp->mem_size);
+ iommu_unmap(rproc->domain, adsp->iova, adsp->mem_size);
}
static int adsp_map_carveout(struct rproc *rproc)
@@ -341,7 +342,6 @@ static int adsp_map_carveout(struct rproc *rproc)
struct qcom_adsp *adsp = rproc->priv;
struct of_phandle_args args;
long long sid;
- unsigned long iova;
int ret;
if (!adsp->has_iommu)
@@ -358,9 +358,9 @@ static int adsp_map_carveout(struct rproc *rproc)
of_node_put(args.np);
/* Add SID configuration for ADSP Firmware to SMMU */
- iova = adsp->mem_phys | (sid << 32);
+ adsp->iova = adsp->mem_phys | (sid << 32);
- ret = iommu_map(rproc->domain, iova, adsp->mem_phys,
+ ret = iommu_map(rproc->domain, adsp->iova, adsp->mem_phys,
adsp->mem_size, IOMMU_READ | IOMMU_WRITE,
GFP_KERNEL);
if (ret) {
diff --git a/drivers/s390/cio/chp.c b/drivers/s390/cio/chp.c
index caa300160b17..1c7b9a39b6f8 100644
--- a/drivers/s390/cio/chp.c
+++ b/drivers/s390/cio/chp.c
@@ -78,6 +78,9 @@ u8 chp_get_sch_opm(struct subchannel *sch)
int opm;
int i;
+ if (!sch->schib.pmcw.dnv)
+ return 0;
+
opm = 0;
chp_id_init(&chpid);
for (i = 0; i < 8; i++) {
diff --git a/drivers/s390/cio/cio.c b/drivers/s390/cio/cio.c
index 21508e4606d5..20ebe0d883c6 100644
--- a/drivers/s390/cio/cio.c
+++ b/drivers/s390/cio/cio.c
@@ -454,7 +454,8 @@ EXPORT_SYMBOL_GPL(cio_commit_config);
/**
* cio_update_schib - Perform stsch and update schib if subchannel is valid.
* @sch: subchannel on which to perform stsch
- * Return zero on success, -ENODEV otherwise.
+ * Return zero on success, -ENODEV if the subchannel is not operational,
+ * -EACCES if the subchannel has no valid device.
*/
int cio_update_schib(struct subchannel *sch)
{
@@ -463,10 +464,12 @@ int cio_update_schib(struct subchannel *sch)
if (stsch(sch->schid, &schib))
return -ENODEV;
- memcpy(&sch->schib, &schib, sizeof(schib));
-
- if (!css_sch_is_valid(&schib))
+ if (!css_sch_is_valid(&schib)) {
+ memset(&sch->schib, 0, sizeof(sch->schib));
return -EACCES;
+ }
+
+ memcpy(&sch->schib, &schib, sizeof(schib));
return 0;
}
diff --git a/drivers/s390/cio/cio.h b/drivers/s390/cio/cio.h
index bad142c536e1..6d28a62bc67a 100644
--- a/drivers/s390/cio/cio.h
+++ b/drivers/s390/cio/cio.h
@@ -7,6 +7,7 @@
#include <linux/mod_devicetable.h>
#include <asm/chpid.h>
#include <asm/cio.h>
+#include <asm/dma-types.h>
#include <asm/fcx.h>
#include <asm/schid.h>
#include <asm/tpi.h>
@@ -49,7 +50,7 @@ struct pmcw {
/* Target SCHIB configuration. */
struct schib_config {
- u64 mba;
+ dma64_t mba;
u32 intparm;
u16 mbi;
u32 isc:3;
@@ -66,7 +67,7 @@ struct schib_config {
struct schib {
struct pmcw pmcw; /* path management control word */
union scsw scsw; /* subchannel status word */
- __u64 mba; /* measurement block address */
+ dma64_t mba; /* measurement block address */
__u8 mda[4]; /* model dependent area */
} __attribute__ ((packed,aligned(4)));
diff --git a/drivers/s390/cio/cmf.c b/drivers/s390/cio/cmf.c
index b7048f2b036e..f39708ce2d2e 100644
--- a/drivers/s390/cio/cmf.c
+++ b/drivers/s390/cio/cmf.c
@@ -184,7 +184,7 @@ static int set_schib(struct ccw_device *cdev, u32 mme, int mbfc,
sch->config.mbfc = mbfc;
/* address can be either a block address or a block index */
if (mbfc)
- sch->config.mba = address;
+ sch->config.mba = address ? virt_to_dma64((void *)address) : 0;
else
sch->config.mbi = address;
diff --git a/drivers/s390/cio/device.c b/drivers/s390/cio/device.c
index 7f8f78b1f8ae..dc6084f1d0f9 100644
--- a/drivers/s390/cio/device.c
+++ b/drivers/s390/cio/device.c
@@ -923,7 +923,7 @@ static int ccw_device_move_to_sch(struct ccw_device *cdev,
if (!sch_is_pseudo_sch(old_sch)) {
spin_lock_irq(&old_sch->lock);
- old_enabled = old_sch->schib.pmcw.ena;
+ old_enabled = old_sch->schib.pmcw.dnv && old_sch->schib.pmcw.ena;
rc = 0;
if (old_enabled)
rc = cio_disable_subchannel(old_sch);
@@ -942,7 +942,7 @@ static int ccw_device_move_to_sch(struct ccw_device *cdev,
CIO_MSG_EVENT(0, "device_move(0.%x.%04x,0.%x.%04x)=%d\n",
cdev->private->dev_id.ssid,
cdev->private->dev_id.devno, sch->schid.ssid,
- sch->schib.pmcw.dev, rc);
+ sch->schid.sch_no, rc);
if (old_enabled) {
/* Try to re-enable the old subchannel. */
spin_lock_irq(&old_sch->lock);
@@ -1208,7 +1208,7 @@ static void io_subchannel_quiesce(struct subchannel *sch)
cdev = sch_get_cdev(sch);
if (cio_is_console(sch->schid))
goto out_unlock;
- if (!sch->schib.pmcw.ena)
+ if (!sch->schib.pmcw.dnv || !sch->schib.pmcw.ena)
goto out_unlock;
ret = cio_disable_subchannel(sch);
if (ret != -EBUSY)
@@ -1255,7 +1255,8 @@ static int recovery_check(struct device *dev, void *data)
switch (cdev->private->state) {
case DEV_STATE_ONLINE:
sch = to_subchannel(cdev->dev.parent);
- if ((sch->schib.pmcw.pam & sch->opm) == sch->vpm)
+ if (sch->schib.pmcw.dnv &&
+ (sch->schib.pmcw.pam & sch->opm) == sch->vpm)
break;
fallthrough;
case DEV_STATE_DISCONNECTED:
diff --git a/drivers/s390/cio/device_fsm.c b/drivers/s390/cio/device_fsm.c
index ab419d40a8a7..b5686c25c83c 100644
--- a/drivers/s390/cio/device_fsm.c
+++ b/drivers/s390/cio/device_fsm.c
@@ -170,6 +170,9 @@ __recover_lost_chpids(struct subchannel *sch, int old_lpm)
int mask, i;
struct chp_id chpid;
+ if (!sch->schib.pmcw.dnv)
+ return;
+
chp_id_init(&chpid);
for (i = 0; i<8; i++) {
mask = 0x80 >> i;
diff --git a/drivers/s390/cio/device_ops.c b/drivers/s390/cio/device_ops.c
index 61c07b4a0fe8..1f7e83fd5087 100644
--- a/drivers/s390/cio/device_ops.c
+++ b/drivers/s390/cio/device_ops.c
@@ -142,6 +142,8 @@ int ccw_device_clear(struct ccw_device *cdev, unsigned long intparm)
if (!cdev || !cdev->dev.parent)
return -ENODEV;
sch = to_subchannel(cdev->dev.parent);
+ if (!sch->schib.pmcw.dnv)
+ return -ENODEV;
if (!sch->schib.pmcw.ena)
return -EINVAL;
if (cdev->private->state == DEV_STATE_NOT_OPER)
@@ -198,6 +200,8 @@ int ccw_device_start_timeout_key(struct ccw_device *cdev, struct ccw1 *cpa,
if (!cdev || !cdev->dev.parent)
return -ENODEV;
sch = to_subchannel(cdev->dev.parent);
+ if (!sch->schib.pmcw.dnv)
+ return -ENODEV;
if (!sch->schib.pmcw.ena)
return -EINVAL;
if (cdev->private->state == DEV_STATE_NOT_OPER)
@@ -379,6 +383,8 @@ int ccw_device_halt(struct ccw_device *cdev, unsigned long intparm)
if (!cdev || !cdev->dev.parent)
return -ENODEV;
sch = to_subchannel(cdev->dev.parent);
+ if (!sch->schib.pmcw.dnv)
+ return -ENODEV;
if (!sch->schib.pmcw.ena)
return -EINVAL;
if (cdev->private->state == DEV_STATE_NOT_OPER)
@@ -413,6 +419,8 @@ int ccw_device_resume(struct ccw_device *cdev)
if (!cdev || !cdev->dev.parent)
return -ENODEV;
sch = to_subchannel(cdev->dev.parent);
+ if (!sch->schib.pmcw.dnv)
+ return -ENODEV;
if (!sch->schib.pmcw.ena)
return -EINVAL;
if (cdev->private->state == DEV_STATE_NOT_OPER)
@@ -482,6 +490,8 @@ struct channel_path_desc_fmt0 *ccw_device_get_chp_desc(struct ccw_device *cdev,
struct chp_id chpid;
sch = to_subchannel(cdev->dev.parent);
+ if (!sch->schib.pmcw.dnv)
+ return NULL;
chp_id_init(&chpid);
chpid.id = sch->schib.pmcw.chpid[chp_idx];
return chp_get_chp_desc(chpid);
@@ -502,9 +512,13 @@ u8 *ccw_device_get_util_str(struct ccw_device *cdev, int chp_idx)
struct chp_id chpid;
u8 *util_str;
+ if (!sch->schib.pmcw.dnv)
+ return NULL;
chp_id_init(&chpid);
chpid.id = sch->schib.pmcw.chpid[chp_idx];
chp = chpid_to_chp(chpid);
+ if (!chp)
+ return NULL;
util_str = kmalloc(sizeof(chp->desc_fmt3.util_str), GFP_KERNEL);
if (!util_str)
@@ -548,6 +562,8 @@ int ccw_device_tm_start_timeout_key(struct ccw_device *cdev, struct tcw *tcw,
int rc;
sch = to_subchannel(cdev->dev.parent);
+ if (!sch->schib.pmcw.dnv)
+ return -ENODEV;
if (!sch->schib.pmcw.ena)
return -EINVAL;
if (cdev->private->state == DEV_STATE_VERIFY) {
@@ -652,6 +668,9 @@ int ccw_device_get_mdc(struct ccw_device *cdev, u8 mask)
struct chp_id chpid;
int mdc = 0, i;
+ if (!sch->schib.pmcw.dnv)
+ return 0;
+
/* Adjust requested path mask to excluded varied off paths. */
if (mask)
mask &= sch->lpm;
@@ -694,6 +713,8 @@ int ccw_device_tm_intrg(struct ccw_device *cdev)
{
struct subchannel *sch = to_subchannel(cdev->dev.parent);
+ if (!sch->schib.pmcw.dnv)
+ return -ENODEV;
if (!sch->schib.pmcw.ena)
return -EINVAL;
if (cdev->private->state != DEV_STATE_ONLINE)
@@ -786,6 +807,8 @@ int ccw_device_get_chpid(struct ccw_device *cdev, int chp_idx, u8 *chpid)
if ((chp_idx < 0) || (chp_idx > 7))
return -EINVAL;
+ if (!sch->schib.pmcw.dnv)
+ return -ENODEV;
mask = 0x80 >> chp_idx;
if (!(sch->schib.pmcw.pim & mask))
return -ENODEV;
diff --git a/drivers/s390/cio/vfio_ccw_fsm.c b/drivers/s390/cio/vfio_ccw_fsm.c
index 5fd94e9d5c61..9a000b0231d6 100644
--- a/drivers/s390/cio/vfio_ccw_fsm.c
+++ b/drivers/s390/cio/vfio_ccw_fsm.c
@@ -399,7 +399,7 @@ static void fsm_close(struct vfio_ccw_private *private,
spin_lock_irq(&sch->lock);
- if (!sch->schib.pmcw.ena)
+ if (!sch->schib.pmcw.dnv || !sch->schib.pmcw.ena)
goto err_unlock;
ret = cio_disable_subchannel(sch);
diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c
index 7b440f4c23c3..ce1ca79c53ba 100644
--- a/drivers/s390/crypto/vfio_ap_ops.c
+++ b/drivers/s390/crypto/vfio_ap_ops.c
@@ -277,9 +277,9 @@ static void vfio_ap_free_aqic_resources(struct vfio_ap_queue *q)
{
if (!q)
return;
- if (q->saved_isc != VFIO_AP_ISC_INVALID &&
- !WARN_ON(!(q->matrix_mdev && q->matrix_mdev->kvm))) {
- kvm_s390_gisc_unregister(q->matrix_mdev->kvm, q->saved_isc);
+ if (q->saved_isc != VFIO_AP_ISC_INVALID) {
+ if (!WARN_ON(!q->matrix_mdev) && q->matrix_mdev->kvm)
+ kvm_s390_gisc_unregister(q->matrix_mdev->kvm, q->saved_isc);
q->saved_isc = VFIO_AP_ISC_INVALID;
}
if (q->saved_iova && !WARN_ON(!q->matrix_mdev)) {
@@ -1935,6 +1935,8 @@ static int apq_status_check(int apqn, struct ap_queue_status *status)
* a value indicating a reset needs to be performed again.
*/
return -EAGAIN;
+ case AP_RESPONSE_Q_NOT_AVAIL:
+ return -ENODEV;
default:
WARN(true,
"failed to verify reset of queue %02x.%04x: TAPQ rc=%u\n",
@@ -1961,6 +1963,10 @@ static void apq_reset_check(struct work_struct *reset_work)
ret = apq_status_check(q->apqn, &status);
if (ret == -EIO)
return;
+ if (ret == -ENODEV) {
+ vfio_ap_free_aqic_resources(q);
+ return;
+ }
if (ret == -EBUSY) {
pr_notice_ratelimited(WAIT_MSG, elapsed,
AP_QID_CARD(q->apqn),
@@ -2004,6 +2010,7 @@ static void vfio_ap_mdev_reset_queue(struct vfio_ap_queue *q)
break;
case AP_RESPONSE_DECONFIGURED:
case AP_RESPONSE_CHECKSTOPPED:
+ case AP_RESPONSE_Q_NOT_AVAIL:
vfio_ap_free_aqic_resources(q);
break;
default:
@@ -2528,12 +2535,15 @@ void vfio_ap_mdev_remove_queue(struct ap_device *apdev)
/*
* If the queue is not in the host's AP configuration, then resetting
* it will fail with response code 01, (APQN not valid); so, let's make
- * sure it is in the host's config.
+ * sure it is in the host's config. If it is not, free the KVM GISC
+ * resources.
*/
if (test_bit_inv(apid, (unsigned long *)matrix_dev->info.apm) &&
test_bit_inv(apqi, (unsigned long *)matrix_dev->info.aqm)) {
vfio_ap_mdev_reset_queue(q);
flush_work(&q->reset_work);
+ } else {
+ vfio_ap_free_aqic_resources(q);
}
done:
diff --git a/drivers/scsi/libiscsi_tcp.c b/drivers/scsi/libiscsi_tcp.c
index 7223bb18b048..d35f93451ee9 100644
--- a/drivers/scsi/libiscsi_tcp.c
+++ b/drivers/scsi/libiscsi_tcp.c
@@ -480,6 +480,9 @@ static int iscsi_tcp_data_in(struct iscsi_conn *conn, struct iscsi_task *task)
int datasn = be32_to_cpu(rhdr->datasn);
unsigned total_in_length = task->sc->sdb.length;
+ if (task->sc->sc_data_direction != DMA_FROM_DEVICE)
+ return ISCSI_ERR_PROTO;
+
/*
* lib iscsi will update this in the completion handling if there
* is status.
diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c
index d578b476d979..116b5ecf15d6 100644
--- a/drivers/scsi/megaraid/megaraid_sas_base.c
+++ b/drivers/scsi/megaraid/megaraid_sas_base.c
@@ -7887,7 +7887,9 @@ megasas_resume(struct device *dev)
goto fail_init_mfi;
}
- if (megasas_get_ctrl_info(instance) != DCMD_SUCCESS)
+ scoped_guard(mutex, &instance->reset_mutex)
+ rval = megasas_get_ctrl_info(instance);
+ if (rval != DCMD_SUCCESS)
goto fail_init_mfi;
tasklet_init(&instance->isr_tasklet, instance->instancet->tasklet,
diff --git a/drivers/scsi/sd_zbc.c b/drivers/scsi/sd_zbc.c
index a8db66428f80..6f566d4f7706 100644
--- a/drivers/scsi/sd_zbc.c
+++ b/drivers/scsi/sd_zbc.c
@@ -595,7 +595,7 @@ int sd_zbc_revalidate_zones(struct scsi_disk *sdkp)
int sd_zbc_read_zones(struct scsi_disk *sdkp, struct queue_limits *lim,
u8 buf[SD_BUF_SIZE])
{
- unsigned int nr_zones;
+ u64 nr_zones;
u32 zone_blocks = 0;
int ret;
@@ -627,6 +627,12 @@ int sd_zbc_read_zones(struct scsi_disk *sdkp, struct queue_limits *lim,
goto err;
nr_zones = round_up(sdkp->capacity, zone_blocks) >> ilog2(zone_blocks);
+ if (nr_zones > INT_MAX) {
+ sd_printk(KERN_ERR, sdkp, "Too many zones (%llu)\n",
+ nr_zones);
+ ret = -EINVAL;
+ goto err;
+ }
sdkp->early_zone_info.nr_zones = nr_zones;
sdkp->early_zone_info.zone_blocks = zone_blocks;
diff --git a/drivers/thermal/gov_step_wise.c b/drivers/thermal/gov_step_wise.c
index ea277c466d8d..4fa4377f0d42 100644
--- a/drivers/thermal/gov_step_wise.c
+++ b/drivers/thermal/gov_step_wise.c
@@ -65,14 +65,12 @@ static unsigned long get_target_state(struct thermal_instance *instance,
min(instance->lower + 1, instance->upper),
instance->upper);
} else if (trend == THERMAL_TREND_DROPPING) {
- if (cur_state <= instance->lower)
- return THERMAL_NO_TARGET;
-
/*
- * If 'throttle' is false, no mitigation is necessary, so
- * request the lower state for this instance.
+ * If 'throttle' is false, no mitigation is necessary and
+ * passive polling is already deactivated, so clear this
+ * instance state by returning THERMAL_NO_TARGET.
*/
- return instance->lower;
+ return THERMAL_NO_TARGET;
}
return instance->target;
diff --git a/fs/autofs/inode.c b/fs/autofs/inode.c
index 9593d407cbc6..54e5c3a9cba2 100644
--- a/fs/autofs/inode.c
+++ b/fs/autofs/inode.c
@@ -51,6 +51,10 @@ void autofs_kill_sb(struct super_block *sb)
if (sbi) {
/* Free wait queues, close pipe */
autofs_catatonic_mode(sbi);
+ if (sbi->pipe) {
+ fput(sbi->pipe);
+ sbi->pipe = NULL;
+ }
put_pid(sbi->oz_pgrp);
}
diff --git a/fs/bpf_fs_kfuncs.c b/fs/bpf_fs_kfuncs.c
index b5a0736b120f..b60e0698ce13 100644
--- a/fs/bpf_fs_kfuncs.c
+++ b/fs/bpf_fs_kfuncs.c
@@ -403,10 +403,6 @@ BTF_ID(func, bpf_lsm_inode_rmdir)
BTF_ID(func, bpf_lsm_inode_setattr)
BTF_ID(func, bpf_lsm_inode_setxattr)
BTF_ID(func, bpf_lsm_inode_unlink)
-#ifdef CONFIG_SECURITY_PATH
-BTF_ID(func, bpf_lsm_path_unlink)
-BTF_ID(func, bpf_lsm_path_rmdir)
-#endif /* CONFIG_SECURITY_PATH */
BTF_SET_END(d_inode_locked_hooks)
bool bpf_lsm_has_d_inode_locked(const struct bpf_prog *prog)
diff --git a/fs/erofs/fscache.c b/fs/erofs/fscache.c
index 7a346e20f7b7..754f7d04488d 100644
--- a/fs/erofs/fscache.c
+++ b/fs/erofs/fscache.c
@@ -67,6 +67,8 @@ static void erofs_fscache_req_complete(struct erofs_fscache_rq *req)
continue;
if (!failed)
folio_mark_uptodate(folio);
+ /* Skip the entire folio before unlocking allows it to be split. */
+ xas_advance(&xas, folio_next_index(folio) - 1);
folio_unlock(folio);
}
rcu_read_unlock();
diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index d2510c818da9..094d2872881d 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -693,19 +693,34 @@ static bool isw_prepare_wbs_switch(struct bdi_writeback *new_wb,
struct inode_switch_wbs_context *isw,
struct list_head *list, int *nr)
{
- struct inode *inode;
+ struct inode *inode, *tmp;
+ LIST_HEAD(scanned);
+ bool full = false;
+
+ /*
+ * Walk from the oldest end and move scanned inodes to the newest
+ * end, so the next scan resumes at unscanned inodes instead of
+ * re-walking an ever-growing run of prepared and skipped ones.
+ * For b_dirty_time this keeps the oldest unscanned inode at the
+ * end move_expired_inodes() picks from; b_attached is unordered.
+ */
+ list_for_each_entry_safe_reverse(inode, tmp, list, i_io_list) {
+ list_move(&inode->i_io_list, &scanned);
- list_for_each_entry(inode, list, i_io_list) {
if (!inode_prepare_wbs_switch(inode, new_wb))
continue;
isw->inodes[*nr] = inode;
(*nr)++;
- if (*nr >= WB_MAX_INODES_PER_ISW - 1)
- return true;
+ if (*nr >= WB_MAX_INODES_PER_ISW - 1) {
+ full = true;
+ break;
+ }
}
- return false;
+ list_splice(&scanned, list);
+
+ return full;
}
/**
diff --git a/fs/hfsplus/super.c b/fs/hfsplus/super.c
index d031c77fb4eb..0d619945d02d 100644
--- a/fs/hfsplus/super.c
+++ b/fs/hfsplus/super.c
@@ -652,7 +652,6 @@ static int hfsplus_fill_super(struct super_block *sb, struct fs_context *fc)
kfree(sbi->s_vhdr_buf);
kfree(sbi->s_backup_vhdr_buf);
out_unload_nls:
- unload_nls(sbi->nls);
unload_nls(nls);
return err;
}
diff --git a/fs/inode.c b/fs/inode.c
index a931126398ce..44891fb7cd43 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -868,7 +868,6 @@ void evict_inodes(struct super_block *sb)
struct inode *inode;
LIST_HEAD(dispose);
-again:
spin_lock(&sb->s_inode_list_lock);
list_for_each_entry(inode, &sb->s_inodes, i_sb_list) {
if (icount_read(inode))
@@ -887,19 +886,19 @@ void evict_inodes(struct super_block *sb)
inode->i_state |= I_FREEING;
inode_lru_list_del(inode);
spin_unlock(&inode->i_lock);
- list_add(&inode->i_lru, &dispose);
/*
- * We can have a ton of inodes to evict at unmount time given
- * enough memory, check to see if we need to go to sleep for a
- * bit so we don't livelock.
+ * Keep this inode out of dispose so it stays on s_inodes while
+ * the list lock is dropped. I_FREEING prevents new references
+ * and leaves eviction to us, so we can resume the walk from it.
*/
if (need_resched()) {
spin_unlock(&sb->s_inode_list_lock);
cond_resched();
dispose_list(&dispose);
- goto again;
+ spin_lock(&sb->s_inode_list_lock);
}
+ list_add(&inode->i_lru, &dispose);
}
spin_unlock(&sb->s_inode_list_lock);
diff --git a/fs/kernfs/mount.c b/fs/kernfs/mount.c
index 76eaf64b9d9e..5d265e63956e 100644
--- a/fs/kernfs/mount.c
+++ b/fs/kernfs/mount.c
@@ -433,8 +433,8 @@ void kernfs_kill_sb(struct super_block *sb)
up_write(&root->kernfs_supers_rwsem);
/*
- * Remove the superblock from fs_supers/s_instances
- * so we can't find it, before freeing kernfs_super_info.
+ * Mark the superblock dead so sget_fc() can't find it,
+ * before freeing kernfs_super_info.
*/
kill_anon_super(sb);
kfree(info);
diff --git a/fs/ntfs3/inode.c b/fs/ntfs3/inode.c
index 1b7bb924fc00..f34e0a786bb6 100644
--- a/fs/ntfs3/inode.c
+++ b/fs/ntfs3/inode.c
@@ -1668,10 +1668,10 @@ int ntfs_create_inode(struct mnt_idmap *idmap, struct inode *dir,
goto out6;
/*
- * Call 'd_instantiate' after inode->i_op is set
+ * Call 'd_instantiate_new' after inode->i_op is set
* but before finish_open.
*/
- d_instantiate(dentry, inode);
+ d_instantiate_new(dentry, inode);
/* Set original time. inode times (i_ctime) may be changed in ntfs_init_acl. */
inode_set_atime_to_ts(inode, ni->i_crtime);
@@ -1719,9 +1719,6 @@ int ntfs_create_inode(struct mnt_idmap *idmap, struct inode *dir,
if (!fnd)
ni_unlock(dir_ni);
- if (!err)
- unlock_new_inode(inode);
-
return err;
}
diff --git a/fs/ocfs2/namei.c b/fs/ocfs2/namei.c
index 20276ad6e17e..42afc21324d4 100644
--- a/fs/ocfs2/namei.c
+++ b/fs/ocfs2/namei.c
@@ -335,13 +335,8 @@ static int ocfs2_mknod(struct mnt_idmap *idmap,
goto leave;
/* calculate meta data/clusters for setting security and acl xattr */
- status = ocfs2_calc_xattr_init(dir, mode, &si, &want_clusters,
- &xattr_credits, &want_meta,
- &acl_state);
- if (status < 0) {
- mlog_errno(status);
- goto leave;
- }
+ ocfs2_calc_xattr_init(dir, mode, &si, &want_clusters, &xattr_credits,
+ &want_meta, &acl_state);
/* Reserve a cluster if creating an extent based directory. */
if (S_ISDIR(mode) && !ocfs2_supports_inline_data(osb)) {
diff --git a/fs/ocfs2/xattr.c b/fs/ocfs2/xattr.c
index 3a796d9159d8..b5029c4e3784 100644
--- a/fs/ocfs2/xattr.c
+++ b/fs/ocfs2/xattr.c
@@ -607,12 +607,11 @@ int ocfs2_calc_security_init(struct inode *dir,
return ret;
}
-int ocfs2_calc_xattr_init(struct inode *dir, umode_t mode,
- struct ocfs2_security_xattr_info *si,
- int *want_clusters, int *xattr_credits,
- int *want_meta, struct ocfs2_acl_state *acl_state)
+void ocfs2_calc_xattr_init(struct inode *dir, umode_t mode,
+ struct ocfs2_security_xattr_info *si,
+ int *want_clusters, int *xattr_credits,
+ int *want_meta, struct ocfs2_acl_state *acl_state)
{
- int ret = 0;
struct ocfs2_super *osb = OCFS2_SB(dir->i_sb);
int s_size = 0, a_size = 0, acl_len = 0, new_clusters;
@@ -634,7 +633,7 @@ int ocfs2_calc_xattr_init(struct inode *dir, umode_t mode,
}
if (!(s_size + a_size))
- return ret;
+ return;
/*
* The max space of security xattr taken inline is
@@ -700,8 +699,6 @@ int ocfs2_calc_xattr_init(struct inode *dir, umode_t mode,
}
}
}
-
- return ret;
}
static int ocfs2_xattr_extend_allocation(struct inode *inode,
diff --git a/fs/ocfs2/xattr.h b/fs/ocfs2/xattr.h
index 5cdd6c6b4006..aeba323e14b3 100644
--- a/fs/ocfs2/xattr.h
+++ b/fs/ocfs2/xattr.h
@@ -57,10 +57,10 @@ int ocfs2_calc_security_init(struct inode *,
int *, int *, struct ocfs2_alloc_context **);
struct ocfs2_acl_state;
-int ocfs2_calc_xattr_init(struct inode *dir, umode_t mode,
- struct ocfs2_security_xattr_info *si,
- int *want_clusters, int *xattr_credits,
- int *want_meta, struct ocfs2_acl_state *acl_state);
+void ocfs2_calc_xattr_init(struct inode *dir, umode_t mode,
+ struct ocfs2_security_xattr_info *si,
+ int *want_clusters, int *xattr_credits,
+ int *want_meta, struct ocfs2_acl_state *acl_state);
/*
* xattrs can live inside an inode, as part of an external xattr block,
diff --git a/fs/smb/client/cached_dir.c b/fs/smb/client/cached_dir.c
index cef855456063..1bd15cc8194c 100644
--- a/fs/smb/client/cached_dir.c
+++ b/fs/smb/client/cached_dir.c
@@ -8,6 +8,7 @@
#include <linux/namei.h>
#include "cifsglob.h"
#include "cifsproto.h"
+#include "../common/smb2status.h"
#include "cifs_debug.h"
#include "smb2proto.h"
#include "cached_dir.h"
@@ -319,25 +320,37 @@ int open_cached_dir(unsigned int xid, struct cifs_tcon *tcon,
rc = compound_send_recv(xid, ses, server,
flags, 2, rqst,
resp_buftype, rsp_iov);
- if (rc) {
- if (rc == -EREMCHG) {
- tcon->need_reconnect = true;
- pr_warn_once("server share %s deleted\n",
- tcon->tree_name);
- }
- goto oshr_free;
+ if (rc == -EREMCHG) {
+ tcon->need_reconnect = true;
+ pr_warn_once("server share %s deleted\n",
+ tcon->tree_name);
}
- cfid->is_open = true;
- spin_lock(&cfids->cfid_list_lock);
+ if (!rsp_iov[0].iov_base || rsp_iov[0].iov_len < sizeof(*o_rsp)) {
+ if (!rc)
+ rc = -EIO;
+ goto oshr_free;
+ }
o_rsp = (struct smb2_create_rsp *)rsp_iov[0].iov_base;
+ if (o_rsp->hdr.Status != STATUS_SUCCESS) {
+ if (!rc)
+ rc = -EIO;
+ goto oshr_free;
+ }
+
oparms.fid->persistent_fid = o_rsp->PersistentFileId;
oparms.fid->volatile_fid = o_rsp->VolatileFileId;
#ifdef CONFIG_CIFS_DEBUG2
oparms.fid->mid = le64_to_cpu(o_rsp->hdr.MessageId);
#endif /* CIFS_DEBUG2 */
+ cfid->is_open = true;
+ atomic_inc(&tcon->num_remote_opens);
+ if (rc)
+ goto oshr_free;
+
+ spin_lock(&cfids->cfid_list_lock);
if (o_rsp->OplockLevel != SMB2_OPLOCK_LEVEL_LEASE) {
spin_unlock(&cfids->cfid_list_lock);
@@ -404,7 +417,6 @@ int open_cached_dir(unsigned int xid, struct cifs_tcon *tcon,
close_cached_dir(cfid);
} else {
*ret_cfid = cfid;
- atomic_inc(&tcon->num_remote_opens);
}
kfree(utf16_path);
diff --git a/fs/smb/client/smb2inode.c b/fs/smb/client/smb2inode.c
index 2e28a965a25d..e419b303ce9c 100644
--- a/fs/smb/client/smb2inode.c
+++ b/fs/smb/client/smb2inode.c
@@ -696,8 +696,10 @@ static int smb2_compound_op(const unsigned int xid, struct cifs_tcon *tcon,
/* smb2_parse_contexts() fills idata->fi.IndexNumber */
rc = smb2_parse_contexts(server, &rsp_iov[0], &oparms->fid->epoch,
oparms->fid->lease_key, &oplock, &idata->fi, NULL);
- if (rc)
+ if (rc) {
cifs_dbg(VFS, "rc: %d parsing context of compound op\n", rc);
+ tmp_rc = rc;
+ }
}
for (i = 0; i < num_cmds; i++) {
diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c
index dcf0a9b9647c..19c82ad224a1 100644
--- a/fs/smb/client/smb2ops.c
+++ b/fs/smb/client/smb2ops.c
@@ -4457,25 +4457,37 @@ smb3_create_lease_buf(u8 *lease_key, u8 oplock, u8 *parent_lease_key, __le32 fla
static __u8
smb2_parse_lease_buf(void *buf, __u16 *epoch, char *lease_key)
{
- struct create_lease *lc = (struct create_lease *)buf;
+ struct create_context *cc = buf;
+ struct lease_context lc;
*epoch = 0; /* not used */
- if (lc->lcontext.LeaseFlags & SMB2_LEASE_FLAG_BREAK_IN_PROGRESS_LE)
+ if (le32_to_cpu(cc->DataLength) != sizeof(lc))
+ return 0;
+
+ memcpy(&lc, (u8 *)cc + le16_to_cpu(cc->DataOffset), sizeof(lc));
+ if (lc.LeaseFlags & SMB2_LEASE_FLAG_BREAK_IN_PROGRESS_LE)
return SMB2_OPLOCK_LEVEL_NOCHANGE;
- return le32_to_cpu(lc->lcontext.LeaseState);
+ return le32_to_cpu(lc.LeaseState);
}
static __u8
smb3_parse_lease_buf(void *buf, __u16 *epoch, char *lease_key)
{
- struct create_lease_v2 *lc = (struct create_lease_v2 *)buf;
+ struct create_context *cc = buf;
+ struct lease_context_v2 lc;
+
+ if (le32_to_cpu(cc->DataLength) != sizeof(lc)) {
+ *epoch = 0;
+ return 0;
+ }
- *epoch = le16_to_cpu(lc->lcontext.Epoch);
- if (lc->lcontext.LeaseFlags & SMB2_LEASE_FLAG_BREAK_IN_PROGRESS_LE)
+ memcpy(&lc, (u8 *)cc + le16_to_cpu(cc->DataOffset), sizeof(lc));
+ *epoch = le16_to_cpu(lc.Epoch);
+ if (lc.LeaseFlags & SMB2_LEASE_FLAG_BREAK_IN_PROGRESS_LE)
return SMB2_OPLOCK_LEVEL_NOCHANGE;
if (lease_key)
- memcpy(lease_key, &lc->lcontext.LeaseKey, SMB2_LEASE_KEY_SIZE);
- return le32_to_cpu(lc->lcontext.LeaseState);
+ memcpy(lease_key, lc.LeaseKey, SMB2_LEASE_KEY_SIZE);
+ return le32_to_cpu(lc.LeaseState);
}
static unsigned int
diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
index c07c5bb32140..e0edfce3fbe5 100644
--- a/fs/smb/client/smb2pdu.c
+++ b/fs/smb/client/smb2pdu.c
@@ -2280,23 +2280,32 @@ create_reconnect_durable_buf(struct cifs_fid *fid)
static void
parse_query_id_ctxt(struct create_context *cc, struct smb2_file_all_info *buf)
{
- struct create_disk_id_rsp *pdisk_id = (struct create_disk_id_rsp *)cc;
+ u16 doff = le16_to_cpu(cc->DataOffset);
+ u32 dlen = le32_to_cpu(cc->DataLength);
+ u8 *beg;
- cifs_dbg(FYI, "parse query id context 0x%llx 0x%llx\n",
- pdisk_id->DiskFileId, pdisk_id->VolumeId);
- buf->IndexNumber = pdisk_id->DiskFileId;
+ if (dlen < sizeof(__le64))
+ return;
+
+ beg = (u8 *)cc + doff;
+ memcpy(&buf->IndexNumber, beg, sizeof(__le64));
+ cifs_dbg(FYI, "parse query id context 0x%llx\n",
+ le64_to_cpu(buf->IndexNumber));
}
static void
parse_posix_ctxt(struct create_context *cc, struct smb2_file_all_info *info,
struct create_posix_rsp *posix)
{
- int sid_len;
u8 *beg = (u8 *)cc + le16_to_cpu(cc->DataOffset);
- u8 *end = beg + le32_to_cpu(cc->DataLength);
+ u32 dlen = le32_to_cpu(cc->DataLength);
+ u8 *end = beg + dlen;
+ int sid_len;
u8 *sid;
memset(posix, 0, sizeof(*posix));
+ if (dlen < 3 * sizeof(__le32))
+ return;
posix->nlink = get_unaligned_le32(beg);
posix->reparse_tag = get_unaligned_le32(beg + 4);
@@ -2332,6 +2341,7 @@ int smb2_parse_contexts(struct TCP_Server_Info *server,
struct smb2_create_rsp *rsp = rsp_iov->iov_base;
struct create_context *cc;
size_t rem, off, len;
+ size_t cc_len;
size_t doff, dlen;
size_t noff, nlen;
char *name;
@@ -2354,29 +2364,41 @@ int smb2_parse_contexts(struct TCP_Server_Info *server,
buf->IndexNumber = 0;
while (rem >= sizeof(*cc)) {
+ off = le32_to_cpu(cc->Next);
+ if (off) {
+ if ((off & 0x7) || off >= rem || off < sizeof(*cc))
+ return -EINVAL;
+ cc_len = off;
+ } else {
+ cc_len = rem;
+ }
+
doff = le16_to_cpu(cc->DataOffset);
dlen = le32_to_cpu(cc->DataLength);
- if (check_add_overflow(doff, dlen, &len) || len > rem)
+ if (doff < sizeof(*cc) ||
+ check_add_overflow(doff, dlen, &len) || len > cc_len)
return -EINVAL;
noff = le16_to_cpu(cc->NameOffset);
nlen = le16_to_cpu(cc->NameLength);
- if (noff + nlen > doff)
+ if (noff < sizeof(*cc) ||
+ check_add_overflow(noff, nlen, &len) || len > cc_len ||
+ (dlen && len > doff))
return -EINVAL;
name = (char *)cc + noff;
switch (nlen) {
case 4:
- if (!strncmp(name, SMB2_CREATE_REQUEST_LEASE, 4)) {
+ if (dlen && !strncmp(name, SMB2_CREATE_REQUEST_LEASE, 4)) {
*oplock = server->ops->parse_lease_buf(cc, epoch,
lease_key);
- } else if (buf &&
+ } else if (dlen && buf &&
!strncmp(name, SMB2_CREATE_QUERY_ON_DISK_ID, 4)) {
parse_query_id_ctxt(cc, buf);
}
break;
case 16:
- if (posix && !memcmp(name, smb3_create_tag_posix, 16))
+ if (dlen && posix && !memcmp(name, smb3_create_tag_posix, 16))
parse_posix_ctxt(cc, buf, posix);
break;
default:
@@ -2388,13 +2410,18 @@ int smb2_parse_contexts(struct TCP_Server_Info *server,
}
off = le32_to_cpu(cc->Next);
- if (!off)
+ if (!off) {
+ rem = 0;
break;
+ }
if (check_sub_overflow(rem, off, &rem))
return -EINVAL;
cc = (struct create_context *)((u8 *)cc + off);
}
+ if (rem)
+ return -EINVAL;
+
if (rsp->OplockLevel != SMB2_OPLOCK_LEVEL_LEASE)
*oplock = rsp->OplockLevel;
diff --git a/fs/squashfs/xz_wrapper.c b/fs/squashfs/xz_wrapper.c
index 6c49481a2f8c..7d54cd524d6d 100644
--- a/fs/squashfs/xz_wrapper.c
+++ b/fs/squashfs/xz_wrapper.c
@@ -57,10 +57,10 @@ static void *squashfs_xz_comp_opts(struct squashfs_sb_info *msblk,
opts->dict_size = le32_to_cpu(comp_opts->dictionary_size);
- /* the dictionary size should be 2^n or 2^n+2^(n+1) */
+ /* the dictionary size should be positive and 2^n or 2^n+2^(n+1) */
n = ffs(opts->dict_size) - 1;
- if (opts->dict_size != (1 << n) && opts->dict_size != (1 << n) +
- (1 << (n + 1))) {
+ if (opts->dict_size <= 0 || (opts->dict_size != (1 << n) &&
+ opts->dict_size != (1 << n) + (1 << (n + 1)))) {
err = -EIO;
goto out;
}
diff --git a/fs/super.c b/fs/super.c
index c293b2efebe0..0877e043a09c 100644
--- a/fs/super.c
+++ b/fs/super.c
@@ -404,11 +404,16 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags,
static void __put_super(struct super_block *s)
{
if (!--s->s_count) {
+ struct file_system_type *type = s->s_type;
+
list_del_init(&s->s_list);
+ hlist_del_init(&s->s_instances);
WARN_ON(s->s_dentry_lru.node);
WARN_ON(s->s_inode_lru.node);
WARN_ON(s->s_mounts);
call_rcu(&s->rcu, destroy_super_rcu);
+ /* The unlink above may touch type->fs_supers, so drop it last. */
+ put_filesystem(type);
}
}
@@ -434,24 +439,17 @@ static void kill_super_notify(struct super_block *sb)
if (sb->s_flags & SB_DEAD)
return;
- /*
- * Remove it from @fs_supers so it isn't found by new
- * sget{_fc}() walkers anymore. Any concurrent mounter still
- * managing to grab a temporary reference is guaranteed to
- * already see SB_DYING and will wait until we notify them about
- * SB_DEAD.
- */
- spin_lock(&sb_lock);
- hlist_del_init(&sb->s_instances);
- spin_unlock(&sb_lock);
-
/*
* Let concurrent mounts know that this thing is really dead.
- * We don't need @sb->s_umount here as every concurrent caller
- * will see SB_DYING and either discard the superblock or wait
- * for SB_DEAD.
+ * sget{_fc}() skips SB_DEAD superblocks and calls test() under
+ * sb_lock, so set it under sb_lock: once we return no test()
+ * runs on this superblock anymore and none will start. Everyone
+ * else already saw SB_DYING and either discarded the superblock
+ * or waits for SB_DEAD.
*/
+ spin_lock(&sb_lock);
super_wake(sb, SB_DEAD);
+ spin_unlock(&sb_lock);
}
/**
@@ -482,7 +480,6 @@ void deactivate_locked_super(struct super_block *s)
list_lru_destroy(&s->s_dentry_lru);
list_lru_destroy(&s->s_inode_lru);
- put_filesystem(fs);
put_super(s);
} else {
super_unlock_excl(s);
@@ -668,12 +665,12 @@ void generic_shutdown_super(struct super_block *sb)
}
/*
* Broadcast to everyone that grabbed a temporary reference to this
- * superblock before we removed it from @fs_supers that the superblock
- * is dying. Every walker of @fs_supers outside of sget{_fc}() will now
- * discard this superblock and treat it as dead.
+ * superblock that it is dying. Every walker of @fs_supers outside
+ * of sget{_fc}() will now discard this superblock and treat it as
+ * dead.
*
- * We leave the superblock on @fs_supers so it can be found by
- * sget{_fc}() until we passed sb->kill_sb().
+ * sget{_fc}() keeps finding the superblock until SB_DEAD is set, so
+ * a concurrent mounter waits until we passed sb->kill_sb().
*/
super_wake(sb, SB_DYING);
super_unlock_excl(sb);
@@ -752,6 +749,9 @@ struct super_block *sget_fc(struct fs_context *fc,
spin_lock(&sb_lock);
if (test) {
hlist_for_each_entry(old, &fc->fs_type->fs_supers, s_instances) {
+ /* Only unlinked at the last passive reference. */
+ if (super_flags(old, SB_DEAD))
+ continue;
if (test(old, fc))
goto share_extant_sb;
}
@@ -828,6 +828,9 @@ struct super_block *sget(struct file_system_type *type,
spin_lock(&sb_lock);
if (test) {
hlist_for_each_entry(old, &type->fs_supers, s_instances) {
+ /* Only unlinked at the last passive reference. */
+ if (super_flags(old, SB_DEAD))
+ continue;
if (!test(old, data))
continue;
if (user_ns != old->s_user_ns) {
diff --git a/fs/xfs/libxfs/xfs_metafile.c b/fs/xfs/libxfs/xfs_metafile.c
index b02e3d6c0868..df999009680a 100644
--- a/fs/xfs/libxfs/xfs_metafile.c
+++ b/fs/xfs/libxfs/xfs_metafile.c
@@ -292,14 +292,14 @@ xfs_metafile_resv_init(
goto out_unlock;
/*
- * Space taken by the per-AG metadata btrees are accounted on-disk as
- * used space. We therefore only hide the space that is reserved but
- * not used by the trees.
+ * Space taken by metadata btrees are accounted on-disk as used space.
+ * We therefore only hide the space that is reserved but not used by
+ * the trees.
*/
if (used > target)
target = used;
else if (target > dblocks_avail)
- target = dblocks_avail;
+ target = max(dblocks_avail, used);
hidden_space = target - used;
error = xfs_dec_fdblocks(mp, hidden_space, true);
diff --git a/fs/xfs/libxfs/xfs_rtrefcount_btree.c b/fs/xfs/libxfs/xfs_rtrefcount_btree.c
index 771ababe7edf..72d778acd79d 100644
--- a/fs/xfs/libxfs/xfs_rtrefcount_btree.c
+++ b/fs/xfs/libxfs/xfs_rtrefcount_btree.c
@@ -617,7 +617,7 @@ xfs_rtrefcountbt_from_disk(
fpp = xfs_rtrefcount_droot_ptr_addr(dblock, 1, maxrecs);
tpp = xfs_rtrefcount_broot_ptr_addr(mp, rblock, 1, rblocklen);
numrecs = be16_to_cpu(dblock->bb_numrecs);
- memcpy(tkp, fkp, 2 * sizeof(*fkp) * numrecs);
+ memcpy(tkp, fkp, sizeof(*fkp) * numrecs);
memcpy(tpp, fpp, sizeof(*fpp) * numrecs);
} else {
frp = xfs_rtrefcount_droot_rec_addr(dblock, 1);
@@ -703,7 +703,7 @@ xfs_rtrefcountbt_to_disk(
fpp = xfs_rtrefcount_broot_ptr_addr(mp, rblock, 1, rblocklen);
tpp = xfs_rtrefcount_droot_ptr_addr(dblock, 1, maxrecs);
numrecs = be16_to_cpu(rblock->bb_numrecs);
- memcpy(tkp, fkp, 2 * sizeof(*fkp) * numrecs);
+ memcpy(tkp, fkp, sizeof(*fkp) * numrecs);
memcpy(tpp, fpp, sizeof(*fpp) * numrecs);
} else {
frp = xfs_rtrefcount_rec_addr(rblock, 1);
diff --git a/fs/xfs/scrub/dir_repair.c b/fs/xfs/scrub/dir_repair.c
index 36432705c44a..ad3d414d9705 100644
--- a/fs/xfs/scrub/dir_repair.c
+++ b/fs/xfs/scrub/dir_repair.c
@@ -1381,9 +1381,24 @@ xrep_dir_live_update(
if (p->delta > 0)
error = xrep_dir_stash_createname(rd, p->name,
p->ip->i_ino);
- else
- error = xrep_dir_stash_removename(rd, p->name,
+ else {
+ /*
+ * xfs_dentry_to_name in unlink or rename-exchange can
+ * pass us names with ftype FT_UNKNOWN, but we really
+ * must know the ftype of the child that is being
+ * removed so that we can do nlink updates correctly
+ * without holding inode references.
+ */
+ struct xfs_name name = {
+ .name = p->name->name,
+ .len = p->name->len,
+ .type = xfs_mode_to_ftype(
+ VFS_IC(p->ip)->i_mode),
+ };
+
+ error = xrep_dir_stash_removename(rd, &name,
p->ip->i_ino);
+ }
mutex_unlock(&rd->pscan.lock);
if (error)
goto out_abort;
diff --git a/fs/xfs/scrub/health.c b/fs/xfs/scrub/health.c
index 3c0f25098b69..13589d9e987f 100644
--- a/fs/xfs/scrub/health.c
+++ b/fs/xfs/scrub/health.c
@@ -202,9 +202,9 @@ xchk_update_health(
* there's no sick flag defined for it, so we branch here ahead of the
* mask check.
*/
- if (sc->sm->sm_type == XFS_SCRUB_TYPE_HEALTHY &&
- !(sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT)) {
- xchk_mark_all_healthy(sc->mp);
+ if (sc->sm->sm_type == XFS_SCRUB_TYPE_HEALTHY) {
+ if (!(sc->sm->sm_flags & XFS_SCRUB_OFLAG_CORRUPT))
+ xchk_mark_all_healthy(sc->mp);
return;
}
diff --git a/fs/xfs/scrub/inode.c b/fs/xfs/scrub/inode.c
index 0d30d65f336a..9a7fb91996d2 100644
--- a/fs/xfs/scrub/inode.c
+++ b/fs/xfs/scrub/inode.c
@@ -607,7 +607,7 @@ xchk_dinode(
}
/* di_forkoff */
- if (XFS_DFORK_BOFF(dip) >= mp->m_sb.sb_inodesize)
+ if (dip->di_forkoff >= (XFS_LITINO(mp) >> 3))
xchk_ino_set_corrupt(sc, ino);
if (naextents != 0 && dip->di_forkoff == 0)
xchk_ino_set_corrupt(sc, ino);
diff --git a/fs/xfs/scrub/inode_repair.c b/fs/xfs/scrub/inode_repair.c
index fdb227dcfd21..9aa7f8d3ff89 100644
--- a/fs/xfs/scrub/inode_repair.c
+++ b/fs/xfs/scrub/inode_repair.c
@@ -1698,7 +1698,7 @@ xrep_inode_blockcounts(
&acount);
if (error)
return error;
- if (count >= sc->mp->m_sb.sb_dblocks)
+ if (acount >= sc->mp->m_sb.sb_dblocks)
return -EFSCORRUPTED;
error = xrep_ino_ensure_extent_count(sc, XFS_ATTR_FORK,
nextents);
diff --git a/fs/xfs/scrub/orphanage.c b/fs/xfs/scrub/orphanage.c
index 3c4f3b542c20..4b7a6691d249 100644
--- a/fs/xfs/scrub/orphanage.c
+++ b/fs/xfs/scrub/orphanage.c
@@ -193,12 +193,16 @@ xrep_orphanage_create(
/* Make sure the orphanage is owned by root. */
error = xrep_chown_orphanage(sc, XFS_I(orphanage_inode));
if (error)
- goto out_dput_orphanage;
+ goto out_rele_orphanage;
/* Stash the reference for later and bail out. */
sc->orphanage = XFS_I(orphanage_inode);
sc->orphanage_ilock_flags = 0;
+ orphanage_inode = NULL;
+out_rele_orphanage:
+ if (orphanage_inode)
+ xchk_irele(sc, XFS_I(orphanage_inode));
out_dput_orphanage:
dput(orphanage_dentry);
out_unlock_root:
diff --git a/fs/xfs/scrub/repair.c b/fs/xfs/scrub/repair.c
index 4d45d39e67f1..99d7f697fbbd 100644
--- a/fs/xfs/scrub/repair.c
+++ b/fs/xfs/scrub/repair.c
@@ -399,6 +399,7 @@ xrep_calc_rtgroup_resblks(
struct xfs_mount *mp = sc->mp;
struct xfs_scrub_metadata *sm = sc->sm;
uint64_t usedlen;
+ xfs_extlen_t refcbt_sz = 0;
xfs_extlen_t rmapbt_sz = 0;
if (!(sm->sm_flags & XFS_SCRUB_IFLAG_REPAIR))
@@ -411,13 +412,27 @@ xrep_calc_rtgroup_resblks(
usedlen = xfs_rtbxlen_to_blen(mp, xfs_rtgroup_extents(mp, sm->sm_agno));
ASSERT(usedlen <= XFS_MAX_RGBLOCKS);
+ if (xfs_has_reflink(mp))
+ refcbt_sz = xfs_rtrefcountbt_calc_size(mp, usedlen);
+
if (xfs_has_rmapbt(mp))
rmapbt_sz = xfs_rtrmapbt_calc_size(mp, usedlen);
+ /*
+ * Guess how many blocks we need to rebuild the rmapbt. For
+ * non-reflink filesystems we can't have more records than used blocks.
+ * However, with reflink it's possible to have more than one rmap
+ * record per rtgroup block. We don't know how many rmaps there could
+ * be in the rtgroup, so we start off with what we hope is an generous
+ * over-estimation.
+ */
+ if (refcbt_sz > 0 && rmapbt_sz > 0)
+ rmapbt_sz *= 2;
+
trace_xrep_calc_rtgroup_resblks_btsize(mp, sm->sm_agno, usedlen,
- rmapbt_sz);
+ rmapbt_sz, refcbt_sz);
- return rmapbt_sz;
+ return max(rmapbt_sz, refcbt_sz);
}
#endif /* CONFIG_XFS_RT */
diff --git a/fs/xfs/scrub/scrub.h b/fs/xfs/scrub/scrub.h
index 6d7d3523b71f..7ade1d005efe 100644
--- a/fs/xfs/scrub/scrub.h
+++ b/fs/xfs/scrub/scrub.h
@@ -40,7 +40,7 @@ static inline int xchk_maybe_relax(struct xchk_relax *widget)
return 0;
widget->resched_nr = 0;
- if (unlikely(widget->next_resched <= jiffies)) {
+ if (unlikely(time_after_eq(jiffies, widget->next_resched))) {
cond_resched();
widget->next_resched = XCHK_RELAX_NEXT;
}
diff --git a/fs/xfs/scrub/trace.h b/fs/xfs/scrub/trace.h
index ed88557140c7..d36109593745 100644
--- a/fs/xfs/scrub/trace.h
+++ b/fs/xfs/scrub/trace.h
@@ -2376,25 +2376,29 @@ TRACE_EVENT(xrep_calc_ag_resblks_btsize,
#ifdef CONFIG_XFS_RT
TRACE_EVENT(xrep_calc_rtgroup_resblks_btsize,
TP_PROTO(struct xfs_mount *mp, xfs_rgnumber_t rgno,
- xfs_rgblock_t usedlen, xfs_rgblock_t rmapbt_sz),
- TP_ARGS(mp, rgno, usedlen, rmapbt_sz),
+ xfs_rgblock_t usedlen, xfs_rgblock_t rmapbt_sz,
+ xfs_rgblock_t refcbt_sz),
+ TP_ARGS(mp, rgno, usedlen, rmapbt_sz, refcbt_sz),
TP_STRUCT__entry(
__field(dev_t, dev)
__field(xfs_rgnumber_t, rgno)
__field(xfs_rgblock_t, usedlen)
__field(xfs_rgblock_t, rmapbt_sz)
+ __field(xfs_rgblock_t, refcbt_sz)
),
TP_fast_assign(
__entry->dev = mp->m_super->s_dev;
__entry->rgno = rgno;
__entry->usedlen = usedlen;
__entry->rmapbt_sz = rmapbt_sz;
+ __entry->refcbt_sz = refcbt_sz;
),
- TP_printk("dev %d:%d rgno 0x%x usedlen %u rmapbt %u",
+ TP_printk("dev %d:%d rgno 0x%x usedlen %u rmapbt %u refcountbt %u",
MAJOR(__entry->dev), MINOR(__entry->dev),
__entry->rgno,
__entry->usedlen,
- __entry->rmapbt_sz)
+ __entry->rmapbt_sz,
+ __entry->refcbt_sz)
);
#endif /* CONFIG_XFS_RT */
diff --git a/fs/xfs/xfs_dquot.c b/fs/xfs/xfs_dquot.c
index 8c89f701537f..cce1f86098bd 100644
--- a/fs/xfs/xfs_dquot.c
+++ b/fs/xfs/xfs_dquot.c
@@ -139,10 +139,14 @@ xfs_qm_adjust_dqlimits(
dq->q_ino.softlimit = defq->ino.soft;
if (!dq->q_ino.hardlimit)
dq->q_ino.hardlimit = defq->ino.hard;
- if (!dq->q_rtb.softlimit)
+ if (!dq->q_rtb.softlimit) {
dq->q_rtb.softlimit = defq->rtb.soft;
- if (!dq->q_rtb.hardlimit)
+ prealloc = 1;
+ }
+ if (!dq->q_rtb.hardlimit) {
dq->q_rtb.hardlimit = defq->rtb.hard;
+ prealloc = 1;
+ }
if (prealloc)
xfs_dquot_set_prealloc_limits(dq);
diff --git a/fs/xfs/xfs_exchrange.c b/fs/xfs/xfs_exchrange.c
index e0f092ff77e9..04da020f3577 100644
--- a/fs/xfs/xfs_exchrange.c
+++ b/fs/xfs/xfs_exchrange.c
@@ -633,6 +633,9 @@ xfs_exchrange_prep(
if (error)
return error;
+ if (fxr->flags & XFS_EXCHANGE_RANGE_DRY_RUN)
+ return 0;
+
trace_xfs_exchrange_flush(fxr, ip1, ip2);
/* Flush the relevant ranges of both files. */
@@ -709,9 +712,11 @@ xfs_exchrange_contents(
* other file write would do. This may involve turning on support for
* logged xattrs if either file has security capabilities.
*/
- error = xfs_exchange_range_finish(fxr);
- if (error)
- goto out_unlock;
+ if (!(fxr->flags & XFS_EXCHANGE_RANGE_DRY_RUN)) {
+ error = xfs_exchange_range_finish(fxr);
+ if (error)
+ goto out_unlock;
+ }
out_unlock:
xfs_iunlock2_io_mmap(ip1, ip2);
@@ -902,7 +907,7 @@ xfs_ioc_commit_range(
if (copy_from_user(&args, argp, sizeof(args)))
return -EFAULT;
- if (args.flags & ~XFS_EXCHANGE_RANGE_ALL_FLAGS)
+ if (args.pad || (args.flags & ~XFS_EXCHANGE_RANGE_ALL_FLAGS))
return -EINVAL;
if (kern_f->magic != XCR_FRESH_MAGIC)
return -EBUSY;
diff --git a/fs/xfs/xfs_icache.c b/fs/xfs/xfs_icache.c
index e44040206851..eab8382bfac4 100644
--- a/fs/xfs/xfs_icache.c
+++ b/fs/xfs/xfs_icache.c
@@ -1650,7 +1650,7 @@ xfs_blockgc_free_dquots(
do_work = true;
}
- if (XFS_IS_UQUOTA_ENFORCED(mp) && gdqp && xfs_dquot_lowsp(gdqp)) {
+ if (XFS_IS_GQUOTA_ENFORCED(mp) && gdqp && xfs_dquot_lowsp(gdqp)) {
icw.icw_gid = make_kgid(mp->m_super->s_user_ns, gdqp->q_id);
icw.icw_flags |= XFS_ICWALK_FLAG_GID;
do_work = true;
diff --git a/fs/xfs/xfs_log_recover.c b/fs/xfs/xfs_log_recover.c
index 33cf2a31780c..dd4020eab9ab 100644
--- a/fs/xfs/xfs_log_recover.c
+++ b/fs/xfs/xfs_log_recover.c
@@ -2736,12 +2736,13 @@ xlog_recover_iunlink_bucket(
{
struct xfs_mount *mp = pag_mount(pag);
struct xfs_inode *prev_ip = NULL;
- struct xfs_inode *ip;
xfs_agino_t prev_agino, agino;
int error = 0;
agino = be32_to_cpu(agi->agi_unlinked[bucket]);
while (agino != NULLAGINO) {
+ struct xfs_inode *ip;
+
error = xfs_iget(mp, NULL, xfs_agino_to_ino(pag, agino), 0, 0,
&ip);
if (error)
@@ -2750,11 +2751,11 @@ xlog_recover_iunlink_bucket(
ASSERT(VFS_I(ip)->i_nlink == 0);
ASSERT(VFS_I(ip)->i_mode != 0);
xfs_iflags_clear(ip, XFS_IRECOVERY);
- agino = ip->i_next_unlinked;
if (prev_ip) {
ip->i_prev_unlinked = prev_agino;
xfs_irele(prev_ip);
+ prev_ip = NULL;
/*
* Ensure the inode is removed from the unlinked list
@@ -2766,18 +2767,20 @@ xlog_recover_iunlink_bucket(
* complete.
*/
error = xfs_inodegc_flush(mp);
- if (error)
- break;
+ if (error) {
+ xfs_irele(ip);
+ return error;
+ }
}
prev_agino = agino;
+ agino = ip->i_next_unlinked;
prev_ip = ip;
}
if (prev_ip) {
int error2;
- ip->i_prev_unlinked = prev_agino;
xfs_irele(prev_ip);
error2 = xfs_inodegc_flush(mp);
diff --git a/fs/xfs/xfs_refcount_item.c b/fs/xfs/xfs_refcount_item.c
index 3728234699a2..830160ba418d 100644
--- a/fs/xfs/xfs_refcount_item.c
+++ b/fs/xfs/xfs_refcount_item.c
@@ -510,6 +510,7 @@ xfs_refcount_recover_work(
struct xfs_cui_log_item *cuip = CUI_ITEM(lip);
struct xfs_trans *tp;
struct xfs_mount *mp = lip->li_log->l_mp;
+ unsigned int dblocks;
bool isrt = xfs_cui_item_isrt(lip);
int i;
int error = 0;
@@ -545,8 +546,11 @@ xfs_refcount_recover_work(
* full btree split on either end of the refcount range.
*/
resv = xlog_recover_resv(&M_RES(mp)->tr_itruncate);
- error = xfs_trans_alloc(mp, &resv, mp->m_refc_maxlevels * 2, 0,
- XFS_TRANS_RESERVE, &tp);
+ if (isrt)
+ dblocks = mp->m_rtrefc_maxlevels * 2;
+ else
+ dblocks = mp->m_refc_maxlevels * 2;
+ error = xfs_trans_alloc(mp, &resv, dblocks, 0, XFS_TRANS_RESERVE, &tp);
if (error)
return error;
diff --git a/fs/xfs/xfs_rmap_item.c b/fs/xfs/xfs_rmap_item.c
index 15f0903f6fd4..973ef02031f8 100644
--- a/fs/xfs/xfs_rmap_item.c
+++ b/fs/xfs/xfs_rmap_item.c
@@ -575,6 +575,7 @@ xfs_rmap_recover_work(
struct xfs_rui_log_item *ruip = RUI_ITEM(lip);
struct xfs_trans *tp;
struct xfs_mount *mp = lip->li_log->l_mp;
+ unsigned int dblocks;
bool isrt = xfs_rui_item_isrt(lip);
int i;
int error = 0;
@@ -598,8 +599,11 @@ xfs_rmap_recover_work(
}
resv = xlog_recover_resv(&M_RES(mp)->tr_itruncate);
- error = xfs_trans_alloc(mp, &resv, mp->m_rmap_maxlevels, 0,
- XFS_TRANS_RESERVE, &tp);
+ if (isrt)
+ dblocks = mp->m_rtrmap_maxlevels;
+ else
+ dblocks = mp->m_rmap_maxlevels;
+ error = xfs_trans_alloc(mp, &resv, dblocks, 0, XFS_TRANS_RESERVE, &tp);
if (error)
return error;
diff --git a/include/drm/drm_client.h b/include/drm/drm_client.h
index 8e2e51f98430..bf9783c97834 100644
--- a/include/drm/drm_client.h
+++ b/include/drm/drm_client.h
@@ -47,12 +47,14 @@ struct drm_client_funcs {
*
* Note that the core does not guarantee exclusion against concurrent
* drm_open(). Clients need to ensure this themselves, for example by
- * using drm_master_internal_acquire() and
- * drm_master_internal_release().
+ * using drm_master_internal_acquire() and drm_master_internal_release().
+ *
+ * If the caller passes force, the client should ignore any present DRM
+ * master and restore the display anyway.
*
* This callback is optional.
*/
- int (*restore)(struct drm_client_dev *client);
+ int (*restore)(struct drm_client_dev *client, bool force);
/**
* @hotplug:
@@ -70,13 +72,8 @@ struct drm_client_funcs {
* Called when suspending the device.
*
* This callback is optional.
- *
- * FIXME: Some callers hold the console lock when invoking this
- * function. This interferes with fbdev emulation, which
- * also tries to acquire the lock. Push the console lock
- * into the callback and remove 'holds_console_lock'.
*/
- int (*suspend)(struct drm_client_dev *client, bool holds_console_lock);
+ int (*suspend)(struct drm_client_dev *client);
/**
* @resume:
@@ -84,13 +81,8 @@ struct drm_client_funcs {
* Called when resuming the device from suspend.
*
* This callback is optional.
- *
- * FIXME: Some callers hold the console lock when invoking this
- * function. This interferes with fbdev emulation, which
- * also tries to acquire the lock. Push the console lock
- * into the callback and remove 'holds_console_lock'.
*/
- int (*resume)(struct drm_client_dev *client, bool holds_console_lock);
+ int (*resume)(struct drm_client_dev *client);
};
/**
diff --git a/include/drm/drm_client_event.h b/include/drm/drm_client_event.h
index 1d544d3a3228..79369c755bc9 100644
--- a/include/drm/drm_client_event.h
+++ b/include/drm/drm_client_event.h
@@ -10,19 +10,19 @@ struct drm_device;
#if defined(CONFIG_DRM_CLIENT)
void drm_client_dev_unregister(struct drm_device *dev);
void drm_client_dev_hotplug(struct drm_device *dev);
-void drm_client_dev_restore(struct drm_device *dev);
-void drm_client_dev_suspend(struct drm_device *dev, bool holds_console_lock);
-void drm_client_dev_resume(struct drm_device *dev, bool holds_console_lock);
+void drm_client_dev_restore(struct drm_device *dev, bool force);
+void drm_client_dev_suspend(struct drm_device *dev);
+void drm_client_dev_resume(struct drm_device *dev);
#else
static inline void drm_client_dev_unregister(struct drm_device *dev)
{ }
static inline void drm_client_dev_hotplug(struct drm_device *dev)
{ }
-static inline void drm_client_dev_restore(struct drm_device *dev)
+static inline void drm_client_dev_restore(struct drm_device *dev, bool force)
{ }
-static inline void drm_client_dev_suspend(struct drm_device *dev, bool holds_console_lock)
+static inline void drm_client_dev_suspend(struct drm_device *dev)
{ }
-static inline void drm_client_dev_resume(struct drm_device *dev, bool holds_console_lock)
+static inline void drm_client_dev_resume(struct drm_device *dev)
{ }
#endif
diff --git a/include/drm/drm_fb_helper.h b/include/drm/drm_fb_helper.h
index 2faf8aaf98dc..dd9a18f8de5a 100644
--- a/include/drm/drm_fb_helper.h
+++ b/include/drm/drm_fb_helper.h
@@ -254,7 +254,8 @@ int drm_fb_helper_set_par(struct fb_info *info);
int drm_fb_helper_check_var(struct fb_var_screeninfo *var,
struct fb_info *info);
-int drm_fb_helper_restore_fbdev_mode_unlocked(struct drm_fb_helper *fb_helper);
+int drm_fb_helper_restore_fbdev_mode_unlocked(struct drm_fb_helper *fb_helper,
+ bool force);
void drm_fb_helper_unregister_info(struct drm_fb_helper *fb_helper);
void drm_fb_helper_fill_info(struct fb_info *info,
@@ -281,7 +282,6 @@ int drm_fb_helper_hotplug_event(struct drm_fb_helper *fb_helper);
int drm_fb_helper_initial_config(struct drm_fb_helper *fb_helper);
int drm_fb_helper_debug_enter(struct fb_info *info);
int drm_fb_helper_debug_leave(struct fb_info *info);
-void drm_fb_helper_lastclose(struct drm_device *dev);
#else
static inline void drm_fb_helper_prepare(struct drm_device *dev,
struct drm_fb_helper *helper,
@@ -397,10 +397,6 @@ static inline int drm_fb_helper_debug_leave(struct fb_info *info)
{
return 0;
}
-
-static inline void drm_fb_helper_lastclose(struct drm_device *dev)
-{
-}
#endif
#endif
diff --git a/include/linux/bpf_mem_alloc.h b/include/linux/bpf_mem_alloc.h
index e45162ef59bb..4ce0d27f8ea2 100644
--- a/include/linux/bpf_mem_alloc.h
+++ b/include/linux/bpf_mem_alloc.h
@@ -14,6 +14,8 @@ struct bpf_mem_alloc {
struct obj_cgroup *objcg;
bool percpu;
struct work_struct work;
+ void (*dtor_ctx_free)(void *ctx);
+ void *dtor_ctx;
};
/* 'size != 0' is for bpf_mem_alloc which manages fixed-size objects.
@@ -32,6 +34,10 @@ int bpf_mem_alloc_percpu_init(struct bpf_mem_alloc *ma, struct obj_cgroup *objcg
/* The percpu allocation with a specific unit size. */
int bpf_mem_alloc_percpu_unit_init(struct bpf_mem_alloc *ma, int size);
void bpf_mem_alloc_destroy(struct bpf_mem_alloc *ma);
+void bpf_mem_alloc_set_dtor(struct bpf_mem_alloc *ma,
+ void (*dtor)(void *obj, void *ctx),
+ void (*dtor_ctx_free)(void *ctx),
+ void *ctx);
/* Check the allocation size for kmalloc equivalent allocator */
int bpf_mem_alloc_check_size(bool percpu, size_t size);
diff --git a/include/linux/if_vlan.h b/include/linux/if_vlan.h
index 15e01935d3fa..89472ba4425f 100644
--- a/include/linux/if_vlan.h
+++ b/include/linux/if_vlan.h
@@ -357,6 +357,9 @@ static inline int __vlan_insert_inner_tag(struct sk_buff *skb,
{
struct vlan_ethhdr *veth;
+ if (unlikely(!pskb_may_pull(skb, mac_len)))
+ return -EINVAL;
+
if (skb_cow_head(skb, VLAN_HLEN) < 0)
return -ENOMEM;
diff --git a/include/linux/rculist.h b/include/linux/rculist.h
index 2abba7552605..e3bc44225692 100644
--- a/include/linux/rculist.h
+++ b/include/linux/rculist.h
@@ -261,6 +261,35 @@ static inline void list_replace_rcu(struct list_head *old,
old->prev = LIST_POISON2;
}
+static inline void __list_splice_rcu(struct list_head *list,
+ struct list_head *prev,
+ struct list_head *next)
+{
+ struct list_head *first = list->next;
+ struct list_head *last = list->prev;
+
+ last->next = next;
+ first->prev = prev;
+ next->prev = last;
+ rcu_assign_pointer(list_next_rcu(prev), first);
+}
+
+/**
+ * list_splice_rcu - splice a non-RCU list into an RCU-protected list,
+ * designed for stacks.
+ * @list: the non RCU-protected list to splice
+ * @head: the place in the existing RCU-protected list to splice
+ *
+ * The list pointed to by @head can be RCU-read traversed concurrently with
+ * this function.
+ */
+static inline void list_splice_rcu(struct list_head *list,
+ struct list_head *head)
+{
+ if (!list_empty(list))
+ __list_splice_rcu(list, head, head->next);
+}
+
/**
* __list_splice_init_rcu - join an RCU-protected list into an existing list.
* @list: the RCU-protected list to splice
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index c9abf7e3accf..37fbdf08a8e0 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -1833,22 +1833,6 @@ static inline void skb_zcopy_set(struct sk_buff *skb, struct ubuf_info *uarg,
}
}
-static inline void skb_zcopy_set_nouarg(struct sk_buff *skb, void *val)
-{
- skb_shinfo(skb)->destructor_arg = (void *)((uintptr_t) val | 0x1UL);
- skb_shinfo(skb)->flags |= SKBFL_ZEROCOPY_FRAG;
-}
-
-static inline bool skb_zcopy_is_nouarg(struct sk_buff *skb)
-{
- return (uintptr_t) skb_shinfo(skb)->destructor_arg & 0x1UL;
-}
-
-static inline void *skb_zcopy_get_nouarg(struct sk_buff *skb)
-{
- return (void *)((uintptr_t) skb_shinfo(skb)->destructor_arg & ~0x1UL);
-}
-
static inline void net_zcopy_put(struct ubuf_info *uarg)
{
if (uarg)
@@ -1871,8 +1855,7 @@ static inline void skb_zcopy_clear(struct sk_buff *skb, bool zerocopy_success)
struct ubuf_info *uarg = skb_zcopy(skb);
if (uarg) {
- if (!skb_zcopy_is_nouarg(skb))
- uarg->ops->complete(skb, uarg, zerocopy_success);
+ uarg->ops->complete(skb, uarg, zerocopy_success);
skb_shinfo(skb)->flags &= ~SKBFL_ALL_ZEROCOPY;
}
@@ -4345,7 +4328,10 @@ skb_header_pointer_careful(const struct sk_buff *skb, int offset,
static inline void * __must_check
skb_pointer_if_linear(const struct sk_buff *skb, int offset, int len)
{
- if (likely(skb_headlen(skb) - offset >= len))
+ unsigned int uoffset = (unsigned int)offset;
+
+ if (likely(uoffset <= skb_headlen(skb) &&
+ (unsigned int)len <= skb_headlen(skb) - uoffset))
return skb->data + offset;
return NULL;
}
diff --git a/include/net/gue.h b/include/net/gue.h
index caefd6da8693..d377155fd0b3 100644
--- a/include/net/gue.h
+++ b/include/net/gue.h
@@ -84,8 +84,9 @@ static inline size_t guehdr_priv_flags_len(__be32 flags)
}
/* Validate standard and private flags. Returns non-zero (meaning invalid)
- * if there is an unknown standard or private flags, or the options length for
- * the flags exceeds the options length specific in hlen of the GUE header.
+ * if there is an unknown standard or private flags, if the options length for
+ * the flags exceeds the options length specified in hlen of the GUE header, or
+ * if a private option contains invalid data.
*/
static inline int validate_gue_flags(struct guehdr *guehdr, size_t optlen)
{
@@ -103,8 +104,8 @@ static inline int validate_gue_flags(struct guehdr *guehdr, size_t optlen)
/* Private flags are last four bytes accounted in
* guehdr_flags_len
*/
- __be32 pflags = *(__be32 *)((void *)&guehdr[1] +
- len - GUE_LEN_PRIV);
+ void *data = (void *)&guehdr[1] + len;
+ __be32 pflags = *(__be32 *)(data - GUE_LEN_PRIV);
if (pflags & ~GUE_PFLAGS_ALL)
return 1;
@@ -112,6 +113,16 @@ static inline int validate_gue_flags(struct guehdr *guehdr, size_t optlen)
len += guehdr_priv_flags_len(pflags);
if (len > optlen)
return 1;
+
+ if (pflags & GUE_PFLAG_REMCSUM) {
+ __be16 *pd = data;
+
+ /* The field offset pd[1] must not be less
+ * than the start pd[0].
+ */
+ if (ntohs(pd[1]) < ntohs(pd[0]))
+ return 1;
+ }
}
return 0;
diff --git a/include/net/ip6_route.h b/include/net/ip6_route.h
index 2e8ca123d146..4137bdf05baa 100644
--- a/include/net/ip6_route.h
+++ b/include/net/ip6_route.h
@@ -94,12 +94,12 @@ static inline struct dst_entry *ip6_route_output(struct net *net,
}
/* Only conditionally release dst if flags indicates
- * !RT6_LOOKUP_F_DST_NOREF or dst is in uncached_list.
+ * !RT6_LOOKUP_F_DST_NOREF or dst is uncached.
*/
static inline void ip6_rt_put_flags(struct rt6_info *rt, int flags)
{
if (!(flags & RT6_LOOKUP_F_DST_NOREF) ||
- !list_empty(&rt->dst.rt_uncached))
+ rt->dst.rt_uncached_list)
ip6_rt_put(rt);
}
diff --git a/include/net/netfilter/nf_conntrack.h b/include/net/netfilter/nf_conntrack.h
index aa0a7c82199e..7e26611eb3c4 100644
--- a/include/net/netfilter/nf_conntrack.h
+++ b/include/net/netfilter/nf_conntrack.h
@@ -184,6 +184,11 @@ static inline void nf_ct_put(struct nf_conn *ct)
nf_ct_destroy(&ct->ct_general);
}
+static inline bool nf_ct_shared(const struct nf_conn *ct)
+{
+ return refcount_read(&ct->ct_general.use) > 1;
+}
+
/* load module; enable/disable conntrack in this namespace */
int nf_ct_netns_get(struct net *net, u8 nfproto);
void nf_ct_netns_put(struct net *net, u8 nfproto);
diff --git a/include/net/nfc/hci.h b/include/net/nfc/hci.h
index 756c11084f65..86ed63e5d533 100644
--- a/include/net/nfc/hci.h
+++ b/include/net/nfc/hci.h
@@ -144,7 +144,7 @@ struct nfc_hci_dev {
data_exchange_cb_t async_cb;
void *async_cb_context;
- u8 *gb;
+ u8 gb[NFC_MAX_GT_LEN];
size_t gb_len;
unsigned long quirks;
diff --git a/include/net/nfc/nfc.h b/include/net/nfc/nfc.h
index c54df042db6b..bcafab5c53e5 100644
--- a/include/net/nfc/nfc.h
+++ b/include/net/nfc/nfc.h
@@ -273,7 +273,8 @@ struct sk_buff *nfc_alloc_recv_skb(unsigned int size, gfp_t gfp);
int nfc_set_remote_general_bytes(struct nfc_dev *dev,
const u8 *gt, u8 gt_len);
-u8 *nfc_get_local_general_bytes(struct nfc_dev *dev, size_t *gb_len);
+u8 *nfc_get_local_general_bytes(struct nfc_dev *dev, u8 *out_gb,
+ size_t gb_max_len, size_t *gb_len);
int nfc_fw_download_done(struct nfc_dev *dev, const char *firmware_name,
u32 result);
diff --git a/include/net/rose.h b/include/net/rose.h
index 95d1f9c582dc..0d957b8dd0ba 100644
--- a/include/net/rose.h
+++ b/include/net/rose.h
@@ -161,17 +161,25 @@ static inline void rose_neigh_put(struct rose_neigh *rose_neigh)
{
if (refcount_dec_and_test(&rose_neigh->use)) {
/* We are dropping the last reference, so we are about to free the
- * neighbour. Its timers may still be armed -- t0timer in particular
- * re-arms itself in rose_t0timer_expiry(). rose_remove_neigh()
- * cancels them before its own put, but callers that drop the final
- * reference without first calling rose_remove_neigh() (the socket
- * heartbeat reaping path) would otherwise kfree() a neighbour with a
- * live timer -> use-after-free. timer_delete_sync() (not the async
- * variant) is required: it waits out a concurrently running handler
- * and loops until the self-rearming timer stays stopped.
+ * neighbour. t0timer is self-rearming: rose_t0timer_expiry() calls
+ * rose_start_t0timer() at its own tail, so a plain timer_delete_sync()
+ * is not enough here. It only guarantees that the callback is not
+ * running *at the moment it returns* -- it does nothing to stop the
+ * very invocation we just waited out from re-arming the timer on its
+ * way out, which races the kfree() below (syzbot: use-after-free read
+ * in ax25_find_cb(), reached via rose_t0timer_expiry() ->
+ * rose_transmit_restart_request() -> rose_send_frame() ->
+ * ax25_send_frame(), dereferencing the freed neigh->digipeat).
+ * timer_shutdown_sync() closes that hole: once it returns, any
+ * further add_timer()/mod_timer() on this timer is silently ignored,
+ * so a self-rearm racing the free can no longer bring the timer back
+ * to life on freed memory. ftimer's handler is a no-op and never
+ * re-arms, but it is shut down the same way here for consistency --
+ * this is final teardown, neither timer has any business firing
+ * again.
*/
- timer_delete_sync(&rose_neigh->ftimer);
- timer_delete_sync(&rose_neigh->t0timer);
+ timer_shutdown_sync(&rose_neigh->ftimer);
+ timer_shutdown_sync(&rose_neigh->t0timer);
if (rose_neigh->ax25)
ax25_cb_put(rose_neigh->ax25);
kfree(rose_neigh->digipeat);
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 12a9672f56eb..e40bad560648 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -1151,9 +1151,9 @@ static inline bool tcp_skb_can_collapse_to(const struct sk_buff *skb)
static inline bool tcp_skb_can_collapse(const struct sk_buff *to,
const struct sk_buff *from)
{
- /* skb_cmp_decrypted() not needed, use tcp_write_collapse_fence() */
return likely(tcp_skb_can_collapse_to(to) &&
mptcp_skb_can_collapse(to, from) &&
+ !skb_cmp_decrypted(to, from) &&
skb_pure_zcopy_same(to, from) &&
skb_frags_readable(to) == skb_frags_readable(from));
}
@@ -2275,7 +2275,7 @@ static inline void tcp_rtx_queue_unlink_and_free(struct sk_buff *skb, struct soc
static inline void tcp_write_collapse_fence(struct sock *sk)
{
- struct sk_buff *skb = tcp_write_queue_tail(sk);
+ struct sk_buff *skb = tcp_write_queue_tail(sk) ?: tcp_rtx_queue_tail(sk);
if (skb)
TCP_SKB_CB(skb)->eor = 1;
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index f92e1c16c356..3509bbe39efa 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -4104,13 +4104,10 @@ int btf_check_and_fixup_fields(const struct btf *btf, struct btf_record *rec)
{
int i;
- /* There are three types that signify ownership of some other type:
- * kptr_ref, bpf_list_head, bpf_rb_root.
- * kptr_ref only supports storing kernel types, which can't store
- * references to program allocated local types.
- *
- * Hence we only need to ensure that bpf_{list_head,rb_root} ownership
- * does not form cycles.
+ /*
+ * Check fields which require the complete BTF and initialize runtime
+ * metadata. Ownership relationships are validated after every record has
+ * been fixed up.
*/
if (IS_ERR_OR_NULL(rec) || !(rec->field_mask & (BPF_GRAPH_ROOT | BPF_UPTR)))
return 0;
@@ -4141,51 +4138,88 @@ int btf_check_and_fixup_fields(const struct btf *btf, struct btf_record *rec)
if (!meta)
return -EFAULT;
rec->fields[i].graph_root.value_rec = meta->record;
+ }
+ return 0;
+}
- /* We need to set value_rec for all root types, but no need
- * to check ownership cycle for a type unless it's also a
- * node type.
- */
- if (!(rec->field_mask & BPF_GRAPH_NODE))
+static int btf_owned_type_idx(const struct btf *btf, struct btf_struct_metas *tab,
+ const struct btf_field *field)
+{
+ struct btf_struct_meta *meta;
+ u32 btf_id;
+
+ if (field->type & BPF_GRAPH_ROOT) {
+ btf_id = field->graph_root.value_btf_id;
+ } else if (field->type == BPF_KPTR_REF || field->type == BPF_KPTR_PERCPU) {
+ if (btf_is_kernel(field->kptr.btf))
+ return -ENOENT;
+ btf_id = field->kptr.btf_id;
+ } else {
+ return -ENOENT;
+ }
+
+ meta = btf_find_struct_meta(btf, btf_id);
+ if (!meta)
+ return field->type & BPF_GRAPH_ROOT ? -EFAULT : -ENOENT;
+ return meta - tab->types;
+}
+
+/*
+ * Each ownership edge adds kernel frames through bpf_obj_free_fields() and
+ * __bpf_obj_drop_impl(). Keep the bound deliberately small because object
+ * destruction can itself run below a BPF call chain. A final pointee without
+ * special fields is not present in the struct metadata table and adds only a
+ * non-recursing drop.
+ */
+#define BTF_MAX_OWNERSHIP_DEPTH 8
+
+static int btf_ownership_depth(const struct btf *btf,
+ struct btf_struct_metas *tab, u8 *depth,
+ int idx, int depth_left)
+{
+ const struct btf_record *rec = tab->types[idx].record;
+ int i, ret, max_depth = 0;
+
+ if (!depth_left)
+ return -ELOOP;
+ if (depth[idx])
+ goto done;
+
+ for (i = 0; i < rec->cnt; i++) {
+ ret = btf_owned_type_idx(btf, tab, &rec->fields[i]);
+ if (ret == -ENOENT)
continue;
+ if (ret < 0)
+ return ret;
+ ret = btf_ownership_depth(btf, tab, depth, ret, depth_left - 1);
+ if (ret < 0)
+ return ret;
+ max_depth = max(max_depth, ret);
+ }
+ depth[idx] = max_depth + 1;
+done:
+ return depth[idx] > depth_left ? -ELOOP : depth[idx];
+}
- /* We need to ensure ownership acyclicity among all types. The
- * proper way to do it would be to topologically sort all BTF
- * IDs based on the ownership edges, since there can be multiple
- * bpf_{list_head,rb_node} in a type. Instead, we use the
- * following resaoning:
- *
- * - A type can only be owned by another type in user BTF if it
- * has a bpf_{list,rb}_node. Let's call these node types.
- * - A type can only _own_ another type in user BTF if it has a
- * bpf_{list_head,rb_root}. Let's call these root types.
- *
- * We ensure that if a type is both a root and node, its
- * element types cannot be root types.
- *
- * To ensure acyclicity:
- *
- * When A is an root type but not a node, its ownership
- * chain can be:
- * A -> B -> C
- * Where:
- * - A is an root, e.g. has bpf_rb_root.
- * - B is both a root and node, e.g. has bpf_rb_node and
- * bpf_list_head.
- * - C is only an root, e.g. has bpf_list_node
- *
- * When A is both a root and node, some other type already
- * owns it in the BTF domain, hence it can not own
- * another root type through any of the ownership edges.
- * A -> B
- * Where:
- * - A is both an root and node.
- * - B is only an node.
- */
- if (meta->record->field_mask & BPF_GRAPH_ROOT)
- return -ELOOP;
+static int btf_check_ownership_depth(const struct btf *btf,
+ struct btf_struct_metas *tab)
+{
+ u8 *depth;
+ int i, ret = 0;
+
+ depth = kvcalloc(tab->cnt, sizeof(*depth), GFP_KERNEL | __GFP_NOWARN);
+ if (!depth)
+ return -ENOMEM;
+
+ for (i = 0; i < tab->cnt; i++) {
+ ret = btf_ownership_depth(btf, tab, depth, i,
+ BTF_MAX_OWNERSHIP_DEPTH);
+ if (ret < 0)
+ break;
+ ret = 0;
}
- return 0;
+ kvfree(depth);
+ return ret;
}
static void __btf_struct_show(const struct btf *btf, const struct btf_type *t,
@@ -5864,6 +5898,10 @@ static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uat
if (err < 0)
goto errout_meta;
}
+
+ err = btf_check_ownership_depth(btf, struct_meta_tab);
+ if (err < 0)
+ goto errout_meta;
}
err = finalize_log(&env->log, uattr, uattr_size);
@@ -7004,7 +7042,7 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
if (btf_type_is_int(t))
return WALK_SCALAR;
- if (!btf_type_is_struct(t))
+ if (!btf_type_is_struct(t) || !t->size)
goto error;
off = (off - moff) % t->size;
diff --git a/kernel/bpf/crypto.c b/kernel/bpf/crypto.c
index 1d024fe7248a..3837d4dcf48d 100644
--- a/kernel/bpf/crypto.c
+++ b/kernel/bpf/crypto.c
@@ -149,8 +149,9 @@ bpf_crypto_ctx_create(const struct bpf_crypto_params *params, u32 params__sz,
const struct bpf_crypto_type *type;
struct bpf_crypto_ctx *ctx;
- if (!params || params->reserved[0] || params->reserved[1] ||
- params__sz != sizeof(struct bpf_crypto_params)) {
+ if (!params ||
+ params__sz != sizeof(struct bpf_crypto_params) ||
+ params->reserved[0] || params->reserved[1]) {
*err = -EINVAL;
return NULL;
}
diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index 7f8e029a6af8..23c9e54f92e1 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -128,6 +128,11 @@ struct htab_elem {
char key[] __aligned(8);
};
+struct htab_btf_record {
+ struct btf_record *record;
+ u32 key_size;
+};
+
static inline bool htab_is_prealloc(const struct bpf_htab *htab)
{
return !(htab->map.map_flags & BPF_F_NO_PREALLOC);
@@ -472,6 +477,87 @@ static int htab_map_alloc_check(union bpf_attr *attr)
return 0;
}
+static void htab_mem_dtor(void *obj, void *ctx)
+{
+ struct htab_btf_record *hrec = ctx;
+ struct htab_elem *elem = obj;
+ void *map_value;
+
+ if (IS_ERR_OR_NULL(hrec->record))
+ return;
+
+ map_value = htab_elem_value(elem, hrec->key_size);
+ bpf_obj_free_fields(hrec->record, map_value);
+}
+
+static void htab_pcpu_mem_dtor(void *obj, void *ctx)
+{
+ void __percpu *pptr = *(void __percpu **)obj;
+ struct htab_btf_record *hrec = ctx;
+ int cpu;
+
+ if (IS_ERR_OR_NULL(hrec->record))
+ return;
+
+ for_each_possible_cpu(cpu)
+ bpf_obj_free_fields(hrec->record, per_cpu_ptr(pptr, cpu));
+}
+
+static void htab_dtor_ctx_free(void *ctx)
+{
+ struct htab_btf_record *hrec = ctx;
+
+ btf_record_free(hrec->record);
+ kfree(ctx);
+}
+
+static int htab_set_dtor(const struct bpf_htab *htab, void (*dtor)(void *, void *))
+{
+ u32 key_size = htab->map.key_size;
+ const struct bpf_mem_alloc *ma;
+ struct htab_btf_record *hrec;
+ int err;
+
+ /* No need for dtors. */
+ if (IS_ERR_OR_NULL(htab->map.record))
+ return 0;
+
+ hrec = kzalloc(sizeof(*hrec), GFP_KERNEL);
+ if (!hrec)
+ return -ENOMEM;
+ hrec->key_size = key_size;
+ hrec->record = btf_record_dup(htab->map.record);
+ if (IS_ERR(hrec->record)) {
+ err = PTR_ERR(hrec->record);
+ kfree(hrec);
+ return err;
+ }
+ ma = htab_is_percpu(htab) ? &htab->pcpu_ma : &htab->ma;
+ /* Kinda sad, but cast away const-ness since we change ma->dtor. */
+ bpf_mem_alloc_set_dtor((struct bpf_mem_alloc *)ma, dtor, htab_dtor_ctx_free, hrec);
+ return 0;
+}
+
+static int htab_map_check_btf(const struct bpf_map *map, const struct btf *btf,
+ const struct btf_type *key_type, const struct btf_type *value_type)
+{
+ struct bpf_htab *htab = container_of(map, struct bpf_htab, map);
+
+ if (btf_type_is_void(key_type))
+ return -EINVAL;
+
+ if (htab_is_prealloc(htab))
+ return 0;
+ /*
+ * We must set the dtor using this callback, as map's BTF record is not
+ * populated in htab_map_alloc(), so it will always appear as NULL.
+ */
+ if (htab_is_percpu(htab))
+ return htab_set_dtor(htab, htab_pcpu_mem_dtor);
+ else
+ return htab_set_dtor(htab, htab_mem_dtor);
+}
+
static struct bpf_map *htab_map_alloc(union bpf_attr *attr)
{
bool percpu = (attr->map_type == BPF_MAP_TYPE_PERCPU_HASH ||
@@ -1675,6 +1761,12 @@ static int htab_lru_percpu_map_lookup_and_delete_elem(struct bpf_map *map,
flags);
}
+/*
+ * Max consecutive empty buckets to walk in one RCU +
+ * instrumentation-disabled section before rescheduling.
+ */
+#define HTAB_BATCH_EMPTY_RESCHED 64
+
static int
__htab_map_lookup_and_delete_batch(struct bpf_map *map,
const union bpf_attr *attr,
@@ -1696,6 +1788,7 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map,
unsigned long flags = 0;
bool locked = false;
struct htab_elem *l;
+ u32 empty_cnt = 0;
struct bucket *b;
int ret = 0;
@@ -1864,12 +1957,21 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map,
}
next_batch:
- /* If we are not copying data, we can go to next bucket and avoid
- * unlocking the rcu.
+ /*
+ * If we are not copying data, we can go to next bucket and avoid
+ * unlocking the rcu. Bound the walk though: after
+ * HTAB_BATCH_EMPTY_RESCHED consecutive empty buckets, fully exit
+ * the critical section (no locks are held here) and reschedule.
*/
if (!bucket_cnt && (batch + 1 < htab->n_buckets)) {
batch++;
- goto again_nocopy;
+ if (++empty_cnt < HTAB_BATCH_EMPTY_RESCHED)
+ goto again_nocopy;
+ empty_cnt = 0;
+ rcu_read_unlock();
+ bpf_enable_instrumentation();
+ cond_resched_tasks_rcu_qs();
+ goto again;
}
rcu_read_unlock();
@@ -1883,11 +1985,13 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map,
}
total += bucket_cnt;
+ empty_cnt = 0;
batch++;
if (batch >= htab->n_buckets) {
ret = -ENOENT;
goto after_loop;
}
+ cond_resched_tasks_rcu_qs();
goto again;
after_loop:
@@ -2266,6 +2370,7 @@ const struct bpf_map_ops htab_map_ops = {
.map_seq_show_elem = htab_map_seq_show_elem,
.map_set_for_each_callback_args = map_set_for_each_callback_args,
.map_for_each_callback = bpf_for_each_hash_elem,
+ .map_check_btf = htab_map_check_btf,
.map_mem_usage = htab_map_mem_usage,
BATCH_OPS(htab),
.map_btf_id = &htab_map_btf_ids[0],
@@ -2288,6 +2393,7 @@ const struct bpf_map_ops htab_lru_map_ops = {
.map_seq_show_elem = htab_map_seq_show_elem,
.map_set_for_each_callback_args = map_set_for_each_callback_args,
.map_for_each_callback = bpf_for_each_hash_elem,
+ .map_check_btf = htab_map_check_btf,
.map_mem_usage = htab_map_mem_usage,
BATCH_OPS(htab_lru),
.map_btf_id = &htab_map_btf_ids[0],
@@ -2461,6 +2567,7 @@ const struct bpf_map_ops htab_percpu_map_ops = {
.map_seq_show_elem = htab_percpu_map_seq_show_elem,
.map_set_for_each_callback_args = map_set_for_each_callback_args,
.map_for_each_callback = bpf_for_each_hash_elem,
+ .map_check_btf = htab_map_check_btf,
.map_mem_usage = htab_map_mem_usage,
BATCH_OPS(htab_percpu),
.map_btf_id = &htab_map_btf_ids[0],
@@ -2481,6 +2588,7 @@ const struct bpf_map_ops htab_lru_percpu_map_ops = {
.map_seq_show_elem = htab_percpu_map_seq_show_elem,
.map_set_for_each_callback_args = map_set_for_each_callback_args,
.map_for_each_callback = bpf_for_each_hash_elem,
+ .map_check_btf = htab_map_check_btf,
.map_mem_usage = htab_map_mem_usage,
BATCH_OPS(htab_lru_percpu),
.map_btf_id = &htab_map_btf_ids[0],
diff --git a/kernel/bpf/memalloc.c b/kernel/bpf/memalloc.c
index bd45dda9dc35..ed69c895cf86 100644
--- a/kernel/bpf/memalloc.c
+++ b/kernel/bpf/memalloc.c
@@ -102,6 +102,8 @@ struct bpf_mem_cache {
int percpu_size;
bool draining;
struct bpf_mem_cache *tgt;
+ void (*dtor)(void *obj, void *ctx);
+ void *dtor_ctx;
/* list of objects to be freed after RCU GP */
struct llist_head free_by_rcu;
@@ -117,6 +119,7 @@ struct bpf_mem_cache {
struct llist_head waiting_for_gp_ttrace;
struct rcu_head rcu_ttrace;
atomic_t call_rcu_ttrace_in_progress;
+ raw_spinlock_t lock;
};
struct bpf_mem_caches {
@@ -212,25 +215,24 @@ static void alloc_bulk(struct bpf_mem_cache *c, int cnt, int node, bool atomic)
gfp = __GFP_NOWARN | __GFP_ACCOUNT;
gfp |= atomic ? GFP_NOWAIT : GFP_KERNEL;
- for (i = 0; i < cnt; i++) {
- /*
- * For every 'c' llist_del_first(&c->free_by_rcu_ttrace); is
- * done only by one CPU == current CPU. Other CPUs might
- * llist_add() and llist_del_all() in parallel.
- */
- obj = llist_del_first(&c->free_by_rcu_ttrace);
- if (!obj)
- break;
- add_obj_to_free_list(c, obj);
- }
- if (i >= cnt)
- return;
+ /*
+ * c->lock serializes concurrent llist_del_first() against
+ * llist_del_all() in __free_rcu() and do_call_rcu_ttrace().
+ */
+ scoped_guard(raw_spinlock_irqsave, &c->lock) {
+ for (i = 0; i < cnt; i++) {
+ obj = llist_del_first(&c->free_by_rcu_ttrace);
+ if (!obj)
+ break;
+ add_obj_to_free_list(c, obj);
+ }
- for (; i < cnt; i++) {
- obj = llist_del_first(&c->waiting_for_gp_ttrace);
- if (!obj)
- break;
- add_obj_to_free_list(c, obj);
+ for (; i < cnt; i++) {
+ obj = llist_del_first(&c->waiting_for_gp_ttrace);
+ if (!obj)
+ break;
+ add_obj_to_free_list(c, obj);
+ }
}
if (i >= cnt)
return;
@@ -260,12 +262,14 @@ static void free_one(void *obj, bool percpu)
kfree(obj);
}
-static int free_all(struct llist_node *llnode, bool percpu)
+static int free_all(struct bpf_mem_cache *c, struct llist_node *llnode, bool percpu)
{
struct llist_node *pos, *t;
int cnt = 0;
llist_for_each_safe(pos, t, llnode) {
+ if (c->dtor)
+ c->dtor((void *)pos + LLIST_NODE_SZ, c->dtor_ctx);
free_one(pos, percpu);
cnt++;
}
@@ -275,8 +279,12 @@ static int free_all(struct llist_node *llnode, bool percpu)
static void __free_rcu(struct rcu_head *head)
{
struct bpf_mem_cache *c = container_of(head, struct bpf_mem_cache, rcu_ttrace);
+ struct llist_node *llnode;
+
+ scoped_guard(raw_spinlock_irqsave, &c->lock)
+ llnode = llist_del_all(&c->waiting_for_gp_ttrace);
- free_all(llist_del_all(&c->waiting_for_gp_ttrace), !!c->percpu_size);
+ free_all(c, llnode, !!c->percpu_size);
atomic_set(&c->call_rcu_ttrace_in_progress, 0);
}
@@ -307,8 +315,9 @@ static void do_call_rcu_ttrace(struct bpf_mem_cache *c)
if (atomic_xchg(&c->call_rcu_ttrace_in_progress, 1)) {
if (unlikely(READ_ONCE(c->draining))) {
- llnode = llist_del_all(&c->free_by_rcu_ttrace);
- free_all(llnode, !!c->percpu_size);
+ scoped_guard(raw_spinlock_irqsave, &c->lock)
+ llnode = llist_del_all(&c->free_by_rcu_ttrace);
+ free_all(c, llnode, !!c->percpu_size);
}
return;
}
@@ -417,7 +426,7 @@ static void check_free_by_rcu(struct bpf_mem_cache *c)
dec_active(c, &flags);
if (unlikely(READ_ONCE(c->draining))) {
- free_all(llist_del_all(&c->waiting_for_gp), !!c->percpu_size);
+ free_all(c, llist_del_all(&c->waiting_for_gp), !!c->percpu_size);
atomic_set(&c->call_rcu_in_progress, 0);
} else {
call_rcu_hurry(&c->rcu, __free_by_rcu);
@@ -542,6 +551,7 @@ int bpf_mem_alloc_init(struct bpf_mem_alloc *ma, int size, bool percpu)
c->objcg = objcg;
c->percpu_size = percpu_size;
c->tgt = c;
+ raw_spin_lock_init(&c->lock);
init_refill_work(c);
prefill_mem_cache(c, cpu);
}
@@ -564,7 +574,7 @@ int bpf_mem_alloc_init(struct bpf_mem_alloc *ma, int size, bool percpu)
c->objcg = objcg;
c->percpu_size = percpu_size;
c->tgt = c;
-
+ raw_spin_lock_init(&c->lock);
init_refill_work(c);
prefill_mem_cache(c, cpu);
}
@@ -616,7 +626,7 @@ int bpf_mem_alloc_percpu_unit_init(struct bpf_mem_alloc *ma, int size)
c->objcg = objcg;
c->percpu_size = percpu_size;
c->tgt = c;
-
+ raw_spin_lock_init(&c->lock);
init_refill_work(c);
prefill_mem_cache(c, cpu);
}
@@ -635,13 +645,13 @@ static void drain_mem_cache(struct bpf_mem_cache *c)
* Except for waiting_for_gp_ttrace list, there are no concurrent operations
* on these lists, so it is safe to use __llist_del_all().
*/
- free_all(llist_del_all(&c->free_by_rcu_ttrace), percpu);
- free_all(llist_del_all(&c->waiting_for_gp_ttrace), percpu);
- free_all(__llist_del_all(&c->free_llist), percpu);
- free_all(__llist_del_all(&c->free_llist_extra), percpu);
- free_all(__llist_del_all(&c->free_by_rcu), percpu);
- free_all(__llist_del_all(&c->free_llist_extra_rcu), percpu);
- free_all(llist_del_all(&c->waiting_for_gp), percpu);
+ free_all(c, llist_del_all(&c->free_by_rcu_ttrace), percpu);
+ free_all(c, llist_del_all(&c->waiting_for_gp_ttrace), percpu);
+ free_all(c, __llist_del_all(&c->free_llist), percpu);
+ free_all(c, __llist_del_all(&c->free_llist_extra), percpu);
+ free_all(c, __llist_del_all(&c->free_by_rcu), percpu);
+ free_all(c, __llist_del_all(&c->free_llist_extra_rcu), percpu);
+ free_all(c, llist_del_all(&c->waiting_for_gp), percpu);
}
static void check_mem_cache(struct bpf_mem_cache *c)
@@ -680,6 +690,9 @@ static void check_leaked_objs(struct bpf_mem_alloc *ma)
static void free_mem_alloc_no_barrier(struct bpf_mem_alloc *ma)
{
+ /* We can free dtor ctx only once all callbacks are done using it. */
+ if (ma->dtor_ctx_free)
+ ma->dtor_ctx_free(ma->dtor_ctx);
check_leaked_objs(ma);
free_percpu(ma->cache);
free_percpu(ma->caches);
@@ -1014,3 +1027,32 @@ int bpf_mem_alloc_check_size(bool percpu, size_t size)
return 0;
}
+
+void bpf_mem_alloc_set_dtor(struct bpf_mem_alloc *ma, void (*dtor)(void *obj, void *ctx),
+ void (*dtor_ctx_free)(void *ctx), void *ctx)
+{
+ struct bpf_mem_caches *cc;
+ struct bpf_mem_cache *c;
+ int cpu, i;
+
+ ma->dtor_ctx_free = dtor_ctx_free;
+ ma->dtor_ctx = ctx;
+
+ if (ma->cache) {
+ for_each_possible_cpu(cpu) {
+ c = per_cpu_ptr(ma->cache, cpu);
+ c->dtor = dtor;
+ c->dtor_ctx = ctx;
+ }
+ }
+ if (ma->caches) {
+ for_each_possible_cpu(cpu) {
+ cc = per_cpu_ptr(ma->caches, cpu);
+ for (i = 0; i < NUM_CACHES; i++) {
+ c = &cc->cache[i];
+ c->dtor = dtor;
+ c->dtor_ctx = ctx;
+ }
+ }
+ }
+}
diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c
index a9a036547c96..884f1eeaba2e 100644
--- a/kernel/bpf/offload.c
+++ b/kernel/bpf/offload.c
@@ -708,6 +708,8 @@ static bool __bpf_offload_dev_match(struct bpf_prog *prog,
return false;
if (offload->netdev == netdev)
return true;
+ if (!bpf_prog_is_offloaded(prog->aux))
+ return false;
ondev1 = bpf_offload_find_netdev(offload->netdev);
ondev2 = bpf_offload_find_netdev(netdev);
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index bab472152d64..dec993c90dc3 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -5941,7 +5941,10 @@ struct bpf_link *bpf_link_get_curr_or_next(u32 *id)
again:
link = idr_get_next(&link_idr, id);
if (link) {
- link = bpf_link_inc_not_zero(link);
+ if (link->id)
+ link = bpf_link_inc_not_zero(link);
+ else
+ link = ERR_PTR(-EAGAIN);
if (IS_ERR(link)) {
(*id)++;
goto again;
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 6ab677c00218..eebbede82070 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1449,6 +1449,26 @@ static bool isolated_cpus_can_update(struct cpumask *add_cpus,
return res;
}
+/*
+ * prstate_housekeeping_conflict - check for partition & housekeeping conflicts
+ * @prstate: partition root state to be checked
+ * @new_cpus: cpu mask
+ * Return: true if there is conflict, false otherwise
+ *
+ * CPUs outside of boot_hk_cpus, if defined, can only be used in an
+ * isolated partition.
+ */
+static bool prstate_housekeeping_conflict(int prstate, struct cpumask *new_cpus)
+{
+ if (!have_boot_isolcpus)
+ return false;
+
+ if ((prstate != PRS_ISOLATED) && !cpumask_subset(new_cpus, boot_hk_cpus))
+ return true;
+
+ return false;
+}
+
static void update_isolation_cpumasks(bool isolcpus_updated)
{
int ret;
@@ -1603,12 +1623,14 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,
* above it or remote partition root underneath it is not allowed.
*/
compute_excpus(cs, tmp->new_cpus);
- WARN_ON_ONCE(cpumask_intersects(tmp->new_cpus, subpartitions_cpus));
if (!cpumask_intersects(tmp->new_cpus, cpu_active_mask) ||
cpumask_subset(top_cpuset.effective_cpus, tmp->new_cpus))
return PERR_INVCPUS;
- if ((new_prs == PRS_ISOLATED) &&
- !isolated_cpus_can_update(tmp->new_cpus, NULL))
+ if (cpumask_intersects(tmp->new_cpus, subpartitions_cpus))
+ return PERR_NOCPUS;
+ if (((new_prs == PRS_ISOLATED) &&
+ !isolated_cpus_can_update(tmp->new_cpus, NULL)) ||
+ prstate_housekeeping_conflict(new_prs, tmp->new_cpus))
return PERR_HKEEPING;
spin_lock_irq(&callback_lock);
@@ -1751,26 +1773,6 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,
remote_partition_disable(cs, tmp);
}
-/*
- * prstate_housekeeping_conflict - check for partition & housekeeping conflicts
- * @prstate: partition root state to be checked
- * @new_cpus: cpu mask
- * Return: true if there is conflict, false otherwise
- *
- * CPUs outside of boot_hk_cpus, if defined, can only be used in an
- * isolated partition.
- */
-static bool prstate_housekeeping_conflict(int prstate, struct cpumask *new_cpus)
-{
- if (!have_boot_isolcpus)
- return false;
-
- if ((prstate != PRS_ISOLATED) && !cpumask_subset(new_cpus, boot_hk_cpus))
- return true;
-
- return false;
-}
-
/**
* update_parent_effective_cpumask - update effective_cpus mask of parent cpuset
* @cs: The cpuset that requests change in partition root state
@@ -3109,8 +3111,9 @@ static int update_prstate(struct cpuset *cs, int new_prs)
* A change in load balance state only, no change in cpumasks.
* Need to update isolated_cpus.
*/
- if ((new_prs == PRS_ISOLATED) &&
- !isolated_cpus_can_update(cs->effective_xcpus, NULL))
+ if (((new_prs == PRS_ISOLATED) &&
+ !isolated_cpus_can_update(cs->effective_xcpus, NULL)) ||
+ prstate_housekeeping_conflict(new_prs, cs->effective_xcpus))
err = PERR_HKEEPING;
else
isolcpus_updated = true;
diff --git a/kernel/cgroup/pids.c b/kernel/cgroup/pids.c
index 8f61114c36dd..5b64c8dde5eb 100644
--- a/kernel/cgroup/pids.c
+++ b/kernel/cgroup/pids.c
@@ -253,6 +253,11 @@ static void pids_event(struct pids_cgroup *pids_forking,
}
if (!cgroup_subsys_on_dfl(pids_cgrp_subsys) ||
cgrp_dfl_root.flags & CGRP_ROOT_PIDS_LOCAL_EVENTS) {
+ /*
+ * pids.events reports the local counter on legacy hierarchies
+ * and when pids_localevents is enabled.
+ */
+ cgroup_file_notify(&p->events_file);
cgroup_file_notify(&p->events_local_file);
return;
}
diff --git a/kernel/events/core.c b/kernel/events/core.c
index ad52e4bda8ff..e98e164919b7 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -3655,6 +3655,9 @@ static void perf_ctx_sched_task_cb(struct perf_event_context *ctx,
list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
cpc = this_cpc(pmu_ctx->pmu);
+ if (cpc->task_epc != pmu_ctx)
+ continue;
+
if (cpc->sched_cb_usage && pmu_ctx->pmu->sched_task)
pmu_ctx->pmu->sched_task(pmu_ctx, task, sched_in);
}
@@ -3805,7 +3808,7 @@ static void __perf_pmu_sched_task(struct perf_cpu_pmu_context *cpc,
perf_ctx_lock(cpuctx, cpuctx->task_ctx);
perf_pmu_disable(pmu);
- pmu->sched_task(cpc->task_epc, task, sched_in);
+ pmu->sched_task(&cpc->epc, task, sched_in);
perf_pmu_enable(pmu);
perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
@@ -3815,15 +3818,17 @@ static void perf_pmu_sched_task(struct task_struct *prev,
struct task_struct *next,
bool sched_in)
{
- struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
struct perf_cpu_pmu_context *cpc, *cpc2;
- /* cpuctx->task_ctx will be handled in perf_event_context_sched_in/out */
- if (prev == next || cpuctx->task_ctx)
+ if (prev == next)
return;
- list_for_each_entry_safe(cpc, cpc2, this_cpu_ptr(&sched_cb_list), sched_cb_entry)
+ list_for_each_entry_safe(cpc, cpc2, this_cpu_ptr(&sched_cb_list), sched_cb_entry) {
+ if (cpc->task_epc)
+ continue;
+
__perf_pmu_sched_task(cpc, sched_in ? next : prev, sched_in);
+ }
}
static void perf_event_switch(struct task_struct *task,
@@ -5311,6 +5316,8 @@ attach_task_ctx_data(struct task_struct *task, struct kmem_cache *ctx_cache,
}
if (refcount_inc_not_zero(&old->refcount)) {
+ if (global)
+ old->global = true;
free_perf_ctx_data(cd); /* unused */
return 0;
}
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index aad9813bd617..dbd871c6235f 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5613,7 +5613,7 @@ void sched_tick(void)
curr = rq->curr;
donor = rq->donor;
- psi_account_irqtime(rq, donor, NULL);
+ psi_account_irqtime(rq, curr, NULL);
update_rq_clock(rq);
hw_pressure = arch_scale_hw_pressure(cpu_of(rq));
diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
index 812e0c34bb80..059675018c71 100644
--- a/kernel/sched/ext.c
+++ b/kernel/sched/ext.c
@@ -1382,7 +1382,12 @@ static void enqueue_task_scx(struct rq *rq, struct task_struct *p, int core_enq_
int sticky_cpu = p->scx.sticky_cpu;
u64 enq_flags = core_enq_flags | rq->scx.extra_enq_flags;
- if (enq_flags & ENQUEUE_WAKEUP)
+ /*
+ * SCX_RQ_IN_WAKEUP promises a task_woken_scx() call once this enqueue
+ * returns. Only the core's wakeup path delivers one. The flags stashed
+ * for a remote activation may carry the wakeup bit without it.
+ */
+ if (core_enq_flags & ENQUEUE_WAKEUP)
rq->scx.flags |= SCX_RQ_IN_WAKEUP;
if (sticky_cpu >= 0)
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 7f469a3c9c17..6a4406acc840 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -3796,7 +3796,7 @@ static void check_flush_dependency(struct workqueue_struct *target_wq,
WARN_ONCE(current->flags & PF_MEMALLOC,
"workqueue: PF_MEMALLOC task %d(%s) is flushing !WQ_MEM_RECLAIM %s:%ps",
current->pid, current->comm, target_wq->name, target_func);
- WARN_ONCE(worker && ((worker->current_pwq->wq->flags &
+ WARN_ONCE(worker && worker->current_pwq && ((worker->current_pwq->wq->flags &
(WQ_MEM_RECLAIM | __WQ_LEGACY)) == WQ_MEM_RECLAIM),
"workqueue: WQ_MEM_RECLAIM %s:%ps is flushing !WQ_MEM_RECLAIM %s:%ps",
worker->current_pwq->wq->name, worker->current_func,
diff --git a/mm/backing-dev.c b/mm/backing-dev.c
index 014252f948a7..48d4cae46c07 100644
--- a/mm/backing-dev.c
+++ b/mm/backing-dev.c
@@ -910,8 +910,9 @@ static void cleanup_offline_cgwbs_workfn(struct work_struct *work)
continue;
spin_unlock_irq(&cgwb_lock);
- while (cleanup_offline_cgwb(wb))
- cond_resched();
+ do {
+ cond_resched_tasks_rcu_qs();
+ } while (cleanup_offline_cgwb(wb));
spin_lock_irq(&cgwb_lock);
wb_put(wb);
diff --git a/mm/damon/core.c b/mm/damon/core.c
index 6fa02025d2af..c0dcb190b06e 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -1746,37 +1746,41 @@ static bool damos_skip_charged_region(struct damon_target *t,
struct damon_region *r = *rp;
struct damos_quota *quota = &s->quota;
unsigned long sz_to_skip;
+ bool skip = false;
/* Skip previously charged regions */
if (quota->charge_target_from) {
if (t != quota->charge_target_from)
return true;
- if (r == damon_last_region(t)) {
- quota->charge_target_from = NULL;
- quota->charge_addr_from = 0;
- return true;
- }
if (quota->charge_addr_from &&
- r->ar.end <= quota->charge_addr_from)
- return true;
+ r->ar.end <= quota->charge_addr_from) {
+ skip = true;
+ goto out;
+ }
if (quota->charge_addr_from && r->ar.start <
quota->charge_addr_from) {
sz_to_skip = ALIGN_DOWN(quota->charge_addr_from -
r->ar.start, min_sz_region);
if (!sz_to_skip) {
- if (damon_sz_region(r) <= min_sz_region)
- return true;
+ if (damon_sz_region(r) <= min_sz_region) {
+ skip = true;
+ goto out;
+ }
sz_to_skip = min_sz_region;
}
damon_split_region_at(t, r, sz_to_skip);
r = damon_next_region(r);
*rp = r;
+ skip = false;
}
+ }
+out:
+ if (r == damon_last_region(t)) {
quota->charge_target_from = NULL;
quota->charge_addr_from = 0;
}
- return false;
+ return skip;
}
static void damos_update_stat(struct damos *s,
diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
index c49f494ec17c..cfa7849041ad 100644
--- a/mm/damon/vaddr.c
+++ b/mm/damon/vaddr.c
@@ -339,22 +339,29 @@ static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned long addr,
}
#ifdef CONFIG_HUGETLB_PAGE
+static bool damon_hugetlb_ptep_mkold(pte_t *pte, struct mm_struct *mm,
+ struct vm_area_struct *vma, unsigned long addr, pte_t *entry)
+{
+ unsigned long psize = huge_page_size(hstate_vma(vma));
+
+ if (!pte_young(*entry))
+ return false;
+ *entry = huge_ptep_get_and_clear(mm, addr, pte, psize);
+ *entry = pte_mkold(*entry);
+ set_huge_pte_at(mm, addr, pte, *entry, psize);
+ return true;
+}
+
static void damon_hugetlb_mkold(pte_t *pte, struct mm_struct *mm,
struct vm_area_struct *vma, unsigned long addr)
{
bool referenced = false;
pte_t entry = huge_ptep_get(mm, addr, pte);
struct folio *folio = pfn_folio(pte_pfn(entry));
- unsigned long psize = huge_page_size(hstate_vma(vma));
folio_get(folio);
- if (pte_young(entry)) {
- referenced = true;
- entry = pte_mkold(entry);
- set_huge_pte_at(mm, addr, pte, entry, psize);
- }
-
+ referenced = damon_hugetlb_ptep_mkold(pte, mm, vma, addr, &entry);
if (mmu_notifier_clear_young(mm, addr,
addr + huge_page_size(hstate_vma(vma))))
referenced = true;
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index d969c9ed91b1..9df625de6dfe 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -138,6 +138,17 @@ static void hugetlb_free_folio(struct folio *folio)
folio_put(folio);
}
+/*
+ * Check if the hstate represents gigantic pages but gigantic page
+ * runtime support is not available. This is a common condition used to
+ * skip operations that cannot be performed on gigantic pages when runtime
+ * support is disabled.
+ */
+static inline bool hstate_is_gigantic_no_runtime(struct hstate *h)
+{
+ return hstate_is_gigantic(h) && !gigantic_page_runtime_supported();
+}
+
static inline bool subpool_is_free(struct hugepage_subpool *spool)
{
if (spool->count)
@@ -1538,7 +1549,7 @@ static void remove_hugetlb_folio(struct hstate *h, struct folio *folio,
VM_BUG_ON_FOLIO(hugetlb_cgroup_from_folio_rsvd(folio), folio);
lockdep_assert_held(&hugetlb_lock);
- if (hstate_is_gigantic(h) && !gigantic_page_runtime_supported())
+ if (hstate_is_gigantic_no_runtime(h))
return;
list_del(&folio->lru);
@@ -1600,7 +1611,7 @@ static void __update_and_free_hugetlb_folio(struct hstate *h,
{
bool clear_flag = folio_test_hugetlb_vmemmap_optimized(folio);
- if (hstate_is_gigantic(h) && !gigantic_page_runtime_supported())
+ if (hstate_is_gigantic_no_runtime(h))
return;
/*
@@ -2115,6 +2126,15 @@ int dissolve_free_hugetlb_folio(struct folio *folio)
struct hstate *h = folio_hstate(folio);
bool adjust_surplus = false;
+ /*
+ * remove_hugetlb_folio()/update_and_free_hugetlb_folio() bail
+ * for gigantic hstates without runtime support, so dissolving one
+ * here would leave it on the free list and, on vmemmap restore
+ * failure, the add_hugetlb_folio() rollback corrupts that list.
+ */
+ if (hstate_is_gigantic_no_runtime(h))
+ goto out;
+
if (!available_huge_pages(h))
goto out;
@@ -2496,7 +2516,7 @@ static void return_unused_surplus_pages(struct hstate *h,
/* Uncommit the reservation */
h->resv_huge_pages -= unused_resv_pages;
- if (hstate_is_gigantic(h) && !gigantic_page_runtime_supported())
+ if (hstate_is_gigantic_no_runtime(h))
goto out;
/*
@@ -3711,7 +3731,7 @@ static void __init hugetlb_init_hstates(void)
* - If CMA allocation is possible, we can not demote
* HUGETLB_PAGE_ORDER or smaller size pages.
*/
- if (hstate_is_gigantic(h) && !gigantic_page_runtime_supported())
+ if (hstate_is_gigantic_no_runtime(h))
continue;
if (hugetlb_cma_total_size() && h->order <= HUGETLB_PAGE_ORDER)
continue;
@@ -4188,7 +4208,7 @@ static ssize_t __nr_hugepages_store_common(bool obey_mempolicy,
int err;
nodemask_t nodes_allowed, *n_mask;
- if (hstate_is_gigantic(h) && !gigantic_page_runtime_supported())
+ if (hstate_is_gigantic_no_runtime(h))
return -EINVAL;
if (nid == NUMA_NO_NODE) {
@@ -5828,18 +5848,21 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma,
hugetlb_vma_lock_write(vma);
i_mmap_lock_write(mapping);
for (; old_addr < old_end; old_addr += sz, new_addr += sz) {
+ const unsigned long offset_to_last_entry =
+ (old_addr | last_addr_mask) - old_addr;
+
src_pte = hugetlb_walk(vma, old_addr, sz);
if (!src_pte) {
- old_addr |= last_addr_mask;
- new_addr |= last_addr_mask;
+ old_addr += offset_to_last_entry;
+ new_addr += offset_to_last_entry;
continue;
}
if (huge_pte_none(huge_ptep_get(mm, old_addr, src_pte)))
continue;
if (huge_pmd_unshare(&tlb, vma, old_addr, src_pte)) {
- old_addr |= last_addr_mask;
- new_addr |= last_addr_mask;
+ old_addr += offset_to_last_entry;
+ new_addr += offset_to_last_entry;
continue;
}
diff --git a/mm/rmap.c b/mm/rmap.c
index 29f93ecf3480..4f7a44093a09 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -146,14 +146,13 @@ static void anon_vma_chain_free(struct anon_vma_chain *anon_vma_chain)
kmem_cache_free(anon_vma_chain_cachep, anon_vma_chain);
}
-static void anon_vma_chain_link(struct vm_area_struct *vma,
- struct anon_vma_chain *avc,
- struct anon_vma *anon_vma)
+static void anon_vma_chain_assign(struct vm_area_struct *vma,
+ struct anon_vma_chain *avc,
+ struct anon_vma *anon_vma)
{
avc->vma = vma;
avc->anon_vma = anon_vma;
list_add(&avc->same_vma, &vma->anon_vma_chain);
- anon_vma_interval_tree_insert(avc, &anon_vma->rb_root);
}
/**
@@ -209,8 +208,13 @@ int __anon_vma_prepare(struct vm_area_struct *vma)
/* page_table_lock to protect against threads */
spin_lock(&mm->page_table_lock);
if (likely(!vma->anon_vma)) {
- vma->anon_vma = anon_vma;
- anon_vma_chain_link(vma, avc, anon_vma);
+ /*
+ * Make anon_vma fields visible before anon_vma is published.
+ * Paired with an address dependency in reusable_anon_vma().
+ */
+ smp_store_release(&vma->anon_vma, anon_vma);
+ anon_vma_chain_assign(vma, avc, anon_vma);
+ anon_vma_interval_tree_insert(avc, &anon_vma->rb_root);
anon_vma->num_active_vmas++;
allocated = NULL;
avc = NULL;
@@ -294,7 +298,8 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)
}
anon_vma = pavc->anon_vma;
root = lock_anon_vma_root(root, anon_vma);
- anon_vma_chain_link(dst, avc, anon_vma);
+ anon_vma_chain_assign(dst, avc, anon_vma);
+ anon_vma_interval_tree_insert(avc, &anon_vma->rb_root);
/*
* Reuse existing anon_vma if it has no vma and only one
@@ -378,8 +383,10 @@ int anon_vma_fork(struct vm_area_struct *vma, struct vm_area_struct *pvma)
get_anon_vma(anon_vma->root);
/* Mark this anon_vma as the one where our new (COWed) pages go. */
vma->anon_vma = anon_vma;
+ anon_vma_chain_assign(vma, avc, anon_vma);
+ /* Now let rmap see it. */
anon_vma_lock_write(anon_vma);
- anon_vma_chain_link(vma, avc, anon_vma);
+ anon_vma_interval_tree_insert(avc, &anon_vma->rb_root);
anon_vma->parent->num_children++;
anon_vma_unlock_write(anon_vma);
diff --git a/mm/vma.c b/mm/vma.c
index dcaa2d8998bf..5beb18b0610b 100644
--- a/mm/vma.c
+++ b/mm/vma.c
@@ -1979,6 +1979,13 @@ static int anon_vma_compatible(struct vm_area_struct *a, struct vm_area_struct *
* acceptable for merging, so we can do all of this optimistically. But
* we do that READ_ONCE() to make sure that we never re-load the pointer.
*
+ * The READ_ONCE() establishes an address dependency between anon_vma and
+ * any access to its fields, which pairs with the assignment to
+ * vma->anon_vma performed with release semantics in __anon_vma_prepare().
+ *
+ * This is especially important as anon_vma's are SLAB_TYPESAFE_BY_RCU so
+ * accessing an uninitialised anon_vma's fields may result in a UAF.
+ *
* IOW: that the "list_is_singular()" test on the anon_vma_chain only
* matters for the 'stable anon_vma' case (ie the thing we want to avoid
* is to return an anon_vma that is "complex" due to having gone through
@@ -1993,6 +2000,7 @@ static struct anon_vma *reusable_anon_vma(struct vm_area_struct *old,
struct vm_area_struct *b)
{
if (anon_vma_compatible(a, b)) {
+ /* Paired with a memory barrier in __anon_vma_prepare(). */
struct anon_vma *anon_vma = READ_ONCE(old->anon_vma);
if (anon_vma && list_is_singular(&old->anon_vma_chain))
diff --git a/net/8021q/vlan_dev.c b/net/8021q/vlan_dev.c
index 560521cf596e..618c8617c2c4 100644
--- a/net/8021q/vlan_dev.c
+++ b/net/8021q/vlan_dev.c
@@ -55,6 +55,11 @@ static int vlan_dev_hard_header(struct sk_buff *skb, struct net_device *dev,
int rc;
if (!(vlan->flags & VLAN_FLAG_REORDER_HDR)) {
+ unsigned int hlen = READ_ONCE(dev->hard_header_len) +
+ READ_ONCE(dev->needed_headroom);
+
+ if (skb_cow_head(skb, hlen) < 0)
+ return -ENOMEM;
vhdr = skb_push(skb, VLAN_HLEN);
vlan_tci = vlan->vlan_id;
diff --git a/net/bluetooth/bnep/core.c b/net/bluetooth/bnep/core.c
index ed3ca38d8a6e..e93c79982abc 100644
--- a/net/bluetooth/bnep/core.c
+++ b/net/bluetooth/bnep/core.c
@@ -273,9 +273,14 @@ static int bnep_rx_extension(struct bnep_session *s, struct sk_buff *skb)
BT_DBG("type 0x%x len %u", h->type, h->len);
+ if (skb->len < h->len) {
+ err = -EILSEQ;
+ break;
+ }
+
switch (h->type & BNEP_TYPE_MASK) {
case BNEP_EXT_CONTROL:
- bnep_rx_control(s, skb->data, skb->len);
+ bnep_rx_control(s, skb->data, h->len);
break;
default:
@@ -376,6 +381,11 @@ static int bnep_rx_frame(struct bnep_session *s, struct sk_buff *skb)
goto badframe;
}
+ if ((type & BNEP_TYPE_MASK) == BNEP_CONTROL) {
+ kfree_skb(skb);
+ return 0;
+ }
+
/* Strip 802.1p header */
if (ntohs(s->eh.h_proto) == ETH_P_8021Q) {
if (!skb_pull(skb, 4))
@@ -454,6 +464,11 @@ static int bnep_tx_frame(struct bnep_session *s, struct sk_buff *skb)
goto send;
}
+ if (skb->len < ETH_HLEN) {
+ kfree_skb(skb);
+ return 0;
+ }
+
iv[il++] = (struct kvec) { &type, 1 };
len++;
diff --git a/net/bluetooth/bnep/netdev.c b/net/bluetooth/bnep/netdev.c
index cc1cff63194f..5d9d851db926 100644
--- a/net/bluetooth/bnep/netdev.c
+++ b/net/bluetooth/bnep/netdev.c
@@ -169,6 +169,12 @@ static netdev_tx_t bnep_net_xmit(struct sk_buff *skb,
BT_DBG("skb %p, dev %p", skb, dev);
+ if (!pskb_may_pull(skb, ETH_HLEN)) {
+ dev->stats.tx_dropped++;
+ kfree_skb(skb);
+ return NETDEV_TX_OK;
+ }
+
#ifdef CONFIG_BT_BNEP_MC_FILTER
if (bnep_net_mc_filter(skb, s)) {
kfree_skb(skb);
@@ -221,7 +227,7 @@ void bnep_net_setup(struct net_device *dev)
dev->addr_len = ETH_ALEN;
ether_setup(dev);
- dev->min_mtu = 0;
+ dev->min_mtu = ETH_MIN_MTU;
dev->max_mtu = ETH_MAX_MTU;
dev->priv_flags &= ~IFF_TX_SKB_SHARING;
dev->netdev_ops = &bnep_netdev_ops;
diff --git a/net/bluetooth/hci_conn.c b/net/bluetooth/hci_conn.c
index 6edc26fe6d2d..d50fa9eea42c 100644
--- a/net/bluetooth/hci_conn.c
+++ b/net/bluetooth/hci_conn.c
@@ -1975,6 +1975,8 @@ struct hci_conn *hci_bind_cis(struct hci_dev *hdev, bdaddr_t *dst,
cis->conn_timeout = timeout;
}
+ hci_conn_hold(cis);
+
if (cis->state == BT_CONNECTED)
return cis;
@@ -2016,7 +2018,6 @@ struct hci_conn *hci_bind_cis(struct hci_dev *hdev, bdaddr_t *dst,
return ERR_PTR(-EINVAL);
}
- hci_conn_hold(cis);
cis->state = BT_BOUND;
return cis;
@@ -2272,10 +2273,13 @@ struct hci_conn *hci_bind_bis(struct hci_dev *hdev, bdaddr_t *dst, __u8 sid,
parent = hci_conn_hash_lookup_big(hdev,
conn->iso_qos.bcast.big);
if (parent && parent != conn) {
+ hci_conn_hold(parent);
link = hci_conn_link(parent, conn);
hci_conn_drop(conn);
- if (!link)
+ if (!link) {
+ hci_conn_drop(parent);
return ERR_PTR(-ENOLINK);
+ }
}
return conn;
@@ -2381,6 +2385,12 @@ struct hci_conn *hci_connect_cis(struct hci_dev *hdev, bdaddr_t *dst,
return cis;
}
+ /* The existing link already owns the hold on its parent. */
+ if (cis->link) {
+ hci_conn_drop(le);
+ return cis;
+ }
+
link = hci_conn_link(le, cis);
hci_conn_drop(cis);
if (!link) {
diff --git a/net/bluetooth/hci_sock.c b/net/bluetooth/hci_sock.c
index 5295ff001e38..694a11fb1286 100644
--- a/net/bluetooth/hci_sock.c
+++ b/net/bluetooth/hci_sock.c
@@ -166,6 +166,7 @@ static bool is_filtered_packet(struct sock *sk, struct sk_buff *skb)
{
struct hci_filter *flt;
int flt_type, flt_event;
+ u8 event;
/* Apply filter */
flt = &hci_pi(sk)->filter;
@@ -179,7 +180,11 @@ static bool is_filtered_packet(struct sock *sk, struct sk_buff *skb)
if (hci_skb_pkt_type(skb) != HCI_EVENT_PKT)
return false;
- flt_event = (*(__u8 *)skb->data & HCI_FLT_EVENT_BITS);
+ if (skb->len < 1)
+ return true;
+
+ event = *(__u8 *)skb->data;
+ flt_event = event & HCI_FLT_EVENT_BITS;
if (!hci_test_bit(flt_event, &flt->event_mask))
return true;
@@ -188,11 +193,17 @@ static bool is_filtered_packet(struct sock *sk, struct sk_buff *skb)
if (!flt->opcode)
return false;
- if (flt_event == HCI_EV_CMD_COMPLETE &&
+ if (event == HCI_EV_CMD_COMPLETE && skb->len < 5)
+ return true;
+
+ if (event == HCI_EV_CMD_COMPLETE &&
flt->opcode != get_unaligned((__le16 *)(skb->data + 3)))
return true;
- if (flt_event == HCI_EV_CMD_STATUS &&
+ if (event == HCI_EV_CMD_STATUS && skb->len < 6)
+ return true;
+
+ if (event == HCI_EV_CMD_STATUS &&
flt->opcode != get_unaligned((__le16 *)(skb->data + 4)))
return true;
@@ -1883,7 +1894,8 @@ static int hci_sock_sendmsg(struct socket *sock, struct msghdr *msg,
u16 ocf = hci_opcode_ocf(opcode);
if (((ogf > HCI_SFLT_MAX_OGF) ||
- !hci_test_bit(ocf & HCI_FLT_OCF_BITS,
+ (ocf > HCI_FLT_OCF_BITS) ||
+ !hci_test_bit(ocf,
&hci_sec_filter.ocf_mask[ogf])) &&
!capable(CAP_NET_RAW)) {
err = -EPERM;
diff --git a/net/bluetooth/iso.c b/net/bluetooth/iso.c
index 14f821aa8ac4..272980ca3038 100644
--- a/net/bluetooth/iso.c
+++ b/net/bluetooth/iso.c
@@ -460,6 +460,7 @@ static int iso_connect_cis(struct sock *sk)
struct hci_dev *hdev;
bdaddr_t src, dst;
u8 src_type;
+ bool already_attached;
int err;
lock_sock(sk);
@@ -530,8 +531,14 @@ static int iso_connect_cis(struct sock *sk)
goto unlock;
}
+ iso_conn_lock(conn);
+ already_attached = iso_pi(sk)->conn == conn && conn->sk == sk;
+ iso_conn_unlock(conn);
+
err = iso_chan_add(conn, sk, NULL);
iso_conn_put(conn);
+ if (already_attached || err == -EBUSY)
+ hci_conn_drop(hcon);
if (err)
goto unlock;
diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
index 9365a98107f2..7d4115603d08 100644
--- a/net/bluetooth/l2cap_core.c
+++ b/net/bluetooth/l2cap_core.c
@@ -6709,9 +6709,17 @@ static int l2cap_stream_rx(struct l2cap_chan *chan, struct l2cap_ctrl *control,
static int l2cap_data_rcv(struct l2cap_chan *chan, struct sk_buff *skb)
{
struct l2cap_ctrl *control = &bt_cb(skb)->l2cap;
- u16 len;
+ u16 len, min_len;
u8 event;
+ min_len = test_bit(FLAG_EXT_CTRL, &chan->flags) ?
+ L2CAP_EXT_CTRL_SIZE : L2CAP_ENH_CTRL_SIZE;
+ if (chan->fcs == L2CAP_FCS_CRC16)
+ min_len += L2CAP_FCS_SIZE;
+
+ if (skb->len < min_len)
+ goto drop;
+
__unpack_control(chan, skb);
len = skb->len;
diff --git a/net/bluetooth/mgmt.c b/net/bluetooth/mgmt.c
index 01f46a985e2b..96950bd53ab8 100644
--- a/net/bluetooth/mgmt.c
+++ b/net/bluetooth/mgmt.c
@@ -499,13 +499,9 @@ static int read_unconf_index_list(struct sock *sk, struct hci_dev *hdev,
read_lock(&hci_dev_list_lock);
- count = 0;
- list_for_each_entry(d, &hci_dev_list, list) {
- if (hci_dev_test_flag(d, HCI_UNCONFIGURED))
- count++;
- }
+ count = list_count_nodes(&hci_dev_list);
- rp_len = sizeof(*rp) + (2 * count);
+ rp_len = sizeof(*rp) + (sizeof(__le16) * count);
rp = kmalloc(rp_len, GFP_ATOMIC);
if (!rp) {
read_unlock(&hci_dev_list_lock);
@@ -2313,6 +2309,8 @@ static void mesh_send_start_complete(struct hci_dev *hdev, void *data, int err)
hci_dev_clear_flag(hdev, HCI_MESH_SENDING);
/* Send Complete Error Code for handle */
mesh_send_complete(hdev, mesh_tx, false);
+ if (err != -ECANCELED)
+ mesh_next(hdev, NULL, 0);
return;
}
@@ -2422,19 +2420,28 @@ static int send_cancel(struct hci_dev *hdev, void *data)
do {
mesh_tx = mgmt_mesh_next(hdev, cmd->sk);
- if (mesh_tx)
- mesh_send_complete(hdev, mesh_tx, false);
+ if (mesh_tx) {
+ if (!hci_cmd_sync_dequeue(hdev, mesh_send_sync,
+ mesh_tx, NULL))
+ mesh_send_complete(hdev, mesh_tx, false);
+ }
} while (mesh_tx);
} else {
mesh_tx = mgmt_mesh_find(hdev, cancel->handle);
- if (mesh_tx && mesh_tx->sk == cmd->sk)
- mesh_send_complete(hdev, mesh_tx, false);
+ if (mesh_tx && mesh_tx->sk == cmd->sk) {
+ if (!hci_cmd_sync_dequeue(hdev, mesh_send_sync,
+ mesh_tx, NULL))
+ mesh_send_complete(hdev, mesh_tx, false);
+ }
}
mgmt_cmd_complete(cmd->sk, hdev->id, MGMT_OP_MESH_SEND_CANCEL,
0, NULL, 0);
+ if (!hci_dev_test_flag(hdev, HCI_MESH_SENDING))
+ mesh_next(hdev, NULL, 0);
+
return 0;
}
diff --git a/net/bluetooth/rfcomm/core.c b/net/bluetooth/rfcomm/core.c
index 45f8cfebcc6a..99c295b9f666 100644
--- a/net/bluetooth/rfcomm/core.c
+++ b/net/bluetooth/rfcomm/core.c
@@ -1817,7 +1817,8 @@ static struct rfcomm_session *rfcomm_recv_frame(struct rfcomm_session *s,
return s;
}
- if (skb->len < sizeof(*hdr) + 1) {
+ if (skb->len < sizeof(*hdr) + 1 ||
+ (!__test_ea(hdr->len) && skb->len < sizeof(*hdr) + 2)) {
kfree_skb(skb);
return s;
}
diff --git a/net/bluetooth/rfcomm/sock.c b/net/bluetooth/rfcomm/sock.c
index 037a7fcab302..ddf2153aa832 100644
--- a/net/bluetooth/rfcomm/sock.c
+++ b/net/bluetooth/rfcomm/sock.c
@@ -785,8 +785,10 @@ static int rfcomm_sock_getsockopt_old(struct socket *sock, int optname, char __u
break;
case RFCOMM_CONNINFO:
- if (sk->sk_state != BT_CONNECTED &&
- !rfcomm_pi(sk)->dlc->defer_setup) {
+ if ((sk->sk_state != BT_CONNECTED &&
+ !(sk->sk_state == BT_CONNECT2 &&
+ rfcomm_pi(sk)->dlc->defer_setup)) ||
+ !rfcomm_pi(sk)->dlc->session) {
err = -ENOTCONN;
break;
}
diff --git a/net/bluetooth/smp.c b/net/bluetooth/smp.c
index 1172df9b7004..51e0545a9e24 100644
--- a/net/bluetooth/smp.c
+++ b/net/bluetooth/smp.c
@@ -2301,6 +2301,23 @@ static u8 smp_cmd_security_req(struct l2cap_conn *conn, struct sk_buff *skb)
bt_dev_dbg(hdev, "conn %p", conn);
+ /* SMP over BR/EDR only covers cross-transport key derivation; the
+ * Security Request procedure has no BR/EDR counterpart. Reject it
+ * here, otherwise smp_ltk_encrypt() finds the peer's LE LTK
+ * (ADDR_LE_DEV_PUBLIC and BDADDR_BREDR are both 0) and issues
+ * HCI_OP_LE_START_ENC on the ACL handle, which the controller
+ * rejects and hci_cs_le_start_enc() turns into a disconnect. Reply
+ * without smp_failure(): this is not an authentication failure, and
+ * MGMT_EV_AUTH_FAILED would make bluetoothd drop the device.
+ */
+ if (hcon->type != LE_LINK) {
+ u8 reason = SMP_CMD_NOTSUPP;
+
+ smp_send_cmd(conn, SMP_CMD_PAIRING_FAIL, sizeof(reason),
+ &reason);
+ return 0;
+ }
+
if (skb->len < sizeof(*rp))
return SMP_INVALID_PARAMS;
diff --git a/net/bridge/br_mdb.c b/net/bridge/br_mdb.c
index 400eb872b403..a80ec51fe18d 100644
--- a/net/bridge/br_mdb.c
+++ b/net/bridge/br_mdb.c
@@ -1523,6 +1523,8 @@ static void br_mdb_flush_pgs(struct net_bridge *br,
}
br_multicast_del_pg(mp, p, pp);
+ /* br_multicast_del_pg() can remove other groups from this list. */
+ pp = &mp->ports;
}
}
diff --git a/net/bridge/br_stp_bpdu.c b/net/bridge/br_stp_bpdu.c
index 7895489ac6fe..e44e448b1157 100644
--- a/net/bridge/br_stp_bpdu.c
+++ b/net/bridge/br_stp_bpdu.c
@@ -52,7 +52,10 @@ static void br_send_bpdu(struct net_bridge_port *p,
LLC_SAP_BSPAN, LLC_PDU_CMD);
llc_pdu_init_as_ui_cmd(skb);
- llc_mac_hdr_init(skb, p->dev->dev_addr, p->br->group_addr);
+ if (llc_mac_hdr_init(skb, p->dev->dev_addr, p->br->group_addr)) {
+ kfree_skb(skb);
+ return;
+ }
skb_reset_mac_header(skb);
diff --git a/net/core/dev.c b/net/core/dev.c
index 48983bac5b53..6569605cc666 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -2863,7 +2863,7 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask,
dev = netdev_get_tx_queue(dev, index)->sb_dev ? : dev;
tc = netdev_txq_to_tc(dev, index);
- if (tc < 0)
+ if (tc < 0 || tc >= num_tc)
return -EINVAL;
}
diff --git a/net/core/filter.c b/net/core/filter.c
index 35427793ddb7..308b86abe460 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -3825,12 +3825,6 @@ static u32 __bpf_skb_min_len(const struct sk_buff *skb)
if (offset > 0)
min_len = offset;
}
- if (skb->ip_summed == CHECKSUM_PARTIAL) {
- offset = skb_checksum_start_offset(skb) +
- skb->csum_offset + sizeof(__sum16);
- if (offset > 0)
- min_len = offset;
- }
return min_len;
}
@@ -3847,6 +3841,11 @@ static int bpf_skb_grow_rcsum(struct sk_buff *skb, unsigned int new_len)
static int bpf_skb_trim_rcsum(struct sk_buff *skb, unsigned int new_len)
{
+ if (skb->ip_summed == CHECKSUM_PARTIAL &&
+ new_len < skb_checksum_start_offset(skb) + skb->csum_offset +
+ sizeof(__sum16))
+ skb->ip_summed = CHECKSUM_NONE;
+
return __skb_trim_rcsum(skb, new_len);
}
@@ -8726,6 +8725,8 @@ static const struct bpf_func_proto *
lwt_seg6local_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
{
switch (func_id) {
+ case BPF_FUNC_skb_pull_data:
+ return NULL;
#if IS_ENABLED(CONFIG_IPV6_SEG6_BPF)
case BPF_FUNC_lwt_seg6_store_bytes:
return &bpf_lwt_seg6_store_bytes_proto;
@@ -10246,11 +10247,12 @@ u32 bpf_sock_convert_ctx_access(enum bpf_access_type type,
target_size));
*insn++ = BPF_JMP_IMM(BPF_JNE, si->dst_reg, NO_QUEUE_MAPPING,
1);
- *insn++ = BPF_MOV64_IMM(si->dst_reg, -1);
+ *insn++ = BPF_MOV32_IMM(si->dst_reg, -1);
#else
- *insn++ = BPF_MOV64_IMM(si->dst_reg, -1);
- *target_size = 2;
+ *insn++ = BPF_MOV32_IMM(si->dst_reg, -1);
#endif
+ *target_size = sizeof_field(struct bpf_sock, rx_queue_mapping);
+
break;
}
@@ -10786,18 +10788,7 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type,
break;
case offsetof(struct bpf_sock_ops, rtt_min):
- BUILD_BUG_ON(sizeof_field(struct tcp_sock, rtt_min) !=
- sizeof(struct minmax));
- BUILD_BUG_ON(sizeof(struct minmax) <
- sizeof(struct minmax_sample));
-
- *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(
- struct bpf_sock_ops_kern, sk),
- si->dst_reg, si->src_reg,
- offsetof(struct bpf_sock_ops_kern, sk));
- *insn++ = BPF_LDX_MEM(BPF_W, si->dst_reg, si->dst_reg,
- offsetof(struct tcp_sock, rtt_min) +
- sizeof_field(struct minmax_sample, t));
+ SOCK_OPS_GET_FIELD(rtt_min, rtt_min.s[0].v, struct tcp_sock);
break;
case offsetof(struct bpf_sock_ops, bpf_sock_ops_cb_flags):
@@ -12496,8 +12487,9 @@ __bpf_kfunc_start_defs();
* @sock: Pointer to socket to be destroyed
*
* Return:
- * On error, may return EPROTONOSUPPORT, EINVAL.
- * EPROTONOSUPPORT if protocol specific destroy handler is not supported.
+ * On error, may return EOPNOTSUPP, or whatever the protocol specific
+ * destroy handler returns.
+ * EOPNOTSUPP if protocol specific destroy handler is not supported.
* 0 otherwise
*/
__bpf_kfunc int bpf_sock_destroy(struct sock_common *sock)
@@ -12509,8 +12501,12 @@ __bpf_kfunc int bpf_sock_destroy(struct sock_common *sock)
* Supporting protocols will need to acquire sock lock in the BPF context
* prior to invoking this kfunc.
*/
- if (!sk->sk_prot->diag_destroy || (sk->sk_protocol != IPPROTO_TCP &&
- sk->sk_protocol != IPPROTO_UDP))
+ if (!sk->sk_prot->diag_destroy)
+ return -EOPNOTSUPP;
+
+ if (sk_fullsock(sk) &&
+ sk->sk_protocol != IPPROTO_TCP &&
+ sk->sk_protocol != IPPROTO_UDP)
return -EOPNOTSUPP;
return sk->sk_prot->diag_destroy(sk, ECONNABORTED);
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 594764eab40d..9ef8b31dbc87 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -5911,7 +5911,8 @@ static int skb_checksum_setup_ipv6(struct sk_buff *skb, bool recalculate)
err = skb_maybe_pull_tail(skb,
off +
sizeof(struct ipv6_opt_hdr),
- MAX_IPV6_HDR_LEN);
+ off +
+ sizeof(struct ipv6_opt_hdr));
if (err < 0)
goto out;
@@ -5926,7 +5927,8 @@ static int skb_checksum_setup_ipv6(struct sk_buff *skb, bool recalculate)
err = skb_maybe_pull_tail(skb,
off +
sizeof(struct ip_auth_hdr),
- MAX_IPV6_HDR_LEN);
+ off +
+ sizeof(struct ip_auth_hdr));
if (err < 0)
goto out;
@@ -5941,7 +5943,8 @@ static int skb_checksum_setup_ipv6(struct sk_buff *skb, bool recalculate)
err = skb_maybe_pull_tail(skb,
off +
sizeof(struct frag_hdr),
- MAX_IPV6_HDR_LEN);
+ off +
+ sizeof(struct frag_hdr));
if (err < 0)
goto out;
diff --git a/net/core/skmsg.c b/net/core/skmsg.c
index c1c315ae2b22..f5c12ef68675 100644
--- a/net/core/skmsg.c
+++ b/net/core/skmsg.c
@@ -1036,6 +1036,10 @@ static int sk_psock_verdict_apply(struct sk_psock *psock, struct sk_buff *skb,
int err = 0;
u32 len, off;
+ if (verdict == __SK_REDIRECT && skb_bpf_ingress(skb) &&
+ skb_bpf_redirect_fetch(skb) == psock->sk)
+ verdict = __SK_PASS;
+
switch (verdict) {
case __SK_PASS:
err = -EIO;
diff --git a/net/core/sock_map.c b/net/core/sock_map.c
index 70bbc78fb079..ebe8e166b6a7 100644
--- a/net/core/sock_map.c
+++ b/net/core/sock_map.c
@@ -41,6 +41,7 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr)
struct bpf_stab *stab;
if (attr->max_entries == 0 ||
+ attr->max_entries > INT_MAX ||
attr->key_size != 4 ||
(attr->value_size != sizeof(u32) &&
attr->value_size != sizeof(u64)) ||
diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c
index 3ce1664e8632..22d5620683df 100644
--- a/net/ipv4/arp.c
+++ b/net/ipv4/arp.c
@@ -1278,6 +1278,7 @@ int arp_ioctl(struct net *net, unsigned int cmd, void __user *arg)
err = copy_from_user(&r, arg, sizeof(struct arpreq));
if (err)
return -EFAULT;
+ r.arp_dev[IFNAMSIZ - 1] = '\0';
break;
default:
return -EINVAL;
diff --git a/net/ipv4/fib_semantics.c b/net/ipv4/fib_semantics.c
index 2aba8c0b9b2b..5ab23a1fa45f 100644
--- a/net/ipv4/fib_semantics.c
+++ b/net/ipv4/fib_semantics.c
@@ -2181,6 +2181,15 @@ static bool fib_good_nh(const struct fib_nh *nh)
return !!(state & NUD_VALID);
}
+static __be32 fib_nh_saddr(struct net *net, const struct fib_info *fi,
+ struct fib_nh *nh, int genid)
+{
+ if (READ_ONCE(nh->nh_saddr_genid) == genid)
+ return READ_ONCE(nh->nh_saddr);
+
+ return fib_info_update_nhc_saddr(net, &nh->nh_common, fi->fib_scope);
+}
+
void fib_select_multipath(struct fib_result *res, int hash,
const struct flowi4 *fl4)
{
@@ -2189,6 +2198,7 @@ void fib_select_multipath(struct fib_result *res, int hash,
bool use_neigh;
int score = -1;
__be32 saddr;
+ int genid;
if (unlikely(res->fi->nh)) {
nexthop_path_fib_result(res, hash);
@@ -2197,6 +2207,7 @@ void fib_select_multipath(struct fib_result *res, int hash,
use_neigh = READ_ONCE(net->ipv4.sysctl_fib_multipath_use_neigh);
saddr = fl4 ? fl4->saddr : 0;
+ genid = saddr ? atomic_read(&net->ipv4.dev_addr_genid) : 0;
change_nexthops(fi) {
int nh_upper_bound, nh_score = 0;
@@ -2209,7 +2220,7 @@ void fib_select_multipath(struct fib_result *res, int hash,
(use_neigh && !fib_good_nh(nexthop_nh)))
continue;
- if (saddr && nexthop_nh->nh_saddr == saddr)
+ if (saddr && fib_nh_saddr(net, fi, nexthop_nh, genid) == saddr)
nh_score += 2;
if (hash <= nh_upper_bound)
nh_score++;
diff --git a/net/ipv4/fou_core.c b/net/ipv4/fou_core.c
index d06c20deb175..fc930274a1fd 100644
--- a/net/ipv4/fou_core.c
+++ b/net/ipv4/fou_core.c
@@ -603,6 +603,10 @@ static int fou_create(struct net *net, struct fou_cfg *cfg,
/* Initial for fou type */
switch (cfg->type) {
case FOU_ENCAP_DIRECT:
+ if (!cfg->protocol) {
+ err = -EINVAL;
+ goto error;
+ }
tunnel_cfg.encap_rcv = fou_udp_recv;
tunnel_cfg.gro_receive = fou_gro_receive;
tunnel_cfg.gro_complete = fou_gro_complete;
diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c
index 6193db63b297..a44c8096725f 100644
--- a/net/ipv4/inet_connection_sock.c
+++ b/net/ipv4/inet_connection_sock.c
@@ -1533,7 +1533,8 @@ void inet_csk_listen_stop(struct sock *sk)
local_bh_enable();
sock_put(child);
- cond_resched();
+ if (!has_current_bpf_ctx())
+ cond_resched();
}
if (queue->fastopenq.rskq_rst_head) {
/* Free all the reqs queued in rskq_rst_head. */
diff --git a/net/ipv4/ip_gre.c b/net/ipv4/ip_gre.c
index 300b802125db..b15cc4ee0d92 100644
--- a/net/ipv4/ip_gre.c
+++ b/net/ipv4/ip_gre.c
@@ -1459,6 +1459,12 @@ static int ipgre_changelink(struct net_device *dev, struct nlattr *tb[],
if (!rtnl_dev_link_net_capable(dev, t->net))
return -EPERM;
+ if (data && data[IFLA_GRE_COLLECT_METADATA] && !t->collect_md) {
+ NL_SET_ERR_MSG(extack,
+ "Enabling collect_md on an existing device is not supported");
+ return -EOPNOTSUPP;
+ }
+
err = ipgre_newlink_encap_setup(dev, data);
if (err)
return err;
@@ -1491,6 +1497,12 @@ static int erspan_changelink(struct net_device *dev, struct nlattr *tb[],
if (!rtnl_dev_link_net_capable(dev, t->net))
return -EPERM;
+ if (data && data[IFLA_GRE_COLLECT_METADATA] && !t->collect_md) {
+ NL_SET_ERR_MSG(extack,
+ "Enabling collect_md on an existing device is not supported");
+ return -EOPNOTSUPP;
+ }
+
err = ipgre_newlink_encap_setup(dev, data);
if (err)
return err;
diff --git a/net/ipv4/ipconfig.c b/net/ipv4/ipconfig.c
index 22a7889876c1..c71776434757 100644
--- a/net/ipv4/ipconfig.c
+++ b/net/ipv4/ipconfig.c
@@ -676,11 +676,40 @@ static const u8 ic_bootp_cookie[4] = { 99, 130, 83, 99 };
#ifdef IPCONFIG_DHCP
+static bool __init
+ic_dhcp_add_option(u8 **options, const u8 *end, u8 type, const void *value,
+ int len)
+{
+ u8 *e = *options;
+
+ /* leave room for the option header and the END marker */
+ if (len > U8_MAX || end - e < len + 3)
+ return false;
+
+ *e++ = type;
+ *e++ = len;
+ memcpy(e, value, len);
+ *options = e + len;
+
+ return true;
+}
+
static void __init
ic_dhcp_init_options(u8 *options, struct ic_device *d)
{
- u8 mt = ((ic_servaddr == NONE)
- ? DHCPDISCOVER : DHCPREQUEST);
+ static const u8 ic_req_params[] = {
+ 1, /* Subnet mask */
+ 3, /* Default gateway */
+ 6, /* DNS server */
+ 12, /* Host name */
+ 15, /* Domain name */
+ 17, /* Boot path */
+ 26, /* MTU */
+ 40, /* NIS domain name */
+ 42, /* NTP servers */
+ };
+ u8 mt = (ic_servaddr == NONE) ? DHCPDISCOVER : DHCPREQUEST;
+ u8 *end = options + sizeof(((struct bootp_pkt *)0)->exten);
u8 *e = options;
int len;
@@ -705,52 +734,25 @@ ic_dhcp_init_options(u8 *options, struct ic_device *d)
e += 4;
}
- /* always? */
- {
- static const u8 ic_req_params[] = {
- 1, /* Subnet mask */
- 3, /* Default gateway */
- 6, /* DNS server */
- 12, /* Host name */
- 15, /* Domain name */
- 17, /* Boot path */
- 26, /* MTU */
- 40, /* NIS domain name */
- 42, /* NTP servers */
- };
-
- *e++ = 55; /* Parameter request list */
- *e++ = sizeof(ic_req_params);
- memcpy(e, ic_req_params, sizeof(ic_req_params));
- e += sizeof(ic_req_params);
-
- if (ic_host_name_set) {
- *e++ = 12; /* host-name */
- len = strlen(utsname()->nodename);
- *e++ = len;
- memcpy(e, utsname()->nodename, len);
- e += len;
- }
- if (*vendor_class_identifier) {
+ *e++ = 55; /* Parameter request list */
+ *e++ = sizeof(ic_req_params);
+ memcpy(e, ic_req_params, sizeof(ic_req_params));
+ e += sizeof(ic_req_params);
+
+ if (ic_host_name_set) {
+ len = strlen(utsname()->nodename);
+ ic_dhcp_add_option(&e, end, 12, utsname()->nodename, len);
+ }
+ if (*vendor_class_identifier) {
+ len = strlen(vendor_class_identifier);
+ if (ic_dhcp_add_option(&e, end, 60, vendor_class_identifier, len))
pr_info("DHCP: sending class identifier \"%s\"\n",
vendor_class_identifier);
- *e++ = 60; /* Class-identifier */
- len = strlen(vendor_class_identifier);
- *e++ = len;
- memcpy(e, vendor_class_identifier, len);
- e += len;
- }
- len = strlen(dhcp_client_identifier + 1);
- /* the minimum length of identifier is 2, include 1 byte type,
- * and can not be larger than the length of options
- */
- if (len >= 1 && len < 312 - (e - options) - 1) {
- *e++ = 61;
- *e++ = len + 1;
- memcpy(e, dhcp_client_identifier, len + 1);
- e += len + 1;
- }
}
+ len = strlen(dhcp_client_identifier + 1);
+ /* the minimum length of identifier is 2, include 1 byte type */
+ if (len >= 1)
+ ic_dhcp_add_option(&e, end, 61, dhcp_client_identifier, len + 1);
*e++ = 255; /* End of the list */
}
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 0cd46896e798..3f9e11a43f35 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -3821,6 +3821,7 @@ void tcp_send_active_reset(struct sock *sk, enum sk_rst_reason reason)
*/
int tcp_send_synack(struct sock *sk)
{
+ struct tcp_sock *tp = tcp_sk(sk);
struct sk_buff *skb;
skb = tcp_rtx_queue_head(sk);
@@ -3838,6 +3839,8 @@ int tcp_send_synack(struct sock *sk)
if (!nskb)
return -ENOMEM;
INIT_LIST_HEAD(&nskb->tcp_tsorted_anchor);
+ if (skb == tp->retransmit_skb_hint)
+ tp->retransmit_skb_hint = nskb;
tcp_highest_sack_replace(sk, skb, nskb);
tcp_rtx_queue_unlink_and_free(skb, sk);
__skb_header_release(nskb);
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index ea7585f6ec32..341786983d42 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -624,14 +624,23 @@ void udp_lib_hash4(struct sock *sk, u16 hash)
struct net *net = sock_net(sk);
struct udp_table *udptable;
- /* Connected udp socket can re-connect to another remote address, which
- * will be handled by rehash. Thus no need to redo hash4 here.
+ udptable = net->ipv4.udp_table;
+ hslot = udp_hashslot(udptable, net, udp_sk(sk)->udp_port_hash);
+
+ /* A connected socket can re-connect to another address. rehash()
+ * relocates it, but only runs when the local address changes, so a
+ * socket bound to a specific address would stay filed under the
+ * previous peer's hash. Move it here.
*/
- if (udp_hashed4(sk))
+ if (udp_hashed4(sk)) {
+ if (udp_sk(sk)->udp_lrpa_hash != hash) {
+ spin_lock_bh(&hslot->lock);
+ udp_rehash4(udptable, sk, hash);
+ spin_unlock_bh(&hslot->lock);
+ }
return;
+ }
- udptable = net->ipv4.udp_table;
- hslot = udp_hashslot(udptable, net, udp_sk(sk)->udp_port_hash);
hslot2 = udp_hashslot2(udptable, udp_sk(sk)->udp_portaddr_hash);
hslot4 = udp_hashslot4(udptable, hash);
udp_sk(sk)->udp_lrpa_hash = hash;
@@ -2220,9 +2229,31 @@ int __udp_disconnect(struct sock *sk, int flags)
}
EXPORT_SYMBOL(__udp_disconnect);
+/* __udp_disconnect() takes a socket out of the 4-tuple hash table only via
+ * ->rehash() or ->unhash(), and neither runs for a socket bound to a
+ * specific address and port. Remove it here, before its peer is cleared.
+ */
+static void udp_unhash4_on_disconnect(struct sock *sk)
+{
+ struct net *net = sock_net(sk);
+ struct udp_table *udptable;
+ struct udp_hslot *hslot;
+
+ if (!udp_hashed4(sk))
+ return;
+
+ udptable = net->ipv4.udp_table;
+ hslot = udp_hashslot(udptable, net, udp_sk(sk)->udp_port_hash);
+
+ spin_lock_bh(&hslot->lock);
+ udp_unhash4(udptable, sk);
+ spin_unlock_bh(&hslot->lock);
+}
+
int udp_disconnect(struct sock *sk, int flags)
{
lock_sock(sk);
+ udp_unhash4_on_disconnect(sk);
__udp_disconnect(sk, flags);
release_sock(sk);
return 0;
@@ -3237,6 +3268,7 @@ int udp_abort(struct sock *sk, int err)
sk->sk_err = err;
sk_error_report(sk);
+ udp_unhash4_on_disconnect(sk);
__udp_disconnect(sk, 0);
out:
diff --git a/net/ipv6/exthdrs_core.c b/net/ipv6/exthdrs_core.c
index 49e31e4ae7b7..ce3724f49e33 100644
--- a/net/ipv6/exthdrs_core.c
+++ b/net/ipv6/exthdrs_core.c
@@ -271,6 +271,9 @@ int ipv6_find_hdr(const struct sk_buff *skb, unsigned int *offset,
hdrlen = ipv6_optlen(hp);
if (!found) {
+ if (skb->len - start < hdrlen)
+ return -EBADMSG;
+
nexthdr = hp->nexthdr;
start += hdrlen;
}
diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
index 75875e4a1ede..9fc2d2677b74 100644
--- a/net/ipv6/ip6_fib.c
+++ b/net/ipv6/ip6_fib.c
@@ -1033,8 +1033,8 @@ static void fib6_purge_rt(struct fib6_info *rt, struct fib6_node *fn,
struct fib6_table *table = rt->fib6_table;
/* Flush all cached dst in exception table */
- rt6_flush_exceptions(rt);
fib6_drop_pcpu_from(rt);
+ rt6_flush_exceptions(rt);
if (rt->nh) {
spin_lock(&rt->nh->lock);
diff --git a/net/ipv6/ip6_gre.c b/net/ipv6/ip6_gre.c
index 81792c940d29..ebd0198458cd 100644
--- a/net/ipv6/ip6_gre.c
+++ b/net/ipv6/ip6_gre.c
@@ -2279,7 +2279,7 @@ static int ip6erspan_changelink(struct net_device *dev, struct nlattr *tb[],
return PTR_ERR(t);
ip6erspan_set_version(data, &p);
- ip6gre_tunnel_unlink_md(ign, t);
+ ip6erspan_tunnel_unlink_md(ign, t);
ip6gre_tunnel_unlink(ign, t);
ip6erspan_tnl_change(t, &p, !tb[IFLA_MTU]);
ip6erspan_tunnel_link_md(ign, t);
diff --git a/net/ipv6/netfilter/ip6t_rpfilter.c b/net/ipv6/netfilter/ip6t_rpfilter.c
index 67c87a88cde4..b5def30c3127 100644
--- a/net/ipv6/netfilter/ip6t_rpfilter.c
+++ b/net/ipv6/netfilter/ip6t_rpfilter.c
@@ -61,7 +61,7 @@ static bool rpfilter_lookup_reverse6(struct net *net, const struct sk_buff *skb,
fl6.flowi6_oif = dev->ifindex;
rt = (void *)ip6_route_lookup(net, &fl6, skb, lookup_flags);
- if (rt->dst.error)
+ if (rt->dst.error || !rt->rt6i_idev)
goto out;
if (rt->rt6i_flags & (RTF_REJECT|RTF_ANYCAST))
diff --git a/net/ipv6/netfilter/ip6t_rt.c b/net/ipv6/netfilter/ip6t_rt.c
index 278b52752f36..2f6261d06a07 100644
--- a/net/ipv6/netfilter/ip6t_rt.c
+++ b/net/ipv6/netfilter/ip6t_rt.c
@@ -96,7 +96,8 @@ static bool rt_mt6(const struct sk_buff *skb, struct xt_action_param *par)
unsigned int i = 0;
for (temp = 0;
- temp < (unsigned int)((hdrlen - 8) / 16);
+ temp < (unsigned int)((hdrlen - 8) / 16) &&
+ i < rtinfo->addrnr;
temp++) {
ap = skb_header_pointer(skb,
ptr
@@ -112,8 +113,6 @@ static bool rt_mt6(const struct sk_buff *skb, struct xt_action_param *par)
if (ipv6_addr_equal(ap, &rtinfo->addrs[i]))
i++;
- if (i == rtinfo->addrnr)
- break;
}
if (i == rtinfo->addrnr)
return ret;
@@ -162,6 +161,12 @@ static int rt_mt6_check(const struct xt_mtchk_param *par)
pr_debug("too many addresses specified\n");
return -EINVAL;
}
+
+ if ((rtinfo->flags & IP6T_RT_FST_MASK) && !rtinfo->addrnr) {
+ pr_info_ratelimited("address list match requested but addrnr is 0\n");
+ return -EINVAL;
+ }
+
if ((rtinfo->flags & (IP6T_RT_RES | IP6T_RT_FST_MASK)) &&
(!(rtinfo->flags & IP6T_RT_TYP) ||
(rtinfo->rt_type != 0) ||
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index 71a38034f5ca..e99b4901091a 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -139,6 +139,7 @@ void rt6_uncached_list_add(struct rt6_info *rt)
{
struct uncached_list *ul = raw_cpu_ptr(&rt6_uncached_list);
+ /* Set once and never cleared: non-NULL marks an uncached route. */
rt->dst.rt_uncached_list = ul;
spin_lock_bh(&ul->lock);
@@ -1725,6 +1726,11 @@ static int rt6_insert_exception(struct rt6_info *nrt,
spin_lock_bh(&rt6_exception_lock);
+ if (f6i->fib6_destroying) {
+ err = -ENOENT;
+ goto out;
+ }
+
bucket = rcu_dereference_protected(nh->rt6i_exception_bucket,
lockdep_is_held(&rt6_exception_lock));
if (!bucket) {
@@ -2716,8 +2722,8 @@ struct dst_entry *ip6_route_output_flags(struct net *net,
rcu_read_lock();
dst = ip6_route_output_flags_noref(net, sk, fl6, flags);
rt6 = dst_rt6_info(dst);
- /* For dst cached in uncached_list, refcnt is already taken. */
- if (list_empty(&rt6->dst.rt_uncached) && !dst_hold_safe(dst)) {
+ /* For an uncached dst, refcnt is already taken. */
+ if (!rt6->dst.rt_uncached_list && !dst_hold_safe(dst)) {
dst = &net->ipv6.ip6_null_entry->dst;
dst_hold(dst);
}
@@ -2826,7 +2832,7 @@ INDIRECT_CALLABLE_SCOPE struct dst_entry *ip6_dst_check(struct dst_entry *dst,
from = rcu_dereference(rt->from);
if (from && (rt->rt6i_flags & RTF_PCPU ||
- unlikely(!list_empty(&rt->dst.rt_uncached))))
+ unlikely(rt->dst.rt_uncached_list)))
dst_ret = rt6_dst_from_check(rt, from, cookie);
else
dst_ret = rt6_check(rt, from, cookie);
diff --git a/net/ipv6/seg6.c b/net/ipv6/seg6.c
index 0246ab6b2c95..80b7d960d770 100644
--- a/net/ipv6/seg6.c
+++ b/net/ipv6/seg6.c
@@ -138,8 +138,8 @@ void seg6_icmp_srh(struct sk_buff *skb, struct inet6_skb_parm *opt)
static struct genl_family seg6_genl_family;
static const struct nla_policy seg6_genl_policy[SEG6_ATTR_MAX + 1] = {
- [SEG6_ATTR_DST] = { .type = NLA_BINARY,
- .len = sizeof(struct in6_addr) },
+ [SEG6_ATTR_DST] =
+ NLA_POLICY_EXACT_LEN(sizeof(struct in6_addr)),
[SEG6_ATTR_DSTLEN] = { .type = NLA_S32, },
[SEG6_ATTR_HMACKEYID] = { .type = NLA_U32, },
[SEG6_ATTR_SECRET] = { .type = NLA_BINARY, },
diff --git a/net/llc/llc_c_ac.c b/net/llc/llc_c_ac.c
index 0779daa8aa8f..a9e0341f4d21 100644
--- a/net/llc/llc_c_ac.c
+++ b/net/llc/llc_c_ac.c
@@ -443,7 +443,7 @@ int llc_conn_ac_resend_i_xxx_x_set_0_or_send_rr(struct sock *sk,
if (likely(!rc))
llc_conn_send_pdu(sk, nskb);
else
- kfree_skb(skb);
+ kfree_skb(nskb);
}
if (rc) {
nr = LLC_I_GET_NR(pdu);
diff --git a/net/llc/llc_s_ac.c b/net/llc/llc_s_ac.c
index 7a0cae9a8111..cc6fb29a9510 100644
--- a/net/llc/llc_s_ac.c
+++ b/net/llc/llc_s_ac.c
@@ -127,6 +127,8 @@ int llc_sap_action_send_xid_r(struct llc_sap *sap, struct sk_buff *skb)
rc = llc_mac_hdr_init(nskb, mac_sa, mac_da);
if (likely(!rc))
rc = dev_queue_xmit(nskb);
+ else
+ kfree_skb(nskb);
out:
return rc;
}
@@ -176,6 +178,8 @@ int llc_sap_action_send_test_r(struct llc_sap *sap, struct sk_buff *skb)
rc = llc_mac_hdr_init(nskb, mac_sa, mac_da);
if (likely(!rc))
rc = dev_queue_xmit(nskb);
+ else
+ kfree_skb(nskb);
out:
return rc;
}
diff --git a/net/llc/llc_sap.c b/net/llc/llc_sap.c
index 6cd03c2ae7d5..5d51ead8a2b3 100644
--- a/net/llc/llc_sap.c
+++ b/net/llc/llc_sap.c
@@ -25,12 +25,12 @@
#include <linux/llc.h>
#include <linux/slab.h>
-static int llc_mac_header_len(unsigned short devtype)
+static int llc_mac_header_len(struct net_device *dev)
{
- switch (devtype) {
+ switch (dev->type) {
case ARPHRD_ETHER:
case ARPHRD_LOOPBACK:
- return sizeof(struct ethhdr);
+ return LL_RESERVED_SPACE(dev);
}
return 0;
}
@@ -51,7 +51,7 @@ struct sk_buff *llc_alloc_frame(struct sock *sk, struct net_device *dev,
int hlen = type == LLC_PDU_TYPE_U ? 3 : 4;
struct sk_buff *skb;
- hlen += llc_mac_header_len(dev->type);
+ hlen += llc_mac_header_len(dev);
skb = alloc_skb(hlen + data_size, GFP_ATOMIC);
if (skb) {
diff --git a/net/mctp/route.c b/net/mctp/route.c
index eb817f1eb5c8..ebccff97ec98 100644
--- a/net/mctp/route.c
+++ b/net/mctp/route.c
@@ -825,7 +825,7 @@ static struct mctp_sk_key *mctp_lookup_prealloc_tag(struct mctp_sock *msk,
spin_lock_irqsave(&mns->keys_lock, flags);
- hlist_for_each_entry(tmp, &mns->keys, hlist) {
+ hlist_for_each_entry(tmp, &msk->keys, sklist) {
if (tmp->net != netid)
continue;
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index e9967d55b8ac..7a1696ac92dc 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -2302,7 +2302,7 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
mptcp_cleanup_rbuf(msk, copied);
err = sk_wait_data(sk, &timeo, last);
if (err < 0) {
- err = copied ? : err;
+ copied = copied ? : err;
goto out_err;
}
}
diff --git a/net/netfilter/ipvs/ip_vs_core.c b/net/netfilter/ipvs/ip_vs_core.c
index 1f62ca2f7330..4e30a903f4bd 100644
--- a/net/netfilter/ipvs/ip_vs_core.c
+++ b/net/netfilter/ipvs/ip_vs_core.c
@@ -1776,6 +1776,12 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related,
/* Ensure the IP header is present in headroom */
if (!pskb_may_pull(skb, hlen_orig))
goto ignore_tunnel;
+ skb_set_transport_header(skb, hlen_orig);
+ /* Before now we may used ihl from skb frag, revalidate it after
+ * copying it into skb head to prevent out-of-bounds access
+ */
+ if (ip_hdr(skb)->ihl * 4 != hlen_orig)
+ goto ignore_tunnel;
IP_VS_DBG(12, "Sending ICMP for %pI4->%pI4: t=%u, c=%u, i=%u\n",
&ip_hdr(skb)->saddr, &ip_hdr(skb)->daddr,
type, code, ntohl(info));
diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c
index 48cf3dad61bd..521159935718 100644
--- a/net/netfilter/nf_conntrack_netlink.c
+++ b/net/netfilter/nf_conntrack_netlink.c
@@ -3395,7 +3395,8 @@ static bool expect_iter_name(struct nf_conntrack_expect *exp, void *data)
struct nf_conntrack_helper *helper;
const char *name = data;
- helper = rcu_dereference(exp->helper);
+ helper = rcu_dereference_protected(exp->helper,
+ lockdep_is_held(&nf_conntrack_expect_lock));
if (!helper)
return false;
diff --git a/net/netfilter/nf_flow_table_offload.c b/net/netfilter/nf_flow_table_offload.c
index 4f346f51d7d7..8e3edc7c37b3 100644
--- a/net/netfilter/nf_flow_table_offload.c
+++ b/net/netfilter/nf_flow_table_offload.c
@@ -993,7 +993,6 @@ static void flow_offload_work_del(struct flow_offload_work *offload)
flow_offload_tuple_del(offload, FLOW_OFFLOAD_DIR_ORIGINAL);
if (test_bit(NF_FLOW_HW_BIDIRECTIONAL, &offload->flow->flags))
flow_offload_tuple_del(offload, FLOW_OFFLOAD_DIR_REPLY);
- set_bit(NF_FLOW_HW_DEAD, &offload->flow->flags);
}
static void flow_offload_tuple_stats(struct flow_offload_work *offload,
@@ -1057,6 +1056,12 @@ static void flow_offload_work_handler(struct work_struct *work)
}
clear_bit(NF_FLOW_HW_PENDING, &offload->flow->flags);
+ if (offload->cmd == FLOW_CLS_DESTROY) {
+ /* Publish after the worker's last flow access. */
+ smp_mb__before_atomic();
+ set_bit(NF_FLOW_HW_DEAD, &offload->flow->flags);
+ }
+
kfree(offload);
}
diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c
index 321a0dd92178..b4c0cae18252 100644
--- a/net/netfilter/nf_tables_api.c
+++ b/net/netfilter/nf_tables_api.c
@@ -6886,11 +6886,14 @@ static int nft_setelem_catchall_insert(const struct net *net,
{
struct nft_set_elem_catchall *catchall;
u8 genmask = nft_genmask_next(net);
+ u64 tstamp = nft_net_tstamp(net);
struct nft_set_ext *ext;
list_for_each_entry(catchall, &set->catchall_list, list) {
ext = nft_set_elem_ext(set, catchall->elem);
- if (nft_set_elem_active(ext, genmask)) {
+ if (nft_set_elem_active(ext, genmask) &&
+ !__nft_set_elem_expired(ext, tstamp) &&
+ !nft_set_elem_is_dead(ext)) {
*priv = catchall->elem;
return -EEXIST;
}
@@ -6983,11 +6986,14 @@ static int nft_setelem_catchall_deactivate(const struct net *net,
struct nft_set_elem *elem)
{
struct nft_set_elem_catchall *catchall;
+ u64 tstamp = nft_net_tstamp(net);
struct nft_set_ext *ext;
list_for_each_entry(catchall, &set->catchall_list, list) {
ext = nft_set_elem_ext(set, catchall->elem);
- if (!nft_is_active_next(net, ext))
+ if (!nft_is_active_next(net, ext) ||
+ __nft_set_elem_expired(ext, tstamp) ||
+ nft_set_elem_is_dead(ext))
continue;
kfree(elem->priv);
@@ -10867,8 +10873,8 @@ static int nf_tables_commit(struct net *net, struct sk_buff *skb)
nft_chain_commit_update(nft_trans_container_chain(trans));
nf_tables_chain_notify(&ctx, NFT_MSG_NEWCHAIN,
&nft_trans_chain_hooks(trans));
- list_splice(&nft_trans_chain_hooks(trans),
- &nft_trans_basechain(trans)->hook_list);
+ list_splice_rcu(&nft_trans_chain_hooks(trans),
+ &nft_trans_basechain(trans)->hook_list);
/* trans destroyed after rcu grace period */
} else {
nft_chain_commit_drop_policy(nft_trans_container_chain(trans));
@@ -10997,8 +11003,8 @@ static int nf_tables_commit(struct net *net, struct sk_buff *skb)
nft_trans_flowtable(trans),
&nft_trans_flowtable_hooks(trans),
NFT_MSG_NEWFLOWTABLE);
- list_splice(&nft_trans_flowtable_hooks(trans),
- &nft_trans_flowtable(trans)->hook_list);
+ list_splice_rcu(&nft_trans_flowtable_hooks(trans),
+ &nft_trans_flowtable(trans)->hook_list);
} else {
nft_clear(net, nft_trans_flowtable(trans));
nf_tables_flowtable_notify(&ctx,
diff --git a/net/netfilter/nfnetlink_queue.c b/net/netfilter/nfnetlink_queue.c
index a65d9a5cb0f0..82187ddd26c1 100644
--- a/net/netfilter/nfnetlink_queue.c
+++ b/net/netfilter/nfnetlink_queue.c
@@ -1340,6 +1340,7 @@ nfqnl_rcv_nl_event(struct notifier_block *this,
if (event == NETLINK_URELEASE && n->protocol == NETLINK_NETFILTER) {
int i;
+ nfnl_lock(NFNL_SUBSYS_QUEUE);
/* destroy all instances for this portid */
spin_lock(&q->instances_lock);
for (i = 0; i < INSTANCE_BUCKETS; i++) {
@@ -1353,6 +1354,7 @@ nfqnl_rcv_nl_event(struct notifier_block *this,
}
}
spin_unlock(&q->instances_lock);
+ nfnl_unlock(NFNL_SUBSYS_QUEUE);
}
return NOTIFY_DONE;
}
@@ -1674,9 +1676,9 @@ static int nfqnl_recv_config(struct sk_buff *skb, const struct nfnl_info *info,
/* Lookup queue under RCU. After peer_portid check (or for new queue
* in BIND case), the queue is owned by the socket sending this message.
- * A socket cannot simultaneously send a message and close, so while
- * processing this CONFIG message, nfqnl_rcv_nl_event() (triggered by
- * socket close) cannot destroy this queue. Safe to use without RCU.
+ * nfqnl_rcv_nl_event() will block on the nfnl subsys mutex that is
+ * held by the caller, so the queue cannot be destroyed in parallel,
+ * even after we drop the RCU read lock.
*/
rcu_read_lock();
queue = instance_lookup(q, queue_num);
diff --git a/net/netfilter/nft_synproxy.c b/net/netfilter/nft_synproxy.c
index a80bdce38546..a6253595a167 100644
--- a/net/netfilter/nft_synproxy.c
+++ b/net/netfilter/nft_synproxy.c
@@ -117,7 +117,8 @@ static void nft_synproxy_do_eval(const struct nft_synproxy *priv,
return;
}
- if (nf_ip_checksum(skb, nft_hook(pkt), thoff, IPPROTO_TCP)) {
+ if (nf_checksum(skb, nft_hook(pkt), thoff, IPPROTO_TCP,
+ nft_pf(pkt))) {
regs->verdict.code = NF_DROP;
return;
}
diff --git a/net/netlink/genetlink.c b/net/netlink/genetlink.c
index 81b4494e2db7..7925a7913bc4 100644
--- a/net/netlink/genetlink.c
+++ b/net/netlink/genetlink.c
@@ -1657,7 +1657,7 @@ static void *ctrl_dumppolicy_prep(struct sk_buff *skb,
}
static int ctrl_dumppolicy_put_op(struct sk_buff *skb,
- struct netlink_callback *cb,
+ struct netlink_callback *cb, u32 cmd,
struct genl_split_ops *doit,
struct genl_split_ops *dumpit)
{
@@ -1678,7 +1678,7 @@ static int ctrl_dumppolicy_put_op(struct sk_buff *skb,
if (!nest_pol)
goto err;
- nest_op = nla_nest_start(skb, doit->cmd);
+ nest_op = nla_nest_start(skb, cmd);
if (!nest_op)
goto err;
@@ -1722,7 +1722,8 @@ static int ctrl_dumppolicy(struct sk_buff *skb, struct netlink_callback *cb)
&doit, &dumpit)))
return -ENOENT;
- if (ctrl_dumppolicy_put_op(skb, cb, &doit, &dumpit))
+ if (ctrl_dumppolicy_put_op(skb, cb, ctx->op,
+ &doit, &dumpit))
return skb->len;
/* done with the per-op policy index list */
@@ -1731,6 +1732,7 @@ static int ctrl_dumppolicy(struct sk_buff *skb, struct netlink_callback *cb)
while (ctx->dump_map) {
if (ctrl_dumppolicy_put_op(skb, cb,
+ ctx->op_iter->cmd,
&ctx->op_iter->doit,
&ctx->op_iter->dumpit))
return skb->len;
diff --git a/net/nfc/core.c b/net/nfc/core.c
index f50e5bab35d8..1d6961c090fb 100644
--- a/net/nfc/core.c
+++ b/net/nfc/core.c
@@ -279,10 +279,10 @@ static struct nfc_target *nfc_find_target(struct nfc_dev *dev, u32 target_idx)
int nfc_dep_link_up(struct nfc_dev *dev, int target_index, u8 comm_mode)
{
- int rc = 0;
- u8 *gb;
- size_t gb_len;
struct nfc_target *target;
+ u8 gb[NFC_MAX_GT_LEN];
+ size_t gb_len = 0;
+ int rc = 0;
pr_debug("dev_name=%s comm %d\n", dev_name(&dev->dev), comm_mode);
@@ -301,7 +301,7 @@ int nfc_dep_link_up(struct nfc_dev *dev, int target_index, u8 comm_mode)
goto error;
}
- gb = nfc_llcp_general_bytes(dev, &gb_len);
+ nfc_get_local_general_bytes(dev, gb, sizeof(gb), &gb_len);
if (gb_len > NFC_MAX_GT_LEN) {
rc = -EINVAL;
goto error;
@@ -644,11 +644,10 @@ int nfc_set_remote_general_bytes(struct nfc_dev *dev, const u8 *gb, u8 gb_len)
}
EXPORT_SYMBOL(nfc_set_remote_general_bytes);
-u8 *nfc_get_local_general_bytes(struct nfc_dev *dev, size_t *gb_len)
+u8 *nfc_get_local_general_bytes(struct nfc_dev *dev, u8 *out_gb,
+ size_t gb_max_len, size_t *gb_len)
{
- pr_debug("dev_name=%s\n", dev_name(&dev->dev));
-
- return nfc_llcp_general_bytes(dev, gb_len);
+ return nfc_llcp_general_bytes(dev, out_gb, gb_max_len, gb_len);
}
EXPORT_SYMBOL(nfc_get_local_general_bytes);
diff --git a/net/nfc/digital_dep.c b/net/nfc/digital_dep.c
index 3982fa084737..968547c306a5 100644
--- a/net/nfc/digital_dep.c
+++ b/net/nfc/digital_dep.c
@@ -1490,14 +1490,14 @@ static int digital_tg_send_atr_res(struct nfc_digital_dev *ddev,
struct digital_atr_req *atr_req)
{
struct digital_atr_res *atr_res;
+ u8 gb[NFC_MAX_GT_LEN];
struct sk_buff *skb;
- u8 *gb, payload_bits;
+ u8 payload_bits;
size_t gb_len;
int rc;
- gb = nfc_get_local_general_bytes(ddev->nfc_dev, &gb_len);
- if (!gb)
- gb_len = 0;
+ nfc_get_local_general_bytes(ddev->nfc_dev, gb, sizeof(gb),
+ &gb_len);
skb = digital_skb_alloc(ddev, sizeof(struct digital_atr_res) + gb_len);
if (!skb)
diff --git a/net/nfc/llcp.h b/net/nfc/llcp.h
index d8345ed57c95..23ae7a0112d3 100644
--- a/net/nfc/llcp.h
+++ b/net/nfc/llcp.h
@@ -91,6 +91,7 @@ struct nfc_llcp_local {
struct hlist_head pending_sdreqs;
struct timer_list sdreq_timer;
struct work_struct sdreq_timeout_work;
+ struct work_struct release_work;
u8 sdreq_next_tid;
/* sockets array */
diff --git a/net/nfc/llcp_commands.c b/net/nfc/llcp_commands.c
index a93bf0b43d50..2d9e0925b1cc 100644
--- a/net/nfc/llcp_commands.c
+++ b/net/nfc/llcp_commands.c
@@ -135,7 +135,7 @@ struct nfc_llcp_sdp_tlv *nfc_llcp_build_sdreq_tlv(u8 tid, const char *uri,
{
struct nfc_llcp_sdp_tlv *sdreq;
- pr_debug("uri: %s, len: %zu\n", uri, uri_len);
+ pr_debug("uri: %.*s, len: %zu\n", (int)uri_len, uri, uri_len);
/* sdreq->tlv_len is u8, takes uri_len, + 3 for header, + 1 for NULL */
if (WARN_ON_ONCE(uri_len > U8_MAX - 4))
diff --git a/net/nfc/llcp_core.c b/net/nfc/llcp_core.c
index 41f8e2f7875a..ee5043da0c8a 100644
--- a/net/nfc/llcp_core.c
+++ b/net/nfc/llcp_core.c
@@ -20,6 +20,8 @@ static LIST_HEAD(llcp_devices);
/* Protects llcp_devices list */
static DEFINE_SPINLOCK(llcp_devices_lock);
+static struct workqueue_struct *llcp_wq;
+
static void nfc_llcp_rx_skb(struct nfc_llcp_local *local, struct sk_buff *skb);
void nfc_llcp_sock_link(struct llcp_sock_list *l, struct sock *sk)
@@ -63,21 +65,33 @@ static void nfc_llcp_socket_purge(struct nfc_llcp_sock *sock)
}
}
+static struct sock *nfc_llcp_sock_list_pop(struct llcp_sock_list *l)
+{
+ struct sock *sk;
+
+ write_lock(&l->lock);
+ sk = sk_head(&l->head);
+ if (sk) {
+ sock_hold(sk);
+ sk_del_node_init(sk);
+ }
+ write_unlock(&l->lock);
+
+ return sk;
+}
+
static void nfc_llcp_socket_release(struct nfc_llcp_local *local, bool device,
int err)
{
struct sock *sk;
- struct hlist_node *tmp;
struct nfc_llcp_sock *llcp_sock;
skb_queue_purge(&local->tx_queue);
- write_lock(&local->sockets.lock);
-
- sk_for_each_safe(sk, tmp, &local->sockets.head) {
+ while ((sk = nfc_llcp_sock_list_pop(&local->sockets))) {
llcp_sock = nfc_llcp_sock(sk);
- bh_lock_sock(sk);
+ lock_sock(sk);
nfc_llcp_socket_purge(llcp_sock);
@@ -91,17 +105,27 @@ static void nfc_llcp_socket_release(struct nfc_llcp_local *local, bool device,
list_for_each_entry_safe(lsk, n,
&llcp_sock->accept_queue,
accept_queue) {
- accept_sk = &lsk->sk;
- bh_lock_sock(accept_sk);
+ bool put_creation = false;
- nfc_llcp_accept_unlink(accept_sk);
-
- if (err)
- accept_sk->sk_err = err;
- accept_sk->sk_state = LLCP_CLOSED;
- accept_sk->sk_state_change(sk);
+ accept_sk = &lsk->sk;
+ lock_sock_nested(accept_sk,
+ SINGLE_DEPTH_NESTING);
+
+ if (nfc_llcp_sock(accept_sk)->parent == sk) {
+ nfc_llcp_accept_unlink(accept_sk);
+ nfc_llcp_sock_unlink(&local->sockets, accept_sk);
+
+ if (err)
+ accept_sk->sk_err = err;
+ accept_sk->sk_state = LLCP_CLOSED;
+ accept_sk->sk_state_change(accept_sk);
+ sock_orphan(accept_sk);
+ put_creation = true;
+ }
- bh_unlock_sock(accept_sk);
+ release_sock(accept_sk);
+ if (put_creation)
+ sock_put(accept_sk); /* creation ref */
}
}
@@ -110,23 +134,18 @@ static void nfc_llcp_socket_release(struct nfc_llcp_local *local, bool device,
sk->sk_state = LLCP_CLOSED;
sk->sk_state_change(sk);
- bh_unlock_sock(sk);
-
- sk_del_node_init(sk);
+ release_sock(sk);
+ sock_put(sk);
}
- write_unlock(&local->sockets.lock);
-
/* If we still have a device, we keep the RAW sockets alive */
if (device == true)
return;
- write_lock(&local->raw_sockets.lock);
-
- sk_for_each_safe(sk, tmp, &local->raw_sockets.head) {
+ while ((sk = nfc_llcp_sock_list_pop(&local->raw_sockets))) {
llcp_sock = nfc_llcp_sock(sk);
- bh_lock_sock(sk);
+ lock_sock(sk);
nfc_llcp_socket_purge(llcp_sock);
@@ -135,26 +154,20 @@ static void nfc_llcp_socket_release(struct nfc_llcp_local *local, bool device,
sk->sk_state = LLCP_CLOSED;
sk->sk_state_change(sk);
- bh_unlock_sock(sk);
-
- sk_del_node_init(sk);
+ release_sock(sk);
+ sock_put(sk);
}
-
- write_unlock(&local->raw_sockets.lock);
}
static struct nfc_llcp_local *nfc_llcp_local_get(struct nfc_llcp_local *local)
{
- /* Since using nfc_llcp_local may result in usage of nfc_dev, whenever
- * we hold a reference to local, we also need to hold a reference to
- * the device to avoid UAF.
- */
- if (!nfc_get_device(local->dev->idx))
+ if (!local)
return NULL;
- kref_get(&local->ref);
+ if (kref_get_unless_zero(&local->ref))
+ return local;
- return local;
+ return NULL;
}
static void local_cleanup(struct nfc_llcp_local *local)
@@ -172,30 +185,34 @@ static void local_cleanup(struct nfc_llcp_local *local)
nfc_llcp_free_sdp_tlv_list(&local->pending_sdreqs);
}
-static void local_release(struct kref *ref)
+static void local_release_work(struct work_struct *work)
{
struct nfc_llcp_local *local;
+ struct nfc_dev *dev;
- local = container_of(ref, struct nfc_llcp_local, ref);
+ local = container_of(work, struct nfc_llcp_local, release_work);
+ dev = local->dev;
local_cleanup(local);
kfree(local);
+ nfc_put_device(dev);
}
-int nfc_llcp_local_put(struct nfc_llcp_local *local)
+static void local_release(struct kref *ref)
{
- struct nfc_dev *dev;
- int ret;
+ struct nfc_llcp_local *local;
- if (local == NULL)
- return 0;
+ local = container_of(ref, struct nfc_llcp_local, ref);
- dev = local->dev;
+ queue_work(llcp_wq, &local->release_work);
+}
- ret = kref_put(&local->ref, local_release);
- nfc_put_device(dev);
+int nfc_llcp_local_put(struct nfc_llcp_local *local)
+{
+ if (!local)
+ return 0;
- return ret;
+ return kref_put(&local->ref, local_release);
}
static struct nfc_llcp_sock *nfc_llcp_sock_get(struct nfc_llcp_local *local,
@@ -341,7 +358,7 @@ static int nfc_llcp_wks_sap(const char *service_name, size_t service_name_len)
{
int sap, num_wks;
- pr_debug("%s\n", service_name);
+ pr_debug("%.*s\n", (int)service_name_len, service_name);
if (service_name == NULL)
return -EINVAL;
@@ -352,7 +369,8 @@ static int nfc_llcp_wks_sap(const char *service_name, size_t service_name_len)
if (wks[sap] == NULL)
continue;
- if (strncmp(wks[sap], service_name, service_name_len) == 0)
+ if (strlen(wks[sap]) == service_name_len &&
+ !strncmp(wks[sap], service_name, service_name_len))
return sap;
}
@@ -635,23 +653,32 @@ static int nfc_llcp_build_gb(struct nfc_llcp_local *local)
return ret;
}
-u8 *nfc_llcp_general_bytes(struct nfc_dev *dev, size_t *general_bytes_len)
+u8 *nfc_llcp_general_bytes(struct nfc_dev *dev, u8 *out_gb, size_t gb_max_len,
+ size_t *general_bytes_len)
{
struct nfc_llcp_local *local;
+ if (!out_gb || !general_bytes_len)
+ return NULL;
+
local = nfc_llcp_find_local(dev);
- if (local == NULL) {
+ if (!local) {
*general_bytes_len = 0;
return NULL;
}
nfc_llcp_build_gb(local);
- *general_bytes_len = local->gb_len;
+ if (local->gb_len) {
+ *general_bytes_len = min_t(size_t, local->gb_len, gb_max_len);
+ memcpy(out_gb, local->gb, *general_bytes_len);
+ } else {
+ *general_bytes_len = 0;
+ }
nfc_llcp_local_put(local);
- return local->gb;
+ return out_gb;
}
int nfc_llcp_set_remote_gb(struct nfc_dev *dev, const u8 *gb, u8 gb_len)
@@ -1074,6 +1101,9 @@ static void nfc_llcp_recv_hdlc(struct nfc_llcp_local *local,
struct sock *sk;
u8 dsap, ssap, ptype, ns, nr;
+ if (!pskb_may_pull(skb, LLCP_HEADER_SIZE + LLCP_SEQUENCE_SIZE))
+ return;
+
ptype = nfc_llcp_ptype(skb);
dsap = nfc_llcp_dsap(skb);
ssap = nfc_llcp_ssap(skb);
@@ -1251,6 +1281,7 @@ static void nfc_llcp_recv_dm(struct nfc_llcp_local *local,
struct nfc_llcp_sock *llcp_sock;
struct sock *sk;
u8 dsap, ssap, reason;
+ bool connecting = false;
dsap = nfc_llcp_dsap(skb);
ssap = nfc_llcp_ssap(skb);
@@ -1262,6 +1293,7 @@ static void nfc_llcp_recv_dm(struct nfc_llcp_local *local,
case LLCP_DM_NOBOUND:
case LLCP_DM_REJ:
llcp_sock = nfc_llcp_connecting_sock_get(local, dsap);
+ connecting = true;
break;
default:
@@ -1276,10 +1308,33 @@ static void nfc_llcp_recv_dm(struct nfc_llcp_local *local,
sk = &llcp_sock->sk;
+ lock_sock(sk);
+
+ /* Check if socket was destroyed whilst waiting for the lock */
+ if (!sk_hashed(sk)) {
+ release_sock(sk);
+ nfc_llcp_sock_put(llcp_sock);
+ return;
+ }
+
+ /*
+ * For DM(NOBOUND)/DM(REJ) the socket is still linked on the
+ * connecting_sockets list. Unlink it here, under the socket lock,
+ * before moving it to LLCP_CLOSED: llcp_sock_release() selects the
+ * list to unlink from by sk_state, so leaving a connecting socket
+ * in the CLOSED state would make it unlink from the wrong list and
+ * corrupt the connecting_sockets list / desync the socket refcount.
+ * This mirrors nfc_llcp_recv_cc().
+ */
+ if (connecting)
+ nfc_llcp_sock_unlink(&local->connecting_sockets, sk);
+
sk->sk_err = ENXIO;
sk->sk_state = LLCP_CLOSED;
sk->sk_state_change(sk);
+ release_sock(sk);
+
nfc_llcp_sock_put(llcp_sock);
}
@@ -1680,6 +1735,7 @@ int nfc_llcp_register_device(struct nfc_dev *ndev)
INIT_WORK(&local->rx_work, nfc_llcp_rx_work);
INIT_WORK(&local->timeout_work, nfc_llcp_timeout_work);
+ INIT_WORK(&local->release_work, local_release_work);
rwlock_init(&local->sockets.lock);
rwlock_init(&local->connecting_sockets.lock);
@@ -1723,10 +1779,23 @@ void nfc_llcp_unregister_device(struct nfc_dev *dev)
int __init nfc_llcp_init(void)
{
- return nfc_llcp_sock_init();
+ int ret;
+
+ llcp_wq = alloc_workqueue("nfc_llcp_wq", WQ_UNBOUND, 0);
+ if (!llcp_wq)
+ return -ENOMEM;
+
+ ret = nfc_llcp_sock_init();
+ if (ret) {
+ destroy_workqueue(llcp_wq);
+ return ret;
+ }
+
+ return 0;
}
void nfc_llcp_exit(void)
{
nfc_llcp_sock_exit();
+ destroy_workqueue(llcp_wq);
}
diff --git a/net/nfc/llcp_sock.c b/net/nfc/llcp_sock.c
index 507447bbceea..0601ddb4304a 100644
--- a/net/nfc/llcp_sock.c
+++ b/net/nfc/llcp_sock.c
@@ -392,11 +392,12 @@ void nfc_llcp_accept_unlink(struct sock *sk)
pr_debug("state %d\n", sk->sk_state);
- list_del_init(&llcp_sock->accept_queue);
- sk_acceptq_removed(llcp_sock->parent);
- llcp_sock->parent = NULL;
-
- sock_put(sk);
+ if (llcp_sock->parent) {
+ list_del_init(&llcp_sock->accept_queue);
+ sk_acceptq_removed(llcp_sock->parent);
+ llcp_sock->parent = NULL;
+ sock_put(sk);
+ }
}
void nfc_llcp_accept_enqueue(struct sock *parent, struct sock *sk)
@@ -423,12 +424,20 @@ struct sock *nfc_llcp_accept_dequeue(struct sock *parent,
list_for_each_entry_safe(lsk, n, &llcp_parent->accept_queue,
accept_queue) {
+ struct nfc_llcp_local *local;
+
sk = &lsk->sk;
- lock_sock(sk);
+ lock_sock_nested(sk, SINGLE_DEPTH_NESTING);
if (sk->sk_state == LLCP_CLOSED) {
- release_sock(sk);
+ local = nfc_llcp_sock(sk)->local;
+
nfc_llcp_accept_unlink(sk);
+ if (local)
+ nfc_llcp_sock_unlink(&local->sockets, sk);
+ sock_orphan(sk);
+ release_sock(sk);
+ sock_put(sk);
continue;
}
@@ -464,7 +473,7 @@ static int llcp_sock_accept(struct socket *sock, struct socket *newsock,
pr_debug("parent %p\n", sk);
- lock_sock_nested(sk, SINGLE_DEPTH_NESTING);
+ lock_sock(sk);
if (sk->sk_state != LLCP_LISTEN) {
ret = -EBADFD;
@@ -490,7 +499,12 @@ static int llcp_sock_accept(struct socket *sock, struct socket *newsock,
release_sock(sk);
timeo = schedule_timeout(timeo);
- lock_sock_nested(sk, SINGLE_DEPTH_NESTING);
+ lock_sock(sk);
+
+ if (sk->sk_state != LLCP_LISTEN) {
+ ret = -EBADFD;
+ break;
+ }
}
__set_current_state(TASK_RUNNING);
remove_wait_queue(sk_sleep(sk), &wait);
@@ -629,13 +643,24 @@ static int llcp_sock_release(struct socket *sock)
list_for_each_entry_safe(lsk, n, &llcp_sock->accept_queue,
accept_queue) {
+ bool put_creation = false;
+
accept_sk = &lsk->sk;
- lock_sock(accept_sk);
+ lock_sock_nested(accept_sk, SINGLE_DEPTH_NESTING);
+
+ if (nfc_llcp_sock(accept_sk)->parent == sk) {
+ nfc_llcp_send_disconnect(lsk);
+ nfc_llcp_accept_unlink(accept_sk);
+ nfc_llcp_sock_unlink(&local->sockets, accept_sk);
- nfc_llcp_send_disconnect(lsk);
- nfc_llcp_accept_unlink(accept_sk);
+ accept_sk->sk_state = LLCP_CLOSED;
+ sock_orphan(accept_sk);
+ put_creation = true;
+ }
release_sock(accept_sk);
+ if (put_creation)
+ sock_put(accept_sk); /* creation ref */
}
}
@@ -734,12 +759,16 @@ static int llcp_sock_connect(struct socket *sock, struct sockaddr *_addr,
llcp_sock->service_name_len = min_t(unsigned int,
addr->service_name_len,
NFC_LLCP_MAX_SERVICE_NAME);
- llcp_sock->service_name = kmemdup(addr->service_name,
- llcp_sock->service_name_len,
- GFP_KERNEL);
- if (!llcp_sock->service_name) {
- ret = -ENOMEM;
- goto sock_llcp_release;
+ if (llcp_sock->service_name_len == 0) {
+ llcp_sock->service_name = NULL;
+ } else {
+ llcp_sock->service_name = kmemdup(addr->service_name,
+ llcp_sock->service_name_len,
+ GFP_KERNEL);
+ if (!llcp_sock->service_name) {
+ ret = -ENOMEM;
+ goto sock_llcp_release;
+ }
}
nfc_llcp_sock_link(&local->connecting_sockets, sk);
diff --git a/net/nfc/nci/core.c b/net/nfc/nci/core.c
index 25ba4cbb00e1..93fa0250a551 100644
--- a/net/nfc/nci/core.c
+++ b/net/nfc/nci/core.c
@@ -780,15 +780,15 @@ static int nci_set_local_general_bytes(struct nfc_dev *nfc_dev)
{
struct nci_dev *ndev = nfc_get_drvdata(nfc_dev);
struct nci_set_config_param param;
+ u8 gb[NFC_MAX_GT_LEN];
int rc;
- param.val = nfc_get_local_general_bytes(nfc_dev, ¶m.len);
- if ((param.val == NULL) || (param.len == 0))
+ nfc_get_local_general_bytes(nfc_dev, gb, sizeof(gb),
+ ¶m.len);
+ if (param.len == 0)
return 0;
- if (param.len > NFC_MAX_GT_LEN)
- return -EINVAL;
-
+ param.val = gb;
param.id = NCI_PN_ATR_REQ_GEN_BYTES;
rc = nci_request(ndev, nci_set_config_req, ¶m,
diff --git a/net/nfc/netlink.c b/net/nfc/netlink.c
index a18e2c503da6..3c599435c99e 100644
--- a/net/nfc/netlink.c
+++ b/net/nfc/netlink.c
@@ -1181,7 +1181,7 @@ static int nfc_genl_llc_sdreq(struct sk_buff *skb, struct genl_info *info)
if (rc != 0) {
rc = -EINVAL;
- goto put_local;
+ goto free_list;
}
if (!sdp_attrs[NFC_SDP_ATTR_URI])
@@ -1200,7 +1200,7 @@ static int nfc_genl_llc_sdreq(struct sk_buff *skb, struct genl_info *info)
sdreq = nfc_llcp_build_sdreq_tlv(tid, uri, uri_len);
if (sdreq == NULL) {
rc = -ENOMEM;
- goto put_local;
+ goto free_list;
}
tlvs_len += sdreq->tlv_len;
@@ -1215,6 +1215,9 @@ static int nfc_genl_llc_sdreq(struct sk_buff *skb, struct genl_info *info)
rc = nfc_llcp_send_snl_sdreq(local, &sdreq_list, tlvs_len);
+free_list:
+ nfc_llcp_free_sdp_tlv_list(&sdreq_list);
+
put_local:
nfc_llcp_local_put(local);
diff --git a/net/nfc/nfc.h b/net/nfc/nfc.h
index 0b1e6466f4fb..82c5dfdad10e 100644
--- a/net/nfc/nfc.h
+++ b/net/nfc/nfc.h
@@ -49,7 +49,8 @@ void nfc_llcp_mac_is_up(struct nfc_dev *dev, u32 target_idx,
int nfc_llcp_register_device(struct nfc_dev *dev);
void nfc_llcp_unregister_device(struct nfc_dev *dev);
int nfc_llcp_set_remote_gb(struct nfc_dev *dev, const u8 *gb, u8 gb_len);
-u8 *nfc_llcp_general_bytes(struct nfc_dev *dev, size_t *general_bytes_len);
+u8 *nfc_llcp_general_bytes(struct nfc_dev *dev, u8 *out_gb, size_t gb_max_len,
+ size_t *general_bytes_len);
int nfc_llcp_data_received(struct nfc_dev *dev, struct sk_buff *skb);
struct nfc_llcp_local *nfc_llcp_find_local(struct nfc_dev *dev);
int nfc_llcp_local_put(struct nfc_llcp_local *local);
diff --git a/net/openvswitch/conntrack.c b/net/openvswitch/conntrack.c
index 89ef350ca91c..04fa06c6f222 100644
--- a/net/openvswitch/conntrack.c
+++ b/net/openvswitch/conntrack.c
@@ -734,6 +734,18 @@ static int __ovs_ct_lookup(struct net *net, struct sw_flow_key *key,
enum ip_conntrack_info ctinfo;
struct nf_conn *ct;
+ /* If the ct entry is not confirmed and shared with some other skb,
+ * e.g., a cloned one, we can't just modify it with the commit as we
+ * must not modify the extension set. Reset.
+ */
+ if (cached && info->commit) {
+ ct = nf_ct_get(skb, &ctinfo);
+ if (ct && !nf_ct_is_confirmed(ct) && nf_ct_shared(ct)) {
+ nf_reset_ct(skb);
+ cached = false;
+ }
+ }
+
if (!cached) {
struct nf_hook_state state = {
.hook = NF_INET_PRE_ROUTING,
@@ -767,8 +779,6 @@ static int __ovs_ct_lookup(struct net *net, struct sw_flow_key *key,
ct = nf_ct_get(skb, &ctinfo);
if (ct) {
- bool add_helper = false;
-
/* Packets starting a new connection must be NATted before the
* helper, so that the helper knows about the NAT. We enforce
* this by delaying both NAT and helper calls for unconfirmed
@@ -800,7 +810,6 @@ static int __ovs_ct_lookup(struct net *net, struct sw_flow_key *key,
GFP_ATOMIC);
if (err)
return err;
- add_helper = true;
/* helper installed, add seqadj if NAT is required */
if (info->nat && !nfct_seqadj(ct)) {
@@ -809,14 +818,14 @@ static int __ovs_ct_lookup(struct net *net, struct sw_flow_key *key,
}
}
- /* Call the helper only if:
- * - nf_conntrack_in() was executed above ("!cached") or a
- * helper was just attached ("add_helper") for a confirmed
- * connection, or
- * - When committing an unconfirmed connection.
+ /* Call the helper only if nf_conntrack_in() was executed
+ * above ("!cached").
+ *
+ * For unconfirmed connections it will be called later during
+ * commit as we need to have all the other extensions allocated
+ * before the call.
*/
- if ((nf_ct_is_confirmed(ct) ? !cached || add_helper :
- info->commit)) {
+ if (nf_ct_is_confirmed(ct) && !cached) {
int err = nf_ct_helper(skb, ct, ctinfo, info->family);
err = verdict_to_errno(err);
@@ -1019,6 +1028,14 @@ static int ovs_ct_commit(struct net *net, struct sw_flow_key *key,
return err;
nf_conn_act_ct_ext_add(skb, ct, ctinfo);
+
+ /* Call the helpers now. We couldn't do this before as
+ * all the extensions must be allocated before the call.
+ */
+ err = nf_ct_helper(skb, ct, ctinfo, info->family);
+ err = verdict_to_errno(err);
+ if (err)
+ return err;
} else if (IS_ENABLED(CONFIG_NF_CONNTRACK_LABELS) &&
labels_nonzero(&info->labels.mask)) {
err = ovs_ct_set_labels(ct, key, &info->labels.value,
diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
index e65f783cc78b..846f24e22cbb 100644
--- a/net/packet/af_packet.c
+++ b/net/packet/af_packet.c
@@ -615,7 +615,7 @@ static int prb_calc_retire_blk_tmo(struct packet_sock *po,
return DEFAULT_PRB_RETIRE_TOV;
div = ecmd.base.speed / 1000;
- mbits = (blk_size_in_bytes * 8) / (1024 * 1024);
+ mbits = (u64)blk_size_in_bytes * 8 / (1024 * 1024);
if (div)
mbits /= div;
@@ -2529,26 +2529,6 @@ static int tpacket_rcv(struct sk_buff *skb, struct net_device *dev,
goto drop_n_restore;
}
-static void tpacket_destruct_skb(struct sk_buff *skb)
-{
- struct packet_sock *po = pkt_sk(skb->sk);
-
- if (likely(po->tx_ring.pg_vec)) {
- void *ph;
- __u32 ts;
-
- ph = skb_zcopy_get_nouarg(skb);
-
- ts = __packet_set_timestamp(po, ph, skb);
- __packet_set_status(po, ph, TP_STATUS_AVAILABLE | ts);
-
- packet_dec_pending(&po->tx_ring);
- complete(&po->skb_completion);
- }
-
- sock_wfree(skb);
-}
-
static int __packet_snd_vnet_parse(struct virtio_net_hdr *vnet_hdr, size_t len)
{
if ((vnet_hdr->flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) &&
@@ -2588,27 +2568,56 @@ static int packet_snd_vnet_parse(struct msghdr *msg, size_t *len,
return 0;
}
+struct tpacket_uarg {
+ struct ubuf_info ubuf;
+ struct packet_sock *po;
+ void *ph;
+};
+
+static void tpacket_ubuf_complete(struct sk_buff *skb, struct ubuf_info *uarg,
+ bool success)
+{
+ struct tpacket_uarg *tu = container_of(uarg, struct tpacket_uarg, ubuf);
+ struct packet_sock *po = tu->po;
+ void *ph = tu->ph;
+ __u32 ts;
+
+ DEBUG_NET_WARN_ON_ONCE(!skb);
+
+ if (!refcount_dec_and_test(&uarg->refcnt))
+ return;
+
+ ts = __packet_set_timestamp(po, ph, skb);
+ __packet_set_status(po, ph, TP_STATUS_AVAILABLE | ts);
+
+ packet_dec_pending(&po->tx_ring);
+ complete(&po->skb_completion);
+
+ kfree(tu);
+ sk_free(&po->sk);
+}
+
+static const struct ubuf_info_ops tpacket_ubuf_ops = {
+ .complete = tpacket_ubuf_complete,
+};
+
static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb,
- void *frame, struct net_device *dev, void *data, int tp_len,
+ struct net_device *dev, void *data, int tp_len,
__be16 proto, unsigned char *addr, int hlen, int copylen,
int hard_header_len,
const struct sockcm_cookie *sockc)
{
- union tpacket_uhdr ph;
int to_write, offset, len, nr_frags, len_max;
struct socket *sock = po->sk.sk_socket;
struct page *page;
int err;
- ph.raw = frame;
-
skb->protocol = proto;
skb->dev = dev;
skb->priority = sockc->priority;
skb->mark = sockc->mark;
skb_set_delivery_type_by_clockid(skb, sockc->transmit_time, po->sk.sk_clockid);
skb_setup_tx_timestamp(skb, sockc);
- skb_zcopy_set_nouarg(skb, ph.raw);
skb_reserve(skb, hlen);
skb_reset_network_header(skb);
@@ -2748,6 +2757,7 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg)
struct virtio_net_hdr vnet_hdr;
bool has_vnet_hdr = false;
struct sockcm_cookie sockc;
+ struct tpacket_uarg *uarg;
__be16 proto;
int err, reserve = 0;
void *ph;
@@ -2875,7 +2885,7 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg)
err = len_sum;
goto out_status;
}
- tp_len = tpacket_fill_skb(po, skb, ph, dev, data, tp_len, proto,
+ tp_len = tpacket_fill_skb(po, skb, dev, data, tp_len, proto,
addr, hlen, copylen, hard_header_len,
&sockc);
if (likely(tp_len >= 0) &&
@@ -2907,7 +2917,24 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg)
virtio_net_hdr_set_proto(skb, &vnet_hdr);
}
- skb->destructor = tpacket_destruct_skb;
+ uarg = kmalloc(sizeof(*uarg), GFP_KERNEL);
+ if (unlikely(!uarg)) {
+ if (likely(len_sum > 0))
+ err = len_sum;
+ else
+ err = -ENOMEM;
+ goto out_status;
+ }
+ uarg->po = po;
+ uarg->ph = ph;
+ uarg->ubuf.ops = &tpacket_ubuf_ops;
+ uarg->ubuf.flags = SKBFL_ZEROCOPY_FRAG;
+ refcount_set(&uarg->ubuf.refcnt, 1);
+
+ /* Hold a sk_wmem_alloc reference until completion */
+ refcount_inc(&po->sk.sk_wmem_alloc);
+ skb_zcopy_init(skb, &uarg->ubuf);
+
__packet_set_status(po, ph, TP_STATUS_SENDING);
packet_inc_pending(&po->tx_ring);
@@ -4486,21 +4513,20 @@ static struct pgv *alloc_pg_vec(struct tpacket_req *req, int order, bool tx_ring
vec->len = block_nr;
pg_vec = vec->pg_vec;
+ if (tx_ring) {
+ vec->deferred = kzalloc_obj(*vec->deferred,
+ GFP_KERNEL | __GFP_NOWARN);
+ if (!vec->deferred)
+ goto out_free_pgvec;
+ vec->deferred->vec = vec;
+ INIT_DELAYED_WORK(&vec->deferred->work,
+ packet_free_pg_vec_work);
+ }
+
for (i = 0; i < block_nr; i++) {
pg_vec[i].buffer = alloc_one_pg_vec_page(order);
if (unlikely(!pg_vec[i].buffer))
goto out_free_pgvec;
-
- if (tx_ring && !vec->deferred &&
- is_vmalloc_addr(pg_vec[i].buffer)) {
- vec->deferred = kzalloc_obj(*vec->deferred,
- GFP_KERNEL | __GFP_NOWARN);
- if (!vec->deferred)
- goto out_free_pgvec;
- vec->deferred->vec = vec;
- INIT_DELAYED_WORK(&vec->deferred->work,
- packet_free_pg_vec_work);
- }
}
out:
diff --git a/net/rds/ib_frmr.c b/net/rds/ib_frmr.c
index bd861191157b..8397aa4a17ac 100644
--- a/net/rds/ib_frmr.c
+++ b/net/rds/ib_frmr.c
@@ -204,19 +204,16 @@ static int rds_ib_map_frmr(struct rds_ib_device *rds_ibdev,
*/
rds_ib_teardown_mr(ibmr);
- ibmr->sg = sg;
- ibmr->sg_len = sg_len;
- ibmr->sg_dma_len = 0;
frmr->sg_byte_len = 0;
- WARN_ON(ibmr->sg_dma_len);
- ibmr->sg_dma_len = ib_dma_map_sg(dev, ibmr->sg, ibmr->sg_len,
+ ibmr->sg_dma_len = ib_dma_map_sg(dev, sg, sg_len,
DMA_BIDIRECTIONAL);
if (unlikely(!ibmr->sg_dma_len)) {
pr_warn("RDS/IB: %s failed!\n", __func__);
return -EBUSY;
}
- frmr->sg_byte_len = 0;
+ ibmr->sg = sg;
+ ibmr->sg_len = sg_len;
frmr->dma_npages = 0;
len = 0;
@@ -264,6 +261,8 @@ static int rds_ib_map_frmr(struct rds_ib_device *rds_ibdev,
ib_dma_unmap_sg(rds_ibdev->dev, ibmr->sg, ibmr->sg_len,
DMA_BIDIRECTIONAL);
ibmr->sg_dma_len = 0;
+ ibmr->sg = NULL;
+ ibmr->sg_len = 0;
return ret;
}
diff --git a/net/rose/rose_in.c b/net/rose/rose_in.c
index 622527f1354f..ac20f318008d 100644
--- a/net/rose/rose_in.c
+++ b/net/rose/rose_in.c
@@ -203,7 +203,18 @@ static int rose_state3_machine(struct sock *sk, struct sk_buff *skb, int framety
break;
default:
- printk(KERN_WARNING "ROSE: unknown %02X in state 3\n", frametype);
+ /*
+ * CALL_ACCEPTED (0x0F) and CLEAR_CONFIRMATION (0x17) may show
+ * up in state 3 as a late or duplicated Call Accepted, or as
+ * crossed clearing / a reused logical channel on a slow AX.25
+ * link. These are harmless protocol races, so drop them
+ * silently. Any other frame type is genuinely unexpected and
+ * is still reported, rate limited to avoid flooding the log.
+ */
+ if (frametype != ROSE_CALL_ACCEPTED &&
+ frametype != ROSE_CLEAR_CONFIRMATION)
+ net_warn_ratelimited("ROSE: unknown %02X in state 3\n",
+ frametype);
break;
}
diff --git a/net/rose/rose_link.c b/net/rose/rose_link.c
index 7746229fdc8c..36b6778d8e54 100644
--- a/net/rose/rose_link.c
+++ b/net/rose/rose_link.c
@@ -263,6 +263,11 @@ void rose_transmit_link(struct sk_buff *skb, struct rose_neigh *neigh)
{
unsigned char *dptr;
+ if (!neigh) {
+ kfree_skb(skb);
+ return;
+ }
+
if (neigh->loopback) {
rose_loopback_queue(skb, neigh);
return;
diff --git a/net/rose/rose_route.c b/net/rose/rose_route.c
index a1e9b05ef6f5..fd196f2e2fd6 100644
--- a/net/rose/rose_route.c
+++ b/net/rose/rose_route.c
@@ -229,8 +229,14 @@ static void rose_remove_neigh(struct rose_neigh *rose_neigh)
{
struct rose_neigh *s;
- timer_delete_sync(&rose_neigh->ftimer);
- timer_delete_sync(&rose_neigh->t0timer);
+ /* Being removed from the list is final teardown for this neighbour:
+ * shut the timers down rather than just deleting them, so a t0timer
+ * self-rearm losing the race with this removal can't bring it back to
+ * life later. See the comment in rose_neigh_put() (include/net/rose.h)
+ * for the full story.
+ */
+ timer_shutdown_sync(&rose_neigh->ftimer);
+ timer_shutdown_sync(&rose_neigh->t0timer);
skb_queue_purge(&rose_neigh->queue);
@@ -851,6 +857,21 @@ void rose_link_device_down(struct net_device *dev)
}
}
+/*
+ * Is this Call Request the very same call as the one an existing route
+ * was built for? Both the loop detection and the reuse of a stale LCI
+ * below rely on this test.
+ */
+static bool rose_same_call(const struct rose_route *rose_route,
+ const rose_address *src_addr,
+ const struct rose_facilities_struct *facilities)
+{
+ return rose_route->rand == facilities->rand &&
+ rosecmp(src_addr, &rose_route->src_addr) == 0 &&
+ ax25cmp(&facilities->dest_call, &rose_route->src_call) == 0 &&
+ ax25cmp(&facilities->source_call, &rose_route->dest_call) == 0;
+}
+
/*
* Route a frame to an appropriate AX.25 connection.
* A NULL ax25_cb indicates an internally generated frame.
@@ -961,6 +982,16 @@ int rose_route_frame(struct sk_buff *skb, ax25_cb *ax25)
goto out;
}
+ memset(&facilities, 0x00, sizeof(struct rose_facilities_struct));
+
+ if (frametype == ROSE_CALL_REQUEST &&
+ !rose_parse_facilities(skb->data + ROSE_CALL_REQ_FACILITIES_OFF,
+ skb->len - ROSE_CALL_REQ_FACILITIES_OFF,
+ &facilities)) {
+ rose_transmit_clear_request(rose_neigh, lci, ROSE_INVALID_FACILITY, 76);
+ goto out;
+ }
+
/*
* Route it to the next in line if we have an entry for it.
*/
@@ -969,7 +1000,21 @@ int rose_route_frame(struct sk_buff *skb, ax25_cb *ax25)
if (rose_route->lci1 == lci &&
rose_route->neigh1 == rose_neigh) {
if (frametype == ROSE_CALL_REQUEST) {
- /* F6FBB - Remove an existing unused route */
+ if (rose_same_call(rose_route, src_addr,
+ &facilities)) {
+ /*
+ * The very same call coming back
+ * to us: a routing loop. Keep
+ * the route, the loop detection
+ * below is what matches on it.
+ */
+ rose_transmit_clear_request(rose_neigh,
+ lci,
+ ROSE_NOT_OBTAINABLE,
+ 120);
+ goto out;
+ }
+ /* Remove an existing unused route */
rose_remove_route(rose_route);
break;
} else if (rose_route->neigh2 != NULL) {
@@ -990,7 +1035,21 @@ int rose_route_frame(struct sk_buff *skb, ax25_cb *ax25)
if (rose_route->lci2 == lci &&
rose_route->neigh2 == rose_neigh) {
if (frametype == ROSE_CALL_REQUEST) {
- /* F6FBB - Remove an existing unused route */
+ if (rose_same_call(rose_route, src_addr,
+ &facilities)) {
+ /*
+ * The very same call coming back
+ * to us: a routing loop. Keep
+ * the route, the loop detection
+ * below is what matches on it.
+ */
+ rose_transmit_clear_request(rose_neigh,
+ lci,
+ ROSE_NOT_OBTAINABLE,
+ 120);
+ goto out;
+ }
+ /* Remove an existing unused route */
rose_remove_route(rose_route);
break;
} else if (rose_route->neigh1 != NULL) {
@@ -1021,24 +1080,12 @@ int rose_route_frame(struct sk_buff *skb, ax25_cb *ax25)
goto out;
}
- memset(&facilities, 0x00, sizeof(struct rose_facilities_struct));
-
- if (!rose_parse_facilities(skb->data + ROSE_CALL_REQ_FACILITIES_OFF,
- skb->len - ROSE_CALL_REQ_FACILITIES_OFF,
- &facilities)) {
- rose_transmit_clear_request(rose_neigh, lci, ROSE_INVALID_FACILITY, 76);
- goto out;
- }
-
/*
* Check for routing loops.
*/
rose_route = rose_route_list;
while (rose_route != NULL) {
- if (rose_route->rand == facilities.rand &&
- rosecmp(src_addr, &rose_route->src_addr) == 0 &&
- ax25cmp(&facilities.dest_call, &rose_route->src_call) == 0 &&
- ax25cmp(&facilities.source_call, &rose_route->dest_call) == 0) {
+ if (rose_same_call(rose_route, src_addr, &facilities)) {
rose_transmit_clear_request(rose_neigh, lci, ROSE_NOT_OBTAINABLE, 120);
goto out;
}
diff --git a/net/sched/act_api.c b/net/sched/act_api.c
index 0914863dbfa2..a0e8f74ca075 100644
--- a/net/sched/act_api.c
+++ b/net/sched/act_api.c
@@ -758,7 +758,7 @@ static int tcf_idr_delete_index(struct tcf_idrinfo *idrinfo, u32 index)
mutex_lock(&idrinfo->lock);
p = idr_find(&idrinfo->action_idr, index);
- if (!p) {
+ if (IS_ERR_OR_NULL(p)) {
mutex_unlock(&idrinfo->lock);
return -ENOENT;
}
diff --git a/net/sched/act_ct.c b/net/sched/act_ct.c
index cbac27ba76af..1adfd61733a1 100644
--- a/net/sched/act_ct.c
+++ b/net/sched/act_ct.c
@@ -430,11 +430,10 @@ static void tcf_ct_flow_table_add(struct tcf_ct_flow_table *ct_ft,
if (test_and_set_bit(IPS_OFFLOAD_BIT, &ct->status))
return;
+ /* NULL if ct is dying (raced flush) or the atomic alloc failed. */
entry = flow_offload_alloc(ct);
- if (!entry) {
- WARN_ON_ONCE(1);
+ if (!entry)
goto err_alloc;
- }
if (tcp) {
ct->proto.tcp.seen[0].flags |= IP_CT_TCP_FLAG_BE_LIBERAL;
@@ -978,11 +977,11 @@ TC_INDIRECT_SCOPE int tcf_ct_act(struct sk_buff *skb, const struct tc_action *a,
struct tcf_result *res)
{
struct net *net = dev_net(skb->dev);
+ bool cached, commit, clear, nat;
enum ip_conntrack_info ctinfo;
struct tcf_ct *c = to_ct(a);
struct nf_conn *tmpl = NULL;
struct nf_hook_state state;
- bool cached, commit, clear;
int nh_ofs, err, retval;
struct tcf_ct_params *p;
bool add_helper = false;
@@ -997,6 +996,7 @@ TC_INDIRECT_SCOPE int tcf_ct_act(struct sk_buff *skb, const struct tc_action *a,
retval = p->action;
commit = p->ct_action & TCA_CT_ACT_COMMIT;
clear = p->ct_action & TCA_CT_ACT_CLEAR;
+ nat = p->ct_action & TCA_CT_ACT_NAT;
tmpl = p->tmpl;
tcf_lastuse_update(&c->tcf_tm);
@@ -1045,6 +1045,19 @@ TC_INDIRECT_SCOPE int tcf_ct_act(struct sk_buff *skb, const struct tc_action *a,
* different zone.
*/
cached = tcf_ct_skb_nfct_cached(net, skb, p);
+
+ /* If the ct entry is not confirmed and shared with some other skb,
+ * e.g., a cloned one, we can't just modify it with a commit or nat
+ * as we must not modify the extension set. Reset.
+ */
+ if (cached && (commit || nat)) {
+ ct = nf_ct_get(skb, &ctinfo);
+ if (ct && !nf_ct_is_confirmed(ct) && nf_ct_shared(ct)) {
+ nf_reset_ct(skb);
+ cached = false;
+ }
+ }
+
if (!cached) {
if (tcf_ct_flow_table_lookup(p, skb, family)) {
skip_add = true;
@@ -1082,25 +1095,31 @@ TC_INDIRECT_SCOPE int tcf_ct_act(struct sk_buff *skb, const struct tc_action *a,
if (err)
goto drop;
add_helper = true;
- if (p->ct_action & TCA_CT_ACT_NAT && !nfct_seqadj(ct)) {
+ if (nat && !nfct_seqadj(ct)) {
if (!nfct_seqadj_ext_add(ct))
goto drop;
}
}
- if (nf_ct_is_confirmed(ct) ? ((!cached && !skip_add) || add_helper) : commit) {
- err = nf_ct_helper(skb, ct, ctinfo, family);
- if (err != NF_ACCEPT)
- goto nf_error;
- }
-
if (commit) {
tcf_ct_act_set_mark(ct, p->mark, p->mark_mask);
tcf_ct_act_set_labels(ct, p->labels, p->labels_mask);
if (!nf_ct_is_confirmed(ct))
nf_conn_act_ct_ext_add(skb, ct, ctinfo);
+ }
+ /* Run helpers for the connection if nf_conntrack_in() was executed
+ * or if we're about to commit. This has to be done after all the
+ * extensions are already added.
+ */
+ if (nf_ct_is_confirmed(ct) ? ((!cached && !skip_add) || add_helper) : commit) {
+ err = nf_ct_helper(skb, ct, ctinfo, family);
+ if (err != NF_ACCEPT)
+ goto nf_error;
+ }
+
+ if (commit) {
/* This will take care of sending queued events
* even if the connection is already confirmed.
*/
diff --git a/net/sched/act_gate.c b/net/sched/act_gate.c
index d09013ae1892..e21f72c0fc45 100644
--- a/net/sched/act_gate.c
+++ b/net/sched/act_gate.c
@@ -681,7 +681,35 @@ static void tcf_gate_stats_update(struct tc_action *a, u64 bytes, u64 packets,
static size_t tcf_gate_get_fill_size(const struct tc_action *act)
{
- return nla_total_size(sizeof(struct tc_gate));
+ struct tcf_gate *gact = to_gate(act);
+ const struct tcf_gate_params *p;
+ struct tcfg_gate_entry *entry;
+ size_t size = nla_total_size(sizeof(struct tc_gate)) /* TCA_GATE_PARMS */
+ + 3 * nla_total_size_64bit(sizeof(u64)) /* TCA_GATE_BASE_TIME
+ * TCA_GATE_CYCLE_TIME
+ * TCA_GATE_CYCLE_TIME_EXT
+ */
+ + nla_total_size(sizeof(s32)) /* TCA_GATE_CLOCKID */
+ + nla_total_size(sizeof(u32)) /* TCA_GATE_FLAGS */
+ + nla_total_size(sizeof(s32)) /* TCA_GATE_PRIORITY */
+ + nla_total_size(0); /* TCA_GATE_ENTRY_LIST */
+ /* TCA_GATE_TM is budgeted by tcf_action_shared_attrs_size() */
+
+ rcu_read_lock();
+ p = rcu_dereference(gact->param);
+ if (p) {
+ list_for_each_entry_rcu(entry, &p->entries, list)
+ /* TCA_GATE_ONE_ENTRY nest and its attributes */
+ size += nla_total_size(0)
+ + nla_total_size(sizeof(u32)) /* TCA_GATE_ENTRY_INDEX */
+ + nla_total_size(0) /* TCA_GATE_ENTRY_GATE */
+ + nla_total_size(sizeof(u32)) /* TCA_GATE_ENTRY_INTERVAL */
+ + nla_total_size(sizeof(s32)) /* TCA_GATE_ENTRY_MAX_OCTETS */
+ + nla_total_size(sizeof(s32)); /* TCA_GATE_ENTRY_IPV */
+ }
+ rcu_read_unlock();
+
+ return size;
}
static void tcf_gate_entry_destructor(void *priv)
diff --git a/net/sched/cls_u32.c b/net/sched/cls_u32.c
index 83a60698cc00..e5e5533f866d 100644
--- a/net/sched/cls_u32.c
+++ b/net/sched/cls_u32.c
@@ -1000,8 +1000,16 @@ static int u32_change(struct net *net, struct sk_buff *in_skb,
return -ENOMEM;
}
} else {
- err = idr_alloc_u32(&tp_c->handle_idr, ht, &handle,
- handle, GFP_KERNEL);
+ /* The IDR is keyed on the mapped id, and that is
+ * what the destroy paths remove. Ask for it here,
+ * so a manual handle colliding with the
+ * auto-allocated id space is rejected (-ENOSPC)
+ * instead of aliasing a future auto id.
+ */
+ u32 id = handle2id(handle);
+
+ err = idr_alloc_u32(&tp_c->handle_idr, ht, &id, id,
+ GFP_KERNEL);
if (err) {
kfree(ht);
return err;
diff --git a/net/sched/em_text.c b/net/sched/em_text.c
index 692e2be1793e..8052c55b2f4d 100644
--- a/net/sched/em_text.c
+++ b/net/sched/em_text.c
@@ -113,7 +113,7 @@ static void em_text_destroy(struct tcf_ematch *m)
static int em_text_dump(struct sk_buff *skb, struct tcf_ematch *m)
{
struct text_match *tm = EM_TEXT_PRIV(m);
- struct tcf_em_text conf;
+ struct tcf_em_text conf = {};
strscpy(conf.algo, tm->config->ops->name);
conf.from_offset = tm->from_offset;
diff --git a/net/sched/sch_hfsc.c b/net/sched/sch_hfsc.c
index 601f99dd989a..df3501dfe01a 100644
--- a/net/sched/sch_hfsc.c
+++ b/net/sched/sch_hfsc.c
@@ -386,6 +386,15 @@ cftree_update(struct hfsc_class *cl)
#define SM_MASK ((1ULL << SM_SHIFT) - 1)
#define ISM_MASK ((1ULL << ISM_SHIFT) - 1)
+/*
+ * Cap on the non-descending hops a classify walk may take before its
+ * filter chain is treated as misconfigured. A flowid binding that was
+ * legal at bind time can become lateral once hfsc_adjust_levels()
+ * raises a class level; a few such hops are legitimate, an unbounded
+ * run means the chain cycles.
+ */
+#define HFSC_CLASSIFY_MAX_DRIFT 8
+
static inline u64
seg_x2y(u64 x, u64 sm)
{
@@ -1133,6 +1142,7 @@ hfsc_classify(struct sk_buff *skb, struct Qdisc *sch, int *qerr)
struct hfsc_class *head, *cl;
struct tcf_result res;
struct tcf_proto *tcf;
+ unsigned int drift;
int result;
if (TC_H_MAJ(skb->priority ^ sch->handle) == 0 &&
@@ -1142,6 +1152,7 @@ hfsc_classify(struct sk_buff *skb, struct Qdisc *sch, int *qerr)
*qerr = NET_XMIT_SUCCESS | __NET_XMIT_BYPASS;
head = &q->root;
+ drift = HFSC_CLASSIFY_MAX_DRIFT;
tcf = rcu_dereference_bh(q->root.filter_list);
while (tcf && (result = tcf_classify(skb, NULL, tcf, &res, false)) >= 0) {
#ifdef CONFIG_NET_CLS_ACT
@@ -1167,6 +1178,17 @@ hfsc_classify(struct sk_buff *skb, struct Qdisc *sch, int *qerr)
if (cl->level == 0)
return cl; /* hit leaf class */
+ /*
+ * flowid binds skip the level check above (res.class is set
+ * at bind time and levels drift after), so a walk can follow
+ * lateral hops without descending; a bounded number of them
+ * is legal, more means the chain cycles.
+ */
+ if (cl->level >= head->level && drift-- == 0) {
+ pr_warn_ratelimited("hfsc: classify hop budget exhausted, dropping packet\n");
+ return NULL;
+ }
+
/* apply inner filter chain */
tcf = rcu_dereference_bh(cl->filter_list);
head = cl;
diff --git a/net/sched/sch_teql.c b/net/sched/sch_teql.c
index eb424b20027e..d3baf17ebe6c 100644
--- a/net/sched/sch_teql.c
+++ b/net/sched/sch_teql.c
@@ -265,14 +265,11 @@ __teql_resolve(struct sk_buff *skb, struct sk_buff *skb_res,
}
if (neigh_event_send(n, skb_res) == 0) {
- int err;
char haddr[MAX_ADDR_LEN];
neigh_ha_snapshot(haddr, n, dev);
- err = dev_hard_header(skb, dev, ntohs(skb_protocol(skb, false)),
- haddr, NULL, skb->len);
-
- if (err < 0)
+ if (dev_hard_header(skb, dev, ntohs(skb_protocol(skb, false)),
+ haddr, NULL, skb->len) < 0)
err = -EINVAL;
} else {
err = (skb_res == NULL) ? -EAGAIN : 1;
diff --git a/net/sctp/associola.c b/net/sctp/associola.c
index febb8a076f80..ca55be94e8d4 100644
--- a/net/sctp/associola.c
+++ b/net/sctp/associola.c
@@ -1285,18 +1285,19 @@ void sctp_assoc_update_retran_path(struct sctp_association *asoc)
/* Manually skip the head element. */
if (&trans->transports == &asoc->peer.transport_addr_list)
continue;
- if (trans->state == SCTP_UNCONFIRMED)
- continue;
- trans_next = sctp_trans_elect_best(trans, trans_next);
- /* Active is good enough for immediate return. */
- if (trans_next->state == SCTP_ACTIVE)
- break;
+ if (trans->state != SCTP_UNCONFIRMED) {
+ trans_next = sctp_trans_elect_best(trans, trans_next);
+ /* Active is good enough for immediate return. */
+ if (trans_next->state == SCTP_ACTIVE)
+ break;
+ }
/* We've reached the end, time to update path. */
if (trans == asoc->peer.retran_path)
break;
}
- asoc->peer.retran_path = trans_next;
+ if (trans_next)
+ asoc->peer.retran_path = trans_next;
pr_debug("%s: association:%p updated new path to addr:%pISpc\n",
__func__, asoc, &asoc->peer.retran_path->ipaddr.sa);
diff --git a/net/sctp/input.c b/net/sctp/input.c
index 864741fae418..9494cfa51106 100644
--- a/net/sctp/input.c
+++ b/net/sctp/input.c
@@ -436,9 +436,10 @@ void sctp_icmp_proto_unreachable(struct sock *sk,
if (timer_pending(&t->proto_unreach_timer))
return;
else {
- if (!mod_timer(&t->proto_unreach_timer,
- jiffies + (HZ/20)))
- sctp_transport_hold(t);
+ sctp_transport_hold(t);
+ if (mod_timer(&t->proto_unreach_timer,
+ jiffies + (HZ / 20)))
+ sctp_transport_put(t);
}
} else {
struct net *net = sock_net(sk);
diff --git a/net/sctp/sm_sideeffect.c b/net/sctp/sm_sideeffect.c
index 0d99b7e8c082..35f540fb15fc 100644
--- a/net/sctp/sm_sideeffect.c
+++ b/net/sctp/sm_sideeffect.c
@@ -244,8 +244,9 @@ void sctp_generate_t3_rtx_event(struct timer_list *t)
pr_debug("%s: sock is busy\n", __func__);
/* Try again later. */
- if (!mod_timer(&transport->T3_rtx_timer, jiffies + (HZ/20)))
- sctp_transport_hold(transport);
+ sctp_transport_hold(transport);
+ if (mod_timer(&transport->T3_rtx_timer, jiffies + (HZ / 20)))
+ sctp_transport_put(transport);
goto out_unlock;
}
@@ -280,8 +281,9 @@ static void sctp_generate_timeout_event(struct sctp_association *asoc,
timeout_type);
/* Try again later. */
- if (!mod_timer(&asoc->timers[timeout_type], jiffies + (HZ/20)))
- sctp_association_hold(asoc);
+ sctp_association_hold(asoc);
+ if (mod_timer(&asoc->timers[timeout_type], jiffies + (HZ / 20)))
+ sctp_association_put(asoc);
goto out_unlock;
}
@@ -378,8 +380,9 @@ void sctp_generate_heartbeat_event(struct timer_list *t)
pr_debug("%s: sock is busy\n", __func__);
/* Try again later. */
- if (!mod_timer(&transport->hb_timer, jiffies + (HZ/20)))
- sctp_transport_hold(transport);
+ sctp_transport_hold(transport);
+ if (mod_timer(&transport->hb_timer, jiffies + (HZ / 20)))
+ sctp_transport_put(transport);
goto out_unlock;
}
@@ -388,8 +391,9 @@ void sctp_generate_heartbeat_event(struct timer_list *t)
timeout = sctp_transport_timeout(transport);
if (elapsed < timeout) {
elapsed = timeout - elapsed;
- if (!mod_timer(&transport->hb_timer, jiffies + elapsed))
- sctp_transport_hold(transport);
+ sctp_transport_hold(transport);
+ if (mod_timer(&transport->hb_timer, jiffies + elapsed))
+ sctp_transport_put(transport);
goto out_unlock;
}
@@ -422,9 +426,10 @@ void sctp_generate_proto_unreach_event(struct timer_list *t)
pr_debug("%s: sock is busy\n", __func__);
/* Try again later. */
- if (!mod_timer(&transport->proto_unreach_timer,
- jiffies + (HZ/20)))
- sctp_transport_hold(transport);
+ sctp_transport_hold(transport);
+ if (mod_timer(&transport->proto_unreach_timer,
+ jiffies + (HZ / 20)))
+ sctp_transport_put(transport);
goto out_unlock;
}
@@ -458,8 +463,9 @@ void sctp_generate_reconf_event(struct timer_list *t)
pr_debug("%s: sock is busy\n", __func__);
/* Try again later. */
- if (!mod_timer(&transport->reconf_timer, jiffies + (HZ / 20)))
- sctp_transport_hold(transport);
+ sctp_transport_hold(transport);
+ if (mod_timer(&transport->reconf_timer, jiffies + (HZ / 20)))
+ sctp_transport_put(transport);
goto out_unlock;
}
@@ -495,8 +501,9 @@ void sctp_generate_probe_event(struct timer_list *t)
pr_debug("%s: sock is busy\n", __func__);
/* Try again later. */
- if (!mod_timer(&transport->probe_timer, jiffies + (HZ / 20)))
- sctp_transport_hold(transport);
+ sctp_transport_hold(transport);
+ if (mod_timer(&transport->probe_timer, jiffies + (HZ / 20)))
+ sctp_transport_put(transport);
goto out_unlock;
}
diff --git a/net/sctp/sm_statefuns.c b/net/sctp/sm_statefuns.c
index 3a8e16b29660..86f26b8b3993 100644
--- a/net/sctp/sm_statefuns.c
+++ b/net/sctp/sm_statefuns.c
@@ -2654,6 +2654,8 @@ static enum sctp_disposition sctp_sf_do_5_2_6_stale(
sctp_add_cmd_sf(commands, SCTP_CMD_REPLY, SCTP_CHUNK(reply));
+ sctp_add_cmd_sf(commands, SCTP_CMD_DISCARD_PACKET, SCTP_NULL());
+
return SCTP_DISPOSITION_CONSUME;
nomem:
diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c
index 34fa809e9504..fff84df00cab 100644
--- a/net/smc/smc_core.c
+++ b/net/smc/smc_core.c
@@ -1835,6 +1835,7 @@ void smcr_port_err(struct smc_ib_device *smcibdev, u8 ibport)
struct smc_link_group *lgr, *n;
int i;
+ spin_lock_bh(&smc_lgr_list.lock);
list_for_each_entry_safe(lgr, n, &smc_lgr_list.list, list) {
if (strncmp(smcibdev->pnetid[ibport - 1], lgr->pnet_id,
SMC_MAX_PNETID_LEN))
@@ -1849,6 +1850,7 @@ void smcr_port_err(struct smc_ib_device *smcibdev, u8 ibport)
smcr_link_down_cond_sched(lnk);
}
}
+ spin_unlock_bh(&smc_lgr_list.lock);
}
static void smc_link_down_work(struct work_struct *work)
diff --git a/net/smc/smc_ib.c b/net/smc/smc_ib.c
index 0052f02756eb..845f15ed06bf 100644
--- a/net/smc/smc_ib.c
+++ b/net/smc/smc_ib.c
@@ -333,6 +333,7 @@ static bool smc_ib_check_link_gid(u8 gid[SMC_GID_SIZE], bool smcrv2,
static void smc_ib_gid_check(struct smc_ib_device *smcibdev, u8 ibport)
{
struct smc_link_group *lgr;
+ bool stale_gid = false;
int i;
spin_lock_bh(&smc_lgr_list.lock);
@@ -348,11 +349,16 @@ static void smc_ib_gid_check(struct smc_ib_device *smcibdev, u8 ibport)
continue;
if (!smc_ib_check_link_gid(lgr->lnk[i].gid,
lgr->smc_version == SMC_V2,
- smcibdev, ibport))
- smcr_port_err(smcibdev, ibport);
+ smcibdev, ibport)) {
+ stale_gid = true;
+ goto out;
+ }
}
}
+out:
spin_unlock_bh(&smc_lgr_list.lock);
+ if (stale_gid)
+ smcr_port_err(smcibdev, ibport);
}
static int smc_ib_remember_port_attr(struct smc_ib_device *smcibdev, u8 ibport)
diff --git a/net/tipc/group.c b/net/tipc/group.c
index 215f2a7d8458..0da7b2713394 100644
--- a/net/tipc/group.c
+++ b/net/tipc/group.c
@@ -797,10 +797,10 @@ void tipc_group_proto_rcv(struct tipc_group *grp, bool *usr_wakeup,
tipc_group_open(m, usr_wakeup);
return;
case GRP_ACK_MSG:
- if (!m)
+ if (!m || !grp->bc_ackers)
return;
acked = msg_grp_bc_acked(hdr);
- if (less_eq(acked, m->bc_acked))
+ if (acked != grp->bc_snd_nxt || m->bc_acked == acked)
return;
m->bc_acked = acked;
if (--grp->bc_ackers)
diff --git a/net/tipc/monitor.c b/net/tipc/monitor.c
index 572b79bf76ce..f47d0dba5e8f 100644
--- a/net/tipc/monitor.c
+++ b/net/tipc/monitor.c
@@ -632,9 +632,10 @@ static void mon_timeout(struct timer_list *t)
{
struct tipc_monitor *mon = timer_container_of(mon, t, timer);
struct tipc_peer *self;
- int best_member_cnt = dom_size(mon->peer_cnt) - 1;
+ int best_member_cnt;
write_lock_bh(&mon->lock);
+ best_member_cnt = dom_size(mon->peer_cnt) - 1;
self = mon->self;
if (self && (best_member_cnt != self->applied)) {
mon_update_local_domain(mon);
diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c
index ea78e2ce754f..ad6c0d61d547 100644
--- a/net/tls/tls_sw.c
+++ b/net/tls/tls_sw.c
@@ -2372,9 +2372,9 @@ ssize_t tls_sw_splice_read(struct socket *sock, loff_t *ppos,
if (copied < 0)
goto splice_requeue;
- if (chunk < rxm->full_len) {
- rxm->offset += len;
- rxm->full_len -= len;
+ if (copied < rxm->full_len) {
+ rxm->offset += copied;
+ rxm->full_len -= copied;
goto splice_requeue;
}
diff --git a/net/xdp/xskmap.c b/net/xdp/xskmap.c
index afa457506274..607f8b7e16d9 100644
--- a/net/xdp/xskmap.c
+++ b/net/xdp/xskmap.c
@@ -124,7 +124,7 @@ static int xsk_map_gen_lookup(struct bpf_map *map, struct bpf_insn *insn_buf)
struct bpf_insn *insn = insn_buf;
*insn++ = BPF_LDX_MEM(BPF_W, ret, index, 0);
- *insn++ = BPF_JMP_IMM(BPF_JGE, ret, map->max_entries, 5);
+ *insn++ = BPF_JMP32_IMM(BPF_JGE, ret, map->max_entries, 5);
*insn++ = BPF_ALU64_IMM(BPF_LSH, ret, ilog2(sizeof(struct xsk_sock *)));
*insn++ = BPF_ALU64_IMM(BPF_ADD, mp, offsetof(struct xsk_map, xsk_map));
*insn++ = BPF_ALU64_REG(BPF_ADD, ret, mp);
diff --git a/rust/kernel/net/phy.rs b/rust/kernel/net/phy.rs
index 1d17bf1ae87d..5963dbbd54af 100644
--- a/rust/kernel/net/phy.rs
+++ b/rust/kernel/net/phy.rs
@@ -123,37 +123,39 @@ pub fn state(&self) -> DeviceState {
/// Gets the current link state.
///
/// It returns true if the link is up.
- #[inline]
pub fn is_link_up(&self) -> bool {
- let phydev = self.0.get().cast_const();
- // SAFETY: By the type invariant of `Device`, `phydev` points to a valid
- // `struct phy_device`, and there is no concurrent write to this field.
- let link = unsafe { bindings::phy_device::link_raw(phydev) };
- link == 1
+ const LINK_IS_UP: u64 = 1;
+ // TODO: the code to access to the bit field will be replaced with automatically
+ // generated code by bindgen when it becomes possible.
+ // SAFETY: The struct invariant ensures that we may access
+ // this field without additional synchronization.
+ let bit_field = unsafe { &(*self.0.get())._bitfield_1 };
+ bit_field.get(15, 1) == LINK_IS_UP
}
/// Gets the current auto-negotiation configuration.
///
/// It returns true if auto-negotiation is enabled.
- #[inline]
pub fn is_autoneg_enabled(&self) -> bool {
- let phydev = self.0.get().cast_const();
- // SAFETY: By the type invariant of `Device`, `phydev` points to a valid
- // `struct phy_device`, and there is no concurrent write to this field.
- let autoneg = unsafe { bindings::phy_device::autoneg_raw(phydev) };
- autoneg == bindings::AUTONEG_ENABLE
+ // TODO: the code to access to the bit field will be replaced with automatically
+ // generated code by bindgen when it becomes possible.
+ // SAFETY: The struct invariant ensures that we may access
+ // this field without additional synchronization.
+ let bit_field = unsafe { &(*self.0.get())._bitfield_1 };
+ bit_field.get(14, 1) == u64::from(bindings::AUTONEG_ENABLE)
}
/// Gets the current auto-negotiation state.
///
/// It returns true if auto-negotiation is completed.
- #[inline]
pub fn is_autoneg_completed(&self) -> bool {
- let phydev = self.0.get().cast_const();
- // SAFETY: By the type invariant of `Device`, `phydev` points to a valid
- // `struct phy_device`, and there is no concurrent write to this field.
- let completed = unsafe { bindings::phy_device::autoneg_complete_raw(phydev) };
- completed == 1
+ const AUTONEG_COMPLETED: u64 = 1;
+ // TODO: the code to access to the bit field will be replaced with automatically
+ // generated code by bindgen when it becomes possible.
+ // SAFETY: The struct invariant ensures that we may access
+ // this field without additional synchronization.
+ let bit_field = unsafe { &(*self.0.get())._bitfield_1 };
+ bit_field.get(16, 1) == AUTONEG_COMPLETED
}
/// Sets the speed of the PHY.
diff --git a/security/ipe/eval.c b/security/ipe/eval.c
index 21439c5be336..4a7be96c84f0 100644
--- a/security/ipe/eval.c
+++ b/security/ipe/eval.c
@@ -134,10 +134,14 @@ static bool evaluate_boot_verified(const struct ipe_eval_ctx *const ctx)
static bool evaluate_dmv_roothash(const struct ipe_eval_ctx *const ctx,
struct ipe_prop *p)
{
- return !!ctx->ipe_bdev &&
- !!ctx->ipe_bdev->root_hash &&
- ipe_digest_eval(p->value,
- ctx->ipe_bdev->root_hash);
+ const struct digest_info *root_hash;
+
+ if (!ctx->ipe_bdev)
+ return false;
+
+ root_hash = rcu_dereference(ctx->ipe_bdev->root_hash);
+
+ return root_hash && ipe_digest_eval(p->value, root_hash);
}
#else
static bool evaluate_dmv_roothash(const struct ipe_eval_ctx *const ctx,
diff --git a/security/ipe/eval.h b/security/ipe/eval.h
index fef65a36468c..b26d1147d070 100644
--- a/security/ipe/eval.h
+++ b/security/ipe/eval.h
@@ -27,7 +27,7 @@ struct ipe_bdev {
#ifdef CONFIG_IPE_PROP_DM_VERITY_SIGNATURE
bool dm_verity_signed;
#endif /* CONFIG_IPE_PROP_DM_VERITY_SIGNATURE */
- struct digest_info *root_hash;
+ struct digest_info __rcu *root_hash;
};
#endif /* CONFIG_IPE_PROP_DM_VERITY */
diff --git a/security/ipe/fs.c b/security/ipe/fs.c
index 0bb9468b8026..84c10eb2423a 100644
--- a/security/ipe/fs.c
+++ b/security/ipe/fs.c
@@ -159,18 +159,16 @@ static ssize_t new_policy(struct file *f, const char __user *data,
}
rc = ipe_new_policyfs_node(p);
- if (rc)
- goto out;
out:
kfree(copy);
if (rc < 0) {
ipe_free_policy(p);
ipe_audit_policy_load(ERR_PTR(rc));
- } else {
- ipe_audit_policy_load(p);
+ return rc;
}
- return (rc < 0) ? rc : len;
+
+ return len;
}
static const struct file_operations np_fops = {
diff --git a/security/ipe/hooks.c b/security/ipe/hooks.c
index d0323b81cd8f..d3092933abe2 100644
--- a/security/ipe/hooks.c
+++ b/security/ipe/hooks.c
@@ -9,6 +9,7 @@
#include <linux/binfmts.h>
#include <linux/mman.h>
#include <linux/blk_types.h>
+#include <linux/rcupdate.h>
#include "ipe.h"
#include "hooks.h"
@@ -204,7 +205,20 @@ void ipe_bdev_free_security(struct block_device *bdev)
{
struct ipe_bdev *blob = ipe_bdev(bdev);
- ipe_digest_free(blob->root_hash);
+ ipe_digest_free(rcu_access_pointer(blob->root_hash));
+}
+
+static void ipe_set_dmverity_roothash(struct ipe_bdev *blob,
+ struct digest_info *info)
+{
+ struct digest_info *old;
+
+ /* Protected by device-mapper's md->suspend_lock */
+ old = rcu_replace_pointer(blob->root_hash, info, true);
+ if (old) {
+ synchronize_rcu();
+ ipe_digest_free(old);
+ }
}
#ifdef CONFIG_IPE_PROP_DM_VERITY_SIGNATURE
@@ -252,8 +266,7 @@ int ipe_bdev_setintegrity(struct block_device *bdev, enum lsm_integrity_type typ
return -EINVAL;
if (!value) {
- ipe_digest_free(blob->root_hash);
- blob->root_hash = NULL;
+ ipe_set_dmverity_roothash(blob, NULL);
return 0;
}
@@ -273,8 +286,7 @@ int ipe_bdev_setintegrity(struct block_device *bdev, enum lsm_integrity_type typ
info->digest_len = digest->digest_len;
- ipe_digest_free(blob->root_hash);
- blob->root_hash = info;
+ ipe_set_dmverity_roothash(blob, info);
return 0;
err:
diff --git a/security/ipe/policy_fs.c b/security/ipe/policy_fs.c
index 9d92d8a14b13..a7aeb57483c6 100644
--- a/security/ipe/policy_fs.c
+++ b/security/ipe/policy_fs.c
@@ -481,6 +481,9 @@ int ipe_new_policyfs_node(struct ipe_policy *p)
inode_lock(root);
p->policyfs = policyfs;
root->i_private = p;
+ /* Only audit signed policies from userspace */
+ if (p->pkcs7)
+ ipe_audit_policy_load(p);
inode_unlock(root);
return 0;
diff --git a/sound/hda/codecs/realtek/alc269.c b/sound/hda/codecs/realtek/alc269.c
index aab2456bacaf..681cdb00a11d 100644
--- a/sound/hda/codecs/realtek/alc269.c
+++ b/sound/hda/codecs/realtek/alc269.c
@@ -3986,6 +3986,7 @@ enum {
ALC245_FIXUP_CLEVO_NOISY_MIC,
ALC269_FIXUP_VAIO_VJFH52_MIC_NO_PRESENCE,
ALC233_FIXUP_MEDION_MTL_SPK,
+ ALC269_FIXUP_STARLABS_LIMIT_INT_MIC_BOOST,
ALC233_FIXUP_STARLABS_STARFIGHTER,
ALC294_FIXUP_BASS_SPEAKER_15,
ALC283_FIXUP_DELL_HP_RESUME,
@@ -6467,9 +6468,15 @@ static const struct hda_fixup alc269_fixups[] = {
{ }
},
},
+ [ALC269_FIXUP_STARLABS_LIMIT_INT_MIC_BOOST] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc269_fixup_limit_int_mic_boost,
+ },
[ALC233_FIXUP_STARLABS_STARFIGHTER] = {
.type = HDA_FIXUP_FUNC,
.v.func = alc233_fixup_starlabs_starfighter,
+ .chained = true,
+ .chain_id = ALC269_FIXUP_STARLABS_LIMIT_INT_MIC_BOOST,
},
[ALC294_FIXUP_BASS_SPEAKER_15] = {
.type = HDA_FIXUP_FUNC,
@@ -7619,6 +7626,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1f66, 0x0105, "Ayaneo Portable Game Player", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x2014, 0x800a, "Positivo ARN50", ALC269_FIXUP_LIMIT_INT_MIC_BOOST),
SND_PCI_QUIRK(0x2039, 0x0001, "Inspur S14-G1", ALC295_FIXUP_CHROME_BOOK),
+ SND_PCI_QUIRK(0x2145, 0x0001, "Star Labs StarFighter", ALC233_FIXUP_STARLABS_STARFIGHTER),
SND_PCI_QUIRK(0x2782, 0x0214, "VAIO VJFE-CL", ALC269_FIXUP_LIMIT_INT_MIC_BOOST),
SND_PCI_QUIRK(0x2782, 0x0228, "Infinix ZERO BOOK 13", ALC269VB_FIXUP_INFINIX_ZERO_BOOK_13),
SND_PCI_QUIRK(0x2782, 0x0232, "CHUWI CoreBook XPro", ALC269VB_FIXUP_CHUWI_COREBOOK_XPRO),
@@ -7699,6 +7707,7 @@ static const struct hda_quirk alc269_fixup_vendor_tbl[] = {
SND_PCI_QUIRK_VENDOR(0x104d, "Sony VAIO", ALC269_FIXUP_SONY_VAIO),
SND_PCI_QUIRK_VENDOR(0x17aa, "Lenovo XPAD", ALC269_FIXUP_LENOVO_XPAD_ACPI),
SND_PCI_QUIRK_VENDOR(0x19e5, "Huawei Matebook", ALC255_FIXUP_MIC_MUTE_LED),
+ SND_PCI_QUIRK_VENDOR(0x2145, "Star Labs", ALC269_FIXUP_STARLABS_LIMIT_INT_MIC_BOOST),
{}
};
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 94ee5f52b866..b21d2f842579 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -6051,6 +6051,13 @@ bpf_object__relocate_core(struct bpf_object *obj, const char *targ_btf_path)
return -EINVAL;
insn = &prog->insns[insn_idx];
+ if (is_ldimm64_insn(insn) && (size_t)insn_idx + 1 >= prog->insns_cnt) {
+ pr_warn("prog '%s': relo #%d: insn #%d (LDIMM64) is truncated\n",
+ prog->name, i, insn_idx);
+ err = -EINVAL;
+ goto out;
+ }
+
err = record_relo_core(prog, rec, insn_idx);
if (err) {
pr_warn("prog '%s': relo #%d: failed to record relocation: %s\n",
diff --git a/tools/lib/bpf/relo_core.c b/tools/lib/bpf/relo_core.c
index 0ccc8f548cba..a6a88e1cad27 100644
--- a/tools/lib/bpf/relo_core.c
+++ b/tools/lib/bpf/relo_core.c
@@ -980,23 +980,30 @@ static int bpf_core_calc_relo(const char *prog_name,
}
/*
- * Turn instruction for which CO_RE relocation failed into invalid one with
+ * Turn instruction for which CO-RE relocation failed into invalid one with
* distinct signature.
*/
-static void bpf_core_poison_insn(const char *prog_name, int relo_idx,
- int insn_idx, struct bpf_insn *insn)
+static int bpf_core_poison_insn(const char *prog_name, int relo_idx,
+ struct bpf_insn *insn, int insn_idx)
{
- pr_debug("prog '%s': relo #%d: substituting insn #%d w/ invalid insn\n",
- prog_name, relo_idx, insn_idx);
- insn->code = BPF_JMP | BPF_CALL;
- insn->dst_reg = 0;
- insn->src_reg = 0;
- insn->off = 0;
- /* if this instruction is reachable (not a dead code),
- * verifier will complain with the following message:
- * invalid func unknown#195896080
- */
- insn->imm = 195896080; /* => 0xbad2310 => "bad relo" */
+ int insn_cnt = is_ldimm64_insn(insn) ? 2 : 1;
+ int i;
+
+ for (i = 0; i < insn_cnt; i++) {
+ pr_debug("prog '%s': relo #%d: substituting insn #%d w/ invalid insn\n",
+ prog_name, relo_idx, insn_idx + i);
+ insn[i].code = BPF_JMP | BPF_CALL;
+ insn[i].dst_reg = 0;
+ insn[i].src_reg = 0;
+ insn[i].off = 0;
+ /*
+ * If this instruction is reachable (not dead code), the verifier
+ * will complain with "invalid func unknown#195896080".
+ */
+ insn[i].imm = 195896080; /* => 0xbad2310 => "bad relo" */
+ }
+
+ return 0;
}
static int insn_bpf_size_to_bytes(struct bpf_insn *insn)
@@ -1047,17 +1054,6 @@ int bpf_core_patch_insn(const char *prog_name, struct bpf_insn *insn,
class = BPF_CLASS(insn->code);
- if (res->poison) {
-poison:
- /* poison second part of ldimm64 to avoid confusing error from
- * verifier about "unknown opcode 00"
- */
- if (is_ldimm64_insn(insn))
- bpf_core_poison_insn(prog_name, relo_idx, insn_idx + 1, insn + 1);
- bpf_core_poison_insn(prog_name, relo_idx, insn_idx, insn);
- return 0;
- }
-
orig_val = res->orig_val;
new_val = res->new_val;
@@ -1065,7 +1061,9 @@ int bpf_core_patch_insn(const char *prog_name, struct bpf_insn *insn,
case BPF_ALU:
case BPF_ALU64:
if (BPF_SRC(insn->code) != BPF_K)
- return -EINVAL;
+ goto bad_insn;
+ if (res->poison)
+ return bpf_core_poison_insn(prog_name, relo_idx, insn, insn_idx);
if (res->validate && insn->imm != orig_val) {
pr_warn("prog '%s': relo #%d: unexpected insn #%d (ALU/ALU64) value: got %u, exp %llu -> %llu\n",
prog_name, relo_idx,
@@ -1082,6 +1080,8 @@ int bpf_core_patch_insn(const char *prog_name, struct bpf_insn *insn,
case BPF_LDX:
case BPF_ST:
case BPF_STX:
+ if (res->poison)
+ return bpf_core_poison_insn(prog_name, relo_idx, insn, insn_idx);
if (res->validate && insn->off != orig_val) {
pr_warn("prog '%s': relo #%d: unexpected insn #%d (LDX/ST/STX) value: got %u, exp %llu -> %llu\n",
prog_name, relo_idx, insn_idx, insn->off, (unsigned long long)orig_val,
@@ -1097,7 +1097,7 @@ int bpf_core_patch_insn(const char *prog_name, struct bpf_insn *insn,
pr_warn("prog '%s': relo #%d: insn #%d (LDX/ST/STX) accesses field incorrectly. "
"Make sure you are accessing pointers, unsigned integers, or fields of matching type and size.\n",
prog_name, relo_idx, insn_idx);
- goto poison;
+ return bpf_core_poison_insn(prog_name, relo_idx, insn, insn_idx);
}
orig_val = insn->off;
@@ -1140,6 +1140,9 @@ int bpf_core_patch_insn(const char *prog_name, struct bpf_insn *insn,
return -EINVAL;
}
+ if (res->poison)
+ return bpf_core_poison_insn(prog_name, relo_idx, insn, insn_idx);
+
imm = (__u32)insn[0].imm | ((__u64)insn[1].imm << 32);
if (res->validate && imm != orig_val) {
pr_warn("prog '%s': relo #%d: unexpected insn #%d (LDIMM64) value: got %llu, exp %llu -> %llu\n",
@@ -1157,6 +1160,7 @@ int bpf_core_patch_insn(const char *prog_name, struct bpf_insn *insn,
break;
}
default:
+bad_insn:
pr_warn("prog '%s': relo #%d: trying to relocate unrecognized insn #%d, code:0x%x, src:0x%x, dst:0x%x, off:0x%x, imm:0x%x\n",
prog_name, relo_idx, insn_idx, insn->code,
insn->src_reg, insn->dst_reg, insn->off, insn->imm);
diff --git a/tools/testing/selftests/bpf/prog_tests/linked_list.c b/tools/testing/selftests/bpf/prog_tests/linked_list.c
index 14c5a7ef0e87..6c3942e6cfaf 100644
--- a/tools/testing/selftests/bpf/prog_tests/linked_list.c
+++ b/tools/testing/selftests/bpf/prog_tests/linked_list.c
@@ -712,7 +712,7 @@ static void test_btf(void)
break;
err = btf__load_into_kernel(btf);
- ASSERT_EQ(err, -ELOOP, "check btf");
+ ASSERT_EQ(err, 0, "check btf");
btf__free(btf);
break;
}
@@ -771,7 +771,7 @@ static void test_btf(void)
break;
err = btf__load_into_kernel(btf);
- ASSERT_EQ(err, -ELOOP, "check btf");
+ ASSERT_EQ(err, 0, "check btf");
btf__free(btf);
break;
}
diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
index 592e533cd4d7..6f7e387f68e5 100755
--- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
+++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
@@ -297,6 +297,8 @@ TEST_MATRIX=(
" C0-4:X2-4:S+ C1-4:X2-4:S+:P2 C2-4:X4:P1 \
. . . X1 . 0 A1:0-1|A2:2-4|A3:2-4 \
A1:P0|A2:P2|A3:P-1 2-4"
+ " CX1-3:P1 CX1-3 CX1-3 . . . P1 . 0 A1:1-3|A2:1-3|A3:1-3 \
+ A1:P1|A2:P0|A3:P-1"
# Remote partition offline tests
" C0-3:S+ C1-3:S+ C2-3 . X2-3 X2-3 X2-3:P2:O2=0 . 0 A1:0-1|A2:1|A3:3 A1:P0|A3:P2 2-3"
diff --git a/tools/testing/selftests/cgroup/test_memcontrol.c b/tools/testing/selftests/cgroup/test_memcontrol.c
index 1f64db1afa21..e383fad6e5bb 100644
--- a/tools/testing/selftests/cgroup/test_memcontrol.c
+++ b/tools/testing/selftests/cgroup/test_memcontrol.c
@@ -26,7 +26,7 @@ static bool has_recursiveprot;
int get_temp_fd(void)
{
- return open(".", O_TMPFILE | O_RDWR | O_EXCL);
+ return open(".", O_TMPFILE | O_RDWR | O_EXCL, 0600);
}
int alloc_pagecache(int fd, size_t size)
diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc
index bc9514428dba..07b1177c1634 100644
--- a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc
+++ b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_non_uniq_symbol.tc
@@ -6,7 +6,7 @@
SYMBOL='name_show'
# We skip this test on kernel where SYMBOL is unique or does not exist.
-if [ "$(grep -c -E "[[:alnum:]]+ t ${SYMBOL}" /proc/kallsyms)" -le '1' ]; then
+if [ "$(grep -c -E "[[:alnum:]]+ t ${SYMBOL}$" /proc/kallsyms)" -le '1' ]; then
exit_unsupported
fi
diff --git a/tools/testing/selftests/nci/nci_dev.c b/tools/testing/selftests/nci/nci_dev.c
index 6dec59d64083..ae9c2216860b 100644
--- a/tools/testing/selftests/nci/nci_dev.c
+++ b/tools/testing/selftests/nci/nci_dev.c
@@ -8,6 +8,7 @@
#include <stdlib.h>
#include <errno.h>
+#include <stdint.h>
#include <string.h>
#include <sys/ioctl.h>
#include <fcntl.h>
@@ -87,6 +88,16 @@ struct msgtemplate {
char buf[MAX_MSG_SIZE];
};
+static int join_thread_status(pthread_t thread)
+{
+ void *thread_ret = NULL;
+
+ if (pthread_join(thread, &thread_ret))
+ return -1;
+
+ return (int)(intptr_t)thread_ret;
+}
+
static int create_nl_socket(void)
{
int fd;
@@ -182,7 +193,7 @@ static int get_family_id(int sd, __u32 pid, __u32 *event_group)
} ans;
struct nlattr *na;
int resp_len;
- __u16 id;
+ __u16 id = 0;
int len;
int rc;
@@ -438,13 +449,13 @@ FIXTURE_SETUP(NCI)
else
rc = pthread_create(&thread_t, NULL, virtual_dev_open,
(void *)&self->virtual_nci_fd);
- ASSERT_GT(rc, -1);
+ ASSERT_EQ(rc, 0);
rc = send_cmd_with_idx(self->sd, self->fid, self->pid,
NFC_CMD_DEV_UP, self->dev_idex);
EXPECT_EQ(rc, 0);
- pthread_join(thread_t, (void **)&status);
+ status = join_thread_status(thread_t);
ASSERT_EQ(status, 0);
self->open_state = true;
}
@@ -509,12 +520,12 @@ FIXTURE_TEARDOWN(NCI)
rc = pthread_create(&thread_t, NULL, virtual_deinit,
(void *)&self->virtual_nci_fd);
- ASSERT_GT(rc, -1);
+ ASSERT_EQ(rc, 0);
rc = send_cmd_with_idx(self->sd, self->fid, self->pid,
NFC_CMD_DEV_DOWN, self->dev_idex);
EXPECT_EQ(rc, 0);
- pthread_join(thread_t, (void **)&status);
+ status = join_thread_status(thread_t);
ASSERT_EQ(status, 0);
}
@@ -585,12 +596,11 @@ int start_polling(int dev_idx, int proto, int virtual_fd, int sd, int fid, int p
void *nla_start_poll_data[2] = {&dev_idx, &proto};
int nla_start_poll_len[2] = {4, 4};
pthread_t thread_t;
- int status;
int rc;
rc = pthread_create(&thread_t, NULL, virtual_poll_start,
(void *)&virtual_fd);
- if (rc < 0)
+ if (rc)
return rc;
rc = send_cmd_mt_nla(sd, fid, pid, NFC_CMD_START_POLL, 2, nla_start_poll_type,
@@ -598,19 +608,17 @@ int start_polling(int dev_idx, int proto, int virtual_fd, int sd, int fid, int p
if (rc != 0)
return rc;
- pthread_join(thread_t, (void **)&status);
- return status;
+ return join_thread_status(thread_t);
}
int stop_polling(int dev_idx, int virtual_fd, int sd, int fid, int pid)
{
pthread_t thread_t;
- int status;
int rc;
rc = pthread_create(&thread_t, NULL, virtual_poll_stop,
(void *)&virtual_fd);
- if (rc < 0)
+ if (rc)
return rc;
rc = send_cmd_with_idx(sd, fid, pid,
@@ -618,8 +626,7 @@ int stop_polling(int dev_idx, int virtual_fd, int sd, int fid, int pid)
if (rc != 0)
return rc;
- pthread_join(thread_t, (void **)&status);
- return status;
+ return join_thread_status(thread_t);
}
TEST_F(NCI, start_poll)
@@ -830,10 +837,14 @@ int disconnect_tag(int nfc_sock, int virtual_fd)
status = pthread_create(&thread_t, NULL, virtual_deactivate_proc,
(void *)&virtual_fd);
+ if (status)
+ return status;
close(nfc_sock);
- pthread_join(thread_t, (void **)&status);
- return status;
+ if (status)
+ return -1;
+
+ return join_thread_status(thread_t);
}
TEST_F(NCI, t4t_tag_read)
@@ -874,13 +885,13 @@ TEST_F(NCI, deinit)
else
rc = pthread_create(&thread_t, NULL, virtual_deinit,
(void *)&self->virtual_nci_fd);
- ASSERT_GT(rc, -1);
+ ASSERT_EQ(rc, 0);
rc = send_cmd_with_idx(self->sd, self->fid, self->pid,
NFC_CMD_DEV_DOWN, self->dev_idex);
EXPECT_EQ(rc, 0);
- pthread_join(thread_t, (void **)&status);
+ status = join_thread_status(thread_t);
self->open_state = 0;
ASSERT_EQ(status, 0);
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index c16115a06b55..ba09b1dabc4b 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -1132,7 +1132,7 @@ static struct kvm *kvm_create_vm(unsigned long type, const char *fdname)
rcuwait_init(&kvm->mn_memslots_update_rcuwait);
xa_init(&kvm->vcpu_array);
#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES
- xa_init(&kvm->mem_attr_array);
+ xa_init_flags(&kvm->mem_attr_array, XA_FLAGS_ACCOUNT);
#endif
INIT_LIST_HEAD(&kvm->gpc_list);
@@ -2453,14 +2453,36 @@ bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end,
return (kvm_get_memory_attributes(kvm, start) & mask) == attrs;
guard(rcu)();
- if (!attrs)
- return !xas_find(&xas, end - 1);
+ /*
+ * Lookup the entry for each index instead of iterating over the xarray
+ * as KVM deletes/nullifies entries to represent "no attributes", and
+ * the xas index is effectively invalid when no entry is found. I.e.
+ * matching non-zero attributes for *every* entry effectively requires
+ * a manually lookup for each index.
+ *
+ * Skip pre-allocated, reserved entries, or restart the lookup if the
+ * xarray was concurrently modified, via xas_retry() ("retry" means the
+ * entry holds an internal xarray value, i.e. is either invalid or NULL
+ * from the caller's perspective).
+ *
+ * Use xas_next() when looking for non-zero attributes to optimize for
+ * the case where the start of the range (or the entire range) doesn't
+ * have any attributes, as xas_next() returns literally the next entry,
+ * whereas xas_next_entry() returns the next non-NULL entry (bounded by
+ * a maximum index).
+ */
for (index = start; index < end; index++) {
do {
- entry = xas_next(&xas);
+ entry = attrs ? xas_next(&xas) :
+ xas_next_entry(&xas, end - 1);
} while (xas_retry(&xas, entry));
+ if (!entry)
+ return !attrs;
+
+ WARN_ON_ONCE(!xa_to_value(entry));
+
if (xas.xa_index != index ||
(xa_to_value(entry) & mask) != attrs)
return false;
^ permalink raw reply [flat|nested] 2+ messages in thread