* Re: Linux 7.3.5
2026-11-17 10:45 Linux 7.3.5 Greg Kroah-Hartman
@ 2026-11-17 10:45 ` Greg Kroah-Hartman
0 siblings, 0 replies; 2+ messages in thread
From: Greg Kroah-Hartman @ 2026-11-17 10:45 UTC (permalink / raw)
To: linux-kernel, akpm, torvalds, stable; +Cc: lwn, jslaby, Greg Kroah-Hartman
diff --git a/Documentation/ABI/testing/sysfs-fs-f2fs b/Documentation/ABI/testing/sysfs-fs-f2fs
index 0cebc89799dd..4f3915948176 100644
--- a/Documentation/ABI/testing/sysfs-fs-f2fs
+++ b/Documentation/ABI/testing/sysfs-fs-f2fs
@@ -271,7 +271,7 @@ Description: Shows all enabled kernel features.
inode_crtime, lost_found, verity, sb_checksum,
casefold, readonly, compression, test_dummy_encryption_v2,
atomic_write, pin_file, encrypted_casefold, linear_lookup,
- fserror.
+ fserror, resizable_tail.
What: /sys/fs/f2fs/<disk>/inject_rate
Date: May 2016
diff --git a/Documentation/admin-guide/pm/intel_idle.rst b/Documentation/admin-guide/pm/intel_idle.rst
index 996e5bd83c42..d1a6251790b8 100644
--- a/Documentation/admin-guide/pm/intel_idle.rst
+++ b/Documentation/admin-guide/pm/intel_idle.rst
@@ -145,11 +145,13 @@ for the other idle state types (C2 and C3) the target residency value is 3 times
the exit latency (again, that is because it reflects the target residency to
exit latency ratio in the majority of cases for the processor models recognized
by ``intel_idle``). The name of the idle state (to be returned by the ``name``
-idle state attribute in ``sysfs``) is either "Cx_LPI" (if it comes from ``_LPI``
-processing) or "Cx_ACPI", where x is the index of that idle state in the final
-list (note that the minimum value of x is 1, because 0 is reserved for the
-"polling" state), and its target residency is based on the exit latency value.
-All of the idle states in the final list are enabled by default in this case.
+idle state attribute in ``sysfs``) either is equal to the state name value that
+comes from ``_LPI`` processing (with the possible fallback to "Cx_LPI" if the
+state name is not provided) or it is "Cx_ACPI", where x is the index of that
+idle state in the final list (note that the minimum value of x is 1, because 0
+is reserved for the "polling" state), and its target residency is based on the
+exit latency value. All of the idle states in the final list are enabled by
+default in this case.
.. _intel-idle-initialization:
diff --git a/Documentation/arch/arm64/silicon-errata.rst b/Documentation/arch/arm64/silicon-errata.rst
index 99a1eb4b833d..95faec78b7f7 100644
--- a/Documentation/arch/arm64/silicon-errata.rst
+++ b/Documentation/arch/arm64/silicon-errata.rst
@@ -273,7 +273,8 @@ stable kernels.
+----------------+-----------------+-----------------+-----------------------------+
| ARM | MMU L1 | #3878312 | N/A |
+----------------+-----------------+-----------------+-----------------------------+
-| ARM | MMU S3 | #3995052 | N/A |
+| ARM | MMU S3 | #3995052, | N/A |
+| | | #3673557 | |
+----------------+-----------------+-----------------+-----------------------------+
| ARM | GIC-700 | #2941627 | ARM64_ERRATUM_2941627 |
+----------------+-----------------+-----------------+-----------------------------+
diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst
index 16c35b491dad..4c241d220bc2 100644
--- a/Documentation/filesystems/overlayfs.rst
+++ b/Documentation/filesystems/overlayfs.rst
@@ -731,7 +731,8 @@ When decoding an overlay file handle, the following steps are followed:
1. Find underlying layer by UUID and path type information.
2. Decode the underlying filesystem file handle to underlying dentry.
- 3. For a lower file handle, lookup the handle in index directory by name.
+ 3. For a lower file handle, encode the decoded underlying inode again and
+ lookup that handle in index directory by name.
4. If a whiteout is found in index, return ESTALE. This represents an
overlay object that was deleted after its file handle was encoded.
5. For a non-directory, instantiate a disconnected overlay dentry from the
diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index e4b66af55ffb..6b641c3cee64 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -351,12 +351,12 @@ with the following files:
"mbm_assign_mode":
The supported counter assignment modes. The enclosed brackets indicate which mode
is enabled. The MBM events associated with counters may reset when "mbm_assign_mode"
- is changed.
+ is changed. See architecture specific notes below.
::
# cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
- [mbm_event]
- default
+ [default]
+ mbm_event
"mbm_event":
@@ -374,20 +374,17 @@ with the following files:
Moving to mbm_event counter assignment mode requires users to assign the counters
to the events. Otherwise, the MBM event counters will return 'Unassigned' when read.
- The mode is beneficial for AMD platforms that support more CTRL_MON
- and MON groups than available hardware counters. By default, this
- feature is enabled on AMD platforms with the ABMC (Assignable Bandwidth
- Monitoring Counters) capability, ensuring counters remain assigned even
- when the corresponding RMID is not actively used by any processor.
+ It is intended for deployments that need to manage counter assignment on
+ platforms where the number of monitoring groups exceeds the available
+ hardware counters. Hardware counters are scarce on these platforms, so
+ mbm_event mode suits workflows that monitor a subset of groups at a time and
+ rotate assignments as needed. The mbm_event mode ensures counters remain
+ assigned even when the corresponding RMID is not actively used.
"default":
In default mode, resctrl assumes there is a hardware counter for each
- event within every CTRL_MON and MON group. On AMD platforms, it is
- recommended to use the mbm_event mode, if supported, to prevent reset of MBM
- events between reads resulting from hardware re-allocating counters. This can
- result in misleading values or display "Unavailable" if no counter is assigned
- to the event.
+ event within every CTRL_MON and MON group.
* To enable "mbm_event" counter assignment mode:
::
@@ -399,6 +396,24 @@ with the following files:
# echo "default" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
+ In default mode on AMD platforms that support more CTRL_MON and MON groups
+ than hardware counters, hardware dynamically shares a smaller pool of
+ counters among RMIDs. One counter from this pool is used per monitoring
+ group and counts every MBM event of that group's RMID. The size of the pool
+ is not enumerated to software (unlike "num_mbm_cntrs"), and "num_rmids" may
+ be much larger. For example, 64 counters in this pool monitor 64 groups.
+ Newer hardware may provide a larger pool.
+
+ On such platforms, while the number of monitoring groups does not exceed
+ that pool, a counter stays attached to each RMID and readings remain
+ accurate. Creating more groups than the pool can cause hardware to
+ re-allocate those counters between successive reads of an event. Bandwidth
+ values may then be misleading, or a read may return "Unavailable" if no
+ counter is allocated to the RMID. There is no user-visible indication when
+ this begins. Users who need stable readings beyond that pool should switch
+ to mbm_event mode, if supported, and assign counters to the groups of
+ interest (rotating assignments as needed).
+
"num_mbm_cntrs":
The maximum number of counters (total of available and assigned counters) in
each domain when the system supports mbm_event mode.
@@ -1783,17 +1798,19 @@ View the llc occupancy snapshot::
Examples on working with mbm_assign_mode
========================================
-a. Check if MBM counter assignment mode is supported.
+a. Check if MBM counter assignment mode is supported. Enable "mbm_event"
+mode. Switching the mbm_assign_mode may reset all the MBM counters (and
+thus all MBM events) of all the resctrl groups.
::
- # mount -t resctrl resctrl /sys/fs/resctrl/
-
+ # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
+ [default]
+ mbm_event
+ # echo "mbm_event" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
# cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
[mbm_event]
default
-The "mbm_event" mode is detected and enabled.
-
b. Check how many assignable counters are supported.
::
@@ -1804,36 +1821,39 @@ c. Check how many assignable counters are available for assignment in each domai
::
# cat /sys/fs/resctrl/info/L3_MON/available_mbm_cntrs
- 0=30;1=30
+ 0=32;1=32
-d. To list the default group's assign states.
+d. To list the default group's assign states. Counters are unassigned when the mode
+is switched to "mbm_event".
::
# cat /sys/fs/resctrl/mbm_L3_assignments
- mbm_total_bytes:0=e;1=e
- mbm_local_bytes:0=e;1=e
+ mbm_total_bytes:0=_;1=_
+ mbm_local_bytes:0=_;1=_
-e. To unassign the counter associated with the mbm_total_bytes event on domain 0.
+e. To assign the counter associated with the mbm_total_bytes event on domain 0
+in exclusive mode.
::
- # echo "mbm_total_bytes:0=_" > /sys/fs/resctrl/mbm_L3_assignments
+ # echo "mbm_total_bytes:0=e" > /sys/fs/resctrl/mbm_L3_assignments
# cat /sys/fs/resctrl/mbm_L3_assignments
- mbm_total_bytes:0=_;1=e
- mbm_local_bytes:0=e;1=e
+ mbm_total_bytes:0=e;1=_
+ mbm_local_bytes:0=_;1=_
f. To unassign the counter associated with the mbm_total_bytes event on all domains.
::
# echo "mbm_total_bytes:*=_" > /sys/fs/resctrl/mbm_L3_assignments
- # cat /sys/fs/resctrl/mbm_L3_assignment
+ # cat /sys/fs/resctrl/mbm_L3_assignments
mbm_total_bytes:0=_;1=_
- mbm_local_bytes:0=e;1=e
+ mbm_local_bytes:0=_;1=_
-g. To assign a counter associated with the mbm_total_bytes event on all domains in
-exclusive mode.
+g. To assign a counter associated with the mbm_total_bytes and mbm_local_bytes
+events on all domains in exclusive mode.
::
# echo "mbm_total_bytes:*=e" > /sys/fs/resctrl/mbm_L3_assignments
+ # echo "mbm_local_bytes:*=e" > /sys/fs/resctrl/mbm_L3_assignments
# cat /sys/fs/resctrl/mbm_L3_assignments
mbm_total_bytes:0=e;1=e
mbm_local_bytes:0=e;1=e
@@ -1890,13 +1910,8 @@ the MBM counters (and thus all MBM events) of all the resctrl groups.
# echo "default" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
# cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
- mbm_event
[default]
-
-m. Unmount the resctrl filesystem.
-::
-
- # umount /sys/fs/resctrl/
+ mbm_event
Intel RDT Errata
================
diff --git a/Documentation/hwmon/asus_rog_ryujin.rst b/Documentation/hwmon/asus_rog_ryujin.rst
index b0d7ce8dd921..24dccef2d2df 100644
--- a/Documentation/hwmon/asus_rog_ryujin.rst
+++ b/Documentation/hwmon/asus_rog_ryujin.rst
@@ -33,6 +33,12 @@ Usage notes
As these are USB HIDs, the driver can be loaded automatically by the kernel and
supports hot swapping.
+The pump of the RYUJIN III also has a 4-pin cable for a motherboard fan header.
+The cooler either hands the pump to its internal controller, which follows the
+PWM of the header its 4-pin cable is plugged into, or runs it at the pump duty
+set over USB. Writing pwm1 selects the duty set over USB. Writing pwm2 sets
+only the internal fan duty and leaves the pump control as it is.
+
Sysfs entries
-------------
diff --git a/Documentation/mm/vmemmap_dedup.rst b/Documentation/mm/vmemmap_dedup.rst
index 9fa8642ded48..66d58fbea5b1 100644
--- a/Documentation/mm/vmemmap_dedup.rst
+++ b/Documentation/mm/vmemmap_dedup.rst
@@ -131,8 +131,19 @@ pages of ``struct page`` (page 1 to page 7) are tail pages.
The optimization is only applied when the size of the struct page is a power
of 2. In this case, all tail pages of the same order are identical. See
compound_head(). This allows us to remap the tail pages of the vmemmap to a
-shared, read-only page. The head page is also remapped to a new page. This
-allows the original vmemmap pages to be freed.
+shared page. The head page is also remapped to a new page. This allows the
+original vmemmap pages to be freed.
+
+The shared page is mapped writable. A lockless get of a page reference can
+read a non-zero refcount before the remap and attempt its atomic update
+after it. The refcount of a tail page is zero, so the update fails without
+changing the page. Nothing else may write to the shared page: it is shared
+by all HugeTLB pages of the same size in the zone, and a stray write would
+corrupt all of them.
+
+HugeTLB pages whose vmemmap is optimized when the memmap is first set up at
+boot keep a read-only mapping of the shared page. Nothing can do a lockless
+get on them before they are handed to HugeTLB.
Here is how things look after remapping::
diff --git a/Makefile b/Makefile
index 05c5b682ad50..51e278ccea9c 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
VERSION = 7
PATCHLEVEL = 3
-SUBLEVEL = 4
+SUBLEVEL = 5
EXTRAVERSION =
NAME = Baby Opossum Posse
diff --git a/arch/arm64/include/asm/sdei.h b/arch/arm64/include/asm/sdei.h
index b2248bd3cb58..a78a5337ff85 100644
--- a/arch/arm64/include/asm/sdei.h
+++ b/arch/arm64/include/asm/sdei.h
@@ -44,7 +44,8 @@ asmlinkage unsigned long __sdei_handler(struct pt_regs *regs,
struct sdei_registered_event *arg);
unsigned long do_sdei_event(struct pt_regs *regs,
- struct sdei_registered_event *arg);
+ struct sdei_registered_event *arg,
+ unsigned long vbar);
unsigned long sdei_arch_get_entry_point(int conduit);
#define sdei_arch_get_entry_point(x) sdei_arch_get_entry_point(x)
diff --git a/arch/arm64/include/asm/suspend.h b/arch/arm64/include/asm/suspend.h
index e9ce68d50ba4..a5abf0615594 100644
--- a/arch/arm64/include/asm/suspend.h
+++ b/arch/arm64/include/asm/suspend.h
@@ -2,7 +2,7 @@
#ifndef __ASM_SUSPEND_H
#define __ASM_SUSPEND_H
-#define NR_CTX_REGS 14
+#define NR_CTX_REGS 15
#define NR_CALLEE_SAVED_REGS 12
/*
diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
index 72c03ccea59f..92b2fe0b2784 100644
--- a/arch/arm64/kernel/entry-common.c
+++ b/arch/arm64/kernel/entry-common.c
@@ -30,6 +30,7 @@
#include <asm/stacktrace.h>
#include <asm/sysreg.h>
#include <asm/system_misc.h>
+#include <asm/vectors.h>
/*
* Handle IRQ/context state management when entering from kernel mode.
@@ -1002,6 +1003,7 @@ asmlinkage void noinstr __noreturn handle_bad_stack(struct pt_regs *regs)
asmlinkage noinstr unsigned long
__sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
{
+ unsigned long elr, spsr, esr, far, vbar;
irqentry_state_t state;
unsigned long ret;
@@ -1027,10 +1029,46 @@ __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
else if (cpu_has_pan())
set_pstate_pan(0);
+ /*
+ * Nothing saved the interrupted context's exception registers on the
+ * way in, and it may be in the middle of exception entry or return
+ * (SDEI ignores DAIF). Save them before anything can take an exception
+ * and overwrite them.
+ */
+ elr = read_sysreg(elr_el1);
+ spsr = read_sysreg(spsr_el1);
+ esr = read_sysreg(esr_el1);
+ far = read_sysreg(far_el1);
+
+ /*
+ * With KPTI, VBAR_EL1 holds the trampoline vectors from the return to
+ * EL0 until the next exception from EL0 maps the kernel. They only
+ * handle exceptions from EL0: an exception taken here would run their
+ * empty current-EL half and never come back. Switch to the kernel
+ * vectors. The IRQ-like resume keeps using the interrupted VBAR, so it
+ * still goes through the trampoline.
+ */
+ vbar = read_sysreg(vbar_el1);
+ if (arm64_kernel_unmapped_at_el0() &&
+ vbar == (unsigned long)raw_cpu_read(this_cpu_vector)) {
+ write_sysreg((unsigned long)vectors, vbar_el1);
+ isb();
+ }
+
state = irqentry_nmi_enter(regs);
- ret = do_sdei_event(regs, arg);
+ ret = do_sdei_event(regs, arg, vbar);
irqentry_nmi_exit(regs, state);
+ /*
+ * Put them back unconditionally. An unchanged ELR does not rule out a
+ * nested exception: one may have returned to the same address.
+ */
+ write_sysreg(elr, elr_el1);
+ write_sysreg(spsr, spsr_el1);
+ write_sysreg(esr, esr_el1);
+ write_sysreg(far, far_el1);
+ write_sysreg(vbar, vbar_el1);
+
return ret;
}
#endif /* CONFIG_ARM_SDE_INTERFACE */
diff --git a/arch/arm64/kernel/machine_kexec_file.c b/arch/arm64/kernel/machine_kexec_file.c
index 854d872dfd0f..14f453dd0946 100644
--- a/arch/arm64/kernel/machine_kexec_file.c
+++ b/arch/arm64/kernel/machine_kexec_file.c
@@ -112,6 +112,8 @@ int load_other_segments(struct kimage *image,
vfree(headers);
goto out_err;
}
+
+ vfree(image->elf_headers);
image->elf_headers = headers;
image->elf_load_addr = kbuf.mem;
image->elf_headers_sz = headers_sz;
diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c
index 778f2a1faac8..1b902da8ba43 100644
--- a/arch/arm64/kernel/sdei.c
+++ b/arch/arm64/kernel/sdei.c
@@ -195,21 +195,49 @@ unsigned long sdei_arch_get_entry_point(int conduit)
return 0;
}
+/*
+ * Exceptions an SDEI handler may take without harm once __sdei_handler() has
+ * restored the interrupted context's exception registers. Their handlers are
+ * safe in NMI context:
+ * - EL1 data aborts on translation, permission and access-flag faults:
+ * faulthandler_disabled() is true in NMI context, so do_page_fault() never
+ * takes the mmap lock. This covers the _nofault() accessors used by the
+ * generic dump_stack()/show_regs() code (e.g. print_worker_info()).
+ * - BRK: kprobes, WARN/BUG, kCFI and UBSAN traps, all NMI-capable.
+ * Anything else taken at EL1 is fatal on its own, or unexpected here.
+ */
+static bool sdei_nested_exception_ok(unsigned long esr)
+{
+ switch (ESR_ELx_EC(esr)) {
+ case ESR_ELx_EC_DABT_CUR:
+ return esr_fsc_is_translation_fault(esr) ||
+ esr_fsc_is_permission_fault(esr) ||
+ esr_fsc_is_access_flag_fault(esr);
+ case ESR_ELx_EC_BRK64:
+ return true;
+ default:
+ return false;
+ }
+}
+
/*
* do_sdei_event() returns one of:
* SDEI_EV_HANDLED - success, return to the interrupted context.
* SDEI_EV_FAILED - failure, return this error code to firmware.
* virtual-address - success, return to this address.
+ *
+ * vbar is the interrupted context's VBAR_EL1, for the IRQ-like resume.
*/
unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
- struct sdei_registered_event *arg)
+ struct sdei_registered_event *arg,
+ unsigned long vbar)
{
u32 mode;
int i, err = 0;
int clobbered_registers = 4;
u64 elr = read_sysreg(elr_el1);
+ u64 nested_elr, nested_esr, nested_far;
u32 kernel_mode = read_sysreg(CurrentEL) | 1; /* +SPSel */
- unsigned long vbar = read_sysreg(vbar_el1);
if (arm64_kernel_unmapped_at_el0())
clobbered_registers++;
@@ -221,18 +249,27 @@ unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
}
err = sdei_event_handler(regs, arg);
- if (err)
- return SDEI_EV_FAILED;
- if (elr != read_sysreg(elr_el1)) {
- /*
- * We took a synchronous exception from the SDEI handler.
- * This could deadlock, and if you interrupt KVM it will
- * hyp-panic instead.
- */
- pr_warn("unsafe: exception during handler\n");
+ /*
+ * A synchronous exception taken by the handler has overwritten the
+ * exception registers; __sdei_handler() puts them back. Only the last
+ * one's syndrome survives; report it if it is not one known to be
+ * harmless.
+ *
+ * If the event interrupted a KVM world switch, VBAR points at the KVM
+ * hyp vectors and any such exception is a hyp panic instead.
+ */
+ nested_elr = read_sysreg(elr_el1);
+ nested_esr = read_sysreg(esr_el1);
+ nested_far = read_sysreg(far_el1);
+ if (nested_elr != elr && !sdei_nested_exception_ok(nested_esr)) {
+ pr_warn_once("unexpected exception during handler: ESR %#llx FAR %#llx near %pS\n",
+ nested_esr, nested_far, (void *)nested_elr);
}
+ if (err)
+ return SDEI_EV_FAILED;
+
mode = regs->pstate & (PSR_MODE32_BIT | PSR_MODE_MASK);
/*
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index b8baba281166..44305b997a02 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -470,6 +470,12 @@ static void amu_read_core_const_ctrs(void *val)
cpu_read_corecnt(&ctrs->corecnt);
}
+static bool cpc_ffh_reg_valid(const struct cpc_reg *reg)
+{
+ return reg->bit_width && reg->bit_width <= 64 &&
+ reg->bit_offset <= 64 - reg->bit_width;
+}
+
static u64 cpc_ffh_extract_bits(const struct cpc_reg *reg, u64 val)
{
val &= GENMASK_ULL(reg->bit_offset + reg->bit_width - 1,
@@ -506,7 +512,8 @@ int cpc_read_ffh_fb_ctrs(int cpu, struct cpc_reg *reg1, u64 *val1,
struct amu_ffh_ctrs ctrs;
int ret;
- if (!is_amu_ctr_reg(reg1) || !is_amu_ctr_reg(reg2))
+ if (!is_amu_ctr_reg(reg1) || !is_amu_ctr_reg(reg2) ||
+ !cpc_ffh_reg_valid(reg1) || !cpc_ffh_reg_valid(reg2))
return -EINVAL;
ret = counters_read_on_cpu(cpu, amu_read_core_const_ctrs, &ctrs);
@@ -530,6 +537,9 @@ int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val)
{
int ret = -EOPNOTSUPP;
+ if (!cpc_ffh_reg_valid(reg))
+ return -EINVAL;
+
switch ((u64)reg->address) {
case CPC_FFH_CTR_CORE:
ret = counters_read_on_cpu(cpu, cpu_read_corecnt, val);
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index bf6d6526639f..cae133805c76 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1362,21 +1362,32 @@ static void kvm_send_hwpoison_signal(unsigned long address, short lsb)
send_sig_mceerr(BUS_MCEERR_AR, (void __user *)address, lsb, current);
}
-static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot,
- unsigned long hva,
+struct kvm_s2_fault_desc {
+ struct kvm_vcpu *vcpu;
+ phys_addr_t fault_ipa;
+ struct kvm_s2_trans *nested;
+ struct kvm_memory_slot *memslot;
+ unsigned long hva;
+};
+
+static bool fault_supports_stage2_huge_mapping(const struct kvm_s2_fault_desc *s2fd,
unsigned long map_size)
{
- gpa_t gpa_start;
+ struct kvm_memory_slot *memslot = s2fd->memslot;
+ unsigned long hva = s2fd->hva;
hva_t uaddr_start, uaddr_end;
+ gpa_t gpa_start;
size_t size;
/* The memslot and the VMA are guaranteed to be aligned to PAGE_SIZE */
if (map_size == PAGE_SIZE)
return true;
- /* pKVM only supports PMD_SIZE huge-mappings */
- if (is_protected_kvm_enabled() && map_size != PMD_SIZE)
- return false;
+ /* pKVM only supports PMD_SIZE huge-mappings for non-protected VMs */
+ if (is_protected_kvm_enabled()) {
+ if (vcpu_is_protected(s2fd->vcpu) || map_size != PMD_SIZE)
+ return false;
+ }
size = memslot->npages * PAGE_SIZE;
@@ -1436,9 +1447,9 @@ static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot,
* Returns the size of the mapping.
*/
static long
-transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot,
- unsigned long hva, kvm_pfn_t *pfnp, gfn_t *gfnp)
+transparent_hugepage_adjust(const struct kvm_s2_fault_desc *s2fd, kvm_pfn_t *pfnp, gfn_t *gfnp)
{
+ struct kvm *kvm = s2fd->vcpu->kvm;
kvm_pfn_t pfn = *pfnp;
gfn_t gfn = *gfnp;
@@ -1447,8 +1458,8 @@ transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot,
* sure that the HVA and IPA are sufficiently aligned and that the
* block map is contained within the memslot.
*/
- if (fault_supports_stage2_huge_mapping(memslot, hva, PMD_SIZE)) {
- int sz = get_user_mapping_size(kvm, hva);
+ if (fault_supports_stage2_huge_mapping(s2fd, PMD_SIZE)) {
+ int sz = get_user_mapping_size(kvm, s2fd->hva);
if (sz < 0)
return sz;
@@ -1585,14 +1596,27 @@ static enum kvm_pgtable_prot adjust_nested_exec_perms(struct kvm *kvm,
return prot;
}
-struct kvm_s2_fault_desc {
- struct kvm_vcpu *vcpu;
- phys_addr_t fault_ipa;
- struct kvm_s2_trans *nested;
- struct kvm_memory_slot *memslot;
- unsigned long hva;
+struct kvm_s2_fault_vma_info {
+ unsigned long mmu_seq;
+ long vma_pagesize;
+ vm_flags_t vm_flags;
+ unsigned long max_map_size;
+ struct page *page;
+ kvm_pfn_t pfn;
+ gfn_t gfn;
+ bool device;
+ bool mte_allowed;
+ bool is_vma_cacheable;
+ bool map_writable;
+ bool map_non_cacheable;
};
+static int kvm_s2_fault_get_vma_info(const struct kvm_s2_fault_desc *s2fd,
+ struct kvm_s2_fault_vma_info *s2vi);
+
+static gfn_t get_canonical_gfn(const struct kvm_s2_fault_desc *s2fd,
+ const struct kvm_s2_fault_vma_info *s2vi);
+
static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
{
bool write_fault, exec_fault;
@@ -1600,12 +1624,10 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_SHARED;
enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R;
struct kvm_pgtable *pgt = s2fd->vcpu->arch.hw_mmu->pgt;
- unsigned long mmu_seq;
- struct page *page;
+ struct kvm_s2_fault_vma_info s2vi = {};
struct kvm *kvm = s2fd->vcpu->kvm;
void *memcache = NULL;
- kvm_pfn_t pfn;
- gfn_t gfn;
+ gfn_t canonical_gfn;
int ret;
if (!perm_fault) {
@@ -1615,24 +1637,23 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
return ret;
}
- if (s2fd->nested)
- gfn = kvm_s2_trans_output(s2fd->nested) >> PAGE_SHIFT;
- else
- gfn = s2fd->fault_ipa >> PAGE_SHIFT;
+ s2vi.vma_pagesize = PAGE_SIZE;
+ s2vi.gfn = ALIGN_DOWN(s2fd->fault_ipa, s2vi.vma_pagesize) >> PAGE_SHIFT;
+ canonical_gfn = get_canonical_gfn(s2fd, &s2vi);
write_fault = kvm_is_write_fault(s2fd->vcpu);
exec_fault = kvm_vcpu_trap_is_exec_fault(s2fd->vcpu);
VM_WARN_ON_ONCE(write_fault && exec_fault);
- mmu_seq = kvm->mmu_invalidate_seq;
+ s2vi.mmu_seq = kvm->mmu_invalidate_seq;
/* Pairs with the smp_wmb() in kvm_mmu_invalidate_end(). */
smp_rmb();
- ret = kvm_gmem_get_pfn(kvm, s2fd->memslot, gfn, &pfn, &page, NULL);
+ ret = kvm_gmem_get_pfn(kvm, s2fd->memslot, canonical_gfn, &s2vi.pfn, &s2vi.page, NULL);
if (ret) {
- kvm_prepare_memory_fault_exit(s2fd->vcpu, s2fd->fault_ipa, PAGE_SIZE,
- write_fault, exec_fault, false);
+ kvm_prepare_memory_fault_exit(s2fd->vcpu, gfn_to_gpa(canonical_gfn),
+ s2vi.vma_pagesize, write_fault, exec_fault, false);
return ret;
}
@@ -1649,7 +1670,7 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
prot = adjust_nested_exec_perms(kvm, s2fd->nested, prot);
kvm_fault_lock(kvm);
- if (mmu_invalidate_retry(kvm, mmu_seq)) {
+ if (mmu_invalidate_retry(kvm, s2vi.mmu_seq)) {
ret = -EAGAIN;
goto out_unlock;
}
@@ -1660,48 +1681,34 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
* PTE, which will be preserved.
*/
prot &= ~KVM_NV_GUEST_MAP_SZ;
- ret = KVM_PGT_FN(kvm_pgtable_stage2_relax_perms)(pgt, s2fd->fault_ipa,
+ ret = KVM_PGT_FN(kvm_pgtable_stage2_relax_perms)(pgt, gfn_to_gpa(s2vi.gfn),
prot, flags);
} else {
- ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, s2fd->fault_ipa, PAGE_SIZE,
- __pfn_to_phys(pfn), prot,
+ ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, gfn_to_gpa(s2vi.gfn),
+ s2vi.vma_pagesize,
+ __pfn_to_phys(s2vi.pfn), prot,
memcache, flags);
}
out_unlock:
- kvm_release_faultin_page(kvm, page, !!ret, prot & KVM_PGTABLE_PROT_W);
+ kvm_release_faultin_page(kvm, s2vi.page, !!ret, prot & KVM_PGTABLE_PROT_W);
kvm_fault_unlock(kvm);
if ((prot & KVM_PGTABLE_PROT_W) && !ret)
- mark_page_dirty_in_slot(kvm, s2fd->memslot, gfn);
+ mark_page_dirty_in_slot(kvm, s2fd->memslot, canonical_gfn);
return ret != -EAGAIN ? ret : 0;
}
-struct kvm_s2_fault_vma_info {
- unsigned long mmu_seq;
- long vma_pagesize;
- vm_flags_t vm_flags;
- unsigned long max_map_size;
- struct page *page;
- kvm_pfn_t pfn;
- gfn_t gfn;
- bool device;
- bool mte_allowed;
- bool is_vma_cacheable;
- bool map_writable;
- bool map_non_cacheable;
-};
-
static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd)
{
unsigned int flags = FOLL_HWPOISON | FOLL_LONGTERM | FOLL_WRITE;
+ struct kvm_s2_fault_vma_info s2vi = {};
struct kvm_vcpu *vcpu = s2fd->vcpu;
struct kvm_pgtable *pgt = vcpu->arch.hw_mmu->pgt;
struct mm_struct *mm = current->mm;
struct kvm *kvm = vcpu->kvm;
void *hyp_memcache;
- struct page *page;
int ret;
hyp_memcache = get_mmu_memcache(vcpu);
@@ -1709,12 +1716,16 @@ static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd)
if (ret)
return -ENOMEM;
+ ret = kvm_s2_fault_get_vma_info(s2fd, &s2vi);
+ if (ret)
+ return ret;
+
ret = account_locked_vm(mm, 1, true);
if (ret)
return ret;
mmap_read_lock(mm);
- ret = pin_user_pages(s2fd->hva, 1, flags, &page);
+ ret = pin_user_pages(s2fd->hva, 1, flags, &s2vi.page);
mmap_read_unlock(mm);
if (ret == -EHWPOISON) {
@@ -1724,7 +1735,7 @@ static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd)
} else if (ret != 1) {
ret = -EFAULT;
goto dec_account;
- } else if (!folio_test_swapbacked(page_folio(page))) {
+ } else if (!folio_test_swapbacked(page_folio(s2vi.page))) {
/*
* We really can't deal with page-cache pages returned by GUP
* because (a) we may trigger writeback of a page for which we
@@ -1744,8 +1755,8 @@ static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd)
}
write_lock(&kvm->mmu_lock);
- ret = pkvm_pgtable_stage2_map(pgt, s2fd->fault_ipa, PAGE_SIZE,
- page_to_phys(page), KVM_PGTABLE_PROT_RWX,
+ ret = pkvm_pgtable_stage2_map(pgt, gfn_to_gpa(s2vi.gfn), PAGE_SIZE,
+ page_to_phys(s2vi.page), KVM_PGTABLE_PROT_RWX,
hyp_memcache, 0);
write_unlock(&kvm->mmu_lock);
if (ret) {
@@ -1756,7 +1767,7 @@ static int pkvm_mem_abort(const struct kvm_s2_fault_desc *s2fd)
return 0;
unpin:
- unpin_user_pages(&page, 1);
+ unpin_user_page(s2vi.page);
dec_account:
account_locked_vm(mm, 1, false);
return ret;
@@ -1779,7 +1790,7 @@ static short kvm_s2_resolve_vma_size(const struct kvm_s2_fault_desc *s2fd,
switch (vma_shift) {
#ifndef __PAGETABLE_PMD_FOLDED
case PUD_SHIFT:
- if (fault_supports_stage2_huge_mapping(s2fd->memslot, s2fd->hva, PUD_SIZE))
+ if (fault_supports_stage2_huge_mapping(s2fd, PUD_SIZE))
break;
fallthrough;
#endif
@@ -1787,7 +1798,7 @@ static short kvm_s2_resolve_vma_size(const struct kvm_s2_fault_desc *s2fd,
vma_shift = PMD_SHIFT;
fallthrough;
case PMD_SHIFT:
- if (fault_supports_stage2_huge_mapping(s2fd->memslot, s2fd->hva, PMD_SIZE))
+ if (fault_supports_stage2_huge_mapping(s2fd, PMD_SIZE))
break;
fallthrough;
case CONT_PTE_SHIFT:
@@ -2027,9 +2038,7 @@ static int kvm_s2_fault_map(const struct kvm_s2_fault_desc *s2fd,
if (perm_fault_granule > PAGE_SIZE) {
mapping_size = perm_fault_granule;
} else {
- mapping_size = transparent_hugepage_adjust(kvm, s2fd->memslot,
- s2fd->hva, &pfn,
- &gfn);
+ mapping_size = transparent_hugepage_adjust(s2fd, &pfn, &gfn);
if (mapping_size < 0) {
ret = mapping_size;
goto out_unlock;
diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
index 8e4c6e4bec12..c298a5b9b12a 100644
--- a/arch/arm64/kvm/pkvm.c
+++ b/arch/arm64/kvm/pkvm.c
@@ -414,6 +414,9 @@ int pkvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size,
u64 end = addr + size;
int ret;
+ if (WARN_ON_ONCE(!PAGE_ALIGNED(addr | size)))
+ return -EINVAL;
+
lockdep_assert_held_write(&kvm->mmu_lock);
mapping = pkvm_mapping_iter_first(&pgt->pkvm_mappings, addr, end - 1);
diff --git a/arch/arm64/kvm/vgic/vgic-mmio-v3.c b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
index 5ab789accc9d..9e6200c5adce 100644
--- a/arch/arm64/kvm/vgic/vgic-mmio-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
@@ -303,9 +303,19 @@ static void vgic_mmio_write_v3r_ctlr(struct kvm_vcpu *vcpu,
if (ctlr != GICR_CTLR_ENABLE_LPIS)
return;
+ /*
+ * Yes, disabling LPIs is painful, since it can be done from
+ * a *remote* vcpu! So let's not take any chance, and make
+ * sure that the target vCPU has written their LRs back to the
+ * irq structures, and release any reference they would have.
+ *
+ * If it hurts, don't do it.
+ */
+ kvm_arm_halt_guest(vcpu->kvm);
vgic_flush_pending_lpis(vcpu);
vgic_its_invalidate_all_caches(vcpu->kvm);
atomic_set_release(&vgic_cpu->ctlr, 0);
+ kvm_arm_resume_guest(vcpu->kvm);
} else {
ctlr = atomic_cmpxchg_acquire(&vgic_cpu->ctlr, 0,
GICR_CTLR_ENABLE_LPIS);
diff --git a/arch/arm64/mm/mmap.c b/arch/arm64/mm/mmap.c
index 32e0771d6477..e212270d4e09 100644
--- a/arch/arm64/mm/mmap.c
+++ b/arch/arm64/mm/mmap.c
@@ -61,7 +61,9 @@ int valid_phys_addr_range(phys_addr_t addr, size_t size)
*/
int valid_mmap_phys_addr_range(unsigned long pfn, size_t size)
{
- return !(((pfn << PAGE_SHIFT) + size) & ~PHYS_MASK);
+ const phys_addr_t limit = PHYS_MASK + 1;
+
+ return size <= limit && pfn <= (limit - size) >> PAGE_SHIFT;
}
static int __init adjust_protection_map(void)
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 12aacc74f764..b1516630904d 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -113,6 +113,10 @@ SYM_FUNC_START(cpu_do_suspend)
alternative_if ARM64_HAS_TCR2
mrs x2, REG_TCR2_EL1
str x2, [x0, #104]
+alternative_else_nop_endif
+alternative_if ARM64_SME
+ mrs_s x2, SYS_TPIDR2_EL0
+ str x2, [x0, #112]
alternative_else_nop_endif
ret
SYM_FUNC_END(cpu_do_suspend)
@@ -152,6 +156,10 @@ alternative_if ARM64_HAS_TCR2
ldr x2, [x0, #104]
msr REG_TCR2_EL1, x2
alternative_else_nop_endif
+alternative_if ARM64_SME
+ ldr x2, [x0, #112]
+ msr_s SYS_TPIDR2_EL0, x2
+alternative_else_nop_endif
msr sctlr_el1, x12
set_this_cpu_offset x13
diff --git a/arch/mips/mm/cex-gen.S b/arch/mips/mm/cex-gen.S
index e528583d1331..6989fe274aab 100644
--- a/arch/mips/mm/cex-gen.S
+++ b/arch/mips/mm/cex-gen.S
@@ -8,6 +8,7 @@
*
* Cache error handler
*/
+#include <asm/addrspace.h>
#include <asm/asm.h>
#include <asm/regdef.h>
#include <asm/mipsregs.h>
@@ -37,6 +38,21 @@
nop
nop
+#if defined(CONFIG_64BIT) && VMLINUX_LOAD_ADDRESS >= XKPHYS && \
+ VMLINUX_LOAD_ADDRESS < XKSEG
+ /*
+ * An XKPHYS kernel may lie outside the 256MB region a j from the
+ * vector reaches, so jump to the handler's uncached XKPHYS address.
+ */
+ PTR_LA k0, cache_parity_error
+ PTR_LI k1, TO_PHYS_MASK
+ and k0, k0, k1
+ PTR_LI k1, UNCAC_BASE
+ or k0, k0, k1
+ jr k0
+ nop
+#else
j cache_parity_error
nop
+#endif
END(except_vec2_generic)
diff --git a/arch/mips/pci/ops-mace.c b/arch/mips/pci/ops-mace.c
index 951d8070fb48..8827e5f2ea46 100644
--- a/arch/mips/pci/ops-mace.c
+++ b/arch/mips/pci/ops-mace.c
@@ -35,15 +35,27 @@ static inline int mkaddr(struct pci_bus *bus, unsigned int devfn,
(reg & 0xfc);
}
+/*
+ * On CRIME 1.1, a MACE PCI register read followed by two register writes
+ * can hang the machine under DMA load, so every register write in this
+ * file is followed by a read. Config cycles to empty slots end in a
+ * master abort; ack it with interrupts still off so the bridge error
+ * interrupt does not report it.
+ */
+static void mace_pci_ack_abort(void)
+{
+ u32 error = mace->pci.error;
+
+ if (error & MACEPCI_ERROR_MASTER_ABORT) {
+ mace->pci.error = error & ~MACEPCI_ERROR_MASTER_ABORT;
+ mace->pci.error;
+ }
+}
static int
mace_pci_read_config(struct pci_bus *bus, unsigned int devfn,
int reg, int size, u32 *val)
{
- u32 control = mace->pci.control;
-
- /* disable master aborts interrupts during config read */
- mace->pci.control = control & ~MACEPCI_CONTROL_MAR_INT;
mace->pci.config_addr = mkaddr(bus, devfn, reg);
switch (size) {
case 1:
@@ -56,9 +68,7 @@ mace_pci_read_config(struct pci_bus *bus, unsigned int devfn,
*val = mace->pci.config_data.l;
break;
}
- /* ack possible master abort */
- mace->pci.error &= ~MACEPCI_ERROR_MASTER_ABORT;
- mace->pci.control = control;
+ mace_pci_ack_abort();
/*
* someone forgot to set the ultra bit for the onboard
* scsi chips; we fake it here
@@ -77,6 +87,7 @@ mace_pci_write_config(struct pci_bus *bus, unsigned int devfn,
int reg, int size, u32 val)
{
mace->pci.config_addr = mkaddr(bus, devfn, reg);
+ mace->pci.error;
switch (size) {
case 1:
mace->pci.config_data.b[(reg & 3) ^ 3] = val;
@@ -88,6 +99,7 @@ mace_pci_write_config(struct pci_bus *bus, unsigned int devfn,
mace->pci.config_data.l = val;
break;
}
+ mace_pci_ack_abort();
DPRINTK("write%d: reg=%08x,val=%02x\n", size * 8, reg, val);
diff --git a/arch/mips/pci/pci-ip32.c b/arch/mips/pci/pci-ip32.c
index 7ae89d0c7099..41ec8453c865 100644
--- a/arch/mips/pci/pci-ip32.c
+++ b/arch/mips/pci/pci-ip32.c
@@ -77,6 +77,8 @@ static irqreturn_t macepci_error(int irq, void *dev)
}
mace->pci.error = flags;
+ /* follow the write with a read, see mace_pci_ack_abort() */
+ mace->pci.error;
return IRQ_HANDLED;
}
@@ -125,10 +127,13 @@ static int __init mace_init(void)
{
PCIBIOS_MIN_IO = 0x1000;
- /* Clear any outstanding errors and enable interrupts */
+ /*
+ * Clear any outstanding errors and enable interrupts. PCI interrupts
+ * are masked in CRIME, so the per-slot enables stay on.
+ */
mace->pci.error_addr = 0;
mace->pci.error = 0;
- mace->pci.control = 0xff008500;
+ mace->pci.control = 0xff0085ff;
printk("MACE PCI rev %d\n", mace->pci.rev);
diff --git a/arch/mips/sgi-ip32/ip32-irq.c b/arch/mips/sgi-ip32/ip32-irq.c
index 29d04468a06b..b9c72af481ed 100644
--- a/arch/mips/sgi-ip32/ip32-irq.c
+++ b/arch/mips/sgi-ip32/ip32-irq.c
@@ -160,38 +160,6 @@ static struct irq_chip crime_edge_interrupt = {
.irq_unmask = crime_enable_irq,
};
-/*
- * This is for MACE PCI interrupts. We can decrease bus traffic by masking
- * as close to the source as possible. This also means we can take the
- * next chunk of the CRIME register in one piece.
- */
-
-static unsigned long macepci_mask;
-
-static void enable_macepci_irq(struct irq_data *d)
-{
- macepci_mask |= MACEPCI_CONTROL_INT(d->irq - MACEPCI_SCSI0_IRQ);
- mace->pci.control = macepci_mask;
- crime_mask |= 1 << (d->irq - CRIME_IRQ_BASE);
- crime->imask = crime_mask;
-}
-
-static void disable_macepci_irq(struct irq_data *d)
-{
- crime_mask &= ~(1 << (d->irq - CRIME_IRQ_BASE));
- crime->imask = crime_mask;
- flush_crime_bus();
- macepci_mask &= ~MACEPCI_CONTROL_INT(d->irq - MACEPCI_SCSI0_IRQ);
- mace->pci.control = macepci_mask;
- flush_mace_bus();
-}
-
-static struct irq_chip ip32_macepci_interrupt = {
- .name = "IP32 MACE PCI",
- .irq_mask = disable_macepci_irq,
- .irq_unmask = enable_macepci_irq,
-};
-
/* This is used for MACE ISA interrupts. That means bits 4-6 in the
* CRIME register.
*/
@@ -299,8 +267,8 @@ static struct irq_chip ip32_maceisa_edge_interrupt = {
.irq_unmask = enable_maceisa_irq,
};
-/* This is used for regular non-ISA, non-PCI MACE interrupts. That means
- * bits 0-3 and 7 in the CRIME register.
+/* This is used for regular non-ISA MACE interrupts. That means bits 0-3
+ * and 7-15 in the CRIME register.
*/
static void enable_mace_irq(struct irq_data *d)
@@ -436,20 +404,13 @@ void __init arch_init_irq(void)
mips_cpu_irq_init();
for (irq = CRIME_IRQ_BASE; irq <= IP32_IRQ_MAX; irq++) {
switch (irq) {
- case MACE_VID_IN1_IRQ ... MACE_PCI_BRIDGE_IRQ:
+ case MACE_VID_IN1_IRQ ... MACEPCI_SHARED2_IRQ:
irq_set_chip_and_handler_name(irq,
&ip32_mace_interrupt,
handle_level_irq,
"level");
break;
- case MACEPCI_SCSI0_IRQ ... MACEPCI_SHARED2_IRQ:
- irq_set_chip_and_handler_name(irq,
- &ip32_macepci_interrupt,
- handle_level_irq,
- "level");
- break;
-
case CRIME_CPUERR_IRQ:
case CRIME_MEMERR_IRQ:
irq_set_chip_and_handler_name(irq,
diff --git a/arch/openrisc/kernel/or32_ksyms.c b/arch/openrisc/kernel/or32_ksyms.c
index 84a937a64e2a..5aabef30ca09 100644
--- a/arch/openrisc/kernel/or32_ksyms.c
+++ b/arch/openrisc/kernel/or32_ksyms.c
@@ -35,6 +35,7 @@ DECLARE_EXPORT(__divsi3);
DECLARE_EXPORT(__umodsi3);
DECLARE_EXPORT(__modsi3);
DECLARE_EXPORT(__muldi3);
+DECLARE_EXPORT(__mulsi3);
DECLARE_EXPORT(__ashrdi3);
DECLARE_EXPORT(__ashldi3);
DECLARE_EXPORT(__lshrdi3);
diff --git a/arch/x86/boot/startup/sme.c b/arch/x86/boot/startup/sme.c
index c07a2c381ed1..2d5cb3f7eee6 100644
--- a/arch/x86/boot/startup/sme.c
+++ b/arch/x86/boot/startup/sme.c
@@ -99,7 +99,7 @@ static void __init sme_clear_pgd(struct sme_populate_pgd_data *ppd)
pgd_t *pgd_p;
pgd_start = ppd->vaddr & PGDIR_MASK;
- pgd_end = ppd->vaddr_end & PGDIR_MASK;
+ pgd_end = (ppd->vaddr_end - 1) & PGDIR_MASK;
pgd_size = (((pgd_end - pgd_start) / PGDIR_SIZE) + 1) * sizeof(pgd_t);
diff --git a/arch/x86/entry/syscall_32.c b/arch/x86/entry/syscall_32.c
index 91123a90710c..4c36e8db0e34 100644
--- a/arch/x86/entry/syscall_32.c
+++ b/arch/x86/entry/syscall_32.c
@@ -77,6 +77,10 @@ static __always_inline void do_syscall_32_irqs_on(struct pt_regs *regs, unsigned
if (likely(nr < IA32_NR_syscalls)) {
nr = array_index_nospec(nr, IA32_NR_syscalls);
regs->ax = ia32_sys_call(regs, (unsigned int)nr);
+ } else {
+ /* See comment in do_syscall_64() */
+ if (nr != -1UL)
+ regs->ax = -ENOSYS;
}
}
diff --git a/arch/x86/entry/syscall_64.c b/arch/x86/entry/syscall_64.c
index 966d3d1b7586..bb95ad0ab786 100644
--- a/arch/x86/entry/syscall_64.c
+++ b/arch/x86/entry/syscall_64.c
@@ -64,15 +64,20 @@ static __always_inline bool do_syscall_x64(struct pt_regs *regs, unsigned long n
return false;
}
-static __always_inline void do_syscall_x32(struct pt_regs *regs, unsigned long nr)
+static __always_inline bool do_syscall_x32(struct pt_regs *regs, unsigned long nr)
{
+ if (!IS_ENABLED(CONFIG_X86_X32_ABI))
+ return false;
+
/* Adjust the starting offset of the table */
nr -= __X32_SYSCALL_BIT;
- if (IS_ENABLED(CONFIG_X86_X32_ABI) && likely(nr < X32_NR_syscalls)) {
+ if (likely(nr < X32_NR_syscalls)) {
nr = array_index_nospec(nr, X32_NR_syscalls);
regs->ax = x32_sys_call(regs, (unsigned int)nr);
+ return true;
}
+ return false;
}
/* Returns true to return using SYSRET, or false to use IRET */
@@ -81,8 +86,21 @@ __visible noinstr bool do_syscall_64(struct pt_regs *regs, long nr)
if (likely(syscall_enter_from_user_mode_randomize_stack(regs, &nr))) {
instrumentation_begin();
- if (!do_syscall_x64(regs, nr))
- do_syscall_x32(regs, nr);
+ if (unlikely(!do_syscall_x64(regs, nr))) {
+ /*
+ * The special treatment of syscall number -1 is a
+ * historical bogosity as the original code used -1 to
+ * indicate syscall abort, which is not distinguishable
+ * from a syscall which is invoked with syscall number
+ * -1. The historic behavior is to keep the return value
+ * in regs->ax unmodified in that case, which means it's
+ * either the preset value -ENOSYS or a value which was
+ * written by a tracer. If the syscall number is invalid
+ * but not -1 then overwrite any tracer modification.
+ */
+ if (!do_syscall_x32(regs, nr) && nr != -1L)
+ regs->ax = -ENOSYS;
+ }
instrumentation_end();
}
diff --git a/arch/x86/include/asm/checksum_32.h b/arch/x86/include/asm/checksum_32.h
index 17da95387997..f74873a5e351 100644
--- a/arch/x86/include/asm/checksum_32.h
+++ b/arch/x86/include/asm/checksum_32.h
@@ -159,7 +159,7 @@ static inline __sum16 csum_ipv6_magic(const struct in6_addr *saddr,
"adcl $0, %0 ;\n"
: "=&r" (sum)
: "r" (saddr), "r" (daddr),
- "r" (htonl(len)), "r" (htonl(proto)), "0" (sum)
+ "g" (htonl(len)), "g" (htonl(proto)), "0" (sum)
: "memory");
return csum_fold(sum);
diff --git a/arch/x86/include/asm/vmware.h b/arch/x86/include/asm/vmware.h
index 4220dae14a2d..fb011cef01e7 100644
--- a/arch/x86/include/asm/vmware.h
+++ b/arch/x86/include/asm/vmware.h
@@ -276,20 +276,22 @@ unsigned long vmware_hypercall_hb_out(unsigned long cmd, unsigned long in2,
asm_inline volatile (
UNWIND_HINT_SAVE
+ "mov %[in6], %%" _ASM_AX "\n\t"
"push %%" _ASM_BP "\n\t"
UNWIND_HINT_UNDEFINED
- "mov %[in6], %%" _ASM_BP "\n\t"
+ "mov %%" _ASM_AX ", %%" _ASM_BP "\n\t"
+ "mov %[magic], %%eax\n\t"
"rep outsb\n\t"
"pop %%" _ASM_BP "\n\t"
UNWIND_HINT_RESTORE
- : "=a" (out0), "=b" (*out1)
- : "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (cmd),
+ : "=&a" (out0), "=b" (*out1)
+ : "b" (cmd),
"c" (in2),
"d" (in3 | VMWARE_HYPERVISOR_PORT_HB),
"S" (in4),
"D" (in5),
- [in6] VMW_BP_CONSTRAINT (in6)
+ [in6] VMW_BP_CONSTRAINT(in6),
+ [magic] "i" (VMWARE_HYPERVISOR_MAGIC)
: "cc", "memory");
return out0;
}
@@ -304,20 +306,22 @@ unsigned long vmware_hypercall_hb_in(unsigned long cmd, unsigned long in2,
asm_inline volatile (
UNWIND_HINT_SAVE
+ "mov %[in6], %%" _ASM_AX "\n\t"
"push %%" _ASM_BP "\n\t"
UNWIND_HINT_UNDEFINED
- "mov %[in6], %%" _ASM_BP "\n\t"
+ "mov %%" _ASM_AX ", %%" _ASM_BP "\n\t"
+ "mov %[magic], %%eax\n\t"
"rep insb\n\t"
"pop %%" _ASM_BP "\n\t"
UNWIND_HINT_RESTORE
- : "=a" (out0), "=b" (*out1)
- : "a" (VMWARE_HYPERVISOR_MAGIC),
- "b" (cmd),
+ : "=&a" (out0), "=b" (*out1)
+ : "b" (cmd),
"c" (in2),
"d" (in3 | VMWARE_HYPERVISOR_PORT_HB),
"S" (in4),
"D" (in5),
- [in6] VMW_BP_CONSTRAINT (in6)
+ [in6] VMW_BP_CONSTRAINT(in6),
+ [magic] "i" (VMWARE_HYPERVISOR_MAGIC)
: "cc", "memory");
return out0;
}
diff --git a/arch/x86/kernel/acpi/cppc.c b/arch/x86/kernel/acpi/cppc.c
index bbade0da5130..174b734e8261 100644
--- a/arch/x86/kernel/acpi/cppc.c
+++ b/arch/x86/kernel/acpi/cppc.c
@@ -6,6 +6,8 @@
#include <linux/bitfield.h>
+#include <linux/limits.h>
+
#include <acpi/cppc_acpi.h>
#include <asm/msr.h>
#include <asm/processor.h>
@@ -45,10 +47,20 @@ bool cpc_ffh_supported(void)
return true;
}
+static bool cpc_ffh_reg_valid(const struct cpc_reg *reg)
+{
+ return reg->address <= U32_MAX && reg->bit_width &&
+ reg->bit_width <= 64 &&
+ reg->bit_offset <= 64 - reg->bit_width;
+}
+
int cpc_read_ffh(int cpunum, struct cpc_reg *reg, u64 *val)
{
int err;
+ if (!cpc_ffh_reg_valid(reg))
+ return -EINVAL;
+
err = rdmsrq_safe_on_cpu(cpunum, reg->address, val);
if (!err) {
u64 mask = GENMASK_ULL(reg->bit_offset + reg->bit_width - 1,
@@ -65,6 +77,9 @@ int cpc_write_ffh(int cpunum, struct cpc_reg *reg, u64 val)
u64 rd_val;
int err;
+ if (!cpc_ffh_reg_valid(reg))
+ return -EINVAL;
+
err = rdmsrq_safe_on_cpu(cpunum, reg->address, &rd_val);
if (!err) {
u64 mask = GENMASK_ULL(reg->bit_offset + reg->bit_width - 1,
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 000ebee2d247..4ced10c6ba0d 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -2088,14 +2088,12 @@ static void __init bhi_select_mitigation(void)
if (bhi_mitigation != BHI_MITIGATION_AUTO)
return;
- if (cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_HOST)) {
- if (cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL))
- bhi_mitigation = BHI_MITIGATION_ON;
- else
- bhi_mitigation = BHI_MITIGATION_VMEXIT_ONLY;
- } else {
+ if (cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL))
+ bhi_mitigation = BHI_MITIGATION_ON;
+ else if (cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_HOST))
+ bhi_mitigation = BHI_MITIGATION_VMEXIT_ONLY;
+ else
bhi_mitigation = BHI_MITIGATION_OFF;
- }
}
static void __init bhi_update_mitigation(void)
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index e3cfa0c10e92..ffd74a68671b 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -214,8 +214,8 @@ union l3_qos_abmc_cfg {
bw_src :12,
reserved1: 3,
is_clos : 1,
- cntr_id : 5,
- reserved : 9,
+ cntr_id :12,
+ reserved : 2,
cntr_en : 1,
cfg_en : 1;
} split;
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 3838e0a13d36..a6a9090c4808 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -470,8 +470,8 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)
r->mon.mbm_cntr_assignable = true;
r->mon.mbm_cntr_configurable = true;
cpuid_count(0x80000020, 5, &eax, &ebx, &ecx, &edx);
- r->mon.num_mbm_cntrs = (ebx & GENMASK(15, 0)) + 1;
- hw_res->mbm_cntr_assign_enabled = true;
+ /* cntr_id is 12 bits and can only encode 4096 counters. */
+ r->mon.num_mbm_cntrs = min((ebx & GENMASK(15, 0)) + 1, BIT(12));
}
r->mon_capable = true;
diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c
index 3c9f60d6ca5a..cb46da482c85 100644
--- a/arch/x86/kernel/nmi.c
+++ b/arch/x86/kernel/nmi.c
@@ -115,6 +115,14 @@ static void nmi_check_duration(struct nmiaction *action, u64 duration)
{
int remainder_ns, decimal_msecs;
+ /* sched_clock() went backwards? */
+ if ((s64)duration < 0) {
+ pr_warn_once("NMI handler (%ps): sched_clock() went backwards by %lld ns on CPU%d\n",
+ action->handler, -(s64)duration,
+ smp_processor_id());
+ return;
+ }
+
if (duration < nmi_longest_ns || duration < action->max_duration)
return;
diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c
index 252c5827d063..3b9b41820c84 100644
--- a/arch/x86/kernel/x86_init.c
+++ b/arch/x86/kernel/x86_init.c
@@ -37,7 +37,7 @@ static void iommu_shutdown_noop(void) { }
bool __init bool_x86_init_noop(void) { return false; }
void x86_op_int_noop(int cpu) { }
int set_rtc_noop(const struct timespec64 *now) { return -EINVAL; }
-void get_rtc_noop(struct timespec64 *now) { }
+void get_rtc_noop(struct timespec64 *now) { *now = (struct timespec64){0}; }
static __initconst const struct of_device_id of_cmos_match[] = {
{ .compatible = "motorola,mc146818" },
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index e56497089b59..2c2ff20f8ad0 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -2310,6 +2310,16 @@ static struct kvm_mmu_page *kvm_mmu_find_shadow_page(struct kvm *kvm,
continue;
}
+ /*
+ * Note! Unlike fully direct MMUs, direct SPs in indirect MMUs
+ * can have an arbitrary number of parents, e.g if the guest is
+ * mapping a huge PTE at many points in its page tables.
+ */
+ if (pte_list_count(&sp->parent_ptes) >= RMAP_RECYCLE_THRESHOLD) {
+ kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list);
+ break;
+ }
+
/* unsync and write-flooding only apply to indirect SPs. */
if (sp->role.direct)
goto out;
diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h
index 6e0153f23136..7454dac59b3a 100644
--- a/arch/x86/kvm/mmu/paging_tmpl.h
+++ b/arch/x86/kvm/mmu/paging_tmpl.h
@@ -339,6 +339,13 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,
gpa_t real_gpa;
gfn_t gfn;
+ /*
+ * KVM might sleep/resched when retrying the walk if the updating A/D
+ * fails because the gPTE changed between reading the gPTE and trying
+ * the CMPXHG to set A/D bits.
+ */
+ might_sleep();
+
trace_kvm_mmu_pagetable_walk(addr, access);
retry_walk:
walker->level = w->cpu_role.base.level;
@@ -479,8 +486,10 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,
addr, write_fault);
if (unlikely(ret < 0))
goto error;
- else if (ret)
+ else if (ret) {
+ cond_resched();
goto retry_walk;
+ }
}
return 1;
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 5165275128fc..a1450c55e946 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -5059,6 +5059,7 @@ static void nested_vmx_restore_host_state(struct kvm_vcpu *vcpu)
__func__, j, h.index, h.value);
goto vmabort;
}
+ break;
}
}
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index c76498d1a162..56a19a7260d3 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -6515,10 +6515,12 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
} else if (vcpu->arch.complete_userspace_io) {
writeback = false;
r = 0;
- } else if (r == EMULATION_RESTART)
+ } else if (r == EMULATION_RESTART) {
+ cond_resched();
goto restart;
- else
+ } else {
r = 1;
+ }
writeback:
if (writeback) {
@@ -9661,18 +9663,23 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
kvm_x86_call(set_efer)(vcpu, 0);
kvm_x86_call(update_exception_bitmap)(vcpu);
+ if (old_cr0 & X86_CR0_PG)
+ kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu);
+
/*
- * On the standard CR0/CR4/EFER modification paths, there are several
- * complex conditions determining whether the MMU has to be reset and/or
- * which PCIDs have to be flushed. However, CR0.WP and the paging-related
- * bits in CR4 and EFER are irrelevant if CR0.PG was '0'; and a reset+flush
- * is needed anyway if CR0.PG was '1' (which can only happen for INIT, as
- * CR0 will be '0' prior to RESET). So we only need to check CR0.PG here.
+ * Unconditionally reset the MMU, even though super strictly speaking,
+ * a MMU reset is only necessary if and only one of the page role bits
+ * is changing. All of the CR0/CR4/EFER bits effectively get covered
+ * by checking CR0.PG=1, as CR0.WP and the paging-related bits in CR4
+ * and EFER are irrelevant if CR0.PG was '0'. Subtly, if the vCPU was
+ * in guest mode (L2) or SMM (although emulating INIT in SMM _should_
+ * be impossible), an MMU reset is also necessary as "guest mode" and
+ * "smm" are both encoded in the shadow page role. As a result, the
+ * only scenarios where an MMU reset is *not* needed is if the vCPU is
+ * in Real Mode or Unpaged Protected Mode, and micro-optimizing those
+ * paths are not worth the risk of running with a stale MMU context.
*/
- if (old_cr0 & X86_CR0_PG) {
- kvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu);
- kvm_mmu_reset_context(vcpu);
- }
+ kvm_mmu_reset_context(vcpu);
/*
* Intel's SDM states that all TLB entries are flushed on INIT. AMD's
diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 0c4da320831c..53dcab39566f 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -320,13 +320,19 @@ static u16 allocate_global_asid(void)
lockdep_assert_held(&global_asid_lock);
- /* The previous allocation hit the edge of available address space */
- if (last_global_asid >= MAX_ASID_AVAILABLE - 1)
- reset_global_asid_space();
-
asid = find_next_zero_bit(global_asid_used, MAX_ASID_AVAILABLE, last_global_asid);
- if (asid >= MAX_ASID_AVAILABLE && !global_asid_available) {
+ /*
+ * Nothing is free between last_global_asid and MAX_ASID_AVAILABLE.
+ * Flush lazy-freed ASIDs and reset the search space.
+ */
+ if (asid >= MAX_ASID_AVAILABLE) {
+ reset_global_asid_space();
+ asid = find_next_zero_bit(global_asid_used, MAX_ASID_AVAILABLE,
+ last_global_asid);
+ }
+
+ if (asid >= MAX_ASID_AVAILABLE) {
/* This should never happen. */
VM_WARN_ONCE(1, "Unable to allocate global ASID despite %d available\n",
global_asid_available);
diff --git a/block/bio.c b/block/bio.c
index f95b63c0604a..90104e6abbcf 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1285,6 +1285,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter,
unsigned mem_align_mask, unsigned len_align_mask)
{
iov_iter_extraction_t flags = 0;
+ struct iov_iter saved_iter;
if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED)))
return -EIO;
@@ -1303,6 +1304,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter,
if (bio->bi_bdev && blk_queue_pci_p2pdma(bio->bi_bdev->bd_disk->queue))
flags |= ITER_ALLOW_P2PDMA;
+ saved_iter = *iter;
do {
ssize_t ret;
@@ -1320,6 +1322,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter,
bio_release_pages(bio, false);
bio_clear_flag(bio, BIO_PAGE_PINNED);
bio->bi_vcnt = 0;
+ *iter = saved_iter;
}
if (!bio->bi_vcnt)
return ret;
diff --git a/block/blk-iocost.c b/block/blk-iocost.c
index 7b8af094ab37..23445d701433 100644
--- a/block/blk-iocost.c
+++ b/block/blk-iocost.c
@@ -2851,6 +2851,9 @@ static void ioc_rqos_done(struct rq_qos *rqos, struct request *rq)
if (!ioc->enabled || !rq->alloc_time_ns || !rq->start_time_ns)
return;
+ if (unlikely(rq->rq_flags & RQF_FLUSH_SEQ))
+ return;
+
switch (req_op(rq)) {
case REQ_OP_READ:
pidx = QOS_RLAT;
diff --git a/block/blk-mq.c b/block/blk-mq.c
index 5a5dc66fc3b1..e21d69767fd4 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -3193,16 +3193,15 @@ void blk_mq_submit_bio(struct bio *bio)
blk_mq_bio_to_request(rq, bio, nr_segs);
+ if (bio_zone_write_plugging(bio))
+ blk_zone_write_plug_init_request(rq);
+
ret = blk_crypto_rq_get_keyslot(rq);
if (ret != BLK_STS_OK) {
- bio_endio_status(bio, ret);
- blk_mq_free_request(rq);
+ blk_mq_end_request(rq, ret);
return;
}
- if (bio_zone_write_plugging(bio))
- blk_zone_write_plug_init_request(rq);
-
if (op_is_flush(bio->bi_opf) && blk_insert_flush(rq))
return;
@@ -3485,8 +3484,10 @@ void blk_mq_free_rqs(struct blk_mq_tag_set *set, struct blk_mq_tags *tags,
if (blk_mq_is_shared_tags(set->flags))
drv_tags = set->shared_tags;
- else
+ else if (hctx_idx < set->nr_hw_queues)
drv_tags = set->tags[hctx_idx];
+ else
+ drv_tags = NULL;
if (tags->static_rqs && set->ops->exit_request) {
int i;
diff --git a/block/elevator.c b/block/elevator.c
index 2161b6eea680..66edf1a5768a 100644
--- a/block/elevator.c
+++ b/block/elevator.c
@@ -812,25 +812,29 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf,
* reference during concurrent disk deletion:
* update_nr_hwq_lock -> kn->active (via del_gendisk -> kobject_del)
* kn->active -> update_nr_hwq_lock (via this sysfs write path)
- *
- * Use the writer lock instead of the reader lock of update_nr_hwq_lock
- * to serialize the two-stage elevator switch steps in
- * elevator_change(): the core switch step under the elevator lock and
- * the elevator_change_done() step outside the elevator lock.
*/
- if (!down_write_trylock(&set->update_nr_hwq_lock)) {
+ if (!down_read_trylock(&set->update_nr_hwq_lock)) {
ret = -EBUSY;
goto out;
}
- if (!blk_queue_no_elv_switch(q)) {
- ret = elevator_change(q, &ctx);
- if (!ret)
- ret = count;
- } else {
- ret = -ENOENT;
+
+ /*
+ * The reader lock is shared, so use QUEUE_FLAG_NO_ELV_SWITCH to allow
+ * only one switch of this queue at a time, and none once the disk is
+ * going away.
+ */
+ if (test_and_set_bit(QUEUE_FLAG_NO_ELV_SWITCH, &q->queue_flags)) {
+ ret = -EBUSY;
+ goto out_unlock;
}
- up_write(&set->update_nr_hwq_lock);
+ ret = elevator_change(q, &ctx);
+ if (!ret)
+ ret = count;
+
+ blk_queue_flag_clear(QUEUE_FLAG_NO_ELV_SWITCH, q);
+out_unlock:
+ up_read(&set->update_nr_hwq_lock);
out:
if (ctx.type)
elevator_put(ctx.type);
diff --git a/block/ioctl.c b/block/ioctl.c
index 64b4e6c0f696..fb2f4b4d2505 100644
--- a/block/ioctl.c
+++ b/block/ioctl.c
@@ -835,8 +835,8 @@ long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg)
return compat_put_ulong(argp, bdev_nr_sectors(bdev));
/* The data is compatible, but the command number is different */
- case BLKBSZGET_32: /* get the logical block size (cf. BLKSSZGET) */
- return put_int(argp, bdev_logical_block_size(bdev));
+ case BLKBSZGET_32: /* get the soft block size (cf. BLKSSZGET) */
+ return put_int(argp, block_size(bdev));
case BLKBSZSET_32:
return blkdev_bszset(file, mode, argp);
case BLKGETSIZE64_32:
diff --git a/drivers/acpi/battery.c b/drivers/acpi/battery.c
index 670853ec3a4d..7169ac7e83f3 100644
--- a/drivers/acpi/battery.c
+++ b/drivers/acpi/battery.c
@@ -583,9 +583,8 @@ static int acpi_battery_get_info(struct acpi_battery *battery)
struct acpi_buffer buffer = { ACPI_ALLOCATE_BUFFER, NULL };
acpi_status status = AE_ERROR;
- status = acpi_evaluate_object(battery->device->handle,
- use_bix ? "_BIX":"_BIF",
- NULL, &buffer);
+ status = acpi_evaluate_object_exclusive(battery->device->handle,
+ use_bix ? "_BIX" : "_BIF", NULL, &buffer);
if (ACPI_FAILURE(status)) {
acpi_handle_info(battery->device->handle,
@@ -624,8 +623,7 @@ static int acpi_battery_get_state(struct acpi_battery *battery)
msecs_to_jiffies(cache_time)))
return 0;
- status = acpi_evaluate_object(battery->device->handle, "_BST",
- NULL, &buffer);
+ status = acpi_evaluate_object_exclusive(battery->device->handle, "_BST", NULL, &buffer);
if (ACPI_FAILURE(status)) {
acpi_handle_info(battery->device->handle,
"_BST evaluation failed: %s",
@@ -675,8 +673,8 @@ static int acpi_battery_set_alarm(struct acpi_battery *battery)
!test_bit(ACPI_BATTERY_ALARM_PRESENT, &battery->flags))
return -ENODEV;
- status = acpi_execute_simple_method(battery->device->handle, "_BTP",
- battery->alarm);
+ status = acpi_execute_simple_method_exclusive(battery->device->handle, "_BTP",
+ battery->alarm);
if (ACPI_FAILURE(status))
return -ENODEV;
diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c
index fef54fcd00b7..8aa0ce556863 100644
--- a/drivers/acpi/cppc_acpi.c
+++ b/drivers/acpi/cppc_acpi.c
@@ -36,6 +36,7 @@
#include <linux/delay.h>
#include <linux/iopoll.h>
#include <linux/ktime.h>
+#include <linux/mutex.h>
#include <linux/rwsem.h>
#include <linux/wait.h>
#include <linux/topology.h>
@@ -93,6 +94,16 @@ static DEFINE_PER_CPU(int, cpu_pcc_subspace_idx);
*/
static DEFINE_PER_CPU(struct cpc_desc *, cpc_desc_ptr);
+/* Protect immutable capability queries against descriptor removal. */
+static DEFINE_MUTEX(cpc_desc_lock);
+
+static void cpc_set_desc(unsigned int cpu, struct cpc_desc *desc)
+{
+ mutex_lock(&cpc_desc_lock);
+ per_cpu(cpc_desc_ptr, cpu) = desc;
+ mutex_unlock(&cpc_desc_lock);
+}
+
/* pcc mapped address + header size + offset within PCC subspace */
#define GET_PCC_VADDR(offs, pcc_ss_id) (pcc_data[pcc_ss_id]->pcc_channel->shmem + \
0x8 + (offs))
@@ -999,12 +1010,12 @@ int acpi_cppc_processor_probe(struct acpi_processor *pr)
}
/* Plug PSD data into this CPU's CPC descriptor. */
- per_cpu(cpc_desc_ptr, pr->id) = cpc_ptr;
+ cpc_set_desc(pr->id, cpc_ptr);
ret = kobject_init_and_add(&cpc_ptr->kobj, &cppc_ktype, &cpu_dev->kobj,
"acpi_cppc");
if (ret) {
- per_cpu(cpc_desc_ptr, pr->id) = NULL;
+ cpc_set_desc(pr->id, NULL);
kobject_put(&cpc_ptr->kobj);
goto out_free;
}
@@ -1056,6 +1067,8 @@ void acpi_cppc_processor_exit(struct acpi_processor *pr)
if (!cpc_ptr)
return;
+ cpc_set_desc(pr->id, NULL);
+
/* Free all the mapped sys mem areas for this CPU */
for (i = 2; i < cpc_ptr->num_entries; i++) {
addr = cpc_ptr->cpc_regs[i-2].sys_mem_vaddr;
@@ -1133,6 +1146,9 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val)
}
*val = 0;
+ if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
+ return cpc_read_ffh(cpu, reg, val);
+
size = GET_BIT_WIDTH(reg);
if (IS_ENABLED(CONFIG_HAS_IOPORT) &&
@@ -1160,8 +1176,6 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val)
}
else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY)
vaddr = reg_res->sys_mem_vaddr;
- else if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
- return cpc_read_ffh(cpu, reg, val);
else
return acpi_os_read_memory((acpi_physical_address)reg->address,
val, size);
@@ -1208,6 +1222,9 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val)
unsigned long flags;
bool locked = false;
+ if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
+ return cpc_write_ffh(cpu, reg, val);
+
size = GET_BIT_WIDTH(reg);
if (IS_ENABLED(CONFIG_HAS_IOPORT) &&
@@ -1233,8 +1250,6 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val)
}
else if (reg->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY)
vaddr = reg_res->sys_mem_vaddr;
- else if (reg->space_id == ACPI_ADR_SPACE_FIXED_HARDWARE)
- return cpc_write_ffh(cpu, reg, val);
else
return acpi_os_write_memory((acpi_physical_address)reg->address,
val, size);
@@ -1400,6 +1415,11 @@ static int cppc_set_reg_val(int cpu, enum cppc_regs reg_idx, u64 val)
reg = &cpc_desc->cpc_regs[reg_idx];
+ /* Integer 1 describes autonomous selection that is always enabled. */
+ if (reg_idx == AUTO_SEL_ENABLE && reg->type == ACPI_TYPE_INTEGER &&
+ reg->cpc_entry.int_value == 1)
+ return val == 1 ? 0 : -EOPNOTSUPP;
+
/* if a register is writeable, it must be a buffer and not null */
if ((reg->type != ACPI_TYPE_BUFFER) || IS_NULL_REG(®->cpc_entry.reg)) {
pr_debug("CPC register is not supported\n");
@@ -1613,6 +1633,9 @@ bool cppc_perf_ctrs_in_pcc_cpu(unsigned int cpu)
{
struct cpc_desc *cpc_desc = per_cpu(cpc_desc_ptr, cpu);
+ if (!cpc_desc)
+ return false;
+
return CPC_IN_PCC(&cpc_desc->cpc_regs[DELIVERED_CTR]) ||
CPC_IN_PCC(&cpc_desc->cpc_regs[REFERENCE_CTR]) ||
CPC_IN_PCC(&cpc_desc->cpc_regs[CTR_WRAP_TIME]);
@@ -1924,6 +1947,32 @@ int cppc_get_auto_sel(int cpu, bool *enable)
}
EXPORT_SYMBOL_GPL(cppc_get_auto_sel);
+/**
+ * cppc_auto_sel_is_immutable - Check for always-enabled autonomous selection.
+ * @cpu: CPU whose _CPC descriptor to check.
+ *
+ * Context: Process context.
+ * Return: true for Integer 1, false for a register or an absent descriptor.
+ */
+bool cppc_auto_sel_is_immutable(int cpu)
+{
+ struct cpc_desc *cpc_desc;
+ struct cpc_register_resource *reg;
+ bool immutable = false;
+
+ mutex_lock(&cpc_desc_lock);
+ cpc_desc = per_cpu(cpc_desc_ptr, cpu);
+ if (cpc_desc) {
+ reg = &cpc_desc->cpc_regs[AUTO_SEL_ENABLE];
+ immutable = reg->type == ACPI_TYPE_INTEGER &&
+ reg->cpc_entry.int_value == 1;
+ }
+ mutex_unlock(&cpc_desc_lock);
+
+ return immutable;
+}
+EXPORT_SYMBOL_GPL(cppc_auto_sel_is_immutable);
+
/**
* cppc_set_auto_sel - Write autonomous selection register.
* @cpu : CPU to which to write register.
diff --git a/drivers/acpi/utils.c b/drivers/acpi/utils.c
index d499b72574ab..ea8cb876c12d 100644
--- a/drivers/acpi/utils.c
+++ b/drivers/acpi/utils.c
@@ -10,6 +10,7 @@
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/slab.h>
#include <linux/init.h>
#include <linux/types.h>
@@ -243,6 +244,28 @@ acpi_extract_package(union acpi_object *package,
EXPORT_SYMBOL(acpi_extract_package);
+static DEFINE_MUTEX(acpi_exclusive_eval_lock);
+
+/**
+ * acpi_evaluate_object_exclusive - Evaluate an object without interleaving
+ * @handle: ACPI handle of the object or its scope.
+ * @pathname: Path of the object relative to @handle.
+ * @arguments: Arguments to pass, or NULL.
+ * @buffer: Buffer for the return value, or NULL.
+ *
+ * Same as acpi_evaluate_object(), but serialized with other exclusive
+ * evaluations. Use it only for methods known to break when interleaved.
+ */
+acpi_status acpi_evaluate_object_exclusive(acpi_handle handle, acpi_string pathname,
+ struct acpi_object_list *arguments,
+ struct acpi_buffer *buffer)
+{
+ guard(mutex)(&acpi_exclusive_eval_lock);
+
+ return acpi_evaluate_object(handle, pathname, arguments, buffer);
+}
+EXPORT_SYMBOL_GPL(acpi_evaluate_object_exclusive);
+
acpi_status
acpi_evaluate_integer(acpi_handle handle,
acpi_string pathname,
@@ -277,6 +300,16 @@ acpi_evaluate_integer(acpi_handle handle,
EXPORT_SYMBOL(acpi_evaluate_integer);
+acpi_status acpi_evaluate_integer_exclusive(acpi_handle handle, acpi_string pathname,
+ struct acpi_object_list *arguments,
+ unsigned long long *data)
+{
+ guard(mutex)(&acpi_exclusive_eval_lock);
+
+ return acpi_evaluate_integer(handle, pathname, arguments, data);
+}
+EXPORT_SYMBOL_GPL(acpi_evaluate_integer_exclusive);
+
int acpi_get_local_u64_address(acpi_handle handle, u64 *addr)
{
acpi_status status;
@@ -685,6 +718,14 @@ acpi_status acpi_execute_simple_method(acpi_handle handle, char *method,
}
EXPORT_SYMBOL(acpi_execute_simple_method);
+acpi_status acpi_execute_simple_method_exclusive(acpi_handle handle, char *method, u64 arg)
+{
+ guard(mutex)(&acpi_exclusive_eval_lock);
+
+ return acpi_execute_simple_method(handle, method, arg);
+}
+EXPORT_SYMBOL_GPL(acpi_execute_simple_method_exclusive);
+
/**
* acpi_evaluate_ej0: Evaluate _EJ0 method for hotplug operations
* @handle: ACPI device handle
diff --git a/drivers/ata/libata-pmp.c b/drivers/ata/libata-pmp.c
index 904f418387d1..43991e54eb08 100644
--- a/drivers/ata/libata-pmp.c
+++ b/drivers/ata/libata-pmp.c
@@ -482,6 +482,20 @@ static void sata_pmp_quirks(struct ata_port *ap)
if (link->pmp == 4)
link->flags |= ATA_LFLAG_DISABLED;
}
+ } else if (vendor == 0x1b4b && devid == 0x9705) {
+ /*
+ * Marvell 88SM9705 quirks
+ *
+ * 1-to-5 PMP with an extra SEMB port. It reports 6 ports and
+ * port 5 is the SEMB, which never answers SRST: EH spends about
+ * a minute per round retrying it, and disk discovery waits for
+ * that. Seen behind a Marvell 88SE9235 in the Synology DX1215
+ * and DX1222.
+ */
+ ata_for_each_link(link, ap, EDGE) {
+ if (link->pmp == 5)
+ link->flags |= ATA_LFLAG_DISABLED;
+ }
}
}
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index b86ac50f16b9..023de0e633b7 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -1984,6 +1984,18 @@ static int ata_scsi_translate(struct ata_device *dev, struct scsi_cmnd *cmd,
if (xlat_func(qc))
goto done;
+ /*
+ * A data-out request may contain pages pinned without write access.
+ * Reject a translated data-in command before issuing it, since its
+ * completion handler may write to the buffer even if the device fails.
+ */
+ if (unlikely(cmd->sc_data_direction == DMA_TO_DEVICE &&
+ ata_is_data(qc->tf.protocol) &&
+ !(qc->tf.flags & ATA_TFLAG_WRITE))) {
+ ata_scsi_set_sense(dev, cmd, ABORTED_COMMAND, 0x4b, 0x0a);
+ goto done;
+ }
+
return ata_scsi_qc_issue(ap, qc);
done:
@@ -2015,6 +2027,12 @@ static void ata_scsi_rbuf_fill(struct ata_device *dev, struct scsi_cmnd *cmd,
unsigned long flags;
unsigned int len;
+ /* A data-out request may contain pages pinned without write access. */
+ if (cmd->sc_data_direction == DMA_TO_DEVICE) {
+ ata_scsi_set_sense(dev, cmd, ABORTED_COMMAND, 0x4b, 0x0a);
+ return;
+ }
+
spin_lock_irqsave(&ata_scsi_rbuf_lock, flags);
memset(ata_scsi_rbuf, 0, ATA_SCSI_RBUF_SIZE);
@@ -3108,6 +3126,13 @@ static unsigned int atapi_xlat(struct ata_queued_cmd *qc)
qc->complete_fn = atapi_qc_complete;
+ /* atapi_fixup_inquiry() writes to the buffer on successful completion. */
+ if (scmd->sc_data_direction == DMA_TO_DEVICE &&
+ scmd->cmnd[0] == INQUIRY) {
+ ata_scsi_set_sense(dev, scmd, ABORTED_COMMAND, 0x4b, 0x0a);
+ return 1;
+ }
+
qc->tf.flags |= ATA_TFLAG_ISADDR | ATA_TFLAG_DEVICE;
if (scmd->sc_data_direction == DMA_TO_DEVICE) {
qc->tf.flags |= ATA_TFLAG_WRITE;
diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
index 1ef01ce18f4f..959e6594013a 100644
--- a/drivers/block/ublk_drv.c
+++ b/drivers/block/ublk_drv.c
@@ -5833,6 +5833,7 @@ static int ublk_ctrl_uring_cmd_permission(struct ublk_device *ub,
case UBLK_CMD_STOP_DEV:
case UBLK_CMD_ADD_DEV:
case UBLK_CMD_DEL_DEV:
+ case UBLK_CMD_DEL_DEV_ASYNC:
case UBLK_CMD_SET_PARAMS:
case UBLK_CMD_START_USER_RECOVERY:
case UBLK_CMD_END_USER_RECOVERY:
diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
index c8bfbf8435ac..10ba8b0b863b 100644
--- a/drivers/block/virtio_blk.c
+++ b/drivers/block/virtio_blk.c
@@ -1596,7 +1596,7 @@ static int virtblk_freeze_priv(struct virtio_device *vdev)
/* Ensure no requests in virtqueues before deleting vqs. */
memflags = blk_mq_freeze_queue(q);
- blk_mq_quiesce_queue_nowait(q);
+ blk_mq_quiesce_queue(q);
blk_mq_unfreeze_queue(q, memflags);
/* Ensure we don't receive any more interrupts */
diff --git a/drivers/bluetooth/btqca.c b/drivers/bluetooth/btqca.c
index 22b08ab05b82..002307000d3c 100644
--- a/drivers/bluetooth/btqca.c
+++ b/drivers/bluetooth/btqca.c
@@ -426,7 +426,8 @@ static int qca_tlv_check_data(struct hci_dev *hdev,
tlv_nvm->data[0] |= 0x80;
/* UART Baud Rate */
- if (soc_type >= QCA_WCN3991)
+ if (soc_type == QCA_WCN3950 ||
+ soc_type >= QCA_WCN3991)
tlv_nvm->data[1] = nvm_baud_rate;
else
tlv_nvm->data[2] = nvm_baud_rate;
diff --git a/drivers/char/hw_random/virtio-rng.c b/drivers/char/hw_random/virtio-rng.c
index 5e83ffa105e4..a9ff520975a7 100644
--- a/drivers/char/hw_random/virtio-rng.c
+++ b/drivers/char/hw_random/virtio-rng.c
@@ -190,6 +190,7 @@ static int probe_common(struct virtio_device *vdev)
err_find:
ida_free(&rng_index_ida, index);
err_ida:
+ vdev->priv = NULL;
kfree(vi);
return err;
}
@@ -198,6 +199,11 @@ static void remove_common(struct virtio_device *vdev)
{
struct virtrng_info *vi = vdev->priv;
+ if (!vi) {
+ virtio_reset_device(vdev);
+ return;
+ }
+
vi->hwrng_removed = true;
vi->data_avail = 0;
vi->data_idx = 0;
@@ -207,6 +213,7 @@ static void remove_common(struct virtio_device *vdev)
virtio_reset_device(vdev);
vdev->config->del_vqs(vdev);
ida_free(&rng_index_ida, vi->index);
+ vdev->priv = NULL;
kfree(vi);
}
diff --git a/drivers/clk/nuvoton/clk-ma35d1.c b/drivers/clk/nuvoton/clk-ma35d1.c
index 9f65d0623517..c914079cee2d 100644
--- a/drivers/clk/nuvoton/clk-ma35d1.c
+++ b/drivers/clk/nuvoton/clk-ma35d1.c
@@ -17,6 +17,7 @@
static DEFINE_SPINLOCK(ma35d1_lock);
#define PLL_MAX_NUM 5
+#define MA35D1_CLK_COUNT 238
/* Clock Control Registers Offset */
#define REG_CLK_PWRCTL 0x00
@@ -467,12 +468,12 @@ static int ma35d1_clocks_probe(struct platform_device *pdev)
int ret;
ma35d1_hw_data = devm_kzalloc(dev,
- struct_size(ma35d1_hw_data, hws, CLK_MAX_IDX),
+ struct_size(ma35d1_hw_data, hws, MA35D1_CLK_COUNT),
GFP_KERNEL);
if (!ma35d1_hw_data)
return -ENOMEM;
- ma35d1_hw_data->num = CLK_MAX_IDX;
+ ma35d1_hw_data->num = MA35D1_CLK_COUNT;
hws = ma35d1_hw_data->hws;
clk_base = devm_platform_ioremap_resource(pdev, 0);
@@ -543,6 +544,8 @@ static int ma35d1_clocks_probe(struct platform_device *pdev)
hws[HCLK3] = ma35d1_clk_fixed_factor(dev, "hclk3", "sysclk1_mux", 1, 2);
hws[PCLK3] = ma35d1_clk_fixed_factor(dev, "pclk3", "sysclk1_mux", 1, 2);
hws[PCLK4] = ma35d1_clk_fixed_factor(dev, "pclk4", "sysclk1_mux", 1, 2);
+ hws[PCLK3_DIV4096] = ma35d1_clk_fixed_factor(dev, "pclk3_div4096", "pclk3", 1, 4096);
+ hws[PCLK4_DIV4096] = ma35d1_clk_fixed_factor(dev, "pclk4_div4096", "pclk4", 1, 4096);
hws[USBPHY0] = ma35d1_clk_fixed("usbphy0", 480000000);
hws[USBPHY1] = ma35d1_clk_fixed("usbphy1", 480000000);
diff --git a/drivers/cpufreq/amd-pstate.c b/drivers/cpufreq/amd-pstate.c
index 6e00d200920a..28b9d4d062b9 100644
--- a/drivers/cpufreq/amd-pstate.c
+++ b/drivers/cpufreq/amd-pstate.c
@@ -1796,7 +1796,7 @@ EXPORT_SYMBOL_FOR_PSTATE_UT(amd_pstate_get_status);
int amd_pstate_update_status(const char *buf, size_t size)
{
- int mode_idx;
+ int cpu, mode_idx, ret = 0;
if (size > strlen("passive") || size < strlen("active"))
return -EINVAL;
@@ -1805,12 +1805,27 @@ int amd_pstate_update_status(const char *buf, size_t size)
if (mode_idx < 0)
return mode_idx;
- if (mode_state_machine[cppc_state][mode_idx]) {
- guard(mutex)(&amd_pstate_driver_lock);
- return mode_state_machine[cppc_state][mode_idx](mode_idx);
+ guard(mutex)(&amd_pstate_driver_lock);
+
+ if (!mode_state_machine[cppc_state][mode_idx])
+ return 0;
+
+ if (mode_idx == AMD_PSTATE_PASSIVE &&
+ !cpu_feature_enabled(X86_FEATURE_CPPC)) {
+ cpus_read_lock();
+ /* Check offline CPUs too, before changing or removing the driver. */
+ for_each_present_cpu(cpu) {
+ if (cppc_auto_sel_is_immutable(cpu)) {
+ ret = -EOPNOTSUPP;
+ break;
+ }
+ }
+ cpus_read_unlock();
+ if (ret)
+ return ret;
}
- return 0;
+ return mode_state_machine[cppc_state][mode_idx](mode_idx);
}
EXPORT_SYMBOL_FOR_PSTATE_UT(amd_pstate_update_status);
diff --git a/drivers/cpufreq/cppc_cpufreq.c b/drivers/cpufreq/cppc_cpufreq.c
index 5f20c4a7ba2f..782b97acf5dc 100644
--- a/drivers/cpufreq/cppc_cpufreq.c
+++ b/drivers/cpufreq/cppc_cpufreq.c
@@ -42,6 +42,7 @@ MODULE_PARM_DESC(fie_disabled, "Disable Frequency Invariance Engine (FIE)");
/* Frequency invariance support */
struct cppc_freq_invariance {
int cpu;
+ bool pcc_work_initialized;
struct irq_work irq_work;
struct kthread_work work;
struct cppc_perf_fb_ctrs prev_perf_fb_ctrs;
@@ -152,7 +153,7 @@ static struct scale_freq_data cppc_sftd_pcc = {
static void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy)
{
- struct scale_freq_data *sftd = &cppc_sftd;
+ struct scale_freq_data *sftd;
struct cppc_freq_invariance *cppc_fi;
int cpu, ret;
@@ -169,7 +170,7 @@ static void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy)
kthread_init_work(&cppc_fi->work, cppc_scale_freq_workfn);
init_irq_work(&cppc_fi->irq_work, cppc_irq_work);
- sftd = &cppc_sftd_pcc;
+ cppc_fi->pcc_work_initialized = true;
}
ret = cppc_get_perf_ctrs(cpu, &cppc_fi->prev_perf_fb_ctrs);
@@ -185,8 +186,15 @@ static void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy)
}
}
- /* Register for freq-invariance */
- topology_set_scale_freq_source(sftd, policy->cpus);
+ /* A shared policy may contain both PCC and non-PCC counters. */
+ for_each_cpu(cpu, policy->cpus) {
+ cppc_fi = &per_cpu(cppc_freq_inv, cpu);
+ if (cppc_fi->pcc_work_initialized)
+ sftd = &cppc_sftd_pcc;
+ else
+ sftd = &cppc_sftd;
+ topology_set_scale_freq_source(sftd, cpumask_of(cpu));
+ }
}
/*
@@ -209,11 +217,12 @@ static void cppc_cpufreq_cpu_fie_exit(struct cpufreq_policy *policy)
topology_clear_scale_freq_source(SCALE_FREQ_SOURCE_CPPC, policy->related_cpus);
for_each_cpu(cpu, policy->related_cpus) {
- if (!cppc_perf_ctrs_in_pcc_cpu(cpu))
- continue;
cppc_fi = &per_cpu(cppc_freq_inv, cpu);
+ if (!cppc_fi->pcc_work_initialized)
+ continue;
irq_work_sync(&cppc_fi->irq_work);
kthread_cancel_work_sync(&cppc_fi->work);
+ cppc_fi->pcc_work_initialized = false;
}
}
@@ -890,6 +899,7 @@ static ssize_t store_auto_select(struct cpufreq_policy *policy,
const char *buf, size_t count)
{
struct cppc_cpudata *cpu_data = policy->driver_data;
+ bool old_auto_sel = cpu_data->perf_ctrls.auto_sel;
bool val;
int ret;
@@ -918,8 +928,8 @@ static ssize_t store_auto_select(struct cpufreq_policy *policy,
if (ret) {
cpu_data->perf_ctrls.min_perf = old_min_perf;
cpu_data->perf_ctrls.max_perf = old_max_perf;
- cppc_set_auto_sel(policy->cpu, false);
- cpu_data->perf_ctrls.auto_sel = false;
+ cppc_set_auto_sel(policy->cpu, old_auto_sel);
+ cpu_data->perf_ctrls.auto_sel = old_auto_sel;
return ret;
}
}
diff --git a/drivers/cxl/core/features.c b/drivers/cxl/core/features.c
index ba6d2a5acb74..754fe5eb4b76 100644
--- a/drivers/cxl/core/features.c
+++ b/drivers/cxl/core/features.c
@@ -97,6 +97,7 @@ get_supported_features(struct cxl_features_state *cxlfs)
kvmalloc_flex(*entries, ent, count);
if (!entries)
return NULL;
+ entries->num_features = count;
struct cxl_mbox_get_sup_feats_out *mbox_out __free(kvfree) =
kvmalloc(cxl_mbox->payload_size, GFP_KERNEL);
@@ -174,7 +175,6 @@ get_supported_features(struct cxl_features_state *cxlfs)
start += num_entries;
} while (remain_feats);
- entries->num_features = count;
entries->num_user_features = user_feats;
return no_free_ptr(entries);
diff --git a/drivers/dma-buf/dma-buf-mapping.c b/drivers/dma-buf/dma-buf-mapping.c
index 833be519e1e6..ef0f22854d01 100644
--- a/drivers/dma-buf/dma-buf-mapping.c
+++ b/drivers/dma-buf/dma-buf-mapping.c
@@ -156,7 +156,7 @@ struct sg_table *dma_buf_phys_vec_to_sgt(struct dma_buf_attachment *attach,
phys_vec[i].paddr);
} else if (dma_use_iova(dma->state)) {
ret = dma_iova_link(attach->dev, dma->state,
- phys_vec[i].paddr, 0,
+ phys_vec[i].paddr, mapped_len,
phys_vec[i].len, dir,
DMA_ATTR_MMIO);
if (ret)
diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c
index 05a3b1f9e0c2..f67a93a37d48 100644
--- a/drivers/dma/qcom/bam_dma.c
+++ b/drivers/dma/qcom/bam_dma.c
@@ -1332,8 +1332,8 @@ static int bam_dma_probe(struct platform_device *pdev)
for (i = 0; i < bdev->num_channels; i++)
bam_channel_init(bdev, &bdev->channels[i], i);
- ret = devm_request_irq(bdev->dev, bdev->irq, bam_dma_irq,
- IRQF_TRIGGER_HIGH, "bam_dma", bdev);
+ ret = request_irq(bdev->irq, bam_dma_irq, IRQF_TRIGGER_NONE,
+ "bam_dma", bdev);
if (ret)
goto err_bam_channel_exit;
@@ -1366,7 +1366,7 @@ static int bam_dma_probe(struct platform_device *pdev)
ret = dma_async_device_register(&bdev->common);
if (ret) {
dev_err(bdev->dev, "failed to register dma async device\n");
- goto err_bam_channel_exit;
+ goto err_free_irq;
}
ret = of_dma_controller_register(pdev->dev.of_node, bam_dma_xlate,
@@ -1385,6 +1385,8 @@ static int bam_dma_probe(struct platform_device *pdev)
err_unregister_dma:
dma_async_device_unregister(&bdev->common);
+err_free_irq:
+ free_irq(bdev->irq, bdev);
err_bam_channel_exit:
for (i = 0; i < bdev->num_channels; i++)
tasklet_kill(&bdev->channels[i].vc.task);
@@ -1410,7 +1412,7 @@ static void bam_dma_remove(struct platform_device *pdev)
/* mask all interrupts for this execution environment */
writel_relaxed(0, bam_addr(bdev, 0, BAM_IRQ_SRCS_MSK_EE));
- devm_free_irq(bdev->dev, bdev->irq, bdev);
+ free_irq(bdev->irq, bdev);
for (i = 0; i < bdev->num_channels; i++) {
bam_dma_terminate_all(&bdev->channels[i].vc.chan);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
index 8b8acf98fdfe..53475f8a3ff1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
@@ -405,7 +405,11 @@ int amdgpu_atif_query_backlight_caps(struct amdgpu_atif *atif)
characteristics.max_input_signal;
atif->backlight_caps.ac_level = characteristics.ac_level;
atif->backlight_caps.dc_level = characteristics.dc_level;
- atif->backlight_caps.data_points = characteristics.number_of_points;
+ if (characteristics.number_of_points > ATIF_QBTC_MAX_DATA_POINTS)
+ pr_warn(FW_BUG "ATIF: backlight curve has %u points, using the first %u\n",
+ characteristics.number_of_points, ATIF_QBTC_MAX_DATA_POINTS);
+ atif->backlight_caps.data_points = min_t(u8, characteristics.number_of_points,
+ ATIF_QBTC_MAX_DATA_POINTS);
memcpy(atif->backlight_caps.luminance_data,
characteristics.data_points,
sizeof(atif->backlight_caps.luminance_data));
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index f462b40220eb..8e0b3db5c15e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -345,7 +345,8 @@ int amdgpu_amdkfd_gpuvm_map_bo_to_kernel(struct kgd_mem *mem, void **kptr,
u64 *size, u32 domain);
void amdgpu_amdkfd_gpuvm_unmap_bo_from_kernel(struct kgd_mem *mem);
-int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart);
+int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart,
+ struct amdkfd_process_info *process_info);
int amdgpu_amdkfd_gpuvm_restore_process_bos(void *process_info,
struct dma_fence __rcu **ef);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index c5c2a73ddd94..6c657d6060d6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -975,11 +975,11 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem,
ret = amdgpu_vm_lock_pd(vm, &exec, 0);
drm_exec_retry_on_contention(&exec);
if (unlikely(ret))
- goto unwind;
+ goto unlock;
ret = drm_exec_lock_obj(&exec, &bo[i]->tbo.base);
drm_exec_retry_on_contention(&exec);
if (unlikely(ret))
- goto unwind;
+ goto unlock;
}
bo_va = amdgpu_vm_bo_find(vm, bo[i]);
@@ -1005,6 +1005,8 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem,
return 0;
+unlock:
+ drm_exec_fini(&exec);
unwind:
for (; i >= 0; i--) {
if (!attachment[i])
@@ -2261,11 +2263,13 @@ int amdgpu_amdkfd_gpuvm_sync_memory(
* amdgpu_amdkfd_map_gtt_bo_to_gart - Map BO to GART and increment reference count
* @bo: Buffer object to be mapped
* @bo_gart: Return bo reference
+ * @process_info: KFD process info of the queue the BO is for
*
* Before return, bo reference count is incremented. To release the reference and unpin/
* unmap the BO, call amdgpu_amdkfd_free_kernel_mem.
*/
-int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart)
+int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo_gart,
+ struct amdkfd_process_info *process_info)
{
int ret;
@@ -2287,8 +2291,8 @@ int amdgpu_amdkfd_map_gtt_bo_to_gart(struct amdgpu_bo *bo, struct amdgpu_bo **bo
goto err_map_bo_gart_failed;
}
- amdgpu_amdkfd_remove_eviction_fence(
- bo, bo->vm_bo->vm->process_info->eviction_fence);
+ /* bo->vm_bo can be the VM of any file which opened the BO */
+ amdgpu_amdkfd_remove_eviction_fence(bo, process_info->eviction_fence);
amdgpu_bo_unreserve(bo);
@@ -2473,6 +2477,9 @@ static int import_obj_create(struct amdgpu_device *adev,
err_remove_mem:
remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info);
+ amdgpu_sync_free(&(*mem)->sync);
+ mutex_destroy(&(*mem)->lock);
+ dma_buf_put(dma_buf);
drm_vma_node_revoke(&obj->vma_node, drm_priv);
err_free_mem:
kfree(*mem);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 86a3dc3655b3..a03a15cfca96 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -149,7 +149,12 @@ static int amdgpu_cs_p1_user_fence(struct amdgpu_cs_parser *p,
if (size != PAGE_SIZE || data->offset > (size - 8))
return -EINVAL;
- if (amdgpu_ttm_tt_get_usermm(p->uf_bo->tbo.ttm))
+ if (amdgpu_ttm_tt_is_userptr(p->uf_bo->tbo.ttm))
+ return -EINVAL;
+
+ /* The GPU writes the user fence through the GART */
+ if (p->uf_bo->preferred_domains != AMDGPU_GEM_DOMAIN_GTT &&
+ p->uf_bo->allowed_domains != AMDGPU_GEM_DOMAIN_GTT)
return -EINVAL;
*offset = data->offset;
@@ -188,18 +193,22 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p,
int ret;
int i;
+ if (cs->in.flags)
+ return -EINVAL;
+
chunk_array = memdup_array_user(u64_to_user_ptr(cs->in.chunks),
cs->in.num_chunks,
sizeof(uint64_t));
if (IS_ERR(chunk_array))
return PTR_ERR(chunk_array);
- p->nchunks = cs->in.num_chunks;
- p->chunks = kvmalloc_objs(struct amdgpu_cs_chunk, p->nchunks);
+ /* parser_fini() walks nchunks chunks, set it once they exist */
+ p->chunks = kvmalloc_objs(struct amdgpu_cs_chunk, cs->in.num_chunks);
if (!p->chunks) {
ret = -ENOMEM;
goto free_chunk;
}
+ p->nchunks = cs->in.num_chunks;
for (i = 0; i < p->nchunks; i++) {
struct drm_amdgpu_cs_chunk __user *chunk_ptr = NULL;
@@ -1071,7 +1080,7 @@ static int amdgpu_cs_patch_ibs(struct amdgpu_cs_parser *p,
if (r)
return r;
- kptr += va_start - (m->start * AMDGPU_GPU_PAGE_SIZE);
+ kptr += va_start - (m->start * AMDGPU_GPU_PAGE_SIZE) + m->offset;
if (ring->funcs->parse_cs) {
memcpy(ib->ptr, kptr, ib->length_dw * 4);
@@ -1309,10 +1318,8 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
fence = &p->jobs[i]->base.s_fence->scheduled;
dma_fence_get(fence);
r = drm_sched_job_add_dependency(&leader->base, fence);
- if (r) {
- dma_fence_put(fence);
+ if (r)
return r;
- }
}
if (p->gang_size > 1) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 5b72e4afb650..948886fffb4a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -5515,7 +5515,21 @@ static void amdgpu_device_halt_activities(struct amdgpu_device *adev,
struct amdgpu_device *tmp_adev = NULL;
int i;
- /* block all schedulers and reset given job's ring */
+ /* Stop job submission first, the preparation below can take seconds */
+ list_for_each_entry(tmp_adev, device_list, reset_list) {
+ for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
+ struct amdgpu_ring *ring = tmp_adev->rings[i];
+
+ if (!amdgpu_ring_sched_ready(ring))
+ continue;
+
+ drm_sched_wqueue_stop(&ring->sched);
+
+ if (need_emergency_restart)
+ amdgpu_job_stop_all_jobs_on_sched(&ring->sched);
+ }
+ }
+
list_for_each_entry(tmp_adev, device_list, reset_list) {
amdgpu_device_set_mp1_state(tmp_adev);
@@ -5551,18 +5565,6 @@ static void amdgpu_device_halt_activities(struct amdgpu_device *adev,
amdgpu_ras_suspend(tmp_adev);
amdgpu_userq_pre_reset(tmp_adev);
-
- for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
- struct amdgpu_ring *ring = tmp_adev->rings[i];
-
- if (!amdgpu_ring_sched_ready(ring))
- continue;
-
- drm_sched_wqueue_stop(&ring->sched);
-
- if (need_emergency_restart)
- amdgpu_job_stop_all_jobs_on_sched(&ring->sched);
- }
atomic_inc(&tmp_adev->gpu_reset_counter);
}
}
@@ -5797,8 +5799,10 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
amdgpu_device_halt_activities(adev, job, reset_context, &device_list,
hive, need_emergency_restart);
- if (need_emergency_restart)
- goto skip_sched_resume;
+ if (need_emergency_restart) {
+ amdgpu_device_gpu_resume(adev, &device_list, need_emergency_restart);
+ goto reset_unlock;
+ }
/*
* Must check guilty signal here since after this point all old
* HW fences are force signaled.
@@ -5808,18 +5812,17 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
if (job && (dma_fence_get_status(&job->hw_fence->base) > 0)) {
job_signaled = true;
dev_info(adev->dev, "Guilty job already signaled, skipping HW reset");
- goto skip_hw_reset;
+ goto post_resume;
}
r = amdgpu_device_asic_reset(adev, &device_list, reset_context);
if (r)
goto reset_unlock;
-skip_hw_reset:
+post_resume:
+ amdgpu_device_gpu_resume(adev, &device_list, need_emergency_restart);
r = amdgpu_device_sched_resume(&device_list, reset_context, job_signaled);
if (r)
goto reset_unlock;
-skip_sched_resume:
- amdgpu_device_gpu_resume(adev, &device_list, need_emergency_restart);
reset_unlock:
amdgpu_device_recovery_put_reset_lock(adev, &device_list);
amdgpu_ras_post_reset(adev, &device_list);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
index 7ab2ad2c5ba9..6d9a3e3b3546 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
@@ -1402,6 +1402,40 @@ static int amdgpu_discovery_sysfs_recurse(struct amdgpu_device *adev,
return 0;
}
+#define list_to_kobj(el) container_of(el, struct kobject, entry)
+
+static void amdgpu_discovery_sysfs_update_harvest(struct amdgpu_device *adev)
+{
+ struct ip_discovery_top *ip_top = adev->discovery.ip_top;
+ struct ip_die_entry *ip_die_entry;
+ struct list_head *el_die, *el_hw_id, *el_hw_inst;
+ struct ip_hw_id *hw_id;
+ struct kset *die_kset;
+ struct ip_hw_instance *ip_inst;
+
+ if (!ip_top)
+ return;
+
+ die_kset = &ip_top->die_kset;
+
+ guard(spinlock)(&die_kset->list_lock);
+ list_for_each(el_die, &die_kset->list) {
+ ip_die_entry = to_ip_die_entry(list_to_kobj(el_die));
+
+ list_for_each(el_hw_id, &ip_die_entry->ip_kset.list) {
+ hw_id = to_ip_hw_id(list_to_kobj(el_hw_id));
+
+ list_for_each(el_hw_inst, &hw_id->hw_id_kset.list) {
+ ip_inst = to_ip_hw_instance(list_to_kobj(el_hw_inst));
+ ip_inst->harvest =
+ amdgpu_discovery_get_harvest_info(
+ adev, ip_inst->hw_id,
+ ip_inst->num_instance);
+ }
+ }
+ }
+}
+
static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev)
{
uint8_t *discovery_bin = adev->discovery.bin;
@@ -1412,9 +1446,13 @@ static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev)
if (!discovery_bin)
return -EINVAL;
- /* If early init already created sysfs in standalone mode, skip normal init */
- if (adev->discovery.ip_top && adev->discovery.ip_top->standalone_mode)
+ /* If early init already created sysfs in standalone mode, update harvest info
+ * now that adev is fully initialized, then skip the rest of normal init.
+ */
+ if (adev->discovery.ip_top && adev->discovery.ip_top->standalone_mode) {
+ amdgpu_discovery_sysfs_update_harvest(adev);
return 0;
+ }
ip_top = kzalloc_obj(*ip_top);
if (!ip_top)
@@ -1469,8 +1507,6 @@ static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev)
/* -------------------------------------------------- */
-#define list_to_kobj(el) container_of(el, struct kobject, entry)
-
static void amdgpu_discovery_sysfs_ip_hw_free(struct ip_hw_id *ip_hw_id)
{
struct list_head *el, *tmp;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
index 9adf3eed8822..ccadbcf38d3f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
@@ -547,6 +547,11 @@ amdgpu_dma_buf_move_notify(struct dma_buf_attachment *attach)
continue;
}
+ if (!amdgpu_vm_ready(vm)) {
+ dma_resv_unlock(resv);
+ continue;
+ }
+
/* Reserve fences for two SDMA page table updates */
r = dma_resv_reserve_fences(resv, 2);
if (!r)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index afd8502a2bc4..cd05921aede5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -1033,6 +1033,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data,
struct drm_file *filp)
{
+ struct amdgpu_device *adev = drm_to_adev(dev);
struct drm_amdgpu_gem_op *args = data;
struct drm_gem_object *gobj;
struct amdgpu_vm_bo_base *base;
@@ -1139,7 +1140,8 @@ int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data,
amdgpu_vm_bo_va_for_each_valid_mapping(bo_va, mapping) {
if (num_mappings < args->num_entries) {
- vm_entries[num_mappings].addr = mapping->start * AMDGPU_GPU_PAGE_SIZE;
+ vm_entries[num_mappings].addr =
+ amdgpu_gmc_sign_extend(mapping->start * AMDGPU_GPU_PAGE_SIZE);
vm_entries[num_mappings].size = (mapping->last - mapping->start + 1) * AMDGPU_GPU_PAGE_SIZE;
vm_entries[num_mappings].offset = mapping->offset;
vm_entries[num_mappings].flags = mapping->flags;
@@ -1149,7 +1151,8 @@ int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data,
amdgpu_vm_bo_va_for_each_invalid_mapping(bo_va, mapping) {
if (num_mappings < args->num_entries) {
- vm_entries[num_mappings].addr = mapping->start * AMDGPU_GPU_PAGE_SIZE;
+ vm_entries[num_mappings].addr =
+ amdgpu_gmc_sign_extend(mapping->start * AMDGPU_GPU_PAGE_SIZE);
vm_entries[num_mappings].size = (mapping->last - mapping->start + 1) * AMDGPU_GPU_PAGE_SIZE;
vm_entries[num_mappings].offset = mapping->offset;
vm_entries[num_mappings].flags = mapping->flags;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
index 35eb87591740..37cf1b808950 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
@@ -1485,7 +1485,8 @@ int amdgpu_ras_reset_error_count(struct amdgpu_device *adev,
return -EOPNOTSUPP;
}
- if (!amdgpu_ras_is_supported(adev, block))
+ if (!amdgpu_ras_is_supported(adev, block) ||
+ !amdgpu_ras_mgr_get_debug_mode(adev))
return -EOPNOTSUPP;
if (amdgpu_sriov_vf(adev))
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
index d6ae9974c952..4f31021b2152 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
@@ -447,10 +447,8 @@ int amdgpu_sync_push_to_job(struct amdgpu_sync *sync, struct amdgpu_job *job)
dma_fence_get(f);
r = drm_sched_job_add_dependency(&job->base, f);
- if (r) {
- dma_fence_put(f);
+ if (r)
return r;
- }
}
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
index b3dc5dd49a64..053ead0d690f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
@@ -365,6 +365,8 @@ static int amdgpu_userq_unmap_helper(struct amdgpu_usermode_queue *queue)
adev->userq_funcs[queue->queue_type];
int r;
+ /* The reset changes the queue state and the MES is unusable meanwhile */
+ guard(rwsem_read)(&adev->reset_domain->sem);
if ((queue->state == AMDGPU_USERQ_STATE_MAPPED) ||
(queue->state == AMDGPU_USERQ_STATE_PREEMPTED)) {
trace_amdgpu_userq_state_start(queue);
@@ -391,6 +393,8 @@ static int amdgpu_userq_map_helper(struct amdgpu_usermode_queue *queue)
adev->userq_funcs[queue->queue_type];
int r;
+ /* The reset changes the queue state and the MES is unusable meanwhile */
+ guard(rwsem_read)(&adev->reset_domain->sem);
if (queue->state == AMDGPU_USERQ_STATE_UNMAPPED) {
trace_amdgpu_userq_state_start(queue);
@@ -751,12 +755,7 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args)
if (!adev->userq_halt_for_enforce_isolation ||
((queue->queue_type != AMDGPU_HW_IP_GFX) &&
(queue->queue_type != AMDGPU_HW_IP_COMPUTE))) {
- /* Serialize the map against an in-progress GPU reset (MES is
- * unresponsive during recovery), matching amdgpu_userq_detach_doorbell().
- */
- down_read(&adev->reset_domain->sem);
r = amdgpu_userq_map_helper(queue);
- up_read(&adev->reset_domain->sem);
if (r) {
drm_file_err(uq_mgr->file, "Failed to map Queue\n");
trace_amdgpu_userq_create_end(queue, r);
@@ -973,7 +972,11 @@ static int amdgpu_userq_validate_vm(void *param, struct amdgpu_bo *bo)
return ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
}
-/* Handle all BOs on the invalidated list, validate them and update the PTs */
+/*
+ * Handle all individual BOs which were moved, lock and validate them and
+ * update the PTs. An evicted individual BO lands on the needs_update list,
+ * the evicted one only holds imported DMA-bufs of compute VMs.
+ */
static int
amdgpu_userq_bo_validate(struct amdgpu_device *adev, struct drm_exec *exec,
struct amdgpu_vm *vm)
@@ -984,8 +987,8 @@ amdgpu_userq_bo_validate(struct amdgpu_device *adev, struct drm_exec *exec,
int ret;
spin_lock(&vm->individual_lock);
- while (!list_empty(&vm->individual.evicted)) {
- bo_va = list_first_entry(&vm->individual.evicted,
+ while (!list_empty(&vm->individual.needs_update)) {
+ bo_va = list_first_entry(&vm->individual.needs_update,
struct amdgpu_bo_va,
base.vm_status);
bo = bo_va->base.bo;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index fc3a8d38d514..c07f1115f654 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -269,19 +269,27 @@ static void amdgpu_vm_bo_reset_state_machine(struct amdgpu_vm *vm)
/*
* Don't use list splice here, we need the special handling for the root
- * PD and set the moved flag appropriately.
+ * PD. All mappings must be written again, also of the vm_bos which are
+ * already waiting for an update of some of them, e.g. after a map with
+ * a delayed update, so set the moved flag for everything that needs an
+ * update.
*/
amdgpu_vm_assert_locked(vm);
list_for_each_entry_safe(vm_bo, tmp, &vm->kernel.idle, vm_status)
amdgpu_vm_bo_needs_update(vm_bo);
+ list_for_each_entry(vm_bo, &vm->kernel.needs_update, vm_status)
+ vm_bo->moved = true;
+
list_for_each_entry_safe(vm_bo, tmp, &vm->always_valid.idle, vm_status)
amdgpu_vm_bo_needs_update(vm_bo);
+ list_for_each_entry(vm_bo, &vm->always_valid.needs_update, vm_status)
+ vm_bo->moved = true;
spin_lock(&vm->individual_lock);
- list_for_each_entry_safe(vm_bo, tmp, &vm->individual.idle, vm_status) {
- vm_bo->moved = true;
+ list_for_each_entry_safe(vm_bo, tmp, &vm->individual.idle, vm_status)
list_move(&vm_bo->vm_status, &vm->individual.needs_update);
- }
+ list_for_each_entry(vm_bo, &vm->individual.needs_update, vm_status)
+ vm_bo->moved = true;
spin_unlock(&vm->individual_lock);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
index eb8cf0d9d9c5..652266e93766 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
@@ -234,7 +234,6 @@ static int amdgpu_vm_sdma_update(struct amdgpu_vm_update_params *p,
dma_fence_get(fence);
r = drm_sched_job_add_dependency(&p->job->base, fence);
if (r) {
- dma_fence_put(fence);
dma_resv_iter_end(&cursor);
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
index f2977fe6d824..04b18056a906 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
@@ -1705,8 +1705,9 @@ static void dce_v10_0_afmt_setmode(struct drm_encoder *encoder,
WREG32(mmAFMT_RAMP_CONTROL2 + dig->afmt->offset, 0x00000001);
WREG32(mmAFMT_RAMP_CONTROL3 + dig->afmt->offset, 0x00000001);
- /* enable audio after to setting up hw */
- dce_v10_0_audio_enable(adev, dig->afmt->pin, true);
+ /* Only enable audio if the display supports it */
+ if (connector->display_info.has_audio)
+ dce_v10_0_audio_enable(adev, dig->afmt->pin, true);
}
static void dce_v10_0_afmt_enable(struct drm_encoder *encoder, bool enable)
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
index c68de0fe1d7d..585b0a50279f 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
@@ -1748,8 +1748,9 @@ static void dce_v6_0_afmt_setmode(struct drm_encoder *encoder,
dce_v6_0_audio_dp_enable(encoder, 1);
}
- /* enable audio after setting up hw */
- dce_v6_0_audio_enable(adev, dig->afmt->pin, true);
+ /* Only enable audio if the display supports it */
+ if (connector->display_info.has_audio)
+ dce_v6_0_audio_enable(adev, dig->afmt->pin, true);
}
static void dce_v6_0_afmt_enable(struct drm_encoder *encoder, bool enable)
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
index c3906270f25e..1390efa4ea51 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
@@ -1653,8 +1653,9 @@ static void dce_v8_0_afmt_setmode(struct drm_encoder *encoder,
WREG32(mmAFMT_RAMP_CONTROL2 + offset, 0x00000001);
WREG32(mmAFMT_RAMP_CONTROL3 + offset, 0x00000001);
- /* enable audio after setting up hw */
- dce_v8_0_audio_enable(adev, dig->afmt->pin, true);
+ /* Only enable audio if the display supports it */
+ if (connector->display_info.has_audio)
+ dce_v8_0_audio_enable(adev, dig->afmt->pin, true);
}
static void dce_v8_0_afmt_enable(struct drm_encoder *encoder, bool enable)
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
index d69c43f1e194..60debaa23762 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
@@ -4080,6 +4080,7 @@ static void gfx_v12_0_update_spm_vmid(struct amdgpu_device *adev,
{
u32 reg, data;
+ amdgpu_gfx_off_ctrl(adev, false);
reg = SOC15_REG_OFFSET(GC, 0, regRLC_SPM_MC_CNTL);
if (amdgpu_sriov_is_pp_one_vf(adev))
data = RREG32_NO_KIQ(reg);
@@ -4093,6 +4094,7 @@ static void gfx_v12_0_update_spm_vmid(struct amdgpu_device *adev,
WREG32_SOC15_NO_KIQ(GC, 0, regRLC_SPM_MC_CNTL, data);
else
WREG32_SOC15(GC, 0, regRLC_SPM_MC_CNTL, data);
+ amdgpu_gfx_off_ctrl(adev, true);
if (ring
&& amdgpu_sriov_is_pp_one_vf(adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
index 333e9c30c091..535c84e55558 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
@@ -168,8 +168,19 @@ static int ih_v6_0_toggle_ring_interrupts(struct amdgpu_device *adev,
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0));
if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
- if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ /*
+ * PSP programming of IH_RB_CNTL is slow enough for MSI to
+ * arrive before this call returns. Publish ih->enabled first
+ * so amdgpu_ih_process() consumes that IV and rearms.
+ */
+ if (enable) {
+ ih->enabled = true;
+ smp_wmb();
+ }
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) {
+ ih->enabled = false;
return -ETIMEDOUT;
+ }
} else {
WREG32(ih_regs->ih_rb_cntl, tmp);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
index c57bcb9a98b1..10bed6f38f30 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
@@ -218,8 +218,10 @@ static int mes_v11_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes,
r = amdgpu_fence_wait_polling(ring,
seq - ring->fence_drv.num_fences_mask,
timeout);
- if (r < 1)
+ if (r < 1) {
+ r = -ETIMEDOUT;
goto error_undo;
+ }
api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off);
api_status->api_completion_fence_addr = status_gpu_addr;
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
index ee72afdb0a1d..bb9892d1bd05 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v12_0.c
@@ -195,8 +195,10 @@ static int mes_v12_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes,
r = amdgpu_fence_wait_polling(ring,
seq - ring->fence_drv.num_fences_mask,
timeout);
- if (r < 1)
+ if (r < 1) {
+ r = -ETIMEDOUT;
goto error_undo;
+ }
api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off);
api_status->api_completion_fence_addr = status_gpu_addr;
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
index 4cd325149b63..bed0eb4b6ec3 100644
--- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
@@ -161,13 +161,52 @@ static int navi10_ih_toggle_ring_interrupts(struct amdgpu_device *adev,
tmp = RREG32(ih_regs->ih_rb_cntl);
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_ENABLE, (enable ? 1 : 0));
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_GPU_TS_ENABLE, 1);
+
+ if (enable) {
+ /* Unset the CLEAR_OVERFLOW bit to make sure the next step
+ * is switching the bit from 0 to 1
+ */
+ tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 0);
+ if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ return -ETIMEDOUT;
+ } else {
+ WREG32_NO_KIQ(ih_regs->ih_rb_cntl, tmp);
+ }
+
+ /* Clear RB_OVERFLOW bit */
+ tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 1);
+ if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ return -ETIMEDOUT;
+ } else {
+ WREG32_NO_KIQ(ih_regs->ih_rb_cntl, tmp);
+ }
+
+ /* Unset the CLEAR_OVERFLOW bit immediately so new overflows
+ * can be detected.
+ */
+ tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_CLEAR, 0);
+ }
+
/* enable_intr field is only valid in ring0 */
if (ih == &adev->irq.ih)
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0));
if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) {
- if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp))
+ /*
+ * PSP programming of IH_RB_CNTL is slow enough for MSI to
+ * arrive before this call returns. Publish ih->enabled first
+ * so amdgpu_ih_process() consumes that IV and rearms.
+ */
+ if (enable) {
+ ih->enabled = true;
+ smp_wmb();
+ }
+ if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) {
+ ih->enabled = false;
return -ETIMEDOUT;
+ }
} else {
WREG32(ih_regs->ih_rb_cntl, tmp);
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
index 87fb42d3c713..64673007ccde 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
@@ -303,7 +303,8 @@ static int init_user_queue(struct process_queue_manager *pqm,
}
retval = amdgpu_amdkfd_map_gtt_bo_to_gart(q_properties->wptr_bo,
- &(*q)->wptr_bo_gart);
+ &(*q)->wptr_bo_gart,
+ pqm->process->kgd_process_info);
if (retval) {
pr_err("Failed to map wptr bo to GART\n");
goto free_gang_ctx_bo;
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_freesync.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_freesync.c
index 7e484adb4120..a51cf46d0e44 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_freesync.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_freesync.c
@@ -128,6 +128,14 @@ void amdgpu_dm_get_freesync_config_for_crtc(
vrefresh >= aconnector->min_vfreq &&
vrefresh <= aconnector->max_vfreq;
+ /*
+ * DC reads allow_freesync while validating the new state, for example
+ * to keep SubVP off a single FreeSync display, but it is otherwise set
+ * only on the first flip after the commit. Set it here, so that a
+ * stream created by this commit is not validated without it.
+ */
+ new_crtc_state->stream->allow_freesync = new_crtc_state->vrr_supported;
+
if (new_crtc_state->vrr_supported) {
new_crtc_state->stream->ignore_msa_timing_param = true;
fs_vid_mode = new_crtc_state->freesync_config.state == VRR_STATE_ACTIVE_FIXED;
@@ -154,6 +162,18 @@ void amdgpu_dm_get_freesync_config_for_crtc(
}
EXPORT_IF_KUNIT(amdgpu_dm_get_freesync_config_for_crtc);
+#define DP_HDR_SDP_EOTF_IDX 2
+
+static bool dm_dp_in_hdr_mode(const struct dc_stream_state *stream)
+{
+ const struct dc_info_packet *hdr = &stream->hdr_static_metadata;
+
+ if (!dc_is_dp_signal(stream->signal) || !hdr->valid)
+ return false;
+
+ return hdr->sb[DP_HDR_SDP_EOTF_IDX] == HDMI_EOTF_SMPTE_ST2084;
+}
+
void amdgpu_dm_update_freesync_state_on_stream(
struct amdgpu_display_manager *dm,
struct dm_crtc_state *new_crtc_state,
@@ -169,6 +189,7 @@ void amdgpu_dm_update_freesync_state_on_stream(
bool pack_sdp_v1_3 = false;
struct amdgpu_dm_connector *aconn;
enum vrr_packet_type packet_type = PACKET_TYPE_VRR;
+ enum color_transfer_func app_tf;
if (!new_stream)
return;
@@ -220,12 +241,16 @@ void amdgpu_dm_update_freesync_state_on_stream(
&new_stream->adaptive_sync_infopacket);
}
+ app_tf = dm_dp_in_hdr_mode(new_stream) ?
+ TRANSFER_FUNC_PQ2084 :
+ TRANSFER_FUNC_UNKNOWN;
+
mod_freesync_build_vrr_infopacket(
dm->freesync_module,
new_stream,
&vrr_params,
packet_type,
- TRANSFER_FUNC_UNKNOWN,
+ app_tf,
&vrr_infopacket,
pack_sdp_v1_3);
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c
index 6ee3da89d058..135eeef68ac4 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn401/dcn401_clk_mgr.c
@@ -538,6 +538,9 @@ static void dcn401_update_clocks_update_dtb_dto(struct clk_mgr_internal *clk_mgr
otg_master = resource_get_otg_master_for_stream(
&context->res_ctx, context->streams[i]);
ASSERT(otg_master);
+ if (!otg_master)
+ continue;
+
ASSERT(otg_master->clock_source);
ASSERT(otg_master->clock_source->funcs->program_pix_clk);
ASSERT(otg_master->stream_res.pix_clk_params.controller_id >= CONTROLLER_ID_D0);
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc.c b/drivers/gpu/drm/amd/display/dc/core/dc.c
index 519ac878ada1..60b903edff2f 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc.c
@@ -2452,6 +2452,8 @@ static enum dc_status dc_commit_state_no_check(struct dc *dc, struct dc_state *c
* update pending complete before optimizing bandwidth.
*/
hwss_wait_for_odm_update_pending_complete(dc, context);
+ /* No DET may stay allocated to a HUBP without a plane (e.g. left over from VBIOS) */
+ hwss_reset_unused_det(dc, context);
/* pplib is notified if disp_num changed */
dc->hwss.optimize_bandwidth(dc, context);
/* Need to do otg sync again as otg could be out of sync due to otg
@@ -4885,8 +4887,10 @@ static void commit_planes_for_stream(struct dc *dc,
dc_exit_ips_for_hw_access(dc);
dc_z10_restore(dc);
- if (update_type == UPDATE_TYPE_FULL && dc->optimized_required)
+ if (update_type == UPDATE_TYPE_FULL && dc->optimized_required) {
+ hwss_reset_unused_det(dc, dc->current_state);
hwss_process_outstanding_hw_updates(dc, dc->current_state);
+ }
if (update_type != UPDATE_TYPE_FAST && dc->res_pool->funcs->prepare_mcache_programming)
dc->res_pool->funcs->prepare_mcache_programming(dc, context);
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c
index ef4987d449cb..d4dbdf7d0506 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c
@@ -3047,6 +3047,37 @@ void hwss_wait_for_odm_update_pending_complete(struct dc *dc, struct dc_state *c
hwss_wait_for_all_blank_complete(dc, context);
}
+void hwss_reset_unused_det(struct dc *dc, struct dc_state *context)
+{
+ struct hubbub *hubbub = dc->res_pool->hubbub;
+ unsigned int i, j;
+
+ if (!hubbub || !hubbub->funcs->program_det_segments)
+ return;
+
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ struct hubp *hubp = dc->res_pool->hubps[i];
+ bool hubp_in_use = false;
+
+ if (!hubp)
+ continue;
+
+ for (j = 0; j < dc->res_pool->pipe_count; j++) {
+ const struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[j];
+
+ /* A plane-less pipe renders from the DPG, its HUBP (assigned with the stream) needs no DET */
+ if (pipe->plane_state && pipe->plane_res.hubp &&
+ pipe->plane_res.hubp->inst == hubp->inst) {
+ hubp_in_use = true;
+ break;
+ }
+ }
+
+ if (!hubp_in_use)
+ hubbub->funcs->program_det_segments(hubbub, hubp->inst, 0);
+ }
+}
+
void hwss_wait_for_no_pipes_pending(struct dc *dc, struct dc_state *context)
{
int i;
diff --git a/drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer.h b/drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer.h
index cdf2562a9a40..0d64a0c1913d 100644
--- a/drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer.h
+++ b/drivers/gpu/drm/amd/display/dc/hwss/hw_sequencer.h
@@ -1807,6 +1807,9 @@ void hwss_wait_for_odm_update_pending_complete(struct dc *dc,
void hwss_wait_for_no_pipes_pending(struct dc *dc,
struct dc_state *context);
+void hwss_reset_unused_det(struct dc *dc,
+ struct dc_state *context);
+
void hwss_wait_for_outstanding_hw_updates(struct dc *dc,
struct dc_state *dc_context);
diff --git a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c
index 24e3bbec15ff..8ff364d465de 100644
--- a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c
+++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_capability.c
@@ -2601,12 +2601,8 @@ bool dp_is_sink_present(struct dc_link *link)
* which indicates we need additional delay
*/
- if (dal_ddc_open(ddc, GPIO_MODE_INPUT,
- GPIO_DDC_CONFIG_TYPE_MODE_I2C) != GPIO_RESULT_OK) {
- dal_ddc_close(ddc);
-
+ if (dal_ddc_open(ddc, GPIO_MODE_INPUT, GPIO_DDC_CONFIG_TYPE_MODE_I2C) != GPIO_RESULT_OK)
return present;
- }
/*
* Read GPIO: DP sink is present if both clock and data pins are zero
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c b/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c
index 73cc8a713556..2cdfe3d7a0ea 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn20/dcn20_optc.c
@@ -449,6 +449,13 @@ void optc2_lock_doublebuffer_disable(struct timing_generator *optc)
void optc2_setup_manual_trigger(struct timing_generator *optc)
{
struct optc *optc1 = DCN10TG_FROM_TG(optc);
+ uint32_t v_total, v_total_max;
+
+ REG_GET(OTG_V_TOTAL_MAX, OTG_V_TOTAL_MAX, &v_total_max);
+ if (!v_total_max) {
+ REG_GET(OTG_V_TOTAL, OTG_V_TOTAL, &v_total);
+ optc->funcs->set_vtotal_min_max(optc, v_total, v_total);
+ }
/* Set the min/max selectors unconditionally so that
* DMCUB fw may change OTG timings when necessary
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn30/dcn30_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn30/dcn30_optc.h
index 16c5610b49ac..b801989f10cf 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn30/dcn30_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn30/dcn30_optc.h
@@ -91,6 +91,12 @@
SRI(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst),\
SRI(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst),\
SRI(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_DATA_RG, OTG, inst),\
+ SRI(OTG_CRC1_DATA_B, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst),\
SR(GSL_SOURCE_SELECT),\
SRI(OTG_TRIGA_MANUAL_TRIG, OTG, inst),\
SRI(OTG_DRR_CONTROL, OTG, inst)
@@ -229,6 +235,7 @@
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL2, OTG_CRC_DSC_MODE, mask_sh),\
SF(OTG0_OTG_CRC_CNTL2, OTG_CRC_DATA_STREAM_COMBINE_MODE, mask_sh),\
@@ -245,6 +252,17 @@
SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\
SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.h
index 119a954fe3e5..85223414c909 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn31/dcn31_optc.h
@@ -83,6 +83,12 @@
SRI(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst),\
SRI(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst),\
SRI(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_DATA_RG, OTG, inst),\
+ SRI(OTG_CRC1_DATA_B, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst),\
SR(GSL_SOURCE_SELECT),\
SRI(OTG_TRIGA_MANUAL_TRIG, OTG, inst),\
SRI(OTG_GLOBAL_CONTROL1, OTG, inst),\
@@ -205,7 +211,9 @@
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_RG, CRC0_R_CR, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_RG, CRC0_G_Y, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_B, CRC0_B_CB, mask_sh),\
@@ -217,6 +225,17 @@
SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\
SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\
@@ -264,8 +283,7 @@
SF(OTG0_INTERRUPT_DEST, OTG0_IHC_OTG_VERTICAL_INTERRUPT2_DEST, mask_sh)
#define OPTC_COMMON_MASK_SH_LIST_DCN31X(mask_sh)\
- OPTC_COMMON_MASK_SH_LIST_DCN3_1(mask_sh),\
- SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh)
+ OPTC_COMMON_MASK_SH_LIST_DCN3_1(mask_sh)
void dcn31_timing_generator_init(struct optc *optc1);
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn314/dcn314_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn314/dcn314_optc.h
index 7b43c3ecd46b..956eecf0a43a 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn314/dcn314_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn314/dcn314_optc.h
@@ -84,6 +84,12 @@
SRI(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst),\
SRI(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst),\
SRI(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_DATA_RG, OTG, inst),\
+ SRI(OTG_CRC1_DATA_B, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst),\
+ SRI(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst),\
SR(GSL_SOURCE_SELECT),\
SRI(OTG_TRIGA_MANUAL_TRIG, OTG, inst),\
SRI(OTG_GLOBAL_CONTROL1, OTG, inst),\
@@ -204,6 +210,7 @@
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_RG, CRC0_R_CR, mask_sh),\
@@ -217,6 +224,17 @@
SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\
SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn32/dcn32_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn32/dcn32_optc.h
index 60c01ec28b65..492760779226 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn32/dcn32_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn32/dcn32_optc.h
@@ -146,6 +146,17 @@
SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\
SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn35/dcn35_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn35/dcn35_optc.h
index 82e0818c8f9f..196596cc9fb6 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn35/dcn35_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn35/dcn35_optc.h
@@ -32,23 +32,12 @@
#define OPTC_COMMON_MASK_SH_LIST_DCN3_5(mask_sh)\
OPTC_COMMON_MASK_SH_LIST_DCN3_2(mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_WINDOW_DB_EN, mask_sh),\
- SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\
- SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\
- SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\
SF(OTG0_OTG_CRC2_DATA_RG, CRC2_R_CR, mask_sh),\
SF(OTG0_OTG_CRC2_DATA_RG, CRC2_G_Y, mask_sh),\
SF(OTG0_OTG_CRC2_DATA_B, CRC2_B_CB, mask_sh),\
SF(OTG0_OTG_CRC3_DATA_RG, CRC3_R_CR, mask_sh),\
SF(OTG0_OTG_CRC3_DATA_RG, CRC3_G_Y, mask_sh),\
SF(OTG0_OTG_CRC3_DATA_B, CRC3_B_CB, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\
- SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWA_X_CONTROL_READBACK, OTG_CRC0_WINDOWA_X_START_READBACK, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWA_X_CONTROL_READBACK, OTG_CRC0_WINDOWA_X_END_READBACK, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWA_Y_CONTROL_READBACK, OTG_CRC0_WINDOWA_Y_START_READBACK, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn401/dcn401_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn401/dcn401_optc.h
index 3add972ca2ff..9e4e9d75400a 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn401/dcn401_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn401/dcn401_optc.h
@@ -110,6 +110,7 @@
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_RG, CRC0_R_CR, mask_sh),\
@@ -123,6 +124,17 @@
SF(OTG0_OTG_CRC0_WINDOWB_X_CONTROL, OTG_CRC0_WINDOWB_X_END, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_START, mask_sh),\
SF(OTG0_OTG_CRC0_WINDOWB_Y_CONTROL, OTG_CRC0_WINDOWB_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_R_CR, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_RG, CRC1_G_Y, mask_sh),\
+ SF(OTG0_OTG_CRC1_DATA_B, CRC1_B_CB, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_X_CONTROL, OTG_CRC1_WINDOWA_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWA_Y_CONTROL, OTG_CRC1_WINDOWA_Y_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_X_CONTROL, OTG_CRC1_WINDOWB_X_END, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_START, mask_sh),\
+ SF(OTG0_OTG_CRC1_WINDOWB_Y_CONTROL, OTG_CRC1_WINDOWB_Y_END, mask_sh),\
SF(OTG0_OTG_TRIGA_MANUAL_TRIG, OTG_TRIGA_MANUAL_TRIG, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL0_READY_SOURCE_SEL, mask_sh),\
SF(GSL_SOURCE_SELECT, GSL1_READY_SOURCE_SEL, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn42/dcn42_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn42/dcn42_optc.h
index bc641f6ecd7e..bdda94489a23 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn42/dcn42_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn42/dcn42_optc.h
@@ -137,6 +137,7 @@
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_R, CRC0_R_CR, mask_sh),\
@@ -316,6 +317,7 @@
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_CHECK_ALL_FIELDS, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_R, CRC0_R_CR, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/optc/dcn60/dcn60_optc.h b/drivers/gpu/drm/amd/display/dc/optc/dcn60/dcn60_optc.h
index ba207c0e498e..ec40c9180030 100644
--- a/drivers/gpu/drm/amd/display/dc/optc/dcn60/dcn60_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/optc/dcn60/dcn60_optc.h
@@ -109,7 +109,9 @@
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_FORCE_DELAY, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_CONT_EN, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC0_SELECT, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_SELECT, mask_sh),\
SF(OTG0_OTG_CRC_CNTL, OTG_CRC_EN, mask_sh),\
+ SF(OTG0_OTG_CRC_CNTL, OTG_CRC1_EN, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_R, CRC0_R_CR, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_G, CRC0_G_Y, mask_sh),\
SF(OTG0_OTG_CRC0_DATA_B, CRC0_B_CB, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
index a2e66761b8ec..81227df68680 100644
--- a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.c
@@ -1941,8 +1941,21 @@ static bool dml1_validate(struct dc *dc, struct dc_state *context, enum dc_valid
out = dcn32_internal_validate_bw(dc, context, pipes, &pipe_cnt, &vlevel, validate_mode);
DC_FP_END();
- if (pipe_cnt == 0)
+ if (pipe_cnt == 0) {
+ /* The context is a copy of the previous state, so without this the
+ * clocks the last display needed stay applied with all displays off.
+ */
+ context->bw_ctx.bw.dcn.clk.socclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dppclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dcfclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dcfclk_deep_sleep_khz = 0;
+ context->bw_ctx.bw.dcn.clk.dramclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.fclk_khz = 0;
+ context->bw_ctx.bw.dcn.clk.p_state_change_support = true;
+ context->bw_ctx.bw.dcn.clk.fclk_p_state_change_support = true;
+ context->bw_ctx.bw.dcn.clk.num_ways = 0;
goto validate_out;
+ }
if (!out)
goto validate_fail;
diff --git a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.h b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.h
index 68e7140f1505..fa8cc89cb3f7 100644
--- a/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.h
+++ b/drivers/gpu/drm/amd/display/dc/resource/dcn32/dcn32_resource.h
@@ -1054,6 +1054,12 @@ unsigned int dcn32_get_max_hw_cursor_size(const struct dc *dc,
SRI_ARR(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst), \
SRI_ARR(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst), \
SRI_ARR(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_DATA_RG, OTG, inst), \
+ SRI_ARR(OTG_CRC1_DATA_B, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst), \
SR_ARR(GSL_SOURCE_SELECT, inst), \
SRI_ARR(OTG_TRIGA_MANUAL_TRIG, OTG, inst), \
SRI_ARR(OTG_GLOBAL_CONTROL1, OTG, inst), \
diff --git a/drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.h b/drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.h
index 8e3473f86490..3cfc1a39a103 100644
--- a/drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.h
+++ b/drivers/gpu/drm/amd/display/dc/resource/dcn401/dcn401_resource.h
@@ -535,6 +535,12 @@ int dcn401_get_power_profile(const struct dc_state *context);
SRI_ARR(OTG_CRC0_WINDOWA_Y_CONTROL, OTG, inst), \
SRI_ARR(OTG_CRC0_WINDOWB_X_CONTROL, OTG, inst), \
SRI_ARR(OTG_CRC0_WINDOWB_Y_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_DATA_RG, OTG, inst), \
+ SRI_ARR(OTG_CRC1_DATA_B, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWA_X_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWA_Y_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWB_X_CONTROL, OTG, inst), \
+ SRI_ARR(OTG_CRC1_WINDOWB_Y_CONTROL, OTG, inst), \
SR_ARR(GSL_SOURCE_SELECT, inst), \
SRI_ARR(OTG_TRIGA_MANUAL_TRIG, OTG, inst), \
SRI_ARR(OTG_GLOBAL_CONTROL1, OTG, inst), \
diff --git a/drivers/gpu/drm/amd/display/modules/freesync/freesync.c b/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
index 4bd060e30799..f154a619b533 100644
--- a/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
+++ b/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
@@ -1281,10 +1281,8 @@ void mod_freesync_handle_v_update(struct mod_freesync *mod_freesync,
* If VRR is inactive, set vtotal min and max to nominal vtotal
*/
if (in_out_vrr->state == VRR_STATE_INACTIVE) {
- in_out_vrr->adjust.v_total_min =
- mod_freesync_calc_v_total_from_refresh(stream,
- in_out_vrr->max_refresh_in_uhz);
- in_out_vrr->adjust.v_total_max = in_out_vrr->adjust.v_total_min;
+ in_out_vrr->adjust.v_total_min = stream->timing.v_total;
+ in_out_vrr->adjust.v_total_max = stream->timing.v_total;
return;
}
}
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
index 8285d4a91589..13e7e33fa0eb 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
@@ -2912,6 +2912,25 @@ static int smu_v13_0_7_mode1_reset(struct smu_context *smu)
return ret;
}
+static void smu_v13_0_7_get_unique_id(struct smu_context *smu)
+{
+ SmuMetricsExternal_t metrics_external;
+ SmuMetrics_t *metrics = &metrics_external.SmuMetrics;
+ struct amdgpu_device *adev = smu->adev;
+ u32 upper32 = 0, lower32 = 0;
+ int ret;
+
+ ret = smu_cmn_get_metrics_table(smu, &metrics_external, false);
+ if (ret)
+ goto out;
+
+ upper32 = metrics->PublicSerialNumberUpper;
+ lower32 = metrics->PublicSerialNumberLower;
+
+out:
+ adev->unique_id = ((u64)upper32 << 32) | lower32;
+}
+
static void smu_v13_0_7_init_msg_ctl(struct smu_context *smu)
{
struct amdgpu_device *adev = smu->adev;
@@ -2995,6 +3014,7 @@ static const struct pptable_funcs smu_v13_0_7_ppt_funcs = {
.enable_uclk_shadow = smu_v13_0_enable_uclk_shadow,
.set_wbrf_exclusion_ranges = smu_v13_0_set_wbrf_exclusion_ranges,
.interrupt_work = smu_v13_0_interrupt_work,
+ .get_unique_id = smu_v13_0_7_get_unique_id,
};
void smu_v13_0_7_set_ppt_funcs(struct smu_context *smu)
diff --git a/drivers/gpu/drm/bridge/nwl-dsi.c b/drivers/gpu/drm/bridge/nwl-dsi.c
index 09992529f3d8..1aab46e1a168 100644
--- a/drivers/gpu/drm/bridge/nwl-dsi.c
+++ b/drivers/gpu/drm/bridge/nwl-dsi.c
@@ -110,6 +110,8 @@ struct nwl_dsi {
int error;
struct nwl_dsi_transfer *xfer;
+
+ unsigned int endpoint;
};
static const struct regmap_config nwl_dsi_regmap_config = {
@@ -821,10 +823,13 @@ static int nwl_dsi_bridge_atomic_check(struct drm_bridge *bridge,
struct drm_connector_state *conn_state)
{
struct drm_display_mode *adjusted_mode = &crtc_state->adjusted_mode;
+ struct nwl_dsi *dsi = bridge_to_dsi(bridge);
/* At least LCDIF + NWL needs active high sync */
- adjusted_mode->flags |= (DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_PVSYNC);
- adjusted_mode->flags &= ~(DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC);
+ if (dsi->endpoint == NWL_DSI_ENDPOINT_LCDIF) {
+ adjusted_mode->flags |= (DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_PVSYNC);
+ adjusted_mode->flags &= ~(DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC);
+ }
/*
* Do a full modeset if crtc_state->active is changed to be true.
@@ -1088,13 +1093,12 @@ static int nwl_dsi_parse_dt(struct nwl_dsi *dsi)
static int nwl_dsi_select_input(struct nwl_dsi *dsi)
{
struct device_node *remote;
- u32 use_dcss = 1;
int ret;
remote = of_graph_get_remote_node(dsi->dev->of_node, 0,
NWL_DSI_ENDPOINT_LCDIF);
if (remote) {
- use_dcss = 0;
+ dsi->endpoint = NWL_DSI_ENDPOINT_LCDIF;
} else {
remote = of_graph_get_remote_node(dsi->dev->of_node, 0,
NWL_DSI_ENDPOINT_DCSS);
@@ -1103,11 +1107,12 @@ static int nwl_dsi_select_input(struct nwl_dsi *dsi)
"No valid input endpoint found\n");
return -EINVAL;
}
+ dsi->endpoint = NWL_DSI_ENDPOINT_DCSS;
}
DRM_DEV_INFO(dsi->dev, "Using %s as input source\n",
- (use_dcss) ? "DCSS" : "LCDIF");
- ret = mux_control_try_select(dsi->mux, use_dcss);
+ (dsi->endpoint == NWL_DSI_ENDPOINT_DCSS) ? "DCSS" : "LCDIF");
+ ret = mux_control_try_select(dsi->mux, dsi->endpoint);
if (ret < 0)
DRM_DEV_ERROR(dsi->dev, "Failed to select input: %d\n", ret);
diff --git a/drivers/gpu/drm/display/drm_dp_helper.c b/drivers/gpu/drm/display/drm_dp_helper.c
index 488fd123a01a..7676285c19f8 100644
--- a/drivers/gpu/drm/display/drm_dp_helper.c
+++ b/drivers/gpu/drm/display/drm_dp_helper.c
@@ -2192,7 +2192,8 @@ static int drm_dp_i2c_xfer(struct i2c_adapter *adapter, struct i2c_msg *msgs,
msg.buffer = msgs[i].buf + j;
msg.size = min(transfer_size, msgs[i].len - j);
- if (j + msg.size == msgs[i].len && aux->no_zero_sized)
+ if (aux->no_zero_sized && i == num - 1 &&
+ j + msg.size == msgs[i].len)
msg.request &= ~DP_AUX_I2C_MOT;
err = drm_dp_i2c_drain_msg(aux, &msg);
diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c
index 63ead8ba6756..5295ec122e0d 100644
--- a/drivers/gpu/drm/drm_crtc.c
+++ b/drivers/gpu/drm/drm_crtc.c
@@ -154,11 +154,8 @@ static void drm_crtc_crc_fini(struct drm_crtc *crtc)
#endif
}
-static const struct dma_fence_ops drm_crtc_fence_ops;
-
static struct drm_crtc *fence_to_crtc(struct dma_fence *fence)
{
- BUG_ON(rcu_access_pointer(fence->ops) != &drm_crtc_fence_ops);
return container_of(fence->extern_lock, struct drm_crtc, fence_lock);
}
@@ -493,14 +490,23 @@ EXPORT_SYMBOL(__drmm_crtc_alloc_with_planes);
* drm_crtc_cleanup - Clean up the core crtc usage
* @crtc: CRTC to cleanup
*
- * This function cleans up @crtc and removes it from the DRM mode setting
- * core. Note that the function does *not* free the crtc structure itself,
- * this is the responsibility of the caller.
+ * This function cleans up @crtc and removes it from the DRM mode setting core,
+ * after first waiting an RCU grace period to ensure @crtc->dev can safely be
+ * dereferenced by our dma_fence_ops.
+ *
+ * Note that the function does *not* free the crtc structure itself, this is the
+ * responsibility of the caller.
*/
void drm_crtc_cleanup(struct drm_crtc *crtc)
{
struct drm_device *dev = crtc->dev;
+ /* Ensure our dma_fence_ops remain valid for an RCU grace period after
+ * the fence is signaled. This is necessary because our dma_fence_ops
+ * dereference crtc->dev.
+ */
+ synchronize_rcu();
+
/* Note that the crtc_list is considered to be static; should we
* remove the drm_crtc at runtime we would have to decrement all
* the indices on the drm_crtc after us in the crtc_list.
diff --git a/drivers/gpu/drm/drm_panic.c b/drivers/gpu/drm/drm_panic.c
index d28ce8878e05..fef47625c526 100644
--- a/drivers/gpu/drm/drm_panic.c
+++ b/drivers/gpu/drm/drm_panic.c
@@ -558,7 +558,8 @@ static void draw_panic_screen_kmsg(struct drm_scanout_buffer *sb)
const struct font_desc *font = get_default_font(sb->width, sb->height, NULL, NULL);
struct drm_rect r_screen = DRM_RECT_INIT(0, 0, sb->width, sb->height);
struct kmsg_dump_iter iter;
- char kmsg_buf[512];
+ /* the size printk formats one record into, too big for the stack */
+ static char kmsg_buf[2048];
size_t kmsg_len;
struct drm_panic_line line;
int yoffset;
@@ -576,6 +577,10 @@ static void draw_panic_screen_kmsg(struct drm_scanout_buffer *sb)
char *start;
char *end;
+ /* a record that does not fit comes back empty, every time */
+ if (!kmsg_len)
+ break;
+
/* ignore terminating NUL and newline */
start = kmsg_buf + kmsg_len - 2;
end = kmsg_buf + kmsg_len - 1;
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
index 03b2576af67f..99c91a9eecf1 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
@@ -168,6 +168,17 @@ u16 etnaviv_buffer_config_pta(struct etnaviv_gpu *gpu, unsigned short id)
return buffer->user_size / 8;
}
+static void etnaviv_cmd_flush_ts(struct etnaviv_cmdbuf *buffer, bool has_blt)
+{
+ CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE, VIVS_TS_FLUSH_CACHE_FLUSH);
+
+ if (has_blt) {
+ CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1);
+ CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1);
+ CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
+ }
+}
+
void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
{
struct etnaviv_cmdbuf *buffer = &gpu->buffer;
@@ -191,7 +202,7 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
unsigned int dwords = 7;
if (has_blt)
- dwords += 10;
+ dwords += 11;
link_target = etnaviv_buffer_reserve(gpu, buffer, dwords);
@@ -204,16 +215,8 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
}
CMD_LOAD_STATE(buffer, VIVS_GL_FLUSH_CACHE, flush);
- if (gpu->exec_state == ETNA_PIPE_3D) {
- if (has_blt) {
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
- } else {
- CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE,
- VIVS_TS_FLUSH_CACHE_FLUSH);
- }
- }
+ if (gpu->exec_state == ETNA_PIPE_3D)
+ etnaviv_cmd_flush_ts(buffer, has_blt);
CMD_SEM(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
CMD_STALL(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
if (has_blt) {
@@ -400,12 +403,12 @@ void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, u32 exec_state,
return_dwords = 7;
/*
- * When the BLT engine is present we need 6 more dwords in the return
+ * When the BLT engine is present we need 7 more dwords in the return
* target: 3 enable/flush/disable + 4 enable/semaphore stall/disable,
- * but we don't need the normal TS flush state.
+ * on top of the normal TS flush state.
*/
if (has_blt)
- return_dwords += 6;
+ return_dwords += 7;
return_target = etnaviv_buffer_reserve(gpu, buffer, return_dwords);
CMD_LINK(cmdbuf, return_dwords, return_target);
@@ -422,14 +425,7 @@ void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, u32 exec_state,
VIVS_GL_FLUSH_CACHE_DEPTH |
VIVS_GL_FLUSH_CACHE_COLOR |
VIVS_GL_FLUSH_CACHE_SHADER_L1);
- if (has_blt) {
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_SET_COMMAND, 0x1);
- CMD_LOAD_STATE(buffer, VIVS_BLT_ENABLE, 0x0);
- } else {
- CMD_LOAD_STATE(buffer, VIVS_TS_FLUSH_CACHE,
- VIVS_TS_FLUSH_CACHE_FLUSH);
- }
+ etnaviv_cmd_flush_ts(buffer, has_blt);
}
CMD_SEM(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
CMD_STALL(buffer, SYNC_RECIPIENT_FE, SYNC_RECIPIENT_PE);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.c b/drivers/gpu/drm/etnaviv/etnaviv_drv.c
index 08aca9035fc1..ec1352c1b09f 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_drv.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.c
@@ -601,8 +601,10 @@ static void etnaviv_unbind(struct device *dev)
component_unbind_all(dev, drm);
- etnaviv_cmdbuf_free(priv->flop_reset_data_ppu);
- kfree(priv->flop_reset_data_ppu);
+ if (priv->flop_reset_data_ppu) {
+ etnaviv_cmdbuf_free(priv->flop_reset_data_ppu);
+ kfree(priv->flop_reset_data_ppu);
+ }
etnaviv_cmdbuf_suballoc_destroy(priv->cmdbuf_suballoc);
diff --git a/drivers/gpu/drm/i915/display/intel_ddi.c b/drivers/gpu/drm/i915/display/intel_ddi.c
index 02a53c9848e1..8de6e13cf2dd 100644
--- a/drivers/gpu/drm/i915/display/intel_ddi.c
+++ b/drivers/gpu/drm/i915/display/intel_ddi.c
@@ -4594,7 +4594,7 @@ intel_ddi_port_sync_transcoders(const struct intel_crtc_state *ref_crtc_state,
* From, xe3lpd onwards we have defeatured this with reference to
* Wa_16024710867
*/
- if (!IS_DISPLAY_VER(display, 9, 20))
+ if (!IS_DISPLAY_VER(display, 9, 30))
return 0;
if (!intel_crtc_has_type(ref_crtc_state, INTEL_OUTPUT_DP))
diff --git a/drivers/gpu/drm/i915/display/intel_display_device.c b/drivers/gpu/drm/i915/display/intel_display_device.c
index f17fc2c68472..c74256359c25 100644
--- a/drivers/gpu/drm/i915/display/intel_display_device.c
+++ b/drivers/gpu/drm/i915/display/intel_display_device.c
@@ -1813,7 +1813,10 @@ static void __intel_display_device_info_runtime_init(struct intel_display *displ
if (intel_display_wa(display, INTEL_DISPLAY_WA_14011765242))
for_each_pipe(display, pipe)
display_runtime->num_scalers[pipe] = 0;
- else if (DISPLAY_VER(display) >= 11) {
+ else if (DISPLAY_VERx100(display) == 3002) {
+ for_each_pipe(display, pipe)
+ display_runtime->num_scalers[pipe] = 1;
+ } else if (DISPLAY_VER(display) >= 11) {
for_each_pipe(display, pipe)
display_runtime->num_scalers[pipe] = 2;
} else if (DISPLAY_VER(display) >= 9) {
diff --git a/drivers/gpu/drm/i915/display/intel_dp.c b/drivers/gpu/drm/i915/display/intel_dp.c
index 3152122e6aef..d79b6367cd57 100644
--- a/drivers/gpu/drm/i915/display/intel_dp.c
+++ b/drivers/gpu/drm/i915/display/intel_dp.c
@@ -475,9 +475,9 @@ bool intel_dp_has_joiner(struct intel_dp *intel_dp)
!connector->panel.vbt.edp.pipe_joiner_enable)
return false;
- return DISPLAY_VER(display) >= 12 ||
+ return ((DISPLAY_VER(display) >= 12 && DISPLAY_VERx100(display) != 3002) ||
(DISPLAY_VER(display) == 11 &&
- encoder->port != PORT_A);
+ encoder->port != PORT_A));
}
static int dg2_max_source_rate(struct intel_dp *intel_dp)
diff --git a/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c b/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c
index 266e042e0023..d84def79c4bf 100644
--- a/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c
+++ b/drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c
@@ -41,6 +41,7 @@
#include "intel_display_types.h"
#include "intel_dp.h"
#include "intel_dp_aux_backlight.h"
+#include "intel_quirks.h"
/*
* DP AUX registers for Intel's proprietary HDR backlight interface. We define
@@ -687,11 +688,30 @@ int intel_dp_aux_init_backlight_funcs(struct intel_connector *connector)
struct drm_device *dev = connector->base.dev;
struct intel_panel *panel = &connector->panel;
bool try_intel_interface = false, try_vesa_interface = false;
+ int enable_dpcd_backlight;
+
+ /*
+ * Some devices have a broken VBT in some Panels, but not others,
+ * advertising INTEL_BACKLIGHT_DISPLAY_DDI, but actually failing to use
+ * the Intel interface. Adding a fallback to the Vesa interface gives
+ * these devices a working backlight for both the broken and non broken
+ * panel VBTs.
+ *
+ * This fallback can not be made the general default as this has broken
+ * other devices in the past whose VBT correctly reports
+ * INTEL_BACKLIGHT_DISPLAY_DDI and whose PWM path is the actual
+ * backlight control, but whose DPCD optimistically advertises
+ * DP_EDP_BACKLIGHT_AUX_ENABLE_CAP / _BRIGHTNESS_AUX_SET_CAP.
+ */
+ enable_dpcd_backlight = display->params.enable_dpcd_backlight;
+ if (enable_dpcd_backlight == INTEL_DP_AUX_BACKLIGHT_AUTO &&
+ intel_has_dpcd_quirk(intel_dp, QUIRK_ENABLE_DPCD_BACKLIGHT))
+ enable_dpcd_backlight = INTEL_DP_AUX_BACKLIGHT_ON;
/* Check the VBT and user's module parameters to figure out which
* interfaces to probe
*/
- switch (display->params.enable_dpcd_backlight) {
+ switch (enable_dpcd_backlight) {
case INTEL_DP_AUX_BACKLIGHT_OFF:
return -ENODEV;
case INTEL_DP_AUX_BACKLIGHT_AUTO:
diff --git a/drivers/gpu/drm/i915/display/intel_plane.c b/drivers/gpu/drm/i915/display/intel_plane.c
index d0f99a87c42e..c1c5a6128614 100644
--- a/drivers/gpu/drm/i915/display/intel_plane.c
+++ b/drivers/gpu/drm/i915/display/intel_plane.c
@@ -140,6 +140,7 @@ intel_plane_duplicate_state(struct drm_plane *plane)
intel_state->dpt_vma = NULL;
intel_state->fence_id = -1;
intel_state->damage = DRM_RECT_INIT(0, 0, 0, 0);
+ intel_state->psr2_sel_fetch_area = DRM_RECT_INIT(0, 0, 0, 0);
/* add reference to fb */
if (intel_state->hw.fb)
@@ -1930,6 +1931,16 @@ static int intel_add_affected_planes(struct intel_atomic_state *state)
ret = intel_joiner_add_affected_planes(state, intel_crtc_joined_pipe_mask(crtc_state));
if (ret)
return ret;
+
+ /*
+ * Add affected plane here to ensure UV and Y plane linking is
+ * in place if UV plane is added as a part of selective update.
+ */
+ if (crtc_state->enable_psr2_sel_fetch) {
+ ret = intel_plane_add_affected(state, crtc);
+ if (ret)
+ return ret;
+ }
}
return 0;
diff --git a/drivers/gpu/drm/i915/display/intel_psr.c b/drivers/gpu/drm/i915/display/intel_psr.c
index c28bcb1a0184..7aaf5b4d3503 100644
--- a/drivers/gpu/drm/i915/display/intel_psr.c
+++ b/drivers/gpu/drm/i915/display/intel_psr.c
@@ -2702,23 +2702,25 @@ static u32 psr2_pipe_srcsz_early_tpt_calc(struct intel_crtc_state *crtc_state,
}
static void clip_area_update(struct drm_rect *overlap_damage_area,
- struct drm_rect *damage_area,
- struct drm_rect *display_area)
+ const struct drm_rect *damage_area,
+ const struct drm_rect *display_area)
{
- if (!drm_rect_intersect(damage_area, display_area))
+ struct drm_rect inter = *damage_area;
+
+ if (!drm_rect_intersect(&inter, display_area))
return;
if (overlap_damage_area->y1 == -1) {
- overlap_damage_area->y1 = damage_area->y1;
- overlap_damage_area->y2 = damage_area->y2;
+ overlap_damage_area->y1 = inter.y1;
+ overlap_damage_area->y2 = inter.y2;
return;
}
- if (damage_area->y1 < overlap_damage_area->y1)
- overlap_damage_area->y1 = damage_area->y1;
+ if (inter.y1 < overlap_damage_area->y1)
+ overlap_damage_area->y1 = inter.y1;
- if (damage_area->y2 > overlap_damage_area->y2)
- overlap_damage_area->y2 = damage_area->y2;
+ if (inter.y2 > overlap_damage_area->y2)
+ overlap_damage_area->y2 = inter.y2;
}
static bool intel_psr2_sel_fetch_pipe_alignment(struct intel_crtc_state *crtc_state)
@@ -2895,7 +2897,7 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
.y2 = crtc_state->hw.adjusted_mode.crtc_vdisplay,
};
bool full_update = false, su_area_changed;
- int i, ret;
+ int i;
/*
* Selective fetch is not always usable, for instance it is dropped
@@ -2913,16 +2915,16 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
if (!crtc_state->enable_psr2_sel_fetch)
return 0;
- if (!psr2_sel_fetch_pipe_state_supported(crtc_state)) {
- full_update = true;
- goto skip_sel_fetch_set_loop;
- }
-
crtc_state->psr2_su_area.x1 = 0;
crtc_state->psr2_su_area.y1 = -1;
crtc_state->psr2_su_area.x2 = drm_rect_width(&display_area);
crtc_state->psr2_su_area.y2 = -1;
+ if (!psr2_sel_fetch_pipe_state_supported(crtc_state)) {
+ full_update = true;
+ goto skip_su_area_calc;
+ }
+
/*
* Calculate minimal selective fetch area of each plane and calculate
* the pipe damaged area.
@@ -3006,23 +3008,19 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
}
if (full_update)
- goto skip_sel_fetch_set_loop;
+ goto skip_su_area_calc;
intel_psr_apply_su_area_workarounds(crtc_state);
- ret = drm_atomic_add_affected_planes(&state->base, &crtc->base);
- if (ret)
- return ret;
-
do {
bool cursor_in_su_area = false;
/*
* Adjust su area to cover cursor fully as necessary
- * (early transport). This needs to be done after
- * drm_atomic_add_affected_planes to ensure visible
- * cursor is added into affected planes even when
- * cursor is not updated by itself.
+ * (early transport). A visible cursor is in the state
+ * even when it is not updated by itself as all planes
+ * of the pipe are added into the state before the plane
+ * checks when selective fetch is enabled.
*/
intel_psr2_sel_fetch_et_alignment(state, crtc, &display_area,
&cursor_in_su_area);
@@ -3042,6 +3040,11 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
break;
} while (su_area_changed);
+skip_su_area_calc:
+ if (full_update)
+ clip_area_update(&crtc_state->psr2_su_area, &display_area,
+ &display_area);
+
/*
* Now that we have the pipe damaged area check if it intersect with
* every plane, if it does set the plane selective fetch area.
@@ -3057,28 +3060,14 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
inter = crtc_state->psr2_su_area;
sel_fetch_area = &new_plane_state->psr2_sel_fetch_area;
- if (!drm_rect_intersect(&inter, &new_plane_state->uapi.dst)) {
- sel_fetch_area->y1 = -1;
- sel_fetch_area->y2 = -1;
- /*
- * if plane sel fetch was previously enabled ->
- * disable it
- */
- if (drm_rect_height(&old_plane_state->psr2_sel_fetch_area) > 0)
- crtc_state->update_planes |= BIT(plane->id);
-
- continue;
- }
-
- if (!psr2_sel_fetch_plane_state_supported(new_plane_state)) {
- full_update = true;
- break;
+ if (drm_rect_intersect(&inter, &new_plane_state->uapi.dst)) {
+ sel_fetch_area->y1 = inter.y1 - new_plane_state->uapi.dst.y1;
+ sel_fetch_area->y2 = inter.y2 - new_plane_state->uapi.dst.y1;
}
- sel_fetch_area = &new_plane_state->psr2_sel_fetch_area;
- sel_fetch_area->y1 = inter.y1 - new_plane_state->uapi.dst.y1;
- sel_fetch_area->y2 = inter.y2 - new_plane_state->uapi.dst.y1;
- crtc_state->update_planes |= BIT(plane->id);
+ if (!drm_rect_equals(&old_plane_state->psr2_sel_fetch_area,
+ sel_fetch_area))
+ crtc_state->update_planes |= BIT(plane->id);
/*
* Sel_fetch_area is calculated for UV plane. Use
@@ -3086,24 +3075,25 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
*/
if (linked) {
struct intel_plane_state *linked_new_plane_state;
+ struct intel_plane_state *linked_old_plane_state;
struct drm_rect *linked_sel_fetch_area;
- linked_new_plane_state = intel_atomic_get_plane_state(state, linked);
- if (IS_ERR(linked_new_plane_state))
- return PTR_ERR(linked_new_plane_state);
+ linked_new_plane_state = intel_atomic_get_new_plane_state(state, linked);
+ if (!linked_new_plane_state)
+ return -EINVAL;
+
+ linked_old_plane_state = intel_atomic_get_old_plane_state(state, linked);
linked_sel_fetch_area = &linked_new_plane_state->psr2_sel_fetch_area;
linked_sel_fetch_area->y1 = sel_fetch_area->y1;
linked_sel_fetch_area->y2 = sel_fetch_area->y2;
- crtc_state->update_planes |= BIT(linked->id);
+
+ if (!drm_rect_equals(&linked_old_plane_state->psr2_sel_fetch_area,
+ linked_sel_fetch_area))
+ crtc_state->update_planes |= BIT(linked->id);
}
}
-skip_sel_fetch_set_loop:
- if (full_update)
- clip_area_update(&crtc_state->psr2_su_area, &display_area,
- &display_area);
-
psr2_man_trk_ctl_calc(crtc_state, full_update);
crtc_state->pipe_srcsz_early_tpt =
psr2_pipe_srcsz_early_tpt_calc(crtc_state, full_update);
diff --git a/drivers/gpu/drm/i915/display/intel_psr_regs.h b/drivers/gpu/drm/i915/display/intel_psr_regs.h
index 16a9e3af198d..428f3da04500 100644
--- a/drivers/gpu/drm/i915/display/intel_psr_regs.h
+++ b/drivers/gpu/drm/i915/display/intel_psr_regs.h
@@ -254,12 +254,12 @@
#define _LNL_SFF_CTL_A 0x60918
#define _LNL_SFF_CTL_B 0x61918
#define LNL_SFF_CTL(tran) _MMIO_TRANS(tran, _LNL_SFF_CTL_A, _LNL_SFF_CTL_B)
-#define LNL_SFF_CTL_SF_SINGLE_FULL_FRAME REG_BIT(1)
+#define LNL_SFF_CTL_SF_SINGLE_FULL_FRAME REG_BIT(0)
#define _LNL_CFF_CTL_A 0x6091c
#define _LNL_CFF_CTL_B 0x6191c
#define LNL_CFF_CTL(tran) _MMIO_TRANS(tran, _LNL_CFF_CTL_A, _LNL_CFF_CTL_B)
-#define LNL_CFF_CTL_SF_CONTINUOUS_FULL_FRAME REG_BIT(1)
+#define LNL_CFF_CTL_SF_CONTINUOUS_FULL_FRAME REG_BIT(0)
/* PSR2 Early transport */
#define _PIPE_SRCSZ_ERLY_TPT_A 0x70074
diff --git a/drivers/gpu/drm/i915/display/intel_quirks.c b/drivers/gpu/drm/i915/display/intel_quirks.c
index 7d7db774d8c7..96eb9a40df9b 100644
--- a/drivers/gpu/drm/i915/display/intel_quirks.c
+++ b/drivers/gpu/drm/i915/display/intel_quirks.c
@@ -100,6 +100,14 @@ static void quirk_disable_psr2(struct intel_display *display)
drm_info(display->drm, "PSR2 support not currently available for this setup, applying disable PSR2 quirk\n");
}
+static void quirk_enable_dpcd_backlight(struct intel_dp *intel_dp)
+{
+ struct intel_display *display = to_intel_display(intel_dp);
+
+ intel_set_dpcd_quirk(intel_dp, QUIRK_ENABLE_DPCD_BACKLIGHT);
+ drm_info(display->drm, "Applying Enable DPCD Backlight quirk\n");
+}
+
struct intel_quirk {
int device;
int subsystem_vendor;
@@ -289,6 +297,22 @@ static const struct intel_dpcd_quirk intel_dpcd_quirks[] = {
.sink_oui = SINK_OUI(0x00, 0x22, 0xb9),
.hook = quirk_disable_edp_panel_replay,
},
+ /* TUXEDO InsanityBook 15 v1 */
+ {
+ .device = 0x591b,
+ .subsystem_vendor = 0x1558,
+ .subsystem_device = 0x9501,
+ .sink_oui = SINK_OUI(0x38, 0xec, 0x11),
+ .hook = quirk_enable_dpcd_backlight,
+ },
+ /* TUXEDO DX1708 */
+ {
+ .device = 0x3e9b,
+ .subsystem_vendor = 0x1558,
+ .subsystem_device = 0x8500,
+ .sink_oui = SINK_OUI(0x38, 0xec, 0x11),
+ .hook = quirk_enable_dpcd_backlight,
+ },
};
void intel_init_quirks(struct intel_display *display)
diff --git a/drivers/gpu/drm/i915/display/intel_quirks.h b/drivers/gpu/drm/i915/display/intel_quirks.h
index 970a4fe52faf..4996419ae76b 100644
--- a/drivers/gpu/drm/i915/display/intel_quirks.h
+++ b/drivers/gpu/drm/i915/display/intel_quirks.h
@@ -23,6 +23,7 @@ enum intel_quirk_id {
QUIRK_EDP_LIMIT_RATE_HBR2,
QUIRK_DISABLE_EDP_PANEL_REPLAY,
QUIRK_DISABLE_PSR2,
+ QUIRK_ENABLE_DPCD_BACKLIGHT,
};
void intel_init_quirks(struct intel_display *display);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index d5aadced7da0..7fdd41a190b4 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -2790,7 +2790,6 @@ add_timeline_fence_array(struct i915_execbuffer *eb,
u64 __user *user_values;
struct eb_fence *f;
u64 nfences;
- int err = 0;
nfences = timeline_fences->fence_count;
if (!nfences)
@@ -2828,6 +2827,7 @@ add_timeline_fence_array(struct i915_execbuffer *eb,
struct drm_syncobj *syncobj;
struct dma_fence *fence = NULL;
u64 point;
+ int err = 0;
if (__copy_from_user(&user_fence,
user_fences++,
diff --git a/drivers/gpu/drm/loongson/lsdc_gem.c b/drivers/gpu/drm/loongson/lsdc_gem.c
index 2fb03487c983..9eba4e1a5ba6 100644
--- a/drivers/gpu/drm/loongson/lsdc_gem.c
+++ b/drivers/gpu/drm/loongson/lsdc_gem.c
@@ -157,14 +157,18 @@ struct drm_gem_object *lsdc_gem_object_create(struct drm_device *ddev,
return ERR_PTR(ret);
}
+ gobj = &lbo->tbo.base;
+ gobj->funcs = &lsdc_gem_object_funcs;
+
if (!sg) {
/* VRAM is filled with random data */
- lsdc_bo_clear(lbo);
+ ret = lsdc_bo_clear(lbo);
+ if (ret) {
+ drm_gem_object_put(gobj);
+ return ERR_PTR(ret);
+ }
}
- gobj = &lbo->tbo.base;
- gobj->funcs = &lsdc_gem_object_funcs;
-
/* tracking the BOs we created */
mutex_lock(&ldev->gem.mutex);
list_add_tail(&lbo->list, &ldev->gem.objects);
diff --git a/drivers/gpu/drm/loongson/lsdc_plane.c b/drivers/gpu/drm/loongson/lsdc_plane.c
index bcc0ffa17bdf..7b9c65c30a9d 100644
--- a/drivers/gpu/drm/loongson/lsdc_plane.c
+++ b/drivers/gpu/drm/loongson/lsdc_plane.c
@@ -230,7 +230,7 @@ static void lsdc_cursor_plane_atomic_async_update(struct drm_plane *plane,
new_state = drm_atomic_get_new_plane_state(state, plane);
- new_fb = plane->state->fb;
+ new_fb = new_state->fb;
plane->state->crtc_x = new_state->crtc_x;
plane->state->crtc_y = new_state->crtc_y;
diff --git a/drivers/gpu/drm/loongson/lsdc_ttm.c b/drivers/gpu/drm/loongson/lsdc_ttm.c
index d7441d96a0dc..7b30dea039ee 100644
--- a/drivers/gpu/drm/loongson/lsdc_ttm.c
+++ b/drivers/gpu/drm/loongson/lsdc_ttm.c
@@ -388,9 +388,13 @@ void lsdc_bo_kunmap(struct lsdc_bo *lbo)
ttm_bo_kunmap(&lbo->kmap);
}
-void lsdc_bo_clear(struct lsdc_bo *lbo)
+int lsdc_bo_clear(struct lsdc_bo *lbo)
{
- lsdc_bo_kmap(lbo);
+ int ret;
+
+ ret = lsdc_bo_kmap(lbo);
+ if (ret)
+ return ret;
if (lbo->is_iomem)
memset_io((void __iomem *)lbo->kptr, 0, lbo->size);
@@ -398,6 +402,8 @@ void lsdc_bo_clear(struct lsdc_bo *lbo)
memset(lbo->kptr, 0, lbo->size);
lsdc_bo_kunmap(lbo);
+
+ return 0;
}
int lsdc_bo_evict_vram(struct drm_device *ddev)
@@ -475,10 +481,8 @@ struct lsdc_bo *lsdc_bo_create(struct drm_device *ddev,
ret = ttm_bo_init_validate(bdev, tbo, bo_type, &lbo->placement, 0,
false, sg, resv, lsdc_bo_destroy);
- if (ret) {
- kfree(lbo);
+ if (ret)
return ERR_PTR(ret);
- }
return lbo;
}
diff --git a/drivers/gpu/drm/loongson/lsdc_ttm.h b/drivers/gpu/drm/loongson/lsdc_ttm.h
index 843e1475064e..df47d9bb0d03 100644
--- a/drivers/gpu/drm/loongson/lsdc_ttm.h
+++ b/drivers/gpu/drm/loongson/lsdc_ttm.h
@@ -89,7 +89,7 @@ size_t lsdc_bo_size(struct lsdc_bo *lbo);
int lsdc_bo_kmap(struct lsdc_bo *lbo);
void lsdc_bo_kunmap(struct lsdc_bo *lbo);
-void lsdc_bo_clear(struct lsdc_bo *lbo);
+int lsdc_bo_clear(struct lsdc_bo *lbo);
int lsdc_bo_evict_vram(struct drm_device *ddev);
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
index aab5a73f467a..ff6858f5e5d3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
+++ b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
@@ -157,6 +157,7 @@ nvkm_subdev_init_(struct nvkm_subdev *subdev)
ret = subdev->func->init(subdev);
if (ret) {
nvkm_error(subdev, "init failed, %d\n", ret);
+ subdev->use.enabled = false;
return ret;
}
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
index 5368e705e7fd..186e09ea0cb1 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
@@ -27,6 +27,7 @@
#include <subdev/bios/init.h>
#include <subdev/bios/pll.h>
#include <subdev/clk/pll.h>
+#include <subdev/vga.h>
int
gf100_devinit_pll_set(struct nvkm_devinit *init, u32 type, u32 freq)
@@ -99,6 +100,25 @@ gf100_devinit_preinit(struct nvkm_devinit *base)
* can use it as a reliable way to know whether we should run devinit.
*/
base->post = ((nvkm_rd32(device, 0x2240c) & BIT(1)) == 0);
+
+ /*
+ * However some VBIOS init sequences miss this bit. So fall back to
+ * the nv50 method of checking if the bit is not set. Running the
+ * VBIOS multiple times may have detrimental effects.
+ */
+ if (base->post) {
+ u64 disable = nvkm_devinit_disable(base);
+ /* magic to detect whether or not x86 vbios code has
+ * executed the devinit scripts to initialise the
+ * board. only works if there's a display engine.
+ */
+ if (!(disable & (1ULL << NVKM_ENGINE_DISP))) {
+ if (nvkm_rdvgac(device, 0, 0x00) ||
+ nvkm_rdvgac(device, 0, 0x1a)) {
+ base->post = false;
+ }
+ }
+ }
}
static const struct nvkm_devinit_func
diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index 673ca9cbf362..0206bbad2ce0 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -130,10 +130,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,
INIT_LIST_HEAD(&entity->list);
entity->guilty = guilty;
- entity->priority = priority;
entity->last_user = current->group_leader;
- entity->rq_priority = drm_sched_policy == DRM_SCHED_POLICY_FAIR ?
- DRM_SCHED_PRIORITY_KERNEL : priority;
entity->num_sched_list = num_sched_list;
entity->sched_list = num_sched_list > 1 ? sched_list : NULL;
RCU_INIT_POINTER(entity->last_scheduled, NULL);
@@ -145,26 +142,26 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,
* not to do this and to fix their DRM calling order.
*/
pr_warn("%s: called with uninitialized scheduler\n", __func__);
- } else {
- enum drm_sched_priority p = entity->priority;
-
+ } else if (priority >= sched_list[0]->num_user_rqs) {
/*
* The "priority" of an entity cannot exceed the number of
* run-queues of a scheduler. Protect against num_rqs being 0,
* by converting to signed. Choose the lowest priority
* available.
*/
- if (p >= sched_list[0]->num_user_rqs) {
- dev_err(sched_list[0]->dev, "entity with out-of-bounds priority:%u num_user_rqs:%u\n",
- p, sched_list[0]->num_user_rqs);
- p = max_t(s32,
- (s32)sched_list[0]->num_user_rqs - 1,
- (s32)DRM_SCHED_PRIORITY_KERNEL);
- entity->priority = p;
- }
- entity->rq = sched_list[0]->sched_rq[entity->rq_priority];
+ dev_err(sched_list[0]->dev, "entity with out-of-bounds priority:%u num_user_rqs:%u\n",
+ priority, sched_list[0]->num_user_rqs);
+ priority = max_t(s32,
+ (s32)sched_list[0]->num_user_rqs - 1,
+ (s32)DRM_SCHED_PRIORITY_KERNEL);
}
+ entity->priority = priority;
+ entity->rq_priority = drm_sched_policy == DRM_SCHED_POLICY_FAIR ?
+ DRM_SCHED_PRIORITY_KERNEL : priority;
+ if (sched_list[0]->sched_rq)
+ entity->rq = sched_list[0]->sched_rq[entity->rq_priority];
+
init_completion(&entity->entity_idle);
/* We start in an idle state. */
@@ -457,6 +454,8 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
{
spin_lock(&entity->lock);
entity->priority = priority;
+ if (drm_sched_policy != DRM_SCHED_POLICY_FAIR)
+ entity->rq_priority = priority;
spin_unlock(&entity->lock);
}
EXPORT_SYMBOL(drm_sched_entity_set_priority);
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index bc88ebdcdbea..cf3df526e773 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -217,6 +217,8 @@ v3d_clean_caches(struct v3d_dev *v3d)
struct drm_device *dev = &v3d->drm;
int core = 0;
+ guard(mutex)(&v3d->cache_clean_lock);
+
trace_v3d_cache_clean_begin(dev);
/* GFXH-1897: Ensure pending flushes complete before writing L2TCACTL */
@@ -233,7 +235,6 @@ v3d_clean_caches(struct v3d_dev *v3d)
drm_err(dev, "Timeout waiting for TMU write combiner flush\n");
}
- mutex_lock(&v3d->cache_clean_lock);
V3D_CORE_WRITE(core, V3D_CTL_L2TCACTL,
V3D_L2TCACTL_L2TFLS |
V3D_SET_FIELD(V3D_L2TCACTL_FLM_CLEAN, V3D_L2TCACTL_FLM));
@@ -243,8 +244,6 @@ v3d_clean_caches(struct v3d_dev *v3d)
drm_err(dev, "Timeout waiting for L2T clean\n");
}
- mutex_unlock(&v3d->cache_clean_lock);
-
trace_v3d_cache_clean_end(dev);
}
diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c
index e6283a75bcdf..002116efc874 100644
--- a/drivers/gpu/drm/v3d/v3d_sched.c
+++ b/drivers/gpu/drm/v3d/v3d_sched.c
@@ -360,6 +360,8 @@ v3d_csd_job_run(struct drm_sched_job *sched_job)
dma_fence_put(job->base.irq_fence);
job->base.irq_fence = dma_fence_get(fence);
+ job->timedout_batches = job->args.cfg[4];
+
trace_v3d_submit_csd(dev, to_v3d_fence(fence)->seqno);
v3d_job_start_stats(&job->base);
diff --git a/drivers/gpu/drm/vc4/vc4_gem.c b/drivers/gpu/drm/vc4/vc4_gem.c
index 6ee4bd77d433..0fef9b48d3c6 100644
--- a/drivers/gpu/drm/vc4/vc4_gem.c
+++ b/drivers/gpu/drm/vc4/vc4_gem.c
@@ -639,13 +639,14 @@ vc4_queue_submit(struct drm_device *dev, struct vc4_exec_info *exec,
fence->seqno = exec->seqno;
exec->fence = &fence->base;
+ /* The job can complete and drop exec->fence as soon as job_lock is
+ * released, so hold our own reference.
+ */
if (out_sync)
- drm_syncobj_replace_fence(out_sync, exec->fence);
+ dma_fence_get(&fence->base);
vc4_attach_fences(exec);
- drm_exec_fini(exec_ctx);
-
list_add_tail(&exec->head, &vc4->bin_job_list);
/* If no bin job was executing and if the render job (if any) has the
@@ -662,6 +663,13 @@ vc4_queue_submit(struct drm_device *dev, struct vc4_exec_info *exec,
spin_unlock_irqrestore(&vc4->job_lock, irqflags);
+ drm_exec_fini(exec_ctx);
+
+ if (out_sync) {
+ drm_syncobj_replace_fence(out_sync, &fence->base);
+ dma_fence_put(&fence->base);
+ }
+
return 0;
}
diff --git a/drivers/gpu/drm/vc4/vc4_hdmi.c b/drivers/gpu/drm/vc4/vc4_hdmi.c
index d4672aa81d85..f2043f907dc5 100644
--- a/drivers/gpu/drm/vc4/vc4_hdmi.c
+++ b/drivers/gpu/drm/vc4/vc4_hdmi.c
@@ -842,8 +842,7 @@ static void vc4_hdmi_disable_scrambling(struct drm_encoder *encoder)
vc4_hdmi->scdc_enabled = false;
- if (delayed_work_pending(&vc4_hdmi->scrambling_work))
- cancel_delayed_work_sync(&vc4_hdmi->scrambling_work);
+ cancel_delayed_work_sync(&vc4_hdmi->scrambling_work);
if (!drm_dev_enter(drm, &idx))
return;
@@ -918,8 +917,6 @@ static void vc4_hdmi_encoder_post_crtc_disable(struct drm_encoder *encoder,
spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
- vc4_hdmi_disable_scrambling(encoder);
-
drm_dev_exit(idx);
out:
@@ -947,6 +944,8 @@ static void vc4_hdmi_encoder_post_crtc_powerdown(struct drm_encoder *encoder,
if (vc4_hdmi->variant->phy_disable)
vc4_hdmi->variant->phy_disable(vc4_hdmi);
+ vc4_hdmi_disable_scrambling(encoder);
+
clk_disable_unprepare(vc4_hdmi->pixel_bvb_clock);
clk_disable_unprepare(vc4_hdmi->pixel_clock);
@@ -1554,6 +1553,8 @@ static void vc4_hdmi_encoder_pre_crtc_configure(struct drm_encoder *encoder,
goto err_disable_pixel_clock;
}
+ vc4_hdmi_enable_scrambling(encoder);
+
if (vc4_hdmi->variant->phy_init)
vc4_hdmi->variant->phy_init(vc4_hdmi, conn_state);
@@ -1688,7 +1689,6 @@ static void vc4_hdmi_encoder_post_crtc_enable(struct drm_encoder *encoder,
}
vc4_hdmi_recenter_fifo(vc4_hdmi);
- vc4_hdmi_enable_scrambling(encoder);
drm_dev_exit(idx);
diff --git a/drivers/gpu/drm/vc4/vc4_hdmi_phy.c b/drivers/gpu/drm/vc4/vc4_hdmi_phy.c
index 56e6a35da357..5040f014bf7e 100644
--- a/drivers/gpu/drm/vc4/vc4_hdmi_phy.c
+++ b/drivers/gpu/drm/vc4/vc4_hdmi_phy.c
@@ -1197,4 +1197,9 @@ void vc6_hdmi_phy_init(struct vc4_hdmi *vc4_hdmi,
void vc6_hdmi_phy_disable(struct vc4_hdmi *vc4_hdmi)
{
+ unsigned long flags;
+
+ spin_lock_irqsave(&vc4_hdmi->hw_lock, flags);
+ vc6_hdmi_reset_phy(vc4_hdmi);
+ spin_unlock_irqrestore(&vc4_hdmi->hw_lock, flags);
}
diff --git a/drivers/hwmon/aquacomputer_d5next.c b/drivers/hwmon/aquacomputer_d5next.c
index 1ca70e726298..3d3786c82931 100644
--- a/drivers/hwmon/aquacomputer_d5next.c
+++ b/drivers/hwmon/aquacomputer_d5next.c
@@ -19,6 +19,7 @@
#include <linux/hwmon.h>
#include <linux/jiffies.h>
#include <linux/ktime.h>
+#include <linux/limits.h>
#include <linux/module.h>
#include <linux/seq_file.h>
#include <linux/unaligned.h>
@@ -132,6 +133,7 @@ static u8 aquaero_secondary_ctrl_report[] = {
#define AQUAERO_FAN_CURRENT_OFFSET 0x06
#define AQUAERO_FAN_POWER_OFFSET 0x08
#define AQUAERO_FAN_SPEED_OFFSET 0x00
+#define AQUAERO_MIN_STATUS_REPORT_SIZE (0x18B + AQUAERO_FAN_POWER_OFFSET + sizeof(u16))
static u16 aquaero_sensor_fan_offsets[] = { 0x167, 0x173, 0x17f, 0x18B };
/* Control report offsets for the Aquaero fan controllers */
@@ -155,6 +157,8 @@ static u16 aquaero_ctrl_fan_offsets[] = { 0x20c, 0x220, 0x234, 0x248 };
#define D5NEXT_5V_VOLTAGE 0x39
#define D5NEXT_12V_VOLTAGE 0x37
#define D5NEXT_VIRTUAL_SENSORS_START 0x3f
+#define D5NEXT_MIN_STATUS_REPORT_SIZE (D5NEXT_PUMP_OFFSET + AQC_FAN_SPEED_OFFSET + \
+ sizeof(u16))
static u16 d5next_sensor_fan_offsets[] = { D5NEXT_PUMP_OFFSET, D5NEXT_FAN_OFFSET };
/* Control report offsets for the D5 Next pump */
@@ -179,11 +183,14 @@ static u16 d5next_ctrl_fan_offsets[] = { 0x97, 0x42 }; /* Pump and fan speed (fr
#define AQUASTREAMULT_FAN_CURRENT_OFFSET 0x00
#define AQUASTREAMULT_FAN_POWER_OFFSET 0x04
#define AQUASTREAMULT_FAN_SPEED_OFFSET 0x06
+#define AQUASTREAMULT_MIN_STATUS_REPORT_SIZE (AQUASTREAMULT_PRESSURE_OFFSET + sizeof(u16))
static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET };
/* Spec and sensor report offset for the Farbwerk RGB controller */
#define FARBWERK_NUM_SENSORS 4
#define FARBWERK_SENSOR_START 0x2f
+#define FARBWERK_MIN_STATUS_REPORT_SIZE (FARBWERK_SENSOR_START + \
+ FARBWERK_NUM_SENSORS * AQC_SENSOR_SIZE)
/* Specs of the Farbwerk 360 RGB controller */
#define FARBWERK360_NUM_SENSORS 4
@@ -193,6 +200,8 @@ static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET };
/* Sensor report offsets for the Farbwerk 360 */
#define FARBWERK360_SENSOR_START 0x32
#define FARBWERK360_VIRTUAL_SENSORS_START 0x3a
+#define FARBWERK360_MIN_STATUS_REPORT_SIZE (FARBWERK360_VIRTUAL_SENSORS_START + \
+ FARBWERK360_NUM_VIRTUAL_SENSORS * AQC_SENSOR_SIZE)
/* Control report offsets for the Farbwerk 360 */
#define FARBWERK360_TEMP_CTRL_OFFSET 0x8
@@ -209,6 +218,7 @@ static u16 aquastreamult_sensor_fan_offsets[] = { AQUASTREAMULT_FAN_OFFSET };
#define OCTO_SENSOR_START 0x3D
#define OCTO_VIRTUAL_SENSORS_START 0x45
#define OCTO_FLOW_SENSOR_OFFSET 0x7B
+#define OCTO_MIN_STATUS_REPORT_SIZE (0xD8 + AQC_FAN_SPEED_OFFSET + sizeof(u16))
static u16 octo_sensor_fan_offsets[] = { 0x7D, 0x8A, 0x97, 0xA4, 0xB1, 0xBE, 0xCB, 0xD8 };
/* Control report offsets for the Octo */
@@ -229,6 +239,7 @@ static u16 octo_ctrl_fan_offsets[] = { 0x5B, 0xB0, 0x105, 0x15A, 0x1AF, 0x204, 0
#define QUADRO_SENSOR_START 0x34
#define QUADRO_VIRTUAL_SENSORS_START 0x3c
#define QUADRO_FLOW_SENSOR_OFFSET 0x6e
+#define QUADRO_MIN_STATUS_REPORT_SIZE (0x97 + AQC_FAN_SPEED_OFFSET + sizeof(u16))
static u16 quadro_sensor_fan_offsets[] = { 0x70, 0x7D, 0x8A, 0x97 };
/* Control report offsets for the Quadro */
@@ -248,6 +259,7 @@ static u16 quadro_ctrl_fan_offsets[] = { 0x37, 0x8c, 0xe1, 0x136 }; /* Fan speed
#define HIGHFLOWNEXT_CONDUCTIVITY 95
#define HIGHFLOWNEXT_5V_VOLTAGE 97
#define HIGHFLOWNEXT_5V_VOLTAGE_USB 99
+#define HIGHFLOWNEXT_MIN_STATUS_REPORT_SIZE (HIGHFLOWNEXT_5V_VOLTAGE_USB + sizeof(u16))
/* Specs of the Leakshield */
#define LEAKSHIELD_NUM_SENSORS 2
@@ -263,6 +275,7 @@ static u16 quadro_ctrl_fan_offsets[] = { 0x37, 0x8c, 0xe1, 0x136 }; /* Fan speed
#define LEAKSHIELD_FLOW_IN 111
#define LEAKSHIELD_RESERVOIR_VOLUME 313
#define LEAKSHIELD_RESERVOIR_FILLED 311
+#define LEAKSHIELD_MIN_STATUS_REPORT_SIZE (LEAKSHIELD_RESERVOIR_VOLUME + sizeof(u16))
/* Specs of the Aquastream XT pump */
#define AQUASTREAMXT_SERIAL_START 0x3a
@@ -554,6 +567,7 @@ struct aqc_data {
const char *name;
int status_report_id; /* Used for legacy devices, report is stored in buffer */
+ u16 min_report_size;
int ctrl_report_id;
int secondary_ctrl_report_id;
int secondary_ctrl_report_size;
@@ -599,9 +613,9 @@ struct aqc_data {
u32 speed_input_min[1];
u32 speed_input_target[1];
u32 speed_input_max[1];
- u32 power_input[8];
+ long power_input[8];
u16 voltage_input[8];
- u16 current_input[8];
+ u32 current_input[8];
/* Label values */
const char *const *temp_label;
@@ -976,7 +990,7 @@ static int aqc_legacy_read(struct aqc_data *priv)
/* Calculation derived from linear regression */
sensor_value = get_unaligned_le16(priv->buffer + AQUASTREAMXT_PUMP_CURR_OFFSET);
- priv->current_input[0] = DIV_ROUND_CLOSEST(sensor_value * 176, 100) - 52;
+ priv->current_input[0] = max(DIV_ROUND_CLOSEST(sensor_value * 176, 100) - 52, 0);
sensor_value = get_unaligned_le16(priv->buffer + AQUASTREAMXT_PUMP_VOLTAGE_OFFSET);
priv->voltage_input[0] = DIV_ROUND_CLOSEST(sensor_value * 1000, 61);
@@ -1006,6 +1020,7 @@ static int aqc_legacy_read(struct aqc_data *priv)
static int aqc_read(struct device *dev, enum hwmon_sensor_types type, u32 attr,
int channel, long *val)
{
+ long power;
int ret;
struct aqc_data *priv = dev_get_drvdata(dev);
@@ -1070,7 +1085,10 @@ static int aqc_read(struct device *dev, enum hwmon_sensor_types type, u32 attr,
}
break;
case hwmon_power:
- *val = priv->power_input[channel];
+ power = READ_ONCE(priv->power_input[channel]);
+ if (power == -ENODATA)
+ return -ENODATA;
+ *val = power;
break;
case hwmon_pwm:
switch (priv->kind) {
@@ -1333,6 +1351,8 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
return 0;
priv = hid_get_drvdata(hdev);
+ if (!priv->min_report_size || size < priv->min_report_size)
+ return 0;
/* Info provided with every report */
priv->serial_number[0] = get_unaligned_be16(data + priv->serial_number_start_offset);
@@ -1368,9 +1388,9 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
priv->speed_input[i] =
get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
priv->fan_structure->speed);
- priv->power_input[i] =
- get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
- priv->fan_structure->power) * 10000;
+ WRITE_ONCE(priv->power_input[i],
+ get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
+ priv->fan_structure->power) * 10000);
priv->voltage_input[i] =
get_unaligned_be16(data + priv->fan_sensor_offsets[i] +
priv->fan_structure->voltage) * 10;
@@ -1410,7 +1430,8 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
priv->speed_input[2] = get_unaligned_be16(data + AQUASTREAMULT_PRESSURE_OFFSET);
priv->speed_input[3] = get_unaligned_be16(data + AQUASTREAMULT_FLOW_SENSOR_OFFSET);
- priv->power_input[1] = get_unaligned_be16(data + AQUASTREAMULT_PUMP_POWER) * 10000;
+ WRITE_ONCE(priv->power_input[1],
+ get_unaligned_be16(data + AQUASTREAMULT_PUMP_POWER) * 10000);
priv->voltage_input[1] = get_unaligned_be16(data + AQUASTREAMULT_PUMP_VOLTAGE) * 10;
@@ -1423,10 +1444,11 @@ static int aqc_raw_event(struct hid_device *hdev, struct hid_report *report, u8
case highflownext:
/* If external temp sensor is not connected, its power reading is also N/A */
if (priv->temp_input[1] == -ENODATA)
- priv->power_input[0] = -ENODATA;
+ WRITE_ONCE(priv->power_input[0], -ENODATA);
else
- priv->power_input[0] =
- get_unaligned_be16(data + HIGHFLOWNEXT_POWER) * 1000000;
+ WRITE_ONCE(priv->power_input[0],
+ min_t(u64, get_unaligned_be16(data + HIGHFLOWNEXT_POWER) *
+ 1000000ULL, LONG_MAX));
priv->voltage_input[0] = get_unaligned_be16(data + HIGHFLOWNEXT_5V_VOLTAGE) * 10;
priv->voltage_input[1] =
@@ -1518,6 +1540,7 @@ static void aqc_debugfs_init(struct aqc_data *priv)
static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
{
struct aqc_data *priv;
+ u16 min_report_size = 0;
int ret;
priv = devm_kzalloc(&hdev->dev, sizeof(*priv), GFP_KERNEL);
@@ -1554,7 +1577,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
* they present. The two other devices have the type of the second element in
* their respective collections set to 1, while the real device has it set to 0.
*/
- if (hdev->collection[1].type != 0) {
+ if (hdev->maxcollection < 2 || hdev->collection[1].type != 0) {
ret = -ENODEV;
goto fail_and_close;
}
@@ -1585,6 +1608,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_fan_power;
priv->voltage_label = label_fan_voltage;
priv->current_label = label_fan_current;
+ min_report_size = AQUAERO_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_D5NEXT:
priv->kind = d5next;
@@ -1610,6 +1634,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_d5next_power;
priv->voltage_label = label_d5next_voltages;
priv->current_label = label_d5next_current;
+ min_report_size = D5NEXT_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_FARBWERK:
priv->kind = farbwerk;
@@ -1620,6 +1645,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->temp_sensor_start_offset = FARBWERK_SENSOR_START;
priv->temp_label = label_temp_sensors;
+ min_report_size = FARBWERK_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_FARBWERK360:
priv->kind = farbwerk360;
@@ -1636,6 +1662,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->temp_label = label_temp_sensors;
priv->virtual_temp_label = label_virtual_temp_sensors;
+ min_report_size = FARBWERK360_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_OCTO:
priv->kind = octo;
@@ -1665,6 +1692,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_fan_power;
priv->voltage_label = label_fan_voltage;
priv->current_label = label_fan_current;
+ min_report_size = OCTO_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_QUADRO:
priv->kind = quadro;
@@ -1694,6 +1722,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_fan_power;
priv->voltage_label = label_fan_voltage;
priv->current_label = label_fan_current;
+ min_report_size = QUADRO_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_HIGHFLOWNEXT:
priv->kind = highflownext;
@@ -1711,6 +1740,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->speed_label = label_highflownext_fan_speed;
priv->power_label = label_highflownext_power;
priv->voltage_label = label_highflownext_voltage;
+ min_report_size = HIGHFLOWNEXT_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_LEAKSHIELD:
/*
@@ -1730,6 +1760,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->temp_label = label_leakshield_temp_sensors;
priv->speed_label = label_leakshield_fan_speed;
+ min_report_size = LEAKSHIELD_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_AQUASTREAMXT:
priv->kind = aquastreamxt;
@@ -1760,6 +1791,7 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
priv->power_label = label_aquastreamult_power;
priv->voltage_label = label_aquastreamult_voltages;
priv->current_label = label_aquastreamult_current;
+ min_report_size = AQUASTREAMULT_MIN_STATUS_REPORT_SIZE;
break;
case USB_PRODUCT_ID_POWERADJUST3:
priv->kind = poweradjust3;
@@ -1833,6 +1865,8 @@ static int aqc_probe(struct hid_device *hdev, const struct hid_device_id *id)
break;
}
+ priv->min_report_size = min_report_size;
+
if (priv->buffer_size != 0) {
priv->checksum_start = 0x01;
priv->checksum_length = priv->buffer_size - 3;
diff --git a/drivers/hwmon/asus_rog_ryujin.c b/drivers/hwmon/asus_rog_ryujin.c
index e297557ca346..a400e523d2e9 100644
--- a/drivers/hwmon/asus_rog_ryujin.c
+++ b/drivers/hwmon/asus_rog_ryujin.c
@@ -49,6 +49,7 @@ static const struct rog_ryujin_device_info rog_ryujin_iii_info = {
#define MAX_REPORT_LENGTH 65
/* Cooler duty report offsets */
+#define RYUJIN_PUMP_SOURCE 3
#define RYUJIN_PUMP_DUTY 4
#define RYUJIN_INTERNAL_FAN_DUTY 5
@@ -69,10 +70,15 @@ static const u8 get_cooler_duty_cmd[] = { RYUJIN_CMD_PREFIX, 0x9A };
static const u8 get_controller_speed_cmd[] = { RYUJIN_CMD_PREFIX, 0xA0 };
static const u8 get_controller_duty_cmd[] = { RYUJIN_CMD_PREFIX, 0xA1 };
+#define RYUJIN_SET_COOLER_CHANNEL_OFFSET 2
#define RYUJIN_SET_COOLER_PUMP_DUTY_OFFSET 3
#define RYUJIN_SET_COOLER_FAN_DUTY_OFFSET 4
static const u8 set_cooler_duty_cmd[] = { RYUJIN_CMD_PREFIX, 0x1A, 0x00, 0x00, 0x00 };
+/* Pump control sources in the set cooler duty command and the cooler duty report */
+#define RYUJIN_SOURCE_HEADER 0
+#define RYUJIN_SOURCE_USB 1
+
#define RYUJIN_SET_CONTROLLER_FAN_DUTY_OFFSET 4
static const u8 set_controller_duty_cmd[] = { RYUJIN_CMD_PREFIX, 0x21, 0x00, 0x00, 0x00 };
@@ -116,6 +122,7 @@ struct rog_ryujin_data {
s32 temp_input[1];
u16 speed_input[6]; /* Pump, internal fan and four controller fan speeds in RPM */
u8 duty_input[3]; /* Pump, internal fan and controller fan duty in PWM */
+ u8 pump_source; /* Pump control source from the cooler duty report */
u8 *buffer;
unsigned long updated; /* jiffies */
@@ -305,6 +312,44 @@ static int rog_ryujin_read_string(struct device *dev, enum hwmon_sensor_types ty
return 0;
}
+/*
+ * On devices with a non-zero duty_channel (the RYUJIN III), the channel byte of
+ * the set cooler duty command selects the pump control source.
+ * RYUJIN_SOURCE_HEADER hands the pump to its internal controller, which follows
+ * the PWM of the header its 4-pin cable is plugged into. RYUJIN_SOURCE_USB
+ * makes the pump follow the pump duty in the command. The cooler duty report
+ * returns the current source.
+ *
+ * A fan duty write sends the reported source back, so that it leaves the pump
+ * control source as it is. A pump duty write selects USB. The cooler does not
+ * pass on a pump duty equal to the one it reports, and after power-on it
+ * reports 40% while a pump handed over at that duty runs at full speed. So
+ * when a pump duty write asks for the reported duty, write a duty 1% away
+ * first.
+ */
+static int rog_ryujin_select_pump_source(struct rog_ryujin_data *priv, u8 *set_cmd,
+ int channel)
+{
+ u8 duty = set_cmd[RYUJIN_SET_COOLER_PUMP_DUTY_OFFSET];
+ int ret;
+
+ if (channel == 1) {
+ set_cmd[RYUJIN_SET_COOLER_CHANNEL_OFFSET] = priv->pump_source;
+ return 0;
+ }
+
+ set_cmd[RYUJIN_SET_COOLER_CHANNEL_OFFSET] = RYUJIN_SOURCE_USB;
+
+ if (duty != rog_ryujin_pwm_to_percent(priv->duty_input[0]))
+ return 0;
+
+ set_cmd[RYUJIN_SET_COOLER_PUMP_DUTY_OFFSET] = duty < 100 ? duty + 1 : duty - 1;
+ ret = rog_ryujin_execute_cmd(priv, set_cmd, SET_CMD_LENGTH, &priv->cooler_duty_set);
+ set_cmd[RYUJIN_SET_COOLER_PUMP_DUTY_OFFSET] = duty;
+
+ return ret;
+}
+
static int rog_ryujin_write_fixed_duty(struct rog_ryujin_data *priv, int channel, int val)
{
u8 set_cmd[SET_CMD_LENGTH];
@@ -322,7 +367,7 @@ static int rog_ryujin_write_fixed_duty(struct rog_ryujin_data *priv, int channel
return ret;
memcpy(set_cmd, set_cooler_duty_cmd, SET_CMD_LENGTH);
- set_cmd[2] = priv->info->duty_channel;
+ set_cmd[RYUJIN_SET_COOLER_CHANNEL_OFFSET] = priv->info->duty_channel;
/* Cooler duties are set as 0-100% */
val = rog_ryujin_pwm_to_percent(val);
@@ -339,6 +384,12 @@ static int rog_ryujin_write_fixed_duty(struct rog_ryujin_data *priv, int channel
set_cmd[RYUJIN_SET_COOLER_FAN_DUTY_OFFSET] = val;
}
+ if (priv->info->duty_channel) {
+ ret = rog_ryujin_select_pump_source(priv, set_cmd, channel);
+ if (ret < 0)
+ return ret;
+ }
+
return rog_ryujin_execute_cmd(priv, set_cmd, SET_CMD_LENGTH, &priv->cooler_duty_set);
} else {
/*
@@ -481,6 +532,7 @@ static int rog_ryujin_raw_event(struct hid_device *hdev, struct hid_report *repo
goto unlock;
}
read_cooler_duty:
+ priv->pump_source = data[RYUJIN_PUMP_SOURCE];
priv->duty_input[0] = rog_ryujin_percent_to_pwm(data[RYUJIN_PUMP_DUTY]);
priv->duty_input[1] = rog_ryujin_percent_to_pwm(data[RYUJIN_INTERNAL_FAN_DUTY]);
diff --git a/drivers/hwmon/asus_wmi_sensors.c b/drivers/hwmon/asus_wmi_sensors.c
index c2dd7ff882f2..502996e97cf1 100644
--- a/drivers/hwmon/asus_wmi_sensors.c
+++ b/drivers/hwmon/asus_wmi_sensors.c
@@ -164,6 +164,7 @@ struct asus_wmi_wmi_info {
struct asus_wmi_sensors {
struct asus_wmi_wmi_info wmi;
+ struct hwmon_chip_info chip_info;
/* lock access to internal cache */
struct mutex lock;
};
@@ -496,11 +497,6 @@ static const struct hwmon_ops asus_wmi_hwmon_ops = {
.read_string = asus_wmi_hwmon_read_string,
};
-static struct hwmon_chip_info asus_wmi_chip_info = {
- .ops = &asus_wmi_hwmon_ops,
- .info = NULL,
-};
-
static int asus_wmi_configure_sensor_setup(struct device *dev,
struct asus_wmi_sensors *sensor_data)
{
@@ -508,7 +504,7 @@ static int asus_wmi_configure_sensor_setup(struct device *dev,
struct hwmon_channel_info *asus_wmi_hwmon_chan;
int nr_count[hwmon_max] = {}, nr_types = 0;
struct asus_wmi_sensor_info *temp_sensor;
- const struct hwmon_chip_info *chip_info;
+ struct hwmon_chip_info *chip_info;
enum hwmon_sensor_types type;
struct device *hwdev;
int i, idx;
@@ -549,8 +545,9 @@ static int asus_wmi_configure_sensor_setup(struct device *dev,
if (!ptr_asus_wmi_ci)
return -ENOMEM;
- asus_wmi_chip_info.info = ptr_asus_wmi_ci;
- chip_info = &asus_wmi_chip_info;
+ chip_info = &sensor_data->chip_info;
+ chip_info->ops = &asus_wmi_hwmon_ops;
+ chip_info->info = ptr_asus_wmi_ci;
sensor_data->wmi.info_by_id = devm_kcalloc(dev, sensor_data->wmi.sensor_count,
sizeof(*sensor_data->wmi.info_by_id),
diff --git a/drivers/hwmon/chipcap2.c b/drivers/hwmon/chipcap2.c
index 9bef767b589e..00f62a33d463 100644
--- a/drivers/hwmon/chipcap2.c
+++ b/drivers/hwmon/chipcap2.c
@@ -20,7 +20,9 @@
#include <linux/i2c.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
+#include <linux/mutex.h>
#include <linux/module.h>
+#include <linux/unaligned.h>
#include <linux/regulator/consumer.h>
#include <linux/mod_devicetable.h>
#include <linux/property.h>
@@ -90,6 +92,9 @@ struct cc2_data {
int irq_low;
int irq_high;
bool process_irqs;
+ /* protects process_irqs and the alarm indicators */
+ struct mutex irq_lock;
+ bool regulator_enabled;
};
/* %RH as a per cent mille from a register value */
@@ -118,20 +123,33 @@ static int cc2_enable(struct cc2_data *data)
{
int ret;
+ if (data->irq_ready > 0) {
+ scoped_guard(mutex, &data->irq_lock)
+ data->process_irqs = false;
+
+ synchronize_irq(data->irq_ready);
+
+ scoped_guard(mutex, &data->irq_lock)
+ reinit_completion(&data->complete);
+ }
+
/* exclusive regulator, check in case a disable failed */
- if (regulator_is_enabled(data->regulator))
+ if (data->regulator_enabled) {
+ scoped_guard(mutex, &data->irq_lock)
+ data->process_irqs = true;
return 0;
-
- /* clear any pending completion */
- try_wait_for_completion(&data->complete);
+ }
ret = regulator_enable(data->regulator);
if (ret < 0)
return ret;
+ data->regulator_enabled = true;
+
usleep_range(CC2_STARTUP_TIME_US, CC2_STARTUP_TIME_US + 125);
- data->process_irqs = true;
+ scoped_guard(mutex, &data->irq_lock)
+ data->process_irqs = true;
return 0;
}
@@ -140,15 +158,36 @@ static void cc2_disable(struct cc2_data *data)
{
int err;
+ /* let the alarms already raised by the last measurement be processed */
+ if (data->irq_low > 0)
+ synchronize_irq(data->irq_low);
+
+ if (data->irq_high > 0)
+ synchronize_irq(data->irq_high);
+
/* ignore alarms triggered by voltage toggling when powering up */
- data->process_irqs = false;
+ scoped_guard(mutex, &data->irq_lock)
+ data->process_irqs = false;
+
+ if (data->irq_ready > 0)
+ synchronize_irq(data->irq_ready);
/* exclusive regulator, check in case an enable failed */
- if (regulator_is_enabled(data->regulator)) {
- err = regulator_disable(data->regulator);
- if (err)
- dev_dbg(&data->client->dev, "Failed to disable device");
+ if (!data->regulator_enabled)
+ return;
+
+ err = regulator_disable(data->regulator);
+ if (err) {
+ dev_dbg(&data->client->dev, "Failed to disable device");
+ return;
}
+
+ data->regulator_enabled = false;
+}
+
+static void cc2_disable_action(void *data)
+{
+ cc2_disable(data);
}
static int cc2_cmd_response_diagnostic(struct device *dev, u8 status)
@@ -167,13 +206,13 @@ static int cc2_cmd_response_diagnostic(struct device *dev, u8 status)
case CC2_RESPONSE_BUSY:
return -EBUSY;
case CC2_RESPONSE_NACK:
- if (resp & CC2_ERR_CORR_EEPROM)
+ if (status & CC2_ERR_CORR_EEPROM)
dev_dbg(dev, "Command failed: corrected EEPROM\n");
- if (resp & CC2_ERR_UNCORR_EEPROM)
+ if (status & CC2_ERR_UNCORR_EEPROM)
dev_dbg(dev, "Command failed: uncorrected EEPROM\n");
- if (resp & CC2_ERR_RAM_PARITY)
+ if (status & CC2_ERR_RAM_PARITY)
dev_dbg(dev, "Command failed: RAM parity\n");
- if (resp & CC2_ERR_RAM_PARITY)
+ if (status & CC2_ERR_CONFIG_LOAD)
dev_dbg(dev, "Command failed: configuration error\n");
return -ENODATA;
default:
@@ -226,7 +265,7 @@ static int cc2_command_mode_start(struct cc2_data *data)
2 * CC2_RESP_START_CM_US);
}
ret = cc2_read_command_status(data->client);
- if (ret != -ETIMEDOUT || i == CC2_CM_RETRIES)
+ if (ret != -ETIMEDOUT || i == CC2_CM_RETRIES - 1)
break;
/* command window missed, prepare for a retry */
@@ -260,8 +299,7 @@ static int cc2_write_reg(struct cc2_data *data, u8 reg, u16 val)
if (ret < 0)
goto disable;
- cpu_to_be16s(&val);
- ret = i2c_smbus_write_word_data(data->client, reg, val);
+ ret = i2c_smbus_write_word_swapped(data->client, reg, val);
if (ret < 0)
goto disable;
@@ -311,7 +349,7 @@ static int cc2_read_reg(struct cc2_data *data, u8 reg, u16 *val)
if (ret != CC2_EEPROM_DATA_LEN)
return ret < 0 ? ret : -EIO;
- *val = be16_to_cpup((__be16 *)&buf[1]);
+ *val = get_unaligned_be16(&buf[1]);
return cc2_read_command_status(data->client);
}
@@ -351,10 +389,10 @@ static int cc2_data_fetch(struct i2c_client *client,
switch (type) {
case hwmon_humidity:
- *val = cc2_rh_convert(be16_to_cpup((__be16 *)&data[0]));
+ *val = cc2_rh_convert(get_unaligned_be16(&data[0]));
break;
case hwmon_temp:
- *val = cc2_temp_convert(be16_to_cpup((__be16 *)&data[2]));
+ *val = cc2_temp_convert(get_unaligned_be16(&data[2]));
break;
default:
return -EINVAL;
@@ -482,8 +520,10 @@ static irqreturn_t cc2_ready_interrupt(int irq, void *data)
{
struct cc2_data *cc2 = data;
- if (cc2->process_irqs)
- complete(&cc2->complete);
+ scoped_guard(mutex, &cc2->irq_lock) {
+ if (cc2->process_irqs)
+ complete(&cc2->complete);
+ }
return IRQ_HANDLED;
}
@@ -492,10 +532,12 @@ static irqreturn_t cc2_low_interrupt(int irq, void *data)
{
struct cc2_data *cc2 = data;
- if (cc2->process_irqs) {
- hwmon_notify_event(cc2->hwmon, hwmon_humidity,
- hwmon_humidity_min_alarm, 0);
- cc2->rh_alarm.low_alarm = true;
+ scoped_guard(mutex, &cc2->irq_lock) {
+ if (cc2->process_irqs) {
+ hwmon_notify_event(cc2->hwmon, hwmon_humidity,
+ hwmon_humidity_min_alarm, 0);
+ cc2->rh_alarm.low_alarm = true;
+ }
}
return IRQ_HANDLED;
@@ -505,10 +547,12 @@ static irqreturn_t cc2_high_interrupt(int irq, void *data)
{
struct cc2_data *cc2 = data;
- if (cc2->process_irqs) {
- hwmon_notify_event(cc2->hwmon, hwmon_humidity,
- hwmon_humidity_max_alarm, 0);
- cc2->rh_alarm.high_alarm = true;
+ scoped_guard(mutex, &cc2->irq_lock) {
+ if (cc2->process_irqs) {
+ hwmon_notify_event(cc2->hwmon, hwmon_humidity,
+ hwmon_humidity_max_alarm, 0);
+ cc2->rh_alarm.high_alarm = true;
+ }
}
return IRQ_HANDLED;
@@ -524,11 +568,13 @@ static int cc2_humidity_min_alarm_status(struct cc2_data *data, long *val)
if (ret < 0)
return ret;
- if (data->rh_alarm.low_alarm) {
- *val = (measurement < min_hyst) ? 1 : 0;
- data->rh_alarm.low_alarm = *val;
- } else {
- *val = 0;
+ scoped_guard(mutex, &data->irq_lock) {
+ if (data->rh_alarm.low_alarm) {
+ *val = (measurement < min_hyst) ? 1 : 0;
+ data->rh_alarm.low_alarm = *val;
+ } else {
+ *val = 0;
+ }
}
return 0;
@@ -544,11 +590,13 @@ static int cc2_humidity_max_alarm_status(struct cc2_data *data, long *val)
if (ret < 0)
return ret;
- if (data->rh_alarm.high_alarm) {
- *val = (measurement > max_hyst) ? 1 : 0;
- data->rh_alarm.high_alarm = *val;
- } else {
- *val = 0;
+ scoped_guard(mutex, &data->irq_lock) {
+ if (data->rh_alarm.high_alarm) {
+ *val = (measurement > max_hyst) ? 1 : 0;
+ data->rh_alarm.high_alarm = *val;
+ } else {
+ *val = 0;
+ }
}
return 0;
@@ -631,28 +679,47 @@ static int cc2_write(struct device *dev, enum hwmon_sensor_types type, u32 attr,
}
}
-static int cc2_request_ready_irq(struct cc2_data *data, struct device *dev)
+static int cc2_get_irqs(struct cc2_data *data, struct device *dev)
{
- int ret = 0;
-
data->irq_ready = fwnode_irq_get_byname(dev_fwnode(dev), "ready");
- if (data->irq_ready > 0) {
+ if (data->irq_ready == -EPROBE_DEFER)
+ return data->irq_ready;
+ if (data->irq_ready > 0)
init_completion(&data->complete);
+
+ data->irq_low = fwnode_irq_get_byname(dev_fwnode(dev), "low");
+ if (data->irq_low == -EPROBE_DEFER)
+ return data->irq_low;
+ if (data->irq_low > 0)
+ data->rh_alarm.low_alarm_visible = true;
+
+ data->irq_high = fwnode_irq_get_byname(dev_fwnode(dev), "high");
+ if (data->irq_high == -EPROBE_DEFER)
+ return data->irq_high;
+ if (data->irq_high > 0)
+ data->rh_alarm.high_alarm_visible = true;
+
+ return 0;
+}
+
+static int cc2_request_data_ready(struct cc2_data *data, struct device *dev)
+{
+ int ret = 0;
+
+ if (data->irq_ready > 0)
ret = devm_request_threaded_irq(dev, data->irq_ready, NULL,
cc2_ready_interrupt,
IRQF_ONESHOT |
IRQF_TRIGGER_RISING,
dev_name(dev), data);
- }
return ret;
}
-static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev)
+static int cc2_request_alarms(struct cc2_data *data, struct device *dev)
{
int ret = 0;
- data->irq_low = fwnode_irq_get_byname(dev_fwnode(dev), "low");
if (data->irq_low > 0) {
ret = devm_request_threaded_irq(dev, data->irq_low, NULL,
cc2_low_interrupt,
@@ -661,11 +728,8 @@ static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev)
dev_name(dev), data);
if (ret)
return ret;
-
- data->rh_alarm.low_alarm_visible = true;
}
- data->irq_high = fwnode_irq_get_byname(dev_fwnode(dev), "high");
if (data->irq_high > 0) {
ret = devm_request_threaded_irq(dev, data->irq_high, NULL,
cc2_high_interrupt,
@@ -674,8 +738,6 @@ static int cc2_request_alarm_irqs(struct cc2_data *data, struct device *dev)
dev_name(dev), data);
if (ret)
return ret;
-
- data->rh_alarm.high_alarm_visible = true;
}
return ret;
@@ -717,6 +779,8 @@ static int cc2_probe(struct i2c_client *client)
i2c_set_clientdata(client, data);
+ mutex_init(&data->irq_lock);
+
data->client = client;
data->regulator = devm_regulator_get_exclusive(dev, "vdd");
@@ -726,13 +790,17 @@ static int cc2_probe(struct i2c_client *client)
device_property_read_string(dev, "label", &data->label);
- ret = cc2_request_ready_irq(data, dev);
+ ret = cc2_get_irqs(data, dev);
if (ret)
- return dev_err_probe(dev, ret, "Failed to request ready irq\n");
+ return dev_err_probe(dev, ret, "Failed to get IRQs\n");
- ret = cc2_request_alarm_irqs(data, dev);
+ ret = devm_add_action_or_reset(dev, cc2_disable_action, data);
if (ret)
- return dev_err_probe(dev, ret, "Failed to request alarm irqs\n");
+ return ret;
+
+ ret = cc2_request_data_ready(data, dev);
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to request ready irq\n");
data->hwmon = devm_hwmon_device_register_with_info(dev, client->name,
data, &cc2_chip_info,
@@ -741,14 +809,11 @@ static int cc2_probe(struct i2c_client *client)
return dev_err_probe(dev, PTR_ERR(data->hwmon),
"Failed to register hwmon device\n");
- return 0;
-}
-
-static void cc2_remove(struct i2c_client *client)
-{
- struct cc2_data *data = i2c_get_clientdata(client);
+ ret = cc2_request_alarms(data, dev);
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to request alarm irqs\n");
- cc2_disable(data);
+ return 0;
}
static const struct i2c_device_id cc2_id[] = {
@@ -783,7 +848,6 @@ static struct i2c_driver cc2_driver = {
.of_match_table = cc2_of_match,
},
.probe = cc2_probe,
- .remove = cc2_remove,
.id_table = cc2_id,
};
module_i2c_driver(cc2_driver);
diff --git a/drivers/hwmon/coretemp.c b/drivers/hwmon/coretemp.c
index 9ed21c8c093a..2d7487b2f80a 100644
--- a/drivers/hwmon/coretemp.c
+++ b/drivers/hwmon/coretemp.c
@@ -491,11 +491,12 @@ init_temp_data(struct platform_data *pdata, unsigned int cpu, int pkg_flag)
struct temp_data *tdata;
if (!pdata->core_data) {
- pdata->nr_cores = topology_num_cores_per_package();
- pdata->core_data = kzalloc_objs(struct temp_data *,
- pdata->nr_cores);
+ int nr_cores = topology_num_cores_per_package();
+
+ pdata->core_data = kzalloc_objs(struct temp_data *, nr_cores);
if (!pdata->core_data)
return NULL;
+ pdata->nr_cores = nr_cores;
}
tdata = kzalloc_obj(struct temp_data);
@@ -695,6 +696,9 @@ static int coretemp_cpu_online(unsigned int cpu)
if (!housekeeping_cpu(cpu, HK_TYPE_MISC))
return 0;
+ if (!pdev)
+ return 0;
+
pdata = platform_get_drvdata(pdev);
if (!pdata->hwmon_dev) {
struct device *hwmon;
@@ -746,6 +750,9 @@ static int coretemp_cpu_offline(unsigned int cpu)
return 0;
/* If the physical CPU device does not exist, just return */
+ if (!pdev)
+ return 0;
+
pd = platform_get_drvdata(pdev);
if (!pd->hwmon_dev)
return 0;
diff --git a/drivers/hwmon/corsair-cpro.c b/drivers/hwmon/corsair-cpro.c
index e6d31d367698..6ce5da37de03 100644
--- a/drivers/hwmon/corsair-cpro.c
+++ b/drivers/hwmon/corsair-cpro.c
@@ -215,10 +215,11 @@ static int set_target(struct ccp_device *ccp, int channel, long val)
int ret;
val = clamp_val(val, 0, 0xFFFF);
- ccp->target[channel] = val;
mutex_lock(&ccp->mutex);
ret = send_usb_cmd(ccp, CTL_SET_FAN_TARGET, channel, val >> 8, val);
+ if (!ret)
+ ccp->target[channel] = val;
mutex_unlock(&ccp->mutex);
return ret;
diff --git a/drivers/hwmon/gpd-fan.c b/drivers/hwmon/gpd-fan.c
index ed212615b2fe..b3b75f99f588 100644
--- a/drivers/hwmon/gpd-fan.c
+++ b/drivers/hwmon/gpd-fan.c
@@ -444,6 +444,7 @@ static void gpd_duo_set_pwm_enable(struct gpd_fan_data *data, enum FAN_PWM_ENABL
break;
case AUTOMATIC:
gpd_ecram_write(data->drvdata, data->drvdata->pwm_write, 0);
+ gpd_ecram_write(data->drvdata, data->drvdata->pwm_write + 1, 0);
break;
}
}
diff --git a/drivers/hwmon/gpio-fan.c b/drivers/hwmon/gpio-fan.c
index 3f78375eeb44..08b58b49be5b 100644
--- a/drivers/hwmon/gpio-fan.c
+++ b/drivers/hwmon/gpio-fan.c
@@ -150,7 +150,10 @@ static int set_fan_speed(struct gpio_fan_data *fan_data, int speed_index)
__set_fan_ctrl(fan_data, fan_data->speed[speed_index].ctrl_val);
- if (fan_data->speed_index > 0 && speed_index == 0) {
+ /* The GPIOs have changed, and a PM put drops the reference on error. */
+ fan_data->speed_index = speed_index;
+
+ if (speed_index == 0) {
int ret;
ret = pm_runtime_put_sync(fan_data->dev);
@@ -158,8 +161,6 @@ static int set_fan_speed(struct gpio_fan_data *fan_data, int speed_index)
return ret;
}
- fan_data->speed_index = speed_index;
-
return 0;
}
diff --git a/drivers/hwmon/hp-wmi-sensors.c b/drivers/hwmon/hp-wmi-sensors.c
index cfa0d6ad0fb3..12950666c981 100644
--- a/drivers/hwmon/hp-wmi-sensors.c
+++ b/drivers/hwmon/hp-wmi-sensors.c
@@ -390,6 +390,7 @@ struct hp_wmi_info {
*/
struct hp_wmi_sensors {
struct wmi_device *wdev;
+ struct hwmon_chip_info chip_info;
struct hp_wmi_info **info_map[hwmon_max];
u8 channel_count[hwmon_max];
bool has_intrusion;
@@ -738,15 +739,21 @@ static long scale_numeric_sensor(const struct hp_wmi_numeric_sensor *nsensor)
val = current_reading;
- for (; unit_modifier < target_modifier; unit_modifier++)
+ for (; val && unit_modifier < target_modifier; unit_modifier++)
val = DIV_ROUND_CLOSEST(val, 10);
- for (; unit_modifier > target_modifier; unit_modifier--) {
- if (val > LONG_MAX / 10) {
- val = LONG_MAX;
- break;
+ if (val) {
+ for (; unit_modifier > target_modifier; unit_modifier--) {
+ if (val > LONG_MAX / 10) {
+ val = LONG_MAX;
+ break;
+ }
+ if (val < LONG_MIN / 10) {
+ val = LONG_MIN;
+ break;
+ }
+ val *= 10;
}
- val *= 10;
}
if (sensor_type == HP_WMI_TYPE_TEMPERATURE) {
@@ -1531,11 +1538,6 @@ static const struct hwmon_ops hp_wmi_hwmon_ops = {
.write = hp_wmi_hwmon_write,
};
-static struct hwmon_chip_info hp_wmi_chip_info = {
- .ops = &hp_wmi_hwmon_ops,
- .info = NULL,
-};
-
static struct hp_wmi_info *match_fan_event(struct hp_wmi_sensors *state,
const char *event_description)
{
@@ -1962,7 +1964,8 @@ static int make_chip_info(struct hp_wmi_sensors *state, bool has_events)
if (!ptr_channel_info)
return -ENOMEM;
- hp_wmi_chip_info.info = ptr_channel_info;
+ state->chip_info.ops = &hp_wmi_hwmon_ops;
+ state->chip_info.info = ptr_channel_info;
for (type = hwmon_chip; type < hwmon_max; type++) {
count = channel_count[type];
@@ -2053,7 +2056,7 @@ static int hp_wmi_sensors_init(struct hp_wmi_sensors *state)
return err;
hwdev = devm_hwmon_device_register_with_info(dev, "hp_wmi_sensors",
- state, &hp_wmi_chip_info,
+ state, &state->chip_info,
NULL);
return PTR_ERR_OR_ZERO(hwdev);
}
diff --git a/drivers/hwmon/lm75.c b/drivers/hwmon/lm75.c
index 2d2d752aeac9..917ac250a648 100644
--- a/drivers/hwmon/lm75.c
+++ b/drivers/hwmon/lm75.c
@@ -123,8 +123,7 @@ struct lm75_data {
static const u8 lm75_sample_set_masks[] = { 0 << 5, 1 << 5, 2 << 5, 3 << 5 };
-#define LM75_ALERT_POLARITY_HIGH_8_BIT (BIT(2))
-#define LM75_ALERT_POLARITY_HIGH_16_BIT (BIT(2) << 8)
+#define LM75_ALERT_POLARITY_HIGH_BIT BIT(2)
#define LM75_SAMPLE_CLEAR_MASK (3 << 5)
/* The structure below stores the configuration values of the supported devices.
@@ -770,11 +769,7 @@ static int lm75_generic_probe(struct device *dev, const char *name,
data->orig_conf = status;
/* Enforce polarity active-low (default) or active-high (devicetree) */
- if (!data->params->config_reg_16bits)
- pol_mask = LM75_ALERT_POLARITY_HIGH_8_BIT;
- else
- pol_mask = LM75_ALERT_POLARITY_HIGH_16_BIT;
-
+ pol_mask = LM75_ALERT_POLARITY_HIGH_BIT;
clr_mask = data->params->clr_mask | pol_mask;
set_mask = data->params->set_mask & ~pol_mask;
if (device_property_read_bool(dev, "ti,alert-polarity-active-high"))
diff --git a/drivers/hwmon/lm87.c b/drivers/hwmon/lm87.c
index c09074f44708..bb4e15b66355 100644
--- a/drivers/hwmon/lm87.c
+++ b/drivers/hwmon/lm87.c
@@ -606,13 +606,16 @@ static DEVICE_ATTR_RO(cpu0_vid);
static ssize_t vrm_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- struct lm87_data *data = dev_get_drvdata(dev);
+ struct i2c_client *client = dev_get_drvdata(dev);
+ struct lm87_data *data = i2c_get_clientdata(client);
+
return sprintf(buf, "%d\n", data->vrm);
}
static ssize_t vrm_store(struct device *dev, struct device_attribute *attr,
const char *buf, size_t count)
{
- struct lm87_data *data = dev_get_drvdata(dev);
+ struct i2c_client *client = dev_get_drvdata(dev);
+ struct lm87_data *data = i2c_get_clientdata(client);
unsigned long val;
int err;
diff --git a/drivers/hwmon/nct6683.c b/drivers/hwmon/nct6683.c
index e1c36c95affb..f34cb62305be 100644
--- a/drivers/hwmon/nct6683.c
+++ b/drivers/hwmon/nct6683.c
@@ -920,7 +920,7 @@ static ssize_t
show_pwm(struct device *dev, struct device_attribute *attr, char *buf)
{
struct nct6683_data *data = nct6683_update_device(dev);
- struct sensor_device_attribute_2 *sattr = to_sensor_dev_attr_2(attr);
+ struct sensor_device_attribute *sattr = to_sensor_dev_attr(attr);
int index = sattr->index;
return sprintf(buf, "%d\n", data->pwm[index]);
@@ -930,7 +930,7 @@ static ssize_t
store_pwm(struct device *dev, struct device_attribute *attr, const char *buf,
size_t count)
{
- struct sensor_device_attribute_2 *sattr = to_sensor_dev_attr_2(attr);
+ struct sensor_device_attribute *sattr = to_sensor_dev_attr(attr);
struct nct6683_data *data = dev_get_drvdata(dev);
int index = sattr->index;
unsigned long val;
diff --git a/drivers/hwmon/peci/dimmtemp.c b/drivers/hwmon/peci/dimmtemp.c
index bd3e8715dfec..617a4cc0f65c 100644
--- a/drivers/hwmon/peci/dimmtemp.c
+++ b/drivers/hwmon/peci/dimmtemp.c
@@ -4,7 +4,6 @@
#include <linux/auxiliary_bus.h>
#include <linux/bitfield.h>
#include <linux/bitops.h>
-#include <linux/devm-helpers.h>
#include <linux/hwmon.h>
#include <linux/jiffies.h>
#include <linux/module.h>
@@ -326,11 +325,13 @@ static int create_dimm_temp_info(struct peci_dimmtemp *priv)
if (ret == -ENODEV) {
dev_dbg(priv->dev, "No DIMMs found\n");
return 0;
- } else if (ret) {
+ }
+ if (ret == -EAGAIN) {
schedule_delayed_work(&priv->detect_work, DIMM_MASK_CHECK_DELAY_JIFFIES);
dev_dbg(priv->dev, "Deferred populating DIMM temp info\n");
- return ret;
}
+ if (ret)
+ return ret;
channels = priv->gen_info->chan_rank_max * priv->gen_info->dimm_idx_max;
@@ -398,10 +399,8 @@ static int peci_dimmtemp_probe(struct auxiliary_device *adev, const struct auxil
"Unexpected PECI revision %#x, some features may be unavailable\n",
peci_dev->info.peci_revision);
- ret = devm_delayed_work_autocancel(priv->dev, &priv->detect_work,
- create_dimm_temp_info_delayed);
- if (ret)
- return ret;
+ auxiliary_set_drvdata(adev, priv);
+ INIT_DELAYED_WORK(&priv->detect_work, create_dimm_temp_info_delayed);
ret = create_dimm_temp_info(priv);
if (ret && ret != -EAGAIN) {
@@ -412,6 +411,13 @@ static int peci_dimmtemp_probe(struct auxiliary_device *adev, const struct auxil
return 0;
}
+static void peci_dimmtemp_remove(struct auxiliary_device *adev)
+{
+ struct peci_dimmtemp *priv = auxiliary_get_drvdata(adev);
+
+ cancel_delayed_work_sync(&priv->detect_work);
+}
+
static int
read_thresholds_hsx(struct peci_dimmtemp *priv, int dimm_order, int chan_rank, u32 *data)
{
@@ -665,6 +671,7 @@ MODULE_DEVICE_TABLE(auxiliary, peci_dimmtemp_ids);
static struct auxiliary_driver peci_dimmtemp_driver = {
.probe = peci_dimmtemp_probe,
+ .remove = peci_dimmtemp_remove,
.id_table = peci_dimmtemp_ids,
};
diff --git a/drivers/hwmon/pmbus/pmbus_core.c b/drivers/hwmon/pmbus/pmbus_core.c
index 5f69c1420b4e..b033bd3b1527 100644
--- a/drivers/hwmon/pmbus/pmbus_core.c
+++ b/drivers/hwmon/pmbus/pmbus_core.c
@@ -1507,11 +1507,14 @@ static int pmbus_thermal_add_sensor(struct pmbus_data *pmbus_data,
tzd = devm_thermal_of_zone_register(dev, index, tdata,
&pmbus_thermal_ops);
/*
- * If CONFIG_THERMAL_OF is disabled, this returns -ENODEV,
- * so ignore that error but forward any other error.
+ * If CONFIG_THERMAL_OF is disabled, devm_thermal_of_zone_register()
+ * returns -ENOTSUPP. If no thermal zone is configured for the sensor,
+ * it returns -ENODEV. Ignore those errors but forward any other error.
*/
- if (IS_ERR(tzd) && (PTR_ERR(tzd) != -ENODEV))
+ if (IS_ERR(tzd) && PTR_ERR(tzd) != -ENODEV && PTR_ERR(tzd) != -ENOTSUPP) {
+ devm_kfree(dev, tdata);
return PTR_ERR(tzd);
+ }
return 0;
}
@@ -1526,10 +1529,11 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data,
{
struct sensor_device_attribute *a;
struct pmbus_sensor *sensor;
+ int err;
sensor = devm_kzalloc(data->dev, sizeof(*sensor), GFP_KERNEL);
if (!sensor)
- return NULL;
+ return ERR_PTR(-ENOMEM);
a = &sensor->attribute;
if (type)
@@ -1554,14 +1558,17 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data,
pmbus_set_sensor, -1);
if (pmbus_add_attribute(data, &a->dev_attr.attr))
- return NULL;
+ return ERR_PTR(-ENOMEM);
sensor->next = data->sensors;
data->sensors = sensor;
/* temperature sensors with _input values are registered with thermal */
- if (class == PSC_TEMPERATURE && strcmp(type, "input") == 0)
- pmbus_thermal_add_sensor(data, sensor, seq);
+ if (class == PSC_TEMPERATURE && strcmp(type, "input") == 0) {
+ err = pmbus_thermal_add_sensor(data, sensor, seq);
+ if (err)
+ return ERR_PTR(err);
+ }
return sensor;
}
@@ -1662,8 +1669,8 @@ static int pmbus_add_limit_attrs(struct i2c_client *client,
page, 0xff, l->reg, attr->class,
attr->update || l->update,
l->readonly, l->writeonly, true);
- if (!curr)
- return -ENOMEM;
+ if (IS_ERR(curr))
+ return PTR_ERR(curr);
if (l->sbit && (info->func[page] & attr->sfunc)) {
ret = pmbus_add_boolean(data, name,
l->alarm, index,
@@ -1702,8 +1709,8 @@ static int pmbus_add_sensor_attrs_one(struct i2c_client *client,
}
base = pmbus_add_sensor(data, name, "input", index, page, phase,
attr->reg, attr->class, true, true, false, true);
- if (!base)
- return -ENOMEM;
+ if (IS_ERR(base))
+ return PTR_ERR(base);
/* No limit and alarm attributes for phase specific sensors */
if (attr->sfunc && phase == 0xff) {
ret = pmbus_add_limit_attrs(client, data, info, name,
@@ -2376,8 +2383,8 @@ static int pmbus_add_fan_ctrl(struct i2c_client *client,
0xff, PMBUS_VIRT_FAN_TARGET_1 + id, PSC_FAN,
false, false, false, true);
- if (!sensor)
- return -ENOMEM;
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
if (!((data->info->func[page] & PMBUS_HAVE_PWM12) ||
(data->info->func[page] & PMBUS_HAVE_PWM34)))
@@ -2387,15 +2394,15 @@ static int pmbus_add_fan_ctrl(struct i2c_client *client,
0xff, PMBUS_VIRT_PWM_1 + id, PSC_PWM,
false, false, false, true);
- if (!sensor)
- return -ENOMEM;
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
sensor = pmbus_add_sensor(data, "pwm", "enable", index, page,
0xff, PMBUS_VIRT_PWM_ENABLE_1 + id, PSC_PWM,
true, false, false, false);
- if (!sensor)
- return -ENOMEM;
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
return 0;
}
@@ -2412,6 +2419,7 @@ static int pmbus_add_fan_attributes(struct i2c_client *client,
int f;
for (f = 0; f < ARRAY_SIZE(pmbus_fan_registers); f++) {
+ struct pmbus_sensor *sensor;
int regval;
if (!(info->func[page] & pmbus_fan_flags[f]))
@@ -2432,10 +2440,11 @@ static int pmbus_add_fan_attributes(struct i2c_client *client,
(!(regval & (PB_FAN_1_INSTALLED >> ((f & 1) * 4)))))
continue;
- if (pmbus_add_sensor(data, "fan", "input", index,
- page, 0xff, pmbus_fan_registers[f],
- PSC_FAN, true, true, false, true) == NULL)
- return -ENOMEM;
+ sensor = pmbus_add_sensor(data, "fan", "input", index,
+ page, 0xff, pmbus_fan_registers[f],
+ PSC_FAN, true, true, false, true);
+ if (IS_ERR(sensor))
+ return PTR_ERR(sensor);
/* Fan control */
if (pmbus_check_word_register(client, page,
diff --git a/drivers/hwmon/pmbus/q54sj108a2.c b/drivers/hwmon/pmbus/q54sj108a2.c
index ecac405d7536..b018a611d9b6 100644
--- a/drivers/hwmon/pmbus/q54sj108a2.c
+++ b/drivers/hwmon/pmbus/q54sj108a2.c
@@ -351,6 +351,8 @@ static int q54sj108a2_probe(struct i2c_client *client)
psu->client = client;
debugfs = pmbus_get_debugfs_dir(client);
+ if (!debugfs)
+ return 0;
q54sj108a2_dir = debugfs_create_dir(client->name, debugfs);
diff --git a/drivers/hwmon/sht15.c b/drivers/hwmon/sht15.c
index 7bb2105d1ae2..48d93c61345c 100644
--- a/drivers/hwmon/sht15.c
+++ b/drivers/hwmon/sht15.c
@@ -31,6 +31,7 @@
#include <linux/bitrev.h>
#include <linux/gpio/consumer.h>
#include <linux/of.h>
+#include <linux/workqueue.h>
/* Commands */
#define SHT15_MEASURE_TEMP 0x03
@@ -976,25 +977,22 @@ static int sht15_probe(struct platform_device *pdev)
goto err_release_reg;
}
- ret = devm_request_irq(&pdev->dev, gpiod_to_irq(data->data),
- sht15_interrupt_fired,
- IRQF_TRIGGER_FALLING,
- "sht15 data",
- data);
+ ret = request_irq(gpiod_to_irq(data->data), sht15_interrupt_fired,
+ IRQF_TRIGGER_FALLING, "sht15 data", data);
if (ret)
goto err_release_reg;
disable_irq_nosync(gpiod_to_irq(data->data));
ret = sht15_connection_reset(data);
if (ret)
- goto err_release_reg;
+ goto err_free_irq;
ret = sht15_soft_reset(data);
if (ret)
- goto err_release_reg;
+ goto err_free_irq;
ret = sysfs_create_group(&pdev->dev.kobj, &sht15_attr_group);
if (ret) {
dev_err(&pdev->dev, "sysfs create failed\n");
- goto err_release_reg;
+ goto err_free_irq;
}
data->hwmon_dev = hwmon_device_register(data->dev);
@@ -1007,9 +1005,13 @@ static int sht15_probe(struct platform_device *pdev)
err_release_sysfs_group:
sysfs_remove_group(&pdev->dev.kobj, &sht15_attr_group);
+err_free_irq:
+ disable_work_sync(&data->read_work);
+ free_irq(gpiod_to_irq(data->data), data);
err_release_reg:
if (!IS_ERR(data->reg)) {
regulator_unregister_notifier(data->reg, &data->nb);
+ cancel_work_sync(&data->update_supply_work);
regulator_disable(data->reg);
}
return ret;
@@ -1023,12 +1025,16 @@ static void sht15_remove(struct platform_device *pdev)
hwmon_device_unregister(data->hwmon_dev);
sysfs_remove_group(&pdev->dev.kobj, &sht15_attr_group);
+ disable_work_sync(&data->read_work);
+ free_irq(gpiod_to_irq(data->data), data);
+
ret = sht15_soft_reset(data);
if (ret)
dev_err(&pdev->dev, "Failed to reset device (%pe)\n", ERR_PTR(ret));
if (!IS_ERR(data->reg)) {
regulator_unregister_notifier(data->reg, &data->nb);
+ cancel_work_sync(&data->update_supply_work);
regulator_disable(data->reg);
}
}
diff --git a/drivers/hwmon/sht4x.c b/drivers/hwmon/sht4x.c
index 0bb9227674a8..f38c106b7090 100644
--- a/drivers/hwmon/sht4x.c
+++ b/drivers/hwmon/sht4x.c
@@ -99,7 +99,9 @@ static int sht4x_read_values(struct sht4x_data *data)
msleep(jiffies_to_msecs(data->heating_complete - curr_jiffies));
if (data->data_pending &&
- time_before64(get_jiffies_64(), data->heating_complete + data->update_interval)) {
+ time_before64(get_jiffies_64(),
+ data->heating_complete +
+ msecs_to_jiffies(data->update_interval))) {
data->data_pending = false;
} else {
next_update = data->last_updated +
diff --git a/drivers/hwmon/tmp108.c b/drivers/hwmon/tmp108.c
index 91d20a1ce112..862b944d3e60 100644
--- a/drivers/hwmon/tmp108.c
+++ b/drivers/hwmon/tmp108.c
@@ -13,7 +13,7 @@
#include <linux/i2c.h>
#include <linux/i3c/device.h>
#include <linux/init.h>
-#include <linux/jiffies.h>
+#include <linux/ktime.h>
#include <linux/regmap.h>
#include <linux/regulator/consumer.h>
#include <linux/slab.h>
@@ -78,7 +78,7 @@
struct tmp108 {
struct regmap *regmap;
u16 orig_config;
- u64 ready_time;
+ ktime_t ready_time;
const struct tmp108_params *params;
};
@@ -121,6 +121,7 @@ static int tmp108_read(struct device *dev, enum hwmon_sensor_types type,
struct tmp108 *tmp108 = dev_get_drvdata(dev);
unsigned int regval;
int err, hyst;
+ s64 delay;
if (type == hwmon_chip) {
if (attr == hwmon_chip_update_interval) {
@@ -137,12 +138,10 @@ static int tmp108_read(struct device *dev, enum hwmon_sensor_types type,
switch (attr) {
case hwmon_temp_input:
- /* Is it too early to return a conversion ? */
- if (time_before64(get_jiffies_64(), tmp108->ready_time)) {
- dev_dbg(dev, "%s: Conversion not ready yet..\n",
- __func__);
- return -EAGAIN;
- }
+ /* Wait if it is too early to return a conversion */
+ delay = ktime_us_delta(tmp108->ready_time, ktime_get());
+ if (delay > 0)
+ usleep_range(delay, delay + 1000);
err = regmap_read(tmp108->regmap, TMP108_REG_TEMP, ®val);
if (err < 0)
return err;
@@ -477,11 +476,8 @@ static int tmp108_common_probe(struct device *dev, struct regmap *regmap, char *
return err;
}
- tmp108->ready_time = get_jiffies_64();
- if ((tmp108->orig_config & TMP108_CONF_MODE_MASK) ==
- TMP108_MODE_SHUTDOWN)
- tmp108->ready_time +=
- msecs_to_jiffies(TMP108_CONVERSION_TIME_MS);
+ tmp108->ready_time = ktime_add_ms(ktime_get(),
+ TMP108_CONVERSION_TIME_MS);
err = devm_add_action_or_reset(dev, tmp108_restore_config, tmp108);
if (err) {
@@ -528,8 +524,8 @@ static int tmp108_resume(struct device *dev)
err = regmap_update_bits(tmp108->regmap, TMP108_REG_CONF,
TMP108_CONF_MODE_MASK, TMP108_MODE_CONTINUOUS);
- tmp108->ready_time = get_jiffies_64() +
- msecs_to_jiffies(TMP108_CONVERSION_TIME_MS);
+ tmp108->ready_time = ktime_add_ms(ktime_get(),
+ TMP108_CONVERSION_TIME_MS);
return err;
}
diff --git a/drivers/idle/intel_idle.c b/drivers/idle/intel_idle.c
index 651408df9c24..ec951c148d4a 100644
--- a/drivers/idle/intel_idle.c
+++ b/drivers/idle/intel_idle.c
@@ -1905,8 +1905,13 @@ static void __init intel_idle_init_cstates_acpi_lpi(struct cpuidle_driver *drv)
state = &drv->states[drv->state_count++];
- scnprintf(state->name, CPUIDLE_NAME_LEN, "C%d_LPI", index + 1);
- strscpy(state->desc, lpi_state->desc, CPUIDLE_DESC_LEN);
+ if (strlen(lpi_state->desc) > 0)
+ strscpy(state->name, lpi_state->desc);
+ else
+ scnprintf(state->name, CPUIDLE_NAME_LEN, "C%d_LPI", index + 1);
+
+ scnprintf(state->desc, CPUIDLE_DESC_LEN, "ACPI LPI MWAIT 0x%02llx",
+ lpi_state->address);
state->exit_latency = lpi_state->wake_latency;
state->target_residency = lpi_state->min_residency;
state->flags = MWAIT2flg(lpi_state->address);
@@ -1956,7 +1961,7 @@ static void __init intel_idle_init_cstates_acpi_cst(struct cpuidle_driver *drv)
state = &drv->states[drv->state_count++];
snprintf(state->name, CPUIDLE_NAME_LEN, "C%d_ACPI", cstate);
- strscpy(state->desc, cx->desc, CPUIDLE_DESC_LEN);
+ strscpy(state->desc, cx->desc);
state->exit_latency = cx->latency;
/*
* For C1-type C-states use the same number for both the exit
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index 0506617b27c2..cfc061a4247a 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -1720,6 +1720,21 @@ static int __init init_iommu_from_acpi(struct amd_iommu *iommu,
return 0;
}
+/*
+ * Apply workaround for Zen1 to Zen5(BRH) generation of processors.
+ * References: CVE-2024-21963
+ */
+static bool __init amd_iommu_force_max_dev_table(void)
+{
+ if (boot_cpu_data.x86 >= 0x17 && boot_cpu_data.x86 <= 0x1a) {
+ if (cpu_feature_enabled(X86_FEATURE_ZEN6))
+ return false;
+ else
+ return true;
+ }
+ return false;
+}
+
/* Allocate PCI segment data structure */
static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id,
struct acpi_table_header *ivrs_base)
@@ -1742,9 +1757,15 @@ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id,
pci_seg->last_bdf = last_bdf;
DUMP_printk("PCI segment : 0x%0x, last bdf : 0x%04x\n", id, last_bdf);
- pci_seg->dev_table_size =
- max(roundup_pow_of_two((last_bdf + 1) * DEV_TABLE_ENTRY_SIZE),
- SZ_4K);
+ if (amd_iommu_force_max_dev_table()) {
+ pr_info("Force device table to maximum size\n");
+ pci_seg->dev_table_size = SZ_2M;
+ } else {
+ pci_seg->dev_table_size =
+ max(roundup_pow_of_two((last_bdf + 1) *
+ DEV_TABLE_ENTRY_SIZE),
+ SZ_4K);
+ }
pci_seg->id = id;
init_llist_head(&pci_seg->dev_data_list);
@@ -2708,6 +2729,15 @@ static int __init init_unity_map_range(struct ivmd_header *m,
return 0;
}
+static bool ivrs_block_type_ivmd(u8 type)
+{
+ if (type == ACPI_IVMD_TYPE ||
+ type == ACPI_IVMD_TYPE_ALL ||
+ type == ACPI_IVMD_TYPE_RANGE)
+ return true;
+ return false;
+}
+
/* iterates over all memory definitions we find in the ACPI table */
static int __init init_memory_definitions(struct acpi_table_header *table)
{
@@ -2719,10 +2749,11 @@ static int __init init_memory_definitions(struct acpi_table_header *table)
while (p < end) {
m = (struct ivmd_header *)p;
+ p += m->length;
+ if (!ivrs_block_type_ivmd(m->type))
+ continue;
if (m->flags & (IVMD_FLAG_UNITY_MAP | IVMD_FLAG_EXCL_RANGE))
init_unity_map_range(m, table);
-
- p += m->length;
}
return 0;
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 893cd2be11a2..b19496f8c1b0 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -489,6 +489,10 @@ static void setup_aliases(struct amd_iommu *iommu, struct device *dev)
clone_aliases(iommu, dev);
}
+/*
+ * We keep dev_data around for unplugged devices and reuse it when the
+ * device is re-plugged - not doing so would introduce a ton of races.
+ */
static struct iommu_dev_data *find_dev_data(struct amd_iommu *iommu, u16 devid)
{
struct iommu_dev_data *dev_data;
@@ -2548,18 +2552,6 @@ static struct iommu_device *amd_iommu_probe_device(struct device *dev)
return iommu_dev;
}
-static void amd_iommu_release_device(struct device *dev)
-{
- struct iommu_dev_data *dev_data = dev_iommu_priv_get(dev);
-
- WARN_ON(dev_data->domain);
-
- /*
- * We keep dev_data around for unplugged devices and reuse it when the
- * device is re-plugged - not doing so would introduce a ton of races.
- */
-}
-
static struct iommu_group *amd_iommu_device_group(struct device *dev)
{
if (dev_is_pci(dev))
@@ -3205,7 +3197,6 @@ const struct iommu_ops amd_iommu_ops = {
.domain_alloc_paging_flags = amd_iommu_domain_alloc_paging_flags,
.domain_alloc_sva = amd_iommu_domain_alloc_sva,
.probe_device = amd_iommu_probe_device,
- .release_device = amd_iommu_release_device,
.device_group = amd_iommu_device_group,
.get_resv_regions = amd_iommu_get_resv_regions,
.is_attach_deferred = amd_iommu_is_attach_deferred,
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index 0a429c64fbf3..3f50298a1c14 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -215,6 +215,13 @@ bool arm_smmu_sva_supported(struct arm_smmu_device *smmu)
if (system_supports_haft())
feat_mask |= ARM_SMMU_FEAT_HAFT;
+ /*
+ * The workaround for ARM_SMMU_OPT_FULL_CONT_RANGE_INV requires range
+ * invalidation support.
+ */
+ if (smmu->options & ARM_SMMU_OPT_FULL_CONT_RANGE_INV)
+ feat_mask |= ARM_SMMU_FEAT_RANGE_INV;
+
if ((smmu->features & feat_mask) != feat_mask)
return false;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 81baafaffc41..b510dd47f231 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2538,6 +2538,37 @@ static void arm_smmu_cmdq_batch_add_range(struct arm_smmu_device *smmu,
}
}
+/*
+ * Generate a range invalidation for ARM_SMMU_OPT_FULL_CONT_RANGE_INV by
+ * ensuring the entire SVA requested range is covered with a single range
+ * invalidation command. The scale is adjusted so that the range invalidation
+ * may extend past the end of the requested range. This ensures that any CONT
+ * the MM is invalidating is covered by a single range invalidation. TTL and
+ * LEAF are always 0 because this is only used by SVA.
+ */
+static bool arm_smmu_cmdq_batch_add_range_inv(struct arm_smmu_device *smmu,
+ struct arm_smmu_cmdq_batch *cmds,
+ struct arm_smmu_cmd *cmd,
+ unsigned long iova, size_t size,
+ u8 tgsz_lg2)
+{
+ u64 cur_tg = iova >> tgsz_lg2;
+ u64 num_tg = ((iova + size - 1) >> tgsz_lg2) - cur_tg + 1;
+ unsigned int scale = fls64((num_tg - 1) / 32);
+
+ if (scale > 31)
+ return false;
+
+ cmd->data[0] |=
+ FIELD_PREP(CMDQ_TLBI_0_NUM,
+ DIV_ROUND_UP_ULL(num_tg, 1ULL << scale) - 1) |
+ FIELD_PREP(CMDQ_TLBI_0_SCALE, scale);
+ cmd->data[1] = FIELD_PREP(CMDQ_TLBI_1_TG, (tgsz_lg2 - 10) / 2) |
+ (cur_tg << tgsz_lg2);
+ arm_smmu_cmdq_batch_add_cmd_p(smmu, cmds, cmd);
+ return true;
+}
+
static bool arm_smmu_inv_size_too_big(struct arm_smmu_device *smmu, size_t size,
size_t granule)
{
@@ -2565,21 +2596,30 @@ static bool arm_smmu_inv_size_too_big(struct arm_smmu_device *smmu, size_t size,
static void arm_smmu_inv_to_cmdq_batch(struct arm_smmu_inv *inv,
struct arm_smmu_cmdq_batch *cmds,
struct arm_smmu_cmd *cmd,
- bool leaf,
+ bool single_range_inv, bool leaf,
unsigned long iova, size_t size,
unsigned int granule)
{
- if (arm_smmu_inv_size_too_big(inv->smmu, size, granule)) {
- struct arm_smmu_cmd nsize_cmd = *cmd;
+ struct arm_smmu_cmd nsize_cmd;
+
+ if (arm_smmu_inv_size_too_big(inv->smmu, size, granule))
+ goto full_inv;
- u64p_replace_bits(&nsize_cmd.data[0], inv->nsize_opcode,
- CMDQ_0_OP);
- arm_smmu_cmdq_batch_add_cmd_p(inv->smmu, cmds, &nsize_cmd);
+ if (single_range_inv && size > granule) {
+ if (!arm_smmu_cmdq_batch_add_range_inv(inv->smmu, cmds, cmd,
+ iova, size, inv->pgsize))
+ goto full_inv;
return;
}
- arm_smmu_cmdq_batch_add_range(inv->smmu, cmds, cmd, leaf,
- iova, size, granule, inv->pgsize);
+ arm_smmu_cmdq_batch_add_range(inv->smmu, cmds, cmd, leaf, iova, size,
+ granule, inv->pgsize);
+ return;
+
+full_inv:
+ nsize_cmd = *cmd;
+ u64p_replace_bits(&nsize_cmd.data[0], inv->nsize_opcode, CMDQ_0_OP);
+ arm_smmu_cmdq_batch_add_cmd_p(inv->smmu, cmds, &nsize_cmd);
}
static inline bool arm_smmu_invs_end_batch(struct arm_smmu_inv *cur,
@@ -2600,7 +2640,8 @@ static inline bool arm_smmu_invs_end_batch(struct arm_smmu_inv *cur,
static void __arm_smmu_domain_inv_range(struct arm_smmu_invs *invs,
unsigned long iova, size_t size,
- unsigned int granule, bool leaf)
+ unsigned int granule,
+ bool single_range_inv, bool leaf)
{
struct arm_smmu_cmdq_batch cmds = {};
struct arm_smmu_inv *cur;
@@ -2630,14 +2671,16 @@ static void __arm_smmu_domain_inv_range(struct arm_smmu_invs *invs,
case INV_TYPE_S1_ASID:
cmd = arm_smmu_make_cmd_tlbi(cur->size_opcode,
cur->id, 0);
- arm_smmu_inv_to_cmdq_batch(cur, &cmds, &cmd, leaf,
- iova, size, granule);
+ arm_smmu_inv_to_cmdq_batch(cur, &cmds, &cmd,
+ single_range_inv, leaf, iova,
+ size, granule);
break;
case INV_TYPE_S2_VMID:
cmd = arm_smmu_make_cmd_tlbi(cur->size_opcode,
0, cur->id);
- arm_smmu_inv_to_cmdq_batch(cur, &cmds, &cmd, leaf,
- iova, size, granule);
+ arm_smmu_inv_to_cmdq_batch(cur, &cmds, &cmd,
+ single_range_inv, leaf, iova,
+ size, granule);
break;
case INV_TYPE_S2_VMID_S1_CLEAR:
/* CMDQ_OP_TLBI_S12_VMALL already flushed S1 entries */
@@ -2684,6 +2727,9 @@ void arm_smmu_domain_inv_range(struct arm_smmu_domain *smmu_domain,
unsigned int granule, bool leaf)
{
struct arm_smmu_invs *invs;
+ bool single_range_inv =
+ smmu_domain->stage == ARM_SMMU_DOMAIN_SVA &&
+ (smmu_domain->smmu->options & ARM_SMMU_OPT_FULL_CONT_RANGE_INV);
/*
* An invalidation request must follow some IOPTE change and then load
@@ -2723,10 +2769,12 @@ void arm_smmu_domain_inv_range(struct arm_smmu_domain *smmu_domain,
unsigned long flags;
read_lock_irqsave(&invs->rwlock, flags);
- __arm_smmu_domain_inv_range(invs, iova, size, granule, leaf);
+ __arm_smmu_domain_inv_range(invs, iova, size, granule,
+ single_range_inv, leaf);
read_unlock_irqrestore(&invs->rwlock, flags);
} else {
- __arm_smmu_domain_inv_range(invs, iova, size, granule, leaf);
+ __arm_smmu_domain_inv_range(invs, iova, size, granule,
+ single_range_inv, leaf);
}
rcu_read_unlock();
@@ -5009,11 +5057,20 @@ static void arm_smmu_device_iidr_probe(struct arm_smmu_device *smmu)
/* Arm errata 2268618, 2812531 */
smmu->features &= ~ARM_SMMU_FEAT_NESTING;
}
+ /* Arm errata 3777127 */
+ smmu->options |= ARM_SMMU_OPT_FULL_CONT_RANGE_INV;
break;
case IIDR_PRODUCTID_ARM_MMU_L1:
+ /* Arm errata 3878312 */
+ smmu->features &= ~ARM_SMMU_FEAT_BTM;
+ break;
case IIDR_PRODUCTID_ARM_MMU_S3:
- /* Arm errata 3878312/3995052 */
+ /* Arm errata 3995052 */
smmu->features &= ~ARM_SMMU_FEAT_BTM;
+ /* Arm errata 3673557 */
+ if (variant < 1 || (variant == 1 && revision < 1))
+ smmu->options |=
+ ARM_SMMU_OPT_FULL_CONT_RANGE_INV;
break;
}
break;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index dd2fee2f560e..094ce28c862f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -934,6 +934,11 @@ struct arm_smmu_device {
#define ARM_SMMU_OPT_MSIPOLL (1 << 2)
#define ARM_SMMU_OPT_CMDQ_FORCE_SYNC (1 << 3)
#define ARM_SMMU_OPT_TEGRA241_CMDQV (1 << 4)
+/*
+ * Range invalidation is mandatory and one range invalidation must fully span an
+ * invalidated CONT
+ */
+#define ARM_SMMU_OPT_FULL_CONT_RANGE_INV (1 << 6)
u32 options;
struct arm_smmu_cmdq cmdq;
diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c
index 283ab372e8da..5ac736cf5bb5 100644
--- a/drivers/iommu/iommufd/io_pagetable.c
+++ b/drivers/iommu/iommufd/io_pagetable.c
@@ -589,7 +589,7 @@ iommu_read_and_clear_dirty(struct iommu_domain *domain,
arg.iopt = iopt;
arg.domain = domain;
arg.dirty = &dirty;
- iova_bitmap_for_each(iter, &arg, __iommu_read_and_clear_dirty);
+ ret = iova_bitmap_for_each(iter, &arg, __iommu_read_and_clear_dirty);
if (!(flags & IOMMU_DIRTY_NO_CLEAR))
iommu_iotlb_sync(domain, &gather);
diff --git a/drivers/iommu/iommufd/iova_bitmap.c b/drivers/iommu/iommufd/iova_bitmap.c
index dac3e657d498..908c70f5362e 100644
--- a/drivers/iommu/iommufd/iova_bitmap.c
+++ b/drivers/iommu/iommufd/iova_bitmap.c
@@ -116,6 +116,9 @@ struct iova_bitmap {
/* length of the IOVA range for the whole bitmap */
size_t length;
+
+ /* first error from recording an IOVA range */
+ int err;
};
/*
@@ -384,7 +387,9 @@ static int iova_bitmap_advance_to(struct iova_bitmap *bitmap,
int iova_bitmap_for_each(struct iova_bitmap *bitmap, void *opaque,
iova_bitmap_fn_t fn)
{
- return fn(bitmap, bitmap->iova, bitmap->length, opaque);
+ int ret = fn(bitmap, bitmap->iova, bitmap->length, opaque);
+
+ return ret ? ret : bitmap->err;
}
EXPORT_SYMBOL_NS_GPL(iova_bitmap_for_each, "IOMMUFD");
@@ -404,6 +409,18 @@ void iova_bitmap_set(struct iova_bitmap *bitmap,
struct iova_bitmap_map *mapped = &bitmap->mapped;
unsigned long cur_bit, last_bit, last_page_idx;
+ /*
+ * If the range cannot be recorded then the system call returns failure.
+ * This is to prevent loosing dirties if userspace is expecting to step
+ * the bitmap through the IOVA space.
+ */
+ if (!length || length > bitmap->length || iova < bitmap->iova ||
+ iova - bitmap->iova > bitmap->length - length) {
+ if (!bitmap->err)
+ bitmap->err = -EPROTO;
+ return;
+ }
+
update_indexes:
if (unlikely(!iova_bitmap_mapped_range(mapped, iova, length))) {
/*
diff --git a/drivers/iommu/virtio-iommu.c b/drivers/iommu/virtio-iommu.c
index 587fc13197f1..fa72ae23b8af 100644
--- a/drivers/iommu/virtio-iommu.c
+++ b/drivers/iommu/virtio-iommu.c
@@ -1227,12 +1227,12 @@ static int viommu_probe(struct virtio_device *vdev)
/* Populate the event queue with buffers */
ret = viommu_fill_evtq(viommu);
if (ret)
- goto err_free_vqs;
+ goto err_reset_vdev;
ret = iommu_device_sysfs_add(&viommu->iommu, dev, NULL, "%s",
virtio_bus_name(vdev));
if (ret)
- goto err_free_vqs;
+ goto err_reset_vdev;
vdev->priv = viommu;
@@ -1244,6 +1244,8 @@ static int viommu_probe(struct virtio_device *vdev)
return 0;
+err_reset_vdev:
+ virtio_reset_device(vdev);
err_free_vqs:
vdev->config->del_vqs(vdev);
diff --git a/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c b/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c
index 62ed92ff1d30..9f5bd9c4ddf9 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-fw-isys.c
@@ -32,7 +32,7 @@ static int handle_proxy_response(struct ipu6_isys *isys, unsigned int req_id)
struct ipu6_fw_isys_proxy_resp_info_abi *resp;
int ret;
- resp = ipu6_recv_get_token(isys->fwcom, IPU6_BASE_PROXY_RECV_QUEUES);
+ resp = ipu6_recv_get_token(isys->fwctx, IPU6_BASE_PROXY_RECV_QUEUES);
if (!resp)
return 1;
@@ -42,7 +42,7 @@ static int handle_proxy_response(struct ipu6_isys *isys, unsigned int req_id)
ret = req_id == resp->request_id ? 0 : -EIO;
- ipu6_recv_put_token(isys->fwcom, IPU6_BASE_PROXY_RECV_QUEUES);
+ ipu6_recv_put_token(isys->fwctx, IPU6_BASE_PROXY_RECV_QUEUES);
return ret;
}
@@ -52,7 +52,7 @@ int ipu6_fw_isys_send_proxy_token(struct ipu6_isys *isys,
unsigned int index,
unsigned int offset, u32 value)
{
- struct ipu6_fw_com_context *ctx = isys->fwcom;
+ struct ipu6_fw_com_context *ctx = isys->fwctx;
struct device *dev = &isys->adev->auxdev.dev;
struct ipu6_fw_proxy_send_queue_token *token;
unsigned int timeout = 1000;
@@ -96,7 +96,7 @@ int ipu6_fw_isys_complex_cmd(struct ipu6_isys *isys,
dma_addr_t dma_mapped_buf,
size_t size, u16 send_type)
{
- struct ipu6_fw_com_context *ctx = isys->fwcom;
+ struct ipu6_fw_com_context *ctx = isys->fwctx;
struct device *dev = &isys->adev->auxdev.dev;
struct ipu6_fw_send_queue_token *token;
@@ -138,7 +138,7 @@ int ipu6_fw_isys_close(struct ipu6_isys *isys)
struct device *dev = &isys->adev->auxdev.dev;
int retry = IPU6_ISYS_CLOSE_RETRY;
unsigned long flags;
- void *fwcom;
+ void *fwctx;
int ret;
/*
@@ -148,9 +148,9 @@ int ipu6_fw_isys_close(struct ipu6_isys *isys)
* spinlock to wait the interrupt handler to be finished
*/
spin_lock_irqsave(&isys->power_lock, flags);
- ret = ipu6_fw_com_close(isys->fwcom);
- fwcom = isys->fwcom;
- isys->fwcom = NULL;
+ ret = ipu6_fw_com_close(isys->fwctx);
+ fwctx = isys->fwctx;
+ isys->fwctx = NULL;
spin_unlock_irqrestore(&isys->power_lock, flags);
if (ret)
dev_err(dev, "Device close failure: %d\n", ret);
@@ -158,14 +158,14 @@ int ipu6_fw_isys_close(struct ipu6_isys *isys)
/* release probably fails if the close failed. Let's try still */
do {
usleep_range(400, 500);
- ret = ipu6_fw_com_release(fwcom, 0);
+ ret = ipu6_fw_com_release(fwctx, 0);
retry--;
} while (ret && retry);
if (ret) {
dev_err(dev, "Device release time out %d\n", ret);
spin_lock_irqsave(&isys->power_lock, flags);
- isys->fwcom = fwcom;
+ isys->fwctx = fwctx;
spin_unlock_irqrestore(&isys->power_lock, flags);
}
@@ -176,11 +176,11 @@ void ipu6_fw_isys_cleanup(struct ipu6_isys *isys)
{
int ret;
- ret = ipu6_fw_com_release(isys->fwcom, 1);
+ ret = ipu6_fw_com_release(isys->fwctx, 1);
if (ret < 0)
dev_warn(&isys->adev->auxdev.dev,
"Device busy, fw_com release failed.");
- isys->fwcom = NULL;
+ isys->fwctx = NULL;
}
static void start_sp(struct ipu6_bus_device *adev)
@@ -212,13 +212,12 @@ static int query_sp(struct ipu6_bus_device *adev)
}
static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
- struct ipu6_fw_com_cfg *fwcom,
+ struct ipu6_fw_com_cfg *fwcom_cfg,
unsigned int num_streams)
{
unsigned int max_send_queues, max_sram_blocks, max_devq_size;
struct ipu6_fw_syscom_queue_config *input_queue_cfg;
struct ipu6_fw_syscom_queue_config *output_queue_cfg;
- struct device *dev = &isys->adev->auxdev.dev;
int type_proxy = IPU6_FW_ISYS_QUEUE_TYPE_PROXY;
int type_dev = IPU6_FW_ISYS_QUEUE_TYPE_DEV;
int type_msg = IPU6_FW_ISYS_QUEUE_TYPE_MSG;
@@ -228,7 +227,6 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
struct ipu6_fw_isys_fw_config *isys_fw_cfg;
u32 num_in_message_queues;
unsigned int max_streams;
- unsigned int size;
unsigned int i;
max_streams = isys->pdata->ipdata->max_streams;
@@ -236,7 +234,7 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
max_sram_blocks = isys->pdata->ipdata->max_sram_blocks;
max_devq_size = isys->pdata->ipdata->max_devq_size;
num_in_message_queues = clamp(num_streams, 1U, max_streams);
- isys_fw_cfg = devm_kzalloc(dev, sizeof(*isys_fw_cfg), GFP_KERNEL);
+ isys_fw_cfg = kzalloc_obj(*isys_fw_cfg);
if (!isys_fw_cfg)
return -ENOMEM;
@@ -248,24 +246,25 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
isys_fw_cfg->num_recv_queues[type_dev] = 0;
isys_fw_cfg->num_recv_queues[type_msg] = 1;
- size = sizeof(*input_queue_cfg) * max_send_queues;
- input_queue_cfg = devm_kzalloc(dev, size, GFP_KERNEL);
+ input_queue_cfg = kzalloc_objs(*input_queue_cfg, max_send_queues);
if (!input_queue_cfg)
- return -ENOMEM;
+ goto err_free_fw_cfg;
- size = sizeof(*output_queue_cfg) * IPU6_N_MAX_RECV_QUEUES;
- output_queue_cfg = devm_kzalloc(dev, size, GFP_KERNEL);
+ output_queue_cfg = kzalloc_objs(*output_queue_cfg,
+ IPU6_N_MAX_RECV_QUEUES);
if (!output_queue_cfg)
- return -ENOMEM;
+ goto err_free_input_queue_cfg;
- fwcom->input = input_queue_cfg;
- fwcom->output = output_queue_cfg;
+ fwcom_cfg->input = input_queue_cfg;
+ fwcom_cfg->output = output_queue_cfg;
- fwcom->num_input_queues = isys_fw_cfg->num_send_queues[type_proxy] +
+ fwcom_cfg->num_input_queues =
+ isys_fw_cfg->num_send_queues[type_proxy] +
isys_fw_cfg->num_send_queues[type_dev] +
isys_fw_cfg->num_send_queues[type_msg];
- fwcom->num_output_queues = isys_fw_cfg->num_recv_queues[type_proxy] +
+ fwcom_cfg->num_output_queues =
+ isys_fw_cfg->num_recv_queues[type_proxy] +
isys_fw_cfg->num_recv_queues[type_dev] +
isys_fw_cfg->num_recv_queues[type_msg];
@@ -280,7 +279,7 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
isys_fw_cfg->buffer_partition.num_gda_pages[i] = 0;
}
- /* FW assumes proxy interface at fwcom queue 0 */
+ /* FW assumes proxy interface at fwcom_cfg queue 0 */
for (i = 0; i < isys_fw_cfg->num_send_queues[type_proxy]; i++) {
input_queue_cfg[i].token_size =
sizeof(struct ipu6_fw_proxy_send_queue_token);
@@ -314,34 +313,51 @@ static int ipu6_isys_fwcom_cfg_init(struct ipu6_isys *isys,
IPU6_ISYS_SIZE_RECV_QUEUE;
}
- fwcom->dmem_addr = isys->pdata->ipdata->hw_variant.dmem_offset;
- fwcom->specific_addr = isys_fw_cfg;
- fwcom->specific_size = sizeof(*isys_fw_cfg);
+ fwcom_cfg->dmem_addr = isys->pdata->ipdata->hw_variant.dmem_offset;
+ fwcom_cfg->specific_addr = isys_fw_cfg;
+ fwcom_cfg->specific_size = sizeof(*isys_fw_cfg);
return 0;
+
+err_free_input_queue_cfg:
+ kfree(input_queue_cfg);
+err_free_fw_cfg:
+ kfree(isys_fw_cfg);
+
+ return -ENOMEM;
+}
+
+static void ipu6_isys_fwcom_cfg_free(struct ipu6_fw_com_cfg *fwcom_cfg)
+{
+ kfree(fwcom_cfg->specific_addr);
+ kfree(fwcom_cfg->output);
+ kfree(fwcom_cfg->input);
}
int ipu6_fw_isys_init(struct ipu6_isys *isys, unsigned int num_streams)
{
struct device *dev = &isys->adev->auxdev.dev;
int retry = IPU6_ISYS_OPEN_RETRY;
- struct ipu6_fw_com_cfg fwcom = {
+ struct ipu6_fw_com_cfg fwcom_cfg = {
.cell_start = start_sp,
.cell_ready = query_sp,
.buttress_boot_offset = SYSCOM_BUTTRESS_FW_PARAMS_ISYS_OFFSET,
};
int ret;
- ipu6_isys_fwcom_cfg_init(isys, &fwcom, num_streams);
+ ret = ipu6_isys_fwcom_cfg_init(isys, &fwcom_cfg, num_streams);
+ if (ret)
+ return ret;
- isys->fwcom = ipu6_fw_com_prepare(&fwcom, isys->adev,
+ isys->fwctx = ipu6_fw_com_prepare(&fwcom_cfg, isys->adev,
isys->pdata->base);
- if (!isys->fwcom) {
+ ipu6_isys_fwcom_cfg_free(&fwcom_cfg);
+ if (!isys->fwctx) {
dev_err(dev, "isys fw com prepare failed\n");
return -EIO;
}
- ret = ipu6_fw_com_open(isys->fwcom);
+ ret = ipu6_fw_com_open(isys->fwctx);
if (ret) {
dev_err(dev, "isys fw com open failed %d\n", ret);
return ret;
@@ -349,7 +365,7 @@ int ipu6_fw_isys_init(struct ipu6_isys *isys, unsigned int num_streams)
do {
usleep_range(400, 500);
- if (ipu6_fw_com_ready(isys->fwcom))
+ if (ipu6_fw_com_ready(isys->fwctx))
break;
retry--;
} while (retry > 0);
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
index 3ac48d2076da..89eb265737d7 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
@@ -1114,7 +1114,7 @@ int ipu6_isys_fw_open(struct ipu6_isys *isys)
*/
ipu6_cleanup_fw_msg_bufs(isys);
- if (isys->fwcom) {
+ if (isys->fwctx) {
/*
* Something went wrong in previous shutdown. As we are now
* restarting isys we can safely delete old context.
@@ -1147,7 +1147,7 @@ void ipu6_isys_fw_close(struct ipu6_isys *isys)
isys->ref_count--;
if (!isys->ref_count) {
ipu6_fw_isys_close(isys);
- if (isys->fwcom) {
+ if (isys->fwctx) {
isys->need_reset = true;
dev_warn(&isys->adev->auxdev.dev,
"failed to close fw isys\n");
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys.c b/drivers/media/pci/intel/ipu6/ipu6-isys.c
index 24db2763de54..e13fbbae8c98 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys.c
@@ -1212,10 +1212,10 @@ static int isys_isr_one(struct ipu6_bus_device *adev)
u32 index;
u64 ts;
- if (!isys->fwcom)
+ if (!isys->fwctx)
return 1;
- resp = ipu6_fw_isys_get_resp(isys->fwcom, IPU6_BASE_MSG_RECV_QUEUES);
+ resp = ipu6_fw_isys_get_resp(isys->fwctx, IPU6_BASE_MSG_RECV_QUEUES);
if (!resp)
return 1;
@@ -1324,7 +1324,7 @@ static int isys_isr_one(struct ipu6_bus_device *adev)
ipu6_isys_put_stream(stream);
leave:
- ipu6_fw_isys_put_resp(isys->fwcom, IPU6_BASE_MSG_RECV_QUEUES);
+ ipu6_fw_isys_put_resp(isys->fwctx, IPU6_BASE_MSG_RECV_QUEUES);
return 0;
}
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys.h b/drivers/media/pci/intel/ipu6/ipu6-isys.h
index 7fb8cb820912..0b139ab24431 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys.h
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys.h
@@ -114,8 +114,7 @@ struct sensor_async_sd {
* @csi2_rx_ctrl_cached: cached shared value between all CSI2 receivers
* @streams_lock: serialise access to streams
* @streams: streams per firmware stream ID
- * @fwcom: fw communication layer private pointer
- * or optional external library private pointer
+ * @fwctx: fw communication layer context pointer
* @phy_termcal_val: the termination calibration value, only used for DWC PHY
* @need_reset: Isys requires d0i0->i3 transition
* @ref_count: total number of callers fw open
@@ -136,7 +135,7 @@ struct ipu6_isys {
spinlock_t streams_lock;
struct ipu6_isys_stream streams[IPU6_ISYS_MAX_STREAMS];
int streams_ref_count[IPU6_ISYS_MAX_STREAMS];
- void *fwcom;
+ void *fwctx;
u32 phy_termcal_val;
bool need_reset;
bool icache_prefetch;
diff --git a/drivers/media/platform/microchip/microchip-isc-base.c b/drivers/media/platform/microchip/microchip-isc-base.c
index 21d108eb93c7..963d5fd53b58 100644
--- a/drivers/media/platform/microchip/microchip-isc-base.c
+++ b/drivers/media/platform/microchip/microchip-isc-base.c
@@ -8,6 +8,7 @@
* Author: Eugen Hristev <eugen.hristev@microchip.com>
*
*/
+#include <linux/bitfield.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/math64.h>
@@ -62,17 +63,17 @@ static inline void isc_update_awb_ctrls(struct isc_device *isc)
/* In here we set our actual hw pipeline config */
regmap_write(isc->regmap, ISC_WB_O_RGR,
- ((ctrls->offset[ISC_HIS_CFG_MODE_R])) |
- ((ctrls->offset[ISC_HIS_CFG_MODE_GR]) << 16));
+ FIELD_PREP(ISC_WB_O_LO, ctrls->offset[ISC_HIS_CFG_MODE_R]) |
+ FIELD_PREP(ISC_WB_O_HI, ctrls->offset[ISC_HIS_CFG_MODE_GR]));
regmap_write(isc->regmap, ISC_WB_O_BGB,
- ((ctrls->offset[ISC_HIS_CFG_MODE_B])) |
- ((ctrls->offset[ISC_HIS_CFG_MODE_GB]) << 16));
+ FIELD_PREP(ISC_WB_O_LO, ctrls->offset[ISC_HIS_CFG_MODE_B]) |
+ FIELD_PREP(ISC_WB_O_HI, ctrls->offset[ISC_HIS_CFG_MODE_GB]));
regmap_write(isc->regmap, ISC_WB_G_RGR,
- ctrls->gain[ISC_HIS_CFG_MODE_R] |
- (ctrls->gain[ISC_HIS_CFG_MODE_GR] << 16));
+ FIELD_PREP(ISC_WB_G_LO, ctrls->gain[ISC_HIS_CFG_MODE_R]) |
+ FIELD_PREP(ISC_WB_G_HI, ctrls->gain[ISC_HIS_CFG_MODE_GR]));
regmap_write(isc->regmap, ISC_WB_G_BGB,
- ctrls->gain[ISC_HIS_CFG_MODE_B] |
- (ctrls->gain[ISC_HIS_CFG_MODE_GB] << 16));
+ FIELD_PREP(ISC_WB_G_LO, ctrls->gain[ISC_HIS_CFG_MODE_B]) |
+ FIELD_PREP(ISC_WB_G_HI, ctrls->gain[ISC_HIS_CFG_MODE_GB]));
}
static inline void isc_reset_awb_ctrls(struct isc_device *isc)
@@ -291,7 +292,8 @@ static int isc_configure(struct isc_device *isc)
struct isc_subdev_entity *subdev = isc->current_subdev;
int ret;
- pfe_cfg0 = isc->config.sd_format->pfe_cfg0_bps;
+ pfe_cfg0 = FIELD_PREP(ISC_PFE_CFG0_BPS_MASK,
+ isc->config.sd_format->pfe_cfg0_bps);
pipeline = isc->config.bits_pipeline;
dcfg = isc->config.dcfg_imode | isc->dcfg;
diff --git a/drivers/media/platform/microchip/microchip-isc-regs.h b/drivers/media/platform/microchip/microchip-isc-regs.h
index e77e1d9a1db8..fe145b142b82 100644
--- a/drivers/media/platform/microchip/microchip-isc-regs.h
+++ b/drivers/media/platform/microchip/microchip-isc-regs.h
@@ -31,11 +31,11 @@
#define ISC_PFE_CFG0_MODE_PROGRESSIVE (0x0 << 4)
#define ISC_PFE_CFG0_MODE_MASK GENMASK(6, 4)
-#define ISC_PFE_CFG0_BPS_EIGHT (0x4 << 28)
-#define ISC_PFG_CFG0_BPS_NINE (0x3 << 28)
-#define ISC_PFG_CFG0_BPS_TEN (0x2 << 28)
-#define ISC_PFG_CFG0_BPS_ELEVEN (0x1 << 28)
-#define ISC_PFG_CFG0_BPS_TWELVE (0x0 << 28)
+#define ISC_PFE_CFG0_BPS_EIGHT 0x4
+#define ISC_PFE_CFG0_BPS_NINE 0x3
+#define ISC_PFE_CFG0_BPS_TEN 0x2
+#define ISC_PFE_CFG0_BPS_ELEVEN 0x1
+#define ISC_PFE_CFG0_BPS_TWELVE 0x0
#define ISC_PFE_CFG0_BPS_MASK GENMASK(30, 28)
#define ISC_PFE_CFG0_COLEN BIT(12)
@@ -149,6 +149,12 @@
/* ISC White Balance Gain for B, GB Register */
#define ISC_WB_G_BGB 0x0000006c
+/* Each WB offset/gain register packs two 13-bit fields, low and high */
+#define ISC_WB_O_LO GENMASK(12, 0) /* R or B offset [12:0] */
+#define ISC_WB_O_HI GENMASK(28, 16) /* GR or GB offset [28:16] */
+#define ISC_WB_G_LO GENMASK(12, 0) /* R or B gain [12:0] */
+#define ISC_WB_G_HI GENMASK(28, 16) /* GR or GB gain [28:16] */
+
/* ISC Color Filter Array Control Register */
#define ISC_CFA_CTRL 0x00000070
diff --git a/drivers/media/platform/microchip/microchip-isc.h b/drivers/media/platform/microchip/microchip-isc.h
index f5e322c2e36b..d7bcd74efff9 100644
--- a/drivers/media/platform/microchip/microchip-isc.h
+++ b/drivers/media/platform/microchip/microchip-isc.h
@@ -62,7 +62,9 @@ struct isc_subdev_entity {
* @mbus_code: V4L2 media bus format code.
* @cfa_baycfg: If this format is RAW BAYER, indicate the type of bayer.
this is either BGBG, RGRG, etc.
- * @pfe_cfg0_bps: Number of hardware data lines connected to the ISC
+ * @pfe_cfg0_bps: ISC_PFE_CFG0 BPS field value (e.g. ISC_PFE_CFG0_BPS_EIGHT),
+ written into PFE_CFG0 with FIELD_PREP(ISC_PFE_CFG0_BPS_MASK)
+ at configure time.
* @raw: If the format is raw bayer.
*/
diff --git a/drivers/media/platform/microchip/microchip-sama5d2-isc.c b/drivers/media/platform/microchip/microchip-sama5d2-isc.c
index 5bd109e93a8f..5e41eee45dbd 100644
--- a/drivers/media/platform/microchip/microchip-sama5d2-isc.c
+++ b/drivers/media/platform/microchip/microchip-sama5d2-isc.c
@@ -147,49 +147,49 @@ static struct isc_format sama5d2_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_SBGGR10,
.mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
- .cfa_baycfg = ISC_BAY_CFG_RGRG,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
+ .cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG10,
.mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG10,
.mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB10,
.mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
.fourcc = V4L2_PIX_FMT_SBGGR12,
.mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG12,
.mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG12,
.mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB12,
.mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
@@ -210,7 +210,7 @@ static struct isc_format sama5d2_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_Y10,
.mbus_code = MEDIA_BUS_FMT_Y10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
},
};
diff --git a/drivers/media/platform/microchip/microchip-sama7g5-isc.c b/drivers/media/platform/microchip/microchip-sama7g5-isc.c
index d978ce448a4d..9e78ea07b013 100644
--- a/drivers/media/platform/microchip/microchip-sama7g5-isc.c
+++ b/drivers/media/platform/microchip/microchip-sama7g5-isc.c
@@ -156,49 +156,49 @@ static struct isc_format sama7g5_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_SBGGR10,
.mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
- .cfa_baycfg = ISC_BAY_CFG_RGRG,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
+ .cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG10,
.mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG10,
.mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB10,
.mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
.fourcc = V4L2_PIX_FMT_SBGGR12,
.mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_BGBG,
},
{
.fourcc = V4L2_PIX_FMT_SGBRG12,
.mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GBGB,
},
{
.fourcc = V4L2_PIX_FMT_SGRBG12,
.mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_GRGR,
},
{
.fourcc = V4L2_PIX_FMT_SRGGB12,
.mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TWELVE,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TWELVE,
.cfa_baycfg = ISC_BAY_CFG_RGRG,
},
{
@@ -224,7 +224,7 @@ static struct isc_format sama7g5_formats_list[] = {
{
.fourcc = V4L2_PIX_FMT_Y10,
.mbus_code = MEDIA_BUS_FMT_Y10_1X10,
- .pfe_cfg0_bps = ISC_PFG_CFG0_BPS_TEN,
+ .pfe_cfg0_bps = ISC_PFE_CFG0_BPS_TEN,
},
};
diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index e289cc71ba98..73b97ec578ec 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -2851,6 +2851,26 @@ static const struct usb_device_id uvc_ids[] = {
.bInterfaceSubClass = 1,
.bInterfaceProtocol = 0,
.driver_info = (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
+ /* Avermedia GC515 */
+ { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
+ | USB_DEVICE_ID_MATCH_INT_INFO,
+ .idVendor = 0x07ca,
+ .idProduct = 0x1515,
+ .bInterfaceClass = USB_CLASS_VIDEO,
+ .bInterfaceSubClass = 1,
+ .bInterfaceProtocol = UVC_PC_PROTOCOL_15,
+ .driver_info = (kernel_ulong_t)&(const struct uvc_device_info){
+ .uvc_version = 0x0110,
+ } },
+ /* AVerMedia Live Gamer HD 2 capture card */
+ { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
+ | USB_DEVICE_ID_MATCH_INT_INFO,
+ .idVendor = 0x07ca,
+ .idProduct = 0x1570,
+ .bInterfaceClass = USB_CLASS_VIDEO,
+ .bInterfaceSubClass = 1,
+ .bInterfaceProtocol = 0,
+ .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_DISABLE_AUTOSUSPEND) },
/* ViMicro Vega */
{ .match_flags = USB_DEVICE_ID_MATCH_DEVICE
| USB_DEVICE_ID_MATCH_INT_INFO,
diff --git a/drivers/mtd/spi-nor/macronix.c b/drivers/mtd/spi-nor/macronix.c
index e97f5cbd9aad..c854a1163a47 100644
--- a/drivers/mtd/spi-nor/macronix.c
+++ b/drivers/mtd/spi-nor/macronix.c
@@ -132,7 +132,9 @@ static const struct flash_info macronix_nor_parts[] = {
}, {
/* MX25L12805D */
.id = SNOR_ID(0xc2, 0x20, 0x18),
+ .size = SZ_16M,
.flags = SPI_NOR_HAS_LOCK | SPI_NOR_4BIT_BP,
+ .no_sfdp_flags = SECT_4K,
}, {
/* MX25L25635E, MX25L25645G */
.id = SNOR_ID(0xc2, 0x20, 0x19),
diff --git a/drivers/net/can/dev/netlink.c b/drivers/net/can/dev/netlink.c
index 766d455950f5..b958c04d53db 100644
--- a/drivers/net/can/dev/netlink.c
+++ b/drivers/net/can/dev/netlink.c
@@ -812,7 +812,7 @@ static size_t can_get_size(const struct net_device *dev)
size += nla_total_size(sizeof(u32)); /* IFLA_CAN_STATE */
size += nla_total_size(sizeof(struct can_ctrlmode)); /* IFLA_CAN_CTRLMODE */
size += nla_total_size(sizeof(u32)); /* IFLA_CAN_RESTART_MS */
- if (priv->do_get_berr_counter) /* IFLA_CAN_BERR_COUNTER */
+ if (priv->do_get_berr_counter && netif_running(dev)) /* IFLA_CAN_BERR_COUNTER */
size += nla_total_size(sizeof(struct can_berr_counter));
if (priv->termination_const) {
size += nla_total_size(sizeof(priv->termination)); /* IFLA_CAN_TERMINATION */
@@ -1006,7 +1006,7 @@ static int can_fill_info(struct sk_buff *skb, const struct net_device *dev)
nla_put(skb, IFLA_CAN_CTRLMODE, sizeof(cm), &cm) ||
nla_put_u32(skb, IFLA_CAN_RESTART_MS, priv->restart_ms) ||
- (priv->do_get_berr_counter &&
+ (priv->do_get_berr_counter && netif_running(dev) &&
!priv->do_get_berr_counter(dev, &bec) &&
nla_put(skb, IFLA_CAN_BERR_COUNTER, sizeof(bec), &bec)) ||
diff --git a/drivers/net/can/dev/skb.c b/drivers/net/can/dev/skb.c
index 14edec5afb57..9e274147ffe5 100644
--- a/drivers/net/can/dev/skb.c
+++ b/drivers/net/can/dev/skb.c
@@ -4,6 +4,7 @@
* Copyright (C) 2008-2009 Wolfgang Grandegger <wg@grandegger.com>
*/
+#include <linux/can/can-ml.h>
#include <linux/can/dev.h>
#include <linux/module.h>
#include <net/can.h>
@@ -54,6 +55,7 @@ int can_put_echo_skb(struct sk_buff *skb, struct net_device *dev,
if (idx >= priv->echo_skb_max) {
netdev_err(dev, "%s: BUG! Trying to access can_priv::echo_skb out of bounds (%u/max %u)\n",
__func__, idx, priv->echo_skb_max);
+ dev_kfree_skb_any(skb);
return -EINVAL;
}
@@ -62,7 +64,7 @@ int can_put_echo_skb(struct sk_buff *skb, struct net_device *dev,
(skb->protocol != htons(ETH_P_CAN) &&
skb->protocol != htons(ETH_P_CANFD) &&
skb->protocol != htons(ETH_P_CANXL))) {
- kfree_skb(skb);
+ dev_kfree_skb_any(skb);
return 0;
}
@@ -90,7 +92,7 @@ int can_put_echo_skb(struct sk_buff *skb, struct net_device *dev,
} else {
/* locking problem with netif_stop_queue() ?? */
netdev_err(dev, "%s: BUG! echo_skb %d is occupied!\n", __func__, idx);
- kfree_skb(skb);
+ dev_kfree_skb_any(skb);
return -EBUSY;
}
@@ -388,7 +390,7 @@ bool can_dropped_invalid_skb(struct net_device *dev, struct sk_buff *skb)
break;
case ETH_P_CANXL:
- if (!can_is_canxl_skb(skb))
+ if (!can_cap_enabled(dev, CAN_CAP_XL) || !can_is_canxl_skb(skb))
goto inval_skb;
break;
@@ -402,7 +404,7 @@ bool can_dropped_invalid_skb(struct net_device *dev, struct sk_buff *skb)
return false;
inval_skb:
- kfree_skb(skb);
+ dev_kfree_skb_any(skb);
dev->stats.tx_dropped++;
return true;
}
diff --git a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
index 0d46f4ce5dca..39f0740421c3 100644
--- a/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
+++ b/drivers/net/can/usb/peak_usb/pcan_usb_fd.c
@@ -755,7 +755,7 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb)
u16 rx_msg_type, rx_msg_size;
size_t rx_msg_min_size;
- if (msg_end - msg_ptr < sizeof(*rx_msg)) {
+ if (msg_end - msg_ptr < sizeof(rx_msg->size)) {
err = -EBADMSG;
break;
}
@@ -766,6 +766,11 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb)
break;
}
+ if (msg_end - msg_ptr < sizeof(*rx_msg)) {
+ err = -EBADMSG;
+ break;
+ }
+
rx_msg_size = le16_to_cpu(rx_msg->size);
rx_msg_type = le16_to_cpu(rx_msg->type);
diff --git a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
index e58f1d2c26bd..eddd8406867f 100644
--- a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
+++ b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
@@ -1934,6 +1934,39 @@ static int atl1c_clean_rx(struct napi_struct *napi, int budget)
break;
}
atl1c_clean_rfd(rfd_ring, rrs, rfd_num);
+
+ /* A link partner whose l2mtu is larger than ours can hand us a
+ * frame that does not fit the buffer the hardware was given:
+ * the descriptor reports the untruncated length while at most
+ * buffer_info->length bytes were written. skb_put() of such a
+ * length would pull in whatever the tail of the buffer holds
+ * and, past the end of the skb, trips skb_over_panic and takes
+ * the host down. Bound it by what the device actually wrote
+ * rather than by skb_tailroom(): the skb carries 64 bytes of
+ * slack for the DMA erratum, so the two are not the same.
+ *
+ * Frames shorter than an Ethernet header plus its FCS are
+ * rejected by the same test; before this, a length below
+ * ETH_FCS_LEN underflowed in skb_put().
+ */
+ if (unlikely(length < ETH_HLEN + ETH_FCS_LEN ||
+ length > buffer_info->length)) {
+ dev_core_stats_rx_dropped_inc(netdev);
+ if (netif_msg_rx_err(adapter))
+ dev_warn_ratelimited(&pdev->dev,
+ "dropped rx frame: %u bytes, buffer %u\n",
+ length, buffer_info->length);
+ kfree_skb_reason(skb, SKB_DROP_REASON_PKT_TOO_BIG);
+ /* atl1c_clean_rfd() has already recycled the RFD, so
+ * count the drop as work: the refill at the end of the
+ * poll only runs when work_done is non-zero, and
+ * sustained oversized traffic would otherwise drain the
+ * ring and stop RX for good.
+ */
+ work_done++;
+ continue;
+ }
+
skb_put(skb, length - ETH_FCS_LEN);
skb->protocol = eth_type_trans(skb, netdev);
atl1c_rx_checksum(adapter, skb, rrs);
diff --git a/drivers/net/ethernet/jme.c b/drivers/net/ethernet/jme.c
index 25ec5f757c68..acb4d41f56e1 100644
--- a/drivers/net/ethernet/jme.c
+++ b/drivers/net/ethernet/jme.c
@@ -1502,7 +1502,8 @@ jme_intr_msi(struct jme_adapter *jme, u32 intrstat)
* all other events are ignored
*/
jwrite32(jme, JME_IEVE, intrstat);
- schedule_work(&jme->linkch_task);
+ if (!test_bit(JME_FLAG_WOL_QUIESCE, &jme->flags))
+ schedule_work(&jme->linkch_task);
goto out_reenable;
}
@@ -1546,7 +1547,8 @@ jme_intr_msi(struct jme_adapter *jme, u32 intrstat)
/*
* Re-enable interrupt
*/
- jwrite32f(jme, JME_IENS, INTR_ENABLE);
+ if (!test_bit(JME_FLAG_WOL_QUIESCE, &jme->flags))
+ jwrite32f(jme, JME_IENS, INTR_ENABLE);
}
static irqreturn_t
@@ -1892,6 +1894,7 @@ jme_powersave_phy(struct jme_adapter *jme)
jme_set_100m_half(jme);
if (jme->reg_pmcs & (PMCS_LFEN | PMCS_LREN))
jme_wait_link(jme);
+ jme_mac_rxclk_on(jme);
jme_clear_pm_enable_wol(jme);
} else {
jme_phy_off(jme);
@@ -3158,6 +3161,21 @@ jme_shutdown(struct pci_dev *pdev)
struct net_device *netdev = pci_get_drvdata(pdev);
struct jme_adapter *jme = netdev_priv(netdev);
+ if (netif_running(netdev)) {
+ set_bit(JME_FLAG_WOL_QUIESCE, &jme->flags);
+ netif_device_detach(netdev);
+ netif_stop_queue(netdev);
+ jme_stop_irq(jme);
+ synchronize_irq(pdev->irq);
+ jme_stop_irq(jme);
+ cancel_work_sync(&jme->linkch_task);
+ tasklet_disable(&jme->pcc_task);
+ napi_disable(&jme->napi);
+ tasklet_disable(&jme->txclean_task);
+ tasklet_disable(&jme->rxclean_task);
+ tasklet_disable(&jme->rxempty_task);
+ }
+
jme_powersave_phy(jme);
pci_pme_active(pdev, true);
}
@@ -3172,11 +3190,16 @@ jme_suspend(struct device *dev)
if (!netif_running(netdev))
return 0;
- atomic_dec(&jme->link_changing);
-
+ set_bit(JME_FLAG_WOL_QUIESCE, &jme->flags);
netif_device_detach(netdev);
netif_stop_queue(netdev);
jme_stop_irq(jme);
+ synchronize_irq(jme->pdev->irq);
+ jme_stop_irq(jme);
+ cancel_work_sync(&jme->linkch_task);
+ tasklet_disable(&jme->pcc_task);
+ napi_disable(&jme->napi);
+ atomic_dec(&jme->link_changing);
tasklet_disable(&jme->txclean_task);
tasklet_disable(&jme->rxclean_task);
@@ -3215,6 +3238,7 @@ jme_resume(struct device *dev)
return 0;
jme_clear_pm_disable_wol(jme);
+ jme_mac_rxclk_off(jme);
jme_phy_on(jme);
if (test_bit(JME_FLAG_SSET, &jme->flags))
jme_set_link_ksettings(netdev, &jme->old_cmd);
@@ -3226,8 +3250,11 @@ jme_resume(struct device *dev)
atomic_inc(&jme->link_changing);
- jme_reset_link(jme);
+ napi_enable(&jme->napi);
+ tasklet_enable(&jme->pcc_task);
+ clear_bit(JME_FLAG_WOL_QUIESCE, &jme->flags);
+ jme_reset_link(jme);
jme_start_irq(jme);
return 0;
diff --git a/drivers/net/ethernet/jme.h b/drivers/net/ethernet/jme.h
index 860494ff3714..ec5dfddf7322 100644
--- a/drivers/net/ethernet/jme.h
+++ b/drivers/net/ethernet/jme.h
@@ -450,6 +450,7 @@ enum jme_flags_bits {
JME_FLAG_SSET = 2,
JME_FLAG_POLL = 5,
JME_FLAG_SHUTDOWN = 6,
+ JME_FLAG_WOL_QUIESCE = 7,
};
#define TX_TIMEOUT (5 * HZ)
diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.c b/drivers/net/ethernet/ti/am65-cpsw-nuss.c
index 6330540979d7..f54cfcb7d153 100644
--- a/drivers/net/ethernet/ti/am65-cpsw-nuss.c
+++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.c
@@ -128,6 +128,7 @@
#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_319 BIT(21)
#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_320 BIT(22)
#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO BIT(23)
+#define AM65_CPSW_PN_TS_CTL_LTYPE2_TS_UNI_EN BIT(24)
/* The PTP event messages - Sync, Delay_Req, Pdelay_Req, and Pdelay_Resp. */
#define AM65_CPSW_TS_EVENT_MSG_TYPE_BITS (BIT(0) | BIT(1) | BIT(2) | BIT(3))
@@ -1857,7 +1858,8 @@ static int am65_cpsw_nuss_hwtstamp_set(struct net_device *ndev,
AM65_CPSW_PN_TS_CTL_LTYPE2_TS_132 |
AM65_CPSW_PN_TS_CTL_LTYPE2_TS_319 |
AM65_CPSW_PN_TS_CTL_LTYPE2_TS_320 |
- AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO;
+ AM65_CPSW_PN_TS_CTL_LTYPE2_TS_TTL_NONZERO |
+ AM65_CPSW_PN_TS_CTL_LTYPE2_TS_UNI_EN;
ts_ctrl = AM65_CPSW_TS_EVENT_MSG_TYPE_BITS <<
AM65_CPSW_PN_TS_CTL_MSG_TYPE_EN_SHIFT;
diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index 7db389ae866d..c1859ffcdaf1 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -1372,9 +1372,10 @@ static int veth_set_channels(struct net_device *dev,
* to identify the range we have to disable
*/
veth_disable_range_safe(dev, new_rx_count, old_rx_count);
- netif_carrier_on(dev);
- if (peer)
+ if (peer && (peer->flags & IFF_UP)) {
+ netif_carrier_on(dev);
netif_carrier_on(peer);
+ }
}
/* update XDP supported features */
diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c
index 3390e341d1e5..7ecd73a5a374 100644
--- a/drivers/net/vxlan/vxlan_core.c
+++ b/drivers/net/vxlan/vxlan_core.c
@@ -996,12 +996,6 @@ static int vxlan_fdb_update_existing(struct vxlan_dev *vxlan,
return -EOPNOTSUPP;
}
- if (rcu_access_pointer(f->nh) &&
- !(state & (NUD_PERMANENT | NUD_NOARP))) {
- NL_SET_ERR_MSG(extack, "Cannot make a nexthop fdb dynamic");
- return -EOPNOTSUPP;
- }
-
/* Do not allow an externally learned entry to take over an entry added
* by the user.
*/
@@ -1263,11 +1257,6 @@ static int vxlan_fdb_add(struct ndmsg *ndm, struct nlattr *tb[],
if (err)
return err;
- if (nhid && !(ndm->ndm_state & (NUD_PERMANENT | NUD_NOARP))) {
- NL_SET_ERR_MSG(extack, "A nexthop fdb cannot be dynamic");
- return -EINVAL;
- }
-
if (vxlan->default_dst.remote_ip.sa.sa_family != ip.sa.sa_family)
return -EAFNOSUPPORT;
@@ -2887,6 +2876,9 @@ static void vxlan_cleanup(struct timer_list *t)
if (f->flags & NTF_EXT_LEARNED)
continue;
+ if (rcu_access_pointer(f->nh))
+ continue;
+
timeout = READ_ONCE(f->updated) + vxlan->cfg.age_interval * HZ;
if (time_before_eq(timeout, jiffies)) {
spin_lock(&vxlan->hash_lock);
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
index 872c48806d09..62d072b6369e 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/cfg80211.c
@@ -6504,6 +6504,7 @@ brcmf_bss_connect_done(struct brcmf_cfg80211_info *cfg,
struct brcmf_cfg80211_profile *profile = &ifp->vif->profile;
struct brcmf_cfg80211_connect_info *conn_info = cfg_to_conn(cfg);
struct cfg80211_connect_resp_params conn_params;
+ bool authorized;
brcmf_dbg(TRACE, "Enter\n");
@@ -6528,7 +6529,13 @@ brcmf_bss_connect_done(struct brcmf_cfg80211_info *cfg,
conn_params.req_ie_len = conn_info->req_ie_len;
conn_params.resp_ie = conn_info->resp_ie;
conn_params.resp_ie_len = conn_info->resp_ie_len;
+ authorized = completed &&
+ ((profile->use_fwsup == BRCMF_PROFILE_FWSUP_SAE) ||
+ (profile->use_fwsup == BRCMF_PROFILE_FWSUP_PSK));
cfg80211_connect_done(ndev, &conn_params, GFP_KERNEL);
+ if (authorized)
+ cfg80211_port_authorized(ndev, profile->bssid, NULL, 0,
+ GFP_KERNEL);
brcmf_dbg(CONN, "Report connect result - connection %s\n",
completed ? "succeeded" : "failed");
}
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c
index 069ba7016654..ff47d4827722 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c
@@ -1639,7 +1639,7 @@ int brcmf_proto_msgbuf_attach(struct brcmf_pub *drvr)
if (!msgbuf->tx_pktids)
goto fail;
msgbuf->rx_pktids = brcmf_msgbuf_init_pktids(NR_RX_PKTIDS,
- DMA_FROM_DEVICE);
+ DMA_BIDIRECTIONAL);
if (!msgbuf->rx_pktids)
goto fail;
diff --git a/drivers/net/wireless/realtek/rtw88/mac80211.c b/drivers/net/wireless/realtek/rtw88/mac80211.c
index 0b622d7663b6..2a9b09fa76e7 100644
--- a/drivers/net/wireless/realtek/rtw88/mac80211.c
+++ b/drivers/net/wireless/realtek/rtw88/mac80211.c
@@ -474,6 +474,7 @@ static int rtw_ops_start_ap(struct ieee80211_hw *hw,
mutex_lock(&rtwdev->mutex);
rtw_write32_set(rtwdev, REG_TCR, BIT_TCR_UPDATE_HGQMD);
+ rtw_write16(rtwdev, REG_ATIMWND, ATIMWND_AP);
rtwdev->ap_active = true;
rtw_store_op_chan(rtwdev, true);
chip->ops->phy_calibration(rtwdev);
@@ -490,6 +491,7 @@ static void rtw_ops_stop_ap(struct ieee80211_hw *hw,
mutex_lock(&rtwdev->mutex);
rtw_write32_clr(rtwdev, REG_TCR, BIT_TCR_UPDATE_HGQMD);
+ rtw_write16(rtwdev, REG_ATIMWND, ATIMWND_DEFAULT);
rtwdev->ap_active = false;
if (!rtw_core_check_sta_active(rtwdev))
rtw_clear_op_chan(rtwdev);
diff --git a/drivers/net/wireless/realtek/rtw88/reg.h b/drivers/net/wireless/realtek/rtw88/reg.h
index 08e9494977e0..0b9ba6f37680 100644
--- a/drivers/net/wireless/realtek/rtw88/reg.h
+++ b/drivers/net/wireless/realtek/rtw88/reg.h
@@ -483,6 +483,8 @@
#define REG_DRVERLYINT 0x0558
#define REG_BCNDMATIM 0x0559
#define REG_ATIMWND 0x055A
+#define ATIMWND_DEFAULT 0x2
+#define ATIMWND_AP 0xa
#define REG_USTIME_TSF 0x055C
#define REG_BCN_MAX_ERR 0x055D
#define REG_RXTSF_OFFSET_CCK 0x055E
diff --git a/drivers/net/wireless/realtek/rtw88/tx.c b/drivers/net/wireless/realtek/rtw88/tx.c
index 8786bbb421c2..701ed7c4f3f7 100644
--- a/drivers/net/wireless/realtek/rtw88/tx.c
+++ b/drivers/net/wireless/realtek/rtw88/tx.c
@@ -196,7 +196,7 @@ void rtw_tx_report_purge_timer(struct timer_list *t)
void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn)
{
struct rtw_tx_report *tx_report = &rtwdev->tx_report;
- unsigned long timeout = RTW_TX_PROBE_TIMEOUT;
+ unsigned long timeout;
unsigned long flags;
u8 *drv_data;
@@ -208,9 +208,10 @@ void rtw_tx_report_enqueue(struct rtw_dev *rtwdev, struct sk_buff *skb, u8 sn)
__skb_queue_tail(&tx_report->queue, skb);
spin_unlock_irqrestore(&tx_report->q_lock, flags);
- if (rtwdev->chip->id == RTW_CHIP_TYPE_8723D &&
- rtwdev->hci.type == RTW_HCI_TYPE_USB)
- timeout = msecs_to_jiffies(2500);
+ if (rtwdev->hci.type == RTW_HCI_TYPE_PCIE)
+ timeout = RTW_TX_PROBE_TIMEOUT_SHORT;
+ else
+ timeout = RTW_TX_PROBE_TIMEOUT_LONG;
mod_timer(&tx_report->purge_timer, jiffies + timeout);
}
diff --git a/drivers/net/wireless/realtek/rtw88/tx.h b/drivers/net/wireless/realtek/rtw88/tx.h
index d34cdeca16f1..168f48e87853 100644
--- a/drivers/net/wireless/realtek/rtw88/tx.h
+++ b/drivers/net/wireless/realtek/rtw88/tx.h
@@ -7,7 +7,8 @@
#define RTK_TX_MAX_AGG_NUM_MASK 0x1f
-#define RTW_TX_PROBE_TIMEOUT msecs_to_jiffies(500)
+#define RTW_TX_PROBE_TIMEOUT_SHORT msecs_to_jiffies(500)
+#define RTW_TX_PROBE_TIMEOUT_LONG msecs_to_jiffies(2500)
struct rtw_tx_desc {
__le32 w0;
diff --git a/drivers/net/wireless/realtek/rtw88/usb.c b/drivers/net/wireless/realtek/rtw88/usb.c
index 64e1c3420e0a..5c766701e632 100644
--- a/drivers/net/wireless/realtek/rtw88/usb.c
+++ b/drivers/net/wireless/realtek/rtw88/usb.c
@@ -2,9 +2,11 @@
/* Copyright(c) 2018-2019 Realtek Corporation
*/
+#include <linux/ip.h>
#include <linux/module.h>
-#include <linux/usb.h>
#include <linux/mutex.h>
+#include <linux/udp.h>
+#include <linux/usb.h>
#include "main.h"
#include "debug.h"
#include "mac.h"
@@ -557,9 +559,61 @@ static int rtw_usb_write_data_h2c(struct rtw_dev *rtwdev, u8 *buf, u32 size)
return rtw_usb_write_data(rtwdev, &pkt_info, buf);
}
-static u8 rtw_usb_tx_queue_mapping_to_qsel(struct sk_buff *skb)
+static bool rtw_usb_bmc_needs_dtim(struct sk_buff *skb)
+{
+ struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
+ struct udphdr *udphdr;
+
+ if (info->control.flags & IEEE80211_TX_CTRL_PORT_CTRL_PROTO)
+ return true;
+
+ if (skb->protocol == htons(ETH_P_ARP))
+ return true;
+
+ if (skb->protocol == htons(ETH_P_IP) &&
+ ip_hdr(skb)->protocol == IPPROTO_UDP) {
+ udphdr = udp_hdr(skb);
+
+ return udphdr->dest == htons(67) || udphdr->dest == htons(68);
+ }
+
+ return false;
+}
+
+#define RTW_USB_HIQ_REFILL_INTERVAL (HZ / 10) /* one unit of budget per 100 ms */
+#define RTW_USB_HIQ_BUDGET_MAX 16
+
+static bool rtw_usb_hiq_take_budget(struct rtw_usb *rtwusb)
+{
+ unsigned long flags, elapsed, add;
+ bool ok;
+
+ spin_lock_irqsave(&rtwusb->hiq_lock, flags);
+
+ elapsed = jiffies - rtwusb->hiq_refill;
+ add = elapsed / RTW_USB_HIQ_REFILL_INTERVAL;
+ if (add) {
+ rtwusb->hiq_budget = min_t(unsigned long, rtwusb->hiq_budget + add,
+ RTW_USB_HIQ_BUDGET_MAX);
+
+ /* keep the unfinished part of the interval for the next unit */
+ rtwusb->hiq_refill = jiffies - elapsed % RTW_USB_HIQ_REFILL_INTERVAL;
+ }
+
+ ok = rtwusb->hiq_budget > 0;
+ if (ok)
+ rtwusb->hiq_budget--;
+
+ spin_unlock_irqrestore(&rtwusb->hiq_lock, flags);
+
+ return ok;
+}
+
+static u8 rtw_usb_tx_queue_mapping_to_qsel(struct rtw_usb *rtwusb,
+ struct sk_buff *skb)
{
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
+ struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
__le16 fc = hdr->frame_control;
u8 qsel;
@@ -567,7 +621,9 @@ static u8 rtw_usb_tx_queue_mapping_to_qsel(struct sk_buff *skb)
qsel = TX_DESC_QSEL_MGMT;
else if (is_broadcast_ether_addr(hdr->addr1) ||
is_multicast_ether_addr(hdr->addr1))
- qsel = TX_DESC_QSEL_HIGH;
+ qsel = (info->flags & IEEE80211_TX_CTL_SEND_AFTER_DTIM) &&
+ rtw_usb_bmc_needs_dtim(skb) && rtw_usb_hiq_take_budget(rtwusb) ?
+ TX_DESC_QSEL_HIGH : skb->priority;
else if (skb_get_queue_mapping(skb) <= IEEE80211_AC_BK)
qsel = skb->priority;
else
@@ -586,7 +642,7 @@ static int rtw_usb_tx_write(struct rtw_dev *rtwdev,
u8 *pkt_desc;
int ep;
- pkt_info->qsel = rtw_usb_tx_queue_mapping_to_qsel(skb);
+ pkt_info->qsel = rtw_usb_tx_queue_mapping_to_qsel(rtwusb, skb);
pkt_desc = skb_push(skb, chip->tx_pkt_desc_sz);
memset(pkt_desc, 0, chip->tx_pkt_desc_sz);
ep = qsel_to_ep(rtwusb, pkt_info->qsel);
@@ -1027,6 +1083,10 @@ static int rtw_usb_init_tx(struct rtw_dev *rtwdev)
struct rtw_usb *rtwusb = rtw_get_usb_priv(rtwdev);
int i;
+ spin_lock_init(&rtwusb->hiq_lock);
+ rtwusb->hiq_budget = RTW_USB_HIQ_BUDGET_MAX;
+ rtwusb->hiq_refill = jiffies;
+
rtwusb->txwq = create_singlethread_workqueue("rtw88_usb: tx wq");
if (!rtwusb->txwq) {
rtw_err(rtwdev, "failed to create TX work queue\n");
diff --git a/drivers/net/wireless/realtek/rtw88/usb.h b/drivers/net/wireless/realtek/rtw88/usb.h
index 9b695b688b24..94ff21dd0846 100644
--- a/drivers/net/wireless/realtek/rtw88/usb.h
+++ b/drivers/net/wireless/realtek/rtw88/usb.h
@@ -75,6 +75,11 @@ struct rtw_usb {
u8 out_ep[RTW_USB_EP_MAX];
int qsel_to_ep[TX_DESC_QSEL_MAX];
+ /* protects hiq_budget and hiq_refill */
+ spinlock_t hiq_lock;
+ u32 hiq_budget;
+ unsigned long hiq_refill;
+
struct workqueue_struct *txwq, *rxwq;
struct sk_buff_head tx_queue[RTW_USB_EP_MAX];
diff --git a/drivers/net/wireless/rsi/rsi_91x_core.c b/drivers/net/wireless/rsi/rsi_91x_core.c
index ead4d4e04328..3281fc94e107 100644
--- a/drivers/net/wireless/rsi/rsi_91x_core.c
+++ b/drivers/net/wireless/rsi/rsi_91x_core.c
@@ -458,7 +458,7 @@ void rsi_core_xmit(struct rsi_common *common, struct sk_buff *skb)
tx_params->sta_id = 0;
}
- if (rsta) {
+ if (rsta && tid < IEEE80211_NUM_TIDS) {
/* Start aggregation if not done for this tid */
if (!rsta->start_tx_aggr[tid]) {
rsta->start_tx_aggr[tid] = true;
diff --git a/drivers/nvme/common/keyring.c b/drivers/nvme/common/keyring.c
index 32d16c53133b..87cb51e1f5a7 100644
--- a/drivers/nvme/common/keyring.c
+++ b/drivers/nvme/common/keyring.c
@@ -35,6 +35,7 @@ struct key *nvme_tls_key_lookup(key_serial_t key_id)
}
if (nvme_tls_psk_revoked(key)) {
pr_err("key id %08x revoked\n", key_id);
+ key_put(key);
return ERR_PTR(-EKEYREVOKED);
}
return key;
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index af71d02b8476..a0811b49710c 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3941,10 +3941,16 @@ static int nvme_subsys_check_duplicate_ids(struct nvme_subsystem *subsys,
return 0;
}
+/* The char device of a namespace rather than of a namespace head */
+static bool nvme_is_ns_cdev(struct device *dev)
+{
+ return dev->parent->class == &nvme_class;
+}
+
static void nvme_cdev_rel(struct device *dev)
{
ida_free(&nvme_ns_chr_minor_ida, MINOR(dev->devt));
- if (dev->parent->class == &nvme_class)
+ if (nvme_is_ns_cdev(dev))
nvme_put_ns(container_of(dev, struct nvme_ns, cdev_device));
else
nvme_put_ns_head(container_of(dev, struct nvme_ns_head,
@@ -3978,6 +3984,14 @@ int nvme_cdev_add(struct cdev *cdev, struct device *cdev_device,
device_initialize(cdev_device);
cdev_init(cdev, fops);
cdev->owner = owner;
+ /* Undone in nvme_cdev_rel() */
+ if (nvme_is_ns_cdev(cdev_device))
+ nvme_get_ns(container_of(cdev_device, struct nvme_ns,
+ cdev_device));
+ else
+ nvme_get_ns_head(container_of(cdev_device,
+ struct nvme_ns_head,
+ cdev_device));
ret = cdev_device_add(cdev, cdev_device);
if (ret)
put_device(cdev_device);
@@ -4010,13 +4024,11 @@ static void nvme_add_ns_cdev(struct nvme_ns *ns)
{
ns->cdev_device.parent = ns->ctrl->device;
- nvme_get_ns(ns); /* Undone in nvme_cdev_rel() */
if (nvme_cdev_add(&ns->cdev, &ns->cdev_device,
&nvme_ns_chr_fops, ns->ctrl->ops->module,
ns->ctrl->instance, ns->head->instance)) {
dev_err(ns->ctrl->device, "Unable to create the ng%dn%d device\n",
ns->ctrl->instance, ns->head->instance);
- nvme_put_ns(ns);
return;
}
set_bit(NVME_NS_CDEV_LIVE, &ns->flags);
@@ -4743,9 +4755,8 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING_NOIO);
mutex_lock(&ctrl->namespaces_lock);
- list_splice_init_rcu(&ctrl->namespaces, &ns_list, synchronize_rcu);
+ list_splice_init_srcu(&ctrl->namespaces, &ns_list, &ctrl->srcu);
mutex_unlock(&ctrl->namespaces_lock);
- synchronize_srcu(&ctrl->srcu);
list_for_each_entry_safe(ns, next, &ns_list, list)
nvme_ns_remove(ns);
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 99ecc73393e5..0920fa9616de 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -648,14 +648,12 @@ static void nvme_add_ns_head_cdev(struct nvme_ns_head *head)
{
head->cdev_device.parent = &head->subsys->dev;
- nvme_get_ns_head(head); /* Undone in nvme_cdev_rel() */
if (nvme_cdev_add(&head->cdev, &head->cdev_device,
&nvme_ns_head_chr_fops, THIS_MODULE,
head->subsys->instance, head->instance)) {
dev_err(disk_to_dev(head->disk),
"Unable to create the ng%dn%d device\n",
head->subsys->instance, head->instance);
- nvme_put_ns_head(head);
return;
}
set_bit(NVME_NSHEAD_CDEV_LIVE, &head->flags);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 77317e5d00f9..b7c5a18b899b 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -4198,6 +4198,8 @@ static const struct pci_device_id nvme_id_table[] = {
.driver_data = NVME_QUIRK_BOGUS_NID, },
{ PCI_DEVICE(0x1344, 0x6004), /* Micron 4100AT */
.driver_data = NVME_QUIRK_DMAPOOL_ALIGN_512, },
+ { PCI_DEVICE(0x1e0f, 0x0032), /* KIOXIA XG10d */
+ .driver_data = NVME_QUIRK_DMAPOOL_ALIGN_512, },
{ PCI_DEVICE(0x1c5c, 0x1504), /* SK Hynix PC400 */
.driver_data = NVME_QUIRK_DISABLE_WRITE_ZEROES, },
{ PCI_DEVICE(0x1c5c, 0x174a), /* SK Hynix P31 SSD */
diff --git a/drivers/nvme/host/pr.c b/drivers/nvme/host/pr.c
index fe7dbe264815..d8b6ac4d83bb 100644
--- a/drivers/nvme/host/pr.c
+++ b/drivers/nvme/host/pr.c
@@ -128,8 +128,12 @@ static int nvme_pr_register(struct block_device *bdev, u64 old_key, u64 new_key,
data.crkey = cpu_to_le64(old_key);
data.nrkey = cpu_to_le64(new_key);
- cdw10 = old_key ? NVME_PR_REGISTER_ACT_REPLACE :
- NVME_PR_REGISTER_ACT_REG;
+ if (!new_key)
+ cdw10 = NVME_PR_REGISTER_ACT_UNREG;
+ else if (old_key)
+ cdw10 = NVME_PR_REGISTER_ACT_REPLACE;
+ else
+ cdw10 = NVME_PR_REGISTER_ACT_REG;
cdw10 |= (flags & PR_FL_IGNORE_KEY) ? NVME_PR_IGNORE_KEY : 0;
cdw10 |= NVME_PR_CPTPL_PERSIST;
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 5b9b540693b5..ff7b9e379614 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -735,7 +735,6 @@ static int nvme_rdma_setup_io_queues(struct nvme_rdma_ctrl *ctrl,
for (i = 0; i < nr_queues; i++) {
struct nvme_rdma_queue *queue = &ctrl->queues[first + i];
- queue->ctrl = ctrl;
queue->queue_size = queue_size;
ctxs[i].queue = queue;
@@ -798,7 +797,6 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
bool pi_capable = false;
int error;
- ctrl->queues[0].ctrl = ctrl;
ctrl->queues[0].queue_size = NVME_AQ_DEPTH;
error = nvme_rdma_alloc_queue(&ctrl->queues[0]);
if (error)
@@ -2284,7 +2282,7 @@ static struct nvme_rdma_ctrl *nvme_rdma_alloc_ctrl(struct device *dev,
struct nvmf_ctrl_options *opts)
{
struct nvme_rdma_ctrl *ctrl;
- int ret;
+ int ret, i;
ctrl = kzalloc_obj(*ctrl);
if (!ctrl)
@@ -2343,6 +2341,9 @@ static struct nvme_rdma_ctrl *nvme_rdma_alloc_ctrl(struct device *dev,
if (!ctrl->queues)
goto out_free_ctrl;
+ for (i = 0; i < ctrl->ctrl.queue_count; i++)
+ ctrl->queues[i].ctrl = ctrl;
+
ret = nvme_init_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops,
0 /* no quirks, we're perfect! */);
if (ret)
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index ce28c9b03817..50297ce8bdde 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -70,6 +70,12 @@ MODULE_PARM_DESC(tls_handshake_timeout,
"nvme TLS handshake timeout in seconds (default 10)");
#endif
+/* read a TLS control record in chunks of this size */
+#define NVME_TCP_TLS_REC_CHUNK 256
+
+/* TLS records read while waiting for the ICResp, the ICResp included */
+#define NVME_TCP_ICRESP_MAX_RECORDS 8
+
static atomic_t nvme_tcp_cpu_queues[NR_CPUS];
enum nvme_tcp_send_state {
@@ -111,6 +117,9 @@ enum nvme_tcp_queue_flags {
NVME_TCP_Q_IO_CPU_SET = 3,
};
+#define NVME_TCP_OPT_NR_QUEUES(opts) ((opts)->nr_io_queues + \
+ (opts)->nr_write_queues + (opts)->nr_poll_queues + 1)
+
enum nvme_tcp_recv_state {
NVME_TCP_RECV_PDU = 0,
NVME_TCP_RECV_DATA,
@@ -1550,23 +1559,161 @@ static void nvme_tcp_free_queue(struct nvme_ctrl *nctrl, int qid)
* nvme_reset_ctrl_work(), queue teardown is immediately followed by
* re-allocation. fput() defers socket cleanup to delayed_fput_work
* in workqueue context, which can race with new queue setup.
+ *
+ * Clear queue->sock under the queue lock so that a sysfs reader in
+ * nvme_tcp_get_address() either observes the NVME_TCP_Q_LIVE bit
+ * cleared or a still valid socket. The queue lock itself is owned
+ * by the controller and only destroyed when the controller is
+ * freed, so readers blocked on it cannot race its teardown.
*/
+ mutex_lock(&queue->queue_lock);
__fput_sync(queue->sock->file);
queue->sock = NULL;
+ mutex_unlock(&queue->queue_lock);
memalloc_noio_restore(noio_flag);
kfree(queue->pdu);
mutex_destroy(&queue->send_mutex);
- mutex_destroy(&queue->queue_lock);
mutex_destroy(&queue->pf_cache_lock);
}
+static int nvme_tcp_recv_tls_alert(struct nvme_tcp_queue *queue,
+ struct msghdr *msg, int len)
+{
+ struct sock *sk = queue->sock->sk;
+ u8 level, description;
+
+ if (len < 2) {
+ dev_err(queue->ctrl->ctrl.device,
+ "queue %d: truncated TLS alert\n",
+ nvme_tcp_queue_id(queue));
+ return -ENOTCONN;
+ }
+
+ /* tls_alert_recv() reads through the iterator */
+ iov_iter_revert(&msg->msg_iter, len);
+ tls_alert_recv(sk, msg, &level, &description);
+
+ if (level == TLS_ALERT_LEVEL_FATAL) {
+ dev_err(queue->ctrl->ctrl.device,
+ "queue %d: fatal TLS alert %u\n",
+ nvme_tcp_queue_id(queue), description);
+ return -ENOTCONN;
+ }
+
+ /* a peer shutting down normally is not worth a warning */
+ if (description == TLS_ALERT_DESC_CLOSE_NOTIFY)
+ dev_dbg(queue->ctrl->ctrl.device,
+ "queue %d: TLS close_notify\n",
+ nvme_tcp_queue_id(queue));
+ else
+ dev_warn(queue->ctrl->ctrl.device,
+ "queue %d: TLS alert %u\n",
+ nvme_tcp_queue_id(queue), description);
+
+ return 0;
+}
+
+/*
+ * Discard a TLS record that arrived in place of the ICResp. A peer may send
+ * a NewSessionTicket as soon as the handshake completes.
+ *
+ * Returns 0 when the record was discarded and the ICResp can be read, or a
+ * negative errno if the connection cannot continue.
+ */
+static int nvme_tcp_skip_tls_record(struct nvme_tcp_queue *queue,
+ struct msghdr *msg, u8 ctype, int len)
+{
+ u8 buf[NVME_TCP_TLS_REC_CHUNK];
+ int ret;
+
+ switch (ctype) {
+ case TLS_RECORD_TYPE_HANDSHAKE:
+ break;
+ case TLS_RECORD_TYPE_ALERT:
+ ret = nvme_tcp_recv_tls_alert(queue, msg, len);
+ return ret ? : -ENOTCONN;
+ default:
+ pr_err("queue %d: unhandled TLS record %u\n",
+ nvme_tcp_queue_id(queue), ctype);
+ return -ENOTCONN;
+ }
+
+ /* the ICResp is in a later record, drain this one first */
+ while (!(msg->msg_flags & MSG_EOR)) {
+ struct kvec iov = {
+ .iov_base = buf,
+ .iov_len = sizeof(buf),
+ };
+ char cbuf[CMSG_LEN(sizeof(char))] = {};
+ /* kTLS fails a control record read without a cmsg buffer */
+ struct msghdr drain = {
+ .msg_control = cbuf,
+ .msg_controllen = sizeof(cbuf),
+ };
+
+ ret = kernel_recvmsg(queue->sock, &drain, &iov, 1, sizeof(buf),
+ 0);
+ if (ret <= 0) {
+ pr_warn("queue %d: failed to drain TLS record: %d\n",
+ nvme_tcp_queue_id(queue), ret);
+ return ret ? : -ECONNRESET;
+ }
+ msg->msg_flags = drain.msg_flags;
+ }
+
+ return 0;
+}
+
+/*
+ * Receive the ICResp. Over TLS, skip the records that a peer may send
+ * ahead of it.
+ *
+ * Returns the length received, or a negative errno.
+ */
+static int nvme_tcp_recv_icresp(struct nvme_tcp_queue *queue,
+ struct nvme_tcp_icresp_pdu *icresp)
+{
+ char cbuf[CMSG_LEN(sizeof(char))];
+ struct msghdr msg;
+ struct kvec iov;
+ u8 ctype;
+ int i, ret;
+
+ for (i = 0; i < NVME_TCP_ICRESP_MAX_RECORDS; i++) {
+ memset(&msg, 0, sizeof(msg));
+ memset(cbuf, 0, sizeof(cbuf));
+ iov.iov_base = icresp;
+ iov.iov_len = sizeof(*icresp);
+ if (nvme_tcp_queue_tls(queue)) {
+ msg.msg_control = cbuf;
+ msg.msg_controllen = sizeof(cbuf);
+ }
+ msg.msg_flags = MSG_WAITALL;
+ ret = kernel_recvmsg(queue->sock, &msg, &iov, 1,
+ iov.iov_len, msg.msg_flags);
+ if (ret <= 0 || !nvme_tcp_queue_tls(queue))
+ return ret;
+
+ ctype = tls_get_record_type(queue->sock->sk,
+ (struct cmsghdr *)cbuf);
+ if (ctype == TLS_RECORD_TYPE_DATA)
+ return ret;
+
+ ret = nvme_tcp_skip_tls_record(queue, &msg, ctype, ret);
+ if (ret < 0)
+ return ret;
+ }
+
+ pr_err("queue %d: too many TLS records before the ICResp\n",
+ nvme_tcp_queue_id(queue));
+ return -EPROTO;
+}
+
static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue)
{
struct nvme_tcp_icreq_pdu *icreq;
struct nvme_tcp_icresp_pdu *icresp;
- char cbuf[CMSG_LEN(sizeof(char))] = {};
- u8 ctype;
struct msghdr msg = {};
struct kvec iov;
bool ctrl_hdgst, ctrl_ddgst;
@@ -1604,16 +1751,7 @@ static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue)
goto free_icresp;
}
- memset(&msg, 0, sizeof(msg));
- iov.iov_base = icresp;
- iov.iov_len = sizeof(*icresp);
- if (nvme_tcp_queue_tls(queue)) {
- msg.msg_control = cbuf;
- msg.msg_controllen = sizeof(cbuf);
- }
- msg.msg_flags = MSG_WAITALL;
- ret = kernel_recvmsg(queue->sock, &msg, &iov, 1,
- iov.iov_len, msg.msg_flags);
+ ret = nvme_tcp_recv_icresp(queue, icresp);
if (ret >= 0 && ret < sizeof(*icresp))
ret = -ECONNRESET;
if (ret < 0) {
@@ -1621,16 +1759,6 @@ static int nvme_tcp_init_connection(struct nvme_tcp_queue *queue)
nvme_tcp_queue_id(queue), ret);
goto free_icresp;
}
- ret = -ENOTCONN;
- if (nvme_tcp_queue_tls(queue)) {
- ctype = tls_get_record_type(queue->sock->sk,
- (struct cmsghdr *)cbuf);
- if (ctype != TLS_RECORD_TYPE_DATA) {
- pr_err("queue %d: unhandled TLS record %d\n",
- nvme_tcp_queue_id(queue), ctype);
- goto free_icresp;
- }
- }
ret = -EINVAL;
if (icresp->hdr.type != nvme_tcp_icresp) {
pr_err("queue %d: bad type returned %d\n",
@@ -1877,7 +2005,6 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
int ret, rcv_pdu_size;
struct file *sock_file;
- mutex_init(&queue->queue_lock);
queue->ctrl = ctrl;
init_llist_head(&queue->req_list);
INIT_LIST_HEAD(&queue->send_list);
@@ -2015,7 +2142,6 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
queue->sock = NULL;
err_destroy_mutex:
mutex_destroy(&queue->send_mutex);
- mutex_destroy(&queue->queue_lock);
mutex_destroy(&queue->pf_cache_lock);
return ret;
}
@@ -2665,6 +2791,9 @@ static void nvme_tcp_stop_ctrl(struct nvme_ctrl *ctrl)
static void nvme_tcp_free_ctrl(struct nvme_ctrl *nctrl)
{
struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl);
+ /* the same count used to size and initialize the queue array */
+ unsigned int nr_queues = NVME_TCP_OPT_NR_QUEUES(nctrl->opts);
+ unsigned int i;
if (list_empty_careful(&ctrl->list))
goto free_ctrl;
@@ -2675,6 +2804,8 @@ static void nvme_tcp_free_ctrl(struct nvme_ctrl *nctrl)
nvmf_free_options(nctrl->opts);
free_ctrl:
+ for (i = 0; i < nr_queues; i++)
+ mutex_destroy(&ctrl->queues[i].queue_lock);
kfree(ctrl->queues);
kfree(ctrl);
}
@@ -2927,17 +3058,18 @@ static int nvme_tcp_get_address(struct nvme_ctrl *ctrl, char *buf, int size)
len = nvmf_get_address(ctrl, buf, size);
- if (!test_bit(NVME_TCP_Q_LIVE, &queue->flags))
- return len;
-
mutex_lock(&queue->queue_lock);
- ret = kernel_getsockname(queue->sock, (struct sockaddr *)&src_addr);
- if (ret > 0) {
- if (len > 0)
- len--; /* strip trailing newline */
- len += scnprintf(buf + len, size - len, "%ssrc_addr=%pISc\n",
- (len) ? "," : "", &src_addr);
+ if (test_bit(NVME_TCP_Q_LIVE, &queue->flags)) {
+ ret = kernel_getsockname(queue->sock,
+ (struct sockaddr *)&src_addr);
+ if (ret > 0) {
+ if (len > 0)
+ len--; /* strip trailing newline */
+ len += scnprintf(buf + len, size - len,
+ "%ssrc_addr=%pISc\n",
+ (len) ? "," : "", &src_addr);
+ }
}
mutex_unlock(&queue->queue_lock);
@@ -3003,7 +3135,7 @@ static struct nvme_tcp_ctrl *nvme_tcp_alloc_ctrl(struct device *dev,
struct nvmf_ctrl_options *opts)
{
struct nvme_tcp_ctrl *ctrl;
- int ret;
+ int ret, i;
ctrl = kzalloc_obj(*ctrl);
if (!ctrl)
@@ -3014,8 +3146,7 @@ static struct nvme_tcp_ctrl *nvme_tcp_alloc_ctrl(struct device *dev,
*/
context_unsafe(INIT_LIST_HEAD(&ctrl->list));
ctrl->ctrl.opts = opts;
- ctrl->ctrl.queue_count = opts->nr_io_queues + opts->nr_write_queues +
- opts->nr_poll_queues + 1;
+ ctrl->ctrl.queue_count = NVME_TCP_OPT_NR_QUEUES(opts);
ctrl->ctrl.sqsize = opts->queue_size - 1;
ctrl->ctrl.kato = opts->kato;
@@ -3073,6 +3204,14 @@ static struct nvme_tcp_ctrl *nvme_tcp_alloc_ctrl(struct device *dev,
goto out_free_ctrl;
}
+ /*
+ * The queue locks are owned by the controller, not the queues:
+ * they outlive the per-queue connect/disconnect cycles so that
+ * sysfs readers can never race their destruction.
+ */
+ for (i = 0; i < ctrl->ctrl.queue_count; i++)
+ mutex_init(&ctrl->queues[i].queue_lock);
+
ret = nvme_init_ctrl(&ctrl->ctrl, dev, &nvme_tcp_ctrl_ops, 0);
if (ret)
goto out_kfree_queues;
diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 7764a3c0195c..3704493dbec9 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -543,6 +543,15 @@ static void nvmet_execute_get_log_page_ana(struct nvmet_req *req)
u16 ngrps = 0;
u16 status;
+ /*
+ * The header and one descriptor per enabled group are the only bytes
+ * written below; zero the whole transfer first so the unwritten tail
+ * does not leak stale kernel memory to the host.
+ */
+ status = nvmet_zero_sgl(req, 0, req->transfer_len);
+ if (status)
+ goto out;
+
status = NVME_SC_INTERNAL;
desc = kmalloc_flex(*desc, nsids, NVMET_MAX_NAMESPACES);
if (!desc)
@@ -1225,8 +1234,13 @@ u16 nvmet_set_feat_kato(struct nvmet_req *req)
{
u32 val32 = le32_to_cpu(req->cmd->common.cdw11);
+ if (val32 > NVMET_MAX_KATO * MSEC_PER_SEC) {
+ req->error_loc = offsetof(struct nvme_common_command, cdw11);
+ return NVME_SC_KA_TIMEOUT_INVALID | NVME_STATUS_DNR;
+ }
+
nvmet_stop_keep_alive_timer(req->sq->ctrl);
- req->sq->ctrl->kato = DIV_ROUND_UP(val32, 1000);
+ req->sq->ctrl->kato = DIV_ROUND_UP_ULL(val32, MSEC_PER_SEC);
nvmet_start_keep_alive_timer(req->sq->ctrl);
nvmet_set_result(req, req->sq->ctrl->kato);
diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index 83b86180c50f..61107bcfc895 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -150,7 +150,7 @@ static ssize_t nvmet_addr_traddr_store(struct config_item *item,
if (nvmet_is_port_enabled(port, __func__))
return -EACCES;
- if (sscanf(page, "%s\n", port->disc_addr.traddr) != 1)
+ if (sscanf(page, "%255s", port->disc_addr.traddr) != 1)
return -EINVAL;
return count;
}
@@ -242,7 +242,7 @@ static ssize_t nvmet_addr_trsvcid_store(struct config_item *item,
if (nvmet_is_port_enabled(port, __func__))
return -EACCES;
- if (sscanf(page, "%s\n", port->disc_addr.trsvcid) != 1)
+ if (sscanf(page, "%31s", port->disc_addr.trsvcid) != 1)
return -EINVAL;
return count;
}
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 07a74c2e1f58..0f5e94bf12bd 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1612,6 +1612,12 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args)
u8 dhchap_status;
int ret;
+ if (kato > NVMET_MAX_KATO * MSEC_PER_SEC) {
+ args->status = NVME_SC_KA_TIMEOUT_INVALID | NVME_STATUS_DNR;
+ args->error_loc = offsetof(struct nvmf_connect_command, kato);
+ return NULL;
+ }
+
args->status = NVME_SC_CONNECT_INVALID_PARAM | NVME_STATUS_DNR;
subsys = nvmet_find_get_subsys(args->port, args->subsysnqn);
if (!subsys) {
@@ -1678,7 +1684,7 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args)
kato = NVMET_DISC_KATO_MS;
/* keep-alive timeout in seconds */
- ctrl->kato = DIV_ROUND_UP(kato, 1000);
+ ctrl->kato = DIV_ROUND_UP_ULL(kato, MSEC_PER_SEC);
ctrl->err_counter = 0;
spin_lock_init(&ctrl->error_lock);
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 2032db380f99..215315ebe42d 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -7,6 +7,7 @@
#define _NVMET_H
#include <linux/dma-mapping.h>
+#include <linux/jiffies.h>
#include <linux/types.h>
#include <linux/device.h>
#include <linux/kref.h>
@@ -702,6 +703,13 @@ void nvmet_add_async_event(struct nvmet_ctrl *ctrl, u8 event_type,
#define NVMET_KAS 10
#define NVMET_DISC_KATO_MS 120000
+/*
+ * Largest keep-alive timeout in seconds that can be reported back in
+ * milliseconds and converted to jiffies without overflowing.
+ */
+#define NVMET_MAX_KATO \
+ min3(U32_MAX / MSEC_PER_SEC, U32_MAX / HZ, MAX_JIFFY_OFFSET / HZ)
+
int __init nvmet_init_configfs(void);
void __exit nvmet_exit_configfs(void);
diff --git a/drivers/pci/controller/cadence/pcie-cadence-plat.c b/drivers/pci/controller/cadence/pcie-cadence-plat.c
index a1ea24fc3b63..4a24302cc5b2 100644
--- a/drivers/pci/controller/cadence/pcie-cadence-plat.c
+++ b/drivers/pci/controller/cadence/pcie-cadence-plat.c
@@ -14,14 +14,6 @@
#define CDNS_PLAT_CPU_TO_BUS_ADDR 0x0FFFFFFF
-/**
- * struct cdns_plat_pcie - private data for this PCIe platform driver
- * @pcie: Cadence PCIe controller
- */
-struct cdns_plat_pcie {
- struct cdns_pcie *pcie;
-};
-
static const struct of_device_id cdns_plat_pcie_of_match[];
static u64 cdns_plat_cpu_addr_fixup(struct cdns_pcie *pcie, u64 cpu_addr)
@@ -36,12 +28,11 @@ static const struct cdns_pcie_ops cdns_plat_ops = {
static int cdns_plat_pcie_probe(struct platform_device *pdev)
{
const struct cdns_plat_pcie_of_data *data;
- struct cdns_plat_pcie *cdns_plat_pcie;
struct device *dev = &pdev->dev;
struct pci_host_bridge *bridge;
+ struct cdns_pcie *pcie;
struct cdns_pcie_ep *ep;
struct cdns_pcie_rc *rc;
- int phy_count;
bool is_rc;
int ret;
@@ -52,11 +43,6 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
is_rc = data->is_rc;
pr_debug(" Started %s with is_rc: %d\n", __func__, is_rc);
- cdns_plat_pcie = devm_kzalloc(dev, sizeof(*cdns_plat_pcie), GFP_KERNEL);
- if (!cdns_plat_pcie)
- return -ENOMEM;
-
- platform_set_drvdata(pdev, cdns_plat_pcie);
if (is_rc) {
if (!IS_ENABLED(CONFIG_PCIE_CADENCE_PLAT_HOST))
return -ENODEV;
@@ -68,9 +54,9 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
rc = pci_host_bridge_priv(bridge);
rc->pcie.dev = dev;
rc->pcie.ops = &cdns_plat_ops;
- cdns_plat_pcie->pcie = &rc->pcie;
+ pcie = &rc->pcie;
- ret = cdns_pcie_init_phy(dev, cdns_plat_pcie->pcie);
+ ret = cdns_pcie_init_phy(dev, pcie);
if (ret) {
dev_err(dev, "failed to init phy\n");
return ret;
@@ -95,9 +81,9 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
ep->pcie.dev = dev;
ep->pcie.ops = &cdns_plat_ops;
- cdns_plat_pcie->pcie = &ep->pcie;
+ pcie = &ep->pcie;
- ret = cdns_pcie_init_phy(dev, cdns_plat_pcie->pcie);
+ ret = cdns_pcie_init_phy(dev, pcie);
if (ret) {
dev_err(dev, "failed to init phy\n");
return ret;
@@ -115,18 +101,17 @@ static int cdns_plat_pcie_probe(struct platform_device *pdev)
goto err_init;
}
+ platform_set_drvdata(pdev, pcie);
+
return 0;
err_init:
err_get_sync:
pm_runtime_put_sync(dev);
pm_runtime_disable(dev);
- cdns_pcie_disable_phy(cdns_plat_pcie->pcie);
- phy_count = cdns_plat_pcie->pcie->phy_count;
- while (phy_count--)
- device_link_del(cdns_plat_pcie->pcie->link[phy_count]);
+ cdns_pcie_disable_phy(pcie);
- return 0;
+ return ret;
}
static void cdns_plat_pcie_shutdown(struct platform_device *pdev)
diff --git a/drivers/pci/controller/cadence/pcie-cadence.c b/drivers/pci/controller/cadence/pcie-cadence.c
index a1eada56edba..0ac980249941 100644
--- a/drivers/pci/controller/cadence/pcie-cadence.c
+++ b/drivers/pci/controller/cadence/pcie-cadence.c
@@ -222,7 +222,6 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie)
struct device_node *np = dev->of_node;
int phy_count;
struct phy **phy;
- struct device_link **link;
int i;
int ret;
const char *name;
@@ -238,10 +237,6 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie)
if (!phy)
return -ENOMEM;
- link = devm_kcalloc(dev, phy_count, sizeof(*link), GFP_KERNEL);
- if (!link)
- return -ENOMEM;
-
for (i = 0; i < phy_count; i++) {
of_property_read_string_index(np, "phy-names", i, &name);
phy[i] = devm_phy_get(dev, name);
@@ -249,17 +244,10 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie)
ret = PTR_ERR(phy[i]);
goto err_phy;
}
- link[i] = device_link_add(dev, &phy[i]->dev, DL_FLAG_STATELESS);
- if (!link[i]) {
- devm_phy_put(dev, phy[i]);
- ret = -EINVAL;
- goto err_phy;
- }
}
pcie->phy_count = phy_count;
pcie->phy = phy;
- pcie->link = link;
ret = cdns_pcie_enable_phy(pcie);
if (ret)
@@ -268,10 +256,8 @@ int cdns_pcie_init_phy(struct device *dev, struct cdns_pcie *pcie)
return 0;
err_phy:
- while (--i >= 0) {
- device_link_del(link[i]);
+ while (--i >= 0)
devm_phy_put(dev, phy[i]);
- }
return ret;
}
diff --git a/drivers/pci/controller/cadence/pcie-cadence.h b/drivers/pci/controller/cadence/pcie-cadence.h
index b4d51adefea5..4a96e1869e53 100644
--- a/drivers/pci/controller/cadence/pcie-cadence.h
+++ b/drivers/pci/controller/cadence/pcie-cadence.h
@@ -260,7 +260,6 @@ struct cdns_plat_pcie_of_data {
* @is_hpa: indicates if the architecture is HPA
* @phy_count: number of supported PHY devices
* @phy: list of pointers to specific PHY control blocks
- * @link: list of pointers to corresponding device link representations
* @ops: Platform-specific ops to control various inputs from Cadence PCIe
* wrapper
* @cdns_pcie_reg_offsets: Register bank offsets for different SoC
@@ -276,7 +275,6 @@ struct cdns_pcie {
bool is_hpa;
int phy_count;
struct phy **phy;
- struct device_link **link;
const struct cdns_pcie_ops *ops;
const struct cdns_plat_pcie_of_data *cdns_pcie_reg_offsets;
int max_link_speed;
diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
index 593388f29bdd..9297d0e607e3 100644
--- a/drivers/pci/controller/dwc/pcie-designware.c
+++ b/drivers/pci/controller/dwc/pcie-designware.c
@@ -577,6 +577,8 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci,
/* The data-less messages only for now */
val |= PCIE_ATU_INHIBIT_PAYLOAD | atu->code;
}
+ if (dw_pcie_ver_is_ge(pci, 460A))
+ val |= PCIE_ATU_DMA_BYPASS;
dw_pcie_writel_atu_ob(pci, atu->index, PCIE_ATU_REGION_CTRL2, val);
/*
diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
index 0735ae940924..668a5ba1c6f5 100644
--- a/drivers/pci/controller/dwc/pcie-designware.h
+++ b/drivers/pci/controller/dwc/pcie-designware.h
@@ -201,6 +201,7 @@
#define PCIE_ATU_ENABLE BIT(31)
#define PCIE_ATU_BAR_MODE_ENABLE BIT(30)
#define PCIE_ATU_CFG_SHIFT_MODE_ENABLE BIT(28)
+#define PCIE_ATU_DMA_BYPASS BIT(27)
#define PCIE_ATU_INHIBIT_PAYLOAD BIT(22)
#define PCIE_ATU_FUNC_NUM_MATCH_EN BIT(19)
#define PCIE_ATU_LOWER_BASE 0x008
diff --git a/drivers/platform/chrome/cros_ec_ishtp.c b/drivers/platform/chrome/cros_ec_ishtp.c
index 3766cef81fe8..b21a27ad3762 100644
--- a/drivers/platform/chrome/cros_ec_ishtp.c
+++ b/drivers/platform/chrome/cros_ec_ishtp.c
@@ -369,7 +369,7 @@ static void ish_event_cb(struct ishtp_cl_device *cl_device)
* @cros_ish_cl: ISHTP client instance
* @reset: true if called from reset handler
*
- * This function complete the initializtion of the client.
+ * This function complete the initialization of the client.
*
* Return: 0 for success, negative error code for failure.
*/
diff --git a/drivers/platform/chrome/cros_usbpd_notify.c b/drivers/platform/chrome/cros_usbpd_notify.c
index d842c05f4db0..25da31ec1996 100644
--- a/drivers/platform/chrome/cros_usbpd_notify.c
+++ b/drivers/platform/chrome/cros_usbpd_notify.c
@@ -64,7 +64,7 @@ static void cros_usbpd_get_event_and_notify(struct device *dev,
/*
* We still send a 0 event out to older devices which don't
- * have the updated device heirarchy.
+ * have the updated device hierarchy.
*/
if (!ec_dev) {
dev_dbg(dev,
@@ -110,26 +110,29 @@ static int cros_usbpd_notify_probe_acpi(struct platform_device *pdev)
if (!pdnotify)
return -ENOMEM;
- /* Get the EC device pointer needed to talk to the EC. */
- ec_dev = dev_get_drvdata(dev->parent);
- if (!ec_dev) {
- /*
- * We continue even for older devices which don't have the
- * correct device heirarchy, namely, GOOG0003 is a child
- * of GOOG0004. If GOOG0003 is a child of GOOG0004 and we
- * can't get a pointer to the Chrome EC device, defer the
- * probe function.
- */
- parent_fwnode = fwnode_get_parent(dev->fwnode);
- if (parent_fwnode) {
- parent_adev = to_acpi_device_node(parent_fwnode);
- if (parent_adev &&
- acpi_dev_hid_match(parent_adev, CREC_DRV_NAME)) {
- return -EPROBE_DEFER;
- }
+ /*
+ * Get the EC device pointer needed to talk to the EC. The parent's
+ * driver data is only a struct cros_ec_device when GOOG0003 is a
+ * child of GOOG0004. On older devices without that hierarchy the
+ * parent may be bound to an unrelated driver (e.g. the ACPI EC
+ * driver for PNP0C09), so don't touch its driver data and continue
+ * without an EC pointer. If GOOG0003 is a child of GOOG0004 and we
+ * can't get a pointer to the Chrome EC device yet, defer the probe.
+ */
+ ec_dev = NULL;
+ parent_fwnode = fwnode_get_parent(dev->fwnode);
+ parent_adev = to_acpi_device_node(parent_fwnode);
+ if (parent_adev && acpi_dev_hid_match(parent_adev, CREC_DRV_NAME)) {
+ ec_dev = dev_get_drvdata(dev->parent);
+ if (!ec_dev) {
+ fwnode_handle_put(parent_fwnode);
+ return -EPROBE_DEFER;
}
- dev_warn(dev, "Couldn't get Chrome EC device pointer.\n");
}
+ fwnode_handle_put(parent_fwnode);
+
+ if (!ec_dev)
+ dev_warn(dev, "Couldn't get Chrome EC device pointer.\n");
pdnotify->dev = dev;
pdnotify->ec = ec_dev;
diff --git a/drivers/ptp/ptp_sysfs.c b/drivers/ptp/ptp_sysfs.c
index dc398c6b7528..7d09256aca72 100644
--- a/drivers/ptp/ptp_sysfs.c
+++ b/drivers/ptp/ptp_sysfs.c
@@ -80,13 +80,17 @@ static ssize_t extts_fifo_show(struct device *dev,
size_t qcnt;
int cnt = 0;
+ spin_lock_irqsave(&ptp->tsevqs_lock, flags);
cnt = list_count_nodes(&ptp->tsevqs);
- if (cnt <= 0)
+ if (cnt <= 0) {
+ spin_unlock_irqrestore(&ptp->tsevqs_lock, flags);
goto out;
+ }
/* The sysfs fifo will always draw from the fist queue */
queue = list_first_entry(&ptp->tsevqs, struct timestamp_event_queue,
qlist);
+ spin_unlock_irqrestore(&ptp->tsevqs_lock, flags);
memset(&event, 0, sizeof(event));
spin_lock_irqsave(&queue->lock, flags);
diff --git a/drivers/scsi/aic7xxx/aic79xx_core.c b/drivers/scsi/aic7xxx/aic79xx_core.c
index 3602a1fc40e4..4abc7948187f 100644
--- a/drivers/scsi/aic7xxx/aic79xx_core.c
+++ b/drivers/scsi/aic7xxx/aic79xx_core.c
@@ -8942,8 +8942,10 @@ ahd_handle_scsi_status(struct ahd_softc *ahd, struct scb *scb)
}
#endif
- if (ahd_perform_autosense(scb) == 0)
+ if (ahd_perform_autosense(scb) == 0) {
+ ahd_done(ahd, scb);
break;
+ }
ahd_compile_devinfo(&devinfo, SCB_GET_OUR_ID(scb),
SCB_GET_TARGET(ahd, scb),
diff --git a/drivers/scsi/aic7xxx/aic79xx_osm.h b/drivers/scsi/aic7xxx/aic79xx_osm.h
index 793fe19993a9..3bcfcfe18f08 100644
--- a/drivers/scsi/aic7xxx/aic79xx_osm.h
+++ b/drivers/scsi/aic7xxx/aic79xx_osm.h
@@ -499,6 +499,8 @@ int ahd_linux_show_info(struct seq_file *,struct Scsi_Host *);
static inline
void ahd_cmd_set_transaction_status(struct scsi_cmnd *cmd, uint32_t status)
{
+ if (!cmd)
+ return;
cmd->result &= ~(CAM_STATUS_MASK << 16);
cmd->result |= status << 16;
}
@@ -512,6 +514,8 @@ void ahd_set_transaction_status(struct scb *scb, uint32_t status)
static inline
void ahd_cmd_set_scsi_status(struct scsi_cmnd *cmd, uint32_t status)
{
+ if (!cmd)
+ return;
cmd->result &= ~0xFFFF;
cmd->result |= status;
}
@@ -525,6 +529,8 @@ void ahd_set_scsi_status(struct scb *scb, uint32_t status)
static inline
uint32_t ahd_cmd_get_transaction_status(struct scsi_cmnd *cmd)
{
+ if (!cmd)
+ return CAM_REQ_INPROG;
return ((cmd->result >> 16) & CAM_STATUS_MASK);
}
@@ -537,6 +543,8 @@ uint32_t ahd_get_transaction_status(struct scb *scb)
static inline
uint32_t ahd_cmd_get_scsi_status(struct scsi_cmnd *cmd)
{
+ if (!cmd)
+ return SAM_STAT_GOOD;
return (cmd->result & 0xFFFF);
}
@@ -564,13 +572,16 @@ u_long ahd_get_transfer_length(struct scb *scb)
static inline
int ahd_get_transfer_dir(struct scb *scb)
{
+ if (!scb->io_ctx)
+ return DMA_NONE;
return (scb->io_ctx->sc_data_direction);
}
static inline
void ahd_set_residual(struct scb *scb, u_long resid)
{
- scsi_set_resid(scb->io_ctx, resid);
+ if (scb->io_ctx)
+ scsi_set_resid(scb->io_ctx, resid);
}
static inline
@@ -582,6 +593,8 @@ void ahd_set_sense_residual(struct scb *scb, u_long resid)
static inline
u_long ahd_get_residual(struct scb *scb)
{
+ if (!scb->io_ctx)
+ return 0;
return scsi_get_resid(scb->io_ctx);
}
@@ -595,11 +608,12 @@ static inline
int ahd_perform_autosense(struct scb *scb)
{
/*
- * We always perform autosense in Linux.
- * On other platforms this is set on a
- * per-transaction basis.
+ * We always perform autosense in Linux, except for the device
+ * reset SCB, which has no command to return sense data to.
*/
- return (1);
+ if (!scb->io_ctx)
+ return 0;
+ return 1;
}
static inline uint32_t
@@ -629,6 +643,8 @@ void ahd_platform_freeze_devq(struct ahd_softc *ahd, struct scb *scb);
static inline void
ahd_freeze_scb(struct scb *scb)
{
+ if (!scb->io_ctx)
+ return;
if ((scb->io_ctx->result & (CAM_DEV_QFRZN << 16)) == 0) {
scb->io_ctx->result |= CAM_DEV_QFRZN << 16;
scb->platform_data->dev->qfrozen++;
diff --git a/drivers/scsi/aic7xxx/aic7xxx_core.c b/drivers/scsi/aic7xxx/aic7xxx_core.c
index b9761f9f0bbc..8ad82171e925 100644
--- a/drivers/scsi/aic7xxx/aic7xxx_core.c
+++ b/drivers/scsi/aic7xxx/aic7xxx_core.c
@@ -5266,9 +5266,13 @@ ahc_init(struct ahc_softc *ahc)
/*
* Assume we have a board at this stage and it has been reset.
+ * PCI controllers keep the host ID ahc_pci_config() chose.
*/
- if ((ahc->flags & AHC_USEDEFAULTS) != 0)
- ahc->our_id = ahc->our_id_b = 7;
+ if ((ahc->flags & AHC_USEDEFAULTS) != 0) {
+ if ((ahc->chip & AHC_PCI) == 0)
+ ahc->our_id = 7;
+ ahc->our_id_b = 7;
+ }
/*
* Default to allowing initiator operations.
diff --git a/drivers/scsi/aic7xxx/aic7xxx_pci.c b/drivers/scsi/aic7xxx/aic7xxx_pci.c
index 2d4c85426dc3..5a04c7d55e4b 100644
--- a/drivers/scsi/aic7xxx/aic7xxx_pci.c
+++ b/drivers/scsi/aic7xxx/aic7xxx_pci.c
@@ -701,6 +701,21 @@ ahc_find_pci_device(ahc_dev_softc_t pci)
return (NULL);
}
+/*
+ * The SGI O2 firmware runs its onboard controllers, in slots 1 and 2 of the
+ * root bus, without the Adaptec BIOS and at host ID 0. It leaves no settings
+ * in scratch RAM, which may still hold power-on contents with bad parity.
+ */
+static bool
+ahc_pci_sgi_o2_onboard(struct ahc_softc *ahc)
+{
+ int slot = ahc_get_pci_slot(ahc->dev_softc);
+
+ return IS_ENABLED(CONFIG_SGI_IP32) &&
+ ahc_get_pci_bus(ahc->dev_softc) == 0 &&
+ (slot == 1 || slot == 2);
+}
+
int
ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry)
{
@@ -782,6 +797,9 @@ ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry)
ahc->bus_intr = ahc_pci_intr;
ahc->bus_chip_init = ahc_pci_chip_init;
+ if (ahc_pci_sgi_o2_onboard(ahc))
+ ahc->flags |= AHC_NO_BIOS_INIT;
+
/* Remember how the card was setup in case there is no SEEPROM */
if ((ahc_inb(ahc, HCNTRL) & POWRDN) == 0) {
ahc_pause(ahc);
@@ -898,7 +916,7 @@ ahc_pci_config(struct ahc_softc *ahc, const struct ahc_pci_identity *entry)
* Assume only one connector and always turn
* on termination.
*/
- our_id = 0x07;
+ our_id = ahc_pci_sgi_o2_onboard(ahc) ? 0 : 0x07;
sxfrctl1 = STPWEN;
}
ahc_outb(ahc, SCSICONF, our_id|ENSPCHK|RESET_SCSI);
diff --git a/drivers/scsi/scsi_bsg.c b/drivers/scsi/scsi_bsg.c
index 5eec248a77a6..e4a65d7a2374 100644
--- a/drivers/scsi/scsi_bsg.c
+++ b/drivers/scsi/scsi_bsg.c
@@ -41,7 +41,7 @@ static void scsi_bsg_uring_task_cb(struct io_tw_req tw_req, io_tw_token_t tw)
u8 sense_len_wr = 0;
if (pdu->bio)
- blk_rq_unmap_user(pdu->bio);
+ ret = blk_rq_unmap_user(pdu->bio);
if (scsi_status_is_check_condition(scmd->result)) {
driver_status = DRIVER_SENSE;
@@ -199,7 +199,7 @@ static int scsi_bsg_sg_io_fn(struct request_queue *q, struct sg_io_v4 *hdr,
struct scsi_cmnd *scmd;
struct request *rq;
struct bio *bio;
- int ret;
+ int ret, unmap_ret;
if (hdr->protocol != BSG_PROTOCOL_SCSI ||
hdr->subprotocol != BSG_SUB_PROTOCOL_SCSI_CMD)
@@ -273,7 +273,9 @@ static int scsi_bsg_sg_io_fn(struct request_queue *q, struct sg_io_v4 *hdr,
else
hdr->dout_resid = scmd->resid_len;
- blk_rq_unmap_user(bio);
+ unmap_ret = blk_rq_unmap_user(bio);
+ if (!ret)
+ ret = unmap_ret;
out_put_request:
blk_mq_free_request(rq);
diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index 9db2201aa553..c5d5d6a91fee 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -349,6 +349,19 @@ int core_enable_device_list_for_node(
new->attach_count++;
mutex_lock(&nacl->lun_entry_mutex);
+ /*
+ * Do not attach a new dev entry to an se_lun that is being torn down.
+ * lun_shutdown is set by core_tpg_remove_lun() before its
+ * core_clear_lun_from_tpg() sweep; checking it here under the same
+ * nacl->lun_entry_mutex the sweep takes guarantees that once the sweep
+ * has passed this ACL no dev entry can be left dangling on the
+ * se_lun->lun_deve_list that is about to be freed.
+ */
+ if (lun->lun_shutdown) {
+ mutex_unlock(&nacl->lun_entry_mutex);
+ ret = -ENODEV;
+ goto free_stats;
+ }
orig = target_nacl_find_deve(nacl, mapped_lun);
if (orig && orig->se_lun) {
struct se_lun *orig_lun = orig->se_lun;
@@ -375,12 +388,12 @@ int core_enable_device_list_for_node(
new->se_lun_acl = lun_acl;
hlist_del_rcu(&orig->link);
hlist_add_head_rcu(&new->link, &nacl->lun_entry_hlist);
- mutex_unlock(&nacl->lun_entry_mutex);
spin_lock(&lun->lun_deve_lock);
list_del(&orig->lun_link);
list_add_tail(&new->lun_link, &lun->lun_deve_list);
spin_unlock(&lun->lun_deve_lock);
+ mutex_unlock(&nacl->lun_entry_mutex);
kref_put(&orig->pr_kref, target_pr_kref_release);
wait_for_completion(&orig->pr_comp);
@@ -393,11 +406,10 @@ int core_enable_device_list_for_node(
new->se_lun = lun;
new->se_lun_acl = lun_acl;
hlist_add_head_rcu(&new->link, &nacl->lun_entry_hlist);
- mutex_unlock(&nacl->lun_entry_mutex);
-
spin_lock(&lun->lun_deve_lock);
list_add_tail(&new->lun_link, &lun->lun_deve_list);
spin_unlock(&lun->lun_deve_lock);
+ mutex_unlock(&nacl->lun_entry_mutex);
target_luns_data_has_changed(nacl, new, true);
return 0;
diff --git a/drivers/target/target_core_fabric_lib.c b/drivers/target/target_core_fabric_lib.c
index 2853b95b2c59..ad024009df56 100644
--- a/drivers/target/target_core_fabric_lib.c
+++ b/drivers/target/target_core_fabric_lib.c
@@ -371,6 +371,16 @@ static bool iscsi_parse_pr_out_transport_id(
pr_err("Missing ISID in iSCSI Initiator port TransportID\n");
return false;
}
+ /*
+ * The registration code reads the ISID as a C string,
+ * and its first 8 bytes with get_unaligned_be64(), so
+ * require 8 bytes after the separator and a NUL inside
+ * the rest of the descriptor.
+ */
+ if (buf + tid_len - p < 8 || !memchr(p, '\0', buf + tid_len - p)) {
+ pr_err("Incomplete or unterminated ISID in iSCSI Initiator port TransportID\n");
+ return false;
+ }
*port_nexus_ptr = p;
/*
* Go ahead and do the lower case conversion of the received
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index ccff9f6e4826..e615e5e30427 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -266,6 +266,15 @@ struct se_node_acl *core_tpg_check_initiator_node_acl(
kref_get(&acl->acl_kref);
acl->dynamic_node_acl = 1;
+ /*
+ * Publish the dynamic NodeACL on tpg->acl_node_list before creating
+ * its demo-mode MappedLUN dev entries below, so that a concurrent LUN
+ * teardown sweep (core_clear_lun_from_tpg(), which walks acl_node_list)
+ * can observe and disable those dev entries. Otherwise a dev entry can
+ * be left on an se_lun->lun_deve_list that is about to be freed.
+ */
+ target_add_node_acl(acl);
+
/*
* Here we only create demo-mode MappedLUNs from the active
* TPG LUNs if the fabric is not explicitly asking for
@@ -275,7 +284,6 @@ struct se_node_acl *core_tpg_check_initiator_node_acl(
(tpg->se_tpg_tfo->tpg_check_demo_mode_login_only(tpg) != 1))
core_tpg_add_node_to_devs(acl, tpg, NULL);
- target_add_node_acl(acl);
return acl;
}
EXPORT_SYMBOL(core_tpg_check_initiator_node_acl);
diff --git a/drivers/target/target_core_user.c b/drivers/target/target_core_user.c
index edc2afd5f4ee..839d5b2e7250 100644
--- a/drivers/target/target_core_user.c
+++ b/drivers/target/target_core_user.c
@@ -2524,6 +2524,11 @@ static int tcmu_set_data_pages_per_blk(struct tcmu_dev *udev, substring_t *arg)
return ret;
}
+ if (val <= 0) {
+ pr_err("Invalid data_pages_per_blk %d.\n", val);
+ return -EINVAL;
+ }
+
if (val > TCMU_MBS_TO_PAGES(udev->data_area_mb)) {
pr_err("Invalid data_pages_per_blk %d: greater than max_data_area_mb %d -> %zd pages).\n",
val, udev->data_area_mb,
diff --git a/drivers/thermal/intel/int340x_thermal/int3403_thermal.c b/drivers/thermal/intel/int340x_thermal/int3403_thermal.c
index 264c9bc8e645..2262de1e14d9 100644
--- a/drivers/thermal/intel/int340x_thermal/int3403_thermal.c
+++ b/drivers/thermal/intel/int340x_thermal/int3403_thermal.c
@@ -217,8 +217,7 @@ static int int3403_add(struct platform_device *pdev)
}
- status = acpi_evaluate_integer(priv->adev->handle, "_TMP",
- NULL, &tmp);
+ status = acpi_evaluate_integer_exclusive(priv->adev->handle, "_TMP", NULL, &tmp);
if (ACPI_FAILURE(status)) {
status = acpi_evaluate_integer(priv->adev->handle, "PTYP",
NULL, &priv->type);
diff --git a/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.c b/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.c
index 965700ee6962..9451fea1f65c 100644
--- a/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.c
+++ b/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.c
@@ -18,7 +18,11 @@ static int int340x_thermal_get_zone_temp(struct thermal_zone_device *zone,
unsigned long long tmp;
acpi_status status;
- status = acpi_evaluate_integer(d->adev->handle, "_TMP", NULL, &tmp);
+ /*
+ * On some ThinkPads _TMP switches the EC page selector without
+ * taking the mutex the battery methods use.
+ */
+ status = acpi_evaluate_integer_exclusive(d->adev->handle, "_TMP", NULL, &tmp);
if (ACPI_FAILURE(status))
return -EIO;
diff --git a/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c b/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c
index b0284c2e2e74..9bdb65f54b6a 100644
--- a/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c
+++ b/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c
@@ -287,7 +287,7 @@ int proc_thermal_add(struct device *dev, struct proc_thermal_device *proc_priv)
if (ret)
return ret;
- status = acpi_evaluate_integer(adev->handle, "_TMP", NULL, &tmp);
+ status = acpi_evaluate_integer_exclusive(adev->handle, "_TMP", NULL, &tmp);
if (ACPI_FAILURE(status)) {
/* there is no _TMP method, add local method */
if (intel_tcc_get_tjmax(-1) > 0)
diff --git a/drivers/vdpa/solidrun/snet_ctrl.c b/drivers/vdpa/solidrun/snet_ctrl.c
index e284c3a06717..09afa0487fd6 100644
--- a/drivers/vdpa/solidrun/snet_ctrl.c
+++ b/drivers/vdpa/solidrun/snet_ctrl.c
@@ -291,7 +291,6 @@ static int snet_send_ctrl_msg(struct snet *snet, u16 opcode, u16 vq_idx)
/* Check for errors */
val = snet_read_ctrl(regs);
- ret = SNET_VAL_TO_ERR(val);
/* Clear the chunk ready bit */
val &= ~SNET_CTRL_CHUNK_RDY_MASK;
@@ -301,6 +300,8 @@ static int snet_send_ctrl_msg(struct snet *snet, u16 opcode, u16 vq_idx)
if (ret)
SNET_WARN(pdev, "Timeout waiting for DPU to complete a control command, err %d\n",
ret);
+ else
+ ret = SNET_VAL_TO_ERR(val);
exit:
mutex_unlock(&snet->ctrl_lock);
diff --git a/drivers/vdpa/vdpa_user/iova_domain.c b/drivers/vdpa/vdpa_user/iova_domain.c
index b6c958224b7c..28a977248a0d 100644
--- a/drivers/vdpa/vdpa_user/iova_domain.c
+++ b/drivers/vdpa/vdpa_user/iova_domain.c
@@ -76,6 +76,10 @@ int vduse_domain_set_map(struct vduse_iova_domain *domain,
for (map = vhost_iotlb_itree_first(iotlb, start, last); map;
map = vhost_iotlb_itree_next(map, start, last)) {
map_file = (struct vdpa_map_file *)map->opaque;
+ if (!map_file || map_file->file == domain->file) {
+ ret = -EINVAL;
+ goto err;
+ }
ret = vduse_iotlb_add_range(domain, map->start, map->last,
map->addr, map->perm,
map_file->file,
@@ -226,6 +230,7 @@ vduse_domain_get_coherent_page(struct vduse_iova_domain *domain, u64 iova)
{
u64 start = iova & PAGE_MASK;
u64 last = start + PAGE_SIZE - 1;
+ struct vdpa_map_file *map_file;
struct vhost_iotlb_map *map;
struct page *page = NULL;
@@ -234,6 +239,10 @@ vduse_domain_get_coherent_page(struct vduse_iova_domain *domain, u64 iova)
if (!map)
goto out;
+ map_file = (struct vdpa_map_file *)map->opaque;
+ if (!map_file || map_file->file != domain->file)
+ goto out;
+
page = pfn_to_page((map->addr + iova - map->start) >> PAGE_SHIFT);
get_page(page);
out:
@@ -579,7 +588,7 @@ static int vduse_domain_mmap(struct file *file, struct vm_area_struct *vma)
{
struct vduse_iova_domain *domain = file->private_data;
- vm_flags_set(vma, VM_DONTDUMP | VM_DONTEXPAND);
+ vm_flags_set(vma, VM_IO | VM_DONTDUMP | VM_DONTEXPAND);
vma->vm_private_data = domain;
vma->vm_ops = &vduse_domain_mmap_ops;
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 4f8c0260bc9e..ddc9bea190de 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -937,7 +937,12 @@ vhost_scsi_copy_iov_to_sgl(struct vhost_scsi_cmd *cmd, struct iov_iter *iter,
}
for_each_sgtable_sg(sg_table, sg, i) {
- page = alloc_page(GFP_KERNEL);
+ /*
+ * Completion copies the full bounce buffer, including bytes the
+ * target did not write. This is already the slow fallback for
+ * misaligned I/O, so zeroing is acceptable here.
+ */
+ page = alloc_page(GFP_KERNEL | __GFP_ZERO);
if (!page) {
ret = -ENOMEM;
goto err;
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index a31786796d4c..c7a9702c2447 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -1151,7 +1151,7 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
unsigned long list_size = PAGE_SIZE / sizeof(struct page *);
unsigned int gup_flags = FOLL_LONGTERM;
unsigned long npages, cur_base, map_pfn, last_pfn = 0;
- unsigned long lock_limit, sz2pin, nchunks, i;
+ unsigned long lock_limit, sz2pin, i;
unsigned long page_offset;
u64 start = iova;
long pinned;
@@ -1187,7 +1187,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
cur_base = uaddr & PAGE_MASK;
iova &= PAGE_MASK;
- nchunks = 0;
while (npages) {
sz2pin = min_t(unsigned long, npages, list_size);
@@ -1202,7 +1201,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
}
goto out;
}
- nchunks++;
if (!last_pfn)
map_pfn = page_to_pfn(page_list[0]);
@@ -1233,7 +1231,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
map_pfn = this_pfn;
iova += csize;
- nchunks = 0;
}
last_pfn = this_pfn;
@@ -1248,7 +1245,7 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
PFN_PHYS(map_pfn), perm, NULL);
out:
if (ret) {
- if (nchunks) {
+ if (last_pfn) {
unsigned long pfn;
/*
@@ -1260,7 +1257,6 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v,
* the corresponding unpinning will be handled by
* vdpa_unmap().
*/
- WARN_ON(!last_pfn);
for (pfn = map_pfn; pfn <= last_pfn; pfn++)
unpin_user_page(pfn_to_page(pfn));
}
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index 44cac11b68d2..0226c4c59709 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -3172,8 +3172,7 @@ static int vhost_add_used_n_in_order(struct vhost_virtqueue *vq,
}
start += nheads[i];
new += nheads[i];
- if (start >= vq->num)
- start -= vq->num;
+ start &= vq->num - 1;
}
if (unlikely(vq->log_used)) {
diff --git a/drivers/vhost/vringh.c b/drivers/vhost/vringh.c
index 9066f9f12ba1..f027f669460f 100644
--- a/drivers/vhost/vringh.c
+++ b/drivers/vhost/vringh.c
@@ -197,8 +197,9 @@ static int move_to_indirect(const struct vringh *vrh,
}
len = vringh32_to_cpu(vrh, desc->len);
- if (unlikely(len % sizeof(struct vring_desc))) {
- vringh_bad("Strange indirect len %u", desc->len);
+ if (unlikely(len < sizeof(struct vring_desc) ||
+ len % sizeof(struct vring_desc))) {
+ vringh_bad("Invalid indirect len %u", desc->len);
return -EINVAL;
}
@@ -333,6 +334,17 @@ __vringh_iov(struct vringh *vrh, u16 i,
if (unlikely(err))
goto fail;
+ if (up_next == -1)
+ count++;
+ else
+ indirect_count++;
+
+ if (count > vrh->vring.num || indirect_count > desc_max) {
+ vringh_bad("Descriptor loop in %p", descs);
+ err = -ELOOP;
+ goto fail;
+ }
+
if (unlikely(desc.flags &
cpu_to_vringh16(vrh, VRING_DESC_F_INDIRECT))) {
u64 a = vringh64_to_cpu(vrh, desc.addr);
@@ -358,17 +370,6 @@ __vringh_iov(struct vringh *vrh, u16 i,
continue;
}
- if (up_next == -1)
- count++;
- else
- indirect_count++;
-
- if (count > vrh->vring.num || indirect_count > desc_max) {
- vringh_bad("Descriptor loop in %p", descs);
- err = -ELOOP;
- goto fail;
- }
-
if (desc.flags & cpu_to_vringh16(vrh, VRING_DESC_F_WRITE))
iov = wiov;
else {
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index abed1fbcf66c..35f75e23c749 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -364,7 +364,7 @@ static struct sk_buff *
vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,
unsigned int out, unsigned int in)
{
- struct virtio_vsock_hdr *hdr;
+ struct virtio_vsock_hdr hdr;
struct iov_iter iov_iter;
struct sk_buff *skb;
size_t payload_len;
@@ -382,34 +382,32 @@ vhost_vsock_alloc_skb(struct vhost_virtqueue *vq,
len > VIRTIO_VSOCK_MAX_PKT_BUF_SIZE + VIRTIO_VSOCK_SKB_HEADROOM)
return NULL;
- /* len contains both payload and hdr */
- skb = virtio_vsock_alloc_skb(len, GFP_KERNEL);
- if (!skb)
- return NULL;
-
iov_iter_init(&iov_iter, ITER_SOURCE, vq->iov, out, len);
- hdr = virtio_vsock_hdr(skb);
- nbytes = copy_from_iter(hdr, sizeof(*hdr), &iov_iter);
- if (nbytes != sizeof(*hdr)) {
+ nbytes = copy_from_iter(&hdr, sizeof(hdr), &iov_iter);
+ if (nbytes != sizeof(hdr)) {
vq_err(vq, "Expected %zu bytes for pkt->hdr, got %zu bytes\n",
- sizeof(*hdr), nbytes);
- kfree_skb(skb);
+ sizeof(hdr), nbytes);
return NULL;
}
- payload_len = le32_to_cpu(hdr->len);
+ payload_len = le32_to_cpu(hdr.len);
+
+ /* The pkt is too big or the length in the header is invalid */
+ if (payload_len > len - sizeof(hdr))
+ return NULL;
+
+ /* Allocate only for the payload declared in the header. */
+ skb = virtio_vsock_alloc_skb(payload_len + sizeof(hdr), GFP_KERNEL);
+ if (!skb)
+ return NULL;
+
+ memcpy(virtio_vsock_hdr(skb), &hdr, sizeof(hdr));
/* No payload */
if (!payload_len)
return skb;
- /* The pkt is too big or the length in the header is invalid */
- if (payload_len + sizeof(*hdr) > len) {
- kfree_skb(skb);
- return NULL;
- }
-
virtio_vsock_skb_put(skb, payload_len);
if (skb_copy_datagram_from_iter(skb, 0, &iov_iter, payload_len)) {
diff --git a/fs/9p/v9fs.c b/fs/9p/v9fs.c
index 274c5157135d..b30ca5145fae 100644
--- a/fs/9p/v9fs.c
+++ b/fs/9p/v9fs.c
@@ -17,6 +17,7 @@
#include <linux/fs_context.h>
#include <linux/slab.h>
#include <linux/seq_file.h>
+#include <linux/sysfs.h>
#include <net/9p/9p.h>
#include <net/9p/client.h>
#include <net/9p/transport.h>
@@ -406,7 +407,7 @@ int v9fs_parse_param(struct fs_context *fc, struct fs_parameter *param)
break;
case Opt_privport:
fd_opts->privport = true;
- rdma_opts->port = true;
+ rdma_opts->privport = true;
break;
}
@@ -617,21 +618,14 @@ static ssize_t caches_show(struct kobject *kobj,
struct kobj_attribute *attr,
char *buf)
{
- ssize_t n = 0, count = 0, limit = PAGE_SIZE;
+ ssize_t count = 0;
struct v9fs_session_info *v9ses;
spin_lock(&v9fs_sessionlist_lock);
list_for_each_entry(v9ses, &v9fs_sessionlist, slist) {
- if (v9ses->cachetag) {
- n = snprintf(buf + count, limit, "%s\n", v9ses->cachetag);
- if (n < 0) {
- count = n;
- break;
- }
-
- count += n;
- limit -= n;
- }
+ if (v9ses->cachetag)
+ count += sysfs_emit_at(buf, count, "%s\n",
+ v9ses->cachetag);
}
spin_unlock(&v9fs_sessionlist_lock);
diff --git a/fs/9p/vfs_inode.c b/fs/9p/vfs_inode.c
index 3829554ca369..2d7e0c1a5353 100644
--- a/fs/9p/vfs_inode.c
+++ b/fs/9p/vfs_inode.c
@@ -495,6 +495,12 @@ static int v9fs_at_to_dotl_flags(int flags)
* nothing and races with concurrent metadata fetches that may already
* have observed the post-unlink value (nlink == 0).
*
+ * The same applies to a regular file whose cached nlink has already been
+ * driven to zero by the server: in the cached modes the attributes come
+ * from GETATTR and are applied verbatim, so decrementing a link count the
+ * server does not maintain only trips the drop_nlink() WARN_ON and
+ * underflows i_nlink.
+ *
* @inode: inode whose nlink is being dropped
*/
static void v9fs_dec_count(struct inode *inode)
@@ -503,6 +509,8 @@ static void v9fs_dec_count(struct inode *inode)
if (!(v9ses->cache & (CACHE_META | CACHE_LOOSE)))
return;
+ if (inode->i_nlink == 0)
+ return;
if (!S_ISDIR(inode->i_mode) || inode->i_nlink > 2)
drop_nlink(inode);
}
diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 576cd76ca0bf..beae9dcb5a3e 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -375,6 +375,19 @@ int btrfs_fileattr_set(struct mnt_idmap *idmap,
}
}
+ /*
+ * Callers write back the whole flag set, so only touch the compression
+ * flags and the compression property if FS_COMPR_FL or FS_NOCOMP_FL
+ * changes.
+ */
+ if (!((fsflags ^ old_fsflags) & (FS_COMPR_FL | FS_NOCOMP_FL))) {
+ /* 1 item for the inode */
+ trans = btrfs_start_transaction(root, 1);
+ if (IS_ERR(trans))
+ return PTR_ERR(trans);
+ goto update_flags;
+ }
+
/*
* The COMPRESS flag can only be changed by users, while the NOCOMPRESS
* flag may be changed automatically if compression code won't make
@@ -2380,6 +2393,17 @@ static noinline int btrfs_ioctl_snap_destroy(struct file *file,
inode = d_inode(dentry);
dest = BTRFS_I(inode)->root;
+
+ /*
+ * A concurrent rename may have changed which subvolume the name
+ * refers to. Verify that the resolved ID still matches the requested ID.
+ */
+ if (destroy_v2 && (vol_args2->flags & BTRFS_SUBVOL_SPEC_BY_ID) &&
+ btrfs_root_id(dest) != vol_args2->subvolid) {
+ ret = -EAGAIN;
+ goto out_end_removing;
+ }
+
if (!capable(CAP_SYS_ADMIN)) {
/*
* Regular user. Only allow this with a special mount
diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index 39cb01ee441a..71b1ec6c0643 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -1857,6 +1857,7 @@ void btrfs_sysfs_add_block_group_type(struct btrfs_block_group *cache)
if (space_info->block_group_kobjs[index]) {
spin_unlock(&space_info->lock);
kobject_put(&rkobj->kobj);
+ memalloc_nofs_restore(nofs_flag);
return;
} else {
space_info->block_group_kobjs[index] = &rkobj->kobj;
diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c
index e07a7b1d3663..f32a533f4100 100644
--- a/fs/ceph/addr.c
+++ b/fs/ceph/addr.c
@@ -2220,6 +2220,12 @@ void ceph_fill_inline_data(struct inode *inode, struct page *locked_page,
doutc(cl, "%p %llx.%llx len %zu locked_page %p\n", inode,
ceph_vinop(inode), len, locked_page);
+ if (len > PAGE_SIZE) {
+ pr_warn_ratelimited_client(cl, "oversized inline data %zu\n",
+ len);
+ len = PAGE_SIZE;
+ }
+
if (len > 0) {
void *kaddr = kmap_atomic(page);
memcpy(kaddr, data, len);
diff --git a/fs/erofs/fileio.c b/fs/erofs/fileio.c
index ebb81a7ffd4c..ac846cfbc563 100644
--- a/fs/erofs/fileio.c
+++ b/fs/erofs/fileio.c
@@ -201,17 +201,29 @@ int erofs_read_meta_folio(struct file *file, struct folio *folio)
.dev = { .m_pa = folio_pos(folio), },
};
struct inode *inode = folio_inode(folio);
+ size_t len = folio_size(folio);
+ loff_t pos, isize;
int err;
err = erofs_map_dev(inode->i_sb, &io.dev);
if (err)
return err;
+ pos = io.dev.m_dif->fsoff + io.dev.m_pa;
+ isize = i_size_read(file_inode(io.dev.m_dif->file));
+ if (pos + folio_size(folio) > isize) {
+ len = isize > pos ? isize - pos : 0;
+ folio_zero_segment(folio, len, folio_size(folio));
+ if (!len) {
+ folio_end_read(folio, true);
+ return 0;
+ }
+ }
+
io.rq = erofs_fileio_rq_alloc(&io.dev);
- io.rq->bio.bi_iter.bi_sector =
- (io.dev.m_dif->fsoff + io.dev.m_pa) >> 9;
+ io.rq->bio.bi_iter.bi_sector = pos >> 9;
erofs_onlinefolio_init(folio);
- bio_add_folio_nofail(&io.rq->bio, folio, folio_size(folio), 0);
+ bio_add_folio_nofail(&io.rq->bio, folio, len, 0);
erofs_fileio_rq_submit(io.rq);
return 0;
}
diff --git a/fs/ext2/file.c b/fs/ext2/file.c
index b9020df7d89e..d4a0a528c342 100644
--- a/fs/ext2/file.c
+++ b/fs/ext2/file.c
@@ -138,7 +138,8 @@ static ssize_t ext2_dio_write_iter(struct kiocb *iocb, struct iov_iter *from)
pos = iocb->ki_pos;
status = generic_perform_write(iocb, from);
if (unlikely(status < 0)) {
- ret = status;
+ if (!ret)
+ ret = status;
goto out_unlock;
}
diff --git a/fs/ext2/inode.c b/fs/ext2/inode.c
index a9245f0cda4d..53edda7217f3 100644
--- a/fs/ext2/inode.c
+++ b/fs/ext2/inode.c
@@ -801,6 +801,12 @@ static int ext2_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
(first_block << blkbits) < i_size_read(inode))
create = 0;
+ ret = ext2_get_blocks(inode, first_block, max_blocks,
+ &bno, &new, &boundary, create);
+ if (ret < 0)
+ return ret;
+
+ iomap->flags = 0;
/*
* Writes that span EOF might trigger an IO size update on completion,
* so consider them to be dirty for the purposes of O_DSYNC even if
@@ -808,13 +814,6 @@ static int ext2_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
*/
if ((flags & IOMAP_WRITE) && offset + length > i_size_read(inode))
iomap->flags |= IOMAP_F_DIRTY;
-
- ret = ext2_get_blocks(inode, first_block, max_blocks,
- &bno, &new, &boundary, create);
- if (ret < 0)
- return ret;
-
- iomap->flags = 0;
iomap->offset = (u64)first_block << blkbits;
iomap->bdev = inode->i_sb->s_bdev;
diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c
index 4b59f30ef45d..a84517023e54 100644
--- a/fs/f2fs/checkpoint.c
+++ b/fs/f2fs/checkpoint.c
@@ -291,7 +291,7 @@ static struct folio *__get_meta_folio(struct f2fs_sb_info *sbi, pgoff_t index,
f2fs_update_iostat(sbi, NULL, FS_META_READ_IO, F2FS_BLKSIZE);
folio_lock(folio);
- if (unlikely(!is_meta_folio(folio))) {
+ if (unlikely(!is_meta_folio(sbi, folio))) {
f2fs_folio_put(folio, true);
goto repeat;
}
@@ -610,7 +610,7 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, long nr_to_write,
enum iostat_type io_type)
{
struct address_space *mapping = META_MAPPING(sbi);
- pgoff_t index = 0, prev = ULONG_MAX;
+ pgoff_t index = 0, next = ULONG_MAX;
struct folio_batch fbatch;
long nwritten = 0;
int nr_folios;
@@ -629,16 +629,15 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, long nr_to_write,
for (i = 0; i < nr_folios; i++) {
struct folio *folio = fbatch.folios[i];
- if (nr_to_write != LONG_MAX && i != 0 &&
- folio->index != prev +
- folio_nr_pages(fbatch.folios[i-1])) {
+ if (nr_to_write != LONG_MAX && next != ULONG_MAX &&
+ folio->index != next) {
folio_batch_release(&fbatch);
goto stop;
}
folio_lock(folio);
- if (unlikely(!is_meta_folio(folio))) {
+ if (unlikely(!is_meta_folio(sbi, folio))) {
continue_unlock:
folio_unlock(folio);
continue;
@@ -659,7 +658,7 @@ long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, long nr_to_write,
break;
}
nwritten += folio_nr_pages(folio);
- prev = folio->index;
+ next = folio->index + folio_nr_pages(folio);
if (unlikely(nwritten >= nr_to_write))
break;
}
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index cfd81bfc28a2..e0abc159b796 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -172,7 +172,8 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task)
dec_page_count(F2FS_F_SB(folio), __read_io_type(folio));
if (bio->bi_status == BLK_STS_OK &&
- F2FS_F_SB(folio)->node_inode && is_node_folio(folio) &&
+ F2FS_F_SB(folio)->node_inode &&
+ is_node_folio(F2FS_F_SB(folio), folio) &&
f2fs_sanity_check_node_footer(F2FS_F_SB(folio),
folio, folio->index, NODE_TYPE_REGULAR, true))
bio->bi_status = BLK_STS_IOERR;
@@ -359,12 +360,12 @@ static void f2fs_write_end_bio(struct bio *bio)
}
}
- if (is_node_folio(folio)) {
+ if (is_node_folio(sbi, folio)) {
f2fs_sanity_check_node_footer(sbi, folio,
folio->index, NODE_TYPE_REGULAR, true);
f2fs_bug_on(sbi, folio->index != nid_of_node(folio));
}
- if (f2fs_in_warm_node_list(folio))
+ if (f2fs_in_warm_node_list(sbi, folio))
f2fs_del_fsync_node_entry(sbi, folio);
dec_page_count(sbi, type);
@@ -399,7 +400,8 @@ static void f2fs_write_end_io(struct bio *bio)
sbi = bio->bi_private;
- if (in_atomic() && bio->bi_iter.bi_size > sbi->max_atc_write_bio_size) {
+ if (bio_in_atomic() &&
+ bio->bi_iter.bi_size > sbi->max_atc_write_bio_size) {
struct work_struct *w;
w = &container_of(bio, struct f2fs_bio, bio)->work;
@@ -500,6 +502,10 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages)
struct bio *bio;
bdev = f2fs_target_device(sbi, fio->new_blkaddr, §or);
+ /* avoid reversed IO if bio is split due to exceeding max_segments */
+ if (npages > 1 && !is_read_io(fio->op) &&
+ f2fs_is_sequential_zone_area(sbi, fio->new_blkaddr))
+ npages = min_t(int, npages, bdev_max_segments(bdev));
bio = bio_alloc_bioset(bdev, npages,
fio->op | fio->op_flags | f2fs_io_flags(fio),
GFP_NOIO, &f2fs_bioset);
@@ -790,7 +796,19 @@ static bool page_is_mergeable(struct f2fs_sb_info *sbi, struct bio *bio,
return false;
if (last_blkaddr + 1 != cur_blkaddr)
return false;
- return bio->bi_bdev == f2fs_target_device(sbi, cur_blkaddr, NULL);
+ if (bio->bi_bdev != f2fs_target_device(sbi, cur_blkaddr, NULL))
+ return false;
+ /* avoid reversed IO if bio is split due to exceeding max_segments/sectors */
+ if (f2fs_is_sequential_zone_area(sbi, cur_blkaddr)) {
+ struct request_queue *q = bdev_get_queue(bio->bi_bdev);
+
+ if (bio->bi_vcnt >= queue_max_segments(q))
+ return false;
+ if (bio_sectors(bio) + (F2FS_BLKSIZE >> SECTOR_SHIFT) >
+ queue_max_sectors(q))
+ return false;
+ }
+ return true;
}
static bool io_type_is_mergeable(struct f2fs_bio_info *io,
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 0c85d90307f7..f0024647153d 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -2316,14 +2316,14 @@ static inline struct address_space *NODE_MAPPING(struct f2fs_sb_info *sbi)
return sbi->node_inode->i_mapping;
}
-static inline bool is_meta_folio(struct folio *folio)
+static inline bool is_meta_folio(struct f2fs_sb_info *sbi, struct folio *folio)
{
- return folio->mapping == META_MAPPING(F2FS_F_SB(folio));
+ return folio->mapping == META_MAPPING(sbi);
}
-static inline bool is_node_folio(struct folio *folio)
+static inline bool is_node_folio(struct f2fs_sb_info *sbi, struct folio *folio)
{
- return folio->mapping == NODE_MAPPING(F2FS_F_SB(folio));
+ return folio->mapping == NODE_MAPPING(sbi);
}
static inline bool is_sbi_flag_set(struct f2fs_sb_info *sbi, unsigned int type)
@@ -3994,7 +3994,7 @@ enum node_type;
int f2fs_check_nid_range(struct f2fs_sb_info *sbi, nid_t nid);
bool f2fs_available_free_memory(struct f2fs_sb_info *sbi, int type);
-bool f2fs_in_warm_node_list(struct folio *folio);
+bool f2fs_in_warm_node_list(struct f2fs_sb_info *sbi, struct folio *folio);
void f2fs_init_fsync_node_info(struct f2fs_sb_info *sbi);
void f2fs_del_fsync_node_entry(struct f2fs_sb_info *sbi, struct folio *folio);
void f2fs_reset_fsync_node_info(struct f2fs_sb_info *sbi);
diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c
index 0a00180c21dc..6220d55709e9 100644
--- a/fs/f2fs/gc.c
+++ b/fs/f2fs/gc.c
@@ -1429,7 +1429,7 @@ static int move_data_block(struct inode *inode, block_t bidx,
F2FS_BLKSIZE);
folio_lock(mfolio);
- if (unlikely(!is_meta_folio(mfolio) ||
+ if (unlikely(!is_meta_folio(fio.sbi, mfolio) ||
!folio_test_uptodate(mfolio))) {
err = -EIO;
f2fs_folio_put(mfolio, true);
@@ -1857,6 +1857,9 @@ static int do_garbage_collect(struct f2fs_sb_info *sbi,
if (block_end_segno > end_segno)
block_end_segno = end_segno;
+ if (unlikely(freezing(current)))
+ goto next_block;
+
if (is_cursec(sbi, GET_SEC_FROM_SEG(sbi, segno))) {
f2fs_err(sbi, "%s: segment %u is used by log",
__func__, segno);
@@ -1924,18 +1927,12 @@ static int do_garbage_collect(struct f2fs_sb_info *sbi,
sbi->next_victim_seg[gc_type] =
(cur_segno + 1 < sec_end_segno) ?
cur_segno + 1 : NULL_SEGNO;
-
- if (unlikely(freezing(current))) {
- folio_put_refs(sum_folio, 2);
- goto stop;
- }
}
next_block:
folio_put_refs(sum_folio, 2);
segno = block_end_segno;
}
-stop:
if (submitted)
f2fs_submit_merged_write(sbi, data_type);
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index 86c2e67e43b6..e166c5a5d170 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -330,9 +330,9 @@ static unsigned int __gang_lookup_nat_set(struct f2fs_nm_info *nm_i,
start, nr);
}
-bool f2fs_in_warm_node_list(struct folio *folio)
+bool f2fs_in_warm_node_list(struct f2fs_sb_info *sbi, struct folio *folio)
{
- return is_node_folio(folio) && IS_DNODE(folio) && is_cold_node(folio);
+ return is_node_folio(sbi, folio) && IS_DNODE(folio) && is_cold_node(folio);
}
void f2fs_init_fsync_node_info(struct f2fs_sb_info *sbi)
@@ -835,8 +835,20 @@ int f2fs_get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode)
if (!dn->inode_folio) {
nfolio[0] = f2fs_get_inode_folio(sbi, nids[0]);
- if (IS_ERR(nfolio[0]))
- return PTR_ERR(nfolio[0]);
+ if (IS_ERR(nfolio[0])) {
+ err = PTR_ERR(nfolio[0]);
+ /*
+ * cp_error drops dirty node caches, so the inode block
+ * of an inode that was never written reads as NEW_ADDR
+ * and returns -ENOENT. It is not a hole; the inode is
+ * gone. Callers treat -ENOENT as a hole and keep
+ * looking block by block (e.g. fiemap up to maxbytes).
+ */
+ if (err == -ENOENT && (f2fs_cp_error(sbi) ||
+ is_sbi_flag_set(sbi, SBI_IS_SHUTDOWN)))
+ err = -EIO;
+ return err;
+ }
} else {
nfolio[0] = dn->inode_folio;
}
@@ -1290,7 +1302,7 @@ int f2fs_truncate_inode_blocks(struct inode *inode, pgoff_t from)
goto fail;
if (offset[1] == 0 && get_nid(folio, offset[0], true)) {
folio_lock(folio);
- BUG_ON(!is_node_folio(folio));
+ BUG_ON(!is_node_folio(sbi, folio));
set_nid(folio, offset[0], 0, true);
folio_unlock(folio);
}
@@ -1605,7 +1617,7 @@ static struct folio *__get_node_folio(struct f2fs_sb_info *sbi, pgoff_t nid,
folio_lock(folio);
- if (unlikely(!is_node_folio(folio))) {
+ if (unlikely(!is_node_folio(sbi, folio))) {
f2fs_folio_put(folio, true);
goto repeat;
}
@@ -1724,7 +1736,7 @@ static struct folio *last_fsync_dnode(struct f2fs_sb_info *sbi, nid_t ino)
folio_lock(folio);
- if (unlikely(!is_node_folio(folio))) {
+ if (unlikely(!is_node_folio(sbi, folio))) {
continue_unlock:
folio_unlock(folio);
continue;
@@ -1834,7 +1846,7 @@ static bool __write_node_folio(struct folio *folio, bool atomic, bool do_fsync,
f2fs_need_dentry_mark(sbi, ino_of_node(folio)));
/* should add to global list before clearing PAGECACHE status */
- if (f2fs_in_warm_node_list(folio)) {
+ if (f2fs_in_warm_node_list(sbi, folio)) {
seq = f2fs_add_fsync_node_entry(sbi, folio);
if (seq_id)
*seq_id = seq;
@@ -1958,7 +1970,7 @@ int f2fs_fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode,
folio_lock(folio);
- if (unlikely(!is_node_folio(folio))) {
+ if (unlikely(!is_node_folio(sbi, folio))) {
continue_unlock:
folio_unlock(folio);
continue;
@@ -2016,7 +2028,7 @@ int f2fs_fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode,
f2fs_debug(sbi, "Retry to write fsync mark: ino=%u, idx=%lx",
ino, last_folio->index);
folio_lock(last_folio);
- if (unlikely(!is_node_folio(last_folio))) {
+ if (unlikely(!is_node_folio(sbi, last_folio))) {
f2fs_folio_put(last_folio, true);
ret = -EAGAIN;
goto out;
@@ -2094,7 +2106,7 @@ void f2fs_flush_inline_data(struct f2fs_sb_info *sbi)
folio_lock(folio);
- if (unlikely(!is_node_folio(folio)))
+ if (unlikely(!is_node_folio(sbi, folio)))
goto unlock;
if (!folio_test_dirty(folio))
goto unlock;
@@ -2166,7 +2178,7 @@ int f2fs_sync_node_pages(struct f2fs_sb_info *sbi,
else if (!folio_trylock(folio))
continue;
- if (unlikely(!is_node_folio(folio))) {
+ if (unlikely(!is_node_folio(sbi, folio))) {
continue_unlock:
folio_unlock(folio);
continue;
@@ -2871,6 +2883,45 @@ int f2fs_recover_inline_xattr(struct inode *inode, struct folio *folio)
f2fs_folio_wait_writeback(ifolio, NODE, true, true);
memcpy(dst_addr, src_addr, inline_size);
update_inode:
+ /*
+ * The xattr node of the checkpoint was removed before fsync, e.g. the
+ * xattrs became small enough to fit inline. Release it as well, or
+ * the inode keeps pointing to the stale xattrs.
+ */
+ if (!ri->i_xattr_nid && F2FS_I(inode)->i_xattr_nid) {
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ struct node_info ni;
+ int err;
+
+ err = f2fs_get_node_info(sbi, F2FS_I(inode)->i_xattr_nid,
+ &ni, false);
+ if (err) {
+ f2fs_folio_put(ifolio, true);
+ return err;
+ }
+
+ if (ni.blk_addr != NEW_ADDR &&
+ !f2fs_is_valid_blkaddr(sbi, ni.blk_addr,
+ DATA_GENERIC_ENHANCE)) {
+ f2fs_err_ratelimited(sbi,
+ "nat entry is corrupted, run fsck to fix it, ino:%u, nid:%u, blkaddr:%u",
+ ni.ino, ni.nid, ni.blk_addr);
+ set_sbi_flag(sbi, SBI_NEED_FSCK);
+ f2fs_handle_error(sbi, ERROR_INCONSISTENT_NAT);
+ f2fs_folio_put(ifolio, true);
+ return -EFSCORRUPTED;
+ }
+
+ f2fs_invalidate_blocks(sbi, ni.blk_addr, 1);
+ dec_valid_node_count(sbi, inode, false);
+ set_node_addr(sbi, &ni, NULL_ADDR, false);
+ /* drop the cached xattr node, which may be dirty */
+ truncate_inode_pages_range(NODE_MAPPING(sbi),
+ F2FS_BLK_TO_BYTES((loff_t)ni.nid),
+ F2FS_BLK_END_BYTES((loff_t)ni.nid));
+ f2fs_i_xnid_write(inode, 0);
+ }
+
f2fs_update_inode(inode, ifolio);
f2fs_folio_put(ifolio, true);
return 0;
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index f3a73b6c24c3..75f78ed7eb90 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -3234,6 +3234,7 @@ static int change_curseg(struct f2fs_sb_info *sbi, int type)
if (IS_ERR(sum_folio)) {
/* GC won't be able to use stale summary pages by cp_error */
memset(curseg->sum_blk, 0, sbi->sum_entry_size);
+ reset_curseg_fields(curseg);
return PTR_ERR(sum_folio);
}
sum_node = SUM_BLK_PAGE_ADDR(sbi, sum_folio, new_segno);
@@ -4158,7 +4159,7 @@ static void do_write_page(struct f2fs_summary *sum, struct f2fs_io_info *fio)
__func__, fio->ino, folio->index, type,
fio->old_blkaddr, fio->new_blkaddr, err);
folio_end_writeback(folio);
- if (f2fs_in_warm_node_list(folio))
+ if (f2fs_in_warm_node_list(fio->sbi, folio))
f2fs_del_fsync_node_entry(fio->sbi, folio);
f2fs_bug_on(fio->sbi, !is_set_ckpt_flags(fio->sbi,
CP_ERROR_FLAG));
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index b9adad85e18c..b18ea09bcbe2 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -4255,6 +4255,7 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi)
block_t avail_node_count, valid_node_count;
unsigned int nat_blocks, nat_bits_bytes, nat_bits_blocks;
unsigned int sit_blk_cnt;
+ unsigned long long ckpt_size, sit_end, nat_end;
int i, j;
total = le32_to_cpu(raw_super->segment_count);
@@ -4386,6 +4387,29 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi)
return 1;
}
+ /*
+ * Nothing above ties the bitmap lengths to sbi->ckpt, which is
+ * (1 + cp_payload) blocks. Bound both ends the way __bitmap_ptr()
+ * lays them out.
+ */
+ ckpt_size = (unsigned long long)(cp_payload + 1) * F2FS_BLKSIZE;
+ if (__is_set_ckpt_flags(ckpt, CP_LARGE_NAT_BITMAP_FLAG)) {
+ nat_end = CP_MIN_CHKSUM_OFFSET + sizeof(__le32) + nat_bitmap_size;
+ sit_end = nat_end + sit_bitmap_size;
+ } else if (cp_payload > 0) {
+ nat_end = CP_MIN_CHKSUM_OFFSET + nat_bitmap_size;
+ sit_end = F2FS_BLKSIZE + sit_bitmap_size;
+ } else {
+ sit_end = CP_MIN_CHKSUM_OFFSET + sit_bitmap_size;
+ nat_end = sit_end + nat_bitmap_size;
+ }
+
+ if (sit_end > ckpt_size || nat_end > ckpt_size) {
+ f2fs_err(sbi, "Bitmaps do not fit the checkpoint pack: sit ends at %llu, nat at %llu, pack is %llu bytes",
+ sit_end, nat_end, ckpt_size);
+ return 1;
+ }
+
if (__is_set_ckpt_flags(ckpt, CP_LARGE_NAT_BITMAP_FLAG) &&
le32_to_cpu(ckpt->checksum_offset) != CP_MIN_CHKSUM_OFFSET) {
f2fs_warn(sbi, "using deprecated layout of large_nat_bitmap, "
diff --git a/fs/f2fs/sysfs.c b/fs/f2fs/sysfs.c
index 811e350a1430..5ee12547171c 100644
--- a/fs/f2fs/sysfs.c
+++ b/fs/f2fs/sysfs.c
@@ -1404,6 +1404,7 @@ F2FS_FEATURE_RO_ATTR(linear_lookup);
#endif
F2FS_FEATURE_RO_ATTR(packed_ssa);
F2FS_FEATURE_RO_ATTR(fserror);
+F2FS_FEATURE_RO_ATTR(resizable_tail);
#define ATTR_LIST(name) (&f2fs_attr_##name.attr)
static struct attribute *f2fs_attrs[] = {
@@ -1574,6 +1575,7 @@ static struct attribute *f2fs_feat_attrs[] = {
#endif
BASE_ATTR_LIST(packed_ssa),
BASE_ATTR_LIST(fserror),
+ BASE_ATTR_LIST(resizable_tail),
NULL,
};
ATTRIBUTE_GROUPS(f2fs_feat);
diff --git a/fs/fat/inode.c b/fs/fat/inode.c
index f775a004cae1..065658642e7f 100644
--- a/fs/fat/inode.c
+++ b/fs/fat/inode.c
@@ -216,7 +216,9 @@ static void fat_write_failed(struct address_space *mapping, loff_t to)
if (to > inode->i_size) {
truncate_pagecache(inode, inode->i_size);
+ down_write(&MSDOS_I(inode)->truncate_lock);
fat_truncate_blocks(inode, inode->i_size);
+ up_write(&MSDOS_I(inode)->truncate_lock);
}
}
diff --git a/fs/nfsd/nfs2acl.c b/fs/nfsd/nfs2acl.c
index 190f5a001900..113ec827ae06 100644
--- a/fs/nfsd/nfs2acl.c
+++ b/fs/nfsd/nfs2acl.c
@@ -15,6 +15,62 @@
#define NFSDDBG_FACILITY NFSDDBG_PROC
+/*
+ * These maps are identical to the NFSv3 maps (nfs3proc.c). This enables
+ * the behavior of the two versions to diverge if needed.
+ */
+static const struct nfsd_access_map nfsd2_regaccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_map nfsd2_diraccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_LOOKUP, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE },
+ { NFS3_ACCESS_DELETE, NFSD_MAY_REMOVE },
+ { 0, 0 }
+};
+
+/*
+ * Some clients - Solaris 2.6 at least, make an access call to the NFS
+ * server to check for access for things like /dev/null (which really,
+ * NFSD doesn't care about). So NFSD provides simple access checking
+ * for those objects, looking mainly at mode bits, ignoring read-only
+ * filesystem checks.
+ */
+static const struct nfsd_access_map nfsd2_otheraccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_maps nfsd2_access_maps = {
+ .regular = nfsd2_regaccess,
+ .directory = nfsd2_diraccess,
+ .other = nfsd2_otheraccess,
+};
+
+static __be32 nfsacld_map_status(__be32 status)
+{
+ switch (status) {
+ case nfserr_nofilehandle:
+ case nfserr_badhandle:
+ status = nfserr_stale;
+ break;
+ case nfserr_wrongsec:
+ status = nfserr_acces;
+ break;
+ }
+ return status;
+}
+
/*
* NULL call.
*/
@@ -79,6 +135,7 @@ static __be32 nfsacld_proc_getacl(struct svc_rqst *rqstp)
/* resp->acl_{access,default} are released in nfssvc_release_getacl. */
out:
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
fail:
@@ -137,6 +194,7 @@ static __be32 nfsacld_proc_setacl(struct svc_rqst *rqstp)
out:
/* argp->acl_{access,default} are released in nfsaclsvc_release_setacl. */
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
out_drop_lock:
@@ -163,6 +221,7 @@ static __be32 nfsacld_proc_getattr(struct svc_rqst *rqstp)
goto out;
resp->status = fh_getattr(&resp->fh, &resp->stat);
out:
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
}
@@ -180,11 +239,14 @@ static __be32 nfsacld_proc_access(struct svc_rqst *rqstp)
fh_copy(&resp->fh, &argp->fh);
resp->access = argp->access;
- resp->status = nfsd_access(rqstp, &resp->fh, &resp->access, NULL);
+
+ resp->status = nfsd_access(rqstp, &resp->fh, &nfsd2_access_maps,
+ &resp->access, NULL);
if (resp->status != nfs_ok)
goto out;
resp->status = fh_getattr(&resp->fh, &resp->stat);
out:
+ resp->status = nfsacld_map_status(resp->status);
return rpc_success;
}
diff --git a/fs/nfsd/nfs3proc.c b/fs/nfsd/nfs3proc.c
index 0904d953d10e..b3c634190701 100644
--- a/fs/nfsd/nfs3proc.c
+++ b/fs/nfsd/nfs3proc.c
@@ -48,6 +48,58 @@ static bool nfsd3_time_in_range(const struct iattr *iap)
return true;
}
+static const struct nfsd_access_map nfsd3_regaccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_map nfsd3_diraccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_LOOKUP, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE },
+ { NFS3_ACCESS_DELETE, NFSD_MAY_REMOVE },
+ { 0, 0 }
+};
+
+/*
+ * Some clients - Solaris 2.6 at least, make an access call to the NFS
+ * server to check for access for things like /dev/null (which really,
+ * NFSD doesn't care about). So NFSD provides simple access checking
+ * for those objects, looking mainly at mode bits, ignoring read-only
+ * filesystem checks.
+ */
+static const struct nfsd_access_map nfsd3_otheraccess[] = {
+ { NFS3_ACCESS_READ, NFSD_MAY_READ },
+ { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_maps nfsd3_access_maps = {
+ .regular = nfsd3_regaccess,
+ .directory = nfsd3_diraccess,
+ .other = nfsd3_otheraccess,
+};
+
+static int nfsd3_iocb_flags(enum nfs3_stable_how how)
+{
+ switch (how) {
+ case NFS_FILE_SYNC:
+ /* persist data and timestamps */
+ return IOCB_DSYNC | IOCB_SYNC;
+ case NFS_DATA_SYNC:
+ /* persist data only */
+ return IOCB_DSYNC;
+ default:
+ return 0;
+ }
+}
+
static __be32 nfsd3_map_status(__be32 status)
{
switch (status) {
@@ -171,7 +223,8 @@ nfsd3_proc_access(struct svc_rqst *rqstp)
fh_copy(&resp->fh, &argp->fh);
resp->access = argp->access;
- resp->status = nfsd_access(rqstp, &resp->fh, &resp->access, NULL);
+ resp->status = nfsd_access(rqstp, &resp->fh, &nfsd3_access_maps,
+ &resp->access, NULL);
resp->status = nfsd3_map_status(resp->status);
return rpc_success;
}
@@ -260,7 +313,8 @@ nfsd3_proc_write(struct svc_rqst *rqstp)
resp->committed = argp->stable;
resp->status = nfsd_write(rqstp, &resp->fh, argp->offset,
&argp->payload, &cnt,
- resp->committed, resp->verf);
+ nfsd3_iocb_flags(resp->committed),
+ resp->verf);
resp->count = cnt;
resp->status = nfsd3_map_status(resp->status);
return rpc_success;
diff --git a/fs/nfsd/nfs3xdr.c b/fs/nfsd/nfs3xdr.c
index e481804bb120..f418cab0bd61 100644
--- a/fs/nfsd/nfs3xdr.c
+++ b/fs/nfsd/nfs3xdr.c
@@ -556,6 +556,8 @@ nfs3svc_decode_writeargs(struct svc_rqst *rqstp, struct xdr_stream *xdr)
return false;
if (xdr_stream_decode_u32(xdr, &args->stable) < 0)
return false;
+ if (args->stable > NFS_FILE_SYNC)
+ return false;
/* opaque data */
if (xdr_stream_decode_u32(xdr, &args->len) < 0)
diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index f3f7f14a9368..f19d8ed80a9d 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -69,6 +69,57 @@ MODULE_PARM_DESC(nfsd4_ssc_umount_timeout,
#define NFSDDBG_FACILITY NFSDDBG_PROC
+static const struct nfsd_access_map nfsd4_regaccess[] = {
+ { NFS4_ACCESS_READ, NFSD_MAY_READ },
+ { NFS4_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS4_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS4_ACCESS_EXTEND, NFSD_MAY_WRITE },
+ { NFS4_ACCESS_XAREAD, NFSD_MAY_READ },
+ { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE },
+ { NFS4_ACCESS_XALIST, NFSD_MAY_READ },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_map nfsd4_diraccess[] = {
+ { NFS4_ACCESS_READ, NFSD_MAY_READ },
+ { NFS4_ACCESS_LOOKUP, NFSD_MAY_EXEC },
+ { NFS4_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
+ { NFS4_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE },
+ { NFS4_ACCESS_DELETE, NFSD_MAY_REMOVE },
+ { NFS4_ACCESS_XAREAD, NFSD_MAY_READ },
+ { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE },
+ { NFS4_ACCESS_XALIST, NFSD_MAY_READ },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_map nfsd4_otheraccess[] = {
+ { NFS4_ACCESS_READ, NFSD_MAY_READ },
+ { NFS4_ACCESS_EXECUTE, NFSD_MAY_EXEC },
+ { NFS4_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { NFS4_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
+ { 0, 0 }
+};
+
+static const struct nfsd_access_maps nfsd4_access_maps = {
+ .regular = nfsd4_regaccess,
+ .directory = nfsd4_diraccess,
+ .other = nfsd4_otheraccess,
+};
+
+static int nfsd4_iocb_flags(enum stable_how4 how)
+{
+ switch (how) {
+ case FILE_SYNC4:
+ /* persist data and timestamps */
+ return IOCB_DSYNC | IOCB_SYNC;
+ case DATA_SYNC4:
+ /* persist data only */
+ return IOCB_DSYNC;
+ default:
+ return 0;
+ }
+}
+
static u32 nfsd_attrmask[] = {
NFSD_WRITEABLE_ATTRS_WORD0,
NFSD_WRITEABLE_ATTRS_WORD1,
@@ -795,10 +846,9 @@ nfsd4_access(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
if (access->ac_req_access & ~access_full)
return nfserr_inval;
-
access->ac_resp_access = access->ac_req_access;
- return nfsd_access(rqstp, &cstate->current_fh, &access->ac_resp_access,
- &access->ac_supported);
+ return nfsd_access(rqstp, &cstate->current_fh, &nfsd4_access_maps,
+ &access->ac_resp_access, &access->ac_supported);
}
static __be32
@@ -1377,7 +1427,7 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
write->wr_how_written = write->wr_stable_how;
status = nfsd_vfs_write(rqstp, &cstate->current_fh, nf,
write->wr_offset, &write->wr_payload,
- &cnt, write->wr_how_written,
+ &cnt, nfsd4_iocb_flags(write->wr_how_written),
(__be32 *)write->wr_verifier.data);
nfsd_file_put(nf);
@@ -1974,7 +2024,7 @@ static void nfsd4_init_copy_res(struct nfsd4_copy *copy, bool sync)
{
copy->cp_res.wr_stable_how =
test_bit(NFSD4_COPY_F_COMMITTED, ©->cp_flags) ?
- NFS_FILE_SYNC : NFS_UNSTABLE;
+ FILE_SYNC4 : UNSTABLE4;
nfsd4_copy_set_sync(copy, sync);
}
diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
index 6441e54283a3..907a914b04cd 100644
--- a/fs/nfsd/nfs4recover.c
+++ b/fs/nfsd/nfs4recover.c
@@ -649,6 +649,7 @@ struct cld_upcall {
struct completion cu_done;
/* daemon has read the whole upcall; protected by cn_lock */
bool cu_inflight;
+ struct rpc_pipe_msg cu_pipe_msg;
union {
struct cld_msg_hdr cu_hdr;
struct cld_msg cu_msg;
@@ -660,22 +661,22 @@ static int
__cld_pipe_upcall(struct rpc_pipe *pipe, void *cmsg, struct nfsd_net *nn)
{
int ret;
- struct rpc_pipe_msg msg;
struct cld_upcall *cup = container_of(cmsg, struct cld_upcall, cu_u);
+ struct rpc_pipe_msg *msg = &cup->cu_pipe_msg;
- memset(&msg, 0, sizeof(msg));
- msg.data = cmsg;
- msg.len = nn->client_tracking_ops->msglen;
+ memset(msg, 0, sizeof(*msg));
+ msg->data = cmsg;
+ msg->len = nn->client_tracking_ops->msglen;
- ret = rpc_queue_upcall(pipe, &msg);
+ ret = rpc_queue_upcall(pipe, msg);
if (ret < 0) {
goto out;
}
wait_for_completion(&cup->cu_done);
- if (msg.errno < 0)
- ret = msg.errno;
+ if (msg->errno < 0)
+ ret = msg->errno;
out:
return ret;
}
@@ -838,8 +839,11 @@ cld_pipe_downcall(struct file *filp, const char __user *src, size_t mlen)
if (status == -EINPROGRESS)
return __cld_pipe_inprogress_downcall(cmsg, nn);
- if (copy_from_user(&cup->cu_u.cu_msg_v2, src, mlen) != 0)
+ if (copy_from_user(&cup->cu_u.cu_msg_v2, src, mlen) != 0) {
+ cup->cu_pipe_msg.errno = -EFAULT;
+ complete(&cup->cu_done);
return -EFAULT;
+ }
complete(&cup->cu_done);
return mlen;
@@ -848,9 +852,8 @@ cld_pipe_downcall(struct file *filp, const char __user *src, size_t mlen)
static void
cld_pipe_destroy_msg(struct rpc_pipe_msg *msg)
{
- struct cld_msg *cmsg = msg->data;
- struct cld_upcall *cup = container_of(cmsg, struct cld_upcall,
- cu_u.cu_msg);
+ struct cld_upcall *cup = container_of(msg, struct cld_upcall,
+ cu_pipe_msg);
struct cld_net *cn = cup->cu_net;
/*
@@ -867,10 +870,34 @@ cld_pipe_destroy_msg(struct rpc_pipe_msg *msg)
complete(&cup->cu_done);
}
+/*
+ * An upcall the daemon has consumed is off every pipe list, so the
+ * purge in rpc_pipe_release() and rpc_close_pipes() cannot reach it.
+ * Release its waiter here instead; no downcall can arrive now.
+ */
+static void
+cld_release_pipe(struct inode *inode)
+{
+ struct nfsd_net *nn = net_generic(inode->i_sb->s_fs_info, nfsd_net_id);
+ struct cld_net *cn = nn->cld_net;
+ struct cld_upcall *cup;
+
+ spin_lock(&cn->cn_lock);
+ list_for_each_entry(cup, &cn->cn_list, cu_list) {
+ if (!cup->cu_inflight)
+ continue;
+ cup->cu_inflight = false;
+ cup->cu_pipe_msg.errno = -EPIPE;
+ complete(&cup->cu_done);
+ }
+ spin_unlock(&cn->cn_lock);
+}
+
static const struct rpc_pipe_ops cld_upcall_ops = {
.upcall = rpc_pipe_generic_upcall,
.downcall = cld_pipe_downcall,
.destroy_msg = cld_pipe_destroy_msg,
+ .release_pipe = cld_release_pipe,
};
static int
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 3bdde4c1bb8e..f0a795940137 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1517,6 +1517,7 @@ hash_delegation_locked(struct nfs4_delegation *dp, struct nfs4_file *fp)
dp->dl_stid.sc_type = SC_TYPE_DELEG;
list_add(&dp->dl_perfile, &fp->fi_delegations);
list_add(&dp->dl_perclnt, &clp->cl_delegations);
+ clp->cl_deleg_count++;
return 0;
}
@@ -1548,6 +1549,7 @@ unhash_delegation_locked(struct nfs4_delegation *dp, unsigned short statusmask)
++dp->dl_time;
spin_lock(&fp->fi_lock);
list_del_init(&dp->dl_perclnt);
+ dp->dl_stid.sc_client->cl_deleg_count--;
list_del_init(&dp->dl_recall_lru);
list_del_init(&dp->dl_perfile);
spin_unlock(&fp->fi_lock);
@@ -7938,6 +7940,7 @@ deleg_reaper(struct nfsd_net *nn)
{
struct list_head *pos, *next;
struct nfs4_client *clp;
+ unsigned int count;
spin_lock(&nn->client_lock);
list_for_each_safe(pos, next, &nn->client_lru) {
@@ -7947,23 +7950,36 @@ deleg_reaper(struct nfsd_net *nn)
continue;
if (clp->cl_state != NFSD4_ACTIVE)
continue;
- if (list_empty(&clp->cl_delegations))
- continue;
if (atomic_read(&clp->cl_delegs_in_recall))
continue;
if (ktime_get_boottime_seconds() - clp->cl_ra_time < 5)
continue;
if (clp->cl_cb_state != NFSD4_CB_UP)
continue;
+ /*
+ * This read races with hash_delegation_locked() and
+ * unhash_delegation_locked() on other CPUs. A stale
+ * count only skews the keep value; the next
+ * laundromat pass sees a more current one.
+ */
+ count = data_race(READ_ONCE(clp->cl_deleg_count));
+ if (!count)
+ continue;
if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &clp->cl_ra->ra_cb.cb_flags))
continue;
/* release in nfsd4_cb_recall_any_release */
kref_get(&clp->cl_nfsdfs.cl_ref);
clp->cl_ra_time = ktime_get_boottime_seconds();
- clp->cl_ra->ra_keep = 0;
+ /*
+ * Ask for a single delegation. Recalling one before it
+ * is needed costs the client an OPEN when it next
+ * touches the file.
+ */
+ clp->cl_ra->ra_keep = count - 1;
clp->cl_ra->ra_bmval[0] = BIT(RCA4_TYPE_MASK_RDATA_DLG) |
- BIT(RCA4_TYPE_MASK_WDATA_DLG);
+ BIT(RCA4_TYPE_MASK_WDATA_DLG) |
+ BIT(RCA4_TYPE_MASK_DIR_DLG);
trace_nfsd_cb_recall_any(clp->cl_ra);
nfsd4_run_cb(&clp->cl_ra->ra_cb);
}
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index ebd0dc8f00fc..f7d2884aba44 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -1606,7 +1606,7 @@ nfsd4_decode_write(struct nfsd4_compoundargs *argp, union nfsd4_op_u *u)
return nfserr_bad_xdr;
if (xdr_stream_decode_u32(argp->xdr, &write->wr_stable_how) < 0)
return nfserr_bad_xdr;
- if (write->wr_stable_how > NFS_FILE_SYNC)
+ if (write->wr_stable_how > FILE_SYNC4)
return nfserr_bad_xdr;
if (xdr_stream_decode_u32(argp->xdr, &write->wr_buflen) < 0)
return nfserr_bad_xdr;
diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index 86aa8d202b4b..3a5d0f5cc0fc 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -2086,8 +2086,13 @@ static int nfsd_nl_validate_listeners(struct genl_info *info)
int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
{
struct net *net = genl_info_net(info);
+ const struct nlattr *bad_attr = NULL;
struct svc_xprt *xprt, *tmp;
+ const char *bad_xprt = NULL;
+ unsigned int rpcb_failures;
const struct nlattr *attr;
+ bool skipped_rpcb = false;
+ bool bad_rpcb = false;
struct svc_serv *serv;
LIST_HEAD(permsocks);
struct nfsd_net *nn;
@@ -2177,13 +2182,16 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
if (delete)
svc_xprt_destroy_all(serv, net, false);
+ rpcb_failures = svc_rpcb_failure_count(serv);
+
/* walk list of addrs again, open any that still don't exist */
nlmsg_for_each_attr_type(attr, NFSD_A_SERVER_SOCK_ADDR, info->nlhdr,
GENL_HDRLEN, rem) {
struct nlattr *tb[NFSD_A_SOCK_MAX + 1];
const char *xcl_name;
struct sockaddr *sa;
- int ret;
+ bool hit_rpcb;
+ int flags, ret;
/* validated up front in nfsd_nl_validate_listeners() */
if (nla_parse_nested(tb, NFSD_A_SOCK_MAX, attr,
@@ -2202,11 +2210,53 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
continue;
}
- ret = svc_xprt_create_from_sa(serv, xcl_name, net, sa, 0,
+ flags = skipped_rpcb ? SVC_SOCK_ANONYMOUS : 0;
+ ret = svc_xprt_create_from_sa(serv, xcl_name, net, sa, flags,
current_cred());
+
+ hit_rpcb = false;
+ if (!skipped_rpcb &&
+ svc_rpcb_failure_count(serv) != rpcb_failures) {
+ skipped_rpcb = true;
+ hit_rpcb = true;
+ if (ret < 0)
+ ret = svc_xprt_create_from_sa(serv, xcl_name,
+ net, sa,
+ SVC_SOCK_ANONYMOUS,
+ current_cred());
+ }
+
/* always save the latest error */
- if (ret < 0)
+ if (ret < 0) {
+ bad_attr = attr;
+ bad_xprt = xcl_name;
+ bad_rpcb = hit_rpcb;
err = ret;
+ }
+ }
+
+ /*
+ * The ack carries the errno of the last entry that failed. Point at
+ * that entry as well, since several entries can share a transport
+ * name and the errno alone cannot tell them apart.
+ */
+ if (err) {
+ NL_SET_BAD_ATTR(info->extack, bad_attr);
+ if (bad_rpcb)
+ NL_SET_ERR_MSG_FMT(info->extack,
+ "cannot create %s listener; rpcbind did not answer",
+ bad_xprt);
+ else if (skipped_rpcb)
+ NL_SET_ERR_MSG_FMT(info->extack,
+ "cannot create %s listener; rpcbind did not answer earlier",
+ bad_xprt);
+ else
+ NL_SET_ERR_MSG_FMT(info->extack,
+ "cannot create %s listener",
+ bad_xprt);
+ } else if (skipped_rpcb) {
+ NL_SET_ERR_MSG(info->extack,
+ "rpcbind did not answer, some listeners are not registered");
}
if (!serv->sv_nrthreads && list_empty(&nn->nfsd_serv->sv_permsocks))
diff --git a/fs/nfsd/nfsproc.c b/fs/nfsd/nfsproc.c
index aaeb222b62ba..48182f78d6be 100644
--- a/fs/nfsd/nfsproc.c
+++ b/fs/nfsd/nfsproc.c
@@ -280,7 +280,7 @@ nfsd_proc_write(struct svc_rqst *rqstp)
fh_copy(&resp->fh, &argp->fh);
resp->status = nfsd_write(rqstp, &resp->fh, argp->offset,
- &argp->payload, &cnt, NFS_DATA_SYNC, NULL);
+ &argp->payload, &cnt, IOCB_DSYNC, NULL);
if (resp->status == nfs_ok)
resp->status = fh_getattr(&resp->fh, &resp->stat);
else if (resp->status == nfserr_jukebox)
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index 9661c08b879e..df35c35890d4 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -610,6 +610,8 @@ struct nfs4_client {
unsigned int cl_state;
atomic_t cl_delegs_in_recall;
+ /* Length of cl_delegations, updated under nn->deleg_lock */
+ unsigned int cl_deleg_count;
struct nfsd4_cb_recall_any *cl_ra;
time64_t cl_ra_time;
diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c
index 8923a9910a08..792c0b327755 100644
--- a/fs/nfsd/vfs.c
+++ b/fs/nfsd/vfs.c
@@ -773,64 +773,21 @@ __be32 nfsd4_vfs_fallocate(struct svc_rqst *rqstp, struct svc_fh *fhp,
}
#endif /* defined(CONFIG_NFSD_V4) */
-/*
- * Check server access rights to a file system object
+/**
+ * nfsd_access - Check caller's access rights to a file system object
+ * @rqstp: RPC transaction context
+ * @fhp: target NFS filehandle
+ * @maps: tables mapping on-the-wire access bits to NFSD_MAY flags
+ * @access: requested access bits on entry, permitted bits on return
+ * @supported: optional output of the access bits the server supports
+ *
+ * Return: nfs_ok on success, otherwise an nfserr status code
*/
-struct accessmap {
- u32 access;
- int how;
-};
-static struct accessmap nfs3_regaccess[] = {
- { NFS3_ACCESS_READ, NFSD_MAY_READ },
- { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
- { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_TRUNC },
- { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE },
-
-#ifdef CONFIG_NFSD_V4
- { NFS4_ACCESS_XAREAD, NFSD_MAY_READ },
- { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE },
- { NFS4_ACCESS_XALIST, NFSD_MAY_READ },
-#endif
-
- { 0, 0 }
-};
-
-static struct accessmap nfs3_diraccess[] = {
- { NFS3_ACCESS_READ, NFSD_MAY_READ },
- { NFS3_ACCESS_LOOKUP, NFSD_MAY_EXEC },
- { NFS3_ACCESS_MODIFY, NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC},
- { NFS3_ACCESS_EXTEND, NFSD_MAY_EXEC|NFSD_MAY_WRITE },
- { NFS3_ACCESS_DELETE, NFSD_MAY_REMOVE },
-
-#ifdef CONFIG_NFSD_V4
- { NFS4_ACCESS_XAREAD, NFSD_MAY_READ },
- { NFS4_ACCESS_XAWRITE, NFSD_MAY_WRITE },
- { NFS4_ACCESS_XALIST, NFSD_MAY_READ },
-#endif
-
- { 0, 0 }
-};
-
-static struct accessmap nfs3_anyaccess[] = {
- /* Some clients - Solaris 2.6 at least, make an access call
- * to the server to check for access for things like /dev/null
- * (which really, the server doesn't care about). So
- * We provide simple access checking for them, looking
- * mainly at mode bits, and we make sure to ignore read-only
- * filesystem checks
- */
- { NFS3_ACCESS_READ, NFSD_MAY_READ },
- { NFS3_ACCESS_EXECUTE, NFSD_MAY_EXEC },
- { NFS3_ACCESS_MODIFY, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
- { NFS3_ACCESS_EXTEND, NFSD_MAY_WRITE|NFSD_MAY_LOCAL_ACCESS },
-
- { 0, 0 }
-};
-
-__be32
-nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp, u32 *access, u32 *supported)
+__be32 nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp,
+ const struct nfsd_access_maps *maps,
+ u32 *access, u32 *supported)
{
- struct accessmap *map;
+ const struct nfsd_access_map *map;
struct svc_export *export;
struct dentry *dentry;
u32 query, result = 0, sresult = 0;
@@ -844,12 +801,11 @@ nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp, u32 *access, u32 *suppor
dentry = fhp->fh_dentry;
if (d_is_reg(dentry))
- map = nfs3_regaccess;
+ map = maps->regular;
else if (d_is_dir(dentry))
- map = nfs3_diraccess;
+ map = maps->directory;
else
- map = nfs3_anyaccess;
-
+ map = maps->other;
query = *access;
for (; map->access; map++) {
@@ -859,7 +815,7 @@ nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp, u32 *access, u32 *suppor
sresult |= map->access;
err2 = nfsd_permission(&rqstp->rq_cred, export,
- dentry, map->how);
+ dentry, map->may);
switch (err2) {
case nfs_ok:
result |= map->access;
@@ -1424,7 +1380,7 @@ nfsd_direct_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
* @offset: Byte offset of start
* @payload: xdr_buf containing the write payload
* @cnt: IN: number of bytes to write, OUT: number of bytes actually written
- * @stable: An NFS stable_how value
+ * @iocb_flags: VFS IOCB_* flags expressing the requested write stability
* @verf: NFS WRITE verifier
*
* Upon return, caller must invoke fh_put on @fhp.
@@ -1436,7 +1392,7 @@ __be32
nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
struct nfsd_file *nf, loff_t offset,
const struct xdr_buf *payload, unsigned long *cnt,
- int stable, __be32 *verf)
+ int iocb_flags, __be32 *verf)
{
struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
struct file *file = nf->nf_file;
@@ -1473,21 +1429,11 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
exp = fhp->fh_export;
if (!EX_ISSYNC(exp))
- stable = NFS_UNSTABLE;
+ iocb_flags = 0;
init_sync_kiocb(&kiocb, file);
kiocb.ki_pos = offset;
- if (likely(!fhp->fh_use_wgather)) {
- switch (stable) {
- case NFS_FILE_SYNC:
- /* persist data and timestamps */
- kiocb.ki_flags |= IOCB_DSYNC | IOCB_SYNC;
- break;
- case NFS_DATA_SYNC:
- /* persist data only */
- kiocb.ki_flags |= IOCB_DSYNC;
- break;
- }
- }
+ if (likely(!fhp->fh_use_wgather))
+ kiocb.ki_flags |= iocb_flags;
nvecs = xdr_buf_to_bvec(rqstp->rq_bvec, rqstp->rq_maxpages, payload);
if (nvecs < 0) {
@@ -1528,7 +1474,7 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
goto out_nfserr;
}
- if (stable && fhp->fh_use_wgather) {
+ if (iocb_flags && fhp->fh_use_wgather) {
host_err = wait_for_concurrent_writes(file);
if (host_err < 0)
commit_reset_write_verifier(nn, rqstp, host_err);
@@ -1619,7 +1565,7 @@ __be32 nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
* @offset: Byte offset of start
* @payload: xdr_buf containing the write payload
* @cnt: IN: number of bytes to write, OUT: number of bytes actually written
- * @stable: An NFS stable_how value
+ * @iocb_flags: VFS IOCB_* flags expressing the requested write stability
* @verf: NFS WRITE verifier
*
* Upon return, caller must invoke fh_put on @fhp.
@@ -1629,8 +1575,8 @@ __be32 nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
*/
__be32
nfsd_write(struct svc_rqst *rqstp, struct svc_fh *fhp, loff_t offset,
- const struct xdr_buf *payload, unsigned long *cnt, int stable,
- __be32 *verf)
+ const struct xdr_buf *payload, unsigned long *cnt,
+ int iocb_flags, __be32 *verf)
{
struct nfsd_file *nf;
__be32 err;
@@ -1642,7 +1588,7 @@ nfsd_write(struct svc_rqst *rqstp, struct svc_fh *fhp, loff_t offset,
goto out;
err = nfsd_vfs_write(rqstp, fhp, nf, offset, payload, cnt,
- stable, verf);
+ iocb_flags, verf);
nfsd_file_put(nf);
out:
trace_nfsd_write_done(rqstp, fhp, offset, *cnt);
diff --git a/fs/nfsd/vfs.h b/fs/nfsd/vfs.h
index 4af2ff9e9dfe..bdb58422a3fb 100644
--- a/fs/nfsd/vfs.h
+++ b/fs/nfsd/vfs.h
@@ -37,6 +37,17 @@
#define NFSD_MAY_CREATE (NFSD_MAY_EXEC|NFSD_MAY_WRITE)
#define NFSD_MAY_REMOVE (NFSD_MAY_EXEC|NFSD_MAY_WRITE|NFSD_MAY_TRUNC)
+struct nfsd_access_map {
+ u32 access;
+ int may;
+};
+
+struct nfsd_access_maps {
+ const struct nfsd_access_map *regular;
+ const struct nfsd_access_map *directory;
+ const struct nfsd_access_map *other;
+};
+
struct nfsd_file;
/*
@@ -100,7 +111,9 @@ __be32 nfsd_create_locked(struct svc_rqst *, struct svc_fh *,
__be32 nfsd_create(struct svc_rqst *, struct svc_fh *,
char *name, int len, struct nfsd_attrs *attrs,
int type, dev_t rdev, struct svc_fh *res);
-__be32 nfsd_access(struct svc_rqst *, struct svc_fh *, u32 *, u32 *);
+__be32 nfsd_access(struct svc_rqst *rqstp, struct svc_fh *fhp,
+ const struct nfsd_access_maps *maps,
+ u32 *access, u32 *supported);
__be32 nfsd_create_setattr(struct svc_rqst *rqstp, struct svc_fh *fhp,
struct svc_fh *resfhp, struct nfsd_attrs *iap);
__be32 nfsd_commit(struct svc_rqst *rqst, struct svc_fh *fhp,
@@ -135,11 +148,13 @@ __be32 nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
u32 *eof);
__be32 nfsd_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
loff_t offset, const struct xdr_buf *payload,
- unsigned long *cnt, int stable, __be32 *verf);
+ unsigned long *cnt, int iocb_flags,
+ __be32 *verf);
__be32 nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
struct nfsd_file *nf, loff_t offset,
const struct xdr_buf *payload,
- unsigned long *cnt, int stable, __be32 *verf);
+ unsigned long *cnt, int iocb_flags,
+ __be32 *verf);
__be32 nfsd_readlink(struct svc_rqst *, struct svc_fh *,
char *, int *);
__be32 nfsd_symlink(struct svc_rqst *, struct svc_fh *,
diff --git a/fs/nfsd/xdr3.h b/fs/nfsd/xdr3.h
index 344203874b4c..cad875d14231 100644
--- a/fs/nfsd/xdr3.h
+++ b/fs/nfsd/xdr3.h
@@ -39,7 +39,7 @@ struct nfsd3_writeargs {
svc_fh fh;
__u64 offset;
__u32 count;
- int stable;
+ __u32 stable;
__u32 len;
struct xdr_buf payload;
};
diff --git a/fs/notify/inotify/inotify_fsnotify.c b/fs/notify/inotify/inotify_fsnotify.c
index 7c326ec2e8a8..6e3af6452dcc 100644
--- a/fs/notify/inotify/inotify_fsnotify.c
+++ b/fs/notify/inotify/inotify_fsnotify.c
@@ -36,7 +36,7 @@ static bool event_compare(struct fsnotify_event *old_fsn,
old = INOTIFY_E(old_fsn);
new = INOTIFY_E(new_fsn);
- if (old->mask & FS_IN_IGNORED)
+ if (old->mask & (FS_IN_IGNORED | FS_MOVE))
return false;
if ((old->mask == new->mask) &&
(old->wd == new->wd) &&
diff --git a/fs/ntfs/ea.c b/fs/ntfs/ea.c
index 6ac551fe8ea6..9e00588ff9b5 100644
--- a/fs/ntfs/ea.c
+++ b/fs/ntfs/ea.c
@@ -699,6 +699,11 @@ static int ntfs_new_attr_flags(struct ntfs_inode *ni, __le32 fattr)
goto err_out;
}
+ if ((new_aflags & ATTR_IS_COMPRESSED) && ni->vol->cluster_size > 4096) {
+ err = -EOPNOTSUPP;
+ goto err_out;
+ }
+
if (!a->non_resident) {
if (!(new_aflags & (ATTR_IS_SPARSE | ATTR_IS_COMPRESSED))) {
err = 0;
diff --git a/fs/ntfs3/xattr.c b/fs/ntfs3/xattr.c
index 594ef6860b93..fb7276d4a428 100644
--- a/fs/ntfs3/xattr.c
+++ b/fs/ntfs3/xattr.c
@@ -146,29 +146,32 @@ static int ntfs_read_ea(struct ntfs_inode *ni, struct EA_FULL **ea,
for (off = 0; off < size; off += ea_size) {
const struct EA_FULL *ef = Add2Ptr(ea_p, off);
u32 bytes = size - off;
- size_t need;
-
- /* Check if we can use field ea->size. */
- if (bytes < sizeof(ef->size))
- goto out1;
+ u32 packed_size;
/* Check if we can use fields ef->name_len and ef->elength. */
if (bytes < offsetof(struct EA_FULL, name))
goto out1;
- /* Size needed to hold this record's name and value. */
- need = struct_size(ef, name,
- 1 + ef->name_len + le16_to_cpu(ef->elength));
+ /*
+ * The name and the value have to fit in the entry whether or
+ * not ef->size is set, because ntfs_get_ea() copies elength
+ * bytes out of this buffer and checks them only against the
+ * caller supplied buffer size.
+ */
+ packed_size =
+ ALIGN(struct_size(ef, name,
+ 1 + ef->name_len +
+ le16_to_cpu(ef->elength)),
+ 4);
if (ef->size) {
ea_size = le32_to_cpu(ef->size);
- /* ef->size must fit the list and cover the record. */
- if (ea_size > bytes || ea_size < need)
+ if (ea_size > bytes || packed_size > ea_size)
goto out1;
continue;
}
- ea_size = ALIGN(need, 4);
+ ea_size = packed_size;
if (ea_size > bytes)
goto out1;
}
diff --git a/fs/ocfs2/cluster/tcp.c b/fs/ocfs2/cluster/tcp.c
index 474fe1414cee..0ba26bd2eba1 100644
--- a/fs/ocfs2/cluster/tcp.c
+++ b/fs/ocfs2/cluster/tcp.c
@@ -391,6 +391,9 @@ static void sc_kref_release(struct kref *kref)
sclog(sc, "releasing\n");
+ /* Unlink before freeing anything the debugfs readers dereference. */
+ o2net_debug_del_sc(sc);
+
if (sc->sc_sock) {
sock_release(sc->sc_sock);
sc->sc_sock = NULL;
@@ -400,8 +403,6 @@ static void sc_kref_release(struct kref *kref)
o2nm_node_put(sc->sc_node);
sc->sc_node = NULL;
- o2net_debug_del_sc(sc);
-
if (sc->sc_page)
__free_page(sc->sc_page);
kfree(sc);
diff --git a/fs/ocfs2/dcache.h b/fs/ocfs2/dcache.h
index 7f246c5692d8..046ab9a80769 100644
--- a/fs/ocfs2/dcache.h
+++ b/fs/ocfs2/dcache.h
@@ -17,9 +17,8 @@ struct ocfs2_dentry_lock {
u64 dl_parent_blkno;
/*
- * The ocfs2_dentry_lock keeps an inode reference until
- * dl_lockres has been destroyed. This is usually done in
- * ->d_iput() anyway, so there should be minimal impact.
+ * The inode reference is dropped before dl_lockres is unlocked.
+ * Callbacks must not use it to find the mount context during teardown.
*/
struct inode *dl_inode;
struct ocfs2_lock_res dl_lockres;
diff --git a/fs/ocfs2/dlm/dlmmaster.c b/fs/ocfs2/dlm/dlmmaster.c
index 612969867ff9..ea48c013f7fb 100644
--- a/fs/ocfs2/dlm/dlmmaster.c
+++ b/fs/ocfs2/dlm/dlmmaster.c
@@ -2525,6 +2525,52 @@ static int dlm_is_lockres_migratable(struct dlm_ctxt *dlm,
*/
+/*
+ * Drop the sender's references on a migration MLE after the attempt has
+ * failed. The finishing assert from the target may already have unlinked
+ * the MLE and consumed its base reference; decide under the locks the
+ * assert handler holds, and only drop the base reference while the MLE is
+ * still hashed. Returns true when the assert got there first: the target
+ * has become the master, so the caller must complete the migration.
+ */
+static bool dlm_put_migration_mle(struct dlm_ctxt *dlm,
+ struct dlm_master_list_entry *mle)
+{
+ bool asserted;
+
+ spin_lock(&dlm->spinlock);
+ spin_lock(&dlm->master_lock);
+ __dlm_mle_detach_hb_events(dlm, mle);
+ asserted = hlist_unhashed(&mle->master_hash_node);
+ if (!asserted) {
+ __dlm_unlink_mle(dlm, mle);
+ __dlm_put_mle(mle);
+ }
+ spin_unlock(&dlm->master_lock);
+ spin_unlock(&dlm->spinlock);
+
+ dlm_put_mle_inuse(mle);
+ return asserted;
+}
+
+/*
+ * The finishing assert got there first: it switched the owner to the
+ * target and cleared DLM_LOCK_RES_MIGRATING. Drop the other nodes' locks
+ * as a completed migration does, unless recovery has taken the lockres
+ * over since, and let the usage be recalculated.
+ */
+static void dlm_migrate_lockres_asserted(struct dlm_ctxt *dlm,
+ struct dlm_lock_resource *res,
+ u8 target)
+{
+ spin_lock(&res->spinlock);
+ if (res->owner == target)
+ dlm_remove_nonlocal_locks(dlm, res);
+ spin_unlock(&res->spinlock);
+
+ dlm_lockres_calc_usage(dlm, res);
+}
+
static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
struct dlm_lock_resource *res, u8 target)
{
@@ -2609,9 +2655,10 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
if (ret < 0) {
if (mle_added) {
- dlm_mle_detach_hb_events(dlm, mle);
- dlm_put_mle(mle);
- dlm_put_mle_inuse(mle);
+ if (dlm_put_migration_mle(dlm, mle)) {
+ dlm_migrate_lockres_asserted(dlm, res, target);
+ ret = 0;
+ }
} else if (mle) {
kmem_cache_free(dlm_mle_cache, mle);
mle = NULL;
@@ -2640,9 +2687,11 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
mlog(0, "migration to node %u failed with %d\n",
target, ret);
/* migration failed, detach and clean up mle */
- dlm_mle_detach_hb_events(dlm, mle);
- dlm_put_mle(mle);
- dlm_put_mle_inuse(mle);
+ if (dlm_put_migration_mle(dlm, mle)) {
+ dlm_migrate_lockres_asserted(dlm, res, target);
+ ret = 0;
+ goto leave;
+ }
spin_lock(&res->spinlock);
res->state &= ~DLM_LOCK_RES_MIGRATING;
wake = 1;
@@ -2686,9 +2735,12 @@ static int dlm_migrate_lockres(struct dlm_ctxt *dlm,
res->lockname.name, target);
ret = -EINVAL;
/* migration failed, detach and clean up mle */
- dlm_mle_detach_hb_events(dlm, mle);
- dlm_put_mle(mle);
- dlm_put_mle_inuse(mle);
+ if (dlm_put_migration_mle(dlm, mle)) {
+ dlm_migrate_lockres_asserted(dlm, res,
+ target);
+ ret = 0;
+ goto leave;
+ }
spin_lock(&res->spinlock);
res->state &= ~DLM_LOCK_RES_MIGRATING;
wake = 1;
diff --git a/fs/ocfs2/dlmglue.c b/fs/ocfs2/dlmglue.c
index 1a83241b6d41..a2a33c34fdf9 100644
--- a/fs/ocfs2/dlmglue.c
+++ b/fs/ocfs2/dlmglue.c
@@ -57,7 +57,6 @@ struct ocfs2_mask_waiter {
#endif
};
-static struct ocfs2_super *ocfs2_get_dentry_osb(struct ocfs2_lock_res *lockres);
static struct ocfs2_super *ocfs2_get_inode_osb(struct ocfs2_lock_res *lockres);
static struct ocfs2_super *ocfs2_get_file_osb(struct ocfs2_lock_res *lockres);
static struct ocfs2_super *ocfs2_get_qinfo_osb(struct ocfs2_lock_res *lockres);
@@ -257,7 +256,6 @@ static const struct ocfs2_lock_res_ops ocfs2_orphan_scan_lops = {
};
static const struct ocfs2_lock_res_ops ocfs2_dentry_lops = {
- .get_osb = ocfs2_get_dentry_osb,
.post_unlock = ocfs2_dentry_post_unlock,
.downconvert_worker = ocfs2_dentry_convert_worker,
.flags = 0,
@@ -308,7 +306,7 @@ static inline struct ocfs2_dentry_lock *ocfs2_lock_res_dl(struct ocfs2_lock_res
{
BUG_ON(lockres->l_type != OCFS2_LOCK_TYPE_DENTRY);
- return (struct ocfs2_dentry_lock *)lockres->l_priv;
+ return container_of(lockres, struct ocfs2_dentry_lock, dl_lockres);
}
static inline struct ocfs2_mem_dqinfo *ocfs2_lock_res_qinfo(struct ocfs2_lock_res *lockres)
@@ -609,13 +607,6 @@ static __u64 ocfs2_get_dentry_lock_ino(struct ocfs2_lock_res *lockres)
return be64_to_cpu(inode_blkno_be);
}
-static struct ocfs2_super *ocfs2_get_dentry_osb(struct ocfs2_lock_res *lockres)
-{
- struct ocfs2_dentry_lock *dl = lockres->l_priv;
-
- return OCFS2_SB(dl->dl_inode->i_sb);
-}
-
void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl,
u64 parent, struct inode *inode)
{
@@ -623,6 +614,7 @@ void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl,
u64 inode_blkno = OCFS2_I(inode)->ip_blkno;
__be64 inode_blkno_be = cpu_to_be64(inode_blkno);
struct ocfs2_lock_res *lockres = &dl->dl_lockres;
+ struct ocfs2_super *osb = OCFS2_SB(inode->i_sb);
ocfs2_lock_res_init_once(lockres);
@@ -651,9 +643,9 @@ void ocfs2_dentry_lock_res_init(struct ocfs2_dentry_lock *dl,
memcpy(&lockres->l_name[OCFS2_DENTRY_LOCK_INO_START], &inode_blkno_be,
sizeof(__be64));
- ocfs2_lock_res_init_common(OCFS2_SB(inode->i_sb), lockres,
+ ocfs2_lock_res_init_common(osb, lockres,
OCFS2_LOCK_TYPE_DENTRY, &ocfs2_dentry_lops,
- dl);
+ osb);
}
static void ocfs2_super_lock_res_init(struct ocfs2_lock_res *res,
@@ -3335,6 +3327,14 @@ static void ocfs2_dlm_shutdown_debug(struct ocfs2_super *osb)
{
struct ocfs2_dlm_debug *dlm_debug = osb->osb_dlm_debug;
+ /*
+ * Drain the readers before the cluster stack goes away: the
+ * locking_state reader resolves ocfs2_dlm_lvb() through the
+ * stack. Every caller removes the parent directory later.
+ */
+ debugfs_lookup_and_remove("locking_state", osb->osb_debug_root);
+ debugfs_lookup_and_remove("locking_filter", osb->osb_debug_root);
+
if (dlm_debug)
ocfs2_put_dlm_debug(dlm_debug);
}
@@ -3349,8 +3349,6 @@ int ocfs2_dlm_init(struct ocfs2_super *osb)
goto local;
}
- ocfs2_dlm_init_debug(osb);
-
/* launch downconvert thread */
osb->dc_task = kthread_run(ocfs2_downconvert_thread, osb, "ocfs2dc-%s",
osb->uuid_str);
@@ -3390,9 +3388,10 @@ int ocfs2_dlm_init(struct ocfs2_super *osb)
ocfs2_orphan_scan_lock_res_init(&osb->osb_orphan_scan.os_lockres, osb);
osb->cconn = conn;
+ if (conn)
+ ocfs2_dlm_init_debug(osb);
bail:
if (status < 0) {
- ocfs2_dlm_shutdown_debug(osb);
if (osb->dc_task)
kthread_stop(osb->dc_task);
}
@@ -3422,10 +3421,11 @@ void ocfs2_dlm_shutdown(struct ocfs2_super *osb,
ocfs2_lock_res_free(&osb->osb_orphan_scan.os_lockres);
if (osb->cconn) {
+ /* The disconnect below may release the selected stack. */
+ ocfs2_dlm_shutdown_debug(osb);
+
ocfs2_cluster_disconnect(osb->cconn, hangup_pending);
osb->cconn = NULL;
-
- ocfs2_dlm_shutdown_debug(osb);
}
}
diff --git a/fs/ocfs2/filecheck.c b/fs/ocfs2/filecheck.c
index 75334e4bd082..e3b79bc749f5 100644
--- a/fs/ocfs2/filecheck.c
+++ b/fs/ocfs2/filecheck.c
@@ -189,6 +189,8 @@ int ocfs2_filecheck_create_sysfs(struct ocfs2_super *osb)
/* Mount failure still calls ocfs2_filecheck_remove_sysfs(). */
entry->fs_fcheck = NULL;
kobject_put(&entry->fs_kobj);
+ /* Keep ocfs2_super alive until the kobject is released. */
+ wait_for_completion(&entry->fs_kobj_unregister);
kfree(fcheck);
return ret;
}
diff --git a/fs/ocfs2/journal.c b/fs/ocfs2/journal.c
index a3938a03e93b..674e70dfbb5c 100644
--- a/fs/ocfs2/journal.c
+++ b/fs/ocfs2/journal.c
@@ -227,13 +227,22 @@ void ocfs2_recovery_disable_quota(struct ocfs2_super *osb)
ocfs2_recovery_disable(osb, OCFS2_REC_QUOTA_WANT_DISABLE);
}
+/* Stop producers and drain completion work without freeing recovery state. */
+void ocfs2_recovery_stop(struct ocfs2_super *osb)
+{
+ ocfs2_recovery_disable(osb, OCFS2_REC_WANT_DISABLE);
+}
+
void ocfs2_recovery_exit(struct ocfs2_super *osb)
{
struct ocfs2_recovery_map *rm;
- /* disable any new recovery threads and wait for any currently
- * running ones to exit. Do this before setting the vol_state. */
- ocfs2_recovery_disable(osb, OCFS2_REC_WANT_DISABLE);
+ /*
+ * Disable producers and drain work before freeing recovery state.
+ * Do this before setting vol_state to VOLUME_DISMOUNTED: completion
+ * work may still be waiting in __ocfs2_wait_on_mount().
+ */
+ ocfs2_recovery_stop(osb);
/*
* Now that recovery is shut down, and the osb is about to be
diff --git a/fs/ocfs2/journal.h b/fs/ocfs2/journal.h
index 19fc920d26b1..4b52bf7b0938 100644
--- a/fs/ocfs2/journal.h
+++ b/fs/ocfs2/journal.h
@@ -147,6 +147,7 @@ void ocfs2_complete_recovery(struct work_struct *work);
void ocfs2_wait_for_recovery(struct ocfs2_super *osb);
int ocfs2_recovery_init(struct ocfs2_super *osb);
+void ocfs2_recovery_stop(struct ocfs2_super *osb);
void ocfs2_recovery_exit(struct ocfs2_super *osb);
void ocfs2_recovery_disable_quota(struct ocfs2_super *osb);
diff --git a/fs/ocfs2/move_extents.c b/fs/ocfs2/move_extents.c
index ad1678ee7cc4..93b45531bf7d 100644
--- a/fs/ocfs2/move_extents.c
+++ b/fs/ocfs2/move_extents.c
@@ -907,6 +907,30 @@ static int __ocfs2_move_extents_range(struct buffer_head *di_bh,
return ret;
}
+static int ocfs2_validate_and_adjust_move_range(struct inode *inode,
+ struct ocfs2_move_extents *range)
+{
+ u64 i_size = i_size_read(inode);
+
+ if (range->me_start > i_size)
+ return -EINVAL;
+
+ if (range->me_len > i_size - range->me_start)
+ range->me_len = i_size - range->me_start;
+
+ /*
+ * The default threshold for defragmentation is 1 MiB, which
+ * matches the maximum OCFS2 cluster size.
+ */
+ if (!range->me_threshold)
+ range->me_threshold = 1024 * 1024;
+
+ if (range->me_threshold > i_size)
+ range->me_threshold = i_size;
+
+ return 0;
+}
+
static int ocfs2_move_extents(struct ocfs2_move_extents_context *context)
{
int status;
@@ -936,6 +960,21 @@ static int ocfs2_move_extents(struct ocfs2_move_extents_context *context)
goto out_rw_unlock;
}
+ status = ocfs2_validate_and_adjust_move_range(inode, context->range);
+ if (status) {
+ mlog_errno(status);
+ goto out_inode_unlock;
+ }
+
+ /* The goal check depends on the adjusted range length. */
+ if (!context->auto_defrag) {
+ status = ocfs2_validate_and_adjust_move_goal(inode, context->range);
+ if (status) {
+ mlog_errno(status);
+ goto out_inode_unlock;
+ }
+ }
+
/*
* remember ip_xattr_sem also needs to be held if necessary
*/
@@ -1028,27 +1067,8 @@ int ocfs2_ioctl_move_extents(struct file *filp, void __user *argp)
goto out_free;
}
- if (range.me_start > i_size_read(inode)) {
- status = -EINVAL;
- goto out_free;
- }
-
- if (range.me_start + range.me_len > i_size_read(inode))
- range.me_len = i_size_read(inode) - range.me_start;
-
context->range = ⦥
- /*
- * ok, the default threshold for the defragmentation
- * is 1M, since our maximum clustersize was 1M also.
- * any thought?
- */
- if (!range.me_threshold)
- range.me_threshold = 1024 * 1024;
-
- if (range.me_threshold > i_size_read(inode))
- range.me_threshold = i_size_read(inode);
-
if (range.me_flags & ~(OCFS2_MOVE_EXT_FL_AUTO_DEFRAG |
OCFS2_MOVE_EXT_FL_PART_DEFRAG)) {
status = -EINVAL;
@@ -1060,23 +1080,11 @@ int ocfs2_ioctl_move_extents(struct file *filp, void __user *argp)
if (range.me_flags & OCFS2_MOVE_EXT_FL_PART_DEFRAG)
context->partial = 1;
- } else {
- /*
- * first best-effort attempt to validate and adjust the goal
- * (physical address in block), while it can't guarantee later
- * operation can succeed all the time since global_bitmap may
- * change a bit over time.
- */
-
- status = ocfs2_validate_and_adjust_move_goal(inode, &range);
- if (status)
- goto out_copy;
}
status = ocfs2_move_extents(context);
if (status)
mlog_errno(status);
-out_copy:
/*
* movement/defragmentation may end up being partially completed,
* that's the reason why we need to return userspace the finished
diff --git a/fs/ocfs2/super.c b/fs/ocfs2/super.c
index 169c576f06bf..e5482c556552 100644
--- a/fs/ocfs2/super.c
+++ b/fs/ocfs2/super.c
@@ -1828,6 +1828,9 @@ static void ocfs2_dismount_volume(struct super_block *sb, int mnt_err)
/* Wait for worker to be done with the work structure in osb */
cancel_work_sync(&osb->dquot_drop_work);
+ /* Stop recovery producers before releasing their local allocators. */
+ ocfs2_recovery_stop(osb);
+
ocfs2_shutdown_local_alloc(osb);
ocfs2_truncate_log_shutdown(osb);
diff --git a/fs/ocfs2/xattr.c b/fs/ocfs2/xattr.c
index 18ebe3cc64ec..4d00a26ece3c 100644
--- a/fs/ocfs2/xattr.c
+++ b/fs/ocfs2/xattr.c
@@ -260,6 +260,7 @@ static int ocfs2_xattr_tree_list_index_block(struct inode *inode,
size_t buffer_size);
static int ocfs2_xattr_create_index_block(struct inode *inode,
+ struct ocfs2_xattr_info *xi,
struct ocfs2_xattr_search *xs,
struct ocfs2_xattr_set_ctxt *ctxt);
@@ -3251,7 +3252,7 @@ static int ocfs2_xattr_block_set(struct inode *inode,
else if ((ret != -ENOSPC) || ctxt->set_abort)
goto end;
else {
- ret = ocfs2_xattr_create_index_block(inode, xs, ctxt);
+ ret = ocfs2_xattr_create_index_block(inode, xi, xs, ctxt);
if (ret)
goto end;
}
@@ -4533,6 +4534,21 @@ static void ocfs2_cp_xattr_block_to_bucket(struct inode *inode,
for (i = 0; i < blks; i++)
memset(bucket_block(bucket, i), 0, blocksize);
+ /*
+ * An empty block has no names or values to carry over, and the
+ * xh_entries[count - 1] below would be xh_entries[-1]. Leave the
+ * bucket empty, but carry over the hash ocfs2_xa_remove_entry() left
+ * in the block's xh_entries[0] so that the bucket still holds the key
+ * it is looked up by.
+ */
+ if (!count) {
+ xh->xh_num_buckets = cpu_to_le16(1);
+ xh->xh_free_start = cpu_to_le16(OCFS2_XATTR_BUCKET_SIZE);
+ xh->xh_entries[0].xe_name_hash =
+ xb_xh->xh_entries[0].xe_name_hash;
+ return;
+ }
+
/*
* Since the xe_name_offset is based on ocfs2_xattr_header,
* there is a offset change corresponding to the change of
@@ -4579,13 +4595,10 @@ static void ocfs2_cp_xattr_block_to_bucket(struct inode *inode,
* real place of the xattr.
*/
static void ocfs2_xattr_update_xattr_search(struct inode *inode,
- struct ocfs2_xattr_search *xs,
- struct buffer_head *old_bh)
+ struct ocfs2_xattr_info *xi,
+ struct ocfs2_xattr_search *xs)
{
- char *buf = old_bh->b_data;
- struct ocfs2_xattr_block *old_xb = (struct ocfs2_xattr_block *)buf;
- struct ocfs2_xattr_header *old_xh = &old_xb->xb_attrs.xb_header;
- int i;
+ int i, count;
xs->header = bucket_xh(xs->bucket);
xs->base = bucket_block(xs->bucket, 0);
@@ -4594,11 +4607,47 @@ static void ocfs2_xattr_update_xattr_search(struct inode *inode,
if (xs->not_found)
return;
- i = xs->here - old_xh->xh_entries;
- xs->here = &xs->header->xh_entries[i];
+ /*
+ * A block holds its entries in insertion order while a bucket holds
+ * them by name hash, so ocfs2_cp_xattr_block_to_bucket() sorted the
+ * copies and the index this entry had in the block no longer says
+ * where it is. It copied every entry and every name unchanged, and
+ * (type, name) pairs are unique within a block, so find it again by
+ * name.
+ *
+ * Search for the name being set, not for the name xs->here carries.
+ * A failed set may have removed that entry, and ocfs2_xa_remove_entry()
+ * then memmoves its successor into the slot. Comparing every entry
+ * rather than narrowing on the hash also finds an entry stored under
+ * the legacy signed hash, which ocfs2_find_xe_in_bucket() would skip.
+ */
+ count = le16_to_cpu(xs->header->xh_count);
+ for (i = 0; i < count; i++) {
+ struct ocfs2_xattr_entry *xe = &xs->header->xh_entries[i];
+
+ if (ocfs2_xattr_get_type(xe) != xi->xi_name_index)
+ continue;
+ if (xe->xe_name_len != xi->xi_name_len)
+ continue;
+ if (!memcmp(xs->base + le16_to_cpu(xe->xe_name_offset),
+ xi->xi_name, xi->xi_name_len)) {
+ xs->here = xe;
+ return;
+ }
+ }
+
+ /*
+ * The entry did not survive the failed set, so this has become an
+ * insert. Point xs->here back inside the bucket: it still refers to
+ * the block being converted, which is about to be rewritten as a tree
+ * root.
+ */
+ xs->not_found = -ENODATA;
+ xs->here = xs->header->xh_entries;
}
static int ocfs2_xattr_create_index_block(struct inode *inode,
+ struct ocfs2_xattr_info *xi,
struct ocfs2_xattr_search *xs,
struct ocfs2_xattr_set_ctxt *ctxt)
{
@@ -4665,7 +4714,7 @@ static int ocfs2_xattr_create_index_block(struct inode *inode,
ocfs2_cp_xattr_block_to_bucket(inode, xb_bh, xs->bucket);
ocfs2_xattr_bucket_journal_dirty(handle, xs->bucket);
- ocfs2_xattr_update_xattr_search(inode, xs, xb_bh);
+ ocfs2_xattr_update_xattr_search(inode, xi, xs);
/* Change from ocfs2_xattr_header to ocfs2_xattr_tree_root */
memset(&xb->xb_attrs, 0, inode->i_sb->s_blocksize -
@@ -5018,7 +5067,15 @@ static int ocfs2_divide_xattr_bucket(struct inode *inode,
start = ocfs2_xattr_find_divide_pos(xh);
if (start == count) {
- xe = &xh->xh_entries[start-1];
+ /*
+ * start == count means every entry shares one hash, so the
+ * first of them carries it too. An empty bucket has no
+ * entries[start - 1], but its entries[0] still carries the
+ * bucket key: either the hash ocfs2_xa_remove_entry()
+ * deliberately leaves behind, or the sentinel a previous
+ * divide wrote.
+ */
+ xe = &xh->xh_entries[0];
/*
* initialized a new empty bucket here.
@@ -5848,11 +5905,20 @@ static int ocfs2_check_xattr_bucket_collision(struct ocfs2_xattr_bucket *bucket,
u32 name_hash)
{
struct ocfs2_xattr_header *xh = bucket_xh(bucket);
+ u16 count = le16_to_cpu(xh->xh_count);
+
+ /*
+ * An empty bucket keeps the hash of the entry that was removed in
+ * xh_entries[0] so that it can still be found, but it holds no
+ * entries to collide with.
+ */
+ if (!count)
+ return 0;
if (name_hash != le32_to_cpu(xh->xh_entries[0].xe_name_hash))
return 0;
- if (xh->xh_entries[le16_to_cpu(xh->xh_count) - 1].xe_name_hash ==
+ if (xh->xh_entries[count - 1].xe_name_hash ==
xh->xh_entries[0].xe_name_hash) {
mlog(ML_ERROR, "Too much hash collision in xattr bucket %llu, "
"hash = %u\n",
@@ -5873,13 +5939,18 @@ static int ocfs2_xattr_set_entry_bucket(struct inode *inode,
struct ocfs2_xattr_search *xs,
struct ocfs2_xattr_set_ctxt *ctxt)
{
- int ret;
+ int ret, found = 0;
+ u16 xe_index;
+ u32 name_hash;
struct ocfs2_xa_loc loc;
trace_ocfs2_xattr_set_entry_bucket(xi->xi_name);
ocfs2_init_xattr_bucket_xa_loc(&loc, xs->bucket,
xs->not_found ? NULL : xs->here);
+ /* Defragmenting below may move this entry, so remember its hash. */
+ name_hash = loc.xl_entry ? le32_to_cpu(loc.xl_entry->xe_name_hash) : 0;
+
ret = ocfs2_xa_set(&loc, xi, ctxt);
if (!ret) {
xs->here = loc.xl_entry;
@@ -5898,6 +5969,29 @@ static int ocfs2_xattr_set_entry_bucket(struct inode *inode,
goto out;
}
+ /*
+ * ocfs2_defrag_xattr_bucket() sorts the entries by name hash and
+ * lib/sort.c is not stable, so entries sharing a hash may have traded
+ * slots. Look the entry up again before writing through it.
+ */
+ if (loc.xl_entry) {
+ ret = ocfs2_find_xe_in_bucket(inode, xs->bucket,
+ xi->xi_name_index, xi->xi_name,
+ name_hash, &xe_index, &found);
+ if (ret) {
+ mlog_errno(ret);
+ goto out;
+ }
+ if (!found) {
+ mlog(ML_ERROR, "xattr entry not found in bucket %llu after defrag\n",
+ (unsigned long long)bucket_blkno(xs->bucket));
+ ret = -EFSCORRUPTED;
+ goto out;
+ }
+ loc.xl_entry = &bucket_xh(xs->bucket)->xh_entries[xe_index];
+ xs->here = loc.xl_entry;
+ }
+
ret = ocfs2_xa_set(&loc, xi, ctxt);
if (!ret) {
xs->here = loc.xl_entry;
diff --git a/fs/open.c b/fs/open.c
index 6b1c14e684a9..d7d9fcc666ab 100644
--- a/fs/open.c
+++ b/fs/open.c
@@ -1147,6 +1147,7 @@ struct file *dentry_open_nonotify(const struct path *path, int flags,
}
return f;
}
+EXPORT_SYMBOL_GPL(dentry_open_nonotify);
/**
* kernel_file_open - open a file for kernel internal use
diff --git a/fs/overlayfs/dir.c b/fs/overlayfs/dir.c
index f716ac13badf..c77af9a79d88 100644
--- a/fs/overlayfs/dir.c
+++ b/fs/overlayfs/dir.c
@@ -74,39 +74,34 @@ static struct dentry *ovl_start_creating_temp(struct ovl_fs *ofs,
&QSTR(name));
}
+/*
+ * The whiteout is the source of a rename, so it has to be hashed.
+ * ovl_create_temp() takes care of that for a casefolded workdir.
+ */
static struct dentry *ovl_whiteout(struct ovl_fs *ofs)
{
int err;
struct dentry *whiteout, *link;
struct dentry *workdir = ofs->workdir;
- struct inode *wdir = workdir->d_inode;
- char name[OVL_TEMPNAME_SIZE];
guard(mutex)(&ofs->whiteout_lock);
if (!ofs->whiteout) {
- whiteout = ovl_start_creating_temp(ofs, workdir, name);
+ whiteout = ovl_create_temp(ofs, workdir, &(struct ovl_cattr) {
+ .mode = S_IFCHR | WHITEOUT_MODE,
+ .rdev = WHITEOUT_DEV });
if (IS_ERR(whiteout))
return whiteout;
- err = ovl_do_whiteout(ofs, wdir, whiteout);
- if (!err)
- ofs->whiteout = dget(whiteout);
- end_creating(whiteout);
- if (err)
- return ERR_PTR(err);
+ ofs->whiteout = whiteout;
}
if (!ofs->no_shared_whiteout) {
- link = ovl_start_creating_temp(ofs, workdir, name);
- if (IS_ERR(link))
+ link = ovl_create_temp(ofs, workdir, &(struct ovl_cattr) {
+ .hardlink = ofs->whiteout });
+ if (!IS_ERR(link))
return link;
- err = ovl_do_link(ofs, ofs->whiteout, wdir, link);
- if (!err)
- whiteout = dget(link);
- end_creating(link);
- if (!err)
- return whiteout;
+ err = PTR_ERR(link);
if (err != -EMLINK) {
pr_warn("Failed to link whiteout - disabling whiteout inode sharing(nlink=%u, err=%u)\n",
ofs->whiteout->d_inode->i_nlink,
@@ -120,9 +115,10 @@ static struct dentry *ovl_whiteout(struct ovl_fs *ofs)
}
int ovl_cleanup_and_whiteout(struct ovl_fs *ofs, struct dentry *dir,
- struct dentry *dentry)
+ struct dentry *dentry, const struct qstr *name)
{
struct dentry *whiteout;
+ struct qstr last = QSTR_LEN(name->name, name->len);
struct renamedata rd = {};
int err;
int flags = 0;
@@ -139,9 +135,17 @@ int ovl_cleanup_and_whiteout(struct ovl_fs *ofs, struct dentry *dir,
rd.old_parent = ofs->workdir;
rd.new_parent = dir;
rd.flags = flags;
- err = start_renaming_two_dentries(&rd, whiteout, dentry);
+ /*
+ * The caller looked up @dentry by @name without a lock on @dir, and a
+ * casefolded directory leaves a negative dentry unhashed. Nothing tells
+ * if such a dentry is still valid, so look up @name again with @dir locked.
+ */
+ err = start_renaming_dentry(&rd, 0, whiteout, &last);
if (!err) {
- err = ovl_do_rename_rd(&rd);
+ if (d_inode(rd.new_dentry) != d_inode(dentry))
+ err = -ESTALE;
+ else
+ err = ovl_do_rename_rd(&rd);
end_renaming(&rd);
}
if (err)
@@ -847,7 +851,7 @@ static int ovl_remove_and_whiteout(struct dentry *dentry,
goto out_dput_upper;
}
- err = ovl_cleanup_and_whiteout(ofs, upperdir, upper);
+ err = ovl_cleanup_and_whiteout(ofs, upperdir, upper, &dentry->d_name);
if (!err)
ovl_dir_modified(dentry->d_parent, true);
diff --git a/fs/overlayfs/export.c b/fs/overlayfs/export.c
index 9aae20c59608..bd998a3d8b95 100644
--- a/fs/overlayfs/export.c
+++ b/fs/overlayfs/export.c
@@ -713,9 +713,9 @@ static struct dentry *ovl_lower_fh_to_d(struct super_block *sb,
}
}
- /* Then lookup indexed upper/whiteout by origin fh */
+ /* Then lookup indexed upper/whiteout by origin */
if (ovl_indexdir(sb)) {
- index = ovl_get_index_fh(ofs, fh);
+ index = ovl_get_index_origin(ofs, origin.dentry);
err = PTR_ERR(index);
if (IS_ERR(index)) {
index = NULL;
diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index 7e1ac70c5d97..e3aa0931bea9 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -658,6 +658,14 @@ int ovl_fileattr_set(struct mnt_idmap *idmap,
unsigned int flags;
int err;
+ /*
+ * Lookup refuses a directory that does not agree with the casefold
+ * state of the mount. Only FS_IOC_SETFLAGS can ask for a change.
+ */
+ if (S_ISDIR(inode->i_mode) && fa->flags_valid &&
+ !!(fa->flags & FS_CASEFOLD_FL) != OVL_FS(inode->i_sb)->casefold)
+ return -EOPNOTSUPP;
+
err = ovl_copy_up(dentry);
if (!err) {
ovl_path_real(dentry, &upperpath);
diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index c6127edb9be9..64d63b260709 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -781,14 +781,21 @@ int ovl_get_index_name(struct ovl_fs *ofs, struct dentry *origin,
return err;
}
-/* Lookup index by file handle for NFS export */
-struct dentry *ovl_get_index_fh(struct ovl_fs *ofs, struct ovl_fh *fh)
+/*
+ * Lookup index by decoded origin for NFS export.
+ *
+ * The name of an index is the hex of the file handle that copy up encoded.
+ * The bytes of the handle to decode cannot be used as the name, because a
+ * lower fs may decode many handles to the same inode. For example, it may
+ * ignore the fid type, extra fid bytes or a zero generation.
+ */
+struct dentry *ovl_get_index_origin(struct ovl_fs *ofs, struct dentry *origin)
{
struct dentry *index;
struct qstr name;
int err;
- err = ovl_get_index_name_fh(fh, &name);
+ err = ovl_get_index_name(ofs, origin, &name);
if (err)
return ERR_PTR(err);
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 91cdd7e15696..a5aa22cc79a8 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -394,14 +394,6 @@ static inline int ovl_do_rename(struct ovl_fs *ofs, struct dentry *olddir,
return ovl_do_rename_rd(&rd);
}
-static inline int ovl_do_whiteout(struct ovl_fs *ofs,
- struct inode *dir, struct dentry *dentry)
-{
- int err = vfs_whiteout(ovl_upper_mnt_idmap(ofs), dir, dentry);
- pr_debug("whiteout(%pd2) = %i\n", dentry, err);
- return err;
-}
-
static inline struct file *ovl_do_tmpfile(struct ovl_fs *ofs,
struct dentry *dentry, umode_t mode)
{
@@ -537,7 +529,14 @@ void ovl_dir_modified(struct dentry *dentry, bool impurity);
u64 ovl_inode_version_get(struct inode *inode);
bool ovl_is_whiteout(struct dentry *dentry);
bool ovl_path_is_whiteout(struct ovl_fs *ofs, const struct path *path);
-struct file *ovl_path_open(const struct path *path, int flags);
+struct file *ovl_path_open_realfile(const struct file *user_file,
+ const struct path *path, int flags);
+
+static inline struct file *ovl_path_open(const struct path *path, int flags)
+{
+ return ovl_path_open_realfile(NULL, path, flags);
+}
+
int ovl_copy_up_start(struct dentry *dentry, int flags);
void ovl_copy_up_end(struct dentry *dentry);
bool ovl_already_copied_up(struct dentry *dentry, int flags);
@@ -749,7 +748,7 @@ int ovl_verify_index(struct ovl_fs *ofs, struct dentry *index);
int ovl_get_index_name_fh(const struct ovl_fh *fh, struct qstr *name);
int ovl_get_index_name(struct ovl_fs *ofs, struct dentry *origin,
struct qstr *name);
-struct dentry *ovl_get_index_fh(struct ovl_fs *ofs, struct ovl_fh *fh);
+struct dentry *ovl_get_index_origin(struct ovl_fs *ofs, struct dentry *origin);
struct dentry *ovl_lookup_index(struct ovl_fs *ofs, struct dentry *upper,
struct dentry *origin, bool verify);
int ovl_path_next(int idx, struct dentry *dentry, struct path *path,
@@ -897,7 +896,7 @@ static inline void ovl_copyflags(struct inode *from, struct inode *to)
/* dir.c */
extern const struct inode_operations ovl_dir_inode_operations;
int ovl_cleanup_and_whiteout(struct ovl_fs *ofs, struct dentry *dir,
- struct dentry *dentry);
+ struct dentry *dentry, const struct qstr *name);
struct ovl_cattr {
dev_t rdev;
umode_t mode;
diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c
index 80366424b3dc..e23331212a29 100644
--- a/fs/overlayfs/readdir.c
+++ b/fs/overlayfs/readdir.c
@@ -973,7 +973,8 @@ static struct file *ovl_dir_open_realfile(const struct file *file,
const struct path *realpath)
{
with_ovl_creds(file_inode(file)->i_sb)
- return ovl_path_open(realpath, O_RDONLY | (file->f_flags & O_LARGEFILE));
+ return ovl_path_open_realfile(file, realpath,
+ O_RDONLY | (file->f_flags & O_LARGEFILE));
}
/*
@@ -1322,7 +1323,8 @@ int ovl_indexdir_cleanup(struct ovl_fs *ofs)
* Whiteout orphan index to block future open by
* handle after overlay nlink dropped to zero.
*/
- err = ovl_cleanup_and_whiteout(ofs, indexdir, index);
+ err = ovl_cleanup_and_whiteout(ofs, indexdir, index,
+ &QSTR_LEN(p->name, p->len));
} else {
/* Cleanup orphan index entries */
err = ovl_cleanup(ofs, indexdir, index);
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 861b0775f9dd..c06bfe9765e2 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -1366,7 +1366,6 @@ static struct dentry *ovl_get_root(struct super_block *sb,
ovl_dentry_set_flag(OVL_E_CONNECTED, root);
ovl_set_upperdata(d_inode(root));
ovl_inode_init(d_inode(root), &oip, ino, fsid);
- WARN_ON(!!IS_CASEFOLDED(d_inode(root)) != ofs->casefold);
ovl_dentry_init_flags(root, upperdentry, oe, DCACHE_OP_WEAK_REVALIDATE);
/* root keeps a reference of upperdentry */
dget(upperdentry);
@@ -1550,6 +1549,17 @@ static int ovl_fill_super_creds(struct fs_context *fc, struct super_block *sb)
if (!sb->s_root)
goto out_free_oe;
+ /*
+ * The casefold flag of an empty layer root may have changed since the
+ * mount options were parsed. The root inode owns oe by now.
+ */
+ if (!!IS_CASEFOLDED(d_inode(sb->s_root)) != ofs->casefold) {
+ pr_err("casefold flag of the root directory has changed\n");
+ dput(sb->s_root);
+ sb->s_root = NULL;
+ return -EINVAL;
+ }
+
return 0;
out_free_oe:
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index b41f4788e4f0..04103a6741ba 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -654,7 +654,8 @@ bool ovl_path_is_whiteout(struct ovl_fs *ofs, const struct path *path)
ovl_path_check_xwhiteout_xattr(ofs, path);
}
-struct file *ovl_path_open(const struct path *path, int flags)
+struct file *ovl_path_open_realfile(const struct file *user_file,
+ const struct path *path, int flags)
{
struct inode *inode = d_inode(path->dentry);
struct mnt_idmap *real_idmap = mnt_idmap(path->mnt);
@@ -682,6 +683,14 @@ struct file *ovl_path_open(const struct path *path, int flags)
if (inode_owner_or_capable(real_idmap, inode))
flags |= O_NOATIME;
+ /*
+ * Directory opens use dentry_open() here, not backing_file_open().
+ * Inherit FMODE_NONOTIFY from a fanotify permission fd so a listener
+ * that also watches the real filesystem does not deadlock.
+ */
+ if (user_file && FMODE_FSNOTIFY_NONE(user_file->f_mode))
+ return dentry_open_nonotify(path, flags, current_cred());
+
return dentry_open(path, flags, current_cred());
}
@@ -1118,7 +1127,7 @@ static void ovl_cleanup_index(struct dentry *dentry)
} else if (ovl_index_all(dentry->d_sb)) {
/* Whiteout orphan index to block future open by handle */
err = ovl_cleanup_and_whiteout(OVL_FS(dentry->d_sb),
- indexdir, index);
+ indexdir, index, &name);
} else {
/* Cleanup orphan index entries */
err = ovl_cleanup(ofs, indexdir, index);
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index a2db957aa799..2eaa79ab1138 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -3314,7 +3314,7 @@ static long do_pagemap_cmd(struct file *file, unsigned int cmd,
return do_pagemap_scan(mm, arg);
default:
- return -EINVAL;
+ return -ENOIOCTLCMD;
}
}
diff --git a/fs/smb/server/mgmt/user_session.c b/fs/smb/server/mgmt/user_session.c
index 1b30e432ebfd..d131809dd08a 100644
--- a/fs/smb/server/mgmt/user_session.c
+++ b/fs/smb/server/mgmt/user_session.c
@@ -89,6 +89,8 @@ static const char *session_state_string(struct ksmbd_session *session)
static const char *session_user_name(struct ksmbd_session *session)
{
+ if (!session->user)
+ return "(Unauthenticated)";
if (user_guest(session->user))
return "(Guest)";
else if (ksmbd_anonymous_user(session->user))
@@ -98,6 +100,8 @@ static const char *session_user_name(struct ksmbd_session *session)
static const char *session_account_type(struct ksmbd_session *session)
{
+ if (!session->user)
+ return "unauthenticated";
if (user_guest(session->user))
return "guest";
if (ksmbd_anonymous_user(session->user))
diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c
index 3f1ab9a20307..1c8037c85de9 100644
--- a/fs/smb/server/smb2pdu.c
+++ b/fs/smb/server/smb2pdu.c
@@ -3097,9 +3097,10 @@ int smb2_tree_disconnect(struct ksmbd_work *work)
goto err_out;
}
+ err = ksmbd_tree_conn_disconnect(sess, tcon);
+
ksmbd_close_tree_conn_fds(work);
- err = ksmbd_tree_conn_disconnect(sess, tcon);
if (err) {
rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED;
goto err_out;
@@ -3452,7 +3453,8 @@ static noinline int smb2_set_stream_name_xattr(const struct path *path,
if (fp->cdoption == FILE_OPEN_LE) {
if (!strcmp(stream_name, "AFP_AfpInfo") &&
test_share_config_flag(fp->tcon->share_conf,
- KSMBD_SHARE_FLAG_TIME_MACHINE)) {
+ KSMBD_SHARE_FLAG_TIME_MACHINE) &&
+ test_tree_conn_flag(fp->tcon, KSMBD_TREE_CONN_FLAG_WRITABLE)) {
/*
* Synthesize an empty AFP_AfpInfo xattr on first access.
* type=0/creator=0 tells macOS to use the file extension
@@ -3478,6 +3480,9 @@ static noinline int smb2_set_stream_name_xattr(const struct path *path,
return -EBADF;
}
+ if (!test_tree_conn_flag(fp->tcon, KSMBD_TREE_CONN_FLAG_WRITABLE))
+ return -EACCES;
+
rc = ksmbd_vfs_setxattr(idmap, path, xattr_stream_name, NULL, 0, 0, false);
if (rc < 0)
pr_err("Failed to store XATTR stream name :%d\n", rc);
@@ -4427,7 +4432,7 @@ int smb2_open(struct ksmbd_work *work)
file_info = FILE_OPENED;
rc = ksmbd_vfs_getattr(&fp->filp->f_path, &stat);
if (rc)
- goto err_out2;
+ goto err_out1;
goto reconnected_fp;
}
@@ -4439,8 +4444,12 @@ int smb2_open(struct ksmbd_work *work)
goto err_out2;
rc = ksmbd_reopen_durable_fd(work, dh_info.fp);
- if (rc)
+ if (rc) {
+ if (rc == -ENOTCONN)
+ rsp->hdr.Status =
+ STATUS_NETWORK_NAME_DELETED;
goto err_out2;
+ }
fp = dh_info.fp;
@@ -4453,7 +4462,7 @@ int smb2_open(struct ksmbd_work *work)
rc = ksmbd_vfs_getattr(&fp->filp->f_path, &stat);
if (rc)
- goto err_out2;
+ goto err_out1;
goto reconnected_fp;
}
@@ -4503,6 +4512,14 @@ int smb2_open(struct ksmbd_work *work)
goto err_out2;
}
}
+
+ /* MS-FSA 2.1.5.1: delete-on-close requires DELETE access */
+ if (req->CreateOptions & FILE_DELETE_ON_CLOSE_LE &&
+ !(smb_map_generic_desired_access(req->DesiredAccess) &
+ FILE_DELETE_LE)) {
+ rc = -EINVAL;
+ goto err_out2;
+ }
}
if (le32_to_cpu(req->CreateDisposition) >
@@ -4772,6 +4789,13 @@ int smb2_open(struct ksmbd_work *work)
maximal_access = daccess;
}
+ /* MAXIMUM_ALLOWED may not have granted the requested DELETE */
+ if (req->CreateOptions & FILE_DELETE_ON_CLOSE_LE &&
+ !(daccess & FILE_DELETE_LE)) {
+ rc = -EACCES;
+ goto err_out;
+ }
+
open_flags = smb2_create_open_flags(file_present, daccess,
req->CreateDisposition,
&may_flags,
@@ -4874,6 +4898,8 @@ int smb2_open(struct ksmbd_work *work)
if (IS_ERR(fp)) {
fput(filp);
rc = PTR_ERR(fp);
+ if (rc == -ENOTCONN)
+ rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED;
fp = NULL;
goto err_out;
}
@@ -5477,11 +5503,20 @@ int smb2_open(struct ksmbd_work *work)
err_out2:
if (!rc) {
- if (!dh_info.replay)
+ if (!dh_info.replay) {
+ smb2_complete_request_open(work);
rc = ksmbd_update_fstate(&work->sess->file_table, fp,
FP_INITED);
- if (!rc)
+ if (rc == -ENOENT)
+ rsp->hdr.Status = STATUS_NETWORK_NAME_DELETED;
+ if (!rc) {
+ work->request_open = fp;
+ work->request_open_chseq =
+ smb3_hdr_channel_sequence(&req->hdr);
+ }
+ } else {
rc = smb2_set_request_open(work, fp, &req->hdr, false, false);
+ }
if (!rc)
rc = ksmbd_iov_pin_rsp(work, (void *)rsp, iov_len);
}
@@ -5496,8 +5531,7 @@ int smb2_open(struct ksmbd_work *work)
rsp->hdr.Status = STATUS_PRIVILEGE_NOT_HELD;
else
rsp->hdr.Status = STATUS_ACCESS_DENIED;
- }
- else if (rc == -ENOENT)
+ } else if (rc == -ENOENT && !rsp->hdr.Status)
rsp->hdr.Status = STATUS_OBJECT_NAME_INVALID;
else if (rc == -EPERM)
rsp->hdr.Status = STATUS_SHARING_VIOLATION;
@@ -7082,17 +7116,8 @@ static int get_file_stream_info(struct ksmbd_work *work,
streamlen *= 2;
kfree(stream_buf);
file_info->StreamNameLength = cpu_to_le32(streamlen);
- /*
- * stream_name_len is the byte length of the xattr's *name*,
- * not its value -- same class of bug ksmbd_stream_eof()
- * (smb2pdu.c) already fixes for EndOfFile/AllocationSize on
- * a stream handle; this enumeration path needs the same
- * real xattr value length, not the name length reused as a
- * size.
- */
- slen = ksmbd_vfs_casexattr_len(file_mnt_idmap(fp->filp),
- path->dentry, stream_name,
- strlen(stream_name) + 1, NULL);
+ slen = ksmbd_vfs_xattr_len(file_mnt_idmap(fp->filp),
+ path->dentry, stream_name);
ssize = slen < 0 ? 0 : (loff_t)slen;
file_info->StreamSize = cpu_to_le64(ssize);
file_info->StreamAllocationSize = cpu_to_le64(ssize);
diff --git a/fs/smb/server/smbacl.c b/fs/smb/server/smbacl.c
index f8bd8e2ed33e..cc3a450e4cb2 100644
--- a/fs/smb/server/smbacl.c
+++ b/fs/smb/server/smbacl.c
@@ -1470,6 +1470,7 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
int i, found = 0;
unsigned int access_bits = 0, denied = 0;
struct smb_ace *others_ace = NULL;
+ bool or_ace = false;
struct posix_acl_entry *pa_entry;
unsigned int sid_type = SIDOWNER;
unsigned short ace_size;
@@ -1502,6 +1503,8 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
is_owner = uid == from_kuid(&init_user_ns, vfsuid_into_kuid(vfsuid));
if (!pdacl->num_aces) {
+ unsigned int allowed = 0;
+
/*
* An empty (present, zero-ACE) DACL grants no access to
* anyone except the object owner's implicit READ_CONTROL
@@ -1510,9 +1513,14 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
* not let trailing bytes after a zero-ACE DACL become an
* implicit grant.
*/
- if (!is_owner ||
- (*pdaccess & ~(FILE_READ_CONTROL_LE | FILE_WRITE_DAC_LE)))
+ if (is_owner)
+ allowed = READ_CONTROL | WRITE_DAC;
+ if (*pdaccess & FILE_MAXIMAL_ACCESS_LE)
+ requested |= allowed;
+ if (!allowed || requested & ~allowed)
rc = -EACCES;
+ else
+ *pdaccess = cpu_to_le32(requested);
goto err_out;
}
@@ -1597,6 +1605,9 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
found = 1;
break;
}
+ if (!compare_sids(&sid_owner_rights, &ace->sid) &&
+ !(ace->flags & INHERIT_ONLY_ACE))
+ or_ace = true;
if (!compare_sids(&sid_everyone, &ace->sid) ||
!compare_sids(&sid_authusers, &ace->sid))
others_ace = ace;
@@ -1618,7 +1629,10 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
else
continue;
- if (id == uid) {
+ /* A group entry grants only to the group's members */
+ if (id == uid &&
+ (pa_entry->e_tag == ACL_USER ||
+ in_group_p(make_kgid(&init_user_ns, id)))) {
mode_to_access_flags(pa_entry->e_perm,
0777,
&access_bits);
@@ -1638,8 +1652,23 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
if (others_ace) {
ace = others_ace;
} else {
- ksmbd_debug(SMB, "Can't find corresponding sid\n");
- rc = -EACCES;
+ unsigned int allowed = 0;
+
+ /*
+ * No ACE names the owner, who may still read and
+ * change the DACL unless OWNER RIGHTS applies
+ * (MS-DTYP 2.5.3.2).
+ */
+ if (is_owner && !or_ace)
+ allowed = READ_CONTROL | WRITE_DAC;
+ if (*pdaccess & FILE_MAXIMAL_ACCESS_LE)
+ requested |= allowed;
+ if (!allowed || requested & ~allowed) {
+ ksmbd_debug(SMB, "Can't find corresponding sid\n");
+ rc = -EACCES;
+ goto err_out;
+ }
+ *pdaccess = cpu_to_le32(requested);
goto err_out;
}
}
@@ -1665,8 +1694,8 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
}
if (granted & ~access_bits) {
- ksmbd_debug(SMB, "Access denied with winACL, granted : %x, access_req : %x\n",
- granted, le32_to_cpu(ace->access_req));
+ ksmbd_debug(SMB, "Access denied with winACL, requested : %x, allowed : %x\n",
+ granted, access_bits);
rc = -EACCES;
goto err_out;
}
diff --git a/fs/smb/server/vfs.c b/fs/smb/server/vfs.c
index 7453b6e72bd4..674e4fcd80cb 100644
--- a/fs/smb/server/vfs.c
+++ b/fs/smb/server/vfs.c
@@ -902,8 +902,8 @@ ssize_t ksmbd_vfs_listxattr(struct dentry *dentry, char **list)
return size;
}
-static ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap,
- struct dentry *dentry, char *xattr_name)
+ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap,
+ struct dentry *dentry, char *xattr_name)
{
return vfs_getxattr(idmap, dentry, xattr_name, NULL, 0);
}
diff --git a/fs/smb/server/vfs.h b/fs/smb/server/vfs.h
index 53044549571a..7cdd49678667 100644
--- a/fs/smb/server/vfs.h
+++ b/fs/smb/server/vfs.h
@@ -109,6 +109,8 @@ ssize_t ksmbd_vfs_getxattr(struct mnt_idmap *idmap,
struct dentry *dentry,
char *xattr_name,
char **xattr_buf);
+ssize_t ksmbd_vfs_xattr_len(struct mnt_idmap *idmap,
+ struct dentry *dentry, char *xattr_name);
ssize_t ksmbd_vfs_casexattr_len(struct mnt_idmap *idmap,
struct dentry *dentry, char *attr_name,
int attr_name_len, char *actual_name);
diff --git a/fs/smb/server/vfs_cache.c b/fs/smb/server/vfs_cache.c
index 1efa9f814560..38196befa03a 100644
--- a/fs/smb/server/vfs_cache.c
+++ b/fs/smb/server/vfs_cache.c
@@ -698,16 +698,18 @@ static void __ksmbd_close_fd(struct ksmbd_file_table *ft, struct ksmbd_file *fp)
}
/*
- * Drop fp's strong reference on conn (taken in ksmbd_open_fd() /
- * ksmbd_reopen_durable_fd()). Durable fps that reached the
- * scavenger have already had fp->conn cleared by session_fd_check(),
- * in which case there is nothing to drop here.
+ * Drop fp's strong connection references. Durable fps that reached
+ * the scavenger have already had these cleared by session_fd_check().
*/
if (fp->conn) {
atomic_dec(&fp->conn->stats.open_files_count);
ksmbd_conn_put(fp->conn);
fp->conn = NULL;
}
+ if (fp->tcon) {
+ ksmbd_tree_connect_put(fp->tcon);
+ fp->tcon = NULL;
+ }
if (ksmbd_stream_fd(fp))
kfree(fp->stream.name);
@@ -1331,14 +1333,14 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp)
fp->filp = filp;
/*
- * fp owns a strong reference on fp->conn for as long as fp->conn is
- * non-NULL, so session_fd_check() and __ksmbd_close_fd() never
- * dereference a dangling pointer. Paired with ksmbd_conn_put() in
- * session_fd_check() (durable preserve), in __ksmbd_close_fd()
- * (final close), and on the error paths below.
+ * fp owns strong references on fp->conn and fp->tcon for as long as
+ * they are non-NULL. Paired with the puts in session_fd_check()
+ * (durable preserve), in __ksmbd_close_fd() (final close), and on the
+ * error paths below.
*/
fp->conn = ksmbd_conn_get(work->conn);
fp->tcon = work->tcon;
+ atomic_inc(&fp->tcon->refcount);
fp->share_conf = work->tcon->share_conf;
atomic_inc(&fp->share_conf->refcount);
fp->volatile_id = KSMBD_NO_FID;
@@ -1351,7 +1353,13 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp)
goto err_out;
}
- ret = __open_id(&work->sess->file_table, fp, OPEN_ID_TYPE_VOLATILE_ID);
+ down_read(&work->sess->tree_conns_lock);
+ if (work->tcon->t_state == TREE_CONNECTED)
+ ret = __open_id(&work->sess->file_table, fp,
+ OPEN_ID_TYPE_VOLATILE_ID);
+ else
+ ret = -ENOTCONN;
+ up_read(&work->sess->tree_conns_lock);
if (ret) {
ksmbd_inode_put(fp->f_ci);
goto err_out;
@@ -1361,7 +1369,8 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp)
return fp;
err_out:
- /* fp->conn was set and refcounted before every branch here. */
+ /* fp->conn and fp->tcon were refcounted before every branch here. */
+ ksmbd_tree_connect_put(fp->tcon);
ksmbd_conn_put(fp->conn);
ksmbd_share_config_put(fp->share_conf);
kmem_cache_free(filp_cache, fp);
@@ -1376,8 +1385,9 @@ struct ksmbd_file *ksmbd_open_fd(struct ksmbd_work *work, struct file *filp)
*
* Return: 0 on success. The FP_NEW -> FP_INITED transition is special:
* -ENOENT if teardown already unpublished @fp by advancing the state or
- * clearing the volatile id. Other state updates preserve the historical
- * fire-and-forget behavior.
+ * clearing the volatile id. On success it also takes a reference for the
+ * opening request, which must be released with ksmbd_fd_put(). Other state
+ * updates preserve the historical fire-and-forget behavior.
*/
int ksmbd_update_fstate(struct ksmbd_file_table *ft, struct ksmbd_file *fp,
unsigned int state)
@@ -1393,6 +1403,8 @@ int ksmbd_update_fstate(struct ksmbd_file_table *ft, struct ksmbd_file *fp,
ret = -ENOENT;
} else {
fp->f_state = state;
+ if (state == FP_INITED)
+ atomic_inc(&fp->refcount);
ret = 0;
}
write_unlock(&ft->lock);
@@ -1813,6 +1825,7 @@ static bool session_fd_check(struct ksmbd_tree_connect *tcon,
struct oplock_info *op;
struct ksmbd_conn *conn;
struct ksmbd_lock *smb_lock, *tmp_lock;
+ struct ksmbd_tree_connect *tree_conn;
if (!is_reconnectable(fp))
return false;
@@ -1834,6 +1847,7 @@ static bool session_fd_check(struct ksmbd_tree_connect *tcon,
ci = fp->f_ci;
down_write(&ci->m_lock);
conn = fp->conn;
+ tree_conn = fp->tcon;
list_for_each_entry_rcu(op, &ci->m_op_list, op_entry,
lockdep_is_held(&ci->m_lock)) {
if (op->o_fp != fp)
@@ -1869,6 +1883,7 @@ static bool session_fd_check(struct ksmbd_tree_connect *tcon,
/* Drop fp's own reference on conn. */
ksmbd_conn_put(conn);
+ ksmbd_tree_connect_put(tree_conn);
return true;
}
@@ -1972,7 +1987,8 @@ int ksmbd_reopen_durable_fd(struct ksmbd_work *work, struct ksmbd_file *fp)
* concurrent teardown that iterates the table can observe a valid
* volatile_id with fp->conn == NULL and preserve a
* partially-initialized fp. fp owns a strong reference on the new
- * conn (see ksmbd_open_fd()); undo it on __open_id() failure.
+ * conn (see ksmbd_open_fd()); undo it on __open_id() failure. Serialize
+ * publication with tree disconnect just as ksmbd_open_fd() does.
*/
spin_lock(&fp->f_lock);
old_f_state = fp->f_state;
@@ -1982,12 +1998,19 @@ int ksmbd_reopen_durable_fd(struct ksmbd_work *work, struct ksmbd_file *fp)
atomic_inc(&fp->share_conf->refcount);
fp->conn = ksmbd_conn_get(conn);
fp->tcon = work->tcon;
+ atomic_inc(&fp->tcon->refcount);
spin_unlock(&fp->f_lock);
atomic_inc(&conn->stats.open_files_count);
write_unlock(&global_ft.lock);
- __open_id(&work->sess->file_table, fp, OPEN_ID_TYPE_VOLATILE_ID);
- if (!has_file_id(fp->volatile_id)) {
+ down_read(&work->sess->tree_conns_lock);
+ if (work->tcon->t_state == TREE_CONNECTED)
+ ret = __open_id(&work->sess->file_table, fp,
+ OPEN_ID_TYPE_VOLATILE_ID);
+ else
+ ret = -ENOTCONN;
+ up_read(&work->sess->tree_conns_lock);
+ if (ret) {
write_lock(&global_ft.lock);
spin_lock(&fp->f_lock);
fp->conn = NULL;
@@ -1998,7 +2021,10 @@ int ksmbd_reopen_durable_fd(struct ksmbd_work *work, struct ksmbd_file *fp)
atomic_dec(&conn->stats.open_files_count);
write_unlock(&global_ft.lock);
ksmbd_conn_put(conn);
+ ksmbd_tree_connect_put(work->tcon);
ksmbd_share_config_put(work->tcon->share_conf);
+ if (ret != -ENOTCONN)
+ ret = -EBADF;
goto out;
}
diff --git a/fs/udf/balloc.c b/fs/udf/balloc.c
index 30cec5600149..75124188e745 100644
--- a/fs/udf/balloc.c
+++ b/fs/udf/balloc.c
@@ -457,6 +457,7 @@ static void udf_table_free_blocks(struct super_block *sb,
int adsize;
+ eloc.partitionReferenceNum = bloc->partitionReferenceNum;
eloc.logicalBlockNum = start;
elen = EXT_RECORDED_ALLOCATED |
(count << sb->s_blocksize_bits);
@@ -532,7 +533,7 @@ static int udf_table_prealloc_blocks(struct super_block *sb,
eloc.logicalBlockNum, elen, first_block);
}
- if (first_block == eloc.logicalBlockNum) {
+ if (ret > 0 && first_block == eloc.logicalBlockNum) {
epos.offset -= adsize;
alloc_count = (elen >> sb->s_blocksize_bits);
@@ -572,7 +573,7 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb,
uint32_t elen, goal_elen = 0;
struct kernel_lb_addr eloc, goal_eloc;
struct extent_position epos, goal_epos;
- int8_t etype;
+ int8_t etype, goal_etype = 0;
struct udf_inode_info *iinfo = UDF_I(table);
int ret = 0;
@@ -623,7 +624,8 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb,
goal_epos.block = epos.block;
goal_epos.offset = epos.offset - adsize;
goal_eloc = eloc;
- goal_elen = (etype << 30) | elen;
+ goal_elen = elen;
+ goal_etype = etype;
}
}
@@ -647,7 +649,8 @@ static udf_pblk_t udf_table_new_block(struct super_block *sb,
goal_elen -= sb->s_blocksize;
if (goal_elen)
- udf_write_aext(table, &goal_epos, &goal_eloc, goal_elen, 1);
+ udf_write_aext(table, &goal_epos, &goal_eloc,
+ (goal_etype << 30) | goal_elen, 1);
else
udf_delete_aext(table, goal_epos, &freed);
brelse(goal_epos.bh);
diff --git a/fs/udf/super.c b/fs/udf/super.c
index 5351755aca3e..7570506ff2dd 100644
--- a/fs/udf/super.c
+++ b/fs/udf/super.c
@@ -1106,6 +1106,73 @@ static int check_partition_desc(struct super_block *sb,
return 0;
}
+/*
+ * Check the Unallocated Space Table once when it is loaded: the allocator
+ * hands out blocks from the start of each extent and trusts that every
+ * extent covers at least one whole block inside the partition.
+ */
+static int udf_check_unalloc_table(struct super_block *sb,
+ struct inode *table, u32 partition_len)
+{
+ struct extent_position epos = {
+ .block = UDF_I(table)->i_location,
+ .offset = sizeof(struct unallocSpaceEntry),
+ };
+ struct kernel_lb_addr eloc;
+ uint32_t elen, blocks;
+ struct kernel_lb_addr seen_block = {};
+ uint32_t seen_offset = 0;
+ u64 steps = 0, period = 1;
+ int8_t etype;
+ int ret;
+
+ /* The walk below assumes the layout of an Unallocated Space Entry */
+ if (!UDF_I(table)->i_use) {
+ udf_err(sb, "unallocated space table is not an unallocated space entry\n");
+ return -EFSCORRUPTED;
+ }
+
+ while ((ret = udf_next_aext(table, &epos, &eloc, &elen, &etype, 1)) > 0) {
+ blocks = elen >> sb->s_blocksize_bits;
+ if (!blocks || (elen & (sb->s_blocksize - 1)) ||
+ eloc.logicalBlockNum >= partition_len ||
+ blocks > partition_len - eloc.logicalBlockNum) {
+ udf_err(sb, "invalid unallocated space table extent (block %u, length %u)\n",
+ eloc.logicalBlockNum, elen);
+ ret = -EFSCORRUPTED;
+ break;
+ }
+ /*
+ * A chain of allocation extents can lead back to itself, and
+ * then the walk returns the same extents forever. Remember a
+ * position at doubling intervals; a walk that comes back to it
+ * is in a loop.
+ */
+ if (epos.block.logicalBlockNum == seen_block.logicalBlockNum &&
+ epos.block.partitionReferenceNum ==
+ seen_block.partitionReferenceNum &&
+ epos.offset == seen_offset) {
+ udf_err(sb, "unallocated space table loops back on itself\n");
+ ret = -EFSCORRUPTED;
+ break;
+ }
+ if (++steps == period) {
+ seen_block = epos.block;
+ seen_offset = epos.offset;
+ steps = 0;
+ period <<= 1;
+ }
+ if (fatal_signal_pending(current)) {
+ udf_err(sb, "interrupted while checking the unallocated space table\n");
+ ret = -EINTR;
+ break;
+ }
+ cond_resched();
+ }
+ brelse(epos.bh);
+ return ret;
+}
+
static int udf_fill_partdesc_info(struct super_block *sb,
struct partitionDesc *p, int p_index)
{
@@ -1166,6 +1233,16 @@ static int udf_fill_partdesc_info(struct super_block *sb,
p_index);
return PTR_ERR(inode);
}
+ err = udf_check_unalloc_table(sb, inode, map->s_partition_len);
+ if (err) {
+ iput(inode);
+ if (err == -EINTR)
+ return err;
+ if (!sb_rdonly(sb))
+ return -EACCES;
+ UDF_SET_FLAG(sb, UDF_FLAG_RW_INCOMPAT);
+ return 0;
+ }
map->s_uspace.s_table = inode;
map->s_partition_flags |= UDF_PART_FLAG_UNALLOC_TABLE;
udf_debug("unallocSpaceTable (part %d) @ %llu\n",
@@ -2233,8 +2310,9 @@ static int udf_fill_super(struct super_block *sb, struct fs_context *fc)
/*
* EACCES is special - we want to propagate to
* upper layers that we cannot handle RW mount.
+ * EINTR means that a fatal signal is pending.
*/
- if (ret == -EACCES)
+ if (ret == -EACCES || ret == -EINTR)
break;
} else
break;
diff --git a/fs/udf/truncate.c b/fs/udf/truncate.c
index 0990f94b8551..75151a59f0da 100644
--- a/fs/udf/truncate.c
+++ b/fs/udf/truncate.c
@@ -124,8 +124,8 @@ void udf_discard_prealloc(struct inode *inode)
{
struct extent_position epos = {};
struct extent_position prev_epos = {};
- struct kernel_lb_addr eloc;
- uint32_t elen;
+ struct kernel_lb_addr eloc, tmpeloc;
+ uint32_t elen, tmpelen;
uint64_t lbcount = 0;
int8_t etype = -1;
struct udf_inode_info *iinfo = UDF_I(inode);
@@ -141,7 +141,8 @@ void udf_discard_prealloc(struct inode *inode)
/* Find the last extent in the file */
while (1) {
- ret = udf_next_aext(inode, &epos, &eloc, &elen, &tmpetype, 0);
+ ret = udf_next_aext(inode, &epos, &tmpeloc, &tmpelen,
+ &tmpetype, 0);
if (ret < 0)
goto out;
if (ret == 0)
diff --git a/fs/xfs/libxfs/xfs_attr_leaf.c b/fs/xfs/libxfs/xfs_attr_leaf.c
index 02ca50442b46..5b1fd6271420 100644
--- a/fs/xfs/libxfs/xfs_attr_leaf.c
+++ b/fs/xfs/libxfs/xfs_attr_leaf.c
@@ -3042,7 +3042,14 @@ xfs_attr3_leaf_setflag(
#endif
entry = &xfs_attr3_leaf_entryp(leaf)[args->index];
- ASSERT((entry->flags & XFS_ATTR_INCOMPLETE) == 0);
+ if (entry->flags & XFS_ATTR_INCOMPLETE) {
+ if (!(args->op_flags & XFS_DA_OP_RECOVERY)) {
+ ASSERT(args->op_flags & XFS_DA_OP_RECOVERY);
+ return -EFSCORRUPTED;
+ }
+ return 0;
+ }
+
entry->flags |= XFS_ATTR_INCOMPLETE;
xfs_trans_log_buf(args->trans, bp,
XFS_DA_LOGRANGE(leaf, entry, sizeof(*entry)));
diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
index d64defeda645..fce2705c67d2 100644
--- a/fs/xfs/libxfs/xfs_bmap.c
+++ b/fs/xfs/libxfs/xfs_bmap.c
@@ -4128,6 +4128,8 @@ xfs_bmapi_finish(
* Returns a negative error code on failure, including -ENOSPC when it could not
* allocate any blocks and -ENOSR when it did allocate blocks to convert a
* delalloc range, but those blocks were before the passed in range.
+ * With XFS_BMAPI_CONVERT_ONLY, it stops at a hole or a delalloc extent instead
+ * of allocating blocks, and returns -ENOSR if it stops before mapping anything.
*/
int
xfs_bmapi_write(
@@ -4187,6 +4189,8 @@ xfs_bmapi_write(
*/
ASSERT((flags & (XFS_BMAPI_PREALLOC | XFS_BMAPI_ZERO)) !=
(XFS_BMAPI_PREALLOC | XFS_BMAPI_ZERO));
+ ASSERT(!(flags & XFS_BMAPI_CONVERT_ONLY) ||
+ (flags & XFS_BMAPI_CONVERT));
if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(ifp)) ||
XFS_TEST_ERROR(mp, XFS_ERRTAG_BMAPIFORMAT)) {
@@ -4235,6 +4239,9 @@ xfs_bmapi_write(
* that we found, if any.
*/
if (need_alloc || wasdelay) {
+ if (flags & XFS_BMAPI_CONVERT_ONLY)
+ break;
+
bma.eof = eof;
bma.conv = !!(flags & XFS_BMAPI_CONVERT);
bma.wasdel = wasdelay;
@@ -4330,10 +4337,13 @@ xfs_bmapi_write(
*
* To avoid a successful return with *nmap set to 0, return the magic
* -ENOSR error code for this particular case so that the caller can
- * handle it.
+ * handle it. The same goes for XFS_BMAPI_CONVERT_ONLY stopping at a
+ * hole or a delalloc extent at the start of the range, in which case
+ * nothing has been changed.
*/
if (!n) {
- ASSERT(bma.nallocs >= *nmap);
+ ASSERT((flags & XFS_BMAPI_CONVERT_ONLY) ? bma.nallocs == 0 :
+ bma.nallocs >= *nmap);
return -ENOSR;
}
*nmap = n;
diff --git a/fs/xfs/libxfs/xfs_bmap.h b/fs/xfs/libxfs/xfs_bmap.h
index d5f2729305fa..76d25e5d5d9c 100644
--- a/fs/xfs/libxfs/xfs_bmap.h
+++ b/fs/xfs/libxfs/xfs_bmap.h
@@ -90,6 +90,13 @@ struct xfs_bmalloca {
/* Try to align allocations to the extent size hint */
#define XFS_BMAPI_EXTSZALIGN (1u << 11)
+/*
+ * Use with XFS_BMAPI_CONVERT to convert unwritten extents without allocating
+ * blocks. Stop at a hole or a delalloc extent: return the mappings found so
+ * far, or -ENOSR if there are none.
+ */
+#define XFS_BMAPI_CONVERT_ONLY (1u << 12)
+
#define XFS_BMAPI_FLAGS \
{ XFS_BMAPI_ENTIRE, "ENTIRE" }, \
{ XFS_BMAPI_METADATA, "METADATA" }, \
@@ -102,7 +109,8 @@ struct xfs_bmalloca {
{ XFS_BMAPI_COWFORK, "COWFORK" }, \
{ XFS_BMAPI_NODISCARD, "NODISCARD" }, \
{ XFS_BMAPI_NORMAP, "NORMAP" },\
- { XFS_BMAPI_EXTSZALIGN, "EXTSZALIGN" }
+ { XFS_BMAPI_EXTSZALIGN, "EXTSZALIGN" }, \
+ { XFS_BMAPI_CONVERT_ONLY, "CONVERT_ONLY" }
static inline int xfs_bmapi_aflag(int w)
diff --git a/fs/xfs/libxfs/xfs_btree.c b/fs/xfs/libxfs/xfs_btree.c
index 60ef7f08b1d3..b978ddc13c15 100644
--- a/fs/xfs/libxfs/xfs_btree.c
+++ b/fs/xfs/libxfs/xfs_btree.c
@@ -2567,7 +2567,7 @@ xfs_btree_lshift(
if (XFS_IS_CORRUPT(tcur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
- goto error0;
+ goto error1;
}
error = xfs_btree_decrement(tcur, level, &i);
@@ -2737,7 +2737,7 @@ xfs_btree_rshift(
if (XFS_IS_CORRUPT(tcur->bc_mp, i != 1)) {
xfs_btree_mark_sick(cur);
error = -EFSCORRUPTED;
- goto error0;
+ goto error1;
}
error = xfs_btree_increment(tcur, level, &i);
diff --git a/fs/xfs/scrub/attr.c b/fs/xfs/scrub/attr.c
index 842d65834afb..f909e2b25158 100644
--- a/fs/xfs/scrub/attr.c
+++ b/fs/xfs/scrub/attr.c
@@ -293,6 +293,78 @@ xchk_xattr_set_map(
return ret;
}
+static unsigned int
+xchk_xattr_local_entry(
+ struct xchk_da_btree *ds,
+ int level,
+ char *buf_end,
+ struct xfs_attr_leafblock *leaf,
+ int idx)
+{
+ struct xfs_attr_leaf_name_local *lentry =
+ xfs_attr3_leaf_name_local(leaf, idx);
+ char *name_end;
+ unsigned int namesize;
+
+ name_end = (char *)lentry + offsetof(typeof(*lentry), nameval[0]);
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ namesize = xfs_attr_leaf_entsize_local(lentry->namelen,
+ be16_to_cpu(lentry->valuelen));
+ name_end = (char *)lentry + namesize;
+ if (lentry->namelen == 0) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ return namesize;
+}
+
+static unsigned int
+xchk_xattr_remote_entry(
+ struct xchk_da_btree *ds,
+ int level,
+ char *buf_end,
+ struct xfs_attr_leafblock *leaf,
+ int idx,
+ struct xfs_attr_leaf_entry *ent)
+{
+ struct xfs_attr_leaf_name_remote *rentry =
+ xfs_attr3_leaf_name_remote(leaf, idx);
+ char *name_end;
+ unsigned int namesize;
+
+ name_end = (char *)rentry + offsetof(typeof(*rentry), name[0]);
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ namesize = xfs_attr_leaf_entsize_remote(rentry->namelen);
+ name_end = (char *)rentry + namesize;
+ if (rentry->namelen == 0) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+ if (rentry->valueblk == 0 && !(ent->flags & XFS_ATTR_INCOMPLETE)) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+ if (name_end > buf_end) {
+ xchk_da_set_corrupt(ds, level);
+ return 0;
+ }
+
+ return namesize;
+}
+
/*
* Check this leaf entry's relations to everything else.
* Returns the number of bytes used for the name/value data.
@@ -311,9 +383,6 @@ xchk_xattr_entry(
{
struct xfs_mount *mp = ds->state->mp;
struct xchk_xattr_buf *ab = ds->sc->buf;
- char *name_end;
- struct xfs_attr_leaf_name_local *lentry;
- struct xfs_attr_leaf_name_remote *rentry;
unsigned int nameidx;
unsigned int namesize;
@@ -333,25 +402,14 @@ xchk_xattr_entry(
}
/* Check the name information. */
- if (ent->flags & XFS_ATTR_LOCAL) {
- lentry = xfs_attr3_leaf_name_local(leaf, idx);
- namesize = xfs_attr_leaf_entsize_local(lentry->namelen,
- be16_to_cpu(lentry->valuelen));
- name_end = (char *)lentry + namesize;
- if (lentry->namelen == 0)
- xchk_da_set_corrupt(ds, level);
- } else {
- rentry = xfs_attr3_leaf_name_remote(leaf, idx);
- namesize = xfs_attr_leaf_entsize_remote(rentry->namelen);
- name_end = (char *)rentry + namesize;
- if (rentry->namelen == 0)
- xchk_da_set_corrupt(ds, level);
- if (rentry->valueblk == 0 &&
- !(ent->flags & XFS_ATTR_INCOMPLETE))
- xchk_da_set_corrupt(ds, level);
- }
- if (name_end > buf_end)
- xchk_da_set_corrupt(ds, level);
+ if (ent->flags & XFS_ATTR_LOCAL)
+ namesize = xchk_xattr_local_entry(ds, level, buf_end, leaf,
+ idx);
+ else
+ namesize = xchk_xattr_remote_entry(ds, level, buf_end, leaf,
+ idx, ent);
+ if (!namesize)
+ return;
if (!xchk_xattr_set_map(ds->sc, ab->usedmap, nameidx, namesize))
xchk_da_set_corrupt(ds, level);
diff --git a/fs/xfs/scrub/attr_repair.c b/fs/xfs/scrub/attr_repair.c
index c7695d4cc8e7..ad688f9f7ce9 100644
--- a/fs/xfs/scrub/attr_repair.c
+++ b/fs/xfs/scrub/attr_repair.c
@@ -285,6 +285,7 @@ xrep_xattr_salvage_local_attr(
struct xfs_attr_leaf_name_local *lentry)
{
struct xchk_xattr_buf *ab = rx->sc->buf;
+ char *name_end;
unsigned char *value;
unsigned int valuelen;
unsigned int namesize;
@@ -293,10 +294,14 @@ xrep_xattr_salvage_local_attr(
* Decode the leaf local entry format. If something seems wrong, we
* junk the attribute.
*/
+ name_end = (char *)lentry + offsetof(typeof(*lentry), nameval[0]);
+ if (name_end > buf_end)
+ return 0;
value = &lentry->nameval[lentry->namelen];
valuelen = be16_to_cpu(lentry->valuelen);
namesize = xfs_attr_leaf_entsize_local(lentry->namelen, valuelen);
- if ((char *)lentry + namesize > buf_end)
+ name_end = (char *)lentry + namesize;
+ if (name_end > buf_end)
return 0;
if (!xrep_xattr_want_salvage(rx, ent->flags, lentry->nameval,
lentry->namelen, value, valuelen))
@@ -333,8 +338,8 @@ xrep_xattr_salvage_remote_attr(
.attr_filter = ent->flags & XFS_ATTR_NSP_ONDISK_MASK,
.namelen = rentry->namelen,
.name = rentry->name,
- .valuelen = be32_to_cpu(rentry->valuelen),
};
+ char *name_end;
unsigned int namesize;
int error;
@@ -343,9 +348,14 @@ xrep_xattr_salvage_remote_attr(
* junk the attribute. Note that we should never find a zero-length
* remote attribute value.
*/
+ name_end = (char *)rentry + offsetof(typeof(*rentry), name[0]);
+ if (name_end > buf_end)
+ return 0;
namesize = xfs_attr_leaf_entsize_remote(rentry->namelen);
- if ((char *)rentry + namesize > buf_end)
+ name_end = (char *)rentry + namesize;
+ if (name_end > buf_end)
return 0;
+ args.valuelen = be32_to_cpu(rentry->valuelen);
if (args.valuelen == 0 ||
!xrep_xattr_want_salvage(rx, ent->flags, rentry->name,
rentry->namelen, NULL, args.valuelen))
diff --git a/fs/xfs/scrub/dir.c b/fs/xfs/scrub/dir.c
index 4ab7e6469248..05fe8bc71427 100644
--- a/fs/xfs/scrub/dir.c
+++ b/fs/xfs/scrub/dir.c
@@ -313,6 +313,74 @@ xchk_dir_actor(
return error;
}
+/*
+ * Deal with an unused directory entry. Returns a positive number of bytes to
+ * advance in the directory block, zero to stop processing, or -1 if corruption
+ * was found.
+ */
+static int
+xchk_dir_check_unused(
+ struct xchk_da_btree *ds,
+ struct xfs_buf *bp,
+ unsigned int iter_off,
+ unsigned int end,
+ xfs_dablk_t rec_bno)
+{
+ struct xfs_dir2_data_unused *dup = bp->b_addr + iter_off;
+ unsigned int advance;
+
+ if (iter_off + sizeof(*dup) > end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ advance = xfs_dir2_data_unusedsize(be16_to_cpu(dup->length));
+ if (!advance || iter_off + advance > end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ return advance;
+}
+
+/*
+ * Check a directory entry. Returns a positive number of bytes to advance
+ * in the directory block, zero to stop processing, or -1 if corruption was
+ * found.
+ */
+static int
+xchk_dir_check_entry(
+ struct xchk_da_btree *ds,
+ struct xfs_buf *bp,
+ unsigned int iter_off,
+ unsigned int end,
+ struct xfs_dir2_data_entry *dent,
+ xfs_dablk_t rec_bno)
+{
+ struct xfs_dir2_data_entry *dep = bp->b_addr + iter_off;
+ struct xfs_mount *mp = ds->state->mp;
+ unsigned int advance;
+
+ /* must have namelen */
+ advance = offsetof(typeof(*dep), name);
+ if (iter_off + advance >= end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ /* Don't walk off the end of the block. */
+ advance = xfs_dir2_data_entsize(mp, dep->namelen);
+ if (!advance || iter_off + advance > end) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ return -1;
+ }
+
+ if (dep == dent)
+ return 0;
+
+ return advance;
+}
+
/* Scrub a directory btree record. */
STATIC int
xchk_dir_rec(
@@ -338,6 +406,7 @@ xchk_dir_rec(
xfs_dahash_t hash;
struct xfs_dir3_icleaf_hdr hdr;
unsigned int tag;
+ bool foundit = false;
int error;
ASSERT(blk->magic == XFS_DIR2_LEAF1_MAGIC ||
@@ -382,28 +451,42 @@ xchk_dir_rec(
dent = bp->b_addr + off;
/* Make sure we got a real directory entry. */
- iter_off = geo->data_entry_offset;
end = xfs_dir3_data_end_offset(geo, bp->b_addr);
if (!end) {
xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
goto out_relse;
}
- for (;;) {
- struct xfs_dir2_data_entry *dep = bp->b_addr + iter_off;
+ iter_off = geo->data_entry_offset;
+ while (iter_off < end) {
struct xfs_dir2_data_unused *dup = bp->b_addr + iter_off;
+ int advance;
- if (iter_off >= end) {
+ /* must have freetag */
+ advance = offsetof(typeof(*dup), length);
+ if (iter_off + advance >= end) {
xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
goto out_relse;
}
- if (be16_to_cpu(dup->freetag) == XFS_DIR2_DATA_FREE_TAG) {
- iter_off += be16_to_cpu(dup->length);
- continue;
- }
- if (dep == dent)
+ if (dup->freetag == cpu_to_be16(XFS_DIR2_DATA_FREE_TAG))
+ advance = xchk_dir_check_unused(ds, bp, iter_off, end,
+ rec_bno);
+ else
+ advance = xchk_dir_check_entry(ds, bp, iter_off, end,
+ dent, rec_bno);
+ if (advance < 0)
+ goto out_relse;
+ if (!advance) {
+ foundit = true;
break;
- iter_off += xfs_dir2_data_entsize(mp, dep->namelen);
+ }
+ iter_off += advance;
+ }
+
+ /* Hash tree must point to the exact dirent */
+ if (!foundit) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ goto out_relse;
}
/* Retrieve the entry, sanity check it, and compare hashes. */
@@ -417,6 +500,13 @@ xchk_dir_rec(
goto out_relse;
}
+ /* Name must not overflow end of block */
+ if ((char *)dent->name + dent->namelen >
+ (char *)bp->b_addr + BBTOB(bp->b_length)) {
+ xchk_fblock_set_corrupt(ds->sc, XFS_DATA_FORK, rec_bno);
+ goto out_relse;
+ }
+
/* Does the directory hash match? */
dname.name = dent->name;
dname.len = dent->namelen;
diff --git a/fs/xfs/scrub/dir_repair.c b/fs/xfs/scrub/dir_repair.c
index bb3f2f462bde..84fe3dda0d42 100644
--- a/fs/xfs/scrub/dir_repair.c
+++ b/fs/xfs/scrub/dir_repair.c
@@ -462,6 +462,63 @@ xrep_dir_salvage_data_entry(
return xrep_dir_salvage_entry(rd, dep->name, dep->namelen, ino);
}
+/*
+ * Deal with an unused directory entry. Returns a positive number of bytes to
+ * advance in the directory block, or zero to stop processing.
+ */
+static int
+xrep_dir_recover_unused(
+ struct xfs_buf *bp,
+ unsigned int offset,
+ unsigned int end)
+{
+ struct xfs_dir2_data_unused *dup = bp->b_addr + offset;
+ unsigned int advance;
+
+ if (offset + sizeof(*dup) > end)
+ return 0;
+
+ advance = xfs_dir2_data_unusedsize(be16_to_cpu(dup->length));
+ if (!advance || offset + advance > end)
+ return 0;
+
+ return advance;
+}
+
+/*
+ * Salvage a directory entry. Returns a positive number of bytes to advance
+ * in the directory block, zero to stop processing, or a negative errno.
+ */
+static int
+xrep_dir_recover_entry(
+ struct xrep_dir *rd,
+ struct xfs_buf *bp,
+ unsigned int offset,
+ unsigned int end)
+{
+ struct xfs_dir2_data_entry *dep = bp->b_addr + offset;
+ unsigned int advance;
+ int error;
+
+ /* must have namelen */
+ advance = offsetof(typeof(*dep), name);
+ if (offset + advance >= end)
+ return 0;
+
+ /* Don't walk off the end of the block. */
+ advance = xfs_dir2_data_entsize(rd->sc->mp, dep->namelen);
+ if (!advance || offset + advance > end)
+ return 0;
+
+ /* Ok, let's save this entry. */
+ error = xrep_dir_salvage_data_entry(rd, dep);
+ if (error < 0)
+ return error;
+
+ return advance;
+}
+
+
/* Try to recover block/data format directory entries. */
STATIC int
xrep_dir_recover_data(
@@ -483,33 +540,25 @@ xrep_dir_recover_data(
while (offset < end) {
struct xfs_dir2_data_unused *dup = bp->b_addr + offset;
- struct xfs_dir2_data_entry *dep = bp->b_addr + offset;
- unsigned int advance;
+ int advance;
if (xchk_should_terminate(rd->sc, &error))
return error;
- /* Skip unused entries. */
- if (be16_to_cpu(dup->freetag) == XFS_DIR2_DATA_FREE_TAG) {
- if (!dup->length)
- break;
- offset += be16_to_cpu(dup->length);
- continue;
- }
+ /* must have freetag */
+ advance = offsetof(typeof(*dup), length);
+ if (offset + advance >= end)
+ break;
- /* Don't walk off the end of the block. */
- advance = xfs_dir2_data_entsize(rd->sc->mp, dep->namelen);
+ if (dup->freetag == cpu_to_be16(XFS_DIR2_DATA_FREE_TAG))
+ advance = xrep_dir_recover_unused(bp, offset, end);
+ else
+ advance = xrep_dir_recover_entry(rd, bp, offset, end);
+ if (advance < 0)
+ return advance;
if (!advance)
break;
offset += advance;
- if (offset > end)
- break;
-
- /* Ok, let's save this entry. */
- error = xrep_dir_salvage_data_entry(rd, dep);
- if (error)
- return error;
-
}
return 0;
@@ -532,6 +581,19 @@ xrep_dir_recover_sf(
hdr = ifp->if_data;
end = (unsigned char *)ifp->if_data + ifp->if_bytes;
+ /* sf header must be big enough for count/i8count */
+ if (!hdr || ifp->if_bytes < 2)
+ return 0;
+
+ /* and large enough for the parent inumber */
+ if (hdr->i8count) {
+ if (ifp->if_bytes < offsetof(struct xfs_dir2_sf_hdr, parent[8]))
+ return 0;
+ } else {
+ if (ifp->if_bytes < offsetof(struct xfs_dir2_sf_hdr, parent[4]))
+ return 0;
+ }
+
ino = xfs_dir2_sf_get_parent_ino(hdr);
trace_xrep_dir_salvaged_parent(rd->sc->ip, ino);
diff --git a/fs/xfs/scrub/quota_repair.c b/fs/xfs/scrub/quota_repair.c
index b4f33f432b12..3be56d73dd66 100644
--- a/fs/xfs/scrub/quota_repair.c
+++ b/fs/xfs/scrub/quota_repair.c
@@ -297,7 +297,6 @@ xrep_quota_block(
xfs_dqid_t id)
{
struct xfs_dqblk *dqblk;
- struct xfs_disk_dquot *ddq;
struct xfs_quotainfo *qi = sc->mp->m_quotainfo;
struct xfs_def_quota *defq = xfs_get_defquota(qi, dqtype);
struct xfs_buf *bp = NULL;
@@ -319,14 +318,21 @@ xrep_quota_block(
break;
case 0:
dqblk = bp->b_addr;
- ddq = &dqblk[0].dd_diskdq;
+ error = 0;
/*
* If there's nothing that would impede a dqiterate, we're
* done.
*/
- if ((ddq->d_type & XFS_DQTYPE_REC_MASK) == dqtype &&
- id == be32_to_cpu(ddq->d_id)) {
+ for (i = 0; i < qi->qi_dqperchunk; i++, dqblk++) {
+ struct xfs_disk_dquot *ddq = &dqblk->dd_diskdq;
+
+ if ((ddq->d_type & XFS_DQTYPE_REC_MASK) != dqtype ||
+ id + i != be32_to_cpu(ddq->d_id))
+ error++;
+ }
+
+ if (!error) {
xfs_trans_brelse(sc->tp, bp);
return 0;
}
@@ -339,7 +345,9 @@ xrep_quota_block(
dqblk = bp->b_addr;
bp->b_ops = &xfs_dquot_buf_ops;
for (i = 0; i < qi->qi_dqperchunk; i++, dqblk++) {
- ddq = &dqblk->dd_diskdq;
+ struct xfs_disk_dquot *ddq = &dqblk->dd_diskdq;
+ bool was_bigtime =
+ !!(ddq->d_type & XFS_DQTYPE_BIGTIME);
trace_xrep_disk_dquot(sc->mp, dqtype, id + i);
@@ -348,8 +356,21 @@ xrep_quota_block(
ddq->d_type = dqtype;
ddq->d_id = cpu_to_be32(id + i);
- if (xfs_has_bigtime(sc->mp) && ddq->d_id)
+ if (xfs_has_bigtime(sc->mp) && ddq->d_id) {
+ /*
+ * If something was broken with this dquot and it was a
+ * non-bigtime dquot, we'll grant everyone a fresh
+ * grace period by zeroing the timer field rather than
+ * try to interpret what might be garbage.
+ */
+ if (!was_bigtime) {
+ ddq->d_btimer = 0;
+ ddq->d_itimer = 0;
+ ddq->d_rtbtimer = 0;
+ }
+
ddq->d_type |= XFS_DQTYPE_BIGTIME;
+ }
xrep_quota_fix_timer(sc->mp, ddq, ddq->d_blk_softlimit,
ddq->d_bcount, &ddq->d_btimer,
diff --git a/fs/xfs/xfs_discard.c b/fs/xfs/xfs_discard.c
index 906221ffe9ba..cdf660344c28 100644
--- a/fs/xfs/xfs_discard.c
+++ b/fs/xfs/xfs_discard.c
@@ -155,6 +155,8 @@ struct xfs_trim_cur {
xfs_agblock_t end;
xfs_extlen_t minlen;
bool by_bno;
+ /* restart point (with count) for the by-length walk */
+ xfs_agblock_t restart_bno;
};
static int
@@ -198,14 +200,15 @@ xfs_trim_gather_extents(
error = xfs_alloc_lookup_le(cur, tcur->start, 0, &i);
if (!error && !i)
error = xfs_alloc_lookup_ge(cur, tcur->start, 0, &i);
- } else if (tcur->start == 0) {
- /* first time through a by-len starts with max length */
- cur = xfs_cntbt_init_cursor(mp, tp, agbp, pag);
- error = xfs_alloc_lookup_ge(cur, 0, tcur->count, &i);
} else {
- /* nth time through a by-len starts where we left off */
+ /*
+ * A by-len walk starts at the last record of the longest
+ * length (restart_bno is NULLAGBLOCK until the first batch
+ * ends) and then where the previous batch stopped.
+ */
cur = xfs_cntbt_init_cursor(mp, tp, agbp, pag);
- error = xfs_alloc_lookup_le(cur, tcur->start, tcur->count, &i);
+ error = xfs_alloc_lookup_le(cur, tcur->restart_bno,
+ tcur->count, &i);
}
if (error)
goto out_del_cursor;
@@ -235,9 +238,15 @@ xfs_trim_gather_extents(
if (--batch <= 0) {
/*
* Update the cursor to point at this extent so we
- * restart the next batch from this extent.
+ * restart the next batch from this extent. The
+ * by-length walk does not visit extents in block
+ * order, so it must keep the start of the range and
+ * restart from restart_bno instead.
*/
- tcur->start = fbno;
+ if (tcur->by_bno)
+ tcur->start = fbno;
+ else
+ tcur->restart_bno = fbno;
tcur->count = flen;
break;
}
@@ -340,6 +349,7 @@ xfs_trim_perag_extents(
.start = start,
.end = end,
.minlen = minlen,
+ .restart_bno = NULLAGBLOCK,
};
int error = 0;
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index d8202da15aca..59bd9a24b3ef 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -668,7 +668,7 @@ xfs_dio_write_end_io(
* they are converted.
*/
if (flags & IOMAP_DIO_UNWRITTEN) {
- error = xfs_iomap_write_unwritten(ip, offset, size, true);
+ error = xfs_iomap_write_unwritten(ip, offset, size, true, 0);
goto out;
}
diff --git a/fs/xfs/xfs_healthmon.c b/fs/xfs/xfs_healthmon.c
index 2bdd747f1ead..05d6f0494139 100644
--- a/fs/xfs/xfs_healthmon.c
+++ b/fs/xfs/xfs_healthmon.c
@@ -114,8 +114,8 @@ xfs_healthmon_attach(
}
refcount_inc(&hm->ref);
- rcu_assign_pointer(mp->m_healthmon, hm);
hm->mount_cookie = (uintptr_t)mp->m_super;
+ rcu_assign_pointer(mp->m_healthmon, hm);
spin_unlock(&xfs_healthmon_lock);
return 0;
diff --git a/fs/xfs/xfs_ioctl32.c b/fs/xfs/xfs_ioctl32.c
index c66e192448a8..e411f9453bfb 100644
--- a/fs/xfs/xfs_ioctl32.c
+++ b/fs/xfs/xfs_ioctl32.c
@@ -123,8 +123,8 @@ xfs_ioctl32_bstat_copyin(
xfs_ioctl32_bstime_copyin(&bstat->bs_atime, &bstat32->bs_atime) ||
xfs_ioctl32_bstime_copyin(&bstat->bs_mtime, &bstat32->bs_mtime) ||
xfs_ioctl32_bstime_copyin(&bstat->bs_ctime, &bstat32->bs_ctime) ||
- get_user(bstat->bs_blocks, &bstat32->bs_size) ||
- get_user(bstat->bs_xflags, &bstat32->bs_size) ||
+ get_user(bstat->bs_blocks, &bstat32->bs_blocks) ||
+ get_user(bstat->bs_xflags, &bstat32->bs_xflags) ||
get_user(bstat->bs_extsize, &bstat32->bs_extsize) ||
get_user(bstat->bs_extents, &bstat32->bs_extents) ||
get_user(bstat->bs_gen, &bstat32->bs_gen) ||
diff --git a/fs/xfs/xfs_ioend.c b/fs/xfs/xfs_ioend.c
index 40695d18dac0..aced9a023db1 100644
--- a/fs/xfs/xfs_ioend.c
+++ b/fs/xfs/xfs_ioend.c
@@ -102,7 +102,7 @@ xfs_end_ioend_write(
else if (ioend->io_flags & IOMAP_IOEND_SHARED)
error = xfs_reflink_end_cow(ip, offset, size);
else if (ioend->io_flags & IOMAP_IOEND_UNWRITTEN)
- error = xfs_iomap_write_unwritten(ip, offset, size, false);
+ error = xfs_iomap_write_unwritten(ip, offset, size, false, 0);
if (!error &&
!(ioend->io_flags & IOMAP_IOEND_DIRECT) &&
diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
index 7c6238fed61e..373ff0e64f14 100644
--- a/fs/xfs/xfs_iomap.c
+++ b/fs/xfs/xfs_iomap.c
@@ -285,6 +285,14 @@ xfs_iomap_write_direct(
ASSERT(count_fsb > 0);
+ /*
+ * xfs_bmapi_write() maps at most XFS_MAX_BMBT_EXTLEN blocks in one
+ * mapping, so don't reserve blocks for more. A pNFS layout range
+ * comes from a client, and its block count may not even fit the
+ * 32-bit reservation. The caller maps the rest with a later call.
+ */
+ count_fsb = min_t(xfs_filblks_t, count_fsb, XFS_MAX_BMBT_EXTLEN);
+
resaligned = xfs_aligned_fsb_count(offset_fsb, count_fsb,
xfs_get_extsz_hint(ip));
if (unlikely(XFS_IS_REALTIME_INODE(ip))) {
@@ -614,7 +622,8 @@ xfs_iomap_write_unwritten(
xfs_inode_t *ip,
xfs_off_t offset,
xfs_off_t count,
- bool update_isize)
+ bool update_isize,
+ uint32_t bmapi_flags)
{
xfs_mount_t *mp = ip->i_mount;
xfs_fileoff_t offset_fsb;
@@ -676,8 +685,21 @@ xfs_iomap_write_unwritten(
*/
nimaps = 1;
error = xfs_bmapi_write(tp, ip, offset_fsb, count_fsb,
- XFS_BMAPI_CONVERT, resblks, &imap,
- &nimaps);
+ XFS_BMAPI_CONVERT | bmapi_flags,
+ resblks, &imap, &nimaps);
+ if (error == -ENOSR && (bmapi_flags & XFS_BMAPI_CONVERT_ONLY)) {
+ /*
+ * Nothing was mapped: the rest of the range starts
+ * with a hole or a delalloc extent. Nothing was
+ * changed either, but xfs_iext_count_extend() may
+ * have logged the inode, and cancelling a dirty
+ * transaction shuts the filesystem down. Commit it,
+ * as xfs_alloc_file_space() does on -ENOSR.
+ */
+ error = xfs_trans_commit(tp);
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+ return error ? error : -EIO;
+ }
if (error)
goto error_on_bmapi_transaction;
diff --git a/fs/xfs/xfs_iomap.h b/fs/xfs/xfs_iomap.h
index f2520a9b3a13..8fc0c6fc8bea 100644
--- a/fs/xfs/xfs_iomap.h
+++ b/fs/xfs/xfs_iomap.h
@@ -15,7 +15,8 @@ struct xfs_zone_alloc_ctx;
int xfs_iomap_write_direct(struct xfs_inode *ip, xfs_fileoff_t offset_fsb,
xfs_fileoff_t count_fsb, unsigned int flags,
struct xfs_bmbt_irec *imap, u64 *sequence);
-int xfs_iomap_write_unwritten(struct xfs_inode *, xfs_off_t, xfs_off_t, bool);
+int xfs_iomap_write_unwritten(struct xfs_inode *ip, xfs_off_t offset,
+ xfs_off_t count, bool update_isize, uint32_t bmapi_flags);
xfs_fileoff_t xfs_iomap_eof_align_last_fsb(struct xfs_inode *ip,
xfs_fileoff_t end_fsb);
diff --git a/fs/xfs/xfs_pnfs.c b/fs/xfs/xfs_pnfs.c
index ab385617003a..7153e7a1e1de 100644
--- a/fs/xfs/xfs_pnfs.c
+++ b/fs/xfs/xfs_pnfs.c
@@ -110,7 +110,15 @@ xfs_fs_map_update_inode(
ip->i_diflags |= XFS_DIFLAG_PREALLOC;
xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
- return xfs_trans_commit(tp);
+ error = xfs_trans_commit(tp);
+ if (error)
+ return error;
+
+ /*
+ * Force the log so that the blocks handed out to the client are
+ * guaranteed to be present even after a server crash.
+ */
+ return xfs_log_force_inode(ip);
}
/*
@@ -134,6 +142,7 @@ xfs_fs_map_blocks(
uint lock_flags;
int error = 0;
u64 seq;
+ bool update_inode = false;
if (xfs_is_shutdown(mp))
return -EIO;
@@ -167,10 +176,9 @@ xfs_fs_map_blocks(
if (!write)
limit = max(limit, round_up(i_size_read(inode),
inode->i_sb->s_blocksize));
- if (offset > limit)
+ if (offset < 0 || offset >= limit)
goto out_unlock;
- if (offset > limit - length)
- length = limit - offset;
+ length = min_t(u64, length, limit - offset);
error = filemap_write_and_wait(inode->i_mapping);
if (error)
@@ -221,20 +229,27 @@ xfs_fs_map_blocks(
end_fsb - offset_fsb, 0, &imap, &seq);
if (error)
goto out_unlock;
+ update_inode = true;
+ } else {
+ xfs_iunlock(ip, lock_flags);
/*
- * Ensure the next transaction is committed synchronously so
- * that the blocks allocated and handed out to the client are
- * guaranteed to be present even after a server crash.
+ * Blocks past EOF that were allocated for buffered writes, such
+ * as speculative preallocation, can be freed by
+ * xfs_free_eofblocks() without breaking the layout unless the
+ * inode is marked as preallocated. Mark it before handing them
+ * out for writing, as for blocks allocated above.
*/
+ if (write && !(ip->i_diflags & XFS_DIFLAG_PREALLOC) &&
+ imap.br_startoff + imap.br_blockcount >
+ XFS_B_TO_FSB(mp, XFS_ISIZE(ip)))
+ update_inode = true;
+ }
+
+ if (update_inode) {
error = xfs_fs_map_update_inode(ip);
- if (!error)
- error = xfs_log_force_inode(ip);
if (error)
goto out_unlock;
-
- } else {
- xfs_iunlock(ip, lock_flags);
}
xfs_iunlock(ip, XFS_IOLOCK_EXCL);
@@ -312,9 +327,7 @@ xfs_fs_commit_blocks(
if (start > size)
continue;
- end = start + maps[i].length;
- if (end > size)
- end = size;
+ end = start + min_t(u64, maps[i].length, size - start);
length = end - start;
if (!length)
@@ -328,7 +341,14 @@ xfs_fs_commit_blocks(
(end - 1) >> PAGE_SHIFT);
WARN_ON_ONCE(error);
- error = xfs_iomap_write_unwritten(ip, start, length, false);
+ /*
+ * A client may only write to blocks that a layout handed out,
+ * and xfs_fs_map_blocks() allocates blocks for any hole in a
+ * write layout, so refuse to allocate blocks for a hole or a
+ * delalloc extent in the committed range.
+ */
+ error = xfs_iomap_write_unwritten(ip, start, length, false,
+ XFS_BMAPI_CONVERT_ONLY);
if (error)
goto out_drop_iolock;
}
diff --git a/include/acpi/acpi_bus.h b/include/acpi/acpi_bus.h
index a10a591c18b2..8b1d329455f7 100644
--- a/include/acpi/acpi_bus.h
+++ b/include/acpi/acpi_bus.h
@@ -32,6 +32,12 @@ acpi_status
acpi_evaluate_integer(acpi_handle handle,
acpi_string pathname,
struct acpi_object_list *arguments, unsigned long long *data);
+acpi_status acpi_evaluate_object_exclusive(acpi_handle handle, acpi_string pathname,
+ struct acpi_object_list *arguments,
+ struct acpi_buffer *buffer);
+acpi_status acpi_evaluate_integer_exclusive(acpi_handle handle, acpi_string pathname,
+ struct acpi_object_list *arguments,
+ unsigned long long *data);
bool acpi_evaluate_reference(acpi_handle handle, acpi_string pathname,
struct acpi_object_list *arguments,
struct acpi_handle_list *list);
@@ -48,6 +54,7 @@ acpi_evaluate_ost(acpi_handle handle, u32 source_event, u32 status_code,
bool acpi_has_method(acpi_handle handle, char *name);
acpi_status acpi_execute_simple_method(acpi_handle handle, char *method,
u64 arg);
+acpi_status acpi_execute_simple_method_exclusive(acpi_handle handle, char *method, u64 arg);
acpi_status acpi_evaluate_ej0(acpi_handle handle);
acpi_status acpi_evaluate_lck(acpi_handle handle, int lock);
acpi_status acpi_evaluate_reg(acpi_handle handle, u8 space_id, u32 function);
diff --git a/include/acpi/cppc_acpi.h b/include/acpi/cppc_acpi.h
index 94a6277edab2..a32b09ec3394 100644
--- a/include/acpi/cppc_acpi.h
+++ b/include/acpi/cppc_acpi.h
@@ -188,6 +188,7 @@ extern int cppc_set_epp(int cpu, u64 epp_val);
extern int cppc_get_auto_act_window(int cpu, u64 *auto_act_window);
extern int cppc_set_auto_act_window(int cpu, u64 auto_act_window);
extern int cppc_get_auto_sel(int cpu, bool *enable);
+bool cppc_auto_sel_is_immutable(int cpu);
extern int cppc_set_auto_sel(int cpu, bool enable);
extern int cppc_get_perf_limited(int cpu, u64 *perf_limited);
extern int cppc_set_perf_limited(int cpu, u64 bits_to_clear);
@@ -289,6 +290,11 @@ static inline int cppc_get_auto_sel(int cpu, bool *enable)
{
return -EOPNOTSUPP;
}
+static inline bool cppc_auto_sel_is_immutable(int cpu)
+{
+ return false;
+}
+
static inline int cppc_set_auto_sel(int cpu, bool enable)
{
return -EOPNOTSUPP;
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index bdcc2778ac64..32ffffd9d7b9 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -398,8 +398,7 @@ struct mmu_gather {
void tlb_flush_mmu(struct mmu_gather *tlb);
static inline void __tlb_adjust_range(struct mmu_gather *tlb,
- unsigned long address,
- unsigned int range_size)
+ unsigned long address, unsigned long range_size)
{
tlb->start = min(tlb->start, address);
tlb->end = max(tlb->end, address + range_size);
diff --git a/include/crypto/mldsa.h b/include/crypto/mldsa.h
index 3ef2676787c9..b156f166a141 100644
--- a/include/crypto/mldsa.h
+++ b/include/crypto/mldsa.h
@@ -48,11 +48,13 @@ enum mldsa_alg {
* Return:
* * 0 if the signature is valid
* * -EBADMSG if the signature and/or public key is malformed
+ * * -EINVAL if @alg is not a valid value
* * -EKEYREJECTED if the signature is invalid but otherwise well-formed
* * -ENOMEM if out of memory so the validity of the signature is unknown
*/
-int mldsa_verify(enum mldsa_alg alg, const u8 *sig, size_t sig_len,
- const u8 *msg, size_t msg_len, const u8 *pk, size_t pk_len);
+int __must_check mldsa_verify(enum mldsa_alg alg, const u8 *sig, size_t sig_len,
+ const u8 *msg, size_t msg_len, const u8 *pk,
+ size_t pk_len);
#if IS_ENABLED(CONFIG_CRYPTO_LIB_MLDSA_KUNIT_TEST)
/* Internal function, exposed only for unit testing */
diff --git a/include/dt-bindings/clock/nuvoton,ma35d1-clk.h b/include/dt-bindings/clock/nuvoton,ma35d1-clk.h
index ba2d70f776a6..764d837d6162 100644
--- a/include/dt-bindings/clock/nuvoton,ma35d1-clk.h
+++ b/include/dt-bindings/clock/nuvoton,ma35d1-clk.h
@@ -248,6 +248,7 @@
#define ADC_GATE 233
#define EADC_DIV 234
#define EADC_GATE 235
-#define CLK_MAX_IDX 236
+#define PCLK3_DIV4096 236
+#define PCLK4_DIV4096 237
#endif /* __DT_BINDINGS_CLOCK_NUVOTON_MA35D1_CLK_H */
diff --git a/include/linux/assoc_array_priv.h b/include/linux/assoc_array_priv.h
index dca733ef6750..37a9f76e1505 100644
--- a/include/linux/assoc_array_priv.h
+++ b/include/linux/assoc_array_priv.h
@@ -80,7 +80,7 @@ struct assoc_array_edit {
struct assoc_array_ptr **leaf_p;
struct assoc_array_ptr *dead_leaf;
struct assoc_array_ptr *new_meta[3];
- struct assoc_array_ptr *excised_meta[1];
+ struct assoc_array_ptr *excised_meta[2];
struct assoc_array_ptr *excised_subtree;
struct assoc_array_ptr **set_backpointers[ASSOC_ARRAY_FAN_OUT];
struct assoc_array_ptr *set_backpointers_to;
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 4f7905c3412b..3989bb0dd43b 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -675,7 +675,7 @@ enum {
QUEUE_FLAG_HCTX_ACTIVE, /* at least one blk-mq hctx is active */
QUEUE_FLAG_SQ_SCHED, /* single queue style io dispatch */
QUEUE_FLAG_DISABLE_WBT_DEF, /* for sched to disable/enable wbt */
- QUEUE_FLAG_NO_ELV_SWITCH, /* can't switch elevator any more */
+ QUEUE_FLAG_NO_ELV_SWITCH, /* a switch is running or disk is going away */
QUEUE_FLAG_QOS_ENABLED, /* qos is enabled */
QUEUE_FLAG_BIO_ISSUE_TIME, /* record bio->issue_time_ns */
QUEUE_FLAG_ZONED_QD1_WRITES, /* Limit zoned devices writes to QD=1 */
@@ -716,8 +716,6 @@ void blk_queue_flag_clear(unsigned int flag, struct request_queue *q);
((q)->limits.features & BLK_FEAT_SKIP_TAGSET_QUIESCE)
#define blk_queue_disable_wbt(q) \
test_bit(QUEUE_FLAG_DISABLE_WBT_DEF, &(q)->queue_flags)
-#define blk_queue_no_elv_switch(q) \
- test_bit(QUEUE_FLAG_NO_ELV_SWITCH, &(q)->queue_flags)
#define blk_queue_zoned_qd1_writes(q) \
test_bit(QUEUE_FLAG_ZONED_QD1_WRITES, &(q)->queue_flags)
diff --git a/include/linux/can/can-ml.h b/include/linux/can/can-ml.h
index 1e99fda2b380..cfefa4a385be 100644
--- a/include/linux/can/can-ml.h
+++ b/include/linux/can/can-ml.h
@@ -45,6 +45,7 @@
#include <linux/can.h>
#include <linux/list.h>
#include <linux/netdevice.h>
+#include <linux/spinlock.h>
/* exposed CAN device capabilities for network layer */
#define CAN_CAP_CC BIT(0) /* CAN CC aka Classical CAN */
@@ -59,6 +60,7 @@
enum { RX_ERR, RX_ALL, RX_FIL, RX_INV, RX_MAX };
struct can_dev_rcv_lists {
+ spinlock_t lock; /* protects the receive lists and entries */
struct hlist_head rx[RX_MAX];
struct hlist_head rx_sff[CAN_SFF_RCV_ARRAY_SZ];
struct hlist_head rx_eff[CAN_EFF_RCV_ARRAY_SZ];
@@ -81,6 +83,7 @@ static inline struct can_ml_priv *can_get_ml_priv(struct net_device *dev)
static inline void can_set_ml_priv(struct net_device *dev,
struct can_ml_priv *ml_priv)
{
+ spin_lock_init(&ml_priv->dev_rcv_lists.lock);
netdev_set_ml_priv(dev, ml_priv, ML_PRIV_CAN);
}
diff --git a/include/linux/can/dev.h b/include/linux/can/dev.h
index 6d0710d6f571..346faddee02e 100644
--- a/include/linux/can/dev.h
+++ b/include/linux/can/dev.h
@@ -176,7 +176,7 @@ static inline bool can_dev_dropped_skb(struct net_device *dev, struct sk_buff *s
return can_dropped_invalid_skb(dev, skb);
invalid_skb:
- kfree_skb(skb);
+ dev_kfree_skb_any(skb);
dev->stats.tx_dropped++;
return true;
}
diff --git a/include/linux/can/skb.h b/include/linux/can/skb.h
index 5d27843862fc..87ace09c0d8b 100644
--- a/include/linux/can/skb.h
+++ b/include/linux/can/skb.h
@@ -78,12 +78,12 @@ static inline struct sk_buff *can_create_echo_skb(struct sk_buff *skb)
nskb = skb_clone(skb, GFP_ATOMIC);
if (unlikely(!nskb)) {
- kfree_skb(skb);
+ dev_kfree_skb_any(skb);
return NULL;
}
can_skb_set_owner(nskb, skb->sk);
- consume_skb(skb);
+ dev_consume_skb_any(skb);
return nskb;
}
diff --git a/include/linux/damon.h b/include/linux/damon.h
index 0c8b7ddef9ab..9772309b2b58 100644
--- a/include/linux/damon.h
+++ b/include/linux/damon.h
@@ -914,6 +914,16 @@ struct damon_ctx {
/* @rnd_state: Per-ctx PRNG state for damon_rand(). */
struct rnd_state rnd_state;
+ /*
+ * number of regions after last merge. Read kdamond_split_regions()
+ * for more details.
+ */
+ unsigned long last_nr_regions;
+ /*
+ * number of damon_split_some_regions() invocations for this context.
+ * Read damon_split_some_regions() for more details.
+ */
+ unsigned long region_split_rotation;
};
/* Get a random number in [@l, @r) using @ctx's lockless PRNG. */
diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
index c745f7ad2298..fac05120ab77 100644
--- a/include/linux/huge_mm.h
+++ b/include/linux/huge_mm.h
@@ -12,9 +12,8 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr,
struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma);
bool huge_pmd_set_accessed(struct vm_fault *vmf);
-int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm,
- pud_t *dst_pud, pud_t *src_pud, unsigned long addr,
- struct vm_area_struct *vma);
+int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
+ pud_t *dst_pud, pud_t *src_pud, unsigned long addr);
#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
void huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud);
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 45ada75dc04e..77d62c930dc3 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -121,10 +121,9 @@ void hugepage_put_subpool(struct hugepage_subpool *spool);
void hugetlb_dup_vma_private(struct vm_area_struct *vma);
void clear_vma_resv_huge_pages(struct vm_area_struct *vma);
-int move_hugetlb_page_tables(struct vm_area_struct *vma,
- struct vm_area_struct *new_vma,
- unsigned long old_addr, unsigned long new_addr,
- unsigned long len);
+unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma,
+ struct vm_area_struct *new_vma, unsigned long old_addr,
+ unsigned long new_addr, unsigned long len);
int copy_hugetlb_page_range(struct mm_struct *, struct mm_struct *,
struct vm_area_struct *, struct vm_area_struct *);
void unmap_hugepage_range(struct vm_area_struct *,
@@ -339,11 +338,9 @@ static inline int copy_hugetlb_page_range(struct mm_struct *dst,
return 0;
}
-static inline int move_hugetlb_page_tables(struct vm_area_struct *vma,
- struct vm_area_struct *new_vma,
- unsigned long old_addr,
- unsigned long new_addr,
- unsigned long len)
+static inline unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma,
+ struct vm_area_struct *new_vma, unsigned long old_addr,
+ unsigned long new_addr, unsigned long len)
{
BUG();
return 0;
diff --git a/include/linux/ksm.h b/include/linux/ksm.h
index d39d0d5483a2..a0d358a1feb6 100644
--- a/include/linux/ksm.h
+++ b/include/linux/ksm.h
@@ -91,8 +91,8 @@ static inline void ksm_exit(struct mm_struct *mm)
* We'd like to make this conditional on vma->vm_flags & VM_MERGEABLE,
* but what if the vma was unmerged while the page was swapped out?
*/
-struct folio *ksm_might_need_to_copy(struct folio *folio,
- struct vm_area_struct *vma, unsigned long addr);
+struct folio *ksm_might_need_to_copy(struct folio *folio, struct page *page,
+ struct vm_area_struct *vma, unsigned long addr);
void rmap_walk_ksm(struct folio *folio, struct rmap_walk_control *rwc);
void folio_migrate_ksm(struct folio *newfolio, struct folio *folio);
@@ -145,7 +145,7 @@ static inline int ksm_madvise(struct vm_area_struct *vma, unsigned long start,
}
static inline struct folio *ksm_might_need_to_copy(struct folio *folio,
- struct vm_area_struct *vma, unsigned long addr)
+ struct page *page, struct vm_area_struct *vma, unsigned long addr)
{
return folio;
}
diff --git a/include/linux/memfd.h b/include/linux/memfd.h
index c159e40e3f34..272479317cf1 100644
--- a/include/linux/memfd.h
+++ b/include/linux/memfd.h
@@ -2,10 +2,19 @@
#ifndef __LINUX_MEMFD_H
#define __LINUX_MEMFD_H
+#include <linux/fcntl.h>
#include <linux/file.h>
#define MEMFD_ANON_NAME "[memfd]"
+static inline int memfd_check_seals_mode(unsigned int seals, umode_t old_mode,
+ umode_t new_mode)
+{
+ if ((seals & F_SEAL_EXEC) && ((old_mode ^ new_mode) & 0111))
+ return -EPERM;
+ return 0;
+}
+
#ifdef CONFIG_MEMFD_CREATE
extern long memfd_fcntl(struct file *file, unsigned int cmd, unsigned int arg);
struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx);
diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h
index a11a44eef521..5db78e5761c5 100644
--- a/include/linux/mmu_notifier.h
+++ b/include/linux/mmu_notifier.h
@@ -483,7 +483,8 @@ mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range)
/*
* This version of mmu_notifier_invalidate_range_start() avoids blocking, but it
* can return an error if a notifier can't proceed without blocking, in which
- * case you're not allowed to modify PTEs in the specified range.
+ * case you're not allowed to modify PTEs in the specified range or call
+ * mmu_notifier_invalidate_range_end().
*
* This is mainly intended for OOM handling.
*/
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 94f9c3ff5416..da8dd04fb462 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -492,6 +492,8 @@ enum lruvec_flags {
* folio->flags, masked by LRU_REFS_MASK.
*/
#define MAX_NR_TIERS 4U
+#define LRU_TIER_MIN 0U
+#define LRU_TIER_MAX (MAX_NR_TIERS - 1)
#ifndef __GENERATING_BOUNDS_H
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index 1a3981c26b23..41b7cdcc674f 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -263,6 +263,12 @@ enum why_no_delegation4 { /* new to v4.1 */
WND4_IS_DIR = 8,
};
+enum stable_how4 {
+ UNSTABLE4 = 0,
+ DATA_SYNC4 = 1,
+ FILE_SYNC4 = 2,
+};
+
enum lock_type4 {
NFS4_UNLOCK_LT = 0,
NFS4_READ_LT = 1,
diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h
index 5663dff53186..842cc0f96147 100644
--- a/include/linux/shmem_fs.h
+++ b/include/linux/shmem_fs.h
@@ -111,8 +111,14 @@ extern unsigned long shmem_get_unmapped_area(struct file *, unsigned long addr,
unsigned long len, unsigned long pgoff, unsigned long flags);
extern int shmem_lock(struct file *file, int lock, struct ucounts *ucounts);
#ifdef CONFIG_SHMEM
+bool shmem_file(struct file *file);
bool shmem_mapping(const struct address_space *mapping);
#else
+static inline bool shmem_file(struct file *file)
+{
+ return false;
+}
+
static inline bool shmem_mapping(const struct address_space *mapping)
{
return false;
@@ -187,15 +193,6 @@ static inline struct page *shmem_read_mapping_page(
mapping_gfp_mask(mapping));
}
-static inline bool shmem_file(struct file *file)
-{
- if (!IS_ENABLED(CONFIG_SHMEM))
- return false;
- if (!file || !file->f_mapping)
- return false;
- return shmem_mapping(file->f_mapping);
-}
-
/* Must be called with inode lock taken exclusive. */
static inline void shmem_freeze(struct inode *inode, bool freeze)
{
diff --git a/include/linux/srcu.h b/include/linux/srcu.h
index 7d9bc06df98d..1c221861ba43 100644
--- a/include/linux/srcu.h
+++ b/include/linux/srcu.h
@@ -18,6 +18,7 @@
#include <linux/mutex.h>
#include <linux/rcupdate.h>
+#include <linux/rculist.h>
#include <linux/workqueue.h>
#include <linux/rcu_segcblist.h>
@@ -103,6 +104,41 @@ void call_srcu(struct srcu_struct *ssp, struct rcu_head *head,
void cleanup_srcu_struct(struct srcu_struct *ssp);
void synchronize_srcu(struct srcu_struct *ssp);
+/**
+ * list_splice_init_srcu - splice an SRCU-protected list into an existing list
+ * @list: the SRCU-protected list to splice
+ * @head: the place in the existing list to splice the first list into
+ * @ssp: the srcu_struct that readers of @list use
+ *
+ * Like list_splice_init_rcu(), but wait for the SRCU readers of @list
+ * instead of RCU ones, so that none of them can follow a next pointer
+ * onto @head.
+ *
+ * Note that this function blocks, and that the caller must prevent any
+ * other updates to @list.
+ */
+static inline void list_splice_init_srcu(struct list_head *list,
+ struct list_head *head,
+ struct srcu_struct *ssp)
+{
+ struct list_head *first = list->next;
+ struct list_head *last = list->prev;
+ struct list_head *next = head->next;
+
+ if (list_empty(list))
+ return;
+
+ INIT_LIST_HEAD_RCU(list);
+ synchronize_srcu(ssp);
+ ASSERT_EXCLUSIVE_ACCESS(*first);
+ ASSERT_EXCLUSIVE_ACCESS(*last);
+
+ last->next = next;
+ rcu_assign_pointer(list_next_rcu(head), first);
+ first->prev = head;
+ next->prev = last;
+}
+
#define SRCU_GET_STATE_COMPLETED 0x1
/**
diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index 3c2b8c355ab3..30344c0d6a9d 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -199,7 +199,8 @@ struct rpc_xprt *rpc_task_get_xprt(struct rpc_clnt *clnt,
int rpcb_create_local(struct net *);
void rpcb_put_local(struct net *);
-int rpcb_register(struct net *, u32, u32, int, unsigned short);
+int rpcb_register(struct net *net, u32 prog, u32 vers, int prot,
+ unsigned short port);
int rpcb_v4_register(struct net *net, const u32 program,
const u32 version,
const struct sockaddr *address,
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index 2db1b9ec5658..5fa9417e034d 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -78,6 +78,7 @@ struct svc_serv {
unsigned int sv_max_payload; /* datagram payload size */
unsigned int sv_max_mesg; /* max_payload + 1 page for overheads */
unsigned int sv_xdrsize; /* XDR buffer size */
+ atomic_t sv_rpcb_failures; /* unanswered rpcbind calls */
struct list_head sv_permsocks; /* all permanent sockets */
struct list_head sv_tempsocks; /* all temporary sockets */
int sv_tmpcnt; /* count of temporary "valid" sockets */
@@ -451,6 +452,7 @@ int sunrpc_set_pool_mode(const char *val);
int sunrpc_get_pool_mode(char *val, size_t size);
void svc_rpcb_cleanup(struct svc_serv *serv, struct net *net);
int svc_bind(struct svc_serv *serv, struct net *net);
+unsigned int svc_rpcb_failure_count(struct svc_serv *serv);
struct svc_serv *svc_create(struct svc_program *, unsigned int,
int (*threadfn)(void *data));
bool svc_rqst_replace_page(struct svc_rqst *rqstp,
@@ -471,8 +473,9 @@ unsigned int svc_serv_maxthreads(const struct svc_serv *serv);
int svc_pool_stats_open(struct svc_info *si, struct file *file);
void svc_process(struct svc_rqst *rqstp);
void svc_process_bc(struct rpc_rqst *req, struct svc_rqst *rqstp);
-int svc_register(const struct svc_serv *, struct net *, const int,
- const unsigned short, const unsigned short);
+int svc_register(struct svc_serv *serv, struct net *net,
+ const int family, const unsigned short proto,
+ const unsigned short port);
void svc_wake_up(struct svc_serv *);
void svc_reserve(struct svc_rqst *rqstp, int space);
diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h
index a4351cffc60c..4c0924c73166 100644
--- a/include/linux/userfaultfd_k.h
+++ b/include/linux/userfaultfd_k.h
@@ -154,8 +154,6 @@ extern int mrwprotect_range(struct userfaultfd_ctx *ctx, unsigned long start,
unsigned long len, bool enable_rwp);
/* move_pages */
-void double_pt_lock(spinlock_t *ptl1, spinlock_t *ptl2);
-void double_pt_unlock(spinlock_t *ptl1, spinlock_t *ptl2);
int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pmd_t dst_pmdval,
struct vm_area_struct *dst_vma,
struct vm_area_struct *src_vma,
diff --git a/include/net/espintcp.h b/include/net/espintcp.h
index c70efd704b6d..7cad3c6822df 100644
--- a/include/net/espintcp.h
+++ b/include/net/espintcp.h
@@ -34,7 +34,8 @@ static inline struct espintcp_ctx *espintcp_getctx(const struct sock *sk)
{
const struct inet_connection_sock *icsk = inet_csk(sk);
- /* RCU is only needed for diag */
- return (__force void *)icsk->icsk_ulp_data;
+ /* pairs with smp_wmb() in espintcp_init_sk() */
+ smp_rmb();
+ return (__force void *)READ_ONCE(icsk->icsk_ulp_data);
}
#endif
diff --git a/include/net/inet_frag.h b/include/net/inet_frag.h
index b02d2472b003..c1fb419a9cf0 100644
--- a/include/net/inet_frag.h
+++ b/include/net/inet_frag.h
@@ -79,6 +79,7 @@ struct frag_v6_compare_key {
* @tstamp_type: stamp has a mono delivery time (EDT)
* @flags: fragment queue flags
* @max_size: maximum received fragment size
+ * @truesize: sum of the truesize of queued fragments
* @fqdir: pointer to struct fqdir
* @rcu: rcu head for freeing deferall
*/
@@ -100,6 +101,7 @@ struct inet_frag_queue {
u8 tstamp_type;
__u8 flags;
u16 max_size;
+ unsigned int truesize;
struct fqdir *fqdir;
struct rcu_head rcu;
};
@@ -172,6 +174,13 @@ extern const u8 ip_frag_ecn_table[16];
#define IPFRAG_DUP 1
#define IPFRAG_OVERLAP 2
#define IPFRAG_MIXED_SECPATH 3
+#define IPFRAG_TOOBIG 4
+
+/* Max sum of the truesize of fragments in a queue. This keeps the sums
+ * done in inet_frag_reasm_prepare() and inet_frag_reasm_finish() well
+ * below INT_MAX. Legitimate traffic stays far below this limit.
+ */
+#define INET_FRAG_MAX_TRUESIZE (INT_MAX / 2)
int inet_frag_queue_insert(struct inet_frag_queue *q, struct sk_buff *skb,
int offset, int end);
void *inet_frag_reasm_prepare(struct inet_frag_queue *q, struct sk_buff *skb,
diff --git a/include/net/netns/can.h b/include/net/netns/can.h
index 48b79f7e6236..54f23681e0d4 100644
--- a/include/net/netns/can.h
+++ b/include/net/netns/can.h
@@ -29,7 +29,7 @@ struct netns_can {
/* receive filters subscribed for 'all' CAN devices */
struct can_dev_rcv_lists *rx_alldev_list;
- spinlock_t rcvlists_lock;
+ spinlock_t rcvlists_lock; /* protects the rcv_entries(_max) stats */
struct timer_list stattimer; /* timer for statistics update */
struct can_pkg_stats *pkg_stats;
struct can_rcv_lists_stats *rcv_lists_stats;
diff --git a/include/net/sctp/sm.h b/include/net/sctp/sm.h
index 3bfd261a53cc..76605d1ee839 100644
--- a/include/net/sctp/sm.h
+++ b/include/net/sctp/sm.h
@@ -252,6 +252,7 @@ struct sctp_chunk *sctp_make_fwdtsn(const struct sctp_association *asoc,
struct sctp_fwdtsn_skip *skiplist);
struct sctp_chunk *sctp_make_auth(const struct sctp_association *asoc,
__u16 key_id);
+void sctp_control_set_owner_w(struct sctp_chunk *chunk);
struct sctp_chunk *sctp_make_strreset_req(const struct sctp_association *asoc,
__u16 stream_num, __be16 *stream_list,
bool out, bool in);
diff --git a/include/sound/cs35l56.h b/include/sound/cs35l56.h
index 747e10ec1a27..ce6383c2c60c 100644
--- a/include/sound/cs35l56.h
+++ b/include/sound/cs35l56.h
@@ -209,6 +209,9 @@ struct snd_ctl_elem_value;
#define CS35L56_GPIO_FN_GPIO 0x00000001
+/* DSP1_FW_VER */
+#define CS35L56_FW_MAIN_VERSION_MASK GENMASK(19, 0)
+
/* Mixer input sources */
#define CS35L56_INPUT_SRC_NONE 0x00
#define CS35L56_INPUT_SRC_ASP1RX1 0x08
@@ -446,6 +449,7 @@ int cs35l56_cal_set_status_get(struct cs35l56_base *cs35l56_base,
int cs35l56_read_prot_status(struct cs35l56_base *cs35l56_base,
bool *fw_missing, unsigned int *fw_version);
void cs35l56_warn_if_firmware_missing(struct cs35l56_base *cs35l56_base);
+bool cs35l56_needs_wait_for_firmware_timer_expiry(struct cs35l56_base *cs35l56_base);
void cs35l56_log_tuning(struct cs35l56_base *cs35l56_base, struct cs_dsp *cs_dsp);
int cs35l56_hw_init(struct cs35l56_base *cs35l56_base);
int cs35l56_get_speaker_id(struct cs35l56_base *cs35l56_base);
diff --git a/include/sound/hwdep.h b/include/sound/hwdep.h
index b0da633184cd..7357727f0660 100644
--- a/include/sound/hwdep.h
+++ b/include/sound/hwdep.h
@@ -57,6 +57,7 @@ struct snd_hwdep {
struct mutex open_mutex;
int used; /* reference counter */
+ struct mutex dsp_mutex; /* serializes DSP load requests */
unsigned int dsp_loaded; /* bit fields of loaded dsp indices */
unsigned int exclusive:1; /* exclusive access mode */
};
diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h
index 5d47d1421c1f..63a86354e822 100644
--- a/include/uapi/linux/iommufd.h
+++ b/include/uapi/linux/iommufd.h
@@ -871,6 +871,12 @@ enum iommufd_hwpt_get_dirty_bitmap_flags {
* Walk the IOMMU pagetables for a given IOVA range to return a bitmap
* with the dirty IOVAs. In doing so it will also by default clear any
* dirty bit metadata set in the IOPTE.
+ *
+ * The IOVA range given must fully enclose any IOPTE. If the range starts or
+ * ends inside a dirty IOPTE then the ioctl will fail. This avoids partially
+ * reporting a dirty range.
+ *
+ * On failure the dirty state becomes unknown and the userspace must restart.
*/
struct iommu_hwpt_get_dirty_bitmap {
__u32 size;
diff --git a/io_uring/cancel.c b/io_uring/cancel.c
index 75a21ecfe69c..6be224ab53dc 100644
--- a/io_uring/cancel.c
+++ b/io_uring/cancel.c
@@ -198,7 +198,8 @@ static int __io_async_cancel(struct io_cancel_data *cd,
break;
if (!all)
return ret;
- nr++;
+ /* poll, futex and waitid return how many they cancelled */
+ nr += max(ret, 1);
} while (1);
/* slow path, try all io-wq's */
@@ -650,7 +651,16 @@ __cold void io_uring_cancel_generic(bool cancel_all, struct io_sq_data *sqd)
}
if (loop) {
+ /*
+ * A request that runs in io-wq and doesn't end makes
+ * each round come here. Don't hold sqd->lock for that
+ * long either.
+ */
+ if (sqd)
+ mutex_unlock(&sqd->lock);
cond_resched();
+ if (sqd)
+ mutex_lock(&sqd->lock);
continue;
}
@@ -678,8 +688,20 @@ __cold void io_uring_cancel_generic(bool cancel_all, struct io_sq_data *sqd)
* avoids a race where a completion comes in before we did
* prepare_to_wait().
*/
- if (inflight == tctx_inflight(tctx, !cancel_all))
+ if (inflight == tctx_inflight(tctx, !cancel_all)) {
+ /*
+ * A zero-copy notification can't be canceled, and a
+ * peer that doesn't read keeps it pending. Don't hold
+ * sqd->lock while waiting for it, or whoever wants the
+ * lock sleeps as long. No ring is used across this,
+ * the next round walks ->ctx_list anew.
+ */
+ if (sqd)
+ mutex_unlock(&sqd->lock);
schedule();
+ if (sqd)
+ mutex_lock(&sqd->lock);
+ }
end_wait:
finish_wait(&tctx->wait, &wait);
} while (1);
diff --git a/io_uring/poll.c b/io_uring/poll.c
index 4f04bda42f7f..fc1800c7097b 100644
--- a/io_uring/poll.c
+++ b/io_uring/poll.c
@@ -128,12 +128,24 @@ static void io_poll_req_insert(struct io_kiocb *req)
hlist_add_head(&req->hash_node, &table->hbs[index].list);
}
+#define IO_POLL_UNMASK (EPOLLERR|EPOLLHUP|EPOLLNVAL)
+
+static __poll_t io_poll_unmask(__poll_t events)
+{
+ /*
+ * A peer that stopped sending is an event for a reader, but a write
+ * only poll must stay armed until the file is writable again.
+ */
+ if (events & (EPOLLIN | EPOLLRDNORM | EPOLLRDBAND))
+ return IO_POLL_UNMASK | EPOLLRDHUP;
+ return IO_POLL_UNMASK;
+}
+
static void io_init_poll_iocb(struct io_poll *poll, __poll_t events)
{
poll->head = NULL;
-#define IO_POLL_UNMASK (EPOLLERR|EPOLLHUP|EPOLLNVAL|EPOLLRDHUP)
/* mask in events that we always want/need */
- poll->events = events | IO_POLL_UNMASK;
+ poll->events = events | io_poll_unmask(events);
INIT_LIST_HEAD(&poll->wait.entry);
init_waitqueue_func_entry(&poll->wait, io_poll_wake);
}
@@ -727,7 +739,7 @@ int io_arm_apoll(struct io_kiocb *req, unsigned issue_flags, __poll_t mask)
int io_arm_poll_handler(struct io_kiocb *req, unsigned issue_flags)
{
const struct io_issue_def *def = &io_issue_defs[req->opcode];
- __poll_t mask = POLLPRI | POLLERR;
+ __poll_t mask = EPOLLERR;
if (!def->pollin && !def->pollout)
return IO_APOLL_ABORTED;
@@ -735,7 +747,8 @@ int io_arm_poll_handler(struct io_kiocb *req, unsigned issue_flags)
return IO_APOLL_ABORTED;
if (def->pollin) {
- mask |= EPOLLIN | EPOLLRDNORM;
+ /* urgent data is an event for a reader only */
+ mask |= EPOLLIN | EPOLLRDNORM | EPOLLPRI;
/* If reading from MSG_ERRQUEUE using recvmsg, ignore POLLIN */
if (req->flags & REQ_F_CLEAR_POLLIN)
@@ -827,12 +840,47 @@ static int io_poll_disarm(struct io_kiocb *req)
return 0;
}
+/*
+ * Cancel every request matching @cd in one pass. Cancelled requests stay
+ * hashed until their task work runs, so finding them one at a time would
+ * walk past all the earlier ones on each lookup.
+ */
+static int io_poll_cancel_all(struct io_ring_ctx *ctx, struct io_cancel_data *cd)
+{
+ unsigned int nr_buckets = 1U << ctx->cancel_table.hash_bits;
+ unsigned int i = 0;
+ struct hlist_node *tmp;
+ struct io_kiocb *req;
+ int nr = 0;
+
+ if (!(cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP |
+ IORING_ASYNC_CANCEL_ANY))) {
+ /* keyed by user_data, so only one bucket can match */
+ i = hash_long(cd->data, ctx->cancel_table.hash_bits);
+ nr_buckets = i + 1;
+ }
+
+ for (; i < nr_buckets; i++) {
+ struct io_hash_bucket *hb = &ctx->cancel_table.hbs[i];
+
+ hlist_for_each_entry_safe(req, tmp, &hb->list, hash_node) {
+ if (!io_cancel_req_match(req, cd))
+ continue;
+ io_poll_cancel_req(req);
+ nr++;
+ }
+ }
+ return nr ?: -ENOENT;
+}
+
static int __io_poll_cancel(struct io_ring_ctx *ctx, struct io_cancel_data *cd)
{
struct io_kiocb *req;
- if (cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP |
- IORING_ASYNC_CANCEL_ANY))
+ if (cd->flags & (IORING_ASYNC_CANCEL_ALL | IORING_ASYNC_CANCEL_ANY))
+ return io_poll_cancel_all(ctx, cd);
+
+ if (cd->flags & (IORING_ASYNC_CANCEL_FD | IORING_ASYNC_CANCEL_OP))
req = io_poll_file_find(ctx, cd);
else
req = io_poll_find(ctx, false, cd);
@@ -971,7 +1019,7 @@ int io_poll_remove(struct io_kiocb *req, unsigned int issue_flags)
poll->events &= ~0xffff;
poll->events |= poll_update->events & 0xffff;
- poll->events |= IO_POLL_UNMASK;
+ poll->events |= io_poll_unmask(poll->events);
}
if (poll_update->update_user_data)
preq->cqe.user_data = poll_update->new_user_data;
diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
index d69baa47c7e7..c1ab760c4591 100644
--- a/io_uring/rsrc.c
+++ b/io_uring/rsrc.c
@@ -521,11 +521,12 @@ static int io_files_update_with_index_alloc(struct io_kiocb *req,
{
struct io_rsrc_update *up = io_kiocb_to_cmd(req, struct io_rsrc_update);
__s32 __user *fds = u64_to_user_ptr(up->arg);
+ struct io_ring_ctx *ctx = req->ctx;
unsigned int done;
struct file *file;
int ret, fd;
- if (!req->ctx->file_table.data.nr)
+ if (!ctx->file_table.data.nr)
return -ENXIO;
for (done = 0; done < up->nr_args; done++) {
@@ -539,15 +540,25 @@ static int io_files_update_with_index_alloc(struct io_kiocb *req,
ret = -EBADF;
break;
}
- ret = io_fixed_fd_install(req, issue_flags, file,
- IORING_FILE_INDEX_ALLOC);
- if (ret < 0)
+ /*
+ * Hold the lock across the put_user(), if it's dropped the
+ * slot can be closed and reused before the error path runs.
+ */
+ io_ring_submit_lock(ctx, issue_flags);
+ ret = __io_fixed_fd_install(ctx, file,
+ IORING_FILE_INDEX_ALLOC);
+ if (ret < 0) {
+ io_ring_submit_unlock(ctx, issue_flags);
+ fput(file);
break;
+ }
if (put_user(ret, &fds[done])) {
- __io_close_fixed(req->ctx, issue_flags, ret);
+ io_fixed_fd_remove(ctx, ret);
+ io_ring_submit_unlock(ctx, issue_flags);
ret = -EFAULT;
break;
}
+ io_ring_submit_unlock(ctx, issue_flags);
}
if (done)
diff --git a/io_uring/sqpoll.c b/io_uring/sqpoll.c
index 49c52284ecdd..d387488db2de 100644
--- a/io_uring/sqpoll.c
+++ b/io_uring/sqpoll.c
@@ -29,6 +29,7 @@
enum {
IO_SQ_THREAD_SHOULD_STOP = 0,
IO_SQ_THREAD_SHOULD_PARK,
+ IO_SQ_THREAD_EXITING,
};
void io_sq_thread_unpark(struct io_sq_data *sqd)
@@ -348,6 +349,18 @@ static int io_sq_thread(void *data)
timeout = jiffies + sqd->sq_thread_idle;
}
+ /*
+ * The SQPOLL thread never returns to userspace, so do the
+ * return-to-userspace work, like memory.high reclaim, here.
+ * It can sleep for a long time, so drop sqd->lock.
+ */
+ if (test_thread_flag(TIF_NOTIFY_RESUME)) {
+ mutex_unlock(&sqd->lock);
+ resume_user_mode_work(NULL);
+ mutex_lock(&sqd->lock);
+ sqd->sq_cpu = raw_smp_processor_id();
+ }
+
cap_entries = !list_is_singular(&sqd->ctx_list);
list_for_each_entry(ctx, &sqd->ctx_list, sqd_list) {
int ret;
@@ -433,6 +446,13 @@ static int io_sq_thread(void *data)
if (io_sq_tw_pending())
io_sq_tw(UINT_MAX);
+ /*
+ * Cancelation drops sqd->lock while it waits, and a ring could attach
+ * to a thread that is on its way out. A fatal signal doesn't set the
+ * stop bit, so mark the thread as exiting here for
+ * io_sq_offload_create() to see.
+ */
+ set_bit(IO_SQ_THREAD_EXITING, &sqd->state);
io_uring_cancel_generic(true, sqd);
rcu_assign_pointer(sqd->thread, NULL);
put_task_struct(current);
@@ -502,7 +522,11 @@ __cold int io_sq_offload_create(struct io_ring_ctx *ctx,
list_add(&ctx->sqd_list, &sqd->ctx_list);
io_sqd_update_thread_idle(sqd);
/* don't attach to a dying SQPOLL thread, would be racy */
- ret = (attached && !sqd->thread) ? -ENXIO : 0;
+ ret = 0;
+ if (attached && (!sqd->thread ||
+ test_bit(IO_SQ_THREAD_SHOULD_STOP, &sqd->state) ||
+ test_bit(IO_SQ_THREAD_EXITING, &sqd->state)))
+ ret = -ENXIO;
io_sq_thread_unpark(sqd);
if (ret < 0)
diff --git a/kernel/events/core.c b/kernel/events/core.c
index c4217c996f5a..9b90b6831de7 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -7634,6 +7634,15 @@ static void perf_sigtrap(struct perf_event *event)
if (event->ctx->task == TASK_TOMBSTONE)
return;
+ /*
+ * The event was removed after this signal was queued, e.g. by exec()
+ * (remove_on_exec), which may also have closed its fd (close-on-exec).
+ * The new program has default signal handlers, so a SIGTRAP would
+ * kill it.
+ */
+ if (READ_ONCE(event->state) <= PERF_EVENT_STATE_EXIT)
+ return;
+
/*
* We'd expect this to only occur if the irq_work is delayed and either
* ctx->task or current has changed in the meantime. This can be the
diff --git a/kernel/fork.c b/kernel/fork.c
index 8ad34646962d..48d7ca064404 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -1535,12 +1535,19 @@ static struct mm_struct *dup_mm(struct task_struct *tsk,
if (!mm)
goto fail_nomem;
+ uprobe_start_dup_mmap();
+ if (mmap_write_lock_killable(oldmm)) {
+ free_mm(mm);
+ goto fail_uprobe;
+ }
+
memcpy(mm, oldmm, sizeof(*mm));
- if (!mm_init(mm, tsk))
- goto fail_nomem;
+ if (!mm_init(mm, tsk)) {
+ mmap_write_unlock(oldmm);
+ goto fail_uprobe;
+ }
- uprobe_start_dup_mmap();
err = dup_mmap(mm, oldmm);
if (err)
goto free_pt;
@@ -1559,9 +1566,10 @@ static struct mm_struct *dup_mm(struct task_struct *tsk,
mm->binfmt = NULL;
mm_init_owner(mm, NULL);
mmput(mm);
- if (err)
- uprobe_end_dup_mmap();
-
+ if (!err)
+ return NULL;
+fail_uprobe:
+ uprobe_end_dup_mmap();
fail_nomem:
return NULL;
}
diff --git a/kernel/panic.c b/kernel/panic.c
index cec271601935..f08453f0d8b4 100644
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -893,7 +893,7 @@ static __init int alloc_taint_buf(void)
size += 1; /* For NULL terminator */
- buf = kmalloc(size, GFP_KERNEL);
+ buf = kzalloc(size, GFP_KERNEL);
if (!buf) {
panic("Failed to allocate taint string buffer");
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index b35e23f93bcd..913630c0b059 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7021,8 +7021,8 @@ static void dequeue_throttled_task(struct task_struct *p, int flags)
WARN_ON_ONCE(p->se.on_rq);
list_del_init(&p->throttle_node);
- /* task blocked after throttled */
- if (flags & DEQUEUE_SLEEP) {
+ /* Task blocked after throttling or is leaving the fair class: */
+ if (flags & (DEQUEUE_SLEEP | DEQUEUE_CLASS)) {
p->throttled = false;
return;
}
diff --git a/lib/assoc_array.c b/lib/assoc_array.c
index b6c9723e12ce..3a7cddb1290b 100644
--- a/lib/assoc_array.c
+++ b/lib/assoc_array.c
@@ -470,33 +470,32 @@ static bool assoc_array_insert_in_empty_tree(struct assoc_array_edit *edit)
}
/*
- * Handle insertion into a terminal node.
+ * The pointer that leads to a node or shortcut with the given back pointer
+ * and parent slot: the root, a slot in a node, or a shortcut's next node.
*/
-static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
- const struct assoc_array_ops *ops,
- const void *index_key,
- struct assoc_array_walk_result *result)
+static struct assoc_array_ptr **assoc_array_parent_link(struct assoc_array *array,
+ struct assoc_array_ptr *back_pointer,
+ int parent_slot)
{
- struct assoc_array_shortcut *shortcut, *new_s0;
- struct assoc_array_node *node, *new_n0, *new_n1, *side;
- struct assoc_array_ptr *ptr;
- unsigned long dissimilarity, base_seg, blank;
- size_t keylen;
- bool have_meta;
- int level, diff;
- int slot, next_slot, free_slot, i, j;
-
- node = result->terminal_node.node;
- level = result->terminal_node.level;
- edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = result->terminal_node.slot;
+ if (!back_pointer)
+ return &array->root;
+ if (assoc_array_ptr_is_node(back_pointer))
+ return &assoc_array_ptr_to_node(back_pointer)->slots[parent_slot];
+ return &assoc_array_ptr_to_shortcut(back_pointer)->next_node;
+}
- pr_devel("-->%s()\n", __func__);
+/*
+ * Insert into a terminal node without changing its shape, if that can be
+ * done: in place of a leaf with the same index key, or in a free slot.
+ */
+static bool assoc_array_insert_in_node(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const void *index_key,
+ struct assoc_array_node *node)
+{
+ struct assoc_array_ptr *ptr;
+ int free_slot, i;
- /* We arrived at a node which doesn't have an onward node or shortcut
- * pointer that we have to follow. This means that (a) the leaf we
- * want must go here (either by insertion or replacement) or (b) we
- * need to split this node and insert in one of the fragments.
- */
free_slot = -1;
/* Firstly, we have to check the leaves in this node to see if there's
@@ -514,7 +513,6 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
pr_devel("replace in slot %d\n", i);
edit->leaf_p = &node->slots[i];
edit->dead_leaf = node->slots[i];
- pr_devel("<--%s() = ok [replace]\n", __func__);
return true;
}
}
@@ -526,28 +524,28 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
pr_devel("insert in free slot %d\n", free_slot);
edit->leaf_p = &node->slots[free_slot];
edit->adjust_count_on = node;
- pr_devel("<--%s() = ok [insert]\n", __func__);
return true;
}
- /* The node has no spare slots - so we're either going to have to split
- * it or insert another node before it.
- *
- * Whatever, we're going to need at least two new nodes - so allocate
- * those now. We may also need a new shortcut, but we deal with that
- * when we need it.
- */
- new_n0 = kzalloc_obj(struct assoc_array_node);
- if (!new_n0)
- return false;
- edit->new_meta[0] = assoc_array_node_to_ptr(new_n0);
- new_n1 = kzalloc_obj(struct assoc_array_node);
- if (!new_n1)
- return false;
- edit->new_meta[1] = assoc_array_node_to_ptr(new_n1);
+ return false;
+}
+
+/*
+ * Fill in the edit script's segment cache for a full node: the segment at
+ * this level of each leaf's index key, or 0xff for a metadata pointer.
+ * The new key's segment, in the last entry, is left to the caller. Returns
+ * true if there are any metadata pointers.
+ */
+static bool assoc_array_fill_segment_cache(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const struct assoc_array_node *node,
+ int level)
+{
+ struct assoc_array_ptr *ptr;
+ unsigned long base_seg;
+ bool have_meta;
+ int i;
- /* We need to find out how similar the leaves are. */
- pr_devel("no spare slots\n");
have_meta = false;
for (i = 0; i < ASSOC_ARRAY_FAN_OUT; i++) {
ptr = node->slots[i];
@@ -561,65 +559,42 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK;
edit->segment_cache[i] = base_seg & ASSOC_ARRAY_FAN_MASK;
}
+ return have_meta;
+}
- if (have_meta) {
- pr_devel("have meta\n");
- goto split_node;
- }
-
- /* The node contains only leaves */
- dissimilarity = 0;
- base_seg = edit->segment_cache[0];
- for (i = 1; i < ASSOC_ARRAY_FAN_OUT; i++)
- dissimilarity |= edit->segment_cache[i] ^ base_seg;
-
- pr_devel("only leaves; dissimilarity=%lx\n", dissimilarity);
+/*
+ * Whether the leaves of a full node and the new key all share one segment,
+ * so that splitting the node cannot make room.
+ */
+static bool assoc_array_all_one_segment(const struct assoc_array_edit *edit)
+{
+ int i;
- if ((dissimilarity & ASSOC_ARRAY_FAN_MASK) == 0) {
- /* The old leaves all cluster in the same slot. We will need
- * to insert a shortcut if the new node wants to cluster with them.
- */
- if ((edit->segment_cache[ASSOC_ARRAY_FAN_OUT] ^ base_seg) == 0)
- goto all_leaves_cluster_together;
+ for (i = 1; i < ASSOC_ARRAY_FAN_OUT + 1; i++)
+ if (edit->segment_cache[i] != edit->segment_cache[0])
+ return false;
+ return true;
+}
- /* Otherwise all the old leaves cluster in the same slot, but
- * the new leaf wants to go into a different slot - so we
- * create a new node (n0) to hold the new leaf and a pointer to
- * a new node (n1) holding all the old leaves.
- *
- * This can be done by falling through to the node splitting
- * path.
- */
- pr_devel("present leaves cluster but not new leaf\n");
- }
+/*
+ * Split a full node into new_n0, which takes its place, and new_n1, below
+ * new_n0 in the slot of a segment that at least two of the leaves share,
+ * the new one included. The caller has filled in the segment cache and
+ * set new_n0's back pointer and parent slot.
+ */
+static void assoc_array_split_node(struct assoc_array_edit *edit,
+ struct assoc_array_node *node,
+ struct assoc_array_node *new_n0,
+ struct assoc_array_node *new_n1)
+{
+ struct assoc_array_shortcut *shortcut;
+ struct assoc_array_node *side;
+ struct assoc_array_ptr *ptr;
+ int slot, next_slot, free_slot, i, j;
-split_node:
pr_devel("split node\n");
- /* We need to split the current node. The node must contain anything
- * from a single leaf (in the one leaf case, this leaf will cluster
- * with the new leaf) and the rest meta-pointers, to all leaves, some
- * of which may cluster.
- *
- * It won't contain the case in which all the current leaves plus the
- * new leaves want to cluster in the same slot.
- *
- * We need to expel at least two leaves out of a set consisting of the
- * leaves in the node and the new leaf. The current meta pointers can
- * just be copied as they shouldn't cluster with any of the leaves.
- *
- * We need a new node (n0) to replace the current one and a new node to
- * take the expelled nodes (n1).
- */
- edit->set[0].to = assoc_array_node_to_ptr(new_n0);
- new_n0->back_pointer = node->back_pointer;
- new_n0->parent_slot = node->parent_slot;
new_n1->back_pointer = assoc_array_node_to_ptr(new_n0);
- new_n1->parent_slot = -1; /* Need to calculate this */
-
-do_split_node:
- pr_devel("do_split_node\n");
-
new_n0->nr_leaves_on_branch = node->nr_leaves_on_branch;
new_n1->nr_leaves_on_branch = 0;
@@ -699,26 +674,29 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
}
}
}
+}
- ptr = node->back_pointer;
- if (!ptr)
- edit->set[0].ptr = &edit->array->root;
- else if (assoc_array_ptr_is_node(ptr))
- edit->set[0].ptr = &assoc_array_ptr_to_node(ptr)->slots[node->parent_slot];
- else
- edit->set[0].ptr = &assoc_array_ptr_to_shortcut(ptr)->next_node;
- edit->excised_meta[0] = assoc_array_node_to_ptr(node);
- pr_devel("<--%s() = ok [split node]\n", __func__);
- return true;
+/*
+ * All the leaves, new and old, want to cluster together in this node in the
+ * same slot, so we have to replace this node with a shortcut to skip over
+ * the identical parts of the key, leading to new_n0. If the node is the
+ * target of a shortcut, the new one replaces that shortcut as well. The
+ * caller then splits the leaves between new_n0 and new_n1 at the shortcut's
+ * far end, for which this fills in the segment cache.
+ */
+static struct assoc_array_shortcut *
+assoc_array_insert_shortcut(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const void *index_key,
+ struct assoc_array_node *node, int level,
+ struct assoc_array_node *new_n0)
+{
+ struct assoc_array_shortcut *new_s0;
+ unsigned long base_seg, blank;
+ size_t keylen;
+ int diff, i;
-all_leaves_cluster_together:
- /* All the leaves, new and old, want to cluster together in this node
- * in the same slot, so we have to replace this node with a shortcut to
- * skip over the identical parts of the key and then place a pair of
- * nodes, one inside the other, at the end of the shortcut and
- * distribute the keys between them.
- *
- * Firstly we need to work out where the leaves start diverging as a
+ /* Firstly we need to work out where the leaves start diverging as a
* bit position into their keys so that we know how big the shortcut
* needs to be.
*
@@ -744,17 +722,28 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
new_s0 = kzalloc_flex(*new_s0, index_key, keylen);
if (!new_s0)
- return false;
+ return NULL;
edit->new_meta[2] = assoc_array_shortcut_to_ptr(new_s0);
- edit->set[0].to = assoc_array_shortcut_to_ptr(new_s0);
new_s0->back_pointer = node->back_pointer;
new_s0->parent_slot = node->parent_slot;
+
+ /* A shortcut has to lead to a node, so a node at the end of a shortcut
+ * is replaced together with that shortcut. The new one holds the index
+ * key from the root, so it covers what the old one did.
+ */
+ if (node->back_pointer &&
+ assoc_array_ptr_is_shortcut(node->back_pointer)) {
+ struct assoc_array_shortcut *above =
+ assoc_array_ptr_to_shortcut(node->back_pointer);
+
+ new_s0->back_pointer = above->back_pointer;
+ new_s0->parent_slot = above->parent_slot;
+ edit->excised_meta[1] = node->back_pointer;
+ }
new_s0->next_node = assoc_array_node_to_ptr(new_n0);
new_n0->back_pointer = assoc_array_shortcut_to_ptr(new_s0);
new_n0->parent_slot = 0;
- new_n1->back_pointer = assoc_array_node_to_ptr(new_n0);
- new_n1->parent_slot = -1; /* Need to calculate this */
new_s0->skip_to_level = level = diff & ~ASSOC_ARRAY_LEVEL_STEP_MASK;
pr_devel("skip_to_level = %d [diff %d]\n", level, diff);
@@ -773,18 +762,104 @@ static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
/* This now reduces to a node splitting exercise for which we'll need
* to regenerate the disparity table.
*/
- for (i = 0; i < ASSOC_ARRAY_FAN_OUT; i++) {
- ptr = node->slots[i];
- base_seg = ops->get_object_key_chunk(assoc_array_ptr_to_leaf(ptr),
- level);
- base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK;
- edit->segment_cache[i] = base_seg & ASSOC_ARRAY_FAN_MASK;
- }
-
+ assoc_array_fill_segment_cache(edit, ops, node, level);
base_seg = ops->get_key_chunk(index_key, level);
base_seg >>= level & ASSOC_ARRAY_KEY_CHUNK_MASK;
edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = base_seg & ASSOC_ARRAY_FAN_MASK;
- goto do_split_node;
+ return new_s0;
+}
+
+/*
+ * Handle insertion into a terminal node.
+ */
+static bool assoc_array_insert_into_terminal_node(struct assoc_array_edit *edit,
+ const struct assoc_array_ops *ops,
+ const void *index_key,
+ struct assoc_array_walk_result *result)
+{
+ struct assoc_array_shortcut *new_s0;
+ struct assoc_array_node *node, *new_n0, *new_n1;
+ bool have_meta;
+ int level;
+
+ node = result->terminal_node.node;
+ level = result->terminal_node.level;
+
+ pr_devel("-->%s()\n", __func__);
+
+ /* We arrived at a node which doesn't have an onward node or shortcut
+ * pointer that we have to follow. This means that (a) the leaf we
+ * want must go here (either by insertion or replacement) or (b) we
+ * need to split this node and insert in one of the fragments.
+ */
+ if (assoc_array_insert_in_node(edit, ops, index_key, node)) {
+ pr_devel("<--%s() = ok [in node]\n", __func__);
+ return true;
+ }
+
+ /* The node has no spare slots - so we're either going to have to split
+ * it or insert another node before it.
+ *
+ * Whatever, we're going to need at least two new nodes - so allocate
+ * those now. We may also need a new shortcut, but we deal with that
+ * when we need it.
+ */
+ new_n0 = kzalloc_obj(struct assoc_array_node);
+ if (!new_n0)
+ return false;
+ edit->new_meta[0] = assoc_array_node_to_ptr(new_n0);
+ new_n1 = kzalloc_obj(struct assoc_array_node);
+ if (!new_n1)
+ return false;
+ edit->new_meta[1] = assoc_array_node_to_ptr(new_n1);
+
+ /* We need to find out how similar the leaves are. */
+ pr_devel("no spare slots\n");
+ have_meta = assoc_array_fill_segment_cache(edit, ops, node, level);
+ edit->segment_cache[ASSOC_ARRAY_FAN_OUT] = result->terminal_node.slot;
+
+ if (!have_meta && assoc_array_all_one_segment(edit)) {
+ /* The node only contains leaves and all the leaves cluster in
+ * the same slot, so we need to insert a shortcut to skip the
+ * common portion of the key and split the node at the
+ * shortcut's far end. The shortcut goes wherever its back
+ * pointer says, which may be further up than the node.
+ */
+ new_s0 = assoc_array_insert_shortcut(edit, ops, index_key,
+ node, level, new_n0);
+ if (!new_s0)
+ return false;
+ edit->set[0].ptr = assoc_array_parent_link(edit->array,
+ new_s0->back_pointer,
+ new_s0->parent_slot);
+ edit->set[0].to = assoc_array_shortcut_to_ptr(new_s0);
+ } else {
+ /* We need to split the current node. The node must contain
+ * anything from a single leaf (in the one leaf case, this leaf
+ * will cluster with the new leaf) and the rest meta-pointers,
+ * to all leaves, some of which may cluster.
+ *
+ * We need to expel at least two leaves out of a set consisting
+ * of the leaves in the node and the new leaf. The current meta
+ * pointers can just be copied as they shouldn't cluster with
+ * any of the leaves. If all the old leaves cluster in one slot
+ * and the new leaf does not, they all go and the new leaf stays.
+ *
+ * We need a new node (n0) to replace the current one and a new
+ * node to take the expelled nodes (n1).
+ */
+ new_n0->back_pointer = node->back_pointer;
+ new_n0->parent_slot = node->parent_slot;
+ edit->set[0].ptr = assoc_array_parent_link(edit->array,
+ new_n0->back_pointer,
+ new_n0->parent_slot);
+ edit->set[0].to = assoc_array_node_to_ptr(new_n0);
+ }
+
+ assoc_array_split_node(edit, node, new_n0, new_n1);
+ edit->excised_meta[0] = assoc_array_node_to_ptr(node);
+ pr_devel("<--%s() = ok [split node]\n", __func__);
+ return true;
}
/*
@@ -843,7 +918,8 @@ static bool assoc_array_insert_mid_shortcut(struct assoc_array_edit *edit,
* zero length - otherwise we just connect the new node directly to the
* parent.
*/
- level += ASSOC_ARRAY_LEVEL_STEP;
+ if (shortcut->back_pointer)
+ level += ASSOC_ARRAY_LEVEL_STEP;
if (diff > level) {
pr_devel("pre-shortcut %d...%d\n", level, diff);
keylen = round_up(diff, ASSOC_ARRAY_KEY_CHUNK_SIZE);
@@ -865,9 +941,11 @@ static bool assoc_array_insert_mid_shortcut(struct assoc_array_edit *edit,
memcpy(new_s0->index_key, shortcut->index_key,
flex_array_size(new_s0, index_key, keylen));
- blank = ULONG_MAX << (diff & ASSOC_ARRAY_KEY_CHUNK_MASK);
- pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, diff, blank);
- new_s0->index_key[keylen - 1] &= ~blank;
+ if (diff & ASSOC_ARRAY_KEY_CHUNK_MASK) {
+ blank = ULONG_MAX << (diff & ASSOC_ARRAY_KEY_CHUNK_MASK);
+ pr_devel("blank off [%zu] %d: %lx\n", keylen - 1, diff, blank);
+ new_s0->index_key[keylen - 1] &= ~blank;
+ }
} else {
pr_devel("no pre-shortcut\n");
edit->set[0].to = assoc_array_node_to_ptr(new_n0);
@@ -1482,16 +1560,25 @@ int assoc_array_gc(struct assoc_array *array,
cursor = array->root;
descend:
- /* If this point is a shortcut, then we need to duplicate it and
- * advance the target cursor.
+ /* The node at this position is duplicated, and so is the shortcut in
+ * front of it if there is one. The node is allocated first: the copy
+ * of the shortcut starts out pointing at the old node, and must not be
+ * in the new tree if the node cannot be allocated, or the cleanup
+ * would follow it into the old tree.
*/
+ new_n = kzalloc_obj(struct assoc_array_node);
+ if (!new_n)
+ goto enomem;
+
if (assoc_array_ptr_is_shortcut(cursor)) {
shortcut = assoc_array_ptr_to_shortcut(cursor);
keylen = round_up(shortcut->skip_to_level, ASSOC_ARRAY_KEY_CHUNK_SIZE);
keylen >>= ASSOC_ARRAY_KEY_CHUNK_SHIFT;
new_s = kmalloc_flex(*new_s, index_key, keylen);
- if (!new_s)
+ if (!new_s) {
+ kfree(new_n);
goto enomem;
+ }
pr_devel("dup shortcut %p -> %p\n", shortcut, new_s);
memcpy(new_s, shortcut, struct_size(new_s, index_key, keylen));
new_s->back_pointer = new_parent;
@@ -1501,11 +1588,7 @@ int assoc_array_gc(struct assoc_array *array,
cursor = shortcut->next_node;
}
- /* Duplicate the node at this position */
node = assoc_array_ptr_to_node(cursor);
- new_n = kzalloc_obj(struct assoc_array_node);
- if (!new_n)
- goto enomem;
pr_devel("dup node %p -> %p\n", node, new_n);
new_n->back_pointer = new_parent;
new_n->parent_slot = node->parent_slot;
diff --git a/lib/base64.c b/lib/base64.c
index 325c7332b049..e46be3a55585 100644
--- a/lib/base64.c
+++ b/lib/base64.c
@@ -52,11 +52,14 @@ static const char base64_tables[][65] = {
INIT_32(0x60, ch_62, ch_63), \
[0x80 ... 0xff] = -1 }
+__diag_push();
+__diag_ignore(clang, all, "-Wconstant-conversion", "https://github.com/llvm/llvm-project/issues/223923");
static const s8 base64_rev_maps[][256] = {
[BASE64_STD] = BASE64_REV_INIT('+', '/'),
[BASE64_URLSAFE] = BASE64_REV_INIT('-', '_'),
[BASE64_IMAP] = BASE64_REV_INIT('+', ',')
};
+__diag_pop();
#undef BASE64_REV_INIT
#undef INIT_32
diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig
index 56f308dd0273..83b4a211c15d 100644
--- a/lib/crypto/Kconfig
+++ b/lib/crypto/Kconfig
@@ -237,7 +237,7 @@ config CRYPTO_LIB_SHA1_ARCH
depends on CRYPTO_LIB_SHA1 && !UML
default y if ARM
default y if ARM64
- default y if MIPS && CPU_CAVIUM_OCTEON
+ default y if MIPS && CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN
default y if PPC
default y if S390
default y if SPARC64
@@ -255,7 +255,7 @@ config CRYPTO_LIB_SHA256_ARCH
depends on CRYPTO_LIB_SHA256 && !UML
default y if ARM && !CPU_V7M
default y if ARM64
- default y if MIPS && CPU_CAVIUM_OCTEON
+ default y if MIPS && CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN
default y if PPC && SPE
default y if RISCV && 64BIT && TOOLCHAIN_HAS_VECTOR_CRYPTO && \
RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS
@@ -275,7 +275,7 @@ config CRYPTO_LIB_SHA512_ARCH
depends on CRYPTO_LIB_SHA512 && !UML
default y if ARM && !CPU_V7M
default y if ARM64
- default y if MIPS && CPU_CAVIUM_OCTEON
+ default y if MIPS && CPU_CAVIUM_OCTEON && CPU_BIG_ENDIAN
default y if RISCV && 64BIT && TOOLCHAIN_HAS_VECTOR_CRYPTO && \
RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS
default y if S390
diff --git a/lib/crypto/mldsa.c b/lib/crypto/mldsa.c
index c96fddc4e7dc..80e0e63b418a 100644
--- a/lib/crypto/mldsa.c
+++ b/lib/crypto/mldsa.c
@@ -306,7 +306,7 @@ static const u8 *decode_t1_elem(struct mldsa_ring_elem *out,
* Reference: FIPS 204 Algorithm 27, sigDecode.
*
* This also validates that the coefficients of z are in range, corresponding
- * the infinity norm check at the end of Algorithm 8, ML-DSA.Verify_internal.
+ * to the infinity norm check at the end of Algorithm 8, ML-DSA.Verify_internal.
*
* Finally, this also converts z to NTT form.
*/
@@ -528,15 +528,21 @@ static size_t encode_w1(u8 out[MAX_W1_ENCODED_LEN],
int mldsa_verify(enum mldsa_alg alg, const u8 *sig, size_t sig_len,
const u8 *msg, size_t msg_len, const u8 *pk, size_t pk_len)
{
- const struct mldsa_parameter_set *params = &mldsa_parameter_sets[alg];
- const int k = params->k, l = params->l;
+ const struct mldsa_parameter_set *params;
+ int k, l;
/* For now this just does pure ML-DSA with an empty context string. */
static const u8 msg_prefix[2] = { /* dom_sep= */ 0, /* ctx_len= */ 0 };
const u8 *ctilde; /* The signer's commitment hash */
- const u8 *t1_encoded = &pk[RHO_LEN]; /* Next encoded element of t_1 */
+ const u8 *t1_encoded; /* Next encoded element of t_1 */
u8 *h; /* The signer's hint vector, length k * N */
size_t w1_enc_len;
+ if (alg != MLDSA44 && alg != MLDSA65 && alg != MLDSA87)
+ return -EINVAL;
+ params = &mldsa_parameter_sets[alg];
+ k = params->k;
+ l = params->l;
+
/* Validate the public key and signature lengths. */
if (pk_len != params->pk_len || sig_len != params->sig_len)
return -EBADMSG;
@@ -587,6 +593,7 @@ int mldsa_verify(enum mldsa_alg alg, const u8 *sig, size_t sig_len,
* The computation is the same for each of the k rows. Just do each row
* before moving on to the next, resulting in only one loop over k.
*/
+ t1_encoded = &pk[RHO_LEN];
for (int i = 0; i < k; i++) {
/*
* tmp = NTT(A) * NTT(z) * 2^-32
diff --git a/lib/crypto/tests/mldsa_kunit.c b/lib/crypto/tests/mldsa_kunit.c
index ec64e10c97f2..d0ee704a3694 100644
--- a/lib/crypto/tests/mldsa_kunit.c
+++ b/lib/crypto/tests/mldsa_kunit.c
@@ -372,6 +372,14 @@ static void test_mldsa_use_hint(struct kunit *test)
}
}
+static void test_mldsa_invalid_alg(struct kunit *test)
+{
+ KUNIT_ASSERT_EQ(test, -EINVAL,
+ mldsa_verify(MLDSA44 - 1, NULL, 0, NULL, 0, NULL, 0));
+ KUNIT_ASSERT_EQ(test, -EINVAL,
+ mldsa_verify(MLDSA87 + 1, NULL, 0, NULL, 0, NULL, 0));
+}
+
static void benchmark_mldsa(struct kunit *test,
const struct mldsa_testvector *tv)
{
@@ -414,6 +422,7 @@ static struct kunit_case mldsa_kunit_cases[] = {
KUNIT_CASE(test_mldsa65),
KUNIT_CASE(test_mldsa87),
KUNIT_CASE(test_mldsa_use_hint),
+ KUNIT_CASE(test_mldsa_invalid_alg),
KUNIT_CASE(benchmark_mldsa44),
KUNIT_CASE(benchmark_mldsa65),
KUNIT_CASE(benchmark_mldsa87),
diff --git a/mm/cma.c b/mm/cma.c
index a10ea37a261d..9ec1d1fd5dff 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -381,6 +381,8 @@ static int __init cma_fixed_reserve(phys_addr_t base, phys_addr_t size)
return -EBUSY;
}
+ accept_memory(base, size);
+
return 0;
}
@@ -678,6 +680,8 @@ int __init cma_declare_contiguous_multi(phys_addr_t total_size,
break;
}
+ accept_memory(mlp->base, size);
+
pr_debug("created region %d: %016llx - %016llx\n",
nr, (u64)mlp->base, (u64)mlp->base + size);
cmrp = &cma->ranges[nr++];
@@ -1020,7 +1024,7 @@ bool cma_release(struct cma *cma, const struct page *pages,
unsigned long count)
{
struct cma_memrange *cmr;
- unsigned long ret = 0;
+ unsigned long leaked = 0;
unsigned long i, pfn;
cmr = find_cma_memrange(cma, pages, count);
@@ -1029,9 +1033,13 @@ bool cma_release(struct cma *cma, const struct page *pages,
pfn = page_to_pfn(pages);
for (i = 0; i < count; i++, pfn++)
- ret += !put_page_testzero(pfn_to_page(pfn));
+ leaked += !page_ref_freeze(pfn_to_page(pfn), 1);
- WARN(ret, "%lu pages are still in use!\n", ret);
+ if (leaked) {
+ pr_warn_ratelimited("%lu pages are still in use, not freeing CMA region!\n",
+ leaked);
+ return true;
+ }
__cma_release_frozen(cma, cmr, pages, count);
diff --git a/mm/damon/core.c b/mm/damon/core.c
index e018635774be..85bcb6da9ad0 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -3200,6 +3200,26 @@ static void damos_trace_esz(struct damon_ctx *c, struct damos *s,
trace_damos_esz(cidx, sidx, quota->esz);
}
+static void damos_fix_stale_charge_pointers(struct damon_ctx *c,
+ struct damos *s)
+{
+ struct damos_quota *quota;
+ struct damon_target *t;
+
+ quota = &s->quota;
+ if (!quota->charge_target_from)
+ return;
+ damon_for_each_target(t, c) {
+ if (t == quota->charge_target_from) {
+ if (damon_nr_regions(t) > 0)
+ return;
+ break;
+ }
+ }
+ quota->charge_target_from = NULL;
+ quota->charge_addr_from = 0;
+}
+
static void damos_adjust_quota(struct damon_ctx *c, struct damos *s)
{
struct damos_quota *quota = &s->quota;
@@ -3235,6 +3255,8 @@ static void damos_adjust_quota(struct damon_ctx *c, struct damos *s)
damos_trace_esz(c, s, quota);
}
+ damos_fix_stale_charge_pointers(c, s);
+
if (!c->ops.get_scheme_score)
return;
@@ -3546,12 +3568,13 @@ static void damon_split_regions_of(struct damon_ctx *ctx,
static void damon_split_some_regions(struct damon_ctx *ctx,
unsigned long split_step)
{
- static unsigned long rotation;
struct damon_target *t;
struct damon_region *r, *next;
- unsigned long offset = rotation++ % split_step;
+ unsigned long offset;
unsigned long idx = 0;
+ offset = ctx->region_split_rotation++ % split_step;
+
damon_for_each_target(t, ctx) {
damon_for_each_region_safe(r, next, t) {
unsigned long sz_region, sz_sub;
@@ -3588,7 +3611,6 @@ static void kdamond_split_regions(struct damon_ctx *ctx)
struct damon_target *t;
unsigned long nr_regions = 0;
unsigned long max_nr_regions = ctx->attrs.max_nr_regions;
- static unsigned long last_nr_regions;
int nr_subregions = 2;
damon_for_each_target(t, ctx)
@@ -3604,7 +3626,7 @@ static void kdamond_split_regions(struct damon_ctx *ctx)
}
/* Maybe the middle of the region has different access frequency */
- if (last_nr_regions == nr_regions &&
+ if (ctx->last_nr_regions == nr_regions &&
nr_regions < max_nr_regions / 3)
nr_subregions = 3;
@@ -3613,7 +3635,7 @@ static void kdamond_split_regions(struct damon_ctx *ctx)
ctx->min_region_sz);
done:
- last_nr_regions = nr_regions;
+ ctx->last_nr_regions = nr_regions;
}
/*
diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c
index 974c456a9e09..20eb8c242f2f 100644
--- a/mm/damon/sysfs-schemes.c
+++ b/mm/damon/sysfs-schemes.c
@@ -1300,7 +1300,10 @@ static ssize_t target_metric_store(struct kobject *kobj,
metric_name = &damos_sysfs_qgoal_metric_names[i];
if (sysfs_streq(buf, metric_name->name)) {
+ if (!mutex_trylock(&damon_sysfs_lock))
+ return -EBUSY;
goal->metric = metric_name->metric;
+ mutex_unlock(&damon_sysfs_lock);
return count;
}
}
@@ -2831,12 +2834,11 @@ static int damon_sysfs_add_scheme_filters(struct damos *scheme,
} else if (filter->type == DAMOS_FILTER_TYPE_TARGET) {
filter->target_idx = sysfs_filter->target_idx;
} else if (filter->type == DAMOS_FILTER_TYPE_HUGEPAGE_SIZE) {
- if (sysfs_filter->sz_range.min >
- sysfs_filter->sz_range.max) {
+ filter->sz_range = sysfs_filter->sz_range;
+ if (filter->sz_range.min > filter->sz_range.max) {
damos_destroy_filter(filter);
return -EINVAL;
}
- filter->sz_range = sysfs_filter->sz_range;
}
damos_add_filter(scheme, filter);
diff --git a/mm/filemap.c b/mm/filemap.c
index 00fd89cf6f55..73a26cd5b6d5 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -227,18 +227,6 @@ void __filemap_remove_folio(struct folio *folio, void *shadow)
page_cache_delete(mapping, folio, shadow);
}
-static void filemap_free_folio(const struct address_space *mapping,
- struct folio *folio)
-{
- void (*free_folio)(struct folio *);
-
- free_folio = mapping->a_ops->free_folio;
- if (free_folio)
- free_folio(folio);
-
- folio_put_refs(folio, folio_nr_pages(folio));
-}
-
/**
* filemap_remove_folio - Remove folio from page cache.
* @folio: The folio.
@@ -250,6 +238,7 @@ static void filemap_free_folio(const struct address_space *mapping,
void filemap_remove_folio(struct folio *folio)
{
struct address_space *mapping = folio->mapping;
+ void (*free_folio)(struct folio *folio);
BUG_ON(!folio_test_locked(folio));
spin_lock(&mapping->host->i_lock);
@@ -258,9 +247,12 @@ void filemap_remove_folio(struct folio *folio)
xa_unlock_irq(&mapping->i_pages);
if (mapping_shrinkable(mapping))
inode_lru_list_add(mapping->host);
+ free_folio = mapping->a_ops->free_folio;
spin_unlock(&mapping->host->i_lock);
- filemap_free_folio(mapping, folio);
+ if (free_folio)
+ free_folio(folio);
+ folio_put_refs(folio, folio_nr_pages(folio));
}
/*
@@ -320,6 +312,7 @@ static void page_cache_delete_batch(struct address_space *mapping,
void delete_from_page_cache_batch(struct address_space *mapping,
struct folio_batch *fbatch)
{
+ void (*free_folio)(struct folio *folio);
int i;
if (!folio_batch_count(fbatch))
@@ -337,10 +330,16 @@ void delete_from_page_cache_batch(struct address_space *mapping,
xa_unlock_irq(&mapping->i_pages);
if (mapping_shrinkable(mapping))
inode_lru_list_add(mapping->host);
+ free_folio = mapping->a_ops->free_folio;
spin_unlock(&mapping->host->i_lock);
- for (i = 0; i < folio_batch_count(fbatch); i++)
- filemap_free_folio(mapping, fbatch->folios[i]);
+ for (i = 0; i < folio_batch_count(fbatch); i++) {
+ struct folio *folio = fbatch->folios[i];
+
+ if (free_folio)
+ free_folio(folio);
+ folio_put_refs(folio, folio_nr_pages(folio));
+ }
}
int filemap_check_errors(struct address_space *mapping)
@@ -1375,10 +1374,22 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr,
* return value based on that state without races.
*
* Also note that WQ_FLAG_WOKEN is sufficient for a non-exclusive
- * waiter, but an exclusive one requires WQ_FLAG_DONE.
+ * waiter, but an exclusive one must also acquire the bit.
*/
- if (behavior == EXCLUSIVE)
- return wait->flags & WQ_FLAG_DONE ? 0 : -EINTR;
+ if (behavior == EXCLUSIVE) {
+ if (wait->flags & WQ_FLAG_DONE)
+ return 0;
+ /*
+ * We might have just been woken up and the waker will not
+ * wake up anybody else. If we ignore that now, the remaining
+ * waiters may not get woken up: take the bit now if it is
+ * still free.
+ */
+ if ((wait->flags & WQ_FLAG_WOKEN) &&
+ !test_and_set_bit(bit_nr, folio_flags(folio, 0)))
+ return 0;
+ return -EINTR;
+ }
return wait->flags & WQ_FLAG_WOKEN ? 0 : -EINTR;
}
@@ -3279,11 +3290,11 @@ loff_t mapping_seek_hole_data(struct address_space *mapping, loff_t start,
* @folio - the folio to lock.
* @fpin - the pointer to the file we may pin (or is already pinned).
*
- * This works similar to lock_folio_or_retry in that it can drop the
- * mmap_lock. It differs in that it actually returns the folio locked
- * if it returns 1 and 0 if it couldn't lock the folio. If we did have
- * to drop the mmap_lock then fpin will point to the pinned file and
- * needs to be fput()'ed at a later point.
+ * Returns 1 with the folio locked, or 0 if locking failed.
+ *
+ * On both entry and return, a non-NULL *fpin means the fault lock has
+ * already been dropped and the caller must eventually fput() the file.
+ * Otherwise the fault lock is still held, including on failure.
*/
static int lock_folio_maybe_drop_mmap(struct vm_fault *vmf, struct folio *folio,
struct file **fpin)
@@ -3300,21 +3311,9 @@ static int lock_folio_maybe_drop_mmap(struct vm_fault *vmf, struct folio *folio,
return 0;
*fpin = maybe_unlock_mmap_for_io(vmf, *fpin);
- if (vmf->flags & FAULT_FLAG_KILLABLE) {
- if (__folio_lock_killable(folio)) {
- /*
- * We didn't have the right flags to drop the
- * fault lock, but all fault_handlers only check
- * for fatal signals if we return VM_FAULT_RETRY,
- * so we need to drop the fault lock here and
- * return 0 if we don't have a fpin.
- */
- if (*fpin == NULL)
- release_fault_lock(vmf);
- return 0;
- }
- } else
- __folio_lock(folio);
+ if (vmf->flags & FAULT_FLAG_KILLABLE)
+ return !__folio_lock_killable(folio);
+ __folio_lock(folio);
return 1;
}
@@ -3543,7 +3542,7 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf)
* vma->vm_mm->mmap_lock must be held on entry.
*
* If our return value has VM_FAULT_RETRY set, it's because the mmap_lock
- * may be dropped before doing I/O or by lock_folio_maybe_drop_mmap().
+ * may be dropped before doing I/O or when aborting an interrupted wait.
*
* If our return value does not have VM_FAULT_RETRY set, the mmap_lock
* has not been released.
@@ -3699,9 +3698,8 @@ vm_fault_t filemap_fault(struct vm_fault *vmf)
out_retry:
/*
- * We dropped the mmap_lock, we need to return to the fault handler to
- * re-find the vma and come back and find our hopefully still populated
- * page.
+ * With fpin set, the fault lock was already dropped. Otherwise it is
+ * still held. Release invalidate_lock before dropping either protection.
*/
if (!IS_ERR(folio))
folio_put(folio);
@@ -3709,6 +3707,8 @@ vm_fault_t filemap_fault(struct vm_fault *vmf)
filemap_invalidate_unlock_shared(mapping);
if (fpin)
fput(fpin);
+ else if (!(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
+ release_fault_lock(vmf);
return ret | VM_FAULT_RETRY;
}
EXPORT_SYMBOL(filemap_fault);
diff --git a/mm/gup.c b/mm/gup.c
index eb898ea1ee22..4f58ad22a756 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -658,9 +658,6 @@ static struct page *follow_huge_pud(struct vm_area_struct *vma,
assert_spin_locked(pud_lockptr(mm, pudp));
- if (!pud_present(pud))
- return NULL;
-
if ((flags & FOLL_WRITE) &&
!can_follow_write_pud(pud, pfn_to_page(pfn), vma, flags))
return NULL;
@@ -896,15 +893,17 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
}
static struct page *follow_pmd_mask(struct vm_area_struct *vma,
- unsigned long address, pud_t *pudp,
- unsigned int flags,
- unsigned long *page_mask)
+ unsigned long address, pud_t *pudp, pud_t pud,
+ unsigned int flags, unsigned long *page_mask)
{
pmd_t *pmd, pmdval;
spinlock_t *ptl;
struct page *page;
struct mm_struct *mm = vma->vm_mm;
+ if (unlikely(pud_bad(pud)))
+ return no_page_table(vma, flags, address);
+
pmd = pmd_offset(pudp, address);
pmdval = pmdp_get_lockless(pmd);
if (pmd_none(pmdval))
@@ -953,18 +952,23 @@ static struct page *follow_pud_mask(struct vm_area_struct *vma,
pud = pudp_get(pudp);
if (!pud_present(pud))
return no_page_table(vma, flags, address);
- if (pud_leaf(pud)) {
- ptl = pud_lock(mm, pudp);
- page = follow_huge_pud(vma, address, pudp, flags, page_mask);
+ if (!pud_leaf(pud))
+ return follow_pmd_mask(vma, address, pudp, pud, flags, page_mask);
+
+ ptl = pud_lock(mm, pudp);
+ pud = *pudp;
+ if (unlikely(!pud_present(pud))) {
spin_unlock(ptl);
- if (page)
- return page;
return no_page_table(vma, flags, address);
}
- if (unlikely(pud_bad(pud)))
- return no_page_table(vma, flags, address);
+ if (unlikely(!pud_leaf(pud))) {
+ spin_unlock(ptl);
+ return follow_pmd_mask(vma, address, pudp, pud, flags, page_mask);
+ }
- return follow_pmd_mask(vma, address, pudp, flags, page_mask);
+ page = follow_huge_pud(vma, address, pudp, flags, page_mask);
+ spin_unlock(ptl);
+ return page;
}
static struct page *follow_p4d_mask(struct vm_area_struct *vma,
@@ -3186,6 +3190,9 @@ static int gup_fast_fallback(unsigned long start, unsigned long nr_pages,
FOLL_PCI_P2PDMA | FOLL_HONOR_NUMA_FAULT)))
return -EINVAL;
+ if (unlikely(!nr_pages))
+ return 0;
+
if (gup_flags & FOLL_PIN)
mm_set_has_pinned_flag(current->mm);
@@ -3193,8 +3200,8 @@ static int gup_fast_fallback(unsigned long start, unsigned long nr_pages,
might_lock_read(¤t->mm->mmap_lock);
start = untagged_addr(start) & PAGE_MASK;
- len = nr_pages << PAGE_SHIFT;
- if (check_add_overflow(start, len, &end))
+ if (check_shl_overflow(nr_pages, PAGE_SHIFT, &len) ||
+ check_add_overflow(start, len, &end))
return -EOVERFLOW;
if (end > TASK_SIZE_MAX)
return -EFAULT;
@@ -3439,7 +3446,7 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
{
unsigned int flags, nr_folios, nr_found;
unsigned int i, pgshift = PAGE_SHIFT;
- pgoff_t start_idx, end_idx;
+ pgoff_t start_idx, end_idx, retry_idx;
struct folio *folio = NULL;
struct folio_batch fbatch;
struct hstate *h;
@@ -3480,6 +3487,7 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
* reason, we try to allocate them and add them to the
* page cache.
*/
+ retry_idx = start_idx;
nr_found = filemap_get_folios_contig(memfd->f_mapping,
&start_idx,
end_idx,
@@ -3492,12 +3500,38 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
for (i = 0; i < nr_found; i++) {
folio = fbatch.folios[i];
+ /*
+ * The folio might have been removed from the
+ * page cache (e.g., truncated or collapsed)
+ * after the lookup. Recheck under the folio
+ * lock.
+ */
+ folio_lock(folio);
+ if (unlikely(folio->mapping != memfd->f_mapping)) {
+ folio_unlock(folio);
+ start_idx = retry_idx;
+ break;
+ }
+
if (try_grab_folio(folio, 1, FOLL_PIN)) {
+ folio_unlock(folio);
folio_batch_release(&fbatch);
ret = -EINVAL;
goto err;
}
+ /*
+ * fallocate() can leave folios in the
+ * page cache that were never cleared.
+ */
+ if (!folio_test_uptodate(folio)) {
+ folio_zero_range(folio, 0, folio_size(folio));
+ flush_dcache_folio(folio);
+ folio_mark_uptodate(folio);
+ }
+ retry_idx = folio_next_index(folio);
+ folio_unlock(folio);
+
if (nr_folios == 0)
*offset = offset_in_folio(folio, start);
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 779c02e0bf6c..5f7e3320d4d0 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2073,11 +2073,13 @@ void touch_pud(struct vm_area_struct *vma, unsigned long addr,
update_mmu_cache_pud(vma, addr, pud);
}
-int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm,
- pud_t *dst_pud, pud_t *src_pud, unsigned long addr,
- struct vm_area_struct *vma)
+int copy_huge_pud(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
+ pud_t *dst_pud, pud_t *src_pud, unsigned long addr)
{
+ struct mm_struct *dst_mm = dst_vma->vm_mm;
+ struct mm_struct *src_mm = src_vma->vm_mm;
spinlock_t *dst_ptl, *src_ptl;
+ struct page *page;
pud_t pud;
int ret;
@@ -2090,11 +2092,20 @@ int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm,
if (unlikely(!pud_trans_huge(pud)))
goto out_unlock;
- /*
- * TODO: once we support anonymous pages, use
- * folio_try_dup_anon_rmap_*() and split if duplicating fails.
- */
- if (vma_is_cow_mapping(vma) && pud_write(pud)) {
+ page = vm_normal_page_pud(src_vma, addr, pud);
+ if (page) {
+ struct folio *folio = page_folio(page);
+
+ folio_get(folio);
+ /*
+ * TODO: once we support anonymous pages, use
+ * folio_try_dup_anon_rmap_*() and split if duplicating fails.
+ */
+ folio_add_file_rmap_pud(folio, page, dst_vma);
+ add_mm_counter(dst_mm, mm_counter_file(folio), HPAGE_PUD_NR);
+ }
+
+ if (vma_is_cow_mapping(src_vma) && pud_write(pud)) {
pudp_set_wrprotect(src_mm, addr, src_pud);
pud = pud_wrprotect(pud);
}
@@ -2483,11 +2494,7 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)
static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
pmd_t pmdval, struct folio *folio, bool is_present)
{
- const bool is_device_private = folio_is_device_private(folio);
-
- /* Present and device private folios are rmappable. */
- if (is_present || is_device_private)
- folio_remove_rmap_pmd(folio, &folio->page, vma);
+ const bool is_device_private = pmd_is_device_private_entry(pmdval);
if (folio_test_anon(folio)) {
add_mm_counter(mm, MM_ANONPAGES, -HPAGE_PMD_NR);
@@ -2502,6 +2509,13 @@ static void zap_huge_pmd_folio(struct mm_struct *mm, struct vm_area_struct *vma,
folio_mark_accessed(folio);
}
+ /*
+ * Present and device private folios are rmappable. Remove the
+ * rmap after transferring young+dirty to the folio.
+ */
+ if (is_present || is_device_private)
+ folio_remove_rmap_pmd(folio, &folio->page, vma);
+
/* Device private folios are pinned. */
if (is_device_private)
folio_put(folio);
@@ -2645,15 +2659,11 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,
return false;
}
- /*
- * We don't have to worry about the ordering of src and dst
- * ptlocks because exclusive mmap_lock prevents deadlock.
- */
- old_ptl = __pmd_trans_huge_lock(old_pmd, vma);
- if (old_ptl) {
- new_ptl = pmd_lockptr(mm, new_pmd);
- if (new_ptl != old_ptl)
- spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
+ old_ptl = pmd_lockptr(mm, old_pmd);
+ new_ptl = pmd_lockptr(mm, new_pmd);
+
+ double_pt_lock(old_ptl, new_ptl);
+ if (likely(pmd_is_huge(*old_pmd))) {
pmd = pmdp_huge_get_and_clear(mm, old_addr, old_pmd);
if (pmd_present(pmd))
force_flush = true;
@@ -2681,11 +2691,10 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,
set_pmd_at(mm, new_addr, new_pmd, pmd);
if (force_flush)
flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
- if (new_ptl != old_ptl)
- spin_unlock(new_ptl);
- spin_unlock(old_ptl);
+ double_pt_unlock(old_ptl, new_ptl);
return true;
}
+ double_pt_unlock(old_ptl, new_ptl);
return false;
}
@@ -3052,6 +3061,8 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma,
spinlock_t *ptl;
pud_t orig_pud;
+ tlb_change_page_size(tlb, HPAGE_PUD_SIZE);
+
ptl = __pud_trans_huge_lock(pud, vma);
if (!ptl)
return 0;
@@ -3507,16 +3518,66 @@ void split_huge_pmd_address(struct vm_area_struct *vma, unsigned long address,
__split_huge_pmd(vma, pmd, address, freeze);
}
-static inline void split_huge_pmd_if_needed(struct vm_area_struct *vma, unsigned long address)
+static bool pud_split_required(const struct vm_area_struct *vma,
+ unsigned long address)
+{
+ if (!IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD))
+ return false;
+ if (IS_ALIGNED(address, HPAGE_PUD_SIZE))
+ return false;
+ /*
+ * Only selected VMAs support mapping PUDs for now. Lift the
+ * restriction once this changes.
+ */
+ if (!vma_is_dax(vma) && !vma_test_any(vma, VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT))
+ return false;
+ return range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PUD_SIZE),
+ ALIGN(address, HPAGE_PUD_SIZE));
+}
+
+static bool pmd_split_required(const struct vm_area_struct *vma,
+ unsigned long address)
{
+ if (IS_ALIGNED(address, HPAGE_PMD_SIZE))
+ return false;
+
+ return range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PMD_SIZE),
+ ALIGN(address, HPAGE_PMD_SIZE));
+}
+
+static void split_huge_leaf(struct vm_area_struct *vma,
+ unsigned long address)
+{
+ const bool split_pud = pud_split_required(vma, address);
+ const bool split_pmd = pmd_split_required(vma, address);
+ pgd_t *pgd;
+ p4d_t *p4d;
+ pud_t *pud;
+ pmd_t *pmd;
+
/*
- * If the new address isn't hpage aligned and it could previously
- * contain an hugepage: check if we need to split an huge pmd.
+ * mmap, VMA and rmap write locks protect the walk; the split helpers
+ * take the page table locks.
*/
- if (!IS_ALIGNED(address, HPAGE_PMD_SIZE) &&
- range_in_vma(vma, ALIGN_DOWN(address, HPAGE_PMD_SIZE),
- ALIGN(address, HPAGE_PMD_SIZE)))
- split_huge_pmd_address(vma, address, false);
+ if (!split_pud && !split_pmd)
+ return;
+
+ pgd = pgd_offset(vma->vm_mm, address);
+ if (!pgd_present(*pgd))
+ return;
+
+ p4d = p4d_offset(pgd, address);
+ if (!p4d_present(*p4d))
+ return;
+
+ pud = pud_offset(p4d, address);
+ if (pud_leaf(*pud))
+ __split_huge_pud(vma, pud, address);
+ if (!split_pmd || !pud_present(*pud))
+ return;
+
+ pmd = pmd_offset(pud, address);
+ __split_huge_pmd(vma, pmd, address, false);
}
void vma_adjust_trans_huge(struct vm_area_struct *vma,
@@ -3525,14 +3586,14 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,
struct vm_area_struct *next)
{
/* Check if we need to split start first. */
- split_huge_pmd_if_needed(vma, start);
+ split_huge_leaf(vma, start);
/* Check if we need to split end next. */
- split_huge_pmd_if_needed(vma, end);
+ split_huge_leaf(vma, end);
/* If we're incrementing next->vm_start, we might need to split it. */
if (next)
- split_huge_pmd_if_needed(next, end);
+ split_huge_leaf(next, end);
}
static void unmap_folio(struct folio *folio)
@@ -4511,8 +4572,7 @@ int split_folio_to_list(struct folio *folio, struct list_head *list)
*
* It is unsafe to call folio_unqueue_deferred_split() until folio refcount is
* zero: because even when the list_lru lock is held, a non-empty
- * _deferred_list might be in use on deferred_split_scan()'s unlocked
- * on-stack list.
+ * _deferred_list might be in use on deferred_split_scan()'s on-stack list.
*
* The list_lru sublist is determined by folio's memcg: it is therefore
* important to unqueue deferred split before changing folio memcg.
@@ -4658,10 +4718,22 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
deferred_split_isolate, &dispose);
list_for_each_entry_safe(folio, next, &dispose, _deferred_list) {
+ struct mem_cgroup *memcg;
+ struct list_lru_one *lru;
bool did_split = false;
bool underused = false;
+ /*
+ * Sync with deferred_split_folio(), which can requeue the
+ * folio once list_empty() is true, and sets PG_partially_mapped.
+ */
+ rcu_read_lock();
+ memcg = folio_memcg(folio);
+ lru = list_lru_lock_irq(&deferred_split_lru, folio_nid(folio),
+ &memcg);
list_del_init(&folio->_deferred_list);
+ list_lru_unlock_irq(lru);
+ rcu_read_unlock();
if (!folio_test_partially_mapped(folio)) {
/*
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index e71e0ab116bf..860c3c6b7673 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -125,6 +125,10 @@ static int __huge_pmd_unshare(struct mmu_gather *tlb,
bool check_locks);
static void hugetlb_unshare_pmds(struct vm_area_struct *vma,
unsigned long start, unsigned long end, bool take_locks);
+static void hugetlb_unmap_range(struct mmu_gather *tlb,
+ struct vm_area_struct *vma, unsigned long start,
+ unsigned long end, struct folio *folio,
+ zap_flags_t zap_flags);
static struct resv_map *vma_resv_map(struct vm_area_struct *vma);
static inline bool subpool_is_free(struct hugepage_subpool *spool)
@@ -4935,10 +4939,11 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
raw_write_seqcount_begin(&src->write_protect_seq);
} else {
/*
- * For shared mappings the vma lock must be held before
- * calling hugetlb_walk() in the src vma. Otherwise, the
- * returned ptep could go away if part of a shared pmd and
- * another thread calls huge_pmd_unshare.
+ * Shared mappings only need copying when we could have
+ * information stored in page tables (e.g., userfaultfd PTE
+ * bits or markers), which is incompatible with PMD sharing.
+ * Still take the hugetlb VMA lock for now to satisfy
+ * hugetlb_walk()'s locking requirements.
*/
hugetlb_vma_lock_read(src_vma);
}
@@ -4946,6 +4951,13 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
last_addr_mask = hugetlb_mask_last_page(h);
for (addr = src_vma->vm_start; addr < src_vma->vm_end; addr += sz) {
spinlock_t *src_ptl, *dst_ptl;
+
+ if (WARN_ON_ONCE(want_pmd_share(src_vma, addr) ||
+ want_pmd_share(dst_vma, addr))) {
+ ret = -EINVAL;
+ break;
+ }
+
src_pte = hugetlb_walk(src_vma, addr, sz);
if (!src_pte) {
addr |= last_addr_mask;
@@ -4957,14 +4969,6 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
break;
}
-#ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING
- /* If the pagetables are shared, there is nothing to do */
- if (ptdesc_pmd_is_shared(virt_to_ptdesc(dst_pte))) {
- addr |= last_addr_mask;
- continue;
- }
-#endif
-
dst_ptl = huge_pte_lock(h, dst, dst_pte);
src_ptl = huge_pte_lockptr(h, src, src_pte);
spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING);
@@ -5115,15 +5119,9 @@ static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr,
spinlock_t *src_ptl, *dst_ptl;
pte_t pte;
- dst_ptl = huge_pte_lock(h, mm, dst_pte);
+ dst_ptl = huge_pte_lockptr(h, mm, dst_pte);
src_ptl = huge_pte_lockptr(h, mm, src_pte);
-
- /*
- * We don't have to worry about the ordering of src and dst ptlocks
- * because exclusive mmap_lock (or the i_mmap_lock) prevents deadlock.
- */
- if (src_ptl != dst_ptl)
- spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING);
+ double_pt_lock(dst_ptl, src_ptl);
pte = huge_ptep_get_and_clear(mm, old_addr, src_pte, sz);
@@ -5152,15 +5150,12 @@ static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr,
set_huge_pte_at(mm, new_addr, dst_pte, pte, sz);
}
- if (src_ptl != dst_ptl)
- spin_unlock(src_ptl);
- spin_unlock(dst_ptl);
+ double_pt_unlock(dst_ptl, src_ptl);
}
-int move_hugetlb_page_tables(struct vm_area_struct *vma,
- struct vm_area_struct *new_vma,
- unsigned long old_addr, unsigned long new_addr,
- unsigned long len)
+unsigned long move_hugetlb_page_tables(struct vm_area_struct *vma,
+ struct vm_area_struct *new_vma, unsigned long old_addr,
+ unsigned long new_addr, unsigned long len)
{
struct hstate *h = hstate_vma(vma);
struct address_space *mapping = vma->vm_file->f_mapping;
@@ -5190,6 +5185,8 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma,
for (; old_addr < old_end; old_addr += sz, new_addr += sz) {
const unsigned long offset_to_last_entry =
(old_addr | last_addr_mask) - old_addr;
+ const unsigned long offset_to_last_dst_entry =
+ (new_addr | last_addr_mask) - new_addr;
src_pte = hugetlb_walk(vma, old_addr, sz);
if (!src_pte) {
@@ -5206,6 +5203,24 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma,
continue;
}
+ /*
+ * Avoid entering the destination PMD sharing path while holding
+ * i_mmap_rwsem for write. Leave destination page table
+ * population to the fault path, which can share the PMD lazily.
+ */
+ if (sz == PMD_SIZE && want_pmd_share(new_vma, new_addr)) {
+ const unsigned long unmap_size =
+ min(old_end - old_addr,
+ offset_to_last_dst_entry + sz);
+
+ hugetlb_unmap_range(&tlb, vma, old_addr,
+ old_addr + unmap_size, NULL,
+ ZAP_FLAG_DROP_MARKER);
+ old_addr += unmap_size - sz;
+ new_addr += unmap_size - sz;
+ continue;
+ }
+
dst_pte = huge_pte_alloc(mm, new_vma, new_addr, sz);
if (!dst_pte)
break;
@@ -5225,9 +5240,10 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma,
return len + old_addr - old_end;
}
-void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
- unsigned long start, unsigned long end,
- struct folio *folio, zap_flags_t zap_flags)
+static void hugetlb_unmap_range(struct mmu_gather *tlb,
+ struct vm_area_struct *vma, unsigned long start,
+ unsigned long end, struct folio *folio,
+ zap_flags_t zap_flags)
{
struct mm_struct *mm = vma->vm_mm;
const bool folio_provided = !!folio;
@@ -5245,13 +5261,6 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
BUG_ON(start & ~huge_page_mask(h));
BUG_ON(end & ~huge_page_mask(h));
- /*
- * This is a hugetlb vma, all the pte entries should point
- * to huge page.
- */
- tlb_change_page_size(tlb, sz);
- tlb_start_vma(tlb, vma);
-
last_addr_mask = hugetlb_mask_last_page(h);
address = start;
for (; address < end; address += sz) {
@@ -5380,6 +5389,15 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
if (folio_provided)
break;
}
+}
+
+void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
+ unsigned long start, unsigned long end,
+ struct folio *folio, zap_flags_t zap_flags)
+{
+ tlb_change_page_size(tlb, huge_page_size(hstate_vma(vma)));
+ tlb_start_vma(tlb, vma);
+ hugetlb_unmap_range(tlb, vma, start, end, folio, zap_flags);
tlb_end_vma(tlb, vma);
huge_pmd_unshare_flush(tlb, vma);
diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c
index 917db0984143..8e5981888937 100644
--- a/mm/hugetlb_vmemmap.c
+++ b/mm/hugetlb_vmemmap.c
@@ -222,10 +222,12 @@ static void vmemmap_remap_pte(pte_t *pte, unsigned long addr,
VM_WARN_ON_ONCE(!PageTail((const struct page *)addr));
/*
- * Remap the tail pages as read-only to catch illegal write
- * operation to the tail pages.
+ * Map the shared tail page writable. A lockless get that read
+ * a non-zero refcount before the remap can still do its
+ * cmpxchg after it. The refcount is zero, so the cmpxchg
+ * fails and leaves the page unchanged.
*/
- entry = mk_pte(walk->vmemmap_tail, PAGE_KERNEL_RO);
+ entry = mk_pte(walk->vmemmap_tail, PAGE_KERNEL);
}
list_add(&page->lru, walk->vmemmap_pages);
diff --git a/mm/internal.h b/mm/internal.h
index 38b1165212c9..526ad50e3c1d 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -584,14 +584,16 @@ void free_pgtables(struct mmu_gather *tlb, struct unmap_desc *desc);
void pmd_install(struct mm_struct *mm, pmd_t *pmd, pgtable_t *pte);
/**
- * sync_with_folio_pmd_zap - sync with concurrent zapping of a folio PMD
+ * sync_with_folio_pmd_zap - sync with concurrent zapping of folios
* @mm: The mm_struct.
* @pmdp: Pointer to the pmd that was found to be pmd_none().
*
* When we find a pmd_none() while unmapping a folio without holding the PTL,
- * zap_huge_pmd() may have cleared the PMD but not yet modified the folio to
- * indicate that it's unmapped. Skipping the PMD without synchronization could
- * make folio unmapping code assume that unmapping failed.
+ * zap_huge_pmd() may have cleared the PMD but not yet updated the folio
+ * mapcount, and zap_empty_pte_table() clears the PMD only after the rmaps
+ * for the table have been removed, but nothing orders the pmd_none() read
+ * against later mapcount reads. Skipping the PMD without synchronization
+ * could make folio unmapping code assume that unmapping failed.
*
* Wait for concurrent zapping to complete by grabbing the PTL.
*/
@@ -619,6 +621,34 @@ static inline void force_page_cache_readahead(struct address_space *mapping,
force_page_cache_ra(&ractl, nr_to_read);
}
+/*
+ * Take two page table locks at the same level in address order such that
+ * we cannot deadlock.
+ */
+static inline void double_pt_lock(spinlock_t *ptl1, spinlock_t *ptl2)
+ __acquires(ptl1)
+ __acquires(ptl2)
+{
+ if (ptl1 > ptl2)
+ swap(ptl1, ptl2);
+ spin_lock(ptl1);
+ if (ptl1 != ptl2)
+ spin_lock_nested(ptl2, SINGLE_DEPTH_NESTING);
+ else
+ __acquire(ptl2);
+}
+
+static inline void double_pt_unlock(spinlock_t *ptl1, spinlock_t *ptl2)
+ __releases(ptl1)
+ __releases(ptl2)
+{
+ spin_unlock(ptl1);
+ if (ptl1 != ptl2)
+ spin_unlock(ptl2);
+ else
+ __release(ptl2);
+}
+
unsigned find_lock_entries(struct address_space *mapping, pgoff_t *start,
pgoff_t end, struct folio_batch *fbatch, pgoff_t *indices);
unsigned find_get_entries(struct address_space *mapping, pgoff_t *start,
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 792166950bba..47ed64f51748 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1928,7 +1928,7 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
result = SCAN_FAIL;
start_pte = pte_offset_map_lock(mm, pmd, haddr, &ptl);
- if (!start_pte) /* mmap_lock + page lock should prevent this */
+ if (!start_pte)
goto drop_folio;
/* step 1: check all mapped PTEs are to the right huge page */
@@ -1959,6 +1959,7 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
}
pte_unmap_unlock(start_pte, ptl);
+ ptl = NULL;
mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, mm,
haddr, haddr + HPAGE_PMD_SIZE);
mmu_notifier_invalidate_range_start(&range);
@@ -1976,7 +1977,7 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
pml = pmd_lock(mm, pmd);
start_pte = pte_offset_map_rw_nolock(mm, pmd, haddr, &pgt_pmd, &ptl);
- if (!start_pte) /* mmap_lock + page lock should prevent this */
+ if (!start_pte)
goto abort;
if (!pml)
spin_lock(ptl);
@@ -2018,9 +2019,10 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
/*
* Must clear entry, or a racing truncate may re-remove it.
* TLB flush can be left until pmdp_collapse_flush() does it.
- * PTE dirty? Shmem page is already dirty; file is read-only.
*/
- clear_ptes(mm, addr, pte, nr_batch_ptes);
+ ptent = get_and_clear_ptes(mm, addr, pte, nr_batch_ptes);
+ if (pte_dirty(ptent))
+ folio_mark_dirty(folio);
folio_remove_rmap_ptes(folio, page, nr_batch_ptes, vma);
nr_mapped_ptes += nr_batch_ptes;
}
@@ -2037,12 +2039,11 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
/* step 4: remove empty page table */
if (!pml) {
pml = pmd_lock(mm, pmd);
- if (ptl != pml) {
+ if (ptl != pml)
spin_lock_nested(ptl, SINGLE_DEPTH_NESTING);
- if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) {
- flush_tlb_mm(mm);
- goto unlock;
- }
+ if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) {
+ flush_tlb_mm(mm);
+ goto unlock;
}
}
pgt_pmd = pmdp_collapse_flush(vma, haddr, pmd);
@@ -2251,6 +2252,7 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
struct file *file, pgoff_t start, struct collapse_control *cc)
{
struct address_space *mapping = file->f_mapping;
+ const bool is_shmem = shmem_mapping(mapping);
struct page *dst;
struct folio *folio, *tmp, *new_folio;
unsigned long new_pfn = -1;
@@ -2259,7 +2261,8 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
XA_STATE_ORDER(xas, &mapping->i_pages, start, HPAGE_PMD_ORDER);
enum scan_result result = SCAN_SUCCEED;
int nr_none = 0;
- bool is_shmem = shmem_file(file);
+ bool do_readahead = false;
+ bool readahead_done = false;
/*
* MADV_COLLAPSE ignores shmem huge config, so do not check shmem
@@ -2277,6 +2280,10 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
mapping_set_update(&xas, mapping);
+retry:
+ result = SCAN_SUCCEED;
+ do_readahead = false;
+ xas_set_order(&xas, start, HPAGE_PMD_ORDER);
__folio_set_locked(new_folio);
if (is_shmem)
__folio_set_swapbacked(new_folio);
@@ -2341,17 +2348,9 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
}
} else { /* !is_shmem */
if (!folio || xa_is_value(folio)) {
- xas_unlock_irq(&xas);
- page_cache_sync_readahead(mapping, &file->f_ra,
- file, index,
- end - index);
- /* drain lru cache to help folio_isolate_lru() */
- lru_add_drain();
- folio = filemap_lock_folio(mapping, index);
- if (IS_ERR(folio)) {
- result = SCAN_FAIL;
- goto xa_unlocked;
- }
+ do_readahead = !readahead_done;
+ result = SCAN_FAIL;
+ goto xa_locked;
} else if (folio_test_dirty(folio)) {
/*
* This page is dirty because it hasn't
@@ -2488,6 +2487,11 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
index += folio_nr_pages(folio);
continue;
out_unlock:
+ /*
+ * The folio may have been unmapped with TTU_BATCH_FLUSH.
+ * Flush before releasing the lock and our last reference.
+ */
+ try_to_unmap_flush();
folio_unlock(folio);
folio_put(folio);
goto xa_unlocked;
@@ -2498,9 +2502,8 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
xa_unlocked:
/*
- * If collapse is successful, flush must be done now before copying.
- * If collapse is unsuccessful, does flush actually need to be done?
- * Do it anyway, to clear the state.
+ * Flush before copying the folios, or releasing them in rollback.
+ * This is a no-op if out_unlock already flushed the batch.
*/
try_to_unmap_flush();
@@ -2674,6 +2677,30 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
new_folio->mapping = NULL;
folio_unlock(new_folio);
+
+ /* Readahead may take invalidate_lock; source folios are unlocked. */
+ if (do_readahead) {
+ for (index = start; index < end;) {
+ folio = filemap_lock_folio(mapping, index);
+ if (IS_ERR(folio)) {
+ page_cache_sync_readahead(mapping, &file->f_ra,
+ file, index, end - index);
+ folio = filemap_lock_folio(mapping, index);
+ if (IS_ERR(folio)) {
+ result = SCAN_FAIL;
+ goto out_put;
+ }
+ }
+ index = folio_next_index(folio);
+ folio_unlock(folio);
+ folio_put(folio);
+ }
+ /* Drain the local LRU batch before retrying isolation. */
+ lru_add_drain();
+ readahead_done = true;
+ goto retry;
+ }
+out_put:
folio_put(new_folio);
out:
VM_BUG_ON(!list_empty(&pagelist));
diff --git a/mm/ksm.c b/mm/ksm.c
index fe42c17490b8..a554e5c0ed07 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -34,6 +34,7 @@
#include <linux/memory.h>
#include <linux/mmu_notifier.h>
#include <linux/swap.h>
+#include <linux/userfaultfd_k.h>
#include <linux/ksm.h>
#include <linux/hashtable.h>
#include <linux/freezer.h>
@@ -475,6 +476,7 @@ static void advisor_stop_scan(void)
#ifdef CONFIG_NUMA
/* Zeroed when merging across nodes is not allowed */
static unsigned int ksm_merge_across_nodes = 1;
+/* Number of allocated stable and unstable tree roots. */
static int ksm_nr_node_ids = 1;
#else
#define ksm_merge_across_nodes 1U
@@ -1462,6 +1464,15 @@ static int replace_page(struct vm_area_struct *vma, struct page *page,
dec_mm_counter(mm, MM_ANONPAGES);
}
+ /* Preserve soft-dirty and uffd protection when replacing the page. */
+ if (pte_soft_dirty(orig_pte))
+ newpte = pte_mksoft_dirty(newpte);
+ if (pte_uffd(orig_pte)) {
+ newpte = pte_mkuffd(newpte);
+ if (userfaultfd_rwp(vma))
+ newpte = pte_modify(newpte, PAGE_NONE);
+ }
+
flush_cache_page(vma, addr, pte_pfn(ptep_get(ptep)));
/*
* No need to notify as we are replacing a read only page with another
@@ -3135,24 +3146,52 @@ void __ksm_exit(struct mm_struct *mm)
trace_ksm_exit(mm);
}
-struct folio *ksm_might_need_to_copy(struct folio *folio,
- struct vm_area_struct *vma, unsigned long addr)
+static bool folio_anon_rmap_compatible(const struct folio *folio,
+ const struct page *page, struct vm_area_struct *vma,
+ unsigned long addr)
{
- struct page *page = folio_page(folio, 0);
- struct anon_vma *anon_vma = folio_anon_vma(folio);
- struct folio *new_folio;
+ const struct anon_vma *anon_vma = folio_anon_vma(folio);
+ struct anon_vma_chain *avc;
- if (folio_test_large(folio))
- return folio;
+ VM_WARN_ON_ONCE_FOLIO(!folio_test_locked(folio), folio);
+ VM_WARN_ON_ONCE_FOLIO(page_folio(page) != folio, folio);
+ VM_WARN_ON_ONCE_FOLIO(folio_test_ksm(folio), folio);
+ vma_assert_stabilised(vma);
+
+ if (!anon_vma)
+ return true;
+ if (page_pgoff(folio, page) != linear_anon_page_index(vma, addr))
+ return false;
+
+ /*
+ * Careful: anon_vma might have been freed, so we cannot safely
+ * dereference it. Add a fast path before we walk the anon_vma chain.
+ */
+ if (anon_vma == vma->anon_vma || anon_vma == vma->anon_vma->root)
+ return true;
+ list_for_each_entry(avc, &vma->anon_vma_chain, same_vma) {
+ if (avc->anon_vma == anon_vma)
+ return true;
+ }
+ return false;
+}
+
+struct folio *ksm_might_need_to_copy(struct folio *folio, struct page *page,
+ struct vm_area_struct *vma, unsigned long addr)
+{
+ struct folio *new_folio;
+
+ /*
+ * A KSM folio is small, but the slot that it left behind can be swapped
+ * in as a part of a large folio by one of the processes that shared it.
+ * So test, and copy, the page of the slot.
+ */
if (folio_test_ksm(folio)) {
if (folio_stable_node(folio) &&
!(ksm_run & KSM_RUN_UNMERGE))
return folio; /* no need to copy it */
- } else if (!anon_vma) {
- return folio; /* no need to copy it */
- } else if (folio->index == linear_anon_page_index(vma, addr) &&
- anon_vma->root == vma->anon_vma->root) {
+ } else if (folio_anon_rmap_compatible(folio, page, vma, addr)) {
return folio; /* still no need to copy it */
}
if (PageHWPoison(page))
@@ -3635,14 +3674,13 @@ static ssize_t merge_across_nodes_store(struct kobject *kobj,
else {
root_stable_tree = buf;
root_unstable_tree = buf + nr_node_ids;
+ ksm_nr_node_ids = nr_node_ids;
/* Stable tree is empty but not the unstable */
root_unstable_tree[0] = one_unstable_tree[0];
}
}
- if (!err) {
+ if (!err)
ksm_merge_across_nodes = knob;
- ksm_nr_node_ids = knob ? 1 : nr_node_ids;
- }
}
mutex_unlock(&ksm_thread_mutex);
diff --git a/mm/madvise.c b/mm/madvise.c
index 54d8397059ba..222c10eb61eb 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -755,6 +755,15 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
if (folio_test_swapcache(folio) || folio_test_dirty(folio)) {
if (!folio_trylock(folio))
continue;
+ /*
+ * KSM folios can gain new mappings at any time, so we must
+ * not clear their dirty flag.
+ */
+ if (folio_test_ksm(folio)) {
+ folio_unlock(folio);
+ continue;
+ }
+
/*
* If we have a large folio at this point, we know it is
* fully mapped so if its mapcount is the same as its
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index b5137e86bed7..b303bfc6b46e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -2419,7 +2419,10 @@ static void high_work_func(struct work_struct *work)
struct mem_cgroup *memcg;
memcg = container_of(work, struct mem_cgroup, high_work);
+ if (!css_tryget_online(&memcg->css))
+ return;
reclaim_high(memcg, MEMCG_CHARGE_BATCH, GFP_KERNEL);
+ css_put(&memcg->css);
}
/*
@@ -4369,6 +4372,8 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css)
page_counter_set_min(&memcg->memory, 0);
page_counter_set_low(&memcg->memory, 0);
+ page_counter_set_high(&memcg->memory, PAGE_COUNTER_MAX);
+ page_counter_set_high(&memcg->swap, PAGE_COUNTER_MAX);
zswap_memcg_offline_cleanup(memcg);
diff --git a/mm/memfd.c b/mm/memfd.c
index 0f6fff004f5e..0e3ff3eebfa8 100644
--- a/mm/memfd.c
+++ b/mm/memfd.c
@@ -482,6 +482,8 @@ struct file *memfd_alloc_file(const char *name, unsigned int flags)
}
file->f_mode |= FMODE_LSEEK | FMODE_PREAD | FMODE_PWRITE;
+ /* a regular file like any other: POSIX.1-2008/SUSv4 Section XSI 2.9.7 */
+ file->f_mode |= FMODE_ATOMIC_POS;
file->f_flags |= O_LARGEFILE;
if (flags & MFD_NOEXEC_SEAL) {
diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c
index 54851d8a195b..ddc135f55eb8 100644
--- a/mm/memory-tiers.c
+++ b/mm/memory-tiers.c
@@ -571,21 +571,14 @@ static struct memory_tier *set_node_memory_tier(int node)
return memtier;
}
-static void destroy_memory_tier(struct memory_tier *memtier)
+static struct memory_tier *clear_node_memory_tier(int node)
{
- list_del(&memtier->list);
- device_unregister(&memtier->dev);
-}
-
-static bool clear_node_memory_tier(int node)
-{
- bool cleared = false;
pg_data_t *pgdat;
struct memory_tier *memtier;
pgdat = NODE_DATA(node);
if (!pgdat)
- return false;
+ return NULL;
/*
* Make sure that anybody looking at NODE_DATA who finds
@@ -603,14 +596,10 @@ static bool clear_node_memory_tier(int node)
synchronize_rcu();
memtype = node_memory_types[node].memtype;
node_clear(node, memtype->nodes);
- if (nodes_empty(memtype->nodes)) {
+ if (nodes_empty(memtype->nodes))
list_del_init(&memtype->tier_sibling);
- if (list_empty(&memtier->memory_types))
- destroy_memory_tier(memtier);
- }
- cleared = true;
}
- return cleared;
+ return memtier;
}
static void release_memtype(struct kref *kref)
@@ -883,13 +872,21 @@ static int __meminit memtier_hotplug_callback(struct notifier_block *self,
{
struct memory_tier *memtier;
struct node_notify *nn = _arg;
+ bool empty = false;
switch (action) {
case NODE_REMOVED_LAST_MEMORY:
mutex_lock(&memory_tier_lock);
- if (clear_node_memory_tier(nn->nid))
+ memtier = clear_node_memory_tier(nn->nid);
+ if (memtier) {
+ empty = list_empty(&memtier->memory_types);
+ if (empty)
+ list_del(&memtier->list);
establish_demotion_targets();
+ }
mutex_unlock(&memory_tier_lock);
+ if (empty)
+ device_unregister(&memtier->dev);
break;
case NODE_ADDED_FIRST_MEMORY:
mutex_lock(&memory_tier_lock);
diff --git a/mm/memory.c b/mm/memory.c
index 9e395ad33ffd..77e0a9bf6321 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -1500,7 +1500,6 @@ copy_pud_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
unsigned long end)
{
struct mm_struct *dst_mm = dst_vma->vm_mm;
- struct mm_struct *src_mm = src_vma->vm_mm;
pud_t *src_pud, *dst_pud;
unsigned long next;
@@ -1514,8 +1513,7 @@ copy_pud_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
int err;
VM_BUG_ON_VMA(next-addr != HPAGE_PUD_SIZE, src_vma);
- err = copy_huge_pud(dst_mm, src_mm,
- dst_pud, src_pud, addr, src_vma);
+ err = copy_huge_pud(dst_vma, src_vma, dst_pud, src_pud, addr);
if (err == -ENOMEM)
return -ENOMEM;
if (!err)
@@ -2073,6 +2071,15 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb,
}
} while (pte += nr, addr += PAGE_SIZE * nr, addr != end);
+ add_mm_rss_vec(mm, rss);
+ lazy_mmu_mode_disable();
+
+ /* Do the actual TLB flush before dropping ptl */
+ if (force_flush) {
+ tlb_flush_mmu_tlbonly(tlb);
+ tlb_flush_rmaps(tlb, vma);
+ }
+
/*
* Fast path: try to hold the pmd lock and unmap the PTE page.
*
@@ -2080,18 +2087,12 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb,
* to hold the pmd lock failed, then we need to recheck all pte entries
* to ensure they are still none, thereby preventing the pte entries
* from being repopulated by another thread.
+ *
+ * Clear the PMD after the rmaps for the table have been removed, in case
+ * an incorrect mapcount is observed after unmapping the folio.
*/
if (can_reclaim_pt && direct_reclaim && addr == end)
direct_reclaim = zap_empty_pte_table(mm, pmd, ptl, &pmdval);
-
- add_mm_rss_vec(mm, rss);
- lazy_mmu_mode_disable();
-
- /* Do the actual TLB flush before dropping ptl */
- if (force_flush) {
- tlb_flush_mmu_tlbonly(tlb);
- tlb_flush_rmaps(tlb, vma);
- }
pte_unmap_unlock(start_pte, ptl);
/*
@@ -2138,13 +2139,15 @@ static inline unsigned long zap_pmd_range(struct mmu_gather *tlb,
addr = next;
continue;
}
+ /* We already synchronized with concurrent zapping. */
+ if (pmd_none(*pmd)) {
+ addr = next;
+ continue;
+ }
/* fall through */
- } else if (details && details->single_folio &&
- folio_test_pmd_mappable(details->single_folio) &&
- next - addr == HPAGE_PMD_SIZE && pmd_none(*pmd)) {
- sync_with_folio_pmd_zap(tlb->mm, pmd);
- }
- if (pmd_none(*pmd)) {
+ } else if (pmd_none(*pmd)) {
+ if (details && details->single_folio)
+ sync_with_folio_pmd_zap(tlb->mm, pmd);
addr = next;
continue;
}
@@ -3700,6 +3703,13 @@ static inline int __wp_page_copy_user(struct page *dst, struct page *src,
return 0;
}
+ /*
+ * __copy_from_user_inatomic() only works as expected for local faults.
+ * Reject remote faults.
+ */
+ if (unlikely(mm != current->mm))
+ return -EFAULT;
+
/*
* If the source page was a PFN mapping, we don't have
* a "struct page" for it. We do a best-effort copy by
@@ -4025,6 +4035,8 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
folio_put(old_folio);
delayacct_wpcopy_end();
+ if (err == -EFAULT)
+ return VM_FAULT_SIGBUS;
return err == -EHWPOISON ? VM_FAULT_HWPOISON : 0;
}
kmsan_copy_page_meta(&new_folio->page, vmf->page);
@@ -4057,8 +4069,11 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
if (unlikely(unshare)) {
if (pte_soft_dirty(vmf->orig_pte))
entry = pte_mksoft_dirty(entry);
- if (pte_uffd(vmf->orig_pte))
+ if (pte_uffd(vmf->orig_pte)) {
entry = pte_mkuffd(entry);
+ if (userfaultfd_rwp(vma))
+ entry = pte_modify(entry, PAGE_NONE);
+ }
} else {
entry = maybe_mkwrite(pte_mkdirty(entry), vma);
}
@@ -4579,8 +4594,8 @@ EXPORT_SYMBOL_GPL(unmap_mapping_pages);
void unmap_mapping_range(struct address_space *mapping,
loff_t const holebegin, loff_t const holelen, int even_cows)
{
- pgoff_t hba = (pgoff_t)(holebegin) >> PAGE_SHIFT;
- pgoff_t hlen = ((pgoff_t)(holelen) + PAGE_SIZE - 1) >> PAGE_SHIFT;
+ pgoff_t hba = (u64)holebegin >> PAGE_SHIFT;
+ pgoff_t hlen = ((u64)holelen + PAGE_SIZE - 1) >> PAGE_SHIFT;
/* Check for overflow. */
if (sizeof(holelen) > sizeof(hlen)) {
@@ -4870,6 +4885,34 @@ static void check_swap_exclusive(struct folio *folio, swp_entry_t entry,
} while (--nr_pages);
}
+/*
+ * Get the address range in which a folio might be mapped, assuming its pages
+ * are mapped consecutively and the folio page at the given idx is mapped at
+ * the given address.
+ */
+static void folio_pte_range(const struct folio *folio,
+ const struct vm_area_struct *vma, unsigned long addr,
+ unsigned long idx, unsigned long *start, unsigned long *end)
+{
+ /* Stay within the page table and VMA boundaries. */
+ const unsigned long pmd_start = addr & PMD_MASK;
+ const unsigned long pmd_end = pmd_start + PMD_SIZE;
+ const unsigned long lower = max(pmd_start, vma->vm_start);
+ /*
+ * We cannot use pmd_addr_end(), as it does not clamp reliably. To
+ * avoid wraparounds, compute using the inclusive end.
+ */
+ const unsigned long upper = min(pmd_end - 1, vma->vm_end - 1) + 1;
+ const unsigned long offset = idx << PAGE_SHIFT;
+
+ /*
+ * Avoid start/end addresses wrapping by not adding/subtracting more
+ * than the lower/upper limits would allow.
+ */
+ *start = addr - min(addr - lower, offset);
+ *end = addr + min(upper - addr, folio_size(folio) - offset);
+}
+
/*
* We enter with either the VMA lock or the mmap_lock held (see
* FAULT_FLAG_VMA_LOCK), and pte mapped but not yet locked.
@@ -5020,7 +5063,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
* folio->index of non-ksm folios would be nonlinear inside the
* anon VMA -- the ksm flag is lost on actual swapout.
*/
- folio = ksm_might_need_to_copy(folio, vma, vmf->address);
+ folio = ksm_might_need_to_copy(folio, page, vma, vmf->address);
if (unlikely(!folio)) {
ret = VM_FAULT_OOM;
folio = swapcache;
@@ -5052,16 +5095,15 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
address = vmf->address;
ptep = vmf->pte;
if (folio_test_large(folio) && folio_test_swapcache(folio)) {
- int nr = folio_nr_pages(folio);
- unsigned long idx = folio_page_idx(folio, page);
- unsigned long folio_start = address - idx * PAGE_SIZE;
- unsigned long folio_end = folio_start + nr * PAGE_SIZE;
+ const int nr = folio_nr_pages(folio);
+ const unsigned long idx = folio_page_idx(folio, page);
+ unsigned long folio_start, folio_end;
pte_t *folio_ptep;
pte_t folio_pte;
- if (unlikely(folio_start < max(address & PMD_MASK, vma->vm_start)))
- goto check_folio;
- if (unlikely(folio_end > pmd_addr_end(address, vma->vm_end)))
+ folio_pte_range(folio, vma, address, idx, &folio_start,
+ &folio_end);
+ if (unlikely((folio_end - folio_start) >> PAGE_SHIFT != nr))
goto check_folio;
folio_ptep = vmf->pte - idx;
@@ -5202,7 +5244,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
if (unlikely(folio != swapcache)) {
folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSIVE);
folio_add_lru_vma(folio, vma);
- folio_put_swap(swapcache, NULL);
+ folio_put_swap(swapcache,
+ folio_file_page(swapcache, swp_offset(entry)));
} else if (!folio_test_anon(folio)) {
/*
* We currently only expect !anon folios that are fully
@@ -6211,16 +6254,11 @@ static void numa_rebuild_large_mapping(struct vm_fault *vmf, struct vm_area_stru
struct folio *folio, pte_t fault_pte,
bool ignore_writable, bool pte_write_upgrade)
{
- int nr = pte_pfn(fault_pte) - folio_pfn(folio);
+ const unsigned long idx = pte_pfn(fault_pte) - folio_pfn(folio);
unsigned long start, end, addr = vmf->address;
- unsigned long addr_start = addr - (nr << PAGE_SHIFT);
- unsigned long pt_start = ALIGN_DOWN(addr, PMD_SIZE);
pte_t *start_ptep;
- /* Stay within the VMA and within the page table. */
- start = max3(addr_start, pt_start, vma->vm_start);
- end = min3(addr_start + folio_size(folio), pt_start + PMD_SIZE,
- vma->vm_end);
+ folio_pte_range(folio, vma, addr, idx, &start, &end);
start_ptep = vmf->pte - ((addr - start) >> PAGE_SHIFT);
/* Restore all PTEs' mapping of the large folio */
@@ -6566,9 +6604,12 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf)
* VM_UFFD_RWP VMA, a protnone PTE without the uffd bit is
* NUMA hinting and must still fall through to do_numa_page().
*/
- if (userfaultfd_pte_rwp(vmf->vma, vmf->orig_pte))
- return do_uffd_rwp(vmf);
- return do_numa_page(vmf);
+ if (userfaultfd_pte_rwp(vmf->vma, vmf->orig_pte)) {
+ if (!(vmf->flags & FAULT_FLAG_UNSHARE))
+ return do_uffd_rwp(vmf);
+ } else {
+ return do_numa_page(vmf);
+ }
}
spin_lock(vmf->ptl);
@@ -6883,8 +6924,12 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
lru_gen_exit_fault();
- /* If the mapping is droppable, then errors due to OOM aren't fatal. */
- if (is_droppable)
+ /*
+ * If the mapping is droppable, then errors due to OOM aren't fatal: the
+ * task returns to user mode, handles its signals and tries again. A
+ * fault from kernel mode has no such return, so it must see the error.
+ */
+ if (is_droppable && (flags & FAULT_FLAG_USER))
ret &= ~VM_FAULT_OOM;
if (flags & FAULT_FLAG_USER) {
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 839c3fe985a0..d9de3b5d5ebe 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -3346,11 +3346,11 @@ void __init numa_policy_init(void)
policy_cache = kmem_cache_create("numa_policy",
sizeof(struct mempolicy),
- 0, SLAB_PANIC, NULL);
+ 0, SLAB_PANIC | SLAB_ACCOUNT, NULL);
sn_cache = kmem_cache_create("shared_policy_node",
sizeof(struct sp_node),
- 0, SLAB_PANIC, NULL);
+ 0, SLAB_PANIC | SLAB_ACCOUNT, NULL);
for_each_node(nid) {
preferred_node_policy[nid] = (struct mempolicy) {
diff --git a/mm/migrate.c b/mm/migrate.c
index 15b45832bcfa..ee78577177d6 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2462,7 +2462,7 @@ static void do_pages_stat_array(struct mm_struct *mm, unsigned long nr_pages,
mmap_read_lock(mm);
for (i = 0; i < nr_pages; i++) {
- unsigned long addr = (unsigned long)(*pages);
+ unsigned long addr = (unsigned long)untagged_addr_remote(mm, *pages);
struct vm_area_struct *vma;
struct folio_walk fw;
struct folio *folio;
@@ -2757,6 +2757,7 @@ int migrate_misplaced_folio(struct folio *folio, int node)
pg_data_t *pgdat = NODE_DATA(node);
int nr_remaining;
unsigned int nr_succeeded;
+ int src_node = folio_nid(folio);
LIST_HEAD(migratepages);
struct mem_cgroup *memcg = get_mem_cgroup_from_folio(folio);
struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat);
@@ -2771,7 +2772,7 @@ int migrate_misplaced_folio(struct folio *folio, int node)
count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded);
count_memcg_events(memcg, NUMA_PAGE_MIGRATE, nr_succeeded);
if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING)
- && !node_is_toptier(folio_nid(folio))
+ && !node_is_toptier(src_node)
&& node_is_toptier(node))
mod_lruvec_state(lruvec, PGPROMOTE_SUCCESS, nr_succeeded);
}
diff --git a/mm/mm_init.c b/mm/mm_init.c
index 1533aebafb68..36ec1ef0712c 100644
--- a/mm/mm_init.c
+++ b/mm/mm_init.c
@@ -2422,8 +2422,6 @@ void __init memblock_free_pages(unsigned long pfn, unsigned int order)
return;
}
- /* pages were reserved and not allocated */
- clear_page_tag_ref(page);
__free_pages_core(page, order, MEMINIT_EARLY);
}
diff --git a/mm/mmap.c b/mm/mmap.c
index 4bf26b0f1e6e..6f22cd76be4f 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1713,8 +1713,7 @@ __latent_entropy int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)
LIST_HEAD(uf);
VMA_ITERATOR(vmi, mm, 0);
- if (mmap_write_lock_killable(oldmm))
- return -EINTR;
+ mmap_assert_write_locked(oldmm);
flush_cache_dup_mm(oldmm);
uprobe_dup_mmap(oldmm, mm);
/*
@@ -1725,11 +1724,6 @@ __latent_entropy int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)
/* No ordering required: file already has been exposed. */
dup_mm_exe_file(mm, oldmm);
- mm->total_vm = oldmm->total_vm;
- mm->data_vm = oldmm->data_vm;
- mm->exec_vm = oldmm->exec_vm;
- mm->stack_vm = oldmm->stack_vm;
-
/* Use __mt_dup() to efficiently build an identical maple tree. */
retval = __mt_dup(&oldmm->mm_mt, &mm->mm_mt, GFP_KERNEL);
if (unlikely(retval))
diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c
index df69ba6e797f..a7bae5de5fed 100644
--- a/mm/mmu_notifier.c
+++ b/mm/mmu_notifier.c
@@ -569,16 +569,25 @@ int __mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range)
{
struct mmu_notifier_subscriptions *subscriptions =
range->mm->notifier_subscriptions;
- int ret;
+ const bool has_itree = subscriptions->has_itree;
+ int ret = 0;
- if (subscriptions->has_itree) {
+ if (has_itree) {
ret = mn_itree_invalidate(subscriptions, range);
if (ret)
return ret;
}
if (!hlist_empty(&subscriptions->list))
- return mn_hlist_invalidate_range_start(subscriptions, range);
- return 0;
+ ret = mn_hlist_invalidate_range_start(subscriptions, range);
+
+ /*
+ * On error, the non-blocking caller is not allowed to call
+ * invalidate_range_end(), as in mn_itree_invalidate().
+ */
+ if (ret && has_itree && !mmu_notifier_range_blockable(range))
+ mn_itree_inv_end(subscriptions);
+
+ return ret;
}
static void
@@ -938,7 +947,6 @@ static int __mmu_interval_notifier_insert(
struct mmu_notifier_subscriptions *subscriptions, unsigned long start,
unsigned long length, const struct mmu_interval_notifier_ops *ops)
{
- interval_sub->mm = mm;
interval_sub->ops = ops;
RB_CLEAR_NODE(&interval_sub->interval_tree.rb);
interval_sub->interval_tree.start = start;
@@ -955,8 +963,12 @@ static int __mmu_interval_notifier_insert(
if (WARN_ON(atomic_read(&mm->mm_users) <= 0))
return -EINVAL;
- /* pairs with mmdrop in mmu_interval_notifier_remove() */
+ /*
+ * Pairs with mmdrop in mmu_interval_notifier_remove(). Drivers take a
+ * non-NULL ->mm for "registered", so set it only with the reference.
+ */
mmgrab(mm);
+ interval_sub->mm = mm;
/*
* If some invalidate_range_start/end region is going on in parallel
diff --git a/mm/mremap.c b/mm/mremap.c
index d404e4b1b6cb..23498d0783c4 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -87,7 +87,7 @@ static pud_t *get_old_pud(struct mm_struct *mm, unsigned long addr)
return NULL;
pud = pud_offset(p4d, addr);
- if (pud_none_or_clear_bad(pud))
+ if (!pud_trans_huge(*pud) && pud_none_or_clear_bad(pud))
return NULL;
return pud;
@@ -202,7 +202,7 @@ static int move_ptes(struct pagetable_move_control *pmc,
struct mm_struct *mm = vma->vm_mm;
pte_t *old_ptep, *new_ptep;
pte_t old_pte, pte;
- pmd_t dummy_pmdval;
+ pmd_t old_pmdval, new_pmdval;
spinlock_t *old_ptl, *new_ptl;
bool force_flush = false;
unsigned long old_addr = pmc->old_addr;
@@ -234,31 +234,30 @@ static int move_ptes(struct pagetable_move_control *pmc,
if (pmc->need_rmap_locks)
take_rmap_locks(vma);
- /*
- * We don't have to worry about the ordering of src and dst
- * pte locks because exclusive mmap_lock prevents deadlock.
- */
- old_ptep = pte_offset_map_lock(mm, old_pmd, old_addr, &old_ptl);
+ old_ptep = pte_offset_map_rw_nolock(mm, old_pmd, old_addr, &old_pmdval,
+ &old_ptl);
if (!old_ptep) {
err = -EAGAIN;
goto out;
}
- /*
- * Now new_pte is none, so collapse_scan_file() path can not find
- * this by traversing file->f_mapping, so there is no concurrency with
- * retract_page_tables(). In addition, we already hold the exclusive
- * mmap_lock, so this new_pte page is stable, so there is no need to get
- * pmdval and do pmd_same() check.
- */
- new_ptep = pte_offset_map_rw_nolock(mm, new_pmd, new_addr, &dummy_pmdval,
- &new_ptl);
+ new_ptep = pte_offset_map_rw_nolock(mm, new_pmd, new_addr, &new_pmdval,
+ &new_ptl);
if (!new_ptep) {
- pte_unmap_unlock(old_ptep, old_ptl);
+ pte_unmap(old_ptep);
+ err = -EAGAIN;
+ goto out;
+ }
+
+ double_pt_lock(old_ptl, new_ptl);
+ if (unlikely(!pmd_same(old_pmdval, pmdp_get_lockless(old_pmd)) ||
+ !pmd_same(new_pmdval, pmdp_get_lockless(new_pmd)))) {
+ double_pt_unlock(old_ptl, new_ptl);
+ pte_unmap(new_ptep);
+ pte_unmap(old_ptep);
err = -EAGAIN;
goto out;
}
- if (new_ptl != old_ptl)
- spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
+
flush_tlb_batched_pending(vma->vm_mm);
lazy_mmu_mode_enable();
@@ -317,10 +316,9 @@ static int move_ptes(struct pagetable_move_control *pmc,
lazy_mmu_mode_disable();
if (force_flush)
flush_tlb_range(vma, old_end - len, old_end);
- if (new_ptl != old_ptl)
- spin_unlock(new_ptl);
+ double_pt_unlock(old_ptl, new_ptl);
pte_unmap(new_ptep - 1);
- pte_unmap_unlock(old_ptep - 1, old_ptl);
+ pte_unmap(old_ptep - 1);
out:
if (pmc->need_rmap_locks)
drop_rmap_locks(vma);
@@ -395,14 +393,9 @@ static bool move_normal_pmd(struct pagetable_move_control *pmc,
if (WARN_ON_ONCE(!pmd_none(*new_pmd)))
return false;
- /*
- * We don't have to worry about the ordering of src and dst
- * ptlocks because exclusive mmap_lock prevents deadlock.
- */
- old_ptl = pmd_lock(mm, old_pmd);
+ old_ptl = pmd_lockptr(mm, old_pmd);
new_ptl = pmd_lockptr(mm, new_pmd);
- if (new_ptl != old_ptl)
- spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
+ double_pt_lock(old_ptl, new_ptl);
pmd = *old_pmd;
@@ -418,9 +411,7 @@ static bool move_normal_pmd(struct pagetable_move_control *pmc,
pmd_populate(mm, new_pmd, pmd_pgtable(pmd));
flush_tlb_range(vma, pmc->old_addr, pmc->old_addr + PMD_SIZE);
out_unlock:
- if (new_ptl != old_ptl)
- spin_unlock(new_ptl);
- spin_unlock(old_ptl);
+ double_pt_unlock(old_ptl, new_ptl);
return res;
}
@@ -452,26 +443,24 @@ static bool move_normal_pud(struct pagetable_move_control *pmc,
if (WARN_ON_ONCE(!pud_none(*new_pud)))
return false;
- /*
- * We don't have to worry about the ordering of src and dst
- * ptlocks because exclusive mmap_lock prevents deadlock.
- */
- old_ptl = pud_lock(mm, old_pud);
+ old_ptl = pud_lockptr(mm, old_pud);
new_ptl = pud_lockptr(mm, new_pud);
- if (new_ptl != old_ptl)
- spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
- /* Clear the pud */
+ double_pt_lock(old_ptl, new_ptl);
+
pud = *old_pud;
+ if (unlikely(pud_none(pud)))
+ goto out_unlock;
+
+ /* Clear the pud */
pud_clear(old_pud);
VM_BUG_ON(!pud_none(*new_pud));
pud_populate(mm, new_pud, pud_pgtable(pud));
flush_tlb_range(vma, pmc->old_addr, pmc->old_addr + PUD_SIZE);
- if (new_ptl != old_ptl)
- spin_unlock(new_ptl);
- spin_unlock(old_ptl);
+out_unlock:
+ double_pt_unlock(old_ptl, new_ptl);
return true;
}
@@ -499,18 +488,17 @@ static bool move_huge_pud(struct pagetable_move_control *pmc,
if (WARN_ON_ONCE(!pud_none(*new_pud)))
return false;
- /*
- * We don't have to worry about the ordering of src and dst
- * ptlocks because exclusive mmap_lock prevents deadlock.
- */
- old_ptl = pud_lock(mm, old_pud);
+ old_ptl = pud_lockptr(mm, old_pud);
new_ptl = pud_lockptr(mm, new_pud);
- if (new_ptl != old_ptl)
- spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
+
+ double_pt_lock(old_ptl, new_ptl);
+ if (unlikely(!pud_trans_huge(*old_pud))) {
+ double_pt_unlock(old_ptl, new_ptl);
+ return false;
+ }
/* Clear the pud */
- pud = *old_pud;
- pud_clear(old_pud);
+ pud = pudp_huge_get_and_clear(mm, pmc->old_addr, old_pud);
VM_BUG_ON(!pud_none(*new_pud));
@@ -518,9 +506,7 @@ static bool move_huge_pud(struct pagetable_move_control *pmc,
/* mark soft_ditry when we add pud level soft dirty support */
set_pud_at(mm, pmc->new_addr, new_pud, pud);
flush_pud_tlb_range(vma, pmc->old_addr, pmc->old_addr + HPAGE_PUD_SIZE);
- if (new_ptl != old_ptl)
- spin_unlock(new_ptl);
- spin_unlock(old_ptl);
+ double_pt_unlock(old_ptl, new_ptl);
return true;
}
@@ -842,11 +828,10 @@ unsigned long move_page_tables(struct pagetable_move_control *pmc)
if (!new_pud)
break;
if (pud_trans_huge(*old_pud)) {
- if (extent == HPAGE_PUD_SIZE) {
- move_pgt_entry(pmc, HPAGE_PUD, old_pud, new_pud);
- /* We ignore and continue on error? */
+ if (extent == HPAGE_PUD_SIZE &&
+ move_pgt_entry(pmc, HPAGE_PUD, old_pud, new_pud))
continue;
- }
+ split_huge_pud(pmc->old, old_pud, pmc->old_addr);
} else if (IS_ENABLED(CONFIG_HAVE_MOVE_PUD) && extent == PUD_SIZE) {
if (move_pgt_entry(pmc, NORMAL_PUD, old_pud, new_pud))
continue;
diff --git a/mm/nommu.c b/mm/nommu.c
index ed44510e3770..6e0f9c7da236 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -1926,7 +1926,7 @@ subsys_initcall(init_admin_reserve);
int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)
{
- mmap_write_lock(oldmm);
+ mmap_assert_write_locked(oldmm);
dup_mm_exe_file(mm, oldmm);
mmap_write_unlock(oldmm);
return 0;
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index eeab25d6ce36..1d6aab8781a5 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -883,6 +883,7 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc,
{
struct wb_domain *dom = dtc_dom(dtc);
struct bdi_writeback *wb = dtc->wb;
+ u64 threshold = thresh;
u64 wb_thresh;
u64 wb_max_thresh;
unsigned long numerator, denominator;
@@ -894,13 +895,14 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc,
fprop_fraction_percpu(&dom->completions, dtc->wb_completions,
&numerator, &denominator);
- wb_thresh = (thresh * (100 * BDI_RATIO_SCALE - bdi_min_ratio)) / (100 * BDI_RATIO_SCALE);
+ wb_thresh = div_u64(threshold * (100 * BDI_RATIO_SCALE - bdi_min_ratio),
+ 100 * BDI_RATIO_SCALE);
wb_thresh *= numerator;
wb_thresh = div64_ul(wb_thresh, denominator);
wb_min_max_ratio(wb, &wb_min_ratio, &wb_max_ratio);
- wb_thresh += (thresh * wb_min_ratio) / (100 * BDI_RATIO_SCALE);
+ wb_thresh += div_u64(threshold * wb_min_ratio, 100 * BDI_RATIO_SCALE);
/*
* It's very possible that wb_thresh is close to 0 not because the
@@ -916,7 +918,7 @@ static unsigned long __wb_calc_thresh(struct dirty_throttle_control *dtc,
wb_thresh = max(wb_thresh, (thresh - dtc->dirty) / 8);
}
- wb_max_thresh = thresh * wb_max_ratio / (100 * BDI_RATIO_SCALE);
+ wb_max_thresh = div_u64(threshold * wb_max_ratio, 100 * BDI_RATIO_SCALE);
if (wb_thresh > wb_max_thresh)
wb_thresh = wb_max_thresh;
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 8ca2289d0491..8714425ffe84 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1634,6 +1634,9 @@ void __meminit __free_pages_core(struct page *page, unsigned int order,
atomic_long_add(nr_pages, &page_zone(page)->managed_pages);
}
+ /* pages were never allocated */
+ clear_page_tag_ref(page);
+
if (page_contains_unaccepted(page, order)) {
if (order == MAX_PAGE_ORDER && __free_unaccepted(page))
return;
@@ -2947,7 +2950,7 @@ static bool free_frozen_page_commit(struct zone *zone,
* kswapd work again by resetting kswapd_failures.
*/
if (kswapd_test_hopeless(pgdat) &&
- next_memory_node(pgdat->node_id) < MAX_NUMNODES)
+ num_node_state(N_MEMORY) > 1)
kswapd_clear_hopeless(pgdat, KSWAPD_CLEAR_HOPELESS_PCP);
}
return ret;
@@ -3256,7 +3259,9 @@ struct page *rmqueue_buddy(struct zone *preferred_zone, struct zone *zone,
* reserves as failing now is worse than failing a
* high-order atomic allocation in the future.
*/
- if (!page && (alloc_flags & (ALLOC_OOM|ALLOC_NON_BLOCK)))
+ if (!page &&
+ ((alloc_flags & ALLOC_OOM) ||
+ (alloc_flags & ALLOC_MASK_ATOMIC) == ALLOC_MASK_ATOMIC))
page = __rmqueue_smallest(zone, order, MIGRATE_HIGHATOMIC);
if (!page) {
@@ -4786,10 +4791,10 @@ static inline struct page *
__alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
struct alloc_context *ac)
{
- bool can_direct_reclaim = gfp_mask & __GFP_DIRECT_RECLAIM;
- bool can_compact = can_direct_reclaim && gfp_compaction_allowed(gfp_mask);
- bool nofail = gfp_mask & __GFP_NOFAIL;
const bool costly_order = order > PAGE_ALLOC_COSTLY_ORDER;
+ bool can_direct_reclaim;
+ bool can_compact;
+ bool nofail;
struct page *page = NULL;
unsigned int alloc_flags;
unsigned long did_some_progress;
@@ -4804,6 +4809,18 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
bool can_retry_reserves = true;
unsigned long alloc_start_time = jiffies;
+ /*
+ * Costly __GFP_NORETRY callers have a cheap fallback, so don't stall
+ * them in reclaim or compaction. __GFP_THISNODE callers are exempt.
+ */
+ if (costly_order && (gfp_mask & __GFP_NORETRY) &&
+ !(gfp_mask & __GFP_THISNODE))
+ gfp_mask &= ~__GFP_DIRECT_RECLAIM;
+
+ can_direct_reclaim = gfp_mask & __GFP_DIRECT_RECLAIM;
+ can_compact = can_direct_reclaim && gfp_compaction_allowed(gfp_mask);
+ nofail = gfp_mask & __GFP_NOFAIL;
+
if (unlikely(nofail)) {
/*
* Also we don't support __GFP_NOFAIL without __GFP_DIRECT_RECLAIM,
@@ -4920,8 +4937,11 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
* Reclaim/compaction cannot run, so defrag_mode's strategy
* of enforcing ALLOC_NOFRAGMENT cannot be fulfilled. Allow
* fallbacks rather than failing the allocation outright.
+ * Not for __GFP_NORETRY: those have a cheap lower order
+ * fallback, so failing beats fragmenting.
*/
if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT) &&
+ !(gfp_mask & __GFP_NORETRY) &&
(gfp_mask & __GFP_KSWAPD_RECLAIM)) {
alloc_flags &= ~ALLOC_NOFRAGMENT;
goto retry;
diff --git a/mm/page_alloc.h b/mm/page_alloc.h
index 2b1017a91928..a9b72cb9520e 100644
--- a/mm/page_alloc.h
+++ b/mm/page_alloc.h
@@ -32,12 +32,11 @@
#define ALLOC_OOM ALLOC_NO_WATERMARKS
#endif
-#define ALLOC_NON_BLOCK 0x10 /* Caller cannot block. Allow access
- * to 25% of the min watermark or
- * 62.5% if __GFP_HIGH is set.
- */
+#define ALLOC_NON_BLOCK 0x10 /* Caller cannot block. */
#define ALLOC_MIN_RESERVE 0x20 /* __GFP_HIGH set. Allow access to 50%
- * of the min watermark.
+ * of the min watermark, or 62.5% if
+ * the caller cannot block either
+ * (ALLOC_NON_BLOCK).
*/
#define ALLOC_CPUSET 0x40 /* check for correct cpuset */
#define ALLOC_CMA 0x80 /* allow allocations from CMA areas */
@@ -58,7 +57,10 @@
#define ALLOC_NO_CODETAG 0x1000
/* Flags that allow allocations below the min watermark. */
-#define ALLOC_RESERVES (ALLOC_NON_BLOCK|ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM)
+#define ALLOC_RESERVES (ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM)
+
+/* Flags that mean GFP_ATOMIC */
+#define ALLOC_MASK_ATOMIC (ALLOC_NON_BLOCK|ALLOC_MIN_RESERVE)
/*
* Structure for holding the mostly immutable allocation parameters passed
diff --git a/mm/rmap.c b/mm/rmap.c
index 6b3480330b92..9ffd3e833e08 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -926,6 +926,7 @@ static bool folio_referenced_one(struct folio *folio,
DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, address, 0);
int ptes = 0, referenced = 0;
unsigned int nr;
+ pte_t pteval;
while (page_vma_mapped_walk(&pvmw)) {
address = pvmw.address;
@@ -970,10 +971,12 @@ static bool folio_referenced_one(struct folio *folio,
return false;
}
- if (pvmw.pte && folio_test_large(folio)) {
+ if (pvmw.pte)
+ pteval = ptep_get(pvmw.pte);
+
+ if (pvmw.pte && pte_present(pteval) && folio_test_large(folio)) {
const unsigned long end_addr = pmd_addr_end(address, vma->vm_end);
const unsigned int max_nr = (end_addr - address) >> PAGE_SHIFT;
- pte_t pteval = ptep_get(pvmw.pte);
nr = folio_pte_batch(folio, pvmw.pte, pteval, max_nr);
}
@@ -983,11 +986,23 @@ static bool folio_referenced_one(struct folio *folio,
* are. —they could be on active/inactive lists or on MGLRU. So the
* simplest approach is to disable this look-around optimization.
*/
- if (lru_gen_enabled() && !lru_gen_switching() && pvmw.pte) {
+ if (lru_gen_enabled() && !lru_gen_switching() && pvmw.pte &&
+ pte_present(pteval)) {
if (lru_gen_look_around(&pvmw, nr))
referenced++;
} else if (pvmw.pte) {
- if (clear_flush_young_ptes_notify(vma, address, pvmw.pte, nr))
+ bool young = false;
+
+ /*
+ * PFN swap PTEs, such as device-exclusive ones, that
+ * actually map pages are "old" from a CPU perspective.
+ * The MMU notifier takes care of any device aspects.
+ */
+ if (pte_present(pteval))
+ young = clear_flush_young_ptes(vma, address, pvmw.pte, nr);
+ young |= mmu_notifier_clear_flush_young(vma->vm_mm, address,
+ address + nr * PAGE_SIZE);
+ if (young)
referenced++;
} else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) {
if (pmdp_clear_flush_young_notify(vma, address,
diff --git a/mm/shmem.c b/mm/shmem.c
index de2362209cc3..597bbc2e7321 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -77,6 +77,7 @@ static struct vfsmount *shm_mnt __ro_after_init;
#include <linux/magic.h>
#include <linux/syscalls.h>
#include <linux/fcntl.h>
+#include <linux/memfd.h>
#include <uapi/linux/memfd.h>
#include <linux/rmap.h>
#include <linux/uuid.h>
@@ -275,6 +276,11 @@ static const struct vm_operations_struct shmem_vm_ops;
static const struct vm_operations_struct shmem_anon_vm_ops;
static struct file_system_type shmem_fs_type;
+bool shmem_file(struct file *file)
+{
+ return file && file_inode(file)->i_fop == &shmem_file_operations;
+}
+
bool shmem_mapping(const struct address_space *mapping)
{
return mapping->a_ops == &shmem_aops;
@@ -968,12 +974,11 @@ static void shmem_delete_from_page_cache(struct folio *folio, void *radswap)
}
/*
- * Remove swap entry from page cache, free the swap and its page cache. Returns
- * the number of pages being freed. 0 means entry not found in XArray (0 pages
- * being freed).
+ * Remove a swap entry or swapin-error marker wholly within [index, end].
+ * Return whether it was removed, counting freed swap pages in nr_swaps_freed.
*/
-static long shmem_free_swap(struct address_space *mapping,
- pgoff_t index, pgoff_t end, void *radswap)
+static bool shmem_free_swap(struct address_space *mapping,
+ pgoff_t index, pgoff_t end, void *radswap, long *nr_swaps_freed)
{
XA_STATE(xas, &mapping->i_pages, index);
const softleaf_t swp = radix_to_swp_entry(radswap);
@@ -994,8 +999,10 @@ static long shmem_free_swap(struct address_space *mapping,
xas_unlock_irq(&xas);
/* A swapin-error marker holds no swap slot, so just drop it. */
- if (nr_pages && softleaf_is_swap(swp))
+ if (nr_pages && softleaf_is_swap(swp)) {
swap_put_entries_direct(swp, nr_pages);
+ *nr_swaps_freed += nr_pages;
+ }
return nr_pages;
}
@@ -1089,12 +1096,10 @@ void shmem_unlock_mapping(struct address_space *mapping)
static struct folio *shmem_get_partial_folio(struct inode *inode, pgoff_t index)
{
+ const gfp_t gfp = mapping_gfp_mask(inode->i_mapping);
struct folio *folio;
- /*
- * At first avoid shmem_get_folio(,,,SGP_READ): that fails
- * beyond i_size, and reports fallocated folios as holes.
- */
+retry:
folio = filemap_get_entry(inode->i_mapping, index);
if (!folio)
return folio;
@@ -1105,13 +1110,14 @@ static struct folio *shmem_get_partial_folio(struct inode *inode, pgoff_t index)
/* The folio has been swapped out */
folio_unlock(folio);
folio_put(folio);
+ goto retry;
}
/*
- * But read a folio back from swap if any of it is within i_size
- * (although in some cases this is just a waste of time).
+ * Truncation can look up a large swap entry at or beyond i_size.
+ * Swap it in directly to bypass the size check in shmem_get_folio().
*/
- folio = NULL;
- shmem_get_folio(inode, index, 0, &folio, SGP_READ);
+ if (shmem_swapin_folio(inode, index, &folio, SGP_READ, gfp, NULL, NULL) == -EEXIST)
+ goto retry;
return folio;
}
@@ -1150,8 +1156,8 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend,
if (xa_is_value(folio)) {
if (unfalloc)
continue;
- nr_swaps_freed += shmem_free_swap(mapping, indices[i],
- end - 1, folio);
+ shmem_free_swap(mapping, indices[i], end - 1,
+ folio, &nr_swaps_freed);
continue;
}
@@ -1218,32 +1224,31 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend,
if (xa_is_value(folio)) {
int order;
- long swaps_freed;
if (unfalloc)
continue;
- swaps_freed = shmem_free_swap(mapping, indices[i],
- end - 1, folio);
- if (!swaps_freed) {
+ if (!shmem_free_swap(mapping, indices[i], end - 1,
+ folio, &nr_swaps_freed)) {
pgoff_t base = indices[i];
order = shmem_confirm_swap(mapping, indices[i],
radix_to_swp_entry(folio));
/*
- * If found a large swap entry cross the end or start
- * border, skip it as the truncate_inode_partial_folio
- * above should have at least zerod its content once.
+ * Skip boundary entries when restarting
+ * the scan to avoid endless loops after
+ * swapin failure.
*/
if (order > 0) {
base = round_down(base, 1 << order);
- if (base < start || base + (1 << order) > end)
+ if (base < start || base + (1 << order) > end) {
+ start = max(start, base + (1 << order));
continue;
+ }
}
/* Swap was replaced by page or extended, retry */
index = base;
break;
}
- nr_swaps_freed += swaps_freed;
continue;
}
@@ -1259,22 +1264,34 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend,
VM_BUG_ON_FOLIO(folio_test_writeback(folio),
folio);
- if (!folio_test_large(folio)) {
+ /*
+ * A shared mapping can have dirty PTEs while
+ * the folio is still clean. Mark the folio
+ * dirty so a failed split cannot discard data
+ * outside the range.
+ */
+ if (!unfalloc && folio_test_large(folio))
+ folio_mark_dirty(folio);
+
+ if (unfalloc || !folio_test_large(folio)) {
truncate_inode_folio(mapping, folio);
} else if (truncate_inode_partial_folio(folio, lstart, lend)) {
/*
- * If we split a page, reset the loop so
- * that we pick up the new sub pages.
- * Otherwise the THP was entirely
- * dropped or the target range was
- * zeroed, so just continue the loop as
- * is.
+ * The folio was split. Retry the scan
+ * to find the split folios.
*/
- if (!folio_test_large(folio)) {
- folio_unlock(folio);
- index = start;
- break;
- }
+ folio_unlock(folio);
+ index = start;
+ break;
+ } else if (folio->mapping) {
+ /*
+ * Splitting failed, but the part in the
+ * range is already zeroed. The folio
+ * was not dropped, so skip it when
+ * restarting the scan to avoid retrying
+ * it forever.
+ */
+ start = max(start, folio_next_index(folio));
}
}
folio_unlock(folio);
@@ -1341,10 +1358,11 @@ static int shmem_setattr(struct mnt_idmap *idmap,
if (error)
return error;
- if ((info->seals & F_SEAL_EXEC) && (attr->ia_valid & ATTR_MODE)) {
- if ((inode->i_mode ^ attr->ia_mode) & 0111) {
- return -EPERM;
- }
+ if (attr->ia_valid & ATTR_MODE) {
+ error = memfd_check_seals_mode(info->seals, inode->i_mode,
+ attr->ia_mode);
+ if (error)
+ return error;
}
if (S_ISREG(inode->i_mode) && (attr->ia_valid & ATTR_SIZE)) {
@@ -1373,7 +1391,8 @@ static int shmem_setattr(struct mnt_idmap *idmap,
if (oldsize > holebegin)
unmap_mapping_range(inode->i_mapping,
holebegin, 0, 1);
- if (info->alloced)
+ /* Swapin-error markers are not counted in alloced. */
+ if (info->alloced || !mapping_empty(inode->i_mapping))
shmem_truncate_range(inode,
newsize, (loff_t)-1);
/* unmap again to remove racily COWed private pages */
@@ -1544,6 +1563,10 @@ static int shmem_unuse_inode(struct inode *inode, unsigned int type)
if (ret < 0)
break;
+ if (signal_pending(current)) {
+ ret = -ERESTARTSYS;
+ break;
+ }
start = indices[folio_batch_count(&fbatch) - 1];
} while (true);
@@ -2171,11 +2194,10 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index,
folio_put_swap(folio, NULL);
swap_cache_del_folio(folio);
/*
- * Don't treat swapin error folio as alloced. Otherwise inode->i_blocks
- * won't be 0 when inode is released and thus trigger WARN_ON(i_blocks)
- * in shmem_evict_inode().
+ * Let shmem_recalc_inode() reduce alloced and release the block charge
+ * for the freed swap pages.
*/
- shmem_recalc_inode(inode, -nr_pages, -nr_pages);
+ shmem_recalc_inode(inode, 0, -nr_pages);
}
static int shmem_split_large_entry(struct inode *inode, pgoff_t index,
@@ -2316,6 +2338,8 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index,
else
error = -ENOMEM;
folio = NULL;
+ /* Make sure to validate the original value. */
+ swap = index_entry;
goto failed;
}
if (fault_type) {
@@ -5198,12 +5222,35 @@ static const struct file_operations shmem_file_operations = {
#endif
};
+#ifdef CONFIG_TMPFS_POSIX_ACL
+static int shmem_set_acl(struct mnt_idmap *idmap, struct dentry *dentry,
+ struct posix_acl *acl, int type)
+{
+ struct inode *inode = d_inode(dentry);
+ umode_t mode = inode->i_mode;
+ int error;
+
+ if (type == ACL_TYPE_ACCESS) {
+ error = posix_acl_equiv_mode(acl, &mode);
+ if (error < 0)
+ return error;
+ error = memfd_check_seals_mode(SHMEM_I(inode)->seals,
+ inode->i_mode, mode);
+ if (error)
+ return error;
+ }
+ return simple_set_acl(idmap, dentry, acl, type);
+}
+#else
+#define shmem_set_acl simple_set_acl
+#endif
+
static const struct inode_operations shmem_inode_operations = {
.getattr = shmem_getattr,
.setattr = shmem_setattr,
#ifdef CONFIG_TMPFS_XATTR
.listxattr = shmem_listxattr,
- .set_acl = simple_set_acl,
+ .set_acl = shmem_set_acl,
.fileattr_get = shmem_fileattr_get,
.fileattr_set = shmem_fileattr_set,
#endif
diff --git a/mm/shmem_quota.c b/mm/shmem_quota.c
index d0b92d6da50f..eeca9efd448a 100644
--- a/mm/shmem_quota.c
+++ b/mm/shmem_quota.c
@@ -234,7 +234,7 @@ static int shmem_acquire_dquot(struct dquot *dquot)
return ret;
}
-static bool shmem_is_empty_dquot(struct dquot *dquot)
+static bool shmem_is_default_dquot(struct dquot *dquot)
{
struct shmem_sb_info *sbinfo = dquot->dq_sb->s_fs_info;
qsize_t bhardlimit;
@@ -248,19 +248,21 @@ static bool shmem_is_empty_dquot(struct dquot *dquot)
ihardlimit = sbinfo->qlimits.grpquota_ihardlimit;
}
- if (test_bit(DQ_FAKE_B, &dquot->dq_flags) ||
- (dquot->dq_dqb.dqb_curspace == 0 &&
- dquot->dq_dqb.dqb_curinodes == 0 &&
- dquot->dq_dqb.dqb_bhardlimit == bhardlimit &&
- dquot->dq_dqb.dqb_ihardlimit == ihardlimit))
- return true;
-
- return false;
+ /*
+ * Defaults are zero usage and soft limits, and the mount's hard limits.
+ * DQ_FAKE_B only means that *all* limits are zero, which need not match
+ * the mount defaults.
+ */
+ return dquot->dq_dqb.dqb_curspace == 0 &&
+ dquot->dq_dqb.dqb_curinodes == 0 &&
+ dquot->dq_dqb.dqb_bsoftlimit == 0 &&
+ dquot->dq_dqb.dqb_isoftlimit == 0 &&
+ dquot->dq_dqb.dqb_bhardlimit == bhardlimit &&
+ dquot->dq_dqb.dqb_ihardlimit == ihardlimit;
}
/*
- * Store limits from dquot in the tree unless it's fake. If it is fake
- * remove the id from the tree since there is no useful information in
- * there.
+ * Remove the ID from the tree if it has no usage and its limits match
+ * the mount defaults. Otherwise, save its limits for the next acquire.
*/
static int shmem_release_dquot(struct dquot *dquot)
{
@@ -295,7 +297,7 @@ static int shmem_release_dquot(struct dquot *dquot)
return -ENOENT;
found:
- if (shmem_is_empty_dquot(dquot)) {
+ if (shmem_is_default_dquot(dquot)) {
/* Remove entry from the tree */
rb_erase(&entry->node, info->dqi_priv);
kfree(entry);
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 601979b97f95..278060e24113 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -2434,7 +2434,8 @@ static int unuse_pte(struct vm_area_struct *vma, pmd_t *pmd,
return 0;
swapcache = folio;
- folio = ksm_might_need_to_copy(folio, vma, addr);
+ folio = ksm_might_need_to_copy(folio,
+ folio_file_page(folio, swp_offset(entry)), vma, addr);
if (unlikely(!folio))
return -ENOMEM;
else if (unlikely(folio == ERR_PTR(-EHWPOISON))) {
@@ -2609,6 +2610,8 @@ static inline int unuse_pmd_range(struct vm_area_struct *vma, pud_t *pud,
pmd = pmd_offset(pud, addr);
do {
cond_resched();
+ if (signal_pending(current))
+ return -ERESTARTSYS;
next = pmd_addr_end(addr, end);
ret = unuse_pte_range(vma, pmd, addr, next, type);
if (ret)
@@ -2823,7 +2826,7 @@ static int try_to_unuse(unsigned int type)
if (swap_usage_in_pages(si)) {
if (!signal_pending(current))
goto retry;
- return -EINTR;
+ return -ERESTARTSYS;
}
success:
diff --git a/mm/truncate.c b/mm/truncate.c
index 29782aced8ba..b7c5d3384748 100644
--- a/mm/truncate.c
+++ b/mm/truncate.c
@@ -436,8 +436,11 @@ void truncate_inode_pages_range(struct address_space *mapping,
}
}
+ if (start >= end)
+ return;
+
index = start;
- while (index < end) {
+ while (true) {
cond_resched();
if (!find_get_entries(mapping, &index, end - 1, &fbatch,
indices)) {
@@ -460,6 +463,7 @@ void truncate_inode_pages_range(struct address_space *mapping,
folio_lock(folio);
VM_BUG_ON_FOLIO(!folio_contains(folio, indices[i]), folio);
folio_wait_writeback(folio);
+ /* Do not handle errors, we'll do another lookup */
truncate_inode_folio(mapping, folio);
folio_unlock(folio);
}
@@ -721,7 +725,12 @@ int invalidate_inode_pages2_range(struct address_space *mapping,
folio_lock(folio);
if (unlikely(folio->mapping != mapping)) {
folio_unlock(folio);
- continue;
+ /*
+ * Folio might have been replaced: look up
+ * again.
+ */
+ index = indices[i];
+ break;
}
VM_BUG_ON_FOLIO(!folio_contains(folio, indices[i]), folio);
folio_wait_writeback(folio);
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 500962be1162..f8e2b7f2f61c 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -115,8 +115,7 @@ struct vm_area_struct *find_vma_and_prepare_anon(struct mm_struct *mm,
vma = vma_lookup(mm, addr);
if (!vma)
vma = ERR_PTR(-ENOENT);
- else if (!(vma->vm_flags & VM_SHARED) &&
- unlikely(anon_vma_prepare(vma)))
+ else if (vma_is_cow_mapping(vma) && unlikely(anon_vma_prepare(vma)))
vma = ERR_PTR(-ENOMEM);
return vma;
@@ -144,7 +143,7 @@ static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm,
* We know we're going to need to use anon_vma, so check
* that early.
*/
- if (!(vma->vm_flags & VM_SHARED) && unlikely(!vma->anon_vma))
+ if (vma_is_cow_mapping(vma) && unlikely(!vma->anon_vma))
vma_end_read(vma);
else
return vma;
@@ -1234,33 +1233,6 @@ int mrwprotect_range(struct userfaultfd_ctx *ctx, unsigned long start,
return 0;
}
-void double_pt_lock(spinlock_t *ptl1,
- spinlock_t *ptl2)
- __acquires(ptl1)
- __acquires(ptl2)
-{
- if (ptl1 > ptl2)
- swap(ptl1, ptl2);
- /* lock in virtual address order to avoid lock inversion */
- spin_lock(ptl1);
- if (ptl1 != ptl2)
- spin_lock_nested(ptl2, SINGLE_DEPTH_NESTING);
- else
- __acquire(ptl2);
-}
-
-void double_pt_unlock(spinlock_t *ptl1,
- spinlock_t *ptl2)
- __releases(ptl1)
- __releases(ptl2)
-{
- spin_unlock(ptl1);
- if (ptl1 != ptl2)
- spin_unlock(ptl2);
- else
- __release(ptl2);
-}
-
static inline bool is_pte_pages_stable(pte_t *dst_pte, pte_t *src_pte,
pte_t orig_dst_pte, pte_t orig_src_pte,
pmd_t *dst_pmd, pmd_t dst_pmdval)
@@ -4021,6 +3993,8 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx,
ret = 0;
for_each_vma_range(vmi, vma, end) {
+ struct userfaultfd_wake_range range;
+
cond_resched();
/* VMA not registered with userfaultfd. */
@@ -4035,18 +4009,17 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx,
start = vma->vm_start;
vma_end = min(end, vma->vm_end);
- if (userfaultfd_missing(vma)) {
- /*
- * Wake any concurrent pending userfault while
- * we unregister, so they will not hang
- * permanently and it avoids userland to call
- * UFFDIO_WAKE explicitly.
- */
- struct userfaultfd_wake_range range;
- range.start = start;
- range.len = vma_end - start;
- wake_userfault(vma->vm_userfaultfd_ctx.ctx, &range);
- }
+ /* Prevent new userfaults from being queued after the wakeup. */
+ vma_start_write(vma);
+
+ /*
+ * Wake any concurrent pending userfault while we unregister,
+ * so they will not hang permanently and it avoids userland to
+ * call UFFDIO_WAKE explicitly.
+ */
+ range.start = start;
+ range.len = vma_end - start;
+ wake_userfault(ctx, &range);
vma = userfaultfd_clear_vma(&vmi, prev, vma,
start, vma_end);
diff --git a/mm/util.c b/mm/util.c
index bf0513d1d3d0..e31eb565445c 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -697,6 +697,19 @@ void *vcalloc_noprof(size_t n, size_t size)
}
EXPORT_SYMBOL(vcalloc_noprof);
+/**
+ * folio_anon_vma - Get a folio's raw anon_vma pointer.
+ * @folio: The folio.
+ *
+ * The returned pointer may be stale for an unmapped folio. The caller
+ * must ensure the anon_vma stays alive before dereferencing it. A folio
+ * reference or lock alone is insufficient; use folio_get_anon_vma() when
+ * the anon_vma's lifetime is not otherwise protected. Note that
+ * folio_get_anon_vma() is only useful for folios that are still mapped.
+ *
+ * Return: The raw anon_vma pointer for an anonymous, non-KSM folio, or
+ * %NULL otherwise.
+ */
struct anon_vma *folio_anon_vma(const struct folio *folio)
{
unsigned long mapping = (unsigned long)folio->mapping;
diff --git a/mm/vma.c b/mm/vma.c
index 3d63f73e723b..e382fd32f0ff 100644
--- a/mm/vma.c
+++ b/mm/vma.c
@@ -535,6 +535,15 @@ void unmap_region(struct unmap_desc *unmap)
tlb_finish_mmu(&tlb);
}
+static int check_split(struct vm_area_struct *vma, unsigned long addr)
+{
+ if (!vma->vm_ops)
+ return 0;
+ if (!vma->vm_ops->may_split)
+ return 0;
+ return vma->vm_ops->may_split(vma, addr);
+}
+
/*
* __split_vma() bypasses sysctl_max_map_count checking. We use this where it
* has already been checked or doesn't make sense to fail.
@@ -551,12 +560,11 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
WARN_ON(vma->vm_start >= addr);
WARN_ON(vma->vm_end <= addr);
- if (vma->vm_ops && vma->vm_ops->may_split) {
- err = vma->vm_ops->may_split(vma, addr);
- if (err)
- return err;
- }
+ err = check_split(vma, addr);
+ if (err)
+ return err;
+ vma_start_write(vma);
new = vm_area_dup(vma);
if (!new)
return -ENOMEM;
@@ -587,7 +595,6 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
if (new->vm_ops && new->vm_ops->open)
new->vm_ops->open(new);
- vma_start_write(vma);
vma_start_write(new);
init_vma_prep(&vp, vma);
@@ -964,6 +971,10 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(
if (vmg->__remove_middle && !can_merge_remove_vma(middle))
return NULL;
+ /* If middle isn't removed then it's resized, i.e. splits. */
+ if (!vmg->__remove_middle && check_split(middle, left_side ? end : start))
+ return NULL;
+
/*
* If we merge both VMAs, then next is also deleted. This implies
* merge_will_delete_vma also.
@@ -1143,6 +1154,7 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
struct vm_area_struct *next = vmg->next;
unsigned long end = vmg->end;
bool can_merge_left, can_merge_right;
+ bool move_iter_back = false;
mmap_assert_write_locked(vmg->mm);
VM_WARN_ON_VMG(vmg->middle, vmg);
@@ -1182,23 +1194,30 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
vmg->end = end;
/* In expand-only case we are already positioned at prev. */
- if (!vmg->just_expand) {
- /* Equivalent to going to the previous range. */
- vma_prev(vmg->vmi);
- }
+ if (!vmg->just_expand)
+ move_iter_back = true;
}
+ if (!can_merge_left && !can_merge_right)
+ return NULL;
+
+ if (move_iter_back)
+ vma_prev(vmg->vmi);
+
/*
* Now try to expand adjacent VMA(s). This takes care of removing the
* following VMA if we have VMAs on both sides.
*/
- if (vmg->target && !vma_expand(vmg)) {
- khugepaged_enter_vma(vmg->target, vmg->vm_flags);
- vmg->state = VMA_MERGE_SUCCESS;
- return vmg->target;
+ if (vma_expand(vmg)) {
+ /* Error occurred, restore iterator. */
+ if (move_iter_back)
+ vma_iter_next_range(vmg->vmi);
+ return NULL;
}
- return NULL;
+ khugepaged_enter_vma(vmg->target, vmg->vm_flags);
+ vmg->state = VMA_MERGE_SUCCESS;
+ return vmg->target;
}
/*
@@ -1293,7 +1312,7 @@ int vma_expand(struct vma_merge_struct *vmg)
if (!ret && vmg->copied_from)
ret = dup_anon_vma(target, vmg->copied_from, &anon_dup);
if (ret)
- return ret;
+ goto nomem;
if (remove_next) {
vma_flags_t next_sticky;
@@ -3241,7 +3260,8 @@ static int acct_stack_growth(struct vm_area_struct *vma,
* Overcommit.. This must be the final test, as it will
* update security statistics.
*/
- if (security_vm_enough_memory_mm(mm, grow))
+ if (vma_test(vma, VMA_ACCOUNT_BIT) &&
+ security_vm_enough_memory_mm(mm, grow))
return -ENOMEM;
return 0;
diff --git a/mm/vma_init.c b/mm/vma_init.c
index baa7e82f47e3..d661f229a734 100644
--- a/mm/vma_init.c
+++ b/mm/vma_init.c
@@ -127,6 +127,9 @@ struct vm_area_struct *vm_area_dup(struct vm_area_struct *orig)
{
struct vm_area_struct *new = kmem_cache_alloc(vm_area_cachep, GFP_KERNEL);
+ /* Must hold a write lock to stabilise atomic VMA flags. */
+ vma_assert_write_locked(orig);
+
if (!new)
return NULL;
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 6a53762d5544..88fd31d13585 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3194,8 +3194,11 @@ struct ctrl_pos {
int gain;
};
-static void read_ctrl_pos(struct lruvec *lruvec, int type, int tier, int gain,
- struct ctrl_pos *pos)
+/*
+ * __noipa works around gcc-16 warning for uninitialized use of pos->refaulted
+ */
+static void __noipa read_ctrl_pos(struct lruvec *lruvec, int type, int tier_min,
+ int tier_max, int gain, struct ctrl_pos *pos)
{
int i;
struct lru_gen_folio *lrugen = &lruvec->lrugen;
@@ -3204,7 +3207,7 @@ static void read_ctrl_pos(struct lruvec *lruvec, int type, int tier, int gain,
pos->gain = gain;
pos->refaulted = pos->total = 0;
- for (i = tier % MAX_NR_TIERS; i <= min(tier, MAX_NR_TIERS - 1); i++) {
+ for (i = tier_min; i <= tier_max; i++) {
pos->refaulted += lrugen->avg_refaulted[type][i] +
atomic_long_read(&lrugen->refaulted[hist][type][i]);
pos->total += lrugen->avg_total[type][i] +
@@ -3771,7 +3774,7 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end,
walk->mm_stats[MM_NONLEAF_FOUND]++;
- if (!walk_pte_range(&val, addr, next, args))
+ if (!walk_pte_range(pmd + i, addr, next, args))
continue;
walk->mm_stats[MM_NONLEAF_ADDED]++;
@@ -3807,7 +3810,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long start, unsigned long end,
if (!pud_present(val) || WARN_ON_ONCE(pud_leaf(val)))
continue;
- walk_pmd_range(&val, addr, next, args);
+ walk_pmd_range(pud + i, addr, next, args);
if (need_resched() || walk->batched >= MAX_LRU_BATCH) {
end = (addr | ~PUD_MASK) + 1;
@@ -4801,16 +4804,16 @@ static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec,
static int get_tier_idx(struct lruvec *lruvec, int type)
{
int tier;
- struct ctrl_pos sp, pv = {};
+ struct ctrl_pos sp, pv;
/*
* To leave a margin for fluctuations, use a larger gain factor (2:3).
* This value is chosen because any other tier would have at least twice
* as many refaults as the first tier.
*/
- read_ctrl_pos(lruvec, type, 0, 2, &sp);
- for (tier = 1; tier < MAX_NR_TIERS; tier++) {
- read_ctrl_pos(lruvec, type, tier, 3, &pv);
+ read_ctrl_pos(lruvec, type, LRU_TIER_MIN, LRU_TIER_MIN, 2, &sp);
+ for (tier = LRU_TIER_MIN + 1; tier <= LRU_TIER_MAX; tier++) {
+ read_ctrl_pos(lruvec, type, tier, tier, 3, &pv);
if (!positive_ctrl_err(&sp, &pv))
break;
}
@@ -4820,7 +4823,7 @@ static int get_tier_idx(struct lruvec *lruvec, int type)
static int get_type_to_scan(struct lruvec *lruvec, int swappiness)
{
- struct ctrl_pos sp, pv = {};
+ struct ctrl_pos sp, pv;
if (swappiness <= MIN_SWAPPINESS + 1)
return LRU_GEN_FILE;
@@ -4831,8 +4834,10 @@ static int get_type_to_scan(struct lruvec *lruvec, int swappiness)
* Compare the sum of all tiers of anon with that of file to determine
* which type to scan.
*/
- read_ctrl_pos(lruvec, LRU_GEN_ANON, MAX_NR_TIERS, swappiness, &sp);
- read_ctrl_pos(lruvec, LRU_GEN_FILE, MAX_NR_TIERS, MAX_SWAPPINESS - swappiness, &pv);
+ read_ctrl_pos(lruvec, LRU_GEN_ANON, LRU_TIER_MIN, LRU_TIER_MAX,
+ swappiness, &sp);
+ read_ctrl_pos(lruvec, LRU_GEN_FILE, LRU_TIER_MIN, LRU_TIER_MAX,
+ MAX_SWAPPINESS - swappiness, &pv);
return positive_ctrl_err(&sp, &pv);
}
diff --git a/net/9p/trans_usbg.c b/net/9p/trans_usbg.c
index 419cda13a7b5..7d600e1288a3 100644
--- a/net/9p/trans_usbg.c
+++ b/net/9p/trans_usbg.c
@@ -149,7 +149,8 @@ static void usb9pfs_tx_complete(struct usb_ep *ep, struct usb_request *req)
{
struct f_usb9pfs *usb9pfs = ep->driver_data;
struct usb_composite_dev *cdev = usb9pfs->function.config->cdev;
- struct p9_req_t *p9_tx_req = req->context;
+ struct p9_client *client;
+ struct p9_req_t *p9_tx_req;
unsigned long flags;
/* reset zero packages */
@@ -165,18 +166,25 @@ static void usb9pfs_tx_complete(struct usb_ep *ep, struct usb_request *req)
ep->name, req->status, req->actual, req->length);
spin_lock_irqsave(&usb9pfs->lock, flags);
- WRITE_ONCE(p9_tx_req->status, REQ_STATUS_SENT);
+ client = usb9pfs->client;
+ p9_tx_req = req->context;
+ req->context = NULL;
- p9_req_put(usb9pfs->client, p9_tx_req);
+ if (!client || !p9_tx_req)
+ goto unlock_complete;
- req->context = NULL;
+ WRITE_ONCE(p9_tx_req->status, REQ_STATUS_SENT);
+
+ p9_req_put(client, p9_tx_req);
+unlock_complete:
spin_unlock_irqrestore(&usb9pfs->lock, flags);
complete(&usb9pfs->send);
}
-static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
+static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs,
+ struct p9_client *client, void *buf)
{
struct p9_req_t *p9_rx_req;
struct p9_fcall rc;
@@ -202,7 +210,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
"mux %p pkt: size: %d bytes tag: %d\n",
usb9pfs, rc.size, rc.tag);
- p9_rx_req = p9_tag_lookup(usb9pfs->client, rc.tag);
+ p9_rx_req = p9_tag_lookup(client, rc.tag);
if (!p9_rx_req || p9_rx_req->status != REQ_STATUS_SENT) {
p9_debug(P9_DEBUG_ERROR, "Unexpected packet tag %d\n", rc.tag);
return NULL;
@@ -212,7 +220,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
p9_debug(P9_DEBUG_ERROR,
"requested packet size too big: %d for tag %d with capacity %zd\n",
rc.size, rc.tag, p9_rx_req->rc.capacity);
- p9_req_put(usb9pfs->client, p9_rx_req);
+ p9_req_put(client, p9_rx_req);
return NULL;
}
@@ -220,7 +228,7 @@ static struct p9_req_t *usb9pfs_rx_header(struct f_usb9pfs *usb9pfs, void *buf)
p9_debug(P9_DEBUG_ERROR,
"No recv fcall for tag %d (req %p), disconnecting!\n",
rc.tag, p9_rx_req);
- p9_req_put(usb9pfs->client, p9_rx_req);
+ p9_req_put(client, p9_rx_req);
return NULL;
}
@@ -231,8 +239,10 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req)
{
struct f_usb9pfs *usb9pfs = ep->driver_data;
struct usb_composite_dev *cdev = usb9pfs->function.config->cdev;
+ struct p9_client *client;
struct p9_req_t *p9_rx_req;
unsigned int req_size = req->actual;
+ unsigned long flags;
int status = REQ_STATUS_RCVD;
if (req->status) {
@@ -241,9 +251,16 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req)
return;
}
- p9_rx_req = usb9pfs_rx_header(usb9pfs, req->buf);
- if (!p9_rx_req)
+ spin_lock_irqsave(&usb9pfs->lock, flags);
+ client = usb9pfs->client;
+ if (!client) {
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
return;
+ }
+
+ p9_rx_req = usb9pfs_rx_header(usb9pfs, client, req->buf);
+ if (!p9_rx_req)
+ goto out_unlock;
if (req_size > p9_rx_req->rc.capacity) {
dev_err(&cdev->gadget->dev,
@@ -257,8 +274,11 @@ static void usb9pfs_rx_complete(struct usb_ep *ep, struct usb_request *req)
p9_rx_req->rc.size = req_size;
- p9_client_cb(usb9pfs->client, p9_rx_req, status);
- p9_req_put(usb9pfs->client, p9_rx_req);
+ p9_client_cb(client, p9_rx_req, status);
+ p9_req_put(client, p9_rx_req);
+
+out_unlock:
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
complete(&usb9pfs->received);
}
@@ -278,18 +298,18 @@ static void disable_usb9pfs(struct f_usb9pfs *usb9pfs)
struct usb_composite_dev *cdev =
usb9pfs->function.config->cdev;
+ disable_ep(cdev, usb9pfs->in_ep);
+ disable_ep(cdev, usb9pfs->out_ep);
+
if (usb9pfs->in_req) {
usb_ep_free_request(usb9pfs->in_ep, usb9pfs->in_req);
usb9pfs->in_req = NULL;
}
if (usb9pfs->out_req) {
- usb_ep_free_request(usb9pfs->out_ep, usb9pfs->out_req);
+ free_ep_req(usb9pfs->out_ep, usb9pfs->out_req);
usb9pfs->out_req = NULL;
}
-
- disable_ep(cdev, usb9pfs->in_ep);
- disable_ep(cdev, usb9pfs->out_ep);
dev_dbg(&cdev->gadget->dev, "%s disabled\n",
usb9pfs->function.name);
}
@@ -314,10 +334,6 @@ static int alloc_requests(struct usb_composite_dev *cdev,
usb9pfs->in_req->complete = usb9pfs_tx_complete;
usb9pfs->out_req->complete = usb9pfs_rx_complete;
- /* length will be set in complete routine */
- usb9pfs->in_req->context = usb9pfs;
- usb9pfs->out_req->context = usb9pfs;
-
return 0;
fail_in:
@@ -391,7 +407,7 @@ static int p9_usbg_create(struct p9_client *client, struct fs_context *fc)
guard(mutex)(&usb9pfs_lock);
list_for_each_entry(dev, &usbg_instance_list, usb9pfs_instance) {
- if (!strncmp(devname, dev->tag, strlen(devname))) {
+ if (!strcmp(devname, dev->tag)) {
if (!dev->inuse) {
dev->inuse = true;
found = 1;
@@ -416,7 +432,9 @@ static int p9_usbg_create(struct p9_client *client, struct fs_context *fc)
client->status = Disconnected;
else
client->status = Connected;
+ spin_lock_irq(&usb9pfs->lock);
usb9pfs->client = client;
+ spin_unlock_irq(&usb9pfs->lock);
client->trans_mod->maxsize = usb9pfs->buflen;
@@ -427,18 +445,26 @@ static int p9_usbg_create(struct p9_client *client, struct fs_context *fc)
static void usb9pfs_clear_tx(struct f_usb9pfs *usb9pfs)
{
+ struct p9_client *client;
struct p9_req_t *req;
+ unsigned long flags;
- guard(spinlock_irqsave)(&usb9pfs->lock);
+ spin_lock_irqsave(&usb9pfs->lock, flags);
+ client = usb9pfs->client;
+ usb9pfs->client = NULL;
+ req = usb9pfs->in_req ? usb9pfs->in_req->context : NULL;
+ if (usb9pfs->in_req)
+ usb9pfs->in_req->context = NULL;
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
- req = usb9pfs->in_req->context;
- if (!req)
+ if (!req || !client)
return;
if (!req->t_err)
req->t_err = -ECONNRESET;
- p9_client_cb(usb9pfs->client, req, REQ_STATUS_ERROR);
+ p9_client_cb(client, req, REQ_STATUS_ERROR);
+ usb9pfs->in_req->context = NULL;
}
static void p9_usbg_close(struct p9_client *client)
@@ -466,6 +492,9 @@ static void p9_usbg_close(struct p9_client *client)
mutex_lock(&usb9pfs_lock);
dev->inuse = false;
mutex_unlock(&usb9pfs_lock);
+
+ disable_usb9pfs(usb9pfs);
+ reinit_completion(&usb9pfs->send);
}
static int p9_usbg_request(struct p9_client *client, struct p9_req_t *p9_req)
@@ -748,8 +777,18 @@ static int usb9pfs_set_alt(struct usb_function *f,
static void usb9pfs_disable(struct usb_function *f)
{
struct f_usb9pfs *usb9pfs = func_to_usb9pfs(f);
+ unsigned long flags;
+ spin_lock_irqsave(&usb9pfs->lock, flags);
+ if (usb9pfs->client)
+ usb9pfs->client->status = Disconnected;
+ spin_unlock_irqrestore(&usb9pfs->lock, flags);
usb9pfs_clear_tx(usb9pfs);
+
+ if (usb9pfs->in_ep->enabled)
+ disable_usb9pfs(usb9pfs);
+
+ reinit_completion(&usb9pfs->send);
}
static struct usb_function *usb9pfs_alloc(struct usb_function_instance *fi)
diff --git a/net/9p/trans_virtio.c b/net/9p/trans_virtio.c
index b0d0094ec8e2..9fa63e3ebf67 100644
--- a/net/9p/trans_virtio.c
+++ b/net/9p/trans_virtio.c
@@ -633,6 +633,11 @@ static int p9_virtio_probe(struct virtio_device *vdev)
err = -EINVAL;
goto out_free_vq;
}
+ if (tag_len > NAME_MAX) {
+ dev_err(&vdev->dev, "mount tag too long (%u bytes)\n", tag_len);
+ err = -EINVAL;
+ goto out_free_vq;
+ }
tag = kzalloc(tag_len + 1, GFP_KERNEL);
if (!tag) {
err = -ENOMEM;
diff --git a/net/9p/trans_xen.c b/net/9p/trans_xen.c
index f9fb2db7a066..71a4d276321e 100644
--- a/net/9p/trans_xen.c
+++ b/net/9p/trans_xen.c
@@ -200,9 +200,19 @@ static void p9_xen_response(struct work_struct *work)
masked_prod, &masked_cons,
XEN_9PFS_RING_SIZE(ring));
+ if (h.size < sizeof(h) ||
+ h.size > xen_9pfs_queued(prod, cons,
+ XEN_9PFS_RING_SIZE(ring))) {
+ dev_warn_ratelimited(&priv->dev->dev,
+ "bad response size %u from backend\n", h.size);
+ break;
+ }
+
req = p9_tag_lookup(priv->client, h.tag);
if (!req || req->status != REQ_STATUS_SENT) {
dev_warn(&priv->dev->dev, "Wrong req tag=%x\n", h.tag);
+ if (req)
+ p9_req_put(priv->client, req);
cons += h.size;
virt_mb();
ring->intf->in_cons = cons;
diff --git a/net/batman-adv/bat_iv_ogm.c b/net/batman-adv/bat_iv_ogm.c
index 53fbdbbe8f4f..1c2ab79adabc 100644
--- a/net/batman-adv/bat_iv_ogm.c
+++ b/net/batman-adv/bat_iv_ogm.c
@@ -1909,6 +1909,11 @@ static void batadv_iv_send_outstanding_bat_ogm_packet(struct work_struct *work)
goto out;
}
+ /* Mark aggregate as full before forcing emit. */
+ spin_lock_bh(&bat_priv->forw_bat_list_lock);
+ forw_packet->num_packets = BATADV_MAX_AGGREGATION_PACKETS;
+ spin_unlock_bh(&bat_priv->forw_bat_list_lock);
+
batadv_iv_ogm_emit(forw_packet);
/* we have to have at least one packet in the queue to determine the
diff --git a/net/batman-adv/distributed-arp-table.c b/net/batman-adv/distributed-arp-table.c
index 011cfdc47fab..86791ddc148f 100644
--- a/net/batman-adv/distributed-arp-table.c
+++ b/net/batman-adv/distributed-arp-table.c
@@ -653,6 +653,7 @@ static void batadv_choose_next_candidate(struct batadv_priv *bat_priv,
* NOT_FOUND
*/
cands[select].type = BATADV_DAT_CANDIDATE_NOT_FOUND;
+ cands[select].orig_node = NULL;
/* iterate over the originator list and find the node with the closest
* dat_address which has not been selected yet
diff --git a/net/batman-adv/main.c b/net/batman-adv/main.c
index d89d44706269..5d5a0be323e6 100644
--- a/net/batman-adv/main.c
+++ b/net/batman-adv/main.c
@@ -294,6 +294,11 @@ void batadv_mesh_free(struct net_device *mesh_iface)
WRITE_ONCE(bat_priv->mesh_state, BATADV_MESH_DEACTIVATING);
+ /* the originator purge worker accesses (among others) the TT tables
+ * and must therefore be stopped before any of them are freed
+ */
+ disable_delayed_work_sync(&bat_priv->orig_work);
+
batadv_purge_outstanding_packets(bat_priv, NULL);
batadv_tp_stop_all(bat_priv);
@@ -483,8 +488,10 @@ int batadv_batman_skb_recv(struct sk_buff *skb, struct net_device *dev,
* shut down otherwise the packet may trigger de-reference errors
* further down in the receive path.
*/
- if (!kref_get_unless_zero(&hard_iface->refcount))
- goto err_out;
+ if (!kref_get_unless_zero(&hard_iface->refcount)) {
+ kfree_skb(skb);
+ return NET_RX_DROP;
+ }
skb = skb_share_check(skb, GFP_ATOMIC);
@@ -540,7 +547,6 @@ int batadv_batman_skb_recv(struct sk_buff *skb, struct net_device *dev,
kfree_skb(skb);
err_put:
batadv_hardif_put(hard_iface);
-err_out:
return NET_RX_DROP;
}
diff --git a/net/batman-adv/multicast_forw.c b/net/batman-adv/multicast_forw.c
index ac97cd8b81dc..9ffed1f237b6 100644
--- a/net/batman-adv/multicast_forw.c
+++ b/net/batman-adv/multicast_forw.c
@@ -950,8 +950,11 @@ static int batadv_mcast_forw_packet(struct batadv_priv *bat_priv,
dest = (u8 *)mcast_tracker + sizeof(*mcast_tracker);
- /* check if full tracker tvlv is within skb length */
+ /* check if full tracker tvlv, including padding, is within skb length */
tvlv_len = sizeof(*mcast_tracker) + ETH_ALEN * num_dests;
+ if (!(num_dests % 2))
+ tvlv_len += 2;
+
if (tvlv_len > skb_network_header_len(skb))
return -EINVAL;
diff --git a/net/batman-adv/tp_meter.c b/net/batman-adv/tp_meter.c
index 022eb13e8ec4..56feb02d2b7e 100644
--- a/net/batman-adv/tp_meter.c
+++ b/net/batman-adv/tp_meter.c
@@ -221,7 +221,7 @@ static void batadv_tp_batctl_notify(enum batadv_tp_meter_reason reason,
unsigned long start_time, u64 total_sent,
u32 cookie)
{
- u32 total_bytes;
+ u64 total_bytes;
u32 test_time;
u8 result;
diff --git a/net/batman-adv/translation-table.c b/net/batman-adv/translation-table.c
index 68f79853032e..ed8dcb8a66b1 100644
--- a/net/batman-adv/translation-table.c
+++ b/net/batman-adv/translation-table.c
@@ -418,6 +418,9 @@ batadv_tt_orig_list_entry_put(struct batadv_tt_orig_list_entry *orig_entry)
* @bat_priv: the bat priv with all the mesh interface information
* @tt_local_entry: the TT entry involved in the event
* @event_flags: flags to store in the event structure
+ *
+ * Only a single change per client is queued. It describes the latest state of
+ * the client and a new event therefore replaces an already queued one.
*/
static void batadv_tt_local_event(struct batadv_priv *bat_priv,
struct batadv_tt_local_entry *tt_local_entry,
@@ -429,14 +432,12 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv,
struct batadv_tt_change_node *entry;
struct batadv_tt_change_node *safe;
bool del_op_requested;
- bool del_op_entry;
size_t changes;
tt_change_node = kmem_cache_alloc(batadv_tt_change_cache, GFP_ATOMIC);
if (!tt_change_node)
return;
- tt_change_node->change.flags = flags;
memset(tt_change_node->change.reserved, 0,
sizeof(tt_change_node->change.reserved));
ether_addr_copy(tt_change_node->change.addr, common->addr);
@@ -444,6 +445,15 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv,
del_op_requested = flags & BATADV_TT_CLIENT_DEL;
+ /* the ROAM flag is only meaningful for the receiver in combination
+ * with a DEL. An ADD with ROAM would be interpreted as a roaming
+ * advertisement
+ */
+ if (!del_op_requested)
+ flags &= ~BATADV_TT_CLIENT_ROAM;
+
+ tt_change_node->change.flags = flags;
+
/* check for ADD+DEL, DEL+ADD, ADD+ADD or DEL+DEL events */
spin_lock_bh(&bat_priv->tt.changes_list_lock);
changes = READ_ONCE(bat_priv->tt.local_changes);
@@ -455,25 +465,12 @@ static void batadv_tt_local_event(struct batadv_priv *bat_priv,
if (entry->change.vid != tt_change_node->change.vid)
continue;
- del_op_entry = entry->change.flags & BATADV_TT_CLIENT_DEL;
- if (del_op_requested != del_op_entry) {
- /* DEL+ADD in the same orig interval have no effect and
- * can be removed to avoid silly behaviour on the
- * receiver side. The other way around (ADD+DEL) can
- * happen in case of roaming of a client still in the
- * NEW state. Roaming of NEW clients is now possible due
- * to automatically recognition of "temporary" clients
- */
- list_del(&entry->list);
- kmem_cache_free(batadv_tt_change_cache, entry);
- changes--;
- } else {
- /* this is a second add or del in the same originator
- * interval. It could mean that flags have been changed
- * (e.g. double add): update them
- */
- entry->change.flags = flags;
- }
+ /* the other nodes may know the client with different
+ * flags. A DEL+ADD must therefore still announce the
+ * current flags and an ADD+DEL must still remove the
+ * client. ADD+ADD and DEL+DEL only update the flags
+ */
+ entry->change.flags = flags;
kmem_cache_free(batadv_tt_change_cache, tt_change_node);
goto update_changes;
@@ -624,6 +621,7 @@ bool batadv_tt_local_add(struct net_device *mesh_iface, const u8 *addr,
bool iif_is_wifi = false;
struct hlist_head *head;
int packet_size_max;
+ bool added = false;
bool ret = false;
u8 remote_flags;
int hash_added;
@@ -656,7 +654,7 @@ bool batadv_tt_local_add(struct net_device *mesh_iface, const u8 *addr,
* flag can be reset like it was never enqueued
*/
tt_local->common.flags &= ~BATADV_TT_CLIENT_PENDING;
- goto add_event;
+ added = true;
}
if (tt_local->common.flags & BATADV_TT_CLIENT_ROAM) {
@@ -737,14 +735,17 @@ bool batadv_tt_local_add(struct net_device *mesh_iface, const u8 *addr,
goto out;
}
-add_event:
- batadv_tt_local_event(bat_priv, tt_local, BATADV_NO_FLAGS);
+ added = true;
check_roaming:
+ if (added)
+ batadv_tt_local_event(bat_priv, tt_local, BATADV_NO_FLAGS);
+
/* Check whether it is a roaming, but don't do anything if the roaming
* process has already been handled
*/
- if (tt_global && !(tt_global->common.flags & BATADV_TT_CLIENT_ROAM)) {
+ if (tt_global &&
+ (roamed_back || !(tt_global->common.flags & BATADV_TT_CLIENT_ROAM))) {
/* These node are probably going to update their tt table */
head = &tt_global->orig_list;
rcu_read_lock();
@@ -1006,22 +1007,24 @@ batadv_tt_prepare_tvlv_local_data(struct batadv_priv *bat_priv,
* batadv_tt_tvlv_container_update() - update the translation table tvlv
* container after local tt changes have been committed
* @bat_priv: the bat priv with all the mesh interface information
+ * @changes_list: list of committed changes which will be consumed
+ * @tt_diff_entries_num: number of entries in @changes_list
*/
-static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv)
+static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv,
+ struct list_head *changes_list,
+ int tt_diff_entries_num)
{
struct batadv_tvlv_tt_change *tt_change;
struct batadv_tt_change_node *entry;
struct batadv_tvlv_tt_data *tt_data;
struct batadv_tt_change_node *safe;
int tt_diff_entries_count = 0;
- int tt_diff_entries_num = 0;
bool drop_changes = false;
size_t tt_extra_len = 0;
int tt_change_len = 0;
int tt_diff_len;
u16 tvlv_len;
- tt_diff_entries_num = READ_ONCE(bat_priv->tt.local_changes);
tt_diff_len = batadv_tt_len(tt_diff_entries_num);
/* if we have too many changes for one packet don't send any
@@ -1039,19 +1042,39 @@ static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv)
tvlv_len = batadv_tt_prepare_tvlv_local_data(bat_priv, &tt_data,
&tt_change, &tt_diff_len);
- if (!tvlv_len)
+ if (!tvlv_len) {
+ /* the changes are lost and the other nodes have to request the
+ * full table. Don't answer TT requests for the new TTVN with
+ * the stale changeset of the previous TTVN. Without changes, the
+ * TTVN was not incremented and the last changeset is still valid
+ */
+ if (drop_changes || tt_diff_entries_num > 0) {
+ list_for_each_entry_safe(entry, safe, changes_list,
+ list) {
+ list_del(&entry->list);
+ kmem_cache_free(batadv_tt_change_cache, entry);
+ }
+
+ spin_lock_bh(&bat_priv->tt.last_changeset_lock);
+ kfree(bat_priv->tt.last_changeset);
+ bat_priv->tt.last_changeset = NULL;
+ bat_priv->tt.last_changeset_len = 0;
+ spin_unlock_bh(&bat_priv->tt.last_changeset_lock);
+ }
+
+ /* announce the already committed TTVN in the next OGM interval
+ * instead of waiting for BATADV_TT_OGM_APPEND_MAX idle rounds
+ */
+ atomic_set(&bat_priv->tt.ogm_append_cnt, 0);
return;
+ }
tt_data->flags = BATADV_TT_OGM_DIFF;
if (!drop_changes && tt_diff_len == 0)
goto container_register;
- spin_lock_bh(&bat_priv->tt.changes_list_lock);
- WRITE_ONCE(bat_priv->tt.local_changes, 0);
-
- list_for_each_entry_safe(entry, safe, &bat_priv->tt.changes_list,
- list) {
+ list_for_each_entry_safe(entry, safe, changes_list, list) {
if (tt_diff_entries_count < tt_diff_entries_num) {
memcpy(tt_change + tt_diff_entries_count,
&entry->change,
@@ -1061,7 +1084,6 @@ static void batadv_tt_tvlv_container_update(struct batadv_priv *bat_priv)
list_del(&entry->list);
kmem_cache_free(batadv_tt_change_cache, entry);
}
- spin_unlock_bh(&bat_priv->tt.changes_list_lock);
tt_extra_len = batadv_tt_len(tt_diff_entries_num -
tt_diff_entries_count);
@@ -2830,8 +2852,12 @@ static bool batadv_tt_local_valid(const void *entry_ptr,
if (tt_common_entry->flags & BATADV_TT_CLIENT_NEW)
return false;
+ /* the entry is (still) announced. A ROAM flag of an entry which is
+ * pending to be removed must not be transmitted. Otherwise, the
+ * receiver would handle it like a roaming advertisement
+ */
if (flags)
- *flags = tt_common_entry->flags;
+ *flags = tt_common_entry->flags & ~BATADV_TT_CLIENT_ROAM;
return true;
}
@@ -3408,21 +3434,30 @@ static void _batadv_tt_update_changes(struct batadv_priv *bat_priv,
u16 tt_num_changes, u8 ttvn)
{
int roams;
+ u8 flags;
int i;
for (i = 0; i < tt_num_changes; i++) {
- if ((tt_change + i)->flags & BATADV_TT_CLIENT_DEL) {
- roams = (tt_change + i)->flags & BATADV_TT_CLIENT_ROAM;
+ flags = (tt_change + i)->flags;
+
+ if (flags & BATADV_TT_CLIENT_DEL) {
+ roams = flags & BATADV_TT_CLIENT_ROAM;
batadv_tt_global_del(bat_priv, orig_node,
(tt_change + i)->addr,
ntohs((tt_change + i)->vid),
"tt removed by changes",
roams);
} else {
+ /* an announced client cannot have roamed away from
+ * orig_node. Only a ROAMING_ADV is allowed to add a
+ * global entry with the ROAM flag
+ */
+ flags &= ~BATADV_TT_CLIENT_ROAM;
+
if (!batadv_tt_global_add(bat_priv, orig_node,
(tt_change + i)->addr,
ntohs((tt_change + i)->vid),
- (tt_change + i)->flags, ttvn))
+ flags, ttvn))
/* In case of problem while storing a
* global_entry, we stop the updating
* procedure without committing the
@@ -3883,11 +3918,25 @@ static void batadv_tt_local_purge_pending_clients(struct batadv_priv *bat_priv)
*/
static void batadv_tt_local_commit_changes_nolock(struct batadv_priv *bat_priv)
{
+ LIST_HEAD(changes_list);
+ int changes_num;
+
lockdep_assert_held(&bat_priv->tt.commit_lock);
- if (READ_ONCE(bat_priv->tt.local_changes) == 0) {
+ /* atomically get the list of changes. The TTVN decision and the
+ * announced changeset must cover the same events. Events queued
+ * afterwards stay in the tt.changes_list for the next TT round
+ */
+ spin_lock_bh(&bat_priv->tt.changes_list_lock);
+ list_splice_init(&bat_priv->tt.changes_list, &changes_list);
+ changes_num = READ_ONCE(bat_priv->tt.local_changes);
+ WRITE_ONCE(bat_priv->tt.local_changes, 0);
+ spin_unlock_bh(&bat_priv->tt.changes_list_lock);
+
+ if (changes_num == 0) {
if (!batadv_atomic_dec_not_zero(&bat_priv->tt.ogm_append_cnt))
- batadv_tt_tvlv_container_update(bat_priv);
+ batadv_tt_tvlv_container_update(bat_priv, &changes_list,
+ changes_num);
return;
}
@@ -3904,7 +3953,7 @@ static void batadv_tt_local_commit_changes_nolock(struct batadv_priv *bat_priv)
/* reset the sending counter */
atomic_set(&bat_priv->tt.ogm_append_cnt, BATADV_TT_OGM_APPEND_MAX);
- batadv_tt_tvlv_container_update(bat_priv);
+ batadv_tt_tvlv_container_update(bat_priv, &changes_list, changes_num);
}
/**
diff --git a/net/batman-adv/types.h b/net/batman-adv/types.h
index 9bdc5a3e799e..db0d4f680e65 100644
--- a/net/batman-adv/types.h
+++ b/net/batman-adv/types.h
@@ -2019,7 +2019,13 @@ struct batadv_forw_packet {
/** @direct_link_flags: direct link flags for aggregated OGM packets */
DECLARE_BITMAP(direct_link_flags, BATADV_MAX_AGGREGATION_PACKETS);
- /** @num_packets: counter for aggregated OGMv1 packets */
+ /**
+ * @num_packets: counter for aggregated OGMv1 packets. Set to
+ * BATADV_MAX_AGGREGATION_PACKETS (independent of the real number of
+ * aggregated packets) by the send worker to prevent further
+ * aggregation while the packet is emitted. It must therefore not be
+ * used to iterate over the aggregated packets
+ */
u8 num_packets;
/** @delayed_work: work queue callback item for packet sending */
diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c
index a92a81846e9d..5b99da7028b9 100644
--- a/net/bluetooth/hci_sync.c
+++ b/net/bluetooth/hci_sync.c
@@ -2687,7 +2687,7 @@ static int hci_pause_advertising_sync(struct hci_dev *hdev)
static int hci_resume_advertising_sync(struct hci_dev *hdev)
{
struct adv_info *adv, *tmp;
- int err;
+ int err = 0;
/* If advertising has not been paused there is nothing to do. */
if (!hdev->advertising_paused)
@@ -2720,13 +2720,19 @@ static int hci_resume_advertising_sync(struct hci_dev *hdev)
*/
if (hci_dev_test_and_clear_flag(hdev, HCI_LE_ADV_0))
err = hci_enable_ext_advertising_sync(hdev, 0x00);
- } else {
+ } else if (hdev->cur_adv_instance) {
/* Schedule for most recent instance to be restarted and begin
* the software rotation loop
*/
err = hci_schedule_adv_instance_sync(hdev,
hdev->cur_adv_instance,
true);
+ } else if (hci_dev_test_flag(hdev, HCI_ADVERTISING) &&
+ !hci_dev_test_flag(hdev, HCI_LE_ADV)) {
+ /* hci_schedule_adv_instance_sync() rejects instance 0x00 while
+ * HCI_ADVERTISING is set, so enable it directly.
+ */
+ err = hci_start_adv_sync(hdev, 0x00);
}
hdev->advertising_paused = false;
@@ -6272,6 +6278,7 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
u8 filter_policy = 0x00;
/* Default is to enable duplicates filter */
u8 filter_dup = LE_SCAN_FILTER_DUP_ENABLE;
+ bool paused;
int err;
bt_dev_dbg(hdev, "");
@@ -6295,6 +6302,12 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
if (err)
goto failed;
+ /* LE Set Random Address is disallowed while advertising is enabled. */
+ paused = !hdev->advertising_paused;
+ err = hci_pause_advertising_sync(hdev);
+ if (err)
+ goto failed;
+
/* All active scans will be done with either a resolvable private
* address (when privacy feature has been enabled) or non-resolvable
* private address.
@@ -6304,6 +6317,9 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
if (err < 0)
own_addr_type = ADDR_LE_DEV_PUBLIC;
+ if (paused)
+ hci_resume_advertising_sync(hdev);
+
if (hci_is_adv_monitoring(hdev) ||
(hci_test_quirk(hdev, HCI_QUIRK_STRICT_DUPLICATE_FILTER) &&
hdev->discovery.result_filtering)) {
@@ -6327,9 +6343,8 @@ static int hci_active_scan_sync(struct hci_dev *hdev, uint16_t interval)
return err;
failed:
- /* Resume advertising if it was paused */
- if (ll_privacy_capable(hdev))
- hci_resume_advertising_sync(hdev);
+ /* No-op when advertising was not paused. */
+ hci_resume_advertising_sync(hdev);
/* Resume passive scanning */
hci_update_passive_scan_sync(hdev);
diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
index b86999899f66..47f0449588ea 100644
--- a/net/bluetooth/l2cap_core.c
+++ b/net/bluetooth/l2cap_core.c
@@ -158,6 +158,28 @@ static struct l2cap_chan *__l2cap_get_chan_by_ident(struct l2cap_conn *conn,
return NULL;
}
+/* Our connection request has been sent and is still outstanding */
+static bool l2cap_chan_connect_pending(struct l2cap_chan *chan)
+{
+ return chan->state == BT_CONNECT &&
+ test_bit(CONF_CONNECT_PEND, &chan->conf_state);
+}
+
+/* Other channels may hold the same ident for a response they still have
+ * to send, so only look at channels waiting for a connection response.
+ */
+static struct l2cap_chan *__l2cap_get_connect_chan_by_ident(struct l2cap_conn *conn,
+ u8 ident)
+{
+ struct l2cap_chan *c;
+
+ list_for_each_entry(c, &conn->chan_l, list) {
+ if (c->ident == ident && l2cap_chan_connect_pending(c))
+ return c;
+ }
+ return NULL;
+}
+
static struct l2cap_chan *__l2cap_global_chan_by_addr(__le16 psm, bdaddr_t *src,
u8 src_type)
{
@@ -4298,7 +4320,7 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn,
if (!chan)
return -EBADSLT;
} else {
- chan = __l2cap_get_chan_by_ident(conn, cmd->ident);
+ chan = __l2cap_get_connect_chan_by_ident(conn, cmd->ident);
if (!chan)
return -EBADSLT;
}
@@ -4311,6 +4333,13 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn,
l2cap_chan_lock(chan);
+ /* Only accept a response matching a connection request we sent and
+ * that is still outstanding. A channel waiting in BT_CONNECT for
+ * security or the feature mask has not sent one yet.
+ */
+ if (!l2cap_chan_connect_pending(chan) || chan->ident != cmd->ident)
+ goto done;
+
switch (result) {
case L2CAP_CR_SUCCESS:
if (__l2cap_get_chan_by_dcid(conn, dcid)) {
@@ -4340,6 +4369,7 @@ static int l2cap_connect_create_rsp(struct l2cap_conn *conn,
break;
}
+done:
l2cap_chan_unlock(chan);
l2cap_chan_put(chan);
@@ -4472,7 +4502,13 @@ static inline int l2cap_config_req(struct l2cap_conn *conn,
goto unlock;
}
- if (!test_and_set_bit(CONF_REQ_SENT, &chan->conf_state)) {
+ /* A channel in BT_CONNECT2 has not passed security or userspace
+ * accept yet. Answer the peer's request, but hold our own until
+ * the channel reaches BT_CONFIG: the CONNECT2 -> CONFIG sites send
+ * it. Connected channels may also need to reconfigure.
+ */
+ if ((chan->state == BT_CONFIG || chan->state == BT_CONNECTED) &&
+ !test_and_set_bit(CONF_REQ_SENT, &chan->conf_state)) {
u8 buf[64];
l2cap_send_cmd(conn, l2cap_get_ident(conn), L2CAP_CONF_REQ,
l2cap_build_conf_req(chan, buf, sizeof(buf)), buf);
@@ -4520,6 +4556,10 @@ static inline int l2cap_config_rsp(struct l2cap_conn *conn,
if (!chan)
return 0;
+ /* Connected channels may also receive reconfiguration responses. */
+ if (chan->state != BT_CONFIG && chan->state != BT_CONNECTED)
+ goto done;
+
switch (result) {
case L2CAP_CONF_SUCCESS:
l2cap_conf_rfc_get(chan, rsp->data, len);
@@ -4587,6 +4627,9 @@ static inline int l2cap_config_rsp(struct l2cap_conn *conn,
if (test_bit(CONF_OUTPUT_DONE, &chan->conf_state)) {
set_default_fcs(chan);
+ if (chan->state != BT_CONFIG)
+ goto done;
+
if (chan->mode == L2CAP_MODE_ERTM ||
chan->mode == L2CAP_MODE_STREAMING)
err = l2cap_ertm_init(chan);
diff --git a/net/can/af_can.c b/net/can/af_can.c
index 34fe3b28d576..167b2dc40eeb 100644
--- a/net/can/af_can.c
+++ b/net/can/af_can.c
@@ -226,7 +226,7 @@ int can_send(struct sk_buff *skb, int loop)
goto inval_skb;
}
- if (unlikely(skb->dev->type != ARPHRD_CAN)) {
+ if (unlikely(!can_get_ml_priv(skb->dev))) {
err = -EPERM;
goto inval_skb;
}
@@ -452,7 +452,7 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id,
/* insert new receiver (dev,canid,mask) -> (func,data) */
- if (dev && (dev->type != ARPHRD_CAN || !can_get_ml_priv(dev)))
+ if (dev && !can_get_ml_priv(dev))
return -ENODEV;
if (dev && !net_eq(net, dev_net(dev)))
@@ -462,9 +462,10 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id,
if (!rcv)
return -ENOMEM;
- spin_lock_bh(&net->can.rcvlists_lock);
-
dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
+
+ spin_lock_bh(&dev_rcv_lists->lock);
+
rcv_list = can_rcv_list_find(&can_id, &mask, dev_rcv_lists);
rcv->can_id = can_id;
@@ -478,6 +479,9 @@ int can_rx_register(struct net *net, struct net_device *dev, canid_t can_id,
hlist_add_head_rcu(&rcv->list, rcv_list);
dev_rcv_lists->entries++;
+ spin_unlock_bh(&dev_rcv_lists->lock);
+
+ spin_lock_bh(&net->can.rcvlists_lock);
rcv_lists_stats->rcv_entries++;
rcv_lists_stats->rcv_entries_max = max(rcv_lists_stats->rcv_entries_max,
rcv_lists_stats->rcv_entries);
@@ -509,6 +513,14 @@ static void can_rx_delete_receiver(struct rcu_head *rp)
*
* Description:
* Removes subscription entry depending on given (subscription) values.
+ * The receive lists of a CAN device are protected by a per-device lock,
+ * so the subscription is found even when the device has been moved to
+ * another network namespace after can_rx_register(). The caller has to
+ * ensure that @dev stays valid, e.g. by holding a reference to @dev or
+ * RTNL. @net should be the namespace that has been used for
+ * can_rx_register(). It selects the 'any' CAN device lists for
+ * @dev == NULL and is otherwise only used for the receive list
+ * statistics, which become imprecise when another namespace is passed.
*/
void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id,
canid_t mask, void (*func)(struct sk_buff *, void *),
@@ -519,15 +531,13 @@ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id,
struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats;
struct can_dev_rcv_lists *dev_rcv_lists;
- if (dev && dev->type != ARPHRD_CAN)
+ if (dev && !can_get_ml_priv(dev))
return;
- if (dev && !net_eq(net, dev_net(dev)))
- return;
+ dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
- spin_lock_bh(&net->can.rcvlists_lock);
+ spin_lock_bh(&dev_rcv_lists->lock);
- dev_rcv_lists = can_dev_rcv_lists_find(net, dev);
rcv_list = can_rcv_list_find(&can_id, &mask, dev_rcv_lists);
/* Search the receiver list for the item to delete. This should
@@ -547,26 +557,26 @@ void can_rx_unregister(struct net *net, struct net_device *dev, canid_t can_id,
* a warning here.
*/
if (!rcv) {
+ spin_unlock_bh(&dev_rcv_lists->lock);
pr_warn("can: receive list entry not found for dev %s, id %03X, mask %03X\n",
DNAME(dev), can_id, mask);
- goto out;
+ return;
}
hlist_del_rcu(&rcv->list);
dev_rcv_lists->entries--;
+ spin_unlock_bh(&dev_rcv_lists->lock);
+
+ spin_lock_bh(&net->can.rcvlists_lock);
if (rcv_lists_stats->rcv_entries > 0)
rcv_lists_stats->rcv_entries--;
-
- out:
spin_unlock_bh(&net->can.rcvlists_lock);
/* schedule the receiver item for deletion */
- if (rcv) {
- if (rcv->sk)
- sock_hold(rcv->sk);
- call_rcu(&rcv->rcu, can_rx_delete_receiver);
- }
+ if (rcv->sk)
+ sock_hold(rcv->sk);
+ call_rcu(&rcv->rcu, can_rx_delete_receiver);
}
EXPORT_SYMBOL(can_rx_unregister);
@@ -684,8 +694,7 @@ static int can_rcv(struct sk_buff *skb, struct net_device *dev,
{
struct can_skb_ext *csx = can_skb_ext_find(skb);
- if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) ||
- !csx || !can_is_can_skb(skb))) {
+ if (unlikely(!can_get_ml_priv(dev) || !csx || !can_is_can_skb(skb))) {
pr_warn_once("PF_CAN: dropped non conform CAN skbuff: dev type %d, len %d\n",
dev->type, skb->len);
@@ -710,8 +719,7 @@ static int canfd_rcv(struct sk_buff *skb, struct net_device *dev,
{
struct can_skb_ext *csx = can_skb_ext_find(skb);
- if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) ||
- !csx || !can_is_canfd_skb(skb))) {
+ if (unlikely(!can_get_ml_priv(dev) || !csx || !can_is_canfd_skb(skb))) {
pr_warn_once("PF_CAN: dropped non conform CAN FD skbuff: dev type %d, len %d\n",
dev->type, skb->len);
@@ -736,8 +744,7 @@ static int canxl_rcv(struct sk_buff *skb, struct net_device *dev,
{
struct can_skb_ext *csx = can_skb_ext_find(skb);
- if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) ||
- !csx || !can_is_canxl_skb(skb))) {
+ if (unlikely(!can_get_ml_priv(dev) || !csx || !can_is_canxl_skb(skb))) {
pr_warn_once("PF_CAN: dropped non conform CAN XL skbuff: dev type %d, len %d\n",
dev->type, skb->len);
@@ -826,6 +833,7 @@ static int can_pernet_init(struct net *net)
net->can.rx_alldev_list = kzalloc_obj(*net->can.rx_alldev_list);
if (!net->can.rx_alldev_list)
goto out;
+ spin_lock_init(&net->can.rx_alldev_list->lock);
net->can.pkg_stats = kzalloc_obj(*net->can.pkg_stats);
if (!net->can.pkg_stats)
goto out_free_rx_alldev_list;
diff --git a/net/can/bcm.c b/net/can/bcm.c
index 3d637a1e0ac1..9288b96724da 100644
--- a/net/can/bcm.c
+++ b/net/can/bcm.c
@@ -54,6 +54,7 @@
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include <linux/can/skb.h>
#include <linux/can/bcm.h>
@@ -963,7 +964,7 @@ static void bcm_remove_op(struct bcm_op *op)
static void bcm_rx_unreg(struct net_device *dev, struct bcm_op *op)
{
if (op->rx_reg_dev == dev) {
- can_rx_unregister(dev_net(dev), dev, op->can_id,
+ can_rx_unregister(sock_net(op->sk), dev, op->can_id,
REGMASK(op->can_id), bcm_rx_handler, op);
/* mark as removed subscription */
@@ -989,21 +990,10 @@ static int bcm_delete_rx_op(struct list_head *ops, struct bcm_msg_head *mh,
/* disable automatic timer on frame reception */
op->flags |= RX_NO_AUTOTIMER;
- /*
- * Don't care if we're bound or not (due to netdev
- * problems) can_rx_unregister() is always a save
- * thing to do here.
- */
if (op->ifindex) {
- /*
- * Only remove subscriptions that had not
- * been removed due to NETDEV_UNREGISTER
- * in bcm_notifier()
- *
- * op->rx_reg_dev is a tracked reference taken
- * when the subscription was registered, so it
- * stays valid here even if a concurrent
- * NETDEV_UNREGISTER already unlisted the dev.
+ /* Only remove subscriptions that had not been
+ * removed in bcm_notify(). op->rx_reg_dev keeps
+ * the device's receive lists accessible.
*/
if (op->rx_reg_dev)
bcm_rx_unreg(op->rx_reg_dev, op);
@@ -1719,7 +1709,7 @@ static int bcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
goto out_release;
}
- if (dev->type != ARPHRD_CAN) {
+ if (!can_get_ml_priv(dev)) {
dev_put(dev);
ret = -ENODEV;
goto out_release;
@@ -1867,7 +1857,7 @@ static int bcm_notifier(struct notifier_block *nb, unsigned long msg,
{
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN)
return NOTIFY_DONE;
@@ -1935,6 +1925,7 @@ static int bcm_release(struct socket *sock)
list_del(&bo->notifier);
spin_unlock(&bcm_notifier_lock);
+ /* bcm_notify() can not access this socket anymore */
lock_sock(sk);
#if IS_ENABLED(CONFIG_PROC_FS)
@@ -1948,29 +1939,20 @@ static int bcm_release(struct socket *sock)
bcm_remove_op(op);
}
- list_for_each_entry_safe(op, next, &bo->rx_ops, list) {
- /*
- * Don't care if we're bound or not (due to netdev problems)
- * can_rx_unregister() is always a save thing to do here.
- */
+ list_for_each_entry(op, &bo->rx_ops, list) {
if (op->ifindex) {
- /*
- * Only remove subscriptions that had not
- * been removed due to NETDEV_UNREGISTER
- * in bcm_notifier()
- *
- * op->rx_reg_dev is a tracked reference taken
- * when the subscription was registered, so it
- * stays valid here even if a concurrent
- * NETDEV_UNREGISTER already unlisted the device.
+ /* Only remove subscriptions that had not been removed
+ * in bcm_notify(). op->rx_reg_dev keeps the device's
+ * receive lists accessible.
*/
if (op->rx_reg_dev)
bcm_rx_unreg(op->rx_reg_dev, op);
- } else
+ } else {
can_rx_unregister(net, NULL, op->can_id,
REGMASK(op->can_id),
bcm_rx_handler, op);
-
+ }
+ cond_resched();
}
synchronize_rcu();
@@ -2024,7 +2006,7 @@ static int bcm_connect(struct socket *sock, struct sockaddr_unsized *uaddr, int
ret = -ENODEV;
goto fail;
}
- if (dev->type != ARPHRD_CAN) {
+ if (!can_get_ml_priv(dev)) {
dev_put(dev);
ret = -ENODEV;
goto fail;
diff --git a/net/can/gw.c b/net/can/gw.c
index 5793cb2420ed..54bb5bd3242a 100644
--- a/net/can/gw.c
+++ b/net/can/gw.c
@@ -52,6 +52,7 @@
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include <linux/can/skb.h>
#include <linux/can/gw.h>
@@ -612,7 +613,7 @@ static int cgw_notifier(struct notifier_block *nb,
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
struct net *net = dev_net(dev);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg == NETDEV_UNREGISTER) {
@@ -1163,7 +1164,7 @@ static int cgw_create_job(struct sk_buff *skb, struct nlmsghdr *nlh,
if (!gwj->src.dev)
goto out;
- if (gwj->src.dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(gwj->src.dev))
goto out;
gwj->dst.dev = __dev_get_by_index(net, gwj->ccgw.dst_idx);
@@ -1171,7 +1172,7 @@ static int cgw_create_job(struct sk_buff *skb, struct nlmsghdr *nlh,
if (!gwj->dst.dev)
goto out;
- if (gwj->dst.dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(gwj->dst.dev))
goto out;
/* is sending the skb back to the incoming interface intended? */
diff --git a/net/can/isotp.c b/net/can/isotp.c
index f5dc9d04bd68..5e2de051fc5e 100644
--- a/net/can/isotp.c
+++ b/net/can/isotp.c
@@ -65,6 +65,7 @@
#include <linux/if_arp.h>
#include <linux/skbuff.h>
#include <linux/can.h>
+#include <linux/can/can-ml.h>
#include <linux/can/core.h>
#include <linux/can/skb.h>
#include <linux/can/isotp.h>
@@ -755,6 +756,14 @@ static void isotp_rcv(struct sk_buff *skb, void *data)
if (skb->len != so->ll.mtu)
return;
+ /* check for correct CAN CC/FD frame content */
+ if (so->ll.mtu == CAN_MTU) {
+ if (!can_is_can_skb(skb))
+ return;
+ } else if (!can_is_canfd_skb(skb)) {
+ return;
+ }
+
cf = (struct canfd_frame *)skb->data;
/* if enabled: check reception of my configured extended address */
@@ -1123,7 +1132,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
int off;
int err;
- if (!so->bound || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN)
+ if (!READ_ONCE(so->bound) || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN)
return -EADDRNOTAVAIL;
/* claim the socket under so->rx_lock: this serializes the claim
@@ -1168,7 +1177,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size)
/* so->bound is only checked once above - a wakeup may have
* unbound/rebound the socket meanwhile => recheck
*/
- if (!so->bound) {
+ if (!READ_ONCE(so->bound)) {
err = -EADDRNOTAVAIL;
goto err_out_drop;
}
@@ -1414,7 +1423,7 @@ static int isotp_recvmsg(struct socket *sock, struct msghdr *msg, size_t size,
if (flags & ~(MSG_DONTWAIT | MSG_TRUNC | MSG_PEEK | MSG_CMSG_COMPAT))
return -EINVAL;
- if (!so->bound)
+ if (!READ_ONCE(so->bound))
return -EADDRNOTAVAIL;
skb = skb_recv_datagram(sk, flags, &ret);
@@ -1488,11 +1497,11 @@ static int isotp_release(struct socket *sock)
list_del(&so->notifier);
spin_unlock(&isotp_notifier_lock);
- rtnl_lock();
lock_sock(sk);
/* remove current filters & unregister
- * tracked reference so->dev is taken at bind() time with rtnl_lock
+ * the tracked reference so->dev keeps the device's receive lists
+ * accessible, even after a namespace change
*/
if (so->bound && so->dev) {
if (isotp_register_rxid(so))
@@ -1507,11 +1516,9 @@ static int isotp_release(struct socket *sock)
}
so->ifindex = 0;
- so->bound = 0;
+ WRITE_ONCE(so->bound, 0);
so->dev = NULL;
- rtnl_unlock();
-
/* Always wait for a grace period before touching the timers below.
* A concurrent NETDEV_UNREGISTER may have already unregistered our
* filters and cleared so->bound in isotp_notify() without waiting
@@ -1541,7 +1548,6 @@ static int isotp_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int l
struct sock *sk = sock->sk;
struct isotp_sock *so = isotp_sk(sk);
struct net *net = sock_net(sk);
- int ifindex;
struct net_device *dev;
canid_t tx_id = addr->can_addr.tp.tx_id;
canid_t rx_id = addr->can_addr.tp.rx_id;
@@ -1579,7 +1585,7 @@ static int isotp_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int l
if (!addr->can_ifindex)
return -ENODEV;
- rtnl_lock();
+ /* lock_sock() serializes the binding with isotp_notify() */
lock_sock(sk);
if (so->bound) {
@@ -1605,61 +1611,83 @@ static int isotp_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int l
goto out;
}
- dev = dev_get_by_index(net, addr->can_ifindex);
- if (!dev) {
- err = -ENODEV;
- goto out;
- }
- if (dev->type != ARPHRD_CAN) {
+ /* Publish the device inside the RCU section of the lookup: When it
+ * is unlisted concurrently, either isotp_notify() sees so->dev after
+ * synchronize_net() or the lookup fails. Check the device before
+ * publishing it.
+ */
+ rcu_read_lock();
+ dev = dev_get_by_index_rcu(net, addr->can_ifindex);
+ if (!dev || !can_get_ml_priv(dev))
err = -ENODEV;
- goto out_put_dev;
- }
- if (READ_ONCE(dev->mtu) < so->ll.mtu) {
+ else if (READ_ONCE(dev->mtu) < so->ll.mtu)
err = -EINVAL;
- goto out_put_dev;
+ if (!err) {
+ dev_hold(dev);
+ WRITE_ONCE(so->dev, dev);
}
- if (!(dev->flags & IFF_UP))
- notify_enetdown = 1;
-
- ifindex = dev->ifindex;
+ rcu_read_unlock();
+ if (err)
+ goto out;
- if (isotp_register_rxid(so))
- can_rx_register(net, dev, rx_id, SINGLE_MASK(rx_id),
- isotp_rcv, sk, "isotp", sk);
+ /* switch to new settings */
+ so->ifindex = dev->ifindex;
+ so->rxid = rx_id;
+ so->txid = tx_id;
/* no consecutive frame echo skb in flight */
WRITE_ONCE(so->cfecho, 0);
- /* register for echo skb's */
- can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id),
- isotp_rcv_echo, sk, "isotpe", sk);
+ /* register for echo skb's first (no echo traffic at this point) */
+ err = can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id),
+ isotp_rcv_echo, sk, "isotpe", sk);
+ if (err)
+ goto out_put_dev;
- /* switch to new settings */
- so->ifindex = ifindex;
- so->rxid = rx_id;
- so->txid = tx_id;
- so->bound = 1;
+ if (isotp_register_rxid(so)) {
+ err = can_rx_register(net, dev, rx_id, SINGLE_MASK(rx_id),
+ isotp_rcv, sk, "isotp", sk);
+ if (err) {
+ /* unregister echo skb subscription */
+ can_rx_unregister(net, dev, tx_id, SINGLE_MASK(tx_id),
+ isotp_rcv_echo, sk);
+ goto out_put_dev;
+ }
+ }
+
+ WRITE_ONCE(so->bound, 1);
/* bind() ok -> hold a reference for so->dev so that isotp_release()
* can safely reach the device later, even if a concurrent
* NETDEV_UNREGISTER has already unlisted it by ifindex.
*/
- so->dev = dev;
netdev_hold(so->dev, &so->dev_tracker, GFP_KERNEL);
+ /* Either isotp_notify() sees so->dev on NETDEV_DOWN or we see the
+ * cleared IFF_UP, pairs with isotp_notify()
+ */
+ smp_mb();
+ if (!(READ_ONCE(dev->flags) & IFF_UP))
+ notify_enetdown = 1;
+
out_put_dev:
- /* remove potential reference from dev_get_by_index() */
+ /* revert so->ifindex and so->dev on failed bind() */
+ if (err) {
+ so->ifindex = 0;
+ WRITE_ONCE(so->dev, NULL);
+ }
+
+ /* remove reference from the device lookup */
dev_put(dev);
out:
- release_sock(sk);
- rtnl_unlock();
-
if (notify_enetdown) {
sk->sk_err = ENETDOWN;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
}
+ release_sock(sk);
+
return err;
}
@@ -1681,8 +1709,8 @@ static int isotp_getname(struct socket *sock, struct sockaddr *uaddr, int peer)
return ISOTP_MIN_NAMELEN;
}
-static int isotp_setsockopt_locked(struct socket *sock, int level, int optname,
- sockptr_t optval, unsigned int optlen)
+static int isotp_setsockopt_locked(struct socket *sock, int optname,
+ const void *val)
{
struct sock *sk = sock->sk;
struct isotp_sock *so = isotp_sk(sk);
@@ -1693,11 +1721,7 @@ static int isotp_setsockopt_locked(struct socket *sock, int level, int optname,
switch (optname) {
case CAN_ISOTP_OPTS:
- if (optlen != sizeof(struct can_isotp_options))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->opt, optval, optlen))
- return -EFAULT;
+ memcpy(&so->opt, val, sizeof(so->opt));
/* no separate rx_ext_address is given => use ext_address */
if (!(so->opt.flags & CAN_ISOTP_RX_EXT_ADDR))
@@ -1722,55 +1746,37 @@ static int isotp_setsockopt_locked(struct socket *sock, int level, int optname,
break;
case CAN_ISOTP_RECV_FC:
- if (optlen != sizeof(struct can_isotp_fc_options))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->rxfc, optval, optlen))
- return -EFAULT;
+ memcpy(&so->rxfc, val, sizeof(so->rxfc));
break;
case CAN_ISOTP_TX_STMIN:
- if (optlen != sizeof(u32))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->force_tx_stmin, optval, optlen))
- return -EFAULT;
+ so->force_tx_stmin = *(const u32 *)val;
break;
case CAN_ISOTP_RX_STMIN:
- if (optlen != sizeof(u32))
- return -EINVAL;
-
- if (copy_from_sockptr(&so->force_rx_stmin, optval, optlen))
- return -EFAULT;
+ so->force_rx_stmin = *(const u32 *)val;
break;
- case CAN_ISOTP_LL_OPTS:
- if (optlen == sizeof(struct can_isotp_ll_options)) {
- struct can_isotp_ll_options ll;
-
- if (copy_from_sockptr(&ll, optval, optlen))
- return -EFAULT;
+ case CAN_ISOTP_LL_OPTS: {
+ const struct can_isotp_ll_options *ll = val;
- /* check for correct ISO 11898-1 DLC data length */
- if (ll.tx_dl != padlen(ll.tx_dl))
- return -EINVAL;
+ /* check for correct ISO 11898-1 DLC data length */
+ if (ll->tx_dl != padlen(ll->tx_dl))
+ return -EINVAL;
- if (ll.mtu != CAN_MTU && ll.mtu != CANFD_MTU)
- return -EINVAL;
+ if (ll->mtu != CAN_MTU && ll->mtu != CANFD_MTU)
+ return -EINVAL;
- if (ll.mtu == CAN_MTU &&
- (ll.tx_dl > CAN_MAX_DLEN || ll.tx_flags != 0))
- return -EINVAL;
+ if (ll->mtu == CAN_MTU &&
+ (ll->tx_dl > CAN_MAX_DLEN || ll->tx_flags != 0))
+ return -EINVAL;
- memcpy(&so->ll, &ll, sizeof(ll));
+ memcpy(&so->ll, ll, sizeof(*ll));
- /* set ll_dl for tx path to similar place as for rx */
- so->tx.ll_dl = ll.tx_dl;
- } else {
- return -EINVAL;
- }
+ /* set ll_dl for tx path to similar place as for rx */
+ so->tx.ll_dl = ll->tx_dl;
break;
+ }
default:
ret = -ENOPROTOOPT;
@@ -1784,13 +1790,49 @@ static int isotp_setsockopt(struct socket *sock, int level, int optname,
{
struct sock *sk = sock->sk;
+ union {
+ struct can_isotp_options opt;
+ struct can_isotp_fc_options rxfc;
+ struct can_isotp_ll_options ll;
+ u32 stmin;
+ } val;
+ unsigned int size;
int ret;
if (level != SOL_CAN_ISOTP)
return -EINVAL;
+ /* copy the option values from user space before lock_sock() */
+ switch (optname) {
+ case CAN_ISOTP_OPTS:
+ size = sizeof(val.opt);
+ break;
+
+ case CAN_ISOTP_RECV_FC:
+ size = sizeof(val.rxfc);
+ break;
+
+ case CAN_ISOTP_TX_STMIN:
+ case CAN_ISOTP_RX_STMIN:
+ size = sizeof(val.stmin);
+ break;
+
+ case CAN_ISOTP_LL_OPTS:
+ size = sizeof(val.ll);
+ break;
+
+ default:
+ return -ENOPROTOOPT;
+ }
+
+ if (optlen != size)
+ return -EINVAL;
+
+ if (copy_from_sockptr(&val, optval, size))
+ return -EFAULT;
+
lock_sock(sk);
- ret = isotp_setsockopt_locked(sock, level, optname, optval, optlen);
+ ret = isotp_setsockopt_locked(sock, optname, &val);
release_sock(sk);
return ret;
}
@@ -1855,12 +1897,21 @@ static void isotp_notify(struct isotp_sock *so, unsigned long msg,
if (!net_eq(dev_net(dev), sock_net(sk)))
return;
- if (so->dev != dev)
+ if (msg == NETDEV_DOWN)
+ smp_mb(); /* pairs with isotp_bind(), see raw_notify() */
+
+ /* lockless fast path, see raw_notify() */
+ if (READ_ONCE(so->dev) != dev)
return;
switch (msg) {
case NETDEV_UNREGISTER:
lock_sock(sk);
+ if (so->dev != dev) {
+ release_sock(sk);
+ break;
+ }
+
/* remove current filters & unregister */
if (so->bound) {
if (isotp_register_rxid(so))
@@ -1875,16 +1926,19 @@ static void isotp_notify(struct isotp_sock *so, unsigned long msg,
}
so->ifindex = 0;
- so->bound = 0;
- so->dev = NULL;
- release_sock(sk);
+ WRITE_ONCE(so->bound, 0);
+ WRITE_ONCE(so->dev, NULL);
sk->sk_err = ENODEV;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
+ release_sock(sk);
break;
case NETDEV_DOWN:
+ if (!READ_ONCE(so->bound))
+ break;
+
sk->sk_err = ENETDOWN;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
@@ -1897,7 +1951,7 @@ static int isotp_notifier(struct notifier_block *nb, unsigned long msg,
{
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN)
return NOTIFY_DONE;
diff --git a/net/can/proc.c b/net/can/proc.c
index de4d05ae3459..fb144d2adc35 100644
--- a/net/can/proc.c
+++ b/net/can/proc.c
@@ -76,16 +76,39 @@ static const char rx_list_name[][8] = {
* af_can statistics stuff
*/
+static void can_reset_pkg_stats(struct can_pkg_stats *pkg_stats)
+{
+ atomic_long_set(&pkg_stats->rx_frames, 0);
+ atomic_long_set(&pkg_stats->tx_frames, 0);
+ atomic_long_set(&pkg_stats->matches, 0);
+
+ pkg_stats->total_rx_rate = 0;
+ pkg_stats->total_tx_rate = 0;
+ pkg_stats->total_rx_match_ratio = 0;
+
+ pkg_stats->current_rx_rate = 0;
+ pkg_stats->current_tx_rate = 0;
+ pkg_stats->current_rx_match_ratio = 0;
+
+ pkg_stats->max_rx_rate = 0;
+ pkg_stats->max_tx_rate = 0;
+ pkg_stats->max_rx_match_ratio = 0;
+
+ atomic_long_set(&pkg_stats->rx_frames_delta, 0);
+ atomic_long_set(&pkg_stats->tx_frames_delta, 0);
+ atomic_long_set(&pkg_stats->matches_delta, 0);
+}
+
static void can_init_stats(struct net *net)
{
struct can_pkg_stats *pkg_stats = net->can.pkg_stats;
struct can_rcv_lists_stats *rcv_lists_stats = net->can.rcv_lists_stats;
/*
- * This memset function is called from a timer context (when
+ * This stats reset is called from a timer context (when
* can_stattimer is active which is the default) OR in a process
* context (reading the proc_fs when can_stattimer is disabled).
*/
- memset(pkg_stats, 0, sizeof(struct can_pkg_stats));
+ can_reset_pkg_stats(pkg_stats);
pkg_stats->jiffies_init = jiffies;
rcv_lists_stats->stats_reset++;
@@ -121,29 +144,23 @@ void can_stat_update(struct timer_list *t)
long rx_frames = atomic_long_read(&pkg_stats->rx_frames);
long tx_frames = atomic_long_read(&pkg_stats->tx_frames);
long matches = atomic_long_read(&pkg_stats->matches);
- long rx_frames_delta = atomic_long_read(&pkg_stats->rx_frames_delta);
- long tx_frames_delta = atomic_long_read(&pkg_stats->tx_frames_delta);
- long matches_delta = atomic_long_read(&pkg_stats->matches_delta);
+ /* fetch and clear values for 'current rate' calculation */
+ long rx_frames_delta = atomic_long_xchg(&pkg_stats->rx_frames_delta, 0);
+ long tx_frames_delta = atomic_long_xchg(&pkg_stats->tx_frames_delta, 0);
+ long matches_delta = atomic_long_xchg(&pkg_stats->matches_delta, 0);
- /* restart counting in timer context on user request */
- if (user_reset)
- can_init_stats(net);
-
- /* restart counting on jiffies overflow */
- if (j < pkg_stats->jiffies_init)
- can_init_stats(net);
-
- /* prevent overflow in calc_rate() */
- if (rx_frames > (LONG_MAX / HZ))
- can_init_stats(net);
-
- /* prevent overflow in calc_rate() */
- if (tx_frames > (LONG_MAX / HZ))
- can_init_stats(net);
-
- /* matches overflow - very improbable */
- if (matches > (LONG_MAX / 100))
+ /*
+ * Restart counting in timer context on user request, on jiffies
+ * overflow or to prevent overflows in calc_rate() and in the
+ * match ratio calculation. The local snapshots are stale after
+ * the reset, so skip the calculation for this interval.
+ */
+ if (user_reset || j < pkg_stats->jiffies_init ||
+ rx_frames > (LONG_MAX / HZ) || tx_frames > (LONG_MAX / HZ) ||
+ matches > (LONG_MAX / 100)) {
can_init_stats(net);
+ goto out;
+ }
/* calc total values */
if (rx_frames)
@@ -172,11 +189,7 @@ void can_stat_update(struct timer_list *t)
if (pkg_stats->max_rx_match_ratio < pkg_stats->current_rx_match_ratio)
pkg_stats->max_rx_match_ratio = pkg_stats->current_rx_match_ratio;
- /* clear values for 'current rate' calculation */
- atomic_long_set(&pkg_stats->tx_frames_delta, 0);
- atomic_long_set(&pkg_stats->rx_frames_delta, 0);
- atomic_long_set(&pkg_stats->matches_delta, 0);
-
+out:
/* restart timer (one second) */
mod_timer(&net->can.stattimer, round_jiffies(jiffies + HZ));
}
diff --git a/net/can/raw.c b/net/can/raw.c
index 0acd4f6c6dd6..0926d5686e41 100644
--- a/net/can/raw.c
+++ b/net/can/raw.c
@@ -85,6 +85,7 @@ struct raw_sock {
struct sock sk;
struct net_device *dev;
netdevice_tracker dev_tracker;
+ struct net_device *bind_dev; /* hint for raw_notify(), no reference */
struct list_head notifier;
int ifindex;
unsigned int bound:1;
@@ -224,6 +225,7 @@ static int raw_enable_filters(struct net *net, struct net_device *dev,
int i;
for (i = 0; i < count; i++) {
+ cond_resched();
err = can_rx_register(net, dev, filter[i].can_id,
filter[i].can_mask,
raw_rcv, sk, "raw", sk);
@@ -258,9 +260,11 @@ static void raw_disable_filters(struct net *net, struct net_device *dev,
{
int i;
- for (i = 0; i < count; i++)
+ for (i = 0; i < count; i++) {
can_rx_unregister(net, dev, filter[i].can_id,
filter[i].can_mask, raw_rcv, sk);
+ cond_resched();
+ }
}
static inline void raw_disable_errfilter(struct net *net,
@@ -305,16 +309,31 @@ static void raw_notify(struct raw_sock *ro, unsigned long msg,
struct net_device *dev)
{
struct sock *sk = &ro->sk;
+ struct net_device *bind_dev;
if (!net_eq(dev_net(dev), sock_net(sk)))
return;
- if (ro->dev != dev)
+ if (msg == NETDEV_DOWN)
+ smp_mb(); /* IFF_UP is cleared before, pairs with raw_bind() */
+
+ /* Lockless fast path, ro->bind_dev covers a concurrent bind().
+ * Read it before ro->dev, pairs with smp_store_release() in
+ * raw_bind(). A stale ro->bind_dev only costs a lock_sock().
+ */
+ bind_dev = smp_load_acquire(&ro->bind_dev);
+ if (READ_ONCE(ro->dev) != dev &&
+ (msg != NETDEV_UNREGISTER || bind_dev != dev))
return;
switch (msg) {
case NETDEV_UNREGISTER:
lock_sock(sk);
+ if (ro->dev != dev) {
+ release_sock(sk);
+ break;
+ }
+
/* remove current filters & unregister */
if (ro->bound) {
raw_disable_allfilters(dev_net(dev), dev, sk);
@@ -326,13 +345,13 @@ static void raw_notify(struct raw_sock *ro, unsigned long msg,
ro->ifindex = 0;
ro->bound = 0;
- ro->dev = NULL;
+ WRITE_ONCE(ro->dev, NULL);
ro->count = 0;
- release_sock(sk);
sk->sk_err = ENODEV;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
+ release_sock(sk);
break;
case NETDEV_DOWN:
@@ -348,7 +367,7 @@ static int raw_notifier(struct notifier_block *nb, unsigned long msg,
{
struct net_device *dev = netdev_notifier_info_to_dev(ptr);
- if (dev->type != ARPHRD_CAN)
+ if (!can_get_ml_priv(dev))
return NOTIFY_DONE;
if (msg != NETDEV_UNREGISTER && msg != NETDEV_DOWN)
return NOTIFY_DONE;
@@ -381,6 +400,7 @@ static int raw_init(struct sock *sk)
ro->bound = 0;
ro->ifindex = 0;
ro->dev = NULL;
+ ro->bind_dev = NULL;
/* set default filter to single entry dfilter */
ro->dfilter.can_id = 0;
@@ -431,13 +451,15 @@ static int raw_release(struct socket *sock)
list_del(&ro->notifier);
spin_unlock(&raw_notifier_lock);
- rtnl_lock();
lock_sock(sk);
- /* remove current filters & unregister */
+ /* remove current filters & unregister
+ * the tracked reference ro->dev keeps the device's receive lists
+ * accessible, even after a namespace change
+ */
if (ro->bound) {
if (ro->dev) {
- raw_disable_allfilters(dev_net(ro->dev), ro->dev, sk);
+ raw_disable_allfilters(net, ro->dev, sk);
netdev_put(ro->dev, &ro->dev_tracker);
} else {
raw_disable_allfilters(net, NULL, sk);
@@ -456,7 +478,6 @@ static int raw_release(struct socket *sock)
sock->sk = NULL;
release_sock(sk);
- rtnl_unlock();
sock_prot_inuse_add(net, sk->sk_prot, -1);
sock_put(sk);
@@ -479,26 +500,34 @@ static int raw_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int len
if (addr->can_family != AF_CAN)
return -EINVAL;
- rtnl_lock();
+ /* lock_sock() serializes the binding with raw_notify() */
lock_sock(sk);
if (ro->bound && addr->can_ifindex == ro->ifindex)
goto out;
if (addr->can_ifindex) {
- dev = dev_get_by_index(sock_net(sk), addr->can_ifindex);
+ /* Publish the device inside the RCU section of the lookup:
+ * When it is unlisted concurrently, either raw_notify() sees
+ * ro->bind_dev after synchronize_net() or the lookup fails.
+ */
+ rcu_read_lock();
+ dev = dev_get_by_index_rcu(sock_net(sk), addr->can_ifindex);
+ if (dev) {
+ dev_hold(dev);
+ /* pairs with smp_load_acquire() in raw_notify() */
+ smp_store_release(&ro->bind_dev, dev);
+ }
+ rcu_read_unlock();
if (!dev) {
err = -ENODEV;
goto out;
}
- if (dev->type != ARPHRD_CAN) {
+ if (!can_get_ml_priv(dev)) {
err = -ENODEV;
goto out_put_dev;
}
- if (!(dev->flags & IFF_UP))
- notify_enetdown = 1;
-
ifindex = dev->ifindex;
/* filters set by default/setsockopt */
@@ -517,7 +546,7 @@ static int raw_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int len
if (ro->bound) {
/* unregister old filters */
if (ro->dev) {
- raw_disable_allfilters(dev_net(ro->dev),
+ raw_disable_allfilters(sock_net(sk),
ro->dev, sk);
/* drop reference to old ro->dev */
netdev_put(ro->dev, &ro->dev_tracker);
@@ -528,24 +557,32 @@ static int raw_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int len
ro->ifindex = ifindex;
ro->bound = 1;
/* bind() ok -> hold a reference for new ro->dev */
- ro->dev = dev;
+ WRITE_ONCE(ro->dev, dev);
if (ro->dev)
netdev_hold(ro->dev, &ro->dev_tracker, GFP_KERNEL);
+
+ if (dev) {
+ /* Either raw_notify() sees ro->dev on NETDEV_DOWN or
+ * we see the cleared IFF_UP, pairs with raw_notify()
+ */
+ smp_mb();
+ if (!(READ_ONCE(dev->flags) & IFF_UP))
+ notify_enetdown = 1;
+ }
}
out_put_dev:
- /* remove potential reference from dev_get_by_index() */
+ /* remove reference from the device lookup */
dev_put(dev);
out:
- release_sock(sk);
- rtnl_unlock();
-
if (notify_enetdown) {
sk->sk_err = ENETDOWN;
if (!sock_flag(sk, SOCK_DEAD))
sk_error_report(sk);
}
+ release_sock(sk);
+
return err;
}
@@ -567,58 +604,36 @@ static int raw_getname(struct socket *sock, struct sockaddr *uaddr,
}
static int raw_setsockopt_locked(struct socket *sock, int optname,
- sockptr_t optval, unsigned int optlen)
+ const void *val, struct can_filter *filter,
+ int count)
{
struct sock *sk = sock->sk;
struct raw_sock *ro = raw_sk(sk);
- struct can_filter *filter = NULL; /* dyn. alloc'ed filters */
- struct can_filter sfilter; /* single filter */
struct net_device *dev = NULL;
- can_err_mask_t err_mask = 0;
- int count = 0;
+ can_err_mask_t err_mask;
int flag;
int err = 0;
switch (optname) {
case CAN_RAW_FILTER:
- if (optlen % sizeof(struct can_filter) != 0)
- return -EINVAL;
-
- if (optlen > CAN_RAW_FILTER_MAX * sizeof(struct can_filter))
- return -EINVAL;
-
- count = optlen / sizeof(struct can_filter);
-
- if (count > 1) {
- /* filter does not fit into dfilter => alloc space */
- filter = memdup_sockptr(optval, optlen);
- if (IS_ERR(filter))
- return PTR_ERR(filter);
- } else if (count == 1) {
- if (copy_from_sockptr(&sfilter, optval, sizeof(sfilter)))
- return -EFAULT;
- }
-
+ /* @filter has been copied by raw_setsockopt(). It is linked
+ * to the socket on success and freed by the caller on error.
+ *
+ * The reg_state check is only a shortcut: On a concurrent
+ * unregistration or namespace change raw_notify() waits for
+ * lock_sock() and removes the filters set here.
+ */
dev = ro->dev;
- if (ro->bound && dev && dev->reg_state != NETREG_REGISTERED) {
- if (count > 1)
- kfree(filter);
+ if (ro->bound && dev &&
+ READ_ONCE(dev->reg_state) != NETREG_REGISTERED)
return -ENODEV;
- }
if (ro->bound) {
/* (try to) register the new filters */
- if (count == 1)
- err = raw_enable_filters(sock_net(sk), dev, sk,
- &sfilter, 1);
- else
- err = raw_enable_filters(sock_net(sk), dev, sk,
- filter, count);
- if (err) {
- if (count > 1)
- kfree(filter);
+ err = raw_enable_filters(sock_net(sk), dev, sk, filter,
+ count);
+ if (err)
return err;
- }
/* remove old filter registrations */
raw_disable_filters(sock_net(sk), dev, sk, ro->filter,
@@ -632,7 +647,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
/* link new filters to the socket */
if (count == 1) {
/* copy filter data for single filter */
- ro->dfilter = sfilter;
+ ro->dfilter = *filter;
filter = &ro->dfilter;
}
ro->filter = filter;
@@ -640,16 +655,12 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_ERR_FILTER:
- if (optlen != sizeof(err_mask))
- return -EINVAL;
-
- if (copy_from_sockptr(&err_mask, optval, optlen))
- return -EFAULT;
-
- err_mask &= CAN_ERR_MASK;
+ err_mask = *(const can_err_mask_t *)val & CAN_ERR_MASK;
+ /* lockless shortcut, see CAN_RAW_FILTER */
dev = ro->dev;
- if (ro->bound && dev && dev->reg_state != NETREG_REGISTERED)
+ if (ro->bound && dev &&
+ READ_ONCE(dev->reg_state) != NETREG_REGISTERED)
return -ENODEV;
/* remove current error mask */
@@ -671,31 +682,15 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_LOOPBACK:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
-
- ro->loopback = !!flag;
+ ro->loopback = !!*(const int *)val;
break;
case CAN_RAW_RECV_OWN_MSGS:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
-
- ro->recv_own_msgs = !!flag;
+ ro->recv_own_msgs = !!*(const int *)val;
break;
case CAN_RAW_FD_FRAMES:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
+ flag = *(const int *)val;
/* Enabling CAN XL includes CAN FD */
if (ro->xl_frames && !flag)
@@ -705,13 +700,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_XL_FRAMES:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
-
- ro->xl_frames = !!flag;
+ ro->xl_frames = !!*(const int *)val;
/* Enabling CAN XL includes CAN FD */
if (ro->xl_frames)
@@ -719,11 +708,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_XL_VCID_OPTS:
- if (optlen != sizeof(ro->raw_vcid_opts))
- return -EINVAL;
-
- if (copy_from_sockptr(&ro->raw_vcid_opts, optval, optlen))
- return -EFAULT;
+ memcpy(&ro->raw_vcid_opts, val, sizeof(ro->raw_vcid_opts));
/* prepare 32 bit values for handling in hot path */
ro->tx_vcid_shifted = ro->raw_vcid_opts.tx_vcid << CANXL_VCID_OFFSET;
@@ -732,13 +717,7 @@ static int raw_setsockopt_locked(struct socket *sock, int optname,
break;
case CAN_RAW_JOIN_FILTERS:
- if (optlen != sizeof(flag))
- return -EINVAL;
-
- if (copy_from_sockptr(&flag, optval, optlen))
- return -EFAULT;
-
- ro->join_filters = !!flag;
+ ro->join_filters = !!*(const int *)val;
break;
default:
@@ -751,18 +730,83 @@ static int raw_setsockopt(struct socket *sock, int level, int optname,
sockptr_t optval, unsigned int optlen)
{
struct sock *sk = sock->sk;
+ struct can_filter *filter = NULL; /* dyn. alloc'ed filters */
+ struct can_filter sfilter; /* single filter */
+ union {
+ can_err_mask_t err_mask;
+ int flag;
+ struct can_raw_vcid_options vcid_opts;
+ } val;
+ unsigned int size = 0;
+ int count = 0;
int err;
if (level != SOL_CAN_RAW)
return -EINVAL;
- rtnl_lock();
+ /* Copy the option values from user space before lock_sock(), so a
+ * stalled copy can not block raw_notify() waiting for lock_sock()
+ * under RTNL.
+ */
+ switch (optname) {
+ case CAN_RAW_FILTER:
+ if (optlen % sizeof(struct can_filter) != 0)
+ return -EINVAL;
+
+ if (optlen > CAN_RAW_FILTER_MAX * sizeof(struct can_filter))
+ return -EINVAL;
+
+ count = optlen / sizeof(struct can_filter);
+
+ if (count > 1) {
+ /* filter does not fit into dfilter => alloc space */
+ filter = memdup_sockptr(optval, optlen);
+ if (IS_ERR(filter))
+ return PTR_ERR(filter);
+ } else if (count == 1) {
+ if (copy_from_sockptr(&sfilter, optval, sizeof(sfilter)))
+ return -EFAULT;
+ filter = &sfilter;
+ }
+ break;
+
+ case CAN_RAW_ERR_FILTER:
+ size = sizeof(val.err_mask);
+ break;
+
+ case CAN_RAW_XL_VCID_OPTS:
+ size = sizeof(val.vcid_opts);
+ break;
+
+ case CAN_RAW_LOOPBACK:
+ case CAN_RAW_RECV_OWN_MSGS:
+ case CAN_RAW_FD_FRAMES:
+ case CAN_RAW_XL_FRAMES:
+ case CAN_RAW_JOIN_FILTERS:
+ size = sizeof(val.flag);
+ break;
+
+ default:
+ return -ENOPROTOOPT;
+ }
+
+ if (size) {
+ if (optlen != size)
+ return -EINVAL;
+
+ if (copy_from_sockptr(&val, optval, size))
+ return -EFAULT;
+ }
+
lock_sock(sk);
- err = raw_setsockopt_locked(sock, optname, optval, optlen);
+ err = raw_setsockopt_locked(sock, optname, &val, filter, count);
release_sock(sk);
- rtnl_unlock();
+
+ /* on error free the filters allocated for CAN_RAW_FILTER above */
+ if (err && count > 1)
+ kfree(filter);
return err;
}
@@ -784,6 +828,7 @@ static int raw_getsockopt(struct socket *sock, int level, int optname,
switch (optname) {
case CAN_RAW_FILTER: {
+ struct can_filter *filter = NULL;
int err = 0;
lock_sock(sk);
@@ -798,15 +843,22 @@ static int raw_getsockopt(struct socket *sock, int level, int optname,
} else {
if (len > fsize)
len = fsize;
- if (copy_to_iter(ro->filter, len,
- &opt->iter_out) != len)
- err = -EFAULT;
+ /* copy to user space after release_sock() */
+ filter = kmemdup(ro->filter, len, GFP_KERNEL);
+ if (!filter)
+ err = -ENOMEM;
}
} else {
len = 0;
}
release_sock(sk);
+ if (filter) {
+ if (copy_to_iter(filter, len, &opt->iter_out) != len)
+ err = -EFAULT;
+ kfree(filter);
+ }
+
if (!err)
opt->optlen = len;
return err;
diff --git a/net/core/dev.c b/net/core/dev.c
index cc7b32ab5608..f803d8ba866f 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4204,7 +4204,9 @@ static enum skb_drop_reason qdisc_pkt_len_segs_init(struct sk_buff *skb)
if (likely(shinfo->gso_type & (SKB_GSO_TCPV4 | SKB_GSO_TCPV6))) {
const struct tcphdr *th;
- if (!pskb_may_pull(skb, hdr_len + sizeof(struct tcphdr)))
+ /* A header in front of skb->data gives a negative offset. */
+ if ((int)hdr_len < 0 ||
+ !pskb_may_pull(skb, hdr_len + sizeof(struct tcphdr)))
return SKB_DROP_REASON_SKB_BAD_GSO;
th = (const struct tcphdr *)(skb->data + hdr_len);
diff --git a/net/core/filter.c b/net/core/filter.c
index 70dc621672f2..d3e5a2843d5e 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -2271,9 +2271,13 @@ static int bpf_out_neigh_v6(struct net *net, struct sk_buff *skb,
static int __bpf_redirect_neigh_v6(struct sk_buff *skb, struct net_device *dev,
struct bpf_nh_params *nh)
{
- const struct ipv6hdr *ip6h = ipv6_hdr(skb);
struct net *net = dev_net(dev);
int err, ret = NET_XMIT_DROP;
+ const struct ipv6hdr *ip6h;
+
+ if (unlikely(!pskb_may_pull(skb, sizeof(*ip6h))))
+ goto out_drop;
+ ip6h = ipv6_hdr(skb);
if (!nh) {
struct dst_entry *dst;
@@ -2380,9 +2384,13 @@ static int bpf_out_neigh_v4(struct net *net, struct sk_buff *skb,
static int __bpf_redirect_neigh_v4(struct sk_buff *skb, struct net_device *dev,
struct bpf_nh_params *nh)
{
- const struct iphdr *ip4h = ip_hdr(skb);
struct net *net = dev_net(dev);
int err, ret = NET_XMIT_DROP;
+ const struct iphdr *ip4h;
+
+ if (unlikely(!pskb_may_pull(skb, sizeof(*ip4h))))
+ goto out_drop;
+ ip4h = ip_hdr(skb);
if (!nh) {
struct flowi4 fl4 = {
@@ -2432,15 +2440,14 @@ static int __bpf_redirect_neigh_v4(struct sk_buff *skb, struct net_device *dev,
static int __bpf_redirect_neigh(struct sk_buff *skb, struct net_device *dev,
struct bpf_nh_params *nh)
{
- struct ethhdr *ethh = eth_hdr(skb);
-
- if (unlikely(skb->mac_header >= skb->network_header))
+ if (unlikely(skb_network_offset(skb) < ETH_HLEN ||
+ !pskb_may_pull(skb, ETH_HLEN)))
goto out;
bpf_push_mac_rcsum(skb);
- if (is_multicast_ether_addr(ethh->h_dest))
+ if (is_multicast_ether_addr(eth_hdr(skb)->h_dest))
goto out;
- skb_pull(skb, sizeof(*ethh));
+ skb_pull(skb, ETH_HLEN);
skb_unset_mac_header(skb);
skb_reset_network_header(skb);
@@ -7989,7 +7996,7 @@ BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags)
if (sk_unhashed(sk))
return -EOPNOTSUPP;
if (sk_is_refcounted(sk) &&
- unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
return -ENOENT;
skb_orphan(skb);
diff --git a/net/core/sock.c b/net/core/sock.c
index e8551df8330f..1108543f9fec 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -3262,7 +3262,7 @@ void __release_sock(struct sock *sk)
spin_unlock_bh(&sk->sk_lock.slock);
- while (1) {
+ do {
next = skb->next;
prefetch(next);
DEBUG_NET_WARN_ON_ONCE(skb_dst_is_noref(skb));
@@ -3270,12 +3270,10 @@ void __release_sock(struct sock *sk)
sk_backlog_rcv(sk, skb);
skb = next;
- if (!skb)
- break;
if (!(++nb & 15))
cond_resched();
- }
+ } while (skb);
spin_lock_bh(&sk->sk_lock.slock);
}
diff --git a/net/ipv4/cipso_ipv4.c b/net/ipv4/cipso_ipv4.c
index 1aacbbeffbc6..e65a3f29d2ca 100644
--- a/net/ipv4/cipso_ipv4.c
+++ b/net/ipv4/cipso_ipv4.c
@@ -2013,6 +2013,10 @@ static int cipso_v4_delopt(struct ip_options_rcu __rcu **opt_ptr)
cipso_ptr = &opt->opt.__data[cipso_off];
cipso_len = cipso_ptr[1];
+ optlen_new = cipso_v4_get_actual_opt_len(opt->opt.__data,
+ opt->opt.optlen);
+ optlen_new -= cipso_len;
+
if (opt->opt.srr > opt->opt.cipso)
opt->opt.srr -= cipso_len;
if (opt->opt.rr > opt->opt.cipso)
@@ -2026,11 +2030,12 @@ static int cipso_v4_delopt(struct ip_options_rcu __rcu **opt_ptr)
memmove(cipso_ptr, cipso_ptr + cipso_len,
opt->opt.optlen - cipso_off - cipso_len);
- optlen_new = cipso_v4_get_actual_opt_len(opt->opt.__data,
- opt->opt.optlen);
hdr_delta = opt->opt.optlen;
opt->opt.optlen = (optlen_new + 3) & ~3;
hdr_delta -= opt->opt.optlen;
+
+ memset(&opt->opt.__data[optlen_new], IPOPT_END,
+ opt->opt.optlen - optlen_new);
} else {
/* only the cipso option was present on the socket so we can
* remove the entire option struct */
diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c
index 8e891c8141f8..a1fe2531cb59 100644
--- a/net/ipv4/inet_connection_sock.c
+++ b/net/ipv4/inet_connection_sock.c
@@ -1060,8 +1060,8 @@ static void reqsk_timer_handler(struct timer_list *t)
* hold another count to prevent use-after-free and
* call reqsk_put() just before return.
*/
- refcount_set(&nreq->rsk_refcnt, 2 + 1);
timer_setup(&nreq->rsk_timer, reqsk_timer_handler, TIMER_PINNED);
+ refcount_set_release(&nreq->rsk_refcnt, 2 + 1);
reqsk_queue_migrated(&inet_csk(nsk)->icsk_accept_queue, req);
req = nreq;
@@ -1445,7 +1445,7 @@ struct sock *inet_csk_complete_hashdance(struct sock *sk, struct sock *child,
goto child_put;
}
- refcount_set(&nreq->rsk_refcnt, 1);
+ refcount_set_release(&nreq->rsk_refcnt, 1);
if (inet_csk_reqsk_queue_add(sk, nreq, child)) {
__NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMIGRATEREQSUCCESS);
reqsk_migrate_reset(req);
@@ -1498,7 +1498,7 @@ void inet_csk_listen_stop(struct sock *sk)
if (nsk) {
nreq = inet_reqsk_clone(req, nsk);
if (nreq) {
- refcount_set(&nreq->rsk_refcnt, 1);
+ refcount_set_release(&nreq->rsk_refcnt, 1);
rcu_read_lock();
if (inet_csk_reqsk_queue_add(nsk, nreq, child)) {
diff --git a/net/ipv4/inet_fragment.c b/net/ipv4/inet_fragment.c
index c18da91c1f03..0b8d22ffafca 100644
--- a/net/ipv4/inet_fragment.c
+++ b/net/ipv4/inet_fragment.c
@@ -336,6 +336,7 @@ void inet_frag_queue_flush(struct inet_frag_queue *q,
q->rb_fragments = RB_ROOT;
q->fragments_tail = NULL;
q->last_run_head = NULL;
+ q->truesize = 0;
}
EXPORT_SYMBOL(inet_frag_queue_flush);
@@ -446,6 +447,9 @@ int inet_frag_queue_insert(struct inet_frag_queue *q, struct sk_buff *skb,
if (last && !secpath_equal(last, skb))
return IPFRAG_MIXED_SECPATH;
+ if (skb->truesize > INET_FRAG_MAX_TRUESIZE - q->truesize)
+ return IPFRAG_TOOBIG;
+
/* An IP fragment is never a GSO packet, but an untrusted source
* (virtio_net_hdr) may have attached GSO metadata to it. Do not let
* that reach the reassembled skb, whose head keeps the first
@@ -510,6 +514,7 @@ int inet_frag_queue_insert(struct inet_frag_queue *q, struct sk_buff *skb,
if (offset)
nf_reset_ct(skb);
+ q->truesize += skb->truesize;
return IPFRAG_OK;
}
EXPORT_SYMBOL(inet_frag_queue_insert);
diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c
index 58d54965036a..7ffa9171d6de 100644
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -550,7 +550,7 @@ struct sock *__inet_lookup_established(const struct net *net,
if (sk->sk_hash != hash)
continue;
if (likely(inet_match(net, sk, acookie, ports, dif, sdif))) {
- if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
goto out;
if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) ||
!inet_match(net, sk, acookie, ports, dif,
diff --git a/net/ipv4/inet_timewait_sock.c b/net/ipv4/inet_timewait_sock.c
index d4c781a0667f..74ff40d42e56 100644
--- a/net/ipv4/inet_timewait_sock.c
+++ b/net/ipv4/inet_timewait_sock.c
@@ -141,7 +141,7 @@ void inet_twsk_hashdance_schedule(struct inet_timewait_sock *tw,
* Also note that after this point, we lost our implicit reference
* so we are not allowed to use tw anymore.
*/
- refcount_set(&tw->tw_refcnt, 3);
+ refcount_set_release(&tw->tw_refcnt, 3);
/* Ensure tw_refcnt has been set before tw is published.
* smp_wmb() provides the necessary memory barrier to enforce this
@@ -328,7 +328,7 @@ void inet_twsk_purge(struct inet_hashinfo *hashinfo)
if (check_net(sock_net(sk)))
continue;
- if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
continue;
if (check_net(sock_net(sk))) {
diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c
index 644030561537..4b2d2713653b 100644
--- a/net/ipv4/syncookies.c
+++ b/net/ipv4/syncookies.c
@@ -205,7 +205,7 @@ struct sock *tcp_get_cookie_sock(struct sock *sk, struct sk_buff *skb,
child = icsk->icsk_af_ops->syn_recv_sock(sk, skb, req, dst,
NULL, &own_req, NULL);
if (child) {
- refcount_set(&req->rsk_refcnt, 1);
+ refcount_set_release(&req->rsk_refcnt, 1);
sock_rps_save_rxhash(child, skb);
if (rsk_drop_req(req)) {
diff --git a/net/ipv4/tcp_fastopen.c b/net/ipv4/tcp_fastopen.c
index 471c78be5513..e40477b6b554 100644
--- a/net/ipv4/tcp_fastopen.c
+++ b/net/ipv4/tcp_fastopen.c
@@ -364,7 +364,7 @@ static struct sock *tcp_fastopen_create_child(struct sock *sk,
tcp_reset_xmit_timer(child, ICSK_TIME_RETRANS,
req->timeout, false);
- refcount_set(&req->rsk_refcnt, 2);
+ refcount_set_release(&req->rsk_refcnt, 2);
sk_mark_napi_id_set(child, skb);
diff --git a/net/ipv4/xfrm4_output.c b/net/ipv4/xfrm4_output.c
index 0ae67d537499..fffa3c4f456e 100644
--- a/net/ipv4/xfrm4_output.c
+++ b/net/ipv4/xfrm4_output.c
@@ -38,9 +38,12 @@ int xfrm4_output(struct net *net, struct sock *sk, struct sk_buff *skb)
void xfrm4_local_error(struct sk_buff *skb, u32 mtu)
{
+ struct sock *sk = skb_to_full_sk(skb);
struct iphdr *hdr;
+ if (!sk)
+ return;
+
hdr = skb->encapsulation ? inner_ip_hdr(skb) : ip_hdr(skb);
- ip_local_error(skb->sk, EMSGSIZE, hdr->daddr,
- inet_sk(skb->sk)->inet_dport, mtu);
+ ip_local_error(sk, EMSGSIZE, hdr->daddr, inet_sk(sk)->inet_dport, mtu);
}
diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c
index a2d3fb187f76..3350330c3f1c 100644
--- a/net/ipv6/addrconf.c
+++ b/net/ipv6/addrconf.c
@@ -1484,7 +1484,8 @@ static int ipv6_create_tempaddr(struct inet6_ifaddr *ifp, bool block)
age = (now - tmp_tstamp + ADDRCONF_TIMER_FUZZ_MINUS) / HZ;
if (cfg.preferred_lft <= regen_advance + age) {
cfg.preferred_lft = regen_advance + age + 1;
- if (cfg.preferred_lft > cfg.valid_lft ||
+ if (cfg.preferred_lft <= regen_advance + age ||
+ cfg.preferred_lft > cfg.valid_lft ||
cfg.preferred_lft > if_public_preferred_lft) {
in6_ifa_put(ifp);
in6_dev_put(idev);
diff --git a/net/ipv6/datagram.c b/net/ipv6/datagram.c
index 38d7b4845281..9672ac979f47 100644
--- a/net/ipv6/datagram.c
+++ b/net/ipv6/datagram.c
@@ -149,6 +149,7 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr_unsized *uaddr,
__be32 fl6_flowlabel = 0;
__be32 old_fl6_flowlabel;
__be16 old_dport;
+ int old_bound_dev_if;
int addr_type;
int err;
@@ -219,6 +220,8 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr_unsized *uaddr,
goto out;
}
+ old_bound_dev_if = sk->sk_bound_dev_if;
+
if (__ipv6_addr_needs_scope_id(addr_type)) {
if (addr_len >= sizeof(struct sockaddr_in6) &&
usin->sin6_scope_id) {
@@ -261,6 +264,7 @@ int __ip6_datagram_connect(struct sock *sk, struct sockaddr_unsized *uaddr,
sk->sk_v6_daddr = old_daddr;
np->flow_label = old_fl6_flowlabel;
inet->inet_dport = old_dport;
+ WRITE_ONCE(sk->sk_bound_dev_if, old_bound_dev_if);
goto out;
}
diff --git a/net/ipv6/inet6_hashtables.c b/net/ipv6/inet6_hashtables.c
index aa5fa1dd062f..1eea584d80df 100644
--- a/net/ipv6/inet6_hashtables.c
+++ b/net/ipv6/inet6_hashtables.c
@@ -105,7 +105,7 @@ struct sock *__inet6_lookup_established(const struct net *net,
continue;
if (!inet6_match(net, sk, saddr, daddr, ports, dif, sdif))
continue;
- if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
+ if (unlikely(!refcount_inc_not_zero_acquire(&sk->sk_refcnt)))
goto out;
if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) ||
diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c
index b965258cf9e5..bdea825434ea 100644
--- a/net/ipv6/raw.c
+++ b/net/ipv6/raw.c
@@ -654,6 +654,9 @@ static int rawv6_send_hdrinc(struct sock *sk, struct msghdr *msg, int length,
if (unlikely(!skb))
return 0;
+ /* l3mdev_ip6_out() may have replaced the skb dst. */
+ rt = dst_rt6_info(skb_dst(skb));
+
/* Acquire rcu_read_lock() in case we need to use rt->rt6i_idev
* in the error path. Since skb has been freed, the dst could
* have been queued for deletion.
diff --git a/net/l2tp/l2tp_core.c b/net/l2tp/l2tp_core.c
index 7e042a3e648c..0cd3e38bf662 100644
--- a/net/l2tp/l2tp_core.c
+++ b/net/l2tp/l2tp_core.c
@@ -315,10 +315,19 @@ EXPORT_SYMBOL_GPL(l2tp_v2_session_get);
struct l2tp_session *l2tp_session_get(const struct net *net, struct sock *sk, int pver,
u32 tunnel_id, u32 session_id)
{
+ struct l2tp_session *session;
+
if (pver == L2TP_HDR_VER_2)
return l2tp_v2_session_get(net, tunnel_id, session_id);
- else
- return l2tp_v3_session_get(net, sk, session_id);
+
+ /* l2tp_v3_session_get() doesn't check sk if the ID hasn't collided */
+ session = l2tp_v3_session_get(net, sk, session_id);
+ if (session && session->tunnel->sock != sk) {
+ l2tp_session_put(session);
+ return NULL;
+ }
+
+ return session;
}
EXPORT_SYMBOL_GPL(l2tp_session_get);
diff --git a/net/mac80211/tdls.c b/net/mac80211/tdls.c
index f663d28d9209..71db07aa301b 100644
--- a/net/mac80211/tdls.c
+++ b/net/mac80211/tdls.c
@@ -2021,7 +2021,7 @@ void
ieee80211_process_tdls_channel_switch(struct ieee80211_sub_if_data *sdata,
struct sk_buff *skb)
{
- struct ieee80211_tdls_data *tf = (void *)skb->data;
+ struct ieee80211_tdls_data *tf;
struct wiphy *wiphy = sdata->local->hw.wiphy;
lockdep_assert_wiphy(wiphy);
@@ -2037,6 +2037,7 @@ ieee80211_process_tdls_channel_switch(struct ieee80211_sub_if_data *sdata,
* The packet/size was already validated by mac80211 Rx path, only look
* at the action type.
*/
+ tf = (void *)skb->data;
switch (tf->action_code) {
case WLAN_TDLS_CHANNEL_SWITCH_REQUEST:
ieee80211_process_tdls_channel_switch_req(sdata, skb);
diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c
index 5714634c8103..27bae2a4fa86 100644
--- a/net/mac80211/tx.c
+++ b/net/mac80211/tx.c
@@ -1130,6 +1130,7 @@ static bool ieee80211_tx_prep_agg(struct ieee80211_tx_data *tx,
struct sk_buff *purge_skb = NULL;
if (test_bit(HT_AGG_STATE_OPERATIONAL, &tid_tx->state)) {
+ info->flags |= IEEE80211_TX_CTL_AMPDU;
reset_agg_timer = true;
} else if (test_bit(HT_AGG_STATE_WANT_START, &tid_tx->state)) {
/*
@@ -1161,6 +1162,7 @@ static bool ieee80211_tx_prep_agg(struct ieee80211_tx_data *tx,
if (!tid_tx) {
/* do nothing, let packet pass through */
} else if (test_bit(HT_AGG_STATE_OPERATIONAL, &tid_tx->state)) {
+ info->flags |= IEEE80211_TX_CTL_AMPDU;
reset_agg_timer = true;
} else {
queued = true;
diff --git a/net/mptcp/pm_kernel.c b/net/mptcp/pm_kernel.c
index 1a7750813235..f7396628490e 100644
--- a/net/mptcp/pm_kernel.c
+++ b/net/mptcp/pm_kernel.c
@@ -906,6 +906,7 @@ static int mptcp_pm_nl_create_listen_socket(struct sock *sk,
* old status is known (TCP_CLOSE).
*/
inet_sk_state_store(newsk, TCP_LISTEN);
+ sock_set_flag(newsk, SOCK_RCU_FREE);
lock_sock(ssk);
WRITE_ONCE(mptcp_subflow_ctx(ssk)->pm_listener, true);
err = __inet_listen_sk(ssk, backlog);
diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c
index 2c72cb50fbf3..0b67b46ef872 100644
--- a/net/mptcp/subflow.c
+++ b/net/mptcp/subflow.c
@@ -379,6 +379,10 @@ static void subflow_prep_synack(const struct sock *sk, struct request_sock *req,
struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(sk);
struct inet_request_sock *ireq = inet_rsk(req);
+ /* MP_JOIN cannot use TFO, do not send a cookie in the SYN/ACK */
+ if (foc && mptcp_subflow_rsk(req)->mp_join)
+ foc->len = -1;
+
/* clear tstamp_ok, as needed depending on cookie */
if (foc && foc->len > -1)
ireq->tstamp_ok = 0;
@@ -887,6 +891,13 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
fallback = true;
} else if (subflow_req->mp_join) {
+ /* a SYN skb here comes from TFO, which MP_JOIN cannot use: just
+ * fall back to the regular path.
+ */
+ if ((TCP_SKB_CB(skb)->tcp_flags & (TCPHDR_SYN | TCPHDR_ACK)) ==
+ TCPHDR_SYN)
+ return NULL;
+
mptcp_get_options(skb, &mp_opt);
if (!(mp_opt.suboptions & OPTION_MPTCP_MPJ_ACK))
fallback = true;
diff --git a/net/qrtr/af_qrtr.c b/net/qrtr/af_qrtr.c
index e7b3647424b8..45cb8e8c36ca 100644
--- a/net/qrtr/af_qrtr.c
+++ b/net/qrtr/af_qrtr.c
@@ -122,6 +122,7 @@ static DEFINE_XARRAY_ALLOC(qrtr_ports);
* @qrtr_tx_flow: xarray of qrtr_tx_flow, keyed by node << 32 | port
* @qrtr_tx_lock: lock for qrtr_tx_flow inserts
* @hello_sent: hello packet send successful
+ * @hello_rcvd: remote has sent a hello packet
* @rx_queue: receive queue
* @item: list item for broadcast list
* @say_hello: delayed work for sending hello packet
@@ -136,6 +137,7 @@ struct qrtr_node {
struct mutex qrtr_tx_lock; /* for qrtr_tx_flow */
bool hello_sent;
+ bool hello_rcvd;
struct sk_buff_head rx_queue;
struct list_head item;
@@ -541,6 +543,11 @@ int qrtr_endpoint_post(struct qrtr_endpoint *ep, const void *data, size_t len)
qrtr_node_assign(node, le32_to_cpu(pkt->server.node));
}
+ if (cb->type == QRTR_TYPE_HELLO) {
+ /* Pairs with smp_load_acquire() in qrtr_hello_work() */
+ smp_store_release(&node->hello_rcvd, true);
+ }
+
if (cb->type == QRTR_TYPE_RESUME_TX) {
qrtr_tx_resume(node, skb);
} else {
@@ -599,6 +606,8 @@ static void qrtr_hello_work(struct work_struct *work)
struct qrtr_node *node;
struct qrtr_sock *ctrl;
struct sk_buff *skb;
+ bool hello_sent;
+ int rc = 0;
node = container_of(to_delayed_work(work), struct qrtr_node, say_hello);
@@ -609,17 +618,45 @@ static void qrtr_hello_work(struct work_struct *work)
return;
}
- skb = qrtr_alloc_ctrl_packet(&pkt, GFP_KERNEL);
- if (!skb) {
- qrtr_port_put(ctrl);
- schedule_delayed_work(&node->say_hello, msecs_to_jiffies(100));
- return;
+ mutex_lock(&node->ep_lock);
+ hello_sent = node->hello_sent;
+ mutex_unlock(&node->ep_lock);
+
+ from.sq_node = qrtr_local_nid;
+ if (!hello_sent) {
+ skb = qrtr_alloc_ctrl_packet(&pkt, GFP_KERNEL);
+ if (!skb)
+ goto retry;
+
+ pkt->cmd = cpu_to_le32(QRTR_TYPE_HELLO);
+ to.sq_node = node->nid;
+ if (qrtr_node_enqueue(node, skb, QRTR_TYPE_HELLO, &from, &to))
+ goto out;
}
+ /* Pairs with smp_store_release() in qrtr_endpoint_post() */
+ if (!smp_load_acquire(&node->hello_rcvd))
+ goto out;
+
+ skb = qrtr_alloc_ctrl_packet(&pkt, GFP_KERNEL);
+ if (!skb)
+ goto retry;
+
pkt->cmd = cpu_to_le32(QRTR_TYPE_HELLO);
- from.sq_node = qrtr_local_nid;
- to.sq_node = node->nid;
- qrtr_node_enqueue(node, skb, QRTR_TYPE_HELLO, &from, &to);
+ to.sq_node = READ_ONCE(node->nid);
+
+ mutex_lock(&node->ep_lock);
+ if (node->ep)
+ rc = qrtr_local_enqueue(NULL, skb, QRTR_TYPE_HELLO, &to, &from);
+ else
+ kfree_skb(skb);
+ mutex_unlock(&node->ep_lock);
+ if (!rc)
+ goto out;
+
+retry:
+ schedule_delayed_work(&node->say_hello, msecs_to_jiffies(100));
+out:
qrtr_port_put(ctrl);
}
diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
index 3e8ee4a691c3..d2e0d20f3ba1 100644
--- a/net/sched/sch_generic.c
+++ b/net/sched/sch_generic.c
@@ -96,9 +96,34 @@ static void qdisc_maybe_clear_missed(struct Qdisc *q,
#define SKB_XOFF_MAGIC ((struct sk_buff *)1UL)
+/* Re-cap a stale queue_mapping; an ONETXQUEUE qdisc pins q->dev_queue. */
+static u16 qdisc_cap_skb_tx_queue(struct Qdisc *q, struct sk_buff *skb)
+{
+ u16 queue = skb_get_queue_mapping(skb);
+ struct net_device *dev = qdisc_dev(q);
+ u16 capped;
+
+ if (q->flags & TCQ_F_ONETXQUEUE)
+ capped = q->dev_queue - netdev_get_tx_queue(dev, 0);
+ else
+ capped = netdev_cap_txqueue(dev, queue);
+
+ if (unlikely(capped != queue))
+ skb_set_queue_mapping(skb, capped);
+
+ return capped;
+}
+
+static struct netdev_queue *skb_cap_tx_queue(struct Qdisc *q,
+ struct sk_buff *skb)
+{
+ return netdev_get_tx_queue(qdisc_dev(q),
+ qdisc_cap_skb_tx_queue(q, skb));
+}
+
static inline struct sk_buff *__skb_dequeue_bad_txq(struct Qdisc *q)
{
- const struct netdev_queue *txq = q->dev_queue;
+ const struct netdev_queue *txq;
spinlock_t *lock = NULL;
struct sk_buff *skb;
@@ -110,7 +135,8 @@ static inline struct sk_buff *__skb_dequeue_bad_txq(struct Qdisc *q)
skb = skb_peek(&q->skb_bad_txq);
if (skb) {
/* check the reason of requeuing without tx lock first */
- txq = skb_get_tx_queue(txq->dev, skb);
+ /* re-cap: real_num_tx_queues may have shrunk */
+ txq = skb_cap_tx_queue(q, skb);
if (!netif_xmit_frozen_or_stopped(txq)) {
skb = __skb_dequeue(&q->skb_bad_txq);
if (qdisc_is_percpu_stats(q)) {
@@ -224,6 +250,8 @@ static void try_bulk_dequeue_skb(struct Qdisc *q,
if (!nskb)
break;
+ /* Every element of an ONETXQUEUE qdisc is pinned to q->dev_queue. */
+ qdisc_cap_skb_tx_queue(q, nskb);
bytelimit -= nskb->len; /* covers GSO len */
skb->next = nskb;
skb = nskb;
@@ -241,7 +269,6 @@ static void try_bulk_dequeue_skb_slow(struct Qdisc *q,
struct sk_buff *skb,
int *packets)
{
- int mapping = skb_get_queue_mapping(skb);
struct sk_buff *nskb;
int cnt = 0;
@@ -249,7 +276,9 @@ static void try_bulk_dequeue_skb_slow(struct Qdisc *q,
nskb = q->dequeue(q);
if (!nskb)
break;
- if (unlikely(skb_get_queue_mapping(nskb) != mapping)) {
+ /* A tc BPF store may have changed the mapping; cap it. */
+ if (unlikely(qdisc_cap_skb_tx_queue(q, nskb) !=
+ skb_get_queue_mapping(skb))) {
qdisc_enqueue_skb_bad_txq(q, nskb);
break;
}
@@ -294,7 +323,7 @@ static struct sk_buff *dequeue_skb(struct Qdisc *q, bool *validate,
if (xfrm_offload(skb))
*validate = true;
/* check the reason of requeuing without tx lock first */
- txq = skb_get_tx_queue(txq->dev, skb);
+ txq = skb_cap_tx_queue(q, skb);
if (!netif_xmit_frozen_or_stopped(txq)) {
skb = __skb_dequeue(&q->gso_skb);
if (qdisc_is_percpu_stats(q)) {
@@ -330,6 +359,8 @@ static struct sk_buff *dequeue_skb(struct Qdisc *q, bool *validate,
skb = q->dequeue(q);
if (skb) {
bulk:
+ /* cap the head; the bulk helpers cap every appended element */
+ qdisc_cap_skb_tx_queue(q, skb);
if (qdisc_may_bulk(q))
try_bulk_dequeue_skb(q, skb, txq, packets, budget);
else
diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c
index 6f3b7273cb16..e900890e9c54 100644
--- a/net/sched/sch_qfq.c
+++ b/net/sched/sch_qfq.c
@@ -517,11 +517,14 @@ static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
sch_tree_lock(sch);
qfq_init_agg(q, new_agg, lmax, weight);
}
+ if (existing && new_agg == cl->agg)
+ goto unlock;
if (existing)
qfq_deact_rm_from_agg(q, cl);
else
qdisc_class_hash_insert(&q->clhash, &cl->common);
qfq_add_to_agg(q, new_agg, cl);
+unlock:
sch_tree_unlock(sch);
qdisc_class_hash_grow(sch, &q->clhash);
diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index 39ac5b97aa3a..0eb0803047a1 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -632,6 +632,11 @@ static int taprio_enqueue(struct sk_buff *skb, struct Qdisc *sch,
int queue;
queue = skb_get_queue_mapping(skb);
+ if (unlikely(queue >= qdisc_dev(sch)->real_num_tx_queues)) {
+ /* A tc BPF program may have stored an out-of-range mapping. */
+ queue = netdev_cap_txqueue(qdisc_dev(sch), queue);
+ skb_set_queue_mapping(skb, queue);
+ }
child = q->qdiscs[queue];
if (unlikely(child == &noop_qdisc))
diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c
index 84a4c97d0f75..0896ec6c75ce 100644
--- a/net/sctp/sm_make_chunk.c
+++ b/net/sctp/sm_make_chunk.c
@@ -94,7 +94,7 @@ static void sctp_control_release_owner(struct sk_buff *skb)
}
}
-static void sctp_control_set_owner_w(struct sctp_chunk *chunk)
+void sctp_control_set_owner_w(struct sctp_chunk *chunk)
{
struct sctp_association *asoc = chunk->asoc;
struct sk_buff *skb = chunk->skb;
@@ -107,7 +107,8 @@ static void sctp_control_set_owner_w(struct sctp_chunk *chunk)
* For now don't do anything for now.
*/
if (chunk->auth) {
- chunk->shkey = asoc->shkey;
+ if (!chunk->shkey)
+ chunk->shkey = asoc->shkey;
sctp_auth_shkey_hold(chunk->shkey);
}
skb->sk = asoc ? asoc->base.sk : NULL;
@@ -136,6 +137,7 @@ int sctp_init_cause(struct sctp_chunk *chunk, __be16 cause_code,
size_t paylen)
{
struct sctp_errhdr err;
+ int chunklen, padlen;
__u16 len;
/* Cause code constants are now defined in network order. */
@@ -143,7 +145,13 @@ int sctp_init_cause(struct sctp_chunk *chunk, __be16 cause_code,
len = sizeof(err) + paylen;
err.length = htons(len);
- if (skb_tailroom(chunk->skb) < len)
+ /* sctp_addto_chunk() pads the chunk up to a 4-byte boundary before
+ * appending the cause, so account for that padding here as well.
+ */
+ chunklen = ntohs(chunk->chunk_hdr->length);
+ padlen = SCTP_PAD4(chunklen) - chunklen;
+
+ if (skb_tailroom(chunk->skb) < padlen + len)
return -ENOSPC;
chunk->subh.err_hdr = sctp_addto_chunk(chunk, sizeof(err), &err);
diff --git a/net/sctp/socket.c b/net/sctp/socket.c
index c616f1609d0c..b5a57f61ee6d 100644
--- a/net/sctp/socket.c
+++ b/net/sctp/socket.c
@@ -148,25 +148,47 @@ static inline void sctp_set_owner_w(struct sctp_chunk *chunk)
static void sctp_clear_owner_w(struct sctp_chunk *chunk)
{
+ /* Keep the shkey alive until the new owner takes its reference. */
+ if (chunk->shkey)
+ sctp_auth_shkey_hold(chunk->shkey);
skb_orphan(chunk->skb);
}
+static void sctp_set_owner_w_migrate(struct sctp_chunk *chunk)
+{
+ struct sctp_shared_key *shkey = chunk->shkey;
+
+ if (chunk->msg)
+ sctp_set_owner_w(chunk);
+ else
+ sctp_control_set_owner_w(chunk);
+
+ if (shkey)
+ sctp_auth_shkey_release(shkey);
+}
+
+static void sctp_process_tx_chunk(struct sctp_association *asoc,
+ struct sctp_chunk *chunk, bool clear,
+ void (*cb)(struct sctp_chunk *))
+{
+ if ((clear && asoc->base.sk == chunk->skb->sk) ||
+ (!clear && asoc->base.sk != chunk->skb->sk))
+ cb(chunk);
+}
+
#define traverse_and_process() \
do { \
msg = chunk->msg; \
if (msg == prev_msg) \
continue; \
- list_for_each_entry(c, &msg->chunks, frag_list) { \
- if ((clear && asoc->base.sk == c->skb->sk) || \
- (!clear && asoc->base.sk != c->skb->sk)) \
- cb(c); \
- } \
+ list_for_each_entry(c, &msg->chunks, frag_list) \
+ sctp_process_tx_chunk(asoc, c, clear, cb); \
prev_msg = msg; \
} while (0)
-static void sctp_for_each_tx_datachunk(struct sctp_association *asoc,
- bool clear,
- void (*cb)(struct sctp_chunk *))
+static void sctp_for_each_tx_chunk(struct sctp_association *asoc,
+ bool clear,
+ void (*cb)(struct sctp_chunk *))
{
struct sctp_datamsg *msg, *prev_msg = NULL;
@@ -189,6 +211,21 @@ static void sctp_for_each_tx_datachunk(struct sctp_association *asoc,
list_for_each_entry(chunk, &q->out_chunk_list, list)
traverse_and_process();
+
+ list_for_each_entry(chunk, &q->control_chunk_list, list)
+ sctp_process_tx_chunk(asoc, chunk, clear, cb);
+
+ list_for_each_entry(chunk, &asoc->asconf_ack_list, transmitted_list)
+ sctp_process_tx_chunk(asoc, chunk, clear, cb);
+
+ list_for_each_entry(chunk, &asoc->addip_chunk_list, list)
+ sctp_process_tx_chunk(asoc, chunk, clear, cb);
+
+ if (asoc->strreset_chunk)
+ sctp_process_tx_chunk(asoc, asoc->strreset_chunk, clear, cb);
+
+ if (asoc->addip_last_asconf)
+ sctp_process_tx_chunk(asoc, asoc->addip_last_asconf, clear, cb);
}
static void sctp_for_each_rx_skb(struct sctp_association *asoc, struct sock *sk,
@@ -9647,9 +9684,9 @@ static int sctp_sock_migrate(struct sock *oldsk, struct sock *newsk,
* paths won't try to lock it and then oldsk.
*/
lock_sock_nested(newsk, SINGLE_DEPTH_NESTING);
- sctp_for_each_tx_datachunk(assoc, true, sctp_clear_owner_w);
+ sctp_for_each_tx_chunk(assoc, true, sctp_clear_owner_w);
sctp_assoc_migrate(assoc, newsk);
- sctp_for_each_tx_datachunk(assoc, false, sctp_set_owner_w);
+ sctp_for_each_tx_chunk(assoc, false, sctp_set_owner_w_migrate);
/* If the association on the newsk is already closed before accept()
* is called, set RCV_SHUTDOWN flag.
diff --git a/net/smc/smc_rx.c b/net/smc/smc_rx.c
index 197fddc6271f..63c178c10bf1 100644
--- a/net/smc/smc_rx.c
+++ b/net/smc/smc_rx.c
@@ -286,6 +286,27 @@ int smc_rx_wait(struct smc_sock *smc, long *timeo, size_t peeked,
return rc;
}
+/* Wait for the pipe to release the spliced data. Unlike smc_rx_wait()
+ * it does not end on shutdown or sk_err, the data is already in the
+ * RMB; smc_conn_kill() wakes it and the caller checks conn->killed.
+ */
+static void smc_rx_wait_splice(struct smc_sock *smc, long *timeo)
+{
+ DEFINE_WAIT_FUNC(wait, woken_wake_function);
+ struct smc_connection *conn = &smc->conn;
+ struct sock *sk = &smc->sk;
+
+ sk_set_bit(SOCKWQ_ASYNC_WAITDATA, sk);
+ add_wait_queue(sk_sleep(sk), &wait);
+ /* pairs with the smp_mb() in wq_has_sleeper(): the release drops
+ * splice_pending, then looks for a sleeper
+ */
+ smp_mb();
+ sk_wait_event(sk, timeo, !atomic_read(&conn->splice_pending), &wait);
+ remove_wait_queue(sk_sleep(sk), &wait);
+ sk_clear_bit(SOCKWQ_ASYNC_WAITDATA, sk);
+}
+
static int smc_rx_recv_urg(struct smc_sock *smc, struct msghdr *msg, int len,
int flags)
{
@@ -443,7 +464,23 @@ int smc_rx_recvmsg(struct smc_sock *smc, struct msghdr *msg,
/* could be just 1 byte, even after waiting on data above */
readable = smc_rx_data_available(conn, peeked_bytes);
splbytes = atomic_read(&conn->splice_pending);
- if (!readable || (msg && splbytes)) {
+ if (msg && splbytes) {
+ /* a pipe holds bytes ahead of these; the copy path
+ * skips the timeout and signal checks
+ */
+ if (!read_remaining)
+ break;
+ if (!timeo || signal_pending(current)) {
+ if (!read_done)
+ read_done = timeo ?
+ sock_intr_errno(timeo) :
+ -EAGAIN;
+ break;
+ }
+ smc_rx_wait_splice(smc, &timeo);
+ continue;
+ }
+ if (!readable) {
if (splbytes)
func = smc_rx_data_available_and_no_splice_pend;
else
diff --git a/net/sunrpc/rpcb_clnt.c b/net/sunrpc/rpcb_clnt.c
index 4c0b7fefee4e..84c06f99e1c7 100644
--- a/net/sunrpc/rpcb_clnt.c
+++ b/net/sunrpc/rpcb_clnt.c
@@ -400,7 +400,8 @@ static struct rpc_clnt *rpcb_create(struct net *net, const char *nodename,
return rpc_create(&args);
}
-static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, struct rpc_message *msg, bool is_set)
+static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt,
+ struct rpc_message *msg, bool is_set)
{
int flags = RPC_TASK_NOCONNECT;
int error, result = 0;
@@ -410,8 +411,23 @@ static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, stru
msg->rpc_resp = &result;
error = rpc_call_sync(clnt, msg, flags);
- if (error < 0)
- return error;
+ if (error < 0) {
+ switch (error) {
+ /* rpcbind answered; the reply itself carries the error */
+ case -EPROTONOSUPPORT:
+ case -EPFNOSUPPORT:
+ case -EOPNOTSUPP:
+ case -EACCES:
+ /* the call never made it onto the wire */
+ case -ENOMEM:
+ case -EMSGSIZE:
+ /* the caller is going away; this says nothing about rpcbind */
+ case -ERESTARTSYS:
+ return error;
+ }
+ /* anything else, we assume that rpcbind isn't functional */
+ return -EIO;
+ }
if (!result)
return -EACCES;
diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c
index 4f402bbf97ba..ca6f90653327 100644
--- a/net/sunrpc/svc.c
+++ b/net/sunrpc/svc.c
@@ -1179,10 +1179,40 @@ int svc_generic_rpcbind_set(struct net *net,
error = svc_rpcbind_set_version(net, progp, version,
family, proto, port);
+ /* -EIO means no answer, not a refusal, so vs_rpcb_optnl must keep it. */
+ if (error == -EIO)
+ return error;
+
return (vers->vs_rpcb_optnl) ? 0 : error;
}
EXPORT_SYMBOL_GPL(svc_generic_rpcbind_set);
+/**
+ * svc_rpcb_failure_count - local rpcbind calls for @serv that got no answer
+ * @serv: RPC service to query
+ *
+ * svc_register() adds one for each of its calls that got no answer. A reply
+ * that refuses one entry does not count, because rpcbind answered and the
+ * next entry may still succeed.
+ *
+ * The count is kept per serv rather than per net. The local rpcbind client
+ * is per-net and lockd shares it, but a count that another service can move
+ * says nothing about this serv's own calls.
+ *
+ * This is for callers that cannot see the svc_register() return, because a
+ * transport class sits in between. Such a caller reads the count before it
+ * starts and compares as it goes, so there is no state to reset between
+ * operations. The count never resets, and callers must not attach meaning
+ * to the value itself.
+ *
+ * Return: the number of unanswered calls since this serv was created.
+ */
+unsigned int svc_rpcb_failure_count(struct svc_serv *serv)
+{
+ return atomic_read(&serv->sv_rpcb_failures);
+}
+EXPORT_SYMBOL_GPL(svc_rpcb_failure_count);
+
/**
* svc_register - register an RPC service with the local portmapper
* @serv: svc_serv struct for the service to register
@@ -1193,10 +1223,11 @@ EXPORT_SYMBOL_GPL(svc_generic_rpcbind_set);
*
* Service is registered for any address in the passed-in protocol family
*/
-int svc_register(const struct svc_serv *serv, struct net *net,
+int svc_register(struct svc_serv *serv, struct net *net,
const int family, const unsigned short proto,
const unsigned short port)
{
+ bool noanswer = false;
unsigned int p, i;
int error = 0;
@@ -1208,10 +1239,16 @@ int svc_register(const struct svc_serv *serv, struct net *net,
struct svc_program *progp = &serv->sv_programs[p];
for (i = 0; i < progp->pg_nvers; i++) {
+ const struct svc_version *vers = progp->pg_vers[i];
int ret;
ret = progp->pg_rpcbind_set(net, progp, i,
family, proto, port);
+ if (ret == -EIO) {
+ noanswer = true;
+ if (vers && vers->vs_rpcb_optnl)
+ ret = 0;
+ }
if (ret < 0) {
printk(KERN_WARNING "svc: failed to register "
"%sv%u RPC service (errno %d).\n",
@@ -1223,6 +1260,9 @@ int svc_register(const struct svc_serv *serv, struct net *net,
}
}
+ if (noanswer)
+ atomic_inc(&serv->sv_rpcb_failures);
+
return error;
}
diff --git a/net/wireless/reg.c b/net/wireless/reg.c
index 11665e0a7efc..44226687349e 100644
--- a/net/wireless/reg.c
+++ b/net/wireless/reg.c
@@ -679,6 +679,9 @@ static bool valid_rule(const u8 *data, unsigned int size, u16 rule_ptr)
/* mandatory fields */
if (rule->len < offsetofend(struct fwdb_rule, max_bw))
return false;
+ /* the rule must lie entirely within the database */
+ if ((u8 *)rule + rule->len > data + size)
+ return false;
if (rule->len >= offsetofend(struct fwdb_rule, wmm_ptr)) {
u32 wmm_ptr = be16_to_cpu(rule->wmm_ptr) << 2;
struct fwdb_wmm_rule *wmm;
diff --git a/net/xfrm/espintcp.c b/net/xfrm/espintcp.c
index 3e72b9f067b9..4052ae5bbb2e 100644
--- a/net/xfrm/espintcp.c
+++ b/net/xfrm/espintcp.c
@@ -434,7 +434,9 @@ static void espintcp_destruct(struct sock *sk)
bool tcp_is_ulp_esp(struct sock *sk)
{
- return sk->sk_prot == &espintcp_prot || sk->sk_prot == &espintcp6_prot;
+ const struct proto *prot = READ_ONCE(sk->sk_prot);
+
+ return prot == &espintcp_prot || prot == &espintcp6_prot;
}
EXPORT_SYMBOL_GPL(tcp_is_ulp_esp);
@@ -466,34 +468,41 @@ static int espintcp_init_sk(struct sock *sk)
__sk_dst_reset(sk);
- strp_check_rcv(&ctx->strp);
skb_queue_head_init(&ctx->ike_queue);
skb_queue_head_init(&ctx->out_queue);
+ INIT_WORK(&ctx->work, espintcp_tx_work);
+
+ ctx->saved_data_ready = sk->sk_data_ready;
+ ctx->saved_write_space = sk->sk_write_space;
+ ctx->saved_destruct = sk->sk_destruct;
+
+ /* avoid using task_frag */
+ sk->sk_allocation = GFP_ATOMIC;
+ sk->sk_use_task_frag = false;
+
+ rcu_assign_pointer(icsk->icsk_ulp_data, ctx);
+
+ /* pairs with smp_rmb() in espintcp_getctx() */
+ smp_wmb();
+
+ WRITE_ONCE(sk->sk_data_ready, espintcp_data_ready);
+ WRITE_ONCE(sk->sk_write_space, espintcp_write_space);
+ sk->sk_destruct = espintcp_destruct;
if (sk->sk_family == AF_INET) {
- sk->sk_prot = &espintcp_prot;
- sk->sk_socket->ops = &espintcp_ops;
+ WRITE_ONCE(sk->sk_prot, &espintcp_prot);
+ WRITE_ONCE(sk->sk_socket->ops, &espintcp_ops);
} else {
mutex_lock(&tcpv6_prot_mutex);
if (!espintcp6_prot.recvmsg)
build_protos(&espintcp6_prot, &espintcp6_ops, sk->sk_prot, sk->sk_socket->ops);
mutex_unlock(&tcpv6_prot_mutex);
- sk->sk_prot = &espintcp6_prot;
- sk->sk_socket->ops = &espintcp6_ops;
+ WRITE_ONCE(sk->sk_prot, &espintcp6_prot);
+ WRITE_ONCE(sk->sk_socket->ops, &espintcp6_ops);
}
- ctx->saved_data_ready = sk->sk_data_ready;
- ctx->saved_write_space = sk->sk_write_space;
- ctx->saved_destruct = sk->sk_destruct;
- sk->sk_data_ready = espintcp_data_ready;
- sk->sk_write_space = espintcp_write_space;
- sk->sk_destruct = espintcp_destruct;
- rcu_assign_pointer(icsk->icsk_ulp_data, ctx);
- INIT_WORK(&ctx->work, espintcp_tx_work);
- /* avoid using task_frag */
- sk->sk_allocation = GFP_ATOMIC;
- sk->sk_use_task_frag = false;
+ strp_check_rcv(&ctx->strp);
return 0;
diff --git a/net/xfrm/xfrm_interface_core.c b/net/xfrm/xfrm_interface_core.c
index 688306bf62c5..9835fbe63ccb 100644
--- a/net/xfrm/xfrm_interface_core.c
+++ b/net/xfrm/xfrm_interface_core.c
@@ -531,8 +531,13 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev)
memset(&fl, 0, sizeof(fl));
+ if (!pskb_inet_may_pull(skb))
+ goto tx_err;
+
switch (skb->protocol) {
case htons(ETH_P_IPV6):
+ if (ipv6_hdr(skb)->version != 6)
+ goto tx_err;
memset(IP6CB(skb), 0, sizeof(*IP6CB(skb)));
xfrm_decode_session(dev_net(dev), skb, &fl, AF_INET6);
if (!dst) {
@@ -548,6 +553,9 @@ static netdev_tx_t xfrmi_xmit(struct sk_buff *skb, struct net_device *dev)
}
break;
case htons(ETH_P_IP):
+ if (ip_hdr(skb)->version != 4 || ip_hdr(skb)->ihl < 5 ||
+ !pskb_network_may_pull(skb, ip_hdr(skb)->ihl * 4))
+ goto tx_err;
memset(IPCB(skb), 0, sizeof(*IPCB(skb)));
xfrm_decode_session(dev_net(dev), skb, &fl, AF_INET);
if (!dst) {
diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
index 513c9f228334..f6f40ba713d5 100644
--- a/net/xfrm/xfrm_policy.c
+++ b/net/xfrm/xfrm_policy.c
@@ -2157,6 +2157,7 @@ static struct xfrm_policy *xfrm_policy_lookup_bytype(struct net *net, u8 type,
struct xfrm_pol_inexact_bin *bin;
struct xfrm_policy *pol, *ret;
struct hlist_head *chain;
+ unsigned int inexact_sequence;
unsigned int sequence;
int err;
@@ -2191,12 +2192,18 @@ static struct xfrm_policy *xfrm_policy_lookup_bytype(struct net *net, u8 type,
goto skip_inexact;
bin = xfrm_policy_inexact_lookup_rcu(net, type, family, dir, if_id);
- if (!bin || !xfrm_policy_find_inexact_candidates(&cand, bin, saddr,
- daddr))
+ if (!bin)
+ goto skip_inexact;
+
+ inexact_sequence = read_seqcount_begin(&bin->count);
+ if (!xfrm_policy_find_inexact_candidates(&cand, bin, saddr, daddr))
goto skip_inexact;
pol = xfrm_policy_eval_candidates(&cand, ret, fl, type,
family, if_id);
+ if (read_seqcount_retry(&bin->count, inexact_sequence))
+ goto retry;
+
if (pol) {
ret = pol;
if (IS_ERR(pol))
diff --git a/rust/kernel/cpufreq.rs b/rust/kernel/cpufreq.rs
index b75d692846a2..f31671129c98 100644
--- a/rust/kernel/cpufreq.rs
+++ b/rust/kernel/cpufreq.rs
@@ -696,8 +696,8 @@ fn clear_data<T: ForeignOwnable>(&mut self) -> Option<T> {
impl<'a> PolicyCpu<'a> {
fn from_cpu(cpu: CpuId) -> Result<Self> {
// SAFETY: It is safe to call `cpufreq_cpu_get` for any valid CPU.
- let ptr =
- NonNull::new(unsafe { bindings::cpufreq_cpu_get(u32::from(cpu)) }).ok_or(ENODEV)?;
+ let ptr = unsafe { bindings::cpufreq_cpu_get(u32::from(cpu)) };
+ let ptr = NonNull::new(ptr).ok_or(ENODEV)?;
Ok(Self(
// SAFETY: `ptr` is non-NULL and `cpufreq_cpu_get()` took a reference on it, so it is
diff --git a/sound/core/hwdep.c b/sound/core/hwdep.c
index c6de5345e946..fb9034897c56 100644
--- a/sound/core/hwdep.c
+++ b/sound/core/hwdep.c
@@ -212,6 +212,7 @@ static int snd_hwdep_dsp_load(struct snd_hwdep *hw,
return -ENXIO;
if (info->index >= 32)
return -EINVAL;
+ guard(mutex)(&hw->dsp_mutex);
/* check whether the dsp was already loaded */
if (hw->dsp_loaded & (1u << info->index))
return -EBUSY;
@@ -380,6 +381,7 @@ int snd_hwdep_new(struct snd_card *card, char *id, int device,
init_waitqueue_head(&hwdep->open_wait);
mutex_init(&hwdep->open_mutex);
+ mutex_init(&hwdep->dsp_mutex);
hwdep->card = card;
hwdep->device = device;
if (id)
diff --git a/sound/core/ump_convert.c b/sound/core/ump_convert.c
index 0a7972d91ec5..ecbd894f4c83 100644
--- a/sound/core/ump_convert.c
+++ b/sound/core/ump_convert.c
@@ -69,7 +69,7 @@ static u32 upscale_14_to_32bit(u16 src)
static int cvt_ump_system_to_legacy(u32 data, unsigned char *buf)
{
buf[0] = ump_message_status_channel(data);
- switch (ump_message_status_code(data)) {
+ switch (buf[0]) {
case UMP_SYSTEM_STATUS_MIDI_TIME_CODE:
case UMP_SYSTEM_STATUS_SONG_SELECT:
buf[1] = (data >> 8) & 0x7f;
diff --git a/sound/hda/codecs/realtek/alc269.c b/sound/hda/codecs/realtek/alc269.c
index 68650004ce0f..49c4d01759b0 100644
--- a/sound/hda/codecs/realtek/alc269.c
+++ b/sound/hda/codecs/realtek/alc269.c
@@ -3537,6 +3537,27 @@ static void alc233_fixup_no_audio_jack(struct hda_codec *codec,
alc_process_coef_fw(codec, alc233_fixup_no_audio_jack_coefs);
}
+static const struct coef_fw alc256_asus_speaker_coefs[] = {
+ WRITE_COEF(0x10, 0x7f20), WRITE_COEF(0x16, 0x0c50), WRITE_COEF(0x35, 0x8d6a),
+ WRITE_COEF(0x37, 0xfe06), WRITE_COEF(0x57, 0x7f7f),
+ {}
+};
+
+static void alc256_fixup_asus_speaker_coefs(struct hda_codec *codec,
+ const struct hda_fixup *fix,
+ int action)
+{
+ /*
+ * The internal speakers of the ASUS ExpertBook PM3606CHA stay silent
+ * with the default COEF values. These are the values set by the
+ * Windows driver. They are applied on every init so that they are
+ * restored after resume, too.
+ */
+ if (action != HDA_FIXUP_ACT_INIT)
+ return;
+ alc_process_coef_fw(codec, alc256_asus_speaker_coefs);
+}
+
static void alc256_fixup_mic_no_presence_and_resume(struct hda_codec *codec,
const struct hda_fixup *fix,
int action)
@@ -4391,6 +4412,8 @@ enum {
ALC285_LENOVO_DAC_RENAME,
ALC287_FIXUP_YOGA9_SPEAKER2_TO_DAC1,
ALC256_FIXUP_IPASON_SMARTBOOK_S1,
+ ALC256_FIXUP_ASUS_SPEAKER_COEFS,
+ ALC245_FIXUP_MINISFORUM_V3_MIC_NO_PRESENCE,
};
/* A special fixup for Lenovo C940 and Yoga Duet 7;
@@ -7208,6 +7231,21 @@ static const struct hda_fixup alc269_fixups[] = {
{ }
},
},
+ [ALC256_FIXUP_ASUS_SPEAKER_COEFS] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc256_fixup_asus_speaker_coefs,
+ .chained = true,
+ .chain_id = ALC256_FIXUP_ASUS_MIC_NO_PRESENCE,
+ },
+ [ALC245_FIXUP_MINISFORUM_V3_MIC_NO_PRESENCE] = {
+ .type = HDA_FIXUP_PINS,
+ .v.pins = (const struct hda_pintbl[]) {
+ { 0x19, 0x04a19150 }, /* headset mic jack, no presence detect */
+ { }
+ },
+ .chained = true,
+ .chain_id = ALC245_FIXUP_BASS_HP_DAC
+ },
};
static const struct hda_quirk alc269_fixup_tbl[] = {
@@ -7335,6 +7373,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1028, 0x097e, "Dell Precision", ALC289_FIXUP_DUAL_SPK),
SND_PCI_QUIRK(0x1028, 0x098d, "Dell Precision", ALC233_FIXUP_ASUS_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1028, 0x09bf, "Dell Precision", ALC233_FIXUP_ASUS_MIC_NO_PRESENCE),
+ SND_PCI_QUIRK(0x1028, 0x09d1, "Alienware m17 R3", ALC289_FIXUP_DUAL_SPK),
SND_PCI_QUIRK(0x1028, 0x0a2e, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC),
SND_PCI_QUIRK(0x1028, 0x0a30, "Dell", ALC236_FIXUP_DELL_AIO_HEADSET_MIC),
SND_PCI_QUIRK(0x1028, 0x0a38, "Dell Latitude 7520", ALC269_FIXUP_DELL4_MIC_NO_PRESENCE_QUIET),
@@ -7605,7 +7644,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x103c, 0x8a2b, "HP Envy 15", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8a2c, "HP Envy 16", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8a2d, "HP Envy 16", ALC287_FIXUP_CS35L41_I2C_2),
- SND_PCI_QUIRK(0x103c, 0x8a2e, "HP Envy 16", ALC287_FIXUP_CS35L41_I2C_2),
+ SND_PCI_QUIRK(0x103c, 0x8a2e, "HP ENVY Laptop 17-cr0xxx", ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS),
SND_PCI_QUIRK(0x103c, 0x8a30, "HP Envy 17", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8a31, "HP ENVY x360 2-in-1 Laptop 15-ey0xxx", ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS),
SND_PCI_QUIRK(0x103c, 0x8a34, "HP Pavilion x360 2-in-1 Laptop 14-ek0xxx", ALC245_FIXUP_HP_MUTE_LED_COEFBIT),
@@ -7990,6 +8029,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1043, 0x31e1, "ASUS B5605CCA", ALC294_FIXUP_ASUS_CS35L41_SPI_2),
SND_PCI_QUIRK(0x1043, 0x31f1, "ASUS B3605CCA", ALC294_FIXUP_ASUS_CS35L41_SPI_2),
SND_PCI_QUIRK(0x1043, 0x3391, "ASUS PM3606CKA", ALC287_FIXUP_CS35L41_I2C_2),
+ SND_PCI_QUIRK(0x1043, 0x3501, "ASUS PM3606CHA", ALC256_FIXUP_ASUS_SPEAKER_COEFS),
SND_PCI_QUIRK(0x1043, 0x3601, "ASUS PM5406CGA", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x1043, 0x3611, "ASUS PM5606CGA", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x1043, 0x3701, "ASUS P5406CCA", ALC245_FIXUP_CS35L41_SPI_2),
@@ -8445,7 +8485,8 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1ee7, 0x2081, "HONOR MRB-XXX M1020", ALC256_FIXUP_HONOR_MRB_XXX_M1020_AUDIO),
SND_PCI_QUIRK(0x1f4c, 0xb020, "Minisforum AI X1 Pro",
ALC245_FIXUP_MINISFORUM_JACK_DETECT),
- SND_PCI_QUIRK(0x1f4c, 0xe001, "Minisforum V3 (SE)", ALC245_FIXUP_BASS_HP_DAC),
+ SND_PCI_QUIRK(0x1f4c, 0xe001, "Minisforum V3 (SE)",
+ ALC245_FIXUP_MINISFORUM_V3_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1f66, 0x0105, "Ayaneo Portable Game Player", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x2014, 0x800a, "Positivo ARN50", ALC269_FIXUP_LIMIT_INT_MIC_BOOST),
SND_PCI_QUIRK(0x2039, 0x0001, "Inspur S14-G1", ALC295_FIXUP_CHROME_BOOK),
diff --git a/sound/hda/codecs/side-codecs/tas2781_hda.c b/sound/hda/codecs/side-codecs/tas2781_hda.c
index e0da299df780..9162173ccb10 100644
--- a/sound/hda/codecs/side-codecs/tas2781_hda.c
+++ b/sound/hda/codecs/side-codecs/tas2781_hda.c
@@ -50,6 +50,31 @@ static void cali_cnv(unsigned char *data, unsigned int base, int offset)
(u32 *)®_data, TASDEV_CALIB_N);
}
+/*
+ * V1 data without CRC is only trusted if every expected device has a
+ * populated, self-consistent entry: R0 (Q27) * 1/R0 (Q31) must be
+ * close to 1.0 in Q58.
+ */
+static bool tas2781_v1_nocrc_valid(const unsigned int *v, int ndev)
+{
+ const u64 one = 1ULL << 58;
+ int i;
+
+ if (ndev <= 0 || ndev > 4)
+ return false;
+
+ for (i = 0; i < ndev; i++) {
+ const unsigned int *e = &v[i * 5];
+ u64 prod = (u64)e[0] * e[1];
+
+ if (!e[0] || !e[1] ||
+ prod < one - one / 10 || prod > one + one / 10)
+ return false;
+ }
+
+ return true;
+}
+
static void tas2781_apply_calib(struct tasdevice_priv *p)
{
struct calidata *cali_data = &p->cali_data;
@@ -145,7 +170,7 @@ static void tas2781_apply_calib(struct tasdevice_priv *p)
* but leave TimeStamp and CRC zeroed.
*/
if (!tmp_val[20] && !tmp_val[21] &&
- tmp_val[0] && tmp_val[1]) {
+ tas2781_v1_nocrc_valid(tmp_val, p->ndev)) {
dev_warn(p->dev,
"%s: V1 data without CRC, accepting\n",
__func__);
diff --git a/sound/pci/cs46xx/dsp_spos.c b/sound/pci/cs46xx/dsp_spos.c
index a77e6f5a14ad..57b0a6321e0b 100644
--- a/sound/pci/cs46xx/dsp_spos.c
+++ b/sound/pci/cs46xx/dsp_spos.c
@@ -1104,6 +1104,8 @@ int cs46xx_dsp_scb_and_task_init (struct snd_cs46xx *chip)
null_scb.entry_point = null_algorithm->address;
ins->the_null_scb = cs46xx_dsp_create_scb(chip, "nullSCB", (u32 *)&null_scb, NULL_SCB_ADDR);
+ if (!ins->the_null_scb)
+ return -ENOMEM;
ins->the_null_scb->task_entry = null_algorithm;
ins->the_null_scb->sub_list_ptr = ins->the_null_scb;
ins->the_null_scb->next_scb_ptr = ins->the_null_scb;
diff --git a/sound/pci/cs46xx/dsp_spos_scb_lib.c b/sound/pci/cs46xx/dsp_spos_scb_lib.c
index fd19365026b4..c5208e42cde9 100644
--- a/sound/pci/cs46xx/dsp_spos_scb_lib.c
+++ b/sound/pci/cs46xx/dsp_spos_scb_lib.c
@@ -274,7 +274,8 @@ _dsp_create_generic_scb (struct snd_cs46xx *chip, char * name, u32 * scb_data, u
dev_dbg(chip->card->dev, "dsp_spos: creating SCB <%s>\n", name);
scb = cs46xx_dsp_create_scb(chip,name,scb_data,dest);
-
+ if (!scb)
+ return NULL;
scb->sub_list_ptr = ins->the_null_scb;
scb->next_scb_ptr = ins->the_null_scb;
diff --git a/sound/soc/amd/yc/acp6x-mach.c b/sound/soc/amd/yc/acp6x-mach.c
index 1be8504571ad..96b0bfeddc06 100644
--- a/sound/soc/amd/yc/acp6x-mach.c
+++ b/sound/soc/amd/yc/acp6x-mach.c
@@ -745,6 +745,13 @@ static const struct dmi_system_id yc_acp_quirk_table[] = {
DMI_MATCH(DMI_BOARD_NAME, "8BD6"),
}
},
+ {
+ .driver_data = &acp6x_card,
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "HP"),
+ DMI_MATCH(DMI_BOARD_NAME, "8EE3"),
+ }
+ },
{
.driver_data = &acp6x_card,
.matches = {
diff --git a/sound/soc/codecs/cs35l56-sdw.c b/sound/soc/codecs/cs35l56-sdw.c
index e10f04d677f4..a49ad69644b5 100644
--- a/sound/soc/codecs/cs35l56-sdw.c
+++ b/sound/soc/codecs/cs35l56-sdw.c
@@ -9,6 +9,7 @@
#include <linux/device.h>
#include <linux/err.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/pm_runtime.h>
#include <linux/regmap.h>
#include <linux/soundwire/sdw.h>
@@ -36,6 +37,9 @@
#define CS35L56_LATE_READ_POLL_US 10
#define CS35L56_LATE_READ_TIMEOUT_US 1000
+static DEFINE_MUTEX(cs35l56_fw_idle_wait_lock);
+static bool cs35l56_fw_idle_wait_pending;
+
static int cs35l56_sdw_poll_mem_status(struct sdw_slave *peripheral,
unsigned int mask,
unsigned int match)
@@ -387,14 +391,34 @@ static int __maybe_unused cs35l56_sdw_runtime_resume(struct device *dev)
return 0;
}
+static int cs35l56_sdw_system_suspend_prepare(struct device *dev)
+{
+ cs35l56_fw_idle_wait_pending = true;
+
+ return 0;
+}
+
static int __maybe_unused cs35l56_sdw_system_suspend(struct device *dev)
{
struct cs35l56_private *cs35l56 = dev_get_drvdata(dev);
+ int ret;
if (cs35l56->sdw_attached)
cs35l56_mask_soundwire_interrupts(cs35l56);
- return cs35l56_system_suspend(dev);
+ ret = cs35l56_system_suspend(dev);
+ if (ret < 0)
+ return ret;
+
+ scoped_guard(mutex, &cs35l56_fw_idle_wait_lock) {
+ if (cs35l56_fw_idle_wait_pending && cs35l56->needs_wait_for_fw_idle) {
+ dev_dbg(cs35l56->base.dev, "Wait for FW timer expiry\n");
+ msleep(CS35L56_FW_REQ_ACTIVE_TIMEOUT_MS + 10);
+ cs35l56_fw_idle_wait_pending = false;
+ }
+ }
+
+ return 0;
}
static int cs35l56_sdw_probe(struct sdw_slave *peripheral, const struct sdw_device_id *id)
@@ -460,6 +484,7 @@ static void cs35l56_sdw_remove(struct sdw_slave *peripheral)
}
static const struct dev_pm_ops cs35l56_sdw_pm = {
+ .prepare = cs35l56_sdw_system_suspend_prepare,
SET_RUNTIME_PM_OPS(cs35l56_sdw_runtime_suspend, cs35l56_sdw_runtime_resume, NULL)
SYSTEM_SLEEP_PM_OPS(cs35l56_sdw_system_suspend, cs35l56_system_resume)
LATE_SYSTEM_SLEEP_PM_OPS(cs35l56_system_suspend_late, cs35l56_system_resume_early)
diff --git a/sound/soc/codecs/cs35l56-shared.c b/sound/soc/codecs/cs35l56-shared.c
index 7b3e37d462d6..b37f65cd4d33 100644
--- a/sound/soc/codecs/cs35l56-shared.c
+++ b/sound/soc/codecs/cs35l56-shared.c
@@ -1379,6 +1379,44 @@ void cs35l56_warn_if_firmware_missing(struct cs35l56_base *cs35l56_base)
}
EXPORT_SYMBOL_NS_GPL(cs35l56_warn_if_firmware_missing, "SND_SOC_CS35L56_SHARED");
+bool cs35l56_needs_wait_for_firmware_timer_expiry(struct cs35l56_base *cs35l56_base)
+{
+ unsigned int fw_ver;
+ bool prot_sts;
+ int ret;
+
+ switch (cs35l56_base->type) {
+ case 0x54:
+ case 0x56:
+ case 0x57:
+ switch (cs35l56_base->rev) {
+ case 0xb0:
+ ret = cs35l56_read_prot_status(cs35l56_base, &prot_sts, &fw_ver);
+ if (ret)
+ return true;
+
+ if ((fw_ver & CS35L56_FW_MAIN_VERSION_MASK) < 0x30d07)
+ return true;
+
+ return false;
+ case 0xb2:
+ ret = cs35l56_read_prot_status(cs35l56_base, &prot_sts, &fw_ver);
+ if (ret)
+ return true;
+
+ if ((fw_ver & CS35L56_FW_MAIN_VERSION_MASK) < 0x40710)
+ return true;
+
+ return false;
+ default:
+ return false;
+ }
+ default:
+ return false;
+ }
+}
+EXPORT_SYMBOL_NS_GPL(cs35l56_needs_wait_for_firmware_timer_expiry, "SND_SOC_CS35L56_SHARED");
+
void cs35l56_log_tuning(struct cs35l56_base *cs35l56_base, struct cs_dsp *cs_dsp)
{
__be32 pid, sid, tid;
diff --git a/sound/soc/codecs/cs35l56.c b/sound/soc/codecs/cs35l56.c
index 279db3f13f1e..573cc71a1910 100644
--- a/sound/soc/codecs/cs35l56.c
+++ b/sound/soc/codecs/cs35l56.c
@@ -1693,6 +1693,10 @@ int cs35l56_system_resume(struct device *dev)
dev_dbg(dev, "system_resume\n");
+ /* Assume the firmware reset to default until we can check it */
+ if (cs35l56->base.init_done)
+ cs35l56->needs_wait_for_fw_idle = true;
+
/*
* We might have done a hard reset or the CS35L56 was power-cycled
* so wait for control port to be ready.
@@ -1707,6 +1711,16 @@ int cs35l56_system_resume(struct device *dev)
if (ret)
return ret;
+ if (cs35l56->sdw_peripheral && cs35l56->base.init_done) {
+ ret = pm_runtime_resume_and_get(cs35l56->base.dev);
+ if (ret < 0)
+ return ret;
+
+ cs35l56->needs_wait_for_fw_idle =
+ cs35l56_needs_wait_for_firmware_timer_expiry(&cs35l56->base);
+ pm_runtime_put_autosuspend(cs35l56->base.dev);
+ }
+
/* Firmware won't have been loaded if the component hasn't probed */
if (!cs35l56->component)
return 0;
@@ -2231,6 +2245,9 @@ int cs35l56_init(struct cs35l56_private *cs35l56)
if (ret)
return dev_err_probe(cs35l56->base.dev, ret, "Failed to write ASP1_CONTROL3\n");
+ cs35l56->needs_wait_for_fw_idle =
+ cs35l56_needs_wait_for_firmware_timer_expiry(&cs35l56->base);
+
cs35l56->base.init_done = true;
complete_all(&cs35l56->init_completion);
diff --git a/sound/soc/codecs/cs35l56.h b/sound/soc/codecs/cs35l56.h
index f7cf8aa653e2..b46f410f0e09 100644
--- a/sound/soc/codecs/cs35l56.h
+++ b/sound/soc/codecs/cs35l56.h
@@ -43,6 +43,7 @@ struct cs35l56_private {
bool soft_resetting;
bool sdw_attached;
bool component_registered;
+ bool needs_wait_for_fw_idle;
struct completion init_completion;
int speaker_id;
diff --git a/sound/soc/codecs/hdac_hda.c b/sound/soc/codecs/hdac_hda.c
index 40d602f79314..9e45b20940f2 100644
--- a/sound/soc/codecs/hdac_hda.c
+++ b/sound/soc/codecs/hdac_hda.c
@@ -633,12 +633,35 @@ static const struct snd_soc_component_driver hdac_hda_hdmi_codec = {
.endianness = 1,
};
+static void hdac_hda_restore_dev_type(void *data)
+{
+ struct hdac_device *hdev = data;
+
+ hdev->type = HDA_DEV_LEGACY;
+}
+
static int hdac_hda_dev_probe(struct hdac_device *hdev)
{
- struct hdac_hda_priv *hda_pvt = dev_get_drvdata(&hdev->dev);
+ struct hdac_hda_priv *hda_pvt;
struct hdac_ext_link *hlink;
int ret;
+ hda_pvt = devm_kzalloc(&hdev->dev, sizeof(*hda_pvt), GFP_KERNEL);
+ if (!hda_pvt)
+ return -ENOMEM;
+
+ hda_pvt->codec = dev_to_hda_codec(&hdev->dev);
+ hda_pvt->dev_index = hdev->addr;
+ /* Intel iDisp (HDMI/DP) codec */
+ if (((hdev->vendor_id >> 16) & 0xFFFF) == 0x8086)
+ hda_pvt->need_display_power = true;
+ dev_set_drvdata(&hdev->dev, hda_pvt);
+
+ /* Allow the legacy codec driver to match again after unbind */
+ ret = devm_add_action(&hdev->dev, hdac_hda_restore_dev_type, hdev);
+ if (ret)
+ return ret;
+
/* hold the ref while we probe */
hlink = snd_hdac_ext_bus_get_hlink_by_name(hdev->bus, dev_name(&hdev->dev));
if (!hlink) {
diff --git a/sound/soc/codecs/tas2770.c b/sound/soc/codecs/tas2770.c
index e79754edac5c..355fcbfd01c7 100644
--- a/sound/soc/codecs/tas2770.c
+++ b/sound/soc/codecs/tas2770.c
@@ -513,7 +513,7 @@ static int tas2770_set_dai_tdm_idle(struct snd_soc_dai *dai,
ret = snd_soc_component_update_bits(component, TAS2770_DIN_PD,
TAS2770_DIN_PD_SDOUT,
TAS2770_DIN_PD_SDOUT);
- if (ret)
+ if (ret < 0)
return ret;
break;
@@ -521,12 +521,12 @@ static int tas2770_set_dai_tdm_idle(struct snd_soc_dai *dai,
ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4,
TAS2770_TDM_CFG_REG4_TX_KEEPER,
TAS2770_TDM_CFG_REG4_TX_KEEPER);
- if (ret)
+ if (ret < 0)
return ret;
ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4,
TAS2770_TDM_CFG_REG4_TX_FILL, 0);
- if (ret)
+ if (ret < 0)
return ret;
break;
@@ -534,25 +534,25 @@ static int tas2770_set_dai_tdm_idle(struct snd_soc_dai *dai,
ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4,
TAS2770_TDM_CFG_REG4_TX_KEEPER,
TAS2770_TDM_CFG_REG4_TX_KEEPER);
- if (ret)
+ if (ret < 0)
return ret;
ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4,
TAS2770_TDM_CFG_REG4_TX_FILL,
TAS2770_TDM_CFG_REG4_TX_FILL);
- if (ret)
+ if (ret < 0)
return ret;
break;
case SND_SOC_DAI_TDM_IDLE_OFF:
ret = snd_soc_component_update_bits(component, TAS2770_DIN_PD,
TAS2770_DIN_PD_SDOUT, 0);
- if (ret)
+ if (ret < 0)
return ret;
ret = snd_soc_component_update_bits(component, TAS2770_TDM_CFG_REG4,
TAS2770_TDM_CFG_REG4_TX_KEEPER, 0);
- if (ret)
+ if (ret < 0)
return ret;
break;
diff --git a/sound/soc/qcom/qdsp6/q6apm.c b/sound/soc/qcom/qdsp6/q6apm.c
index a79bbb34c777..8a756249e588 100644
--- a/sound/soc/qcom/qdsp6/q6apm.c
+++ b/sound/soc/qcom/qdsp6/q6apm.c
@@ -884,8 +884,6 @@ static int apm_probe(gpr_device_t *gdev)
if (!apm)
return -ENOMEM;
- dev_set_drvdata(dev, apm);
-
mutex_init(&apm->lock);
apm->dev = dev;
apm->gdev = gdev;
@@ -899,6 +897,7 @@ static int apm_probe(gpr_device_t *gdev)
idr_init(&apm->modules_idr);
+ dev_set_drvdata(dev, apm);
g_apm = apm;
q6apm_get_apm_state(apm);
@@ -942,6 +941,9 @@ static int apm_callback(const struct gpr_resp_pkt *data, void *priv, int op)
const struct gpr_hdr *hdr = &data->hdr;
int graph_id, is_pos_buf;
+ if (!apm)
+ return 0;
+
result = data->payload;
switch (hdr->opcode) {
diff --git a/sound/soc/qcom/qdsp6/q6prm.c b/sound/soc/qcom/qdsp6/q6prm.c
index 1f3ce4cc0837..bcc73127d54d 100644
--- a/sound/soc/qcom/qdsp6/q6prm.c
+++ b/sound/soc/qcom/qdsp6/q6prm.c
@@ -192,6 +192,9 @@ static int prm_callback(const struct gpr_resp_pkt *data, void *priv, int op)
const struct gpr_ibasic_rsp_result_t *result;
const struct gpr_hdr *hdr = &data->hdr;
+ if (!prm)
+ return 0;
+
switch (hdr->opcode) {
case PRM_CMD_RSP_REQUEST_HW_RSC:
case PRM_CMD_RSP_RELEASE_HW_RSC:
diff --git a/sound/soc/sof/intel/hda-codec.c b/sound/soc/sof/intel/hda-codec.c
index fd371850b0d6..32fcc1cd3f92 100644
--- a/sound/soc/sof/intel/hda-codec.c
+++ b/sound/soc/sof/intel/hda-codec.c
@@ -15,8 +15,6 @@
#include "hda.h"
#if IS_ENABLED(CONFIG_SND_SOC_SOF_HDA_AUDIO_CODEC)
-#include "../../codecs/hdac_hda.h"
-
#define CODEC_PROBE_RETRIES 3
#define IDISP_VID_INTEL 0x80860000
@@ -148,7 +146,6 @@ static struct hda_codec *hda_codec_device_init(struct hdac_bus *bus, int addr, i
/* probe individual codec */
static int hda_codec_probe(struct snd_sof_dev *sdev, int address)
{
- struct hdac_hda_priv *hda_priv;
struct hda_bus *hbus = sof_to_hbus(sdev);
struct hda_codec *codec;
u32 hda_cmd = (address << 28) | (AC_NODE_ROOT << 20) |
@@ -168,27 +165,15 @@ static int hda_codec_probe(struct snd_sof_dev *sdev, int address)
dev_dbg(sdev->dev, "HDA codec #%d probed OK: response: %x\n",
address, resp);
- hda_priv = devm_kzalloc(sdev->dev, sizeof(*hda_priv), GFP_KERNEL);
- if (!hda_priv)
- return -ENOMEM;
-
codec = hda_codec_device_init(&hbus->core, address, HDA_DEV_LEGACY);
ret = PTR_ERR_OR_ZERO(codec);
if (ret < 0)
return ret;
- hda_priv->codec = codec;
- hda_priv->dev_index = address;
- dev_set_drvdata(&codec->core.dev, hda_priv);
-
- if ((resp & 0xFFFF0000) == IDISP_VID_INTEL) {
- if (!hbus->core.audio_component) {
- dev_dbg(sdev->dev,
- "iDisp hw present but no driver\n");
- ret = -ENOENT;
- goto out;
- }
- hda_priv->need_display_power = true;
+ if ((resp & 0xFFFF0000) == IDISP_VID_INTEL && !hbus->core.audio_component) {
+ dev_dbg(sdev->dev, "iDisp hw present but no driver\n");
+ ret = -ENOENT;
+ goto out;
}
if (is_generic_config(hbus))
diff --git a/sound/soc/sof/intel/hda-sdw-bpt.c b/sound/soc/sof/intel/hda-sdw-bpt.c
index ba9fbd662b85..9d97d8676251 100644
--- a/sound/soc/sof/intel/hda-sdw-bpt.c
+++ b/sound/soc/sof/intel/hda-sdw-bpt.c
@@ -456,6 +456,12 @@ int hda_sdw_bpt_close(struct device *dev, int link_id, struct hdac_ext_stream *b
int ret;
int ret1;
+ if (bpt_rx_stream && hdac_stream(bpt_rx_stream)->running)
+ hda_sdw_bpt_dma_disable(dev, bpt_rx_stream);
+
+ if (bpt_tx_stream && hdac_stream(bpt_tx_stream)->running)
+ hda_sdw_bpt_dma_disable(dev, bpt_tx_stream);
+
/*
* In the case of SoundWire we need to reset the PCMSyCM registers.
* Need to continue depreparing the DMA buffers even if this fails.
diff --git a/sound/usb/bcd2000/bcd2000.c b/sound/usb/bcd2000/bcd2000.c
index 2bd49bf82748..0fd086cac955 100644
--- a/sound/usb/bcd2000/bcd2000.c
+++ b/sound/usb/bcd2000/bcd2000.c
@@ -183,10 +183,24 @@ static int bcd2000_midi_output_open(struct snd_rawmidi_substream *substream)
static int bcd2000_midi_output_close(struct snd_rawmidi_substream *substream)
{
struct bcd2000 *bcd2k = substream->rmidi->private_data;
+ struct urb *urb = NULL;
- if (bcd2k->midi_out_active && bcd2k->midi_out_urb) {
- usb_kill_urb(bcd2k->midi_out_urb);
- bcd2k->midi_out_active = 0;
+ /*
+ * Reference the URB under the lock so that it stays valid until
+ * usb_kill_urb() returns, even if a disconnect frees it in the
+ * meantime. usb_kill_urb() cannot run under the lock, as the
+ * completion handler takes the same lock.
+ */
+ scoped_guard(spinlock_irqsave, &bcd2k->midi_lock) {
+ if (bcd2k->midi_out_active && bcd2k->midi_out_urb) {
+ urb = usb_get_urb(bcd2k->midi_out_urb);
+ bcd2k->midi_out_active = 0;
+ }
+ }
+
+ if (urb) {
+ usb_kill_urb(urb);
+ usb_put_urb(urb);
}
return 0;
diff --git a/sound/usb/caiaq/input.c b/sound/usb/caiaq/input.c
index 8d924330c54c..309a361ca9f7 100644
--- a/sound/usb/caiaq/input.c
+++ b/sound/usb/caiaq/input.c
@@ -840,6 +840,8 @@ int snd_usb_caiaq_input_init(struct snd_usb_caiaqdev *cdev)
return 0;
exit_free_idev:
+ usb_free_urb(cdev->ep4_in_urb);
+ cdev->ep4_in_urb = NULL;
input_free_device(input);
cdev->input_dev = NULL;
return ret;
diff --git a/sound/usb/line6/capture.c b/sound/usb/line6/capture.c
index ceee51eeda0d..32922b7f2ffc 100644
--- a/sound/usb/line6/capture.c
+++ b/sound/usb/line6/capture.c
@@ -94,6 +94,18 @@ void line6_capture_copy(struct snd_line6_pcm *line6pcm, char *fbuf, int fsize)
if (runtime == NULL)
return;
+ /*
+ * snd_line6_capture_open() constrains the buffer to hold at least
+ * two packets, so this cannot happen for a conforming device. Drop
+ * the packet rather than overrun the single wrap below.
+ */
+ if (unlikely(frames > runtime->buffer_size)) {
+ dev_err_ratelimited(line6pcm->line6->ifcdev,
+ "packet of %d frames exceeds %lu-frame buffer, dropping\n",
+ frames, runtime->buffer_size);
+ return;
+ }
+
if (line6pcm->in.pos_done + frames > runtime->buffer_size) {
/*
The transferred area goes over buffer boundary,
@@ -231,6 +243,23 @@ static int snd_line6_capture_open(struct snd_pcm_substream *substream)
if (err < 0)
return err;
+ /*
+ * line6_capture_copy() wraps the ring buffer once per packet and
+ * advances pos_done by the packet, which only works when the buffer
+ * holds more than one packet: with exactly one, pos_done gains a
+ * buffer and loses it again on every packet. The packet size comes
+ * from the device, so bound the buffer by two packets here. The
+ * impulse-response path copies a playback URB into this ring, so
+ * take the larger of the two endpoints.
+ */
+ err = snd_pcm_hw_constraint_minmax(runtime,
+ SNDRV_PCM_HW_PARAM_BUFFER_BYTES,
+ 2 * max(line6pcm->max_packet_size_in,
+ line6pcm->max_packet_size_out),
+ UINT_MAX);
+ if (err < 0)
+ return err;
+
line6_pcm_acquire(line6pcm, LINE6_STREAM_CAPTURE_HELPER, false);
runtime->hw = line6pcm->properties->capture_hw;
diff --git a/sound/usb/midi.c b/sound/usb/midi.c
index 7e5b1b13360f..1099e5b81411 100644
--- a/sound/usb/midi.c
+++ b/sound/usb/midi.c
@@ -113,6 +113,7 @@ struct snd_usb_midi {
unsigned int opened[2];
unsigned char disconnected;
unsigned char input_running;
+ unsigned char keep_input_running;
struct snd_kcontrol *roland_load_ctl;
};
@@ -1149,41 +1150,42 @@ static int substream_open(struct snd_rawmidi_substream *substream, int dir,
int open)
{
struct snd_usb_midi *umidi = substream->rmidi->private_data;
- struct snd_kcontrol *ctl;
+ struct snd_ctl_elem_id ctl_id;
+ bool activate_ctl = false;
+ bool active;
guard(rwsem_read)(&umidi->disc_rwsem);
if (umidi->disconnected)
return open ? -ENODEV : 0;
- guard(mutex)(&umidi->mutex);
- if (open) {
- if (!umidi->opened[0] && !umidi->opened[1]) {
- if (umidi->roland_load_ctl) {
- ctl = umidi->roland_load_ctl;
- ctl->vd[0].access |=
- SNDRV_CTL_ELEM_ACCESS_INACTIVE;
- snd_ctl_notify(umidi->card,
- SNDRV_CTL_EVENT_MASK_INFO, &ctl->id);
- update_roland_altsetting(umidi);
+ scoped_guard(mutex, &umidi->mutex) {
+ if (open) {
+ if (!umidi->opened[0] && !umidi->opened[1]) {
+ if (umidi->roland_load_ctl) {
+ ctl_id = umidi->roland_load_ctl->id;
+ activate_ctl = true;
+ active = false;
+ update_roland_altsetting(umidi);
+ }
}
- }
- umidi->opened[dir]++;
- if (umidi->opened[1])
- snd_usbmidi_input_start(&umidi->list);
- } else {
- umidi->opened[dir]--;
- if (!umidi->opened[1])
- snd_usbmidi_input_stop(&umidi->list);
- if (!umidi->opened[0] && !umidi->opened[1]) {
- if (umidi->roland_load_ctl) {
- ctl = umidi->roland_load_ctl;
- ctl->vd[0].access &=
- ~SNDRV_CTL_ELEM_ACCESS_INACTIVE;
- snd_ctl_notify(umidi->card,
- SNDRV_CTL_EVENT_MASK_INFO, &ctl->id);
+ umidi->opened[dir]++;
+ if (umidi->opened[1])
+ snd_usbmidi_input_start(&umidi->list);
+ } else {
+ umidi->opened[dir]--;
+ if (!umidi->opened[1] && !umidi->keep_input_running)
+ snd_usbmidi_input_stop(&umidi->list);
+ if (!umidi->opened[0] && !umidi->opened[1]) {
+ if (umidi->roland_load_ctl) {
+ ctl_id = umidi->roland_load_ctl->id;
+ activate_ctl = true;
+ active = true;
+ }
}
}
}
+ if (activate_ctl)
+ snd_ctl_activate_id(umidi->card, &ctl_id, active);
return 0;
}
@@ -2262,6 +2264,10 @@ static int snd_usbmidi_detect_roland(struct snd_usb_midi *umidi,
struct usb_host_interface *hostif;
u8 *cs_desc;
+ /* TD-11 becomes unusable if input isn't serviced continuously */
+ if (umidi->usb_id == USB_ID(0x0582, 0x0151))
+ umidi->keep_input_running = 1;
+
intf = umidi->iface;
if (!intf)
return -ENOENT;
@@ -2469,7 +2475,7 @@ void snd_usbmidi_input_start(struct list_head *p)
int i;
umidi = list_entry(p, struct snd_usb_midi, list);
- if (umidi->input_running || !umidi->opened[1])
+ if (umidi->input_running || (!umidi->opened[1] && !umidi->keep_input_running))
return;
for (i = 0; i < MIDI_MAX_ENDPOINTS; ++i)
snd_usbmidi_input_start_ep(umidi, umidi->endpoints[i].in);
@@ -2644,6 +2650,10 @@ int __snd_usbmidi_create(struct snd_card *card,
if (err < 0)
goto exit;
+ /* start input for devices that need continuous servicing */
+ if (umidi->keep_input_running)
+ snd_usbmidi_input_start(&umidi->list);
+
usb_autopm_get_interface_no_resume(umidi->iface);
list_add_tail(&umidi->list, midi_list);
diff --git a/sound/usb/qcom/qc_audio_offload.c b/sound/usb/qcom/qc_audio_offload.c
index e4bfd43a2488..5ee568e0454c 100644
--- a/sound/usb/qcom/qc_audio_offload.c
+++ b/sound/usb/qcom/qc_audio_offload.c
@@ -1969,8 +1969,13 @@ static int qc_usb_audio_probe(struct auxiliary_device *auxdev,
QMI_UAUDIO_STREAM_REQ_MSG_V01_MAX_MSG_LEN,
&uaudio_svc_ops_options,
&uaudio_stream_req_handlers);
+ if (ret)
+ goto free_qmi_hdl;
+
ret = qmi_add_server(svc->uaudio_svc_hdl, UAUDIO_STREAM_SERVICE_ID_V01,
UAUDIO_STREAM_SERVICE_VERS_V01, 0);
+ if (ret)
+ goto release_qmi_hdl;
uaudio_svc = svc;
@@ -1987,7 +1992,9 @@ static int qc_usb_audio_probe(struct auxiliary_device *auxdev,
release_qmi:
qc_usb_audio_cleanup_qmi_dev();
+release_qmi_hdl:
qmi_handle_release(svc->uaudio_svc_hdl);
+free_qmi_hdl:
kfree(svc->uaudio_svc_hdl);
free_svc:
kfree(svc);
diff --git a/sound/usb/usx2y/us122l.c b/sound/usb/usx2y/us122l.c
index a5db0d044ef9..9b4ec9bdc138 100644
--- a/sound/usb/usx2y/us122l.c
+++ b/sound/usb/usx2y/us122l.c
@@ -129,19 +129,26 @@ static const struct vm_operations_struct usb_stream_hwdep_vm_ops = {
static int usb_stream_hwdep_open(struct snd_hwdep *hw, struct file *file)
{
struct us122l *us122l = hw->private_data;
- struct usb_interface *iface;
+ struct usb_interface *iface, *iface0 = NULL;
if (hw->used >= 2)
return -EBUSY;
- if (!us122l->first)
- us122l->first = file;
-
if (us122l->is_us144) {
- iface = usb_ifnum_to_if(us122l->dev, 0);
- usb_autopm_get_interface(iface);
+ iface0 = usb_ifnum_to_if(us122l->dev, 0);
+ if (!iface0)
+ return -ENODEV;
}
+
iface = usb_ifnum_to_if(us122l->dev, 1);
+ if (!iface)
+ return -ENODEV;
+
+ if (!us122l->first)
+ us122l->first = file;
+
+ if (iface0)
+ usb_autopm_get_interface(iface0);
usb_autopm_get_interface(iface);
return 0;
}
@@ -153,10 +160,12 @@ static int usb_stream_hwdep_release(struct snd_hwdep *hw, struct file *file)
if (us122l->is_us144) {
iface = usb_ifnum_to_if(us122l->dev, 0);
- usb_autopm_put_interface(iface);
+ if (iface)
+ usb_autopm_put_interface(iface);
}
iface = usb_ifnum_to_if(us122l->dev, 1);
- usb_autopm_put_interface(iface);
+ if (iface)
+ usb_autopm_put_interface(iface);
if (us122l->first == file)
us122l->first = NULL;
guard(mutex)(&us122l->mutex);
@@ -179,6 +188,8 @@ static int usb_stream_hwdep_mmap(struct snd_hwdep *hw,
offset = area->vm_pgoff << PAGE_SHIFT;
guard(mutex)(&us122l->mutex);
s = us122l->sk.s;
+ if (!s)
+ return -ENODEV;
read = offset < s->read_size;
if (read) {
if (area->vm_flags & VM_WRITE)
diff --git a/sound/usb/usx2y/usX2Yhwdep.c b/sound/usb/usx2y/usX2Yhwdep.c
index 7f153a2fcdd8..3800c5b5887c 100644
--- a/sound/usb/usx2y/usX2Yhwdep.c
+++ b/sound/usb/usx2y/usX2Yhwdep.c
@@ -179,6 +179,9 @@ static int snd_usx2y_hwdep_dsp_load(struct snd_hwdep *hw,
int lret, err;
char *buf;
+ if (priv->chip_status & USX2Y_STAT_CHIP_HUP)
+ return -ENODEV;
+
buf = memdup_user(dsp->image, dsp->length);
if (IS_ERR(buf))
return PTR_ERR(buf);
@@ -201,11 +204,13 @@ static int snd_usx2y_hwdep_dsp_load(struct snd_hwdep *hw,
err = usx2y_in04_init(priv);
if (err) {
dev_err(&dev->dev, "usx2y_in04_init error\n");
+ priv->chip_status |= USX2Y_STAT_CHIP_HUP;
return err;
}
err = usx2y_create_alsa_devices(hw->card);
if (err) {
dev_err(&dev->dev, "usx2y_create_alsa_devices error %i\n", err);
+ priv->chip_status |= USX2Y_STAT_CHIP_HUP;
return err;
}
priv->chip_status |= USX2Y_STAT_CHIP_INIT;
diff --git a/sound/usb/usx2y/usb_stream.c b/sound/usb/usx2y/usb_stream.c
index 3122cf653273..7d621280fba3 100644
--- a/sound/usb/usx2y/usb_stream.c
+++ b/sound/usb/usx2y/usb_stream.c
@@ -682,6 +682,10 @@ int usb_stream_start(struct usb_stream_kernel *sk)
s->sync_packet = -1;
s->inpacket_head = -1;
sk->iso_frame_balance = 0;
+ sk->idle_inurb = sk->inurb[USB_STREAM_NURBS - 2];
+ sk->idle_outurb = sk->outurb[USB_STREAM_NURBS - 2];
+ sk->completed_inurb = sk->inurb[USB_STREAM_NURBS - 1];
+ sk->completed_outurb = sk->outurb[USB_STREAM_NURBS - 1];
++try;
for (u = 0; u < 2; u++) {
struct urb *inurb = sk->inurb[u];
@@ -739,11 +743,6 @@ int usb_stream_start(struct usb_stream_kernel *sk)
return -EFAULT;
}
- sk->idle_inurb = sk->inurb[USB_STREAM_NURBS - 2];
- sk->idle_outurb = sk->outurb[USB_STREAM_NURBS - 2];
- sk->completed_inurb = sk->inurb[USB_STREAM_NURBS - 1];
- sk->completed_outurb = sk->outurb[USB_STREAM_NURBS - 1];
-
/* wait, check */
{
int wait_ms = 3000;
diff --git a/sound/usb/usx2y/usbusx2yaudio.c b/sound/usb/usx2y/usbusx2yaudio.c
index a3a0bc13632b..798c9bb5386a 100644
--- a/sound/usb/usx2y/usbusx2yaudio.c
+++ b/sound/usb/usx2y/usbusx2yaudio.c
@@ -51,6 +51,18 @@ static int usx2y_urb_capt_retire(struct snd_usx2y_substream *subs)
dev_dbg(&usx2y->dev->dev, "%s: 0 == len ERROR!\n", __func__);
continue;
}
+ /*
+ * snd_usx2y_pcm_open() constrains the buffer to hold at least
+ * two URBs, so this cannot happen for a device that keeps its
+ * packets at 50 frames. Drop the packet rather than exceed the
+ * single wrap below.
+ */
+ if (unlikely(len > runtime->buffer_size)) {
+ dev_err_ratelimited(&usx2y->dev->dev,
+ "%s: packet of %i frames exceeds %lu-frame buffer, dropping\n",
+ __func__, len, runtime->buffer_size);
+ continue;
+ }
/* copy a data chunk */
if ((hwptr_done + len) > runtime->buffer_size) {
@@ -106,14 +118,31 @@ static int usx2y_urb_play_prepare(struct snd_usx2y_substream *subs,
__func__, counts);
return -EPIPE;
}
+ if (counts * usx2y->stride > subs->maxpacksize) {
+ dev_err_ratelimited(&usx2y->dev->dev,
+ "%s: packet of %i bytes exceeds maxpacksize %u\n",
+ __func__, counts * usx2y->stride,
+ subs->maxpacksize);
+ return -EPIPE;
+ }
/* set up descriptor */
urb->iso_frame_desc[pack].offset = pack ?
urb->iso_frame_desc[pack - 1].offset +
urb->iso_frame_desc[pack - 1].length :
0;
- urb->iso_frame_desc[pack].length = cap_urb->iso_frame_desc[pack].actual_length;
+ urb->iso_frame_desc[pack].length = counts * usx2y->stride;
}
if (atomic_read(&subs->state) >= STATE_PRERUNNING) {
+ /*
+ * The copy below wraps once, which needs count <= buffer_size.
+ * snd_usx2y_pcm_open() guarantees it; guard anyway.
+ */
+ if (unlikely(count > runtime->buffer_size)) {
+ dev_err_ratelimited(&usx2y->dev->dev,
+ "%s: URB of %i frames exceeds %lu-frame buffer, stopping\n",
+ __func__, count, runtime->buffer_size);
+ return -EPIPE;
+ }
if (subs->hwptr + count > runtime->buffer_size) {
/* err, the transferred area goes over buffer boundary.
* copy the data to the temp buffer.
@@ -888,6 +917,18 @@ static int snd_usx2y_pcm_open(struct snd_pcm_substream *substream)
runtime->private_data = subs;
subs->pcm_substream = substream;
snd_pcm_hw_constraint_minmax(runtime, SNDRV_PCM_HW_PARAM_PERIOD_TIME, 1000, 200000);
+ /*
+ * Capture wraps the ring buffer once per packet and playback once
+ * per URB of nr_of_packs() packets of at most 50 frames (see
+ * usx2y_urb_play_prepare()). Playback needs the buffer to hold more
+ * than one URB and capture more than one packet; use the URB bound
+ * for both, and two URBs rather than one, because the position only
+ * advances when a URB is less than the whole buffer. The bound is in
+ * frames and does not depend on the altsetting, which is selected
+ * in prepare.
+ */
+ snd_pcm_hw_constraint_minmax(runtime, SNDRV_PCM_HW_PARAM_BUFFER_SIZE,
+ 2 * nr_of_packs() * 50, UINT_MAX);
return 0;
}
diff --git a/tools/perf/tests/tool_pmu.c b/tools/perf/tests/tool_pmu.c
index c6c5ebf0e935..667160f9e2cd 100644
--- a/tools/perf/tests/tool_pmu.c
+++ b/tools/perf/tests/tool_pmu.c
@@ -1,8 +1,16 @@
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+#include <errno.h>
+#include <fcntl.h>
+#include <string.h>
+#include <unistd.h>
+#include <internal/xyarray.h>
+#include "cpumap.h"
#include "debug.h"
#include "evlist.h"
+#include "evsel.h"
#include "parse-events.h"
#include "tests.h"
+#include "thread_map.h"
#include "tool_pmu.h"
static int do_test(enum tool_pmu_event ev, bool with_pmu)
@@ -100,9 +108,80 @@ static int test__tool_pmu_with_pmu(struct test_suite *test __maybe_unused,
return TEST_OK;
}
+static int test__tool_pmu_incremental_open_unwind(struct test_suite *test __maybe_unused,
+ int subtest __maybe_unused)
+{
+ struct evlist *evlist = evlist__new();
+ struct parse_events_error err;
+ struct perf_cpu_map *cpus = NULL;
+ struct perf_thread_map *threads = NULL;
+ struct evsel *evsel;
+ int ret = TEST_FAIL, fd0 = -1;
+
+ if (!evlist)
+ return TEST_FAIL;
+
+ parse_events_error__init(&err);
+ if (parse_events(evlist, "tool/user_time/", &err)) {
+ parse_events_error__exit(&err);
+ evlist__put(evlist);
+ return TEST_FAIL;
+ }
+ parse_events_error__exit(&err);
+
+ evsel = evlist__first(evlist);
+ cpus = perf_cpu_map__new("0,1");
+ threads = thread_map__new_by_tid(getpid());
+ if (!cpus || !threads)
+ goto out;
+
+ /* Step 1: Open CPU index 0 successfully */
+ if (evsel__open_per_cpu_and_thread(evsel, cpus, 0, threads) < 0) {
+ pr_debug("Failed to open CPU index 0\n");
+ goto out;
+ }
+
+ fd0 = (*(int *)xyarray__entry(evsel->core.fd, 0, 0));
+ if (fd0 < 0 || fcntl(fd0, F_GETFD) < 0) {
+ pr_debug("CPU index 0 FD is invalid (%d)\n", fd0);
+ goto out;
+ }
+
+ /* Step 2: Intentionally trigger failure on CPU index 1 */
+ evsel->core.attr.sample_period = 1; /* Not supported for tool PMU -> -EINVAL */
+ if (evsel__open_per_cpu_and_thread(evsel, cpus, 1, threads) >= 0) {
+ pr_debug("Unexpected success opening CPU index 1 with sample_period=1\n");
+ goto out;
+ }
+
+ /*
+ * Step 3: Check that CPU index 0's FD was NOT destroyed by CPU index 1's unwind.
+ */
+ if ((*(int *)xyarray__entry(evsel->core.fd, 0, 0)) != fd0) {
+ pr_debug("FAILED: CPU 0 FD overwritten: FD(evsel, 0, 0)=%d, expected %d\n",
+ (*(int *)xyarray__entry(evsel->core.fd, 0, 0)), fd0);
+ goto out;
+ }
+
+ if (fcntl(fd0, F_GETFD) < 0) {
+ pr_debug("FAILED: CPU 0 FD %d was closed by error unwind! errno=%d (%s)\n",
+ fd0, errno, strerror(errno));
+ goto out;
+ }
+
+ ret = TEST_OK;
+out:
+ evsel__close(evsel);
+ perf_cpu_map__put(cpus);
+ perf_thread_map__put(threads);
+ evlist__put(evlist);
+ return ret;
+}
+
static struct test_case tests__tool_pmu[] = {
TEST_CASE("Parsing without PMU name", tool_pmu_without_pmu),
TEST_CASE("Parsing with PMU name", tool_pmu_with_pmu),
+ TEST_CASE("Incremental open error unwind boundary", tool_pmu_incremental_open_unwind),
{ .name = NULL, }
};
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index d4cb455f4a7d..5db43520f25e 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -2974,6 +2974,19 @@ static bool evsel__detect_missing_features(struct evsel *evsel, struct perf_cpu
return false;
}
+static void evsel__close_fds(struct evsel *evsel, int start_cpu_map_idx,
+ int cpu_map_idx, int thread, int nthreads)
+{
+ do {
+ while (--thread >= 0) {
+ if (FD(evsel, cpu_map_idx, thread) >= 0)
+ close(FD(evsel, cpu_map_idx, thread));
+ FD(evsel, cpu_map_idx, thread) = -1;
+ }
+ thread = nthreads;
+ } while (--cpu_map_idx >= start_cpu_map_idx);
+}
+
static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,
struct perf_thread_map *threads,
int start_cpu_map_idx, int end_cpu_map_idx)
@@ -3109,8 +3122,14 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,
if (err == -EMFILE && rlimit__increase_nofile(&set_rlimit))
goto retry_open;
- if (err == -EINVAL && evsel__detect_missing_features(evsel, cpu))
+ if (err == -EINVAL && evsel__detect_missing_features(evsel, cpu)) {
+ /*
+ * The restart opens the events again, so close what this
+ * attempt opened before its entries are overwritten.
+ */
+ evsel__close_fds(evsel, start_cpu_map_idx, idx, thread, nthreads);
goto fallback_missing_features;
+ }
if (evsel__precise_ip_fallback(evsel))
goto retry_open;
@@ -3120,14 +3139,7 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,
threads->err_thread = thread;
old_errno = errno;
- do {
- while (--thread >= 0) {
- if (FD(evsel, idx, thread) >= 0)
- close(FD(evsel, idx, thread));
- FD(evsel, idx, thread) = -1;
- }
- thread = nthreads;
- } while (--idx >= 0);
+ evsel__close_fds(evsel, start_cpu_map_idx, idx, thread, nthreads);
errno = old_errno;
out:
if (err)
diff --git a/tools/perf/util/hwmon_pmu.c b/tools/perf/util/hwmon_pmu.c
index ed544dca70c3..96aeb379e169 100644
--- a/tools/perf/util/hwmon_pmu.c
+++ b/tools/perf/util/hwmon_pmu.c
@@ -803,7 +803,7 @@ int evsel__hwmon_pmu_open(struct evsel *evsel,
FD(evsel, idx, thread) = -1;
}
thread = nthreads;
- } while (--idx >= 0);
+ } while (--idx >= start_cpu_map_idx);
close(dir);
return err;
}
diff --git a/tools/perf/util/tool_pmu.c b/tools/perf/util/tool_pmu.c
index 5c30854b4644..d2f7591747d9 100644
--- a/tools/perf/util/tool_pmu.c
+++ b/tools/perf/util/tool_pmu.c
@@ -349,7 +349,7 @@ int evsel__tool_pmu_open(struct evsel *evsel,
FD(evsel, idx, thread) = -1;
}
thread = nthreads;
- } while (--idx >= 0);
+ } while (--idx >= start_cpu_map_idx);
errno = old_errno;
return err;
}
diff --git a/tools/testing/selftests/filesystems/fscontext_ns/fscontext_ns_test.c b/tools/testing/selftests/filesystems/fscontext_ns/fscontext_ns_test.c
index 0f30a3e6e197..535bf1f4e1a3 100644
--- a/tools/testing/selftests/filesystems/fscontext_ns/fscontext_ns_test.c
+++ b/tools/testing/selftests/filesystems/fscontext_ns/fscontext_ns_test.c
@@ -193,7 +193,7 @@ FIXTURE_VARIANT_ADD(fscontext_ns, binfmt_misc) {
FIXTURE_VARIANT_ADD(fscontext_ns, overlay) {
.fsname = "overlay",
- .expected_errno = EIO,
+ .expected_errno = EINVAL,
};
TEST_F(fscontext_ns, create_from_descendant_userns)
diff --git a/tools/testing/selftests/net/fib_nexthops.sh b/tools/testing/selftests/net/fib_nexthops.sh
index 431d7bed7622..adbec51917eb 100755
--- a/tools/testing/selftests/net/fib_nexthops.sh
+++ b/tools/testing/selftests/net/fib_nexthops.sh
@@ -425,6 +425,14 @@ stop_ip_monitor()
return $rc
}
+fdb_entry_aged()
+{
+ local mac=$1
+ local dev=$2
+
+ ! $BRIDGE fdb get to $mac dev $dev self &> /dev/null
+}
+
check_nexthop_fdb_support()
{
$IP nexthop help 2>&1 | grep -q fdb
@@ -538,14 +546,18 @@ ipv6_fdb_grp_fcnal()
log_test $? 0 "Fdb mac add with nexthop group and static state"
run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic"
- log_test $? 255 "Fdb mac add with nexthop group and dynamic state"
+ log_test $? 0 "Fdb mac add with nexthop group and dynamic state"
+
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self dynamic extern_learn"
+ log_test $? 0 "Fdb mac add with nexthop group and extern_learn"
- run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self"
- run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic"
- log_test $? 255 "Fdb mac replace with nexthop group and dynamic state"
+ run_cmd "$IP link set dev vx10 up type vxlan ageing 1"
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:18 dev vx10 dst 2001:db8:91::11 self dynamic"
+ slowwait 20 fdb_entry_aged 02:02:00:00:00:18 vx10
+ log_test $? 0 "Fdb mac without nexthop group aged out"
- run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic"
- log_test $? 255 "Fdb mac append with nexthop group and dynamic state"
+ run_cmd "$BRIDGE fdb get to 02:02:00:00:00:16 dev vx10 self"
+ log_test $? 0 "Fdb mac with nexthop group not aged out"
run_cmd "$IP -6 ro add 2001:db8:101::1/128 nhid 66"
log_test $? 2 "Route add with fdb nexthop"
@@ -688,14 +700,18 @@ ipv4_fdb_grp_fcnal()
log_test $? 0 "Fdb mac add with nexthop group and static state"
run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic"
- log_test $? 255 "Fdb mac add with nexthop group and dynamic state"
+ log_test $? 0 "Fdb mac add with nexthop group and dynamic state"
+
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self dynamic extern_learn"
+ log_test $? 0 "Fdb mac add with nexthop group and extern_learn"
- run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self"
- run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic"
- log_test $? 255 "Fdb mac replace with nexthop group and dynamic state"
+ run_cmd "$IP link set dev vx10 up type vxlan ageing 1"
+ run_cmd "$BRIDGE fdb add 02:02:00:00:00:18 dev vx10 dst 10.0.0.3 self dynamic"
+ slowwait 20 fdb_entry_aged 02:02:00:00:00:18 vx10
+ log_test $? 0 "Fdb mac without nexthop group aged out"
- run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic"
- log_test $? 255 "Fdb mac append with nexthop group and dynamic state"
+ run_cmd "$BRIDGE fdb get to 02:02:00:00:00:16 dev vx10 self"
+ log_test $? 0 "Fdb mac with nexthop group not aged out"
run_cmd "$IP ro add 172.16.0.0/22 nhid 16"
log_test $? 2 "Route add with fdb nexthop"
^ permalink raw reply [flat|nested] 2+ messages in thread