From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 03B4F3B9DA2; Wed, 19 Aug 2026 16:52:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787158369; cv=none; b=D1Axt3zbhtzFeOG25paiyt8iB+YNpt6r2Rjv/1Dncd38jorNIDpBmhixxkpic+pSVFdHKmfRwu7qvOZKpghQkYFAezLCKsJLtKr2KMXPPA7ku3keTYDnaVRrIibGvsiOIG7gAMJDwbdXXzO4UxJVh3BCVMUhMYnID62Luuq2Bgk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787158369; c=relaxed/simple; bh=eN9Gl9UYMWpkDoAqJayGy+S2jWAvYtO3SfCj5ifqyWA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DX4034qxL8UkwbMu9QmpqC7VpmmIN2LP1GOL/tTTimL3+UqmbLmKBM0Uw0A9OLdHZzCkOg007y+P0PK+dYd4h+rk6j6tnxSkgILsXnS6drS6qL7JDGZ1UlBPwElS25crNhss+YPq8fWypDDkiu08zbKmAsrVqEL6PrHzvzIF9Pc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b=TBaedlK4; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="TBaedlK4" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C383B1F00A3A; Wed, 19 Aug 2026 16:52:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=korg; t=1787158354; bh=NjvKyDkY7EDi8KPQElERsbg6QXiqakVa+XYwm2bVmBI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=TBaedlK4oj8s0qHN5Rj0s1AvwB1oxt4hp9FZlLUBho4dibTRRieBgwMx3jLSi6xcl fhCzQNjTj6QlY+ihTbrg8OgAYGAjxIjPcD3dZpXZS6cHAIDRWM0A4k26CpYPLXm/0w XAY4/D7as9LNT5IInqXgJJEdxykPdXojoCNlW7vo= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org, akpm@linux-foundation.org, torvalds@linux-foundation.org, stable@vger.kernel.org Cc: lwn@lwn.net, jslaby@suse.cz, Greg Kroah-Hartman Subject: Re: Linux 6.6.152 Date: Wed, 19 Aug 2026 18:52:08 +0200 Message-ID: <2026081908-deity-carol-8fd7@gregkh> X-Mailer: git-send-email 2.55.0 In-Reply-To: <2026081908-empirical-reoccur-9c01@gregkh> References: <2026081908-empirical-reoccur-9c01@gregkh> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit diff --git a/Documentation/devicetree/bindings/crypto/qcom,inline-crypto-engine.yaml b/Documentation/devicetree/bindings/crypto/qcom,inline-crypto-engine.yaml index 7da9aa82d837..69731ff62c65 100644 --- a/Documentation/devicetree/bindings/crypto/qcom,inline-crypto-engine.yaml +++ b/Documentation/devicetree/bindings/crypto/qcom,inline-crypto-engine.yaml @@ -21,6 +21,16 @@ properties: maxItems: 1 clocks: + minItems: 1 + maxItems: 2 + + clock-names: + minItems: 1 + items: + - const: core + - const: iface + + power-domains: maxItems: 1 required: @@ -38,6 +48,10 @@ examples: compatible = "qcom,sm8550-inline-crypto-engine", "qcom,inline-crypto-engine"; reg = <0x01d88000 0x8000>; - clocks = <&gcc GCC_UFS_PHY_ICE_CORE_CLK>; + clocks = <&gcc GCC_UFS_PHY_ICE_CORE_CLK>, + <&gcc GCC_UFS_PHY_AHB_CLK>; + clock-names = "core", + "iface"; + power-domains = <&gcc UFS_PHY_GDSC>; }; ... diff --git a/Documentation/driver-api/driver-model/devres.rst b/Documentation/driver-api/driver-model/devres.rst index 8be086b3f829..3fe1711dd587 100644 --- a/Documentation/driver-api/driver-model/devres.rst +++ b/Documentation/driver-api/driver-model/devres.rst @@ -426,6 +426,7 @@ REGULATOR devm_regulator_bulk_put() devm_regulator_get() devm_regulator_get_enable() + devm_regulator_get_enable_read_voltage() devm_regulator_get_enable_optional() devm_regulator_get_exclusive() devm_regulator_get_optional() diff --git a/Makefile b/Makefile index 05c17f548757..13d82f5eefd2 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 VERSION = 6 PATCHLEVEL = 6 -SUBLEVEL = 151 +SUBLEVEL = 152 EXTRAVERSION = NAME = Pinguïn Aangedreven diff --git a/arch/arm/boot/dts/broadcom/bcm-ns.dtsi b/arch/arm/boot/dts/broadcom/bcm-ns.dtsi index 88fda18af1f8..06acd72f9835 100644 --- a/arch/arm/boot/dts/broadcom/bcm-ns.dtsi +++ b/arch/arm/boot/dts/broadcom/bcm-ns.dtsi @@ -131,7 +131,7 @@ axi@18000000 { /* PCIe Controller 2 */ <0x00014000 0 &gic GIC_SPI 138 IRQ_TYPE_LEVEL_HIGH>, - <0x00014000 1 &gic GIC_SPI 138 IRQ_TYPE_LEVEL_HIGH>, + <0x00014000 1 &gic GIC_SPI 139 IRQ_TYPE_LEVEL_HIGH>, <0x00014000 2 &gic GIC_SPI 140 IRQ_TYPE_LEVEL_HIGH>, <0x00014000 3 &gic GIC_SPI 141 IRQ_TYPE_LEVEL_HIGH>, <0x00014000 4 &gic GIC_SPI 142 IRQ_TYPE_LEVEL_HIGH>, diff --git a/arch/arm/mach-npcm/platsmp.c b/arch/arm/mach-npcm/platsmp.c index 41891d3aa124..4c1fc9983746 100644 --- a/arch/arm/mach-npcm/platsmp.c +++ b/arch/arm/mach-npcm/platsmp.c @@ -32,6 +32,7 @@ static int npcm7xx_smp_boot_secondary(unsigned int cpu, goto out; } gcr_base = of_iomap(gcr_np, 0); + of_node_put(gcr_np); if (!gcr_base) { pr_err("could not iomap gcr"); ret = -ENOMEM; @@ -60,6 +61,7 @@ static void __init npcm7xx_smp_prepare_cpus(unsigned int max_cpus) return; } scu_base = of_iomap(scu_np, 0); + of_node_put(scu_np); if (!scu_base) { pr_err("could not iomap scu"); return; diff --git a/arch/s390/kvm/pci.c b/arch/s390/kvm/pci.c index 17fedfc7bb99..d9fcbf41025e 100644 --- a/arch/s390/kvm/pci.c +++ b/arch/s390/kvm/pci.c @@ -167,7 +167,7 @@ static int kvm_zpci_set_airq(struct zpci_dev *zdev) fib.fmt0.noi = airq_iv_end(zdev->aibv); fib.fmt0.aibv = virt_to_phys(zdev->aibv->vector); fib.fmt0.aibvo = 0; - fib.fmt0.aisb = virt_to_phys(aift->sbv->vector + (zdev->aisb / 64) * 8); + fib.fmt0.aisb = virt_to_phys(aift->sbv->vector) + (zdev->aisb / 64) * 8; fib.fmt0.aisbo = zdev->aisb & 63; fib.gd = zdev->gisa; @@ -191,34 +191,54 @@ static int kvm_zpci_clear_airq(struct zpci_dev *zdev) return cc ? -EIO : 0; } -static inline void unaccount_mem(unsigned long nr_pages) +static inline void unaccount_mem(struct kvm_zdev *kzdev, unsigned long nr_pages) { - struct user_struct *user = get_uid(current_user()); + struct user_struct *user = kzdev->user_account; + struct mm_struct *mm_account = kzdev->mm_account; - if (user) + if (user) { atomic_long_sub(nr_pages, &user->locked_vm); - if (current->mm) - atomic64_sub(nr_pages, ¤t->mm->pinned_vm); + free_uid(user); + kzdev->user_account = NULL; + } + + if (mm_account) { + atomic64_sub(nr_pages, &mm_account->pinned_vm); + mmdrop(mm_account); + kzdev->mm_account = NULL; + } } -static inline int account_mem(unsigned long nr_pages) +static inline int account_mem(struct kvm_zdev *kzdev, unsigned long nr_pages) { struct user_struct *user = get_uid(current_user()); unsigned long page_limit, cur_pages, new_pages; + int rc = 0; page_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; + cur_pages = atomic_long_read(&user->locked_vm); do { - cur_pages = atomic_long_read(&user->locked_vm); new_pages = cur_pages + nr_pages; - if (new_pages > page_limit) - return -ENOMEM; - } while (atomic_long_cmpxchg(&user->locked_vm, cur_pages, - new_pages) != cur_pages); + if (new_pages > page_limit) { + rc = -ENOMEM; + goto out; + } + } while (!atomic_long_try_cmpxchg(&user->locked_vm, &cur_pages, new_pages)); - atomic64_add(nr_pages, ¤t->mm->pinned_vm); + if (current->mm) { + mmgrab(current->mm); + atomic64_add(nr_pages, ¤t->mm->pinned_vm); + } + + kzdev->user_account = user; + kzdev->mm_account = current->mm; return 0; + +out: + free_uid(user); + return rc; } static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, @@ -295,14 +315,17 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, } /* Account for pinned pages, roll back on failure */ - if (account_mem(pcount)) + rc = account_mem(zdev->kzdev, pcount); + if (rc) goto unpin2; /* AISB must be allocated before we can fill in GAITE */ mutex_lock(&aift->aift_lock); bit = airq_iv_alloc_bit(aift->sbv); - if (bit == -1UL) + if (bit == -1UL) { + rc = -ENOMEM; goto unlock; + } zdev->aisb = bit; /* store the summary bit number */ zdev->aibv = airq_iv_create(msi_vecs, AIRQ_IV_DATA | AIRQ_IV_BITLOCK | @@ -336,24 +359,39 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, aift->kzdev[zdev->aisb] = zdev->kzdev; spin_unlock_irq(&aift->gait_lock); - /* Update guest FIB for re-issue */ - fib->fmt0.aisbo = zdev->aisb & 63; - fib->fmt0.aisb = virt_to_phys(aift->sbv->vector + (zdev->aisb / 64) * 8); - fib->fmt0.isc = gisc; - /* Save some guest fib values in the host for later use */ - zdev->kzdev->fib.fmt0.isc = fib->fmt0.isc; + zdev->kzdev->fib.fmt0.isc = gisc; zdev->kzdev->fib.fmt0.aibv = fib->fmt0.aibv; - mutex_unlock(&aift->aift_lock); /* Issue the clp to setup the irq now */ rc = kvm_zpci_set_airq(zdev); - return rc; + if (!rc) { + mutex_unlock(&aift->aift_lock); + return rc; + } + + /* Start cleanup */ + zdev->kzdev->fib.fmt0.isc = 0; + zdev->kzdev->fib.fmt0.aibv = 0; + + spin_lock_irq(&aift->gait_lock); + gaite->count--; + gaite->aisb = 0; + gaite->gisc = 0; + gaite->aisbo = 0; + gaite->gisa = 0; + aift->kzdev[zdev->aisb] = NULL; + spin_unlock_irq(&aift->gait_lock); + + airq_iv_release(zdev->aibv); + zdev->aibv = NULL; free_aisb: airq_iv_free_bit(aift->sbv, zdev->aisb); zdev->aisb = 0; unlock: + if (pcount > 0) + unaccount_mem(zdev->kzdev, pcount); mutex_unlock(&aift->aift_lock); unpin2: if (fib->fmt0.sum == 1) @@ -424,7 +462,7 @@ static int kvm_s390_pci_aif_disable(struct zpci_dev *zdev, bool force) pcount++; } if (pcount > 0) - unaccount_mem(pcount); + unaccount_mem(kzdev, pcount); out: mutex_unlock(&aift->aift_lock); diff --git a/arch/s390/kvm/pci.h b/arch/s390/kvm/pci.h index ff0972dd5e71..fdf8c7bf4ed0 100644 --- a/arch/s390/kvm/pci.h +++ b/arch/s390/kvm/pci.h @@ -22,6 +22,8 @@ struct kvm_zdev { struct kvm *kvm; struct zpci_fib fib; struct list_head entry; + struct user_struct *user_account; + struct mm_struct *mm_account; }; struct zpci_gaite { diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 6c4b588d557f..50a8c5701084 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2299,6 +2299,9 @@ static union kvm_mmu_page_role kvm_mmu_child_role(u64 *sptep, bool direct, role.direct = direct; role.passthrough = 0; + WARN_ON_ONCE(role.invalid); + role.invalid = 0; + /* * If the guest has 4-byte PTEs then that means it's using 32-bit, * 2-level, non-PAE paging. KVM shadows such guests with PAE paging diff --git a/drivers/ata/pata_sl82c105.c b/drivers/ata/pata_sl82c105.c index 93882e976ede..2a5ce46af5a0 100644 --- a/drivers/ata/pata_sl82c105.c +++ b/drivers/ata/pata_sl82c105.c @@ -264,6 +264,7 @@ static struct ata_port_operations sl82c105_port_ops = { static int sl82c105_bridge_revision(struct pci_dev *pdev) { struct pci_dev *bridge; + u8 revision; /* * The bridge should be part of the same device, but function 0. @@ -285,8 +286,9 @@ static int sl82c105_bridge_revision(struct pci_dev *pdev) /* * We need to find function 0's revision, not function 1 */ + revision = bridge->revision; pci_dev_put(bridge); - return bridge->revision; + return revision; } static void sl82c105_fixup(struct pci_dev *pdev) diff --git a/drivers/counter/microchip-tcb-capture.c b/drivers/counter/microchip-tcb-capture.c index 461f57f66631..fbd1afe178a3 100644 --- a/drivers/counter/microchip-tcb-capture.c +++ b/drivers/counter/microchip-tcb-capture.c @@ -309,7 +309,7 @@ static int mchp_tc_probe(struct platform_device *pdev) char clk_name[7]; struct regmap *regmap; struct clk *clk[3]; - int channel; + u32 channel; int ret, i; counter = devm_counter_alloc(&pdev->dev, sizeof(*priv)); @@ -343,7 +343,7 @@ static int mchp_tc_probe(struct platform_device *pdev) priv->channel[i] = channel; - snprintf(clk_name, sizeof(clk_name), "t%d_clk", channel); + snprintf(clk_name, sizeof(clk_name), "t%u_clk", channel); clk[i] = of_clk_get_by_name(np->parent, clk_name); if (IS_ERR(clk[i])) { diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 5eea14076a92..8672838fb67c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3977,6 +3977,8 @@ static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) iounmap(adev->rmmio); adev->rmmio = NULL; + if (adev->mman.aper_base_kaddr) + iounmap(adev->mman.aper_base_kaddr); adev->mman.aper_base_kaddr = NULL; /* Memory manager related */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 1499638182bc..df23a77a07d6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -1887,23 +1887,17 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) /* Change the size here instead of the init above so only lpfn is affected */ amdgpu_ttm_set_buffer_funcs_status(adev, false); #ifdef CONFIG_64BIT - if (adev->gmc.xgmi.connected_to_cpu) { - void *kaddr = devm_memremap(adev->dev, adev->gmc.aper_base, - adev->gmc.visible_vram_size, - MEMREMAP_WB); - if (IS_ERR(kaddr)) - return PTR_ERR(kaddr); - adev->mman.aper_base_kaddr = (__force void __iomem *)kaddr; - } else if (adev->gmc.is_app_apu) { +#ifdef CONFIG_X86 + if (adev->gmc.xgmi.connected_to_cpu) + adev->mman.aper_base_kaddr = ioremap_cache(adev->gmc.aper_base, + adev->gmc.visible_vram_size); + else if (adev->gmc.is_app_apu) DRM_DEBUG_DRIVER( "No need to ioremap when real vram size is 0\n"); - } else { - adev->mman.aper_base_kaddr = devm_ioremap_wc(adev->dev, - adev->gmc.aper_base, - adev->gmc.visible_vram_size); - if (!adev->mman.aper_base_kaddr) - return -ENOMEM; - } + else +#endif + adev->mman.aper_base_kaddr = ioremap_wc(adev->gmc.aper_base, + adev->gmc.visible_vram_size); #endif /* @@ -2050,6 +2044,8 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) */ void amdgpu_ttm_fini(struct amdgpu_device *adev) { + int idx; + if (!adev->mman.initialized) return; @@ -2072,7 +2068,14 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev) amdgpu_ttm_fw_reserve_vram_fini(adev); amdgpu_ttm_drv_reserve_vram_fini(adev); - adev->mman.aper_base_kaddr = NULL; + if (drm_dev_enter(adev_to_drm(adev), &idx)) { + + if (adev->mman.aper_base_kaddr) + iounmap(adev->mman.aper_base_kaddr); + adev->mman.aper_base_kaddr = NULL; + + drm_dev_exit(idx); + } amdgpu_vram_mgr_fini(adev); amdgpu_gtt_mgr_fini(adev); diff --git a/drivers/gpu/drm/bridge/parade-ps8640.c b/drivers/gpu/drm/bridge/parade-ps8640.c index 14d4dcf239da..61ea855e55d8 100644 --- a/drivers/gpu/drm/bridge/parade-ps8640.c +++ b/drivers/gpu/drm/bridge/parade-ps8640.c @@ -258,8 +258,14 @@ static ssize_t ps8640_aux_transfer_msg(struct drm_dp_aux *aux, addr_len[PAGE0_SWAUX_LENGTH - base] = (len == 0) ? SWAUX_NO_PAYLOAD : ((len - 1) & SWAUX_LENGTH_MASK); - regmap_bulk_write(map, PAGE0_SWAUX_ADDR_7_0, addr_len, - ARRAY_SIZE(addr_len)); + ret = regmap_bulk_write(map, PAGE0_SWAUX_ADDR_7_0, addr_len, + ARRAY_SIZE(addr_len)); + if (ret) { + DRM_DEV_ERROR(dev, + "failed to write AUX address %#x, len %zu: %d\n", + msg->address, len, ret); + return ret; + } if (len && (request == DP_AUX_NATIVE_WRITE || request == DP_AUX_I2C_WRITE)) { @@ -275,13 +281,22 @@ static ssize_t ps8640_aux_transfer_msg(struct drm_dp_aux *aux, } } - regmap_write(map, PAGE0_SWAUX_CTRL, SWAUX_SEND); + ret = regmap_write(map, PAGE0_SWAUX_CTRL, SWAUX_SEND); + if (ret) { + DRM_DEV_ERROR(dev, "failed to start AUX transfer: %d\n", ret); + return ret; + } /* Zero delay loop because i2c transactions are slow already */ - regmap_read_poll_timeout(map, PAGE0_SWAUX_CTRL, data, - !(data & SWAUX_SEND), 0, 50 * 1000); + ret = regmap_read_poll_timeout(map, PAGE0_SWAUX_CTRL, data, + !(data & SWAUX_SEND), 0, 50 * 1000); + if (ret) { + DRM_DEV_ERROR(dev, "failed to complete AUX transfer: %d\n", + ret); + return ret; + } - regmap_read(map, PAGE0_SWAUX_STATUS, &data); + ret = regmap_read(map, PAGE0_SWAUX_STATUS, &data); if (ret) { DRM_DEV_ERROR(dev, "failed to read PAGE0_SWAUX_STATUS: %d\n", ret); diff --git a/drivers/hwmon/ads7828.c b/drivers/hwmon/ads7828.c index 809e830f52a6..b9ed146a24cc 100644 --- a/drivers/hwmon/ads7828.c +++ b/drivers/hwmon/ads7828.c @@ -108,12 +108,11 @@ static int ads7828_probe(struct i2c_client *client) struct ads7828_data *data; struct device *hwmon_dev; unsigned int vref_mv = ADS7828_INT_VREF_MV; - unsigned int vref_uv; + int vref_uv; bool diff_input = false; bool ext_vref = false; unsigned int regval; enum ads7828_chips chip; - struct regulator *reg; data = devm_kzalloc(dev, sizeof(struct ads7828_data), GFP_KERNEL); if (!data) @@ -127,9 +126,11 @@ static int ads7828_probe(struct i2c_client *client) } else if (dev->of_node) { diff_input = of_property_read_bool(dev->of_node, "ti,differential-input"); - reg = devm_regulator_get_optional(dev, "vref"); - if (!IS_ERR(reg)) { - vref_uv = regulator_get_voltage(reg); + vref_uv = devm_regulator_get_enable_read_voltage(dev, "vref"); + if (vref_uv < 0) { + if (vref_uv != -ENODEV) + return vref_uv; + } else { vref_mv = DIV_ROUND_CLOSEST(vref_uv, 1000); if (vref_mv < ADS7828_EXT_VREF_MV_MIN || vref_mv > ADS7828_EXT_VREF_MV_MAX) diff --git a/drivers/hwmon/corsair-psu.c b/drivers/hwmon/corsair-psu.c index 4ab73bcef1d1..61d1b22bc63e 100644 --- a/drivers/hwmon/corsair-psu.c +++ b/drivers/hwmon/corsair-psu.c @@ -709,7 +709,7 @@ static int vendor_show(struct seq_file *seqf, void *unused) { struct corsairpsu_data *priv = seqf->private; - seq_printf(seqf, "%s\n", priv->vendor); + seq_printf(seqf, "%.*s\n", REPLY_SIZE, priv->vendor); return 0; } @@ -719,7 +719,7 @@ static int product_show(struct seq_file *seqf, void *unused) { struct corsairpsu_data *priv = seqf->private; - seq_printf(seqf, "%s\n", priv->product); + seq_printf(seqf, "%.*s\n", REPLY_SIZE, priv->product); return 0; } diff --git a/drivers/hwmon/nzxt-smart2.c b/drivers/hwmon/nzxt-smart2.c index 5bbe6f3f8af4..00c8d3bc3e34 100644 --- a/drivers/hwmon/nzxt-smart2.c +++ b/drivers/hwmon/nzxt-smart2.c @@ -760,7 +760,11 @@ static int nzxt_smart2_hid_probe(struct hid_device *hdev, hid_device_io_start(hdev); - init_device(drvdata, UPDATE_INTERVAL_DEFAULT_MS); + ret = init_device(drvdata, UPDATE_INTERVAL_DEFAULT_MS); + if (ret) { + dev_err(&hdev->dev, "init_device failed: %d\n", ret); + goto out_hw_close; + } drvdata->hwmon = hwmon_device_register_with_info(&hdev->dev, "nzxtsmart2", drvdata, diff --git a/drivers/hwmon/pmbus/lm25066.c b/drivers/hwmon/pmbus/lm25066.c index 929fa6d34efd..8a94c4cfb5d1 100644 --- a/drivers/hwmon/pmbus/lm25066.c +++ b/drivers/hwmon/pmbus/lm25066.c @@ -14,10 +14,11 @@ #include #include #include -#include +#include +#include #include "pmbus.h" -enum chips { lm25056, lm25066, lm5064, lm5066, lm5066i }; +enum chips { lm25056 = 1, lm25066, lm5064, lm5066, lm5066i }; #define LM25066_READ_VAUX 0xd0 #define LM25066_MFR_READ_IIN 0xd1 @@ -468,8 +469,6 @@ static int lm25066_probe(struct i2c_client *client) struct lm25066_data *data; struct pmbus_driver_info *info; const struct __coeff *coeff; - const struct of_device_id *of_id; - const struct i2c_device_id *i2c_id; if (!i2c_check_functionality(client->adapter, I2C_FUNC_SMBUS_READ_BYTE_DATA)) @@ -484,14 +483,8 @@ static int lm25066_probe(struct i2c_client *client) if (config < 0) return config; - i2c_id = i2c_match_id(lm25066_id, client); + data->id = (enum chips)(unsigned long)i2c_get_match_data(client); - of_id = of_match_device(lm25066_of_match, &client->dev); - if (of_id && (unsigned long)of_id->data != i2c_id->driver_data) - dev_notice(&client->dev, "Device mismatch: %s in device tree, %s detected\n", - of_id->name, i2c_id->name); - - data->id = i2c_id->driver_data; info = &data->info; info->pages = 1; @@ -548,8 +541,8 @@ static int lm25066_probe(struct i2c_client *client) if (of_property_read_u32(client->dev.of_node, "shunt-resistor-micro-ohms", &shunt)) shunt = 1000; - info->m[PSC_CURRENT_IN] = info->m[PSC_CURRENT_IN] * shunt / 1000; - info->m[PSC_POWER] = info->m[PSC_POWER] * shunt / 1000; + info->m[PSC_CURRENT_IN] = DIV_ROUND_CLOSEST_ULL((u64)info->m[PSC_CURRENT_IN] * shunt, 1000); + info->m[PSC_POWER] = DIV_ROUND_CLOSEST_ULL((u64)info->m[PSC_POWER] * shunt, 1000); #if IS_ENABLED(CONFIG_SENSORS_LM25066_REGULATOR) /* LM25056 doesn't support OPERATION */ diff --git a/drivers/input/evdev.c b/drivers/input/evdev.c index 95f90699d2b1..db014dae9500 100644 --- a/drivers/input/evdev.c +++ b/drivers/input/evdev.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include "input-compat.h" @@ -67,8 +68,10 @@ static size_t evdev_get_mask_cnt(unsigned int type) [EV_SND] = SND_CNT, [EV_FF] = FF_CNT, }; + unsigned long mask = array_index_mask_nospec(type, EV_CNT); - return (type < EV_CNT) ? counts[type] : 0; + /* Returns 0 for out-of-bounds types, including speculatively */ + return counts[type & mask] & mask; } /* requires the buffer lock to be held */ @@ -146,11 +149,11 @@ static void __evdev_queue_syn_dropped(struct evdev_client *client) struct timespec64 ts = ktime_to_timespec64(ev_time[client->clk_type]); struct input_event ev; + memset(&ev, 0, sizeof(ev)); ev.input_event_sec = ts.tv_sec; ev.input_event_usec = ts.tv_nsec / NSEC_PER_USEC; ev.type = EV_SYN; ev.code = SYN_DROPPED; - ev.value = 0; client->buffer[client->head++] = ev; client->head &= client->bufsize - 1; @@ -218,20 +221,20 @@ static void __pass_event(struct evdev_client *client, client->head &= client->bufsize - 1; if (unlikely(client->head == client->tail)) { + struct input_event ev; + + memset(&ev, 0, sizeof(ev)); + ev.input_event_sec = event->input_event_sec; + ev.input_event_usec = event->input_event_usec; + ev.type = EV_SYN; + ev.code = SYN_DROPPED; + /* * This effectively "drops" all unconsumed events, leaving * EV_SYN/SYN_DROPPED plus the newest event in the queue. */ client->tail = (client->head - 2) & (client->bufsize - 1); - - client->buffer[client->tail] = (struct input_event) { - .input_event_sec = event->input_event_sec, - .input_event_usec = event->input_event_usec, - .type = EV_SYN, - .code = SYN_DROPPED, - .value = 0, - }; - + client->buffer[client->tail] = ev; client->packet_head = client->tail; } @@ -253,6 +256,8 @@ static void evdev_pass_values(struct evdev_client *client, if (client->revoked) return; + memset(&event, 0, sizeof(event)); + ts = ktime_to_timespec64(ev_time[client->clk_type]); event.input_event_sec = ts.tv_sec; event.input_event_usec = ts.tv_nsec / NSEC_PER_USEC; diff --git a/drivers/misc/fastrpc.c b/drivers/misc/fastrpc.c index 35e311dfa49d..0a5ab9ea89d4 100644 --- a/drivers/misc/fastrpc.c +++ b/drivers/misc/fastrpc.c @@ -468,6 +468,7 @@ static void fastrpc_channel_ctx_free(struct kref *ref) cctx = container_of(ref, struct fastrpc_channel_ctx, refcount); + idr_destroy(&cctx->ctx_idr); kfree(cctx); } @@ -1275,10 +1276,12 @@ static int fastrpc_internal_invoke(struct fastrpc_user *fl, u32 kernel, } if (err == -ERESTARTSYS) { + spin_lock(&fl->lock); list_for_each_entry_safe(buf, b, &fl->mmaps, node) { list_del(&buf->node); list_add_tail(&buf->node, &fl->cctx->invoke_interrupted_mmaps); } + spin_unlock(&fl->lock); } if (err) @@ -1644,7 +1647,7 @@ static int fastrpc_device_open(struct inode *inode, struct file *filp) dev_err(&cctx->rpdev->dev, "No session available\n"); mutex_destroy(&fl->mutex); kfree(fl); - + fastrpc_channel_ctx_put(cctx); return -EBUSY; } @@ -1872,9 +1875,6 @@ static int fastrpc_req_munmap_impl(struct fastrpc_user *fl, struct fastrpc_buf * &args[0]); if (!err) { dev_dbg(dev, "unmmap\tpt 0x%09lx OK\n", buf->raddr); - spin_lock(&fl->lock); - list_del(&buf->node); - spin_unlock(&fl->lock); fastrpc_buf_free(buf); } else { dev_err(dev, "unmmap\tpt 0x%09lx ERROR\n", buf->raddr); @@ -1888,6 +1888,7 @@ static int fastrpc_req_munmap(struct fastrpc_user *fl, char __user *argp) struct fastrpc_buf *buf = NULL, *iter, *b; struct fastrpc_req_munmap req; struct device *dev = fl->sctx->dev; + int err; if (copy_from_user(&req, argp, sizeof(req))) return -EFAULT; @@ -1895,6 +1896,7 @@ static int fastrpc_req_munmap(struct fastrpc_user *fl, char __user *argp) spin_lock(&fl->lock); list_for_each_entry_safe(iter, b, &fl->mmaps, node) { if ((iter->raddr == req.vaddrout) && (iter->size == req.size)) { + list_del(&iter->node); buf = iter; break; } @@ -1907,7 +1909,14 @@ static int fastrpc_req_munmap(struct fastrpc_user *fl, char __user *argp) return -EINVAL; } - return fastrpc_req_munmap_impl(fl, buf); + err = fastrpc_req_munmap_impl(fl, buf); + if (err) { + spin_lock(&fl->lock); + list_add_tail(&buf->node, &fl->mmaps); + spin_unlock(&fl->lock); + } + + return err; } static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp) diff --git a/drivers/misc/mei/client.c b/drivers/misc/mei/client.c index 699fa2362ed4..133c36a353bd 100644 --- a/drivers/misc/mei/client.c +++ b/drivers/misc/mei/client.c @@ -447,18 +447,24 @@ static void mei_io_tx_list_free_cl(struct list_head *head, } /** - * mei_io_list_free_fp - free cb from a list that matches file pointer + * mei_io_rd_list_free_fp - free cb from a rd_completed list that matches file pointer * - * @head: io list + * @cl: host client * @fp: file pointer (matching cb file object), may be NULL */ -static void mei_io_list_free_fp(struct list_head *head, const struct file *fp) +static void mei_io_rd_list_free_fp(struct mei_cl *cl, const struct file *fp) { struct mei_cl_cb *cb, *next; + LIST_HEAD(cmpl_list); - list_for_each_entry_safe(cb, next, head, list) + spin_lock(&cl->rd_completed_lock); + list_for_each_entry_safe(cb, next, &cl->rd_completed, list) if (!fp || fp == cb->fp) - mei_io_cb_free(cb); + list_move(&cb->list, &cmpl_list); + spin_unlock(&cl->rd_completed_lock); + + list_for_each_entry_safe(cb, next, &cmpl_list, list) + mei_io_cb_free(cb); } /** @@ -587,9 +593,7 @@ int mei_cl_flush_queues(struct mei_cl *cl, const struct file *fp) mei_io_list_flush_cl(&cl->dev->ctrl_rd_list, cl); mei_cl_free_pending(cl); } - spin_lock(&cl->rd_completed_lock); - mei_io_list_free_fp(&cl->rd_completed, fp); - spin_unlock(&cl->rd_completed_lock); + mei_io_rd_list_free_fp(cl, fp); return 0; } @@ -1426,7 +1430,7 @@ void mei_cl_add_rd_completed(struct mei_cl *cl, struct mei_cl_cb *cb) } /** - * mei_cl_del_rd_completed - free read completed callback with lock + * mei_cl_del_rd_completed - unlink read completed callback with lock and free it * * @cl: host client * @cb: callback block @@ -1435,8 +1439,9 @@ void mei_cl_add_rd_completed(struct mei_cl *cl, struct mei_cl_cb *cb) void mei_cl_del_rd_completed(struct mei_cl *cl, struct mei_cl_cb *cb) { spin_lock(&cl->rd_completed_lock); - mei_io_cb_free(cb); + list_del_init(&cb->list); spin_unlock(&cl->rd_completed_lock); + mei_io_cb_free(cb); } /** diff --git a/drivers/net/bonding/bond_alb.c b/drivers/net/bonding/bond_alb.c index 7edf0fd58c34..ce6a3a0a2e15 100644 --- a/drivers/net/bonding/bond_alb.c +++ b/drivers/net/bonding/bond_alb.c @@ -1535,8 +1535,8 @@ void bond_alb_monitor(struct work_struct *work) struct bonding *bond = container_of(work, struct bonding, alb_work.work); struct alb_bond_info *bond_info = &(BOND_ALB_INFO(bond)); + struct slave *slave, *curr; struct list_head *iter; - struct slave *slave; if (!bond_has_slaves(bond)) { atomic_set(&bond_info->tx_rebalance_counter, 0); @@ -1598,9 +1598,11 @@ void bond_alb_monitor(struct work_struct *work) * because a slave was disabled then * it can now leave promiscuous mode. */ - dev_set_promiscuity(rtnl_dereference(bond->curr_active_slave)->dev, - -1); - bond_info->primary_is_promisc = 0; + curr = rtnl_dereference(bond->curr_active_slave); + if (bond_info->primary_is_promisc && curr) { + dev_set_promiscuity(curr->dev, -1); + bond_info->primary_is_promisc = 0; + } rtnl_unlock(); rcu_read_lock(); diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_ring.c b/drivers/net/ethernet/aquantia/atlantic/aq_ring.c index 3f004d08307f..ec7654823f4e 100644 --- a/drivers/net/ethernet/aquantia/atlantic/aq_ring.c +++ b/drivers/net/ethernet/aquantia/atlantic/aq_ring.c @@ -336,6 +336,35 @@ bool aq_ring_tx_clean(struct aq_ring_s *self) return !!budget; } +void aq_ring_tx_deinit(struct aq_ring_s *self) +{ + if (!self) + return; + + for (; self->sw_head != self->sw_tail; + self->sw_head = aq_ring_next_dx(self, self->sw_head)) { + struct aq_ring_buff_s *buff = &self->buff_ring[self->sw_head]; + struct device *ndev = aq_nic_get_dev(self->aq_nic); + + if (buff->is_mapped) { + if (buff->is_sop) { + dma_unmap_single(ndev, buff->pa, buff->len, + DMA_TO_DEVICE); + } else { + dma_unmap_page(ndev, buff->pa, buff->len, + DMA_TO_DEVICE); + } + } + + if (buff->is_eop) { + if (buff->skb) + dev_kfree_skb_any(buff->skb); + else if (buff->xdpf) + xdp_return_frame(buff->xdpf); + } + } +} + static void aq_rx_checksum(struct aq_ring_s *self, struct aq_ring_buff_s *buff, struct sk_buff *skb) @@ -897,15 +926,29 @@ int aq_ring_rx_fill(struct aq_ring_s *self) void aq_ring_rx_deinit(struct aq_ring_s *self) { - if (!self) + unsigned int i; + + if (!self || !self->buff_ring) return; - for (; self->sw_head != self->sw_tail; - self->sw_head = aq_ring_next_dx(self, self->sw_head)) { - struct aq_ring_buff_s *buff = &self->buff_ring[self->sw_head]; + /* Release every page still owned by the ring. + * + * Walking [sw_head, sw_tail) is not enough: refill is batched + * (aq_ring_rx_fill() waits for AQ_CFG_RX_REFILL_THRES free slots), + * so slots that were cleaned but not yet reposted accumulate in the + * [sw_tail, sw_head) gap, and they keep their page for reuse. Walk + * the whole ring and release whatever is left. + */ + for (i = 0; i < self->size; i++) { + struct aq_ring_buff_s *buff = &self->buff_ring[i]; + + if (!buff->rxdata.page) + continue; aq_free_rxpage(&buff->rxdata, aq_nic_get_dev(self->aq_nic)); } + + self->sw_head = self->sw_tail; } void aq_ring_free(struct aq_ring_s *self) diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_ring.h b/drivers/net/ethernet/aquantia/atlantic/aq_ring.h index d627ace850ff..67503c512676 100644 --- a/drivers/net/ethernet/aquantia/atlantic/aq_ring.h +++ b/drivers/net/ethernet/aquantia/atlantic/aq_ring.h @@ -199,6 +199,7 @@ void aq_ring_update_queue_state(struct aq_ring_s *ring); void aq_ring_queue_wake(struct aq_ring_s *ring); void aq_ring_queue_stop(struct aq_ring_s *ring); bool aq_ring_tx_clean(struct aq_ring_s *self); +void aq_ring_tx_deinit(struct aq_ring_s *self); int aq_xdp_xmit(struct net_device *dev, int num_frames, struct xdp_frame **frames, u32 flags); int aq_ring_rx_clean(struct aq_ring_s *self, diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_vec.c b/drivers/net/ethernet/aquantia/atlantic/aq_vec.c index 9769ab4f9bef..62b5967b6dcc 100644 --- a/drivers/net/ethernet/aquantia/atlantic/aq_vec.c +++ b/drivers/net/ethernet/aquantia/atlantic/aq_vec.c @@ -275,7 +275,7 @@ void aq_vec_deinit(struct aq_vec_s *self) for (i = 0U; self->tx_rings > i; ++i) { ring = self->ring[i]; - aq_ring_tx_clean(&ring[AQ_VEC_TX_ID]); + aq_ring_tx_deinit(&ring[AQ_VEC_TX_ID]); aq_ring_rx_deinit(&ring[AQ_VEC_RX_ID]); } diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index afe700575f53..e305b02b775e 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -3715,7 +3715,17 @@ static int bnxt_init_one_rx_ring(struct bnxt *bp, int ring_nr) if ((bp->flags & BNXT_FLAG_AGG_RINGS)) { type = ((u32)BNXT_RX_PAGE_SIZE << RX_BD_LEN_SHIFT) | - RX_BD_TYPE_RX_AGG_BD | RX_BD_FLAGS_SOP; + RX_BD_TYPE_RX_AGG_BD; + + /* Disable EOP if TPA is enabled to prevent overlapping zero + * padding with the next segment's data. On P7_PLUS, EOP will + * automatically disable Relaxed Ordering (RO) to prevent + * potential data corruption (and may degrade performance). On + * older chips, RO will not be automatically disabled and may + * cause corruption. + */ + if (!(bp->flags & BNXT_FLAG_TPA)) + type |= RX_BD_FLAGS_AGG_EOP; bnxt_init_rxbd_pages(ring, type); } diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index bc1ff1085da7..739ed9d0b5ab 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -104,6 +104,7 @@ struct rx_bd { #define RX_BD_TYPE_48B_BD_SIZE (2 << 4) #define RX_BD_TYPE_64B_BD_SIZE (3 << 4) #define RX_BD_FLAGS_SOP (1 << 6) + #define RX_BD_FLAGS_AGG_EOP (1 << 6) #define RX_BD_FLAGS_EOP (1 << 7) #define RX_BD_FLAGS_BUFFERS (3 << 8) #define RX_BD_FLAGS_1_BUFFER_PACKET (0 << 8) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c index 404b433f1bc0..d8f39776481b 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c @@ -475,12 +475,15 @@ static int bnxt_ptp_enable(struct ptp_clock_info *ptp_info, return rc; case PTP_CLK_REQ_PPS: /* Configure PHC PPS IN */ - rc = bnxt_ptp_cfg_pin(bp, 0, BNXT_PPS_PIN_PPS_IN); + pin_id = 0; + if (!on) + break; + rc = bnxt_ptp_cfg_pin(bp, pin_id, BNXT_PPS_PIN_PPS_IN); if (rc) return rc; rc = bnxt_ptp_cfg_event(bp, BNXT_PPS_EVENT_INTERNAL); if (!rc) - ptp->pps_info.pins[0].event = BNXT_PPS_EVENT_INTERNAL; + ptp->pps_info.pins[pin_id].event = BNXT_PPS_EVENT_INTERNAL; return rc; default: netdev_err(ptp->bp->dev, "Unrecognized PIN function\n"); diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c index 49297b83c3fd..ec7fe943e568 100644 --- a/drivers/net/ethernet/freescale/fec_main.c +++ b/drivers/net/ethernet/freescale/fec_main.c @@ -3322,8 +3322,15 @@ static void fec_enet_free_buffers(struct net_device *ndev) for (q = 0; q < fep->num_rx_queues; q++) { rxq = fep->rx_queue[q]; - for (i = 0; i < rxq->bd.ring_size; i++) - page_pool_put_full_page(rxq->page_pool, rxq->rx_skb_info[i].page, false); + for (i = 0; i < rxq->bd.ring_size; i++) { + struct page *page = rxq->rx_skb_info[i].page; + + if (!page) + continue; + + page_pool_put_full_page(rxq->page_pool, page, false); + rxq->rx_skb_info[i].page = NULL; + } for (i = 0; i < XDP_STATS_TOTAL; i++) rxq->stats[i] = 0; diff --git a/drivers/net/ethernet/hisilicon/hix5hd2_gmac.c b/drivers/net/ethernet/hisilicon/hix5hd2_gmac.c index 26d22bb04b87..e8cedc260865 100644 --- a/drivers/net/ethernet/hisilicon/hix5hd2_gmac.c +++ b/drivers/net/ethernet/hisilicon/hix5hd2_gmac.c @@ -1287,7 +1287,6 @@ static int hix5hd2_dev_remove(struct platform_device *pdev) struct net_device *ndev = platform_get_drvdata(pdev); struct hix5hd2_priv *priv = netdev_priv(ndev); - netif_napi_del(&priv->napi); unregister_netdev(ndev); mdiobus_unregister(priv->bus); mdiobus_free(priv->bus); diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c index 635cfb9a3e2c..1d1e5cea9395 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c @@ -72,10 +72,12 @@ static void otx2_get_egress_burst_cfg(struct otx2_nic *nic, u32 burst, if (burst) { *burst_exp = ilog2(burst) ? ilog2(burst) - 1 : 0; tmp = burst - rounddown_pow_of_two(burst); - if (burst < max_mantissa) + if (burst <= max_mantissa) { *burst_mantissa = tmp * 2; - else + } else { + WARN_ON(*burst_exp < 7); *burst_mantissa = tmp / (1ULL << (*burst_exp - 7)); + } } else { *burst_exp = MAX_BURST_EXPONENT; *burst_mantissa = max_mantissa; diff --git a/drivers/net/ethernet/marvell/prestera/prestera_pci.c b/drivers/net/ethernet/marvell/prestera/prestera_pci.c index 35857dc19542..5db2c9e5e077 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_pci.c +++ b/drivers/net/ethernet/marvell/prestera/prestera_pci.c @@ -684,6 +684,9 @@ static int prestera_fw_hdr_parse(struct prestera_fw *fw) struct prestera_fw_header *hdr; u32 magic; + if (fw->bin->size < sizeof(*hdr)) + return -EINVAL; + hdr = (struct prestera_fw_header *)fw->bin->data; magic = be32_to_cpu(hdr->magic_number); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c index 0a33ab5f53fd..dba0d73ac37d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c @@ -1050,13 +1050,11 @@ struct mlx5_fw_tracer *mlx5_fw_tracer_create(struct mlx5_core_dev *dev) tracer = kvzalloc(sizeof(*tracer), GFP_KERNEL); if (!tracer) - return ERR_PTR(-ENOMEM); + return NULL; tracer->work_queue = create_singlethread_workqueue("mlx5_fw_tracer"); - if (!tracer->work_queue) { - err = -ENOMEM; + if (!tracer->work_queue) goto free_tracer; - } tracer->dev = dev; @@ -1098,7 +1096,7 @@ struct mlx5_fw_tracer *mlx5_fw_tracer_create(struct mlx5_core_dev *dev) destroy_workqueue(tracer->work_queue); free_tracer: kvfree(tracer); - return ERR_PTR(err); + return NULL; } static int fw_tracer_event(struct notifier_block *nb, unsigned long action, void *data); @@ -1109,7 +1107,7 @@ int mlx5_fw_tracer_init(struct mlx5_fw_tracer *tracer) struct mlx5_core_dev *dev; int err; - if (IS_ERR_OR_NULL(tracer)) + if (!tracer) return 0; if (!tracer->str_db.loaded) @@ -1159,7 +1157,7 @@ int mlx5_fw_tracer_init(struct mlx5_fw_tracer *tracer) /* Stop tracer + Cleanup HW resources */ void mlx5_fw_tracer_cleanup(struct mlx5_fw_tracer *tracer) { - if (IS_ERR_OR_NULL(tracer)) + if (!tracer) return; mutex_lock(&tracer->state_lock); @@ -1188,7 +1186,7 @@ void mlx5_fw_tracer_cleanup(struct mlx5_fw_tracer *tracer) /* Free software resources (Buffers, etc ..) */ void mlx5_fw_tracer_destroy(struct mlx5_fw_tracer *tracer) { - if (IS_ERR_OR_NULL(tracer)) + if (!tracer) return; mlx5_core_dbg(tracer->dev, "FWTracer: Destroy\n"); @@ -1240,7 +1238,7 @@ int mlx5_fw_tracer_reload(struct mlx5_fw_tracer *tracer) struct mlx5_core_dev *dev; int err; - if (IS_ERR_OR_NULL(tracer)) + if (!tracer) return 0; dev = tracer->dev; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_priv.h b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_priv.h index 6cc23af66b5b..a23d0df42f02 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_priv.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_priv.h @@ -30,6 +30,7 @@ enum { MLX5E_TC_FLOW_FLAG_FAILED = MLX5E_TC_FLOW_BASE + 9, MLX5E_TC_FLOW_FLAG_SAMPLE = MLX5E_TC_FLOW_BASE + 10, MLX5E_TC_FLOW_FLAG_USE_ACT_STATS = MLX5E_TC_FLOW_BASE + 11, + MLX5E_TC_FLOW_FLAG_PEER = MLX5E_TC_FLOW_BASE + 12, }; struct mlx5e_tc_flow_parse_attr { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c index f1f422505731..8c52bbeabb7e 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c @@ -2043,7 +2043,8 @@ static void mlx5e_tc_del_flow(struct mlx5e_priv *priv, if (mlx5e_is_eswitch_flow(flow)) { struct mlx5_devcom_comp_dev *devcom = flow->priv->mdev->priv.eswitch->devcom; - if (!mlx5_devcom_for_each_peer_begin(devcom)) { + if (flow_flag_test(flow, PEER) || + !mlx5_devcom_for_each_peer_begin(devcom)) { mlx5e_tc_del_fdb_flow(priv, flow); return; } @@ -4489,6 +4490,7 @@ static int mlx5e_tc_add_fdb_peer_flow(struct flow_cls_offload *f, else in_mdev = priv->mdev; + flow_flags |= BIT(MLX5E_TC_FLOW_FLAG_PEER); parse_attr = flow->attr->parse_attr; peer_flow = __mlx5e_add_fdb_flow(peer_priv, f, flow_flags, parse_attr->filter_dev, diff --git a/drivers/net/thunderbolt/main.c b/drivers/net/thunderbolt/main.c index 5d7d11d75b0b..ceaba08cd378 100644 --- a/drivers/net/thunderbolt/main.c +++ b/drivers/net/thunderbolt/main.c @@ -389,11 +389,16 @@ static void tbnet_tear_down(struct tbnet *net, bool send_logout) break; } - tb_ring_stop(net->rx_ring.ring); - tb_ring_stop(net->tx_ring.ring); - tbnet_free_buffers(&net->rx_ring); - tbnet_free_buffers(&net->tx_ring); - + /* Tear the paths down before stopping the rings. This mirrors + * tbnet_connected_work(), which enables the paths last so the + * Rx ring is primed before packets can arrive. Stopping a + * ring zeroes its descriptor base and tbnet_free_buffers() + * unmaps and frees the frame buffers, leaving anything still + * in flight with nowhere to drain to; + * __tb_path_deactivate_hop() then waits for the hop's + * 'pending' bit, which on some host routers never clears in + * that state. + */ ret = tb_xdomain_disable_paths(net->xd, net->local_transmit_path, net->tx_ring.ring->hop, @@ -402,6 +407,11 @@ static void tbnet_tear_down(struct tbnet *net, bool send_logout) if (ret) netdev_warn(net->dev, "failed to disable DMA paths\n"); + tb_ring_stop(net->rx_ring.ring); + tb_ring_stop(net->tx_ring.ring); + tbnet_free_buffers(&net->rx_ring); + tbnet_free_buffers(&net->tx_ring); + tb_xdomain_release_in_hopid(net->xd, net->remote_transmit_path); net->remote_transmit_path = 0; } @@ -928,12 +938,8 @@ static int tbnet_open(struct net_device *dev) netif_carrier_off(dev); - flags = RING_FLAG_FRAME; - /* Only enable full E2E if the other end supports it too */ - if (tbnet_e2e && net->svc->prtcstns & TBNET_E2E) - flags |= RING_FLAG_E2E; - - ring = tb_ring_alloc_tx(xd->tb->nhi, -1, TBNET_RING_SIZE, flags); + ring = tb_ring_alloc_tx(xd->tb->nhi, -1, TBNET_RING_SIZE, + RING_FLAG_FRAME); if (!ring) { netdev_err(dev, "failed to allocate Tx ring\n"); return -ENOMEM; @@ -952,6 +958,11 @@ static int tbnet_open(struct net_device *dev) sof_mask = BIT(TBIP_PDF_FRAME_START); eof_mask = BIT(TBIP_PDF_FRAME_END); + flags = RING_FLAG_FRAME; + /* Only enable full E2E if the other end supports it too */ + if (tbnet_e2e && net->svc->prtcstns & TBNET_E2E) + flags |= RING_FLAG_E2E; + ring = tb_ring_alloc_rx(xd->tb->nhi, -1, TBNET_RING_SIZE, flags, net->tx_ring.ring->hop, sof_mask, eof_mask, tbnet_start_poll, net); diff --git a/drivers/net/usb/ax88179_178a.c b/drivers/net/usb/ax88179_178a.c index 73de34179f35..619093d33373 100644 --- a/drivers/net/usb/ax88179_178a.c +++ b/drivers/net/usb/ax88179_178a.c @@ -1491,8 +1491,10 @@ ax88179_tx_fixup(struct usbnet *dev, struct sk_buff *skb, gfp_t flags) headroom = skb_headroom(skb) - 8; - if ((dev->net->features & NETIF_F_SG) && skb_linearize(skb)) + if ((dev->net->features & NETIF_F_SG) && skb_linearize(skb)) { + dev_kfree_skb_any(skb); return NULL; + } if ((skb_header_cloned(skb) || headroom < 0) && pskb_expand_head(skb, headroom < 0 ? 8 : 0, 0, GFP_ATOMIC)) { diff --git a/drivers/net/veth.c b/drivers/net/veth.c index d85a11e9477f..e2fa5ba5a032 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -902,18 +902,24 @@ static struct sk_buff *veth_xdp_rcv_skb(struct veth_rq *rq, skb_reset_mac_header(skb); - /* check if bpf_xdp_adjust_tail was used */ - off = xdp->data_end - orig_data_end; - if (off != 0) - __skb_put(skb, off); /* positive on grow, negative on shrink */ - /* XDP frag metadata (e.g. nr_frags) are updated in eBPF helpers - * (e.g. bpf_xdp_adjust_tail), we need to update data_len here. + * (e.g. bpf_xdp_adjust_tail). Remove the old fragment contribution + * from skb->len before updating data_len, then add the new one back. */ - if (xdp_buff_has_frags(xdp)) + skb->len -= skb->data_len; + if (xdp_buff_has_frags(xdp)) { skb->data_len = skb_shinfo(skb)->xdp_frags_size; - else + skb->len += skb->data_len; + } else { skb->data_len = 0; + } + + /* Synchronize the skb tail with XDP's updated linear area. */ + off = xdp->data_end - orig_data_end; + if (off != 0) { + skb_set_tail_pointer(skb, xdp->data_end - xdp->data); + skb->len += off; /* positive on grow, negative on shrink */ + } skb->protocol = eth_type_trans(skb, rq->dev); diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c index 0e9aeb03fcc7..9d3546f95fa4 100644 --- a/drivers/net/vxlan/vxlan_core.c +++ b/drivers/net/vxlan/vxlan_core.c @@ -4338,7 +4338,7 @@ static int vxlan_changelink(struct net_device *dev, struct nlattr *tb[], if (change_igmp && vxlan_addr_multicast(&dst->remote_ip)) err = vxlan_multicast_leave(vxlan); - if (conf.age_interval != vxlan->cfg.age_interval) + if (netif_running(dev) && conf.age_interval != vxlan->cfg.age_interval) mod_timer(&vxlan->age_timer, jiffies); netdev_adjacent_change_commit(dst->remote_dev, lowerdev, dev); diff --git a/drivers/pinctrl/renesas/pinctrl-rzg2l.c b/drivers/pinctrl/renesas/pinctrl-rzg2l.c index ac629c72d592..b713cc8e0fde 100644 --- a/drivers/pinctrl/renesas/pinctrl-rzg2l.c +++ b/drivers/pinctrl/renesas/pinctrl-rzg2l.c @@ -710,7 +710,7 @@ static int rzg2l_pinctrl_pinconf_set(struct pinctrl_dev *pctldev, } default: - return -EOPNOTSUPP; + return -ENOTSUPP; } } @@ -759,7 +759,7 @@ static int rzg2l_pinctrl_pinconf_group_get(struct pinctrl_dev *pctldev, /* Check config matching between to pin */ if (i && prev_config != *config) - return -EOPNOTSUPP; + return -ENOTSUPP; prev_config = *config; } diff --git a/drivers/ptp/ptp_ocp.c b/drivers/ptp/ptp_ocp.c index 4899fdf9bdf7..2d21699586ce 100644 --- a/drivers/ptp/ptp_ocp.c +++ b/drivers/ptp/ptp_ocp.c @@ -1671,9 +1671,11 @@ ptp_ocp_devlink_info_get(struct devlink *devlink, struct devlink_info_req *req, if (err) return err; + snprintf(buf, sizeof(buf), "%.*s", OCP_BOARD_ID_LEN, + (const char *)bp->board_id); err = devlink_info_version_fixed_put(req, DEVLINK_INFO_VERSION_GENERIC_BOARD_ID, - bp->board_id); + buf); if (err) return err; diff --git a/drivers/regulator/devres.c b/drivers/regulator/devres.c index 90bb0d178885..7111c46e9de1 100644 --- a/drivers/regulator/devres.c +++ b/drivers/regulator/devres.c @@ -145,6 +145,65 @@ struct regulator *devm_regulator_get_optional(struct device *dev, } EXPORT_SYMBOL_GPL(devm_regulator_get_optional); +/** + * devm_regulator_get_enable_read_voltage - Resource managed regulator get and + * enable that returns the voltage + * @dev: device to supply + * @id: supply name or regulator ID. + * + * Get and enable regulator for duration of the device life-time. + * regulator_disable() and regulator_put() are automatically called on driver + * detach. See regulator_get_optional(), regulator_enable(), and + * regulator_get_voltage() for more information. + * + * This is a convenience function for supplies that provide a reference voltage + * where the consumer driver just needs to know the voltage and keep the + * regulator enabled. + * + * In cases where the supply is not strictly required, callers can check for + * -ENODEV error and handle it accordingly. + * + * Returns: voltage in microvolts on success, or an error code on failure. + */ +int devm_regulator_get_enable_read_voltage(struct device *dev, const char *id) +{ + struct regulator *r; + int ret; + + /* + * Since we need a real voltage, we use devm_regulator_get_optional() + * rather than getting a dummy regulator with devm_regulator_get() and + * then letting regulator_get_voltage() fail with -EINVAL. This way, the + * caller can handle the -ENODEV error code if needed instead of the + * ambiguous -EINVAL. + */ + r = devm_regulator_get_optional(dev, id); + if (IS_ERR(r)) + return PTR_ERR(r); + + ret = regulator_enable(r); + if (ret) + goto err_regulator_put; + + ret = devm_add_action_or_reset(dev, regulator_action_disable, r); + if (ret) + goto err_regulator_put; + + ret = regulator_get_voltage(r); + if (ret < 0) + goto err_release_action; + + return ret; + +err_release_action: + devm_release_action(dev, regulator_action_disable, r); +err_regulator_put: + devm_regulator_put(r); + + return ret; +} +EXPORT_SYMBOL_GPL(devm_regulator_get_enable_read_voltage); + static int devm_regulator_match(struct device *dev, void *res, void *data) { struct regulator **r = res; diff --git a/drivers/s390/crypto/zcrypt_ccamisc.c b/drivers/s390/crypto/zcrypt_ccamisc.c index c3e3f3a3d8f9..b8526219eceb 100644 --- a/drivers/s390/crypto/zcrypt_ccamisc.c +++ b/drivers/s390/crypto/zcrypt_ccamisc.c @@ -946,7 +946,8 @@ static int _ip_cprb_helper(u16 cardnr, u16 domain, const u8 *clr_key_value, int clr_key_bit_size, u8 *key_token, - int *key_token_size) + int *key_token_size, + bool scrub) { int rc, n; u8 *mem, *ptr; @@ -1087,7 +1088,7 @@ static int _ip_cprb_helper(u16 cardnr, u16 domain, *key_token_size = t->len; out: - free_cprbmem(mem, PARMBSIZE, 0); + free_cprbmem(mem, PARMBSIZE, scrub); return rc; } @@ -1130,7 +1131,8 @@ int cca_clr2cipherkey(u16 card, u16 dom, u32 keybitsize, u32 keygenflags, * 4/4 COMPLETE the secure cipher key import */ rc = _ip_cprb_helper(card, dom, "AES ", "FIRST ", "MIN3PART", - exorbuf, keybitsize, token, &tokensize); + exorbuf, keybitsize, token, &tokensize, + true); if (rc) { DEBUG_ERR( "%s clear key import 1/4 with CSNBKPI2 failed, rc=%d\n", @@ -1138,7 +1140,8 @@ int cca_clr2cipherkey(u16 card, u16 dom, u32 keybitsize, u32 keygenflags, goto out; } rc = _ip_cprb_helper(card, dom, "AES ", "ADD-PART", NULL, - clrkey, keybitsize, token, &tokensize); + clrkey, keybitsize, token, &tokensize, + true); if (rc) { DEBUG_ERR( "%s clear key import 2/4 with CSNBKPI2 failed, rc=%d\n", @@ -1146,7 +1149,8 @@ int cca_clr2cipherkey(u16 card, u16 dom, u32 keybitsize, u32 keygenflags, goto out; } rc = _ip_cprb_helper(card, dom, "AES ", "ADD-PART", NULL, - exorbuf, keybitsize, token, &tokensize); + exorbuf, keybitsize, token, &tokensize, + true); if (rc) { DEBUG_ERR( "%s clear key import 3/4 with CSNBKPI2 failed, rc=%d\n", @@ -1154,7 +1158,8 @@ int cca_clr2cipherkey(u16 card, u16 dom, u32 keybitsize, u32 keygenflags, goto out; } rc = _ip_cprb_helper(card, dom, "AES ", "COMPLETE", NULL, - NULL, keybitsize, token, &tokensize); + NULL, keybitsize, token, &tokensize, + true); if (rc) { DEBUG_ERR( "%s clear key import 4/4 with CSNBKPI2 failed, rc=%d\n", @@ -1172,7 +1177,8 @@ int cca_clr2cipherkey(u16 card, u16 dom, u32 keybitsize, u32 keygenflags, *keybufsize = tokensize; out: - kfree(token); + memzero_explicit(exorbuf, sizeof(exorbuf)); + kfree_sensitive(token); return rc; } EXPORT_SYMBOL(cca_clr2cipherkey); diff --git a/drivers/scsi/scsi_debug.c b/drivers/scsi/scsi_debug.c index 573bb3fcecd5..5d8561ec323b 100644 --- a/drivers/scsi/scsi_debug.c +++ b/drivers/scsi/scsi_debug.c @@ -3077,8 +3077,8 @@ static bool comp_write_worker(struct sdeb_store_info *sip, u64 lba, u32 num, if (!res) return res; if (rest) - res = memcmp(fsp, arr + ((num - rest) * lb_size), - rest * lb_size); + res = !memcmp(fsp, arr + ((num - rest) * lb_size), + rest * lb_size); if (!res) return res; if (compare_only) diff --git a/drivers/spi/spi-fsl-dspi.c b/drivers/spi/spi-fsl-dspi.c index 3206c84c6f22..b7dc2add9114 100644 --- a/drivers/spi/spi-fsl-dspi.c +++ b/drivers/spi/spi-fsl-dspi.c @@ -751,8 +751,12 @@ static void dspi_setup_accel(struct fsl_dspi *dspi) struct spi_transfer *xfer = dspi->cur_transfer; bool odd = !!(dspi->len & 1); - /* No accel for frames not multiple of 8 bits at the moment */ - if (xfer->bits_per_word % 8) + /* + * No accel for DMA transfers or frames not multiples of 8 bits at the + * moment. + */ + if (dspi->devtype_data->trans_mode == DSPI_DMA_MODE || + xfer->bits_per_word % 8) goto no_accel; if (!odd && dspi->len <= dspi->devtype_data->fifo_size * 2) { @@ -761,10 +765,7 @@ static void dspi_setup_accel(struct fsl_dspi *dspi) dspi->oper_bits_per_word = 8; } else { /* Start off with maximum supported by hardware */ - if (dspi->devtype_data->trans_mode == DSPI_XSPI_MODE) - dspi->oper_bits_per_word = 32; - else - dspi->oper_bits_per_word = 16; + dspi->oper_bits_per_word = 32; /* * And go down only if the buffer can't be sent with diff --git a/drivers/staging/rtl8723bs/core/rtw_ieee80211.c b/drivers/staging/rtl8723bs/core/rtw_ieee80211.c index 95e4d2924224..003946633633 100644 --- a/drivers/staging/rtl8723bs/core/rtw_ieee80211.c +++ b/drivers/staging/rtl8723bs/core/rtw_ieee80211.c @@ -371,6 +371,9 @@ unsigned char *rtw_get_wpa_ie(unsigned char *pie, int *wpa_ie_len, int limit) pbuf = rtw_get_ie(pbuf, WLAN_EID_VENDOR_SPECIFIC, &len, limit_new); if (pbuf) { + if (len < 6) + goto check_next_ie; + /* check if oui matches... */ if (memcmp((pbuf + 2), wpa_oui_type, sizeof(wpa_oui_type))) goto check_next_ie; diff --git a/drivers/staging/rtl8723bs/core/rtw_mlme_ext.c b/drivers/staging/rtl8723bs/core/rtw_mlme_ext.c index f4c0fd703c5a..98b46182afd6 100644 --- a/drivers/staging/rtl8723bs/core/rtw_mlme_ext.c +++ b/drivers/staging/rtl8723bs/core/rtw_mlme_ext.c @@ -905,7 +905,7 @@ unsigned int OnAuthClient(struct adapter *padapter, union recv_frame *precv_fram p = rtw_get_ie(pframe + WLAN_HDR_A3_LEN + _AUTH_IE_OFFSET_, WLAN_EID_CHALLENGE, (int *)&len, pkt_len - WLAN_HDR_A3_LEN - _AUTH_IE_OFFSET_); - if (!p) + if (!p || len != WLAN_AUTH_CHALLENGE_LEN) goto authclnt_fail; memcpy((void *)(pmlmeinfo->chg_txt), (void *)(p + 2), len); diff --git a/drivers/staging/rtl8723bs/core/rtw_wlan_util.c b/drivers/staging/rtl8723bs/core/rtw_wlan_util.c index aba340530378..3faf0da668c5 100644 --- a/drivers/staging/rtl8723bs/core/rtw_wlan_util.c +++ b/drivers/staging/rtl8723bs/core/rtw_wlan_util.c @@ -731,6 +731,9 @@ int WMM_param_handler(struct adapter *padapter, struct ndis_80211_var_ie *pIE) return false; } + if (pIE->length != WLAN_WMM_LEN) + return false; + if (!memcmp(&(pmlmeinfo->WMM_param), (pIE->data + 6), sizeof(struct WMM_para_element))) return false; else diff --git a/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c b/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c index 557b9c6513e1..941d6b8406b6 100644 --- a/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c +++ b/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c @@ -2043,6 +2043,8 @@ static netdev_tx_t rtw_cfg80211_monitor_if_xmit_entry(struct sk_buff *skb, struc /* Skip the ratio tap header */ skb_pull(skb, rtap_len); + if (skb->len < dot11_hdr_len) + goto fail; dot11_hdr = (struct ieee80211_hdr *)skb->data; frame_control = le16_to_cpu(dot11_hdr->frame_control); @@ -2055,6 +2057,8 @@ static netdev_tx_t rtw_cfg80211_monitor_if_xmit_entry(struct sk_buff *skb, struc qos_len = 2; if ((frame_control & 0x0300) == 0x0300) dot11_hdr_len += 6; + if (skb->len < dot11_hdr_len + qos_len + snap_len) + goto fail; memcpy(dst_mac_addr, dot11_hdr->addr1, sizeof(dst_mac_addr)); memcpy(src_mac_addr, dot11_hdr->addr2, sizeof(src_mac_addr)); diff --git a/drivers/thermal/thermal_hwmon.c b/drivers/thermal/thermal_hwmon.c index 0e0c4061792c..79a4de5f9f0e 100644 --- a/drivers/thermal/thermal_hwmon.c +++ b/drivers/thermal/thermal_hwmon.c @@ -218,7 +218,8 @@ int thermal_add_hwmon_sysfs(struct thermal_zone_device *tz) if (new_hwmon_device) hwmon_device_unregister(hwmon->device); free_mem: - kfree(hwmon); + if (new_hwmon_device) + kfree(hwmon); return result; } diff --git a/drivers/thunderbolt/eeprom.c b/drivers/thunderbolt/eeprom.c index eb241b270f79..6e76c3432a9a 100644 --- a/drivers/thunderbolt/eeprom.c +++ b/drivers/thunderbolt/eeprom.c @@ -392,9 +392,16 @@ static int tb_drom_parse_entry_port(struct tb_switch *sw, return -EIO; } port->link_nr = entry->link_nr; - if (entry->has_dual_link_port) + if (entry->has_dual_link_port) { + if (entry->dual_link_port_nr > sw->config.max_port_number) { + tb_sw_warn(sw, + "port entry has invalid dual link port number %u\n", + entry->dual_link_port_nr); + return -EIO; + } port->dual_link_port = &port->sw->ports[entry->dual_link_port_nr]; + } } return 0; } diff --git a/drivers/thunderbolt/icm.c b/drivers/thunderbolt/icm.c index 623aa81a8833..c07fed6c117e 100644 --- a/drivers/thunderbolt/icm.c +++ b/drivers/thunderbolt/icm.c @@ -2291,7 +2291,7 @@ static int icm_usb4_switch_op(struct tb_switch *sw, u16 opcode, u32 *metadata, if (tx_data_len) { request.data_len_valid |= ICM_USB4_SWITCH_DATA_VALID; if (tx_data_len < ARRAY_SIZE(request.data)) - request.data_len_valid = + request.data_len_valid |= tx_data_len & ICM_USB4_SWITCH_DATA_LEN_MASK; memcpy(request.data, tx_data, tx_data_len * sizeof(u32)); } diff --git a/drivers/tty/serial/8250/8250_dma.c b/drivers/tty/serial/8250/8250_dma.c index 745a7399c959..daf66b0a2855 100644 --- a/drivers/tty/serial/8250/8250_dma.c +++ b/drivers/tty/serial/8250/8250_dma.c @@ -201,11 +201,12 @@ void serial8250_rx_dma_flush(struct uart_8250_port *p) { struct uart_8250_dma *dma = p->dma; - if (dma->rx_running) { - dmaengine_pause(dma->rxchan); - __dma_rx_complete(p); - dmaengine_terminate_async(dma->rxchan); - } + if (!dma || !dma->rxchan || !dma->rx_running) + return; + + dmaengine_pause(dma->rxchan); + __dma_rx_complete(p); + dmaengine_terminate_async(dma->rxchan); } EXPORT_SYMBOL_GPL(serial8250_rx_dma_flush); @@ -314,6 +315,7 @@ void serial8250_release_dma(struct uart_8250_port *p) /* Release RX resources */ dmaengine_terminate_sync(dma->rxchan); + dma->rx_running = 0; dma_free_coherent(dma->rxchan->device->dev, dma->rx_size, dma->rx_buf, dma->rx_addr); dma_release_channel(dma->rxchan); diff --git a/drivers/tty/vt/keyboard.c b/drivers/tty/vt/keyboard.c index 18b3c197c134..5a4f2f67f08a 100644 --- a/drivers/tty/vt/keyboard.c +++ b/drivers/tty/vt/keyboard.c @@ -1405,7 +1405,7 @@ static void kbd_keycode(unsigned int keycode, int down, bool hw_raw) struct keyboard_notifier_param param = { .vc = vc, .value = keycode, .down = down }; int rc; - tty = vc->port.tty; + tty = tty_port_tty_get(&vc->port); if (tty && (!tty->driver_data)) { /* No driver data? Strange. Okay we fix it then. */ @@ -1465,9 +1465,12 @@ static void kbd_keycode(unsigned int keycode, int down, bool hw_raw) * characters get aren't echoed locally. This makes key repeat * usable with slow applications and under heavy loads. */ + tty_kref_put(tty); return; } + tty_kref_put(tty); + param.shift = shift_final = (shift_state | kbd->slockstate) ^ kbd->lockstate; param.ledstate = kbd->ledflagstate; key_map = key_maps[shift_final]; diff --git a/drivers/tty/vt/vt_ioctl.c b/drivers/tty/vt/vt_ioctl.c index 5b21b60547da..e8abf4572dae 100644 --- a/drivers/tty/vt/vt_ioctl.c +++ b/drivers/tty/vt/vt_ioctl.c @@ -408,6 +408,8 @@ static int vt_k_ioctl(struct tty_struct *tty, unsigned int cmd, /* this could be folded into KDSKBMODE, but for compatibility reasons it is not so easy to fold KDGKBMETA into KDGKBMODE */ case KDSKBMETA: + if (!perm) + return -EPERM; return vt_do_kdskbmeta(console, arg); case KDGKBMETA: diff --git a/drivers/usb/atm/cxacru.c b/drivers/usb/atm/cxacru.c index db9a8f2731f1..071d2a079f14 100644 --- a/drivers/usb/atm/cxacru.c +++ b/drivers/usb/atm/cxacru.c @@ -700,6 +700,8 @@ static int cxacru_cm(struct cxacru_data *instance, enum cxacru_cm_request cm, ret = offd; usb_dbg(instance->usbatm, "cm %#x\n", cm); fail: + if (ret < 0) + usb_kill_urb(instance->rcv_urb); mutex_unlock(&instance->cm_serialize); err: return ret; diff --git a/drivers/usb/cdns3/cdnsp-gadget.c b/drivers/usb/cdns3/cdnsp-gadget.c index fb192b120d77..89d3d8167e59 100644 --- a/drivers/usb/cdns3/cdnsp-gadget.c +++ b/drivers/usb/cdns3/cdnsp-gadget.c @@ -154,9 +154,9 @@ static void cdnsp_set_apb_timeout_value(struct cdnsp_device *pdev) offset = cdnsp_find_next_ext_cap(base, offset, D_XEC_PRE_REGS_CAP); reg = base + offset + REG_CHICKEN_BITS_3_OFFSET; - val = le32_to_cpu(readl(reg)); + val = readl(reg); val = CHICKEN_APB_TIMEOUT_SET(val, cdns->override_apb_timeout); - writel(cpu_to_le32(val), reg); + writel(val, reg); } static void cdnsp_set_chicken_bits_2(struct cdnsp_device *pdev, u32 bit) diff --git a/drivers/usb/gadget/function/f_ncm.c b/drivers/usb/gadget/function/f_ncm.c index 40e20a0ec471..6535f3393e31 100644 --- a/drivers/usb/gadget/function/f_ncm.c +++ b/drivers/usb/gadget/function/f_ncm.c @@ -1161,7 +1161,7 @@ static int ncm_unwrap_ntb(struct gether *port, unsigned char *ntb_ptr = skb->data; __le16 *tmp; unsigned index, index2; - int ndp_index; + unsigned int ndp_index; unsigned dg_len, dg_len2; unsigned ndp_len; unsigned block_len; diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 3645d83f240d..5e2f7f447767 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -998,6 +998,7 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, unsigned int gup_flags = FOLL_LONGTERM; unsigned long npages, cur_base, map_pfn, last_pfn = 0; unsigned long lock_limit, sz2pin, nchunks, i; + unsigned long page_offset; u64 start = iova; long pinned; int ret = 0; @@ -1010,7 +1011,13 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, if (perm & VHOST_ACCESS_WO) gup_flags |= FOLL_WRITE; - npages = PFN_UP(size + (iova & ~PAGE_MASK)); + page_offset = iova & ~PAGE_MASK; + if (size > ULONG_MAX - page_offset) { + ret = -EINVAL; + goto free; + } + + npages = PFN_UP(size + page_offset); if (!npages) { ret = -EINVAL; goto free; diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 147cfb64bba2..fd8298490a31 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -1937,6 +1937,14 @@ static long vhost_vring_set_num_addr(struct vhost_dev *d, BUG(); } + /* + * The metadata cache holds the IOTLB mapping that backed the previous + * desc/avail/used addresses and vring size, both of which are being + * replaced here. iotlb_access_ok() takes a cache hit as proof that the + * region was validated, so the stale entries have to go. + */ + __vhost_vq_meta_reset(vq); + mutex_unlock(&vq->mutex); return r; diff --git a/drivers/video/fbdev/core/bitblit.c b/drivers/video/fbdev/core/bitblit.c index ed853a2aec2f..8b58f94ce44c 100644 --- a/drivers/video/fbdev/core/bitblit.c +++ b/drivers/video/fbdev/core/bitblit.c @@ -274,9 +274,14 @@ static void bit_cursor(struct vc_data *vc, struct fb_info *info, int mode, if (!vc->vc_font.data) return; - c = scr_readw((u16 *) vc->vc_pos); + c = scr_readw((u16 *) vc->vc_pos); attribute = get_attribute(info, c); - src = vc->vc_font.data + ((c & charmask) * (w * vc->vc_font.height)); + c &= charmask; + + /* Clamp to font size, same as bit_putcs_aligned() */ + if (c >= vc->vc_font.charcount) + c = 0; + src = vc->vc_font.data + (c * (w * vc->vc_font.height)); if (par->cursor_state.image.data != src || par->cursor_reset) { diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index b1e2b3f732cc..e843531201be 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -10640,6 +10640,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from, } kvfree(pages); out: + extent_changeset_free(data_reserved); if (ret >= 0) iocb->ki_pos += encoded->len; return ret; diff --git a/fs/crypto/fscrypt_private.h b/fs/crypto/fscrypt_private.h index 14b26036055e..8be25746fd72 100644 --- a/fs/crypto/fscrypt_private.h +++ b/fs/crypto/fscrypt_private.h @@ -422,6 +422,19 @@ fscrypt_is_key_prepared(struct fscrypt_prepared_key *prep_key, /* keyring.c */ +/* + * fscrypt_master_key_user - a user's claim to a master key + */ +struct fscrypt_master_key_user { + struct list_head link; + kuid_t uid; + /* + * This 'struct key' contains no secret. It exists solely to charge the + * appropriate user's key quota. + */ + struct key *quota_key; +}; + /* * fscrypt_master_key_secret - secret key material of an in-use master key */ @@ -506,19 +519,18 @@ struct fscrypt_master_key { struct fscrypt_key_specifier mk_spec; /* - * Keyring which contains a key of type 'key_type_fscrypt_user' for each - * user who has added this key. Normally each key will be added by just - * one user, but it's possible that multiple users share a key, and in - * that case we need to keep track of those users so that one user can't - * remove the key before the others want it removed too. + * List of user claims to this key (struct fscrypt_master_key_user). + * Normally each key will be added by just one user, but it's possible + * that multiple users share a key, and in that case we need to keep + * track of those users so that one user can't remove the key before the + * others want it removed too. * - * This is NULL for v1 policy keys; those can only be added by root. + * Used only for v2 policy keys. v1 policy keys can be added only by + * root, so user tracking doesn't apply to them. * - * Locking: protected by ->mk_sem. (We don't just rely on the keyrings - * subsystem semaphore ->mk_users->sem, as we need support for atomic - * search+insert along with proper synchronization with ->mk_secret.) + * Locking: protected by ->mk_sem. */ - struct key *mk_users; + struct list_head mk_users; /* * List of inodes that were unlocked using this key. This allows the diff --git a/fs/crypto/keyring.c b/fs/crypto/keyring.c index 7cbb1fd872ac..f2a0d89d5f5e 100644 --- a/fs/crypto/keyring.c +++ b/fs/crypto/keyring.c @@ -64,18 +64,19 @@ static void fscrypt_free_master_key(struct rcu_head *head) kfree_sensitive(mk); } +static void clear_mk_users(struct fscrypt_master_key *mk); + void fscrypt_put_master_key(struct fscrypt_master_key *mk) { if (!refcount_dec_and_test(&mk->mk_struct_refs)) return; /* - * No structural references left, so free ->mk_users, and also free the + * No structural references left, so clear ->mk_users, and also free the * fscrypt_master_key struct itself after an RCU grace period ensures * that concurrent keyring lookups can no longer find it. */ WARN_ON_ONCE(refcount_read(&mk->mk_active_refs) != 0); - key_put(mk->mk_users); - mk->mk_users = NULL; + clear_mk_users(mk); call_rcu(&mk->mk_rcu_head, fscrypt_free_master_key); } @@ -145,8 +146,8 @@ static void fscrypt_user_key_describe(const struct key *key, struct seq_file *m) } /* - * Type of key in ->mk_users. Each key of this type represents a particular - * user who has added a particular master key. + * Type of fscrypt_master_key_user::quota_key. This contains no secret; it + * exists solely to charge a user's key quota. * * Note that the name of this key type really should be something like * ".fscrypt-user" instead of simply ".fscrypt". But the shorter name is chosen @@ -160,30 +161,9 @@ static struct key_type key_type_fscrypt_user = { .describe = fscrypt_user_key_describe, }; -#define FSCRYPT_MK_USERS_DESCRIPTION_SIZE \ - (CONST_STRLEN("fscrypt-") + 2 * FSCRYPT_KEY_IDENTIFIER_SIZE + \ - CONST_STRLEN("-users") + 1) - #define FSCRYPT_MK_USER_DESCRIPTION_SIZE \ (2 * FSCRYPT_KEY_IDENTIFIER_SIZE + CONST_STRLEN(".uid.") + 10 + 1) -static void format_mk_users_keyring_description( - char description[FSCRYPT_MK_USERS_DESCRIPTION_SIZE], - const u8 mk_identifier[FSCRYPT_KEY_IDENTIFIER_SIZE]) -{ - sprintf(description, "fscrypt-%*phN-users", - FSCRYPT_KEY_IDENTIFIER_SIZE, mk_identifier); -} - -static void format_mk_user_description( - char description[FSCRYPT_MK_USER_DESCRIPTION_SIZE], - const u8 mk_identifier[FSCRYPT_KEY_IDENTIFIER_SIZE]) -{ - - sprintf(description, "%*phN.uid.%u", FSCRYPT_KEY_IDENTIFIER_SIZE, - mk_identifier, __kuid_val(current_fsuid())); -} - /* Create ->s_master_keys if needed. Synchronized by fscrypt_add_key_mutex. */ static int allocate_filesystem_keyring(struct super_block *sb) { @@ -319,91 +299,94 @@ fscrypt_find_master_key(struct super_block *sb, return mk; } -static int allocate_master_key_users_keyring(struct fscrypt_master_key *mk) +/* Find the current user's claim in ->mk_users. ->mk_sem must be held. */ +static struct fscrypt_master_key_user * +find_master_key_user(struct fscrypt_master_key *mk) { - char description[FSCRYPT_MK_USERS_DESCRIPTION_SIZE]; - struct key *keyring; - - format_mk_users_keyring_description(description, - mk->mk_spec.u.identifier); - keyring = keyring_alloc(description, GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, - current_cred(), KEY_POS_SEARCH | - KEY_USR_SEARCH | KEY_USR_READ | KEY_USR_VIEW, - KEY_ALLOC_NOT_IN_QUOTA, NULL, NULL); - if (IS_ERR(keyring)) - return PTR_ERR(keyring); - - mk->mk_users = keyring; - return 0; -} + struct fscrypt_master_key_user *mk_user; + kuid_t uid = current_fsuid(); -/* - * Find the current user's "key" in the master key's ->mk_users. - * Returns ERR_PTR(-ENOKEY) if not found. - */ -static struct key *find_master_key_user(struct fscrypt_master_key *mk) -{ - char description[FSCRYPT_MK_USER_DESCRIPTION_SIZE]; - key_ref_t keyref; - - format_mk_user_description(description, mk->mk_spec.u.identifier); - - /* - * We need to mark the keyring reference as "possessed" so that we - * acquire permission to search it, via the KEY_POS_SEARCH permission. - */ - keyref = keyring_search(make_key_ref(mk->mk_users, true /*possessed*/), - &key_type_fscrypt_user, description, false); - if (IS_ERR(keyref)) { - if (PTR_ERR(keyref) == -EAGAIN || /* not found */ - PTR_ERR(keyref) == -EKEYREVOKED) /* recently invalidated */ - keyref = ERR_PTR(-ENOKEY); - return ERR_CAST(keyref); + list_for_each_entry(mk_user, &mk->mk_users, link) { + if (uid_eq(mk_user->uid, uid)) + return mk_user; } - return key_ref_to_ptr(keyref); + return NULL; } /* - * Give the current user a "key" in ->mk_users. This charges the user's quota + * Give the current user a claim in ->mk_users. This charges the user's quota * and marks the master key as added by the current user, so that it cannot be * removed by another user with the key. Either ->mk_sem must be held for * write, or the master key must be still undergoing initialization. */ static int add_master_key_user(struct fscrypt_master_key *mk) { + kuid_t uid = current_fsuid(); char description[FSCRYPT_MK_USER_DESCRIPTION_SIZE]; - struct key *mk_user; + struct key *quota_key; + struct fscrypt_master_key_user *mk_user; int err; - format_mk_user_description(description, mk->mk_spec.u.identifier); - mk_user = key_alloc(&key_type_fscrypt_user, description, - current_fsuid(), current_gid(), current_cred(), - KEY_POS_SEARCH | KEY_USR_VIEW, 0, NULL); - if (IS_ERR(mk_user)) - return PTR_ERR(mk_user); + snprintf(description, sizeof(description), "%*phN.uid.%u", + FSCRYPT_KEY_IDENTIFIER_SIZE, mk->mk_spec.u.identifier, + __kuid_val(uid)); + quota_key = key_alloc(&key_type_fscrypt_user, description, uid, + current_gid(), current_cred(), + KEY_POS_SEARCH | KEY_USR_VIEW, 0, NULL); + if (IS_ERR(quota_key)) + return PTR_ERR(quota_key); + + err = key_instantiate_and_link(quota_key, NULL, 0, NULL, NULL); + if (err) { + key_put(quota_key); + return err; + } - err = key_instantiate_and_link(mk_user, NULL, 0, mk->mk_users, NULL); - key_put(mk_user); - return err; + mk_user = kzalloc(sizeof(*mk_user), GFP_KERNEL); + if (!mk_user) { + key_put(quota_key); + return -ENOMEM; + } + mk_user->uid = uid; + mk_user->quota_key = quota_key; + list_add(&mk_user->link, &mk->mk_users); + return 0; +} + +static void unlink_and_free_mk_user(struct fscrypt_master_key_user *mk_user) +{ + list_del(&mk_user->link); + key_put(mk_user->quota_key); + kfree(mk_user); } /* - * Remove the current user's "key" from ->mk_users. + * Remove the current user's claim from ->mk_users. * ->mk_sem must be held for write. * - * Returns 0 if removed, -ENOKEY if not found, or another -errno code. + * Returns 0 if removed or -ENOKEY if not found. */ static int remove_master_key_user(struct fscrypt_master_key *mk) { - struct key *mk_user; - int err; + struct fscrypt_master_key_user *mk_user; mk_user = find_master_key_user(mk); - if (IS_ERR(mk_user)) - return PTR_ERR(mk_user); - err = key_unlink(mk->mk_users, mk_user); - key_put(mk_user); - return err; + if (!mk_user) + return -ENOKEY; + unlink_and_free_mk_user(mk_user); + return 0; +} + +/* + * Clear ->mk_users. Either ->mk_sem must be held for write, or 'mk' must have + * no structural references left. + */ +static void clear_mk_users(struct fscrypt_master_key *mk) +{ + struct fscrypt_master_key_user *mk_user, *tmp; + + list_for_each_entry_safe(mk_user, tmp, &mk->mk_users, link) + unlink_and_free_mk_user(mk_user); } /* @@ -426,13 +409,12 @@ static int add_new_master_key(struct super_block *sb, refcount_set(&mk->mk_struct_refs, 1); mk->mk_spec = *mk_spec; + INIT_LIST_HEAD(&mk->mk_users); + INIT_LIST_HEAD(&mk->mk_decrypted_inodes); spin_lock_init(&mk->mk_decrypted_inodes_lock); if (mk_spec->type == FSCRYPT_KEY_SPEC_TYPE_IDENTIFIER) { - err = allocate_master_key_users_keyring(mk); - if (err) - goto out_put; err = add_master_key_user(mk); if (err) goto out_put; @@ -460,19 +442,13 @@ static int add_existing_master_key(struct fscrypt_master_key *mk, int err; /* - * If the current user is already in ->mk_users, then there's nothing to - * do. Otherwise, we need to add the user to ->mk_users. (Neither is - * applicable for v1 policy keys, which have NULL ->mk_users.) + * For v2 policy keys (FSCRYPT_KEY_SPEC_TYPE_IDENTIFIER): If the current + * user is already in ->mk_users, then there's nothing to do. + * Otherwise, add the user to ->mk_users. */ - if (mk->mk_users) { - struct key *mk_user = find_master_key_user(mk); - - if (mk_user != ERR_PTR(-ENOKEY)) { - if (IS_ERR(mk_user)) - return PTR_ERR(mk_user); - key_put(mk_user); + if (mk->mk_spec.type == FSCRYPT_KEY_SPEC_TYPE_IDENTIFIER) { + if (find_master_key_user(mk) != NULL) return 0; - } err = add_master_key_user(mk); if (err) return err; @@ -819,7 +795,6 @@ int fscrypt_verify_key_added(struct super_block *sb, { struct fscrypt_key_specifier mk_spec; struct fscrypt_master_key *mk; - struct key *mk_user; int err; mk_spec.type = FSCRYPT_KEY_SPEC_TYPE_IDENTIFIER; @@ -831,13 +806,10 @@ int fscrypt_verify_key_added(struct super_block *sb, goto out; } down_read(&mk->mk_sem); - mk_user = find_master_key_user(mk); - if (IS_ERR(mk_user)) { - err = PTR_ERR(mk_user); - } else { - key_put(mk_user); + if (find_master_key_user(mk) != NULL) err = 0; - } + else + err = -ENOKEY; up_read(&mk->mk_sem); fscrypt_put_master_key(mk); out: @@ -1030,16 +1002,18 @@ static int do_remove_key(struct file *filp, void __user *_uarg, bool all_users) down_write(&mk->mk_sem); /* If relevant, remove current user's (or all users) claim to the key */ - if (mk->mk_users && mk->mk_users->keys.nr_leaves_on_tree != 0) { - if (all_users) - err = keyring_clear(mk->mk_users); - else + if (!list_empty(&mk->mk_users)) { + if (all_users) { + clear_mk_users(mk); + err = 0; + } else { err = remove_master_key_user(mk); + } if (err) { up_write(&mk->mk_sem); goto out_put_key; } - if (mk->mk_users->keys.nr_leaves_on_tree != 0) { + if (!list_empty(&mk->mk_users)) { /* * Other users have still added the key too. We removed * the current user's claim to the key, but we still @@ -1127,6 +1101,8 @@ int fscrypt_ioctl_get_key_status(struct file *filp, void __user *uarg) struct super_block *sb = file_inode(filp)->i_sb; struct fscrypt_get_key_status_arg arg; struct fscrypt_master_key *mk; + kuid_t uid; + const struct fscrypt_master_key_user *mk_user; int err; if (copy_from_user(&arg, uarg, sizeof(arg))) @@ -1159,19 +1135,13 @@ int fscrypt_ioctl_get_key_status(struct file *filp, void __user *uarg) } arg.status = FSCRYPT_KEY_STATUS_PRESENT; - if (mk->mk_users) { - struct key *mk_user; - arg.user_count = mk->mk_users->keys.nr_leaves_on_tree; - mk_user = find_master_key_user(mk); - if (!IS_ERR(mk_user)) { + uid = current_fsuid(); + list_for_each_entry(mk_user, &mk->mk_users, link) { + arg.user_count++; + if (uid_eq(mk_user->uid, uid)) arg.status_flags |= FSCRYPT_KEY_STATUS_FLAG_ADDED_BY_SELF; - key_put(mk_user); - } else if (mk_user != ERR_PTR(-ENOKEY)) { - err = PTR_ERR(mk_user); - goto out_release_key; - } } err = 0; out_release_key: diff --git a/fs/crypto/policy.c b/fs/crypto/policy.c index f4456ecb3f87..2ba184ceb4d7 100644 --- a/fs/crypto/policy.c +++ b/fs/crypto/policy.c @@ -505,7 +505,7 @@ int fscrypt_ioctl_set_policy(struct file *filp, const void __user *arg) return -EFAULT; policy.version = version; - if (!inode_owner_or_capable(&nop_mnt_idmap, inode)) + if (!inode_owner_or_capable(file_mnt_idmap(filp), inode)) return -EACCES; ret = mnt_want_write_file(filp); diff --git a/fs/namespace.c b/fs/namespace.c index 646d9e7d41ee..f185de5f5df4 100644 --- a/fs/namespace.c +++ b/fs/namespace.c @@ -4050,6 +4050,11 @@ SYSCALL_DEFINE3(fsmount, int, fs_fd, unsigned int, flags, ret = PTR_ERR(newmount.mnt); goto err_unlock; } + if (newmount.mnt->mnt_sb->s_flags & SB_NOUSER) { + mntput(newmount.mnt); + ret = -EINVAL; + goto err_unlock; + } newmount.dentry = dget(fc->root); newmount.mnt->mnt_flags = mnt_flags; diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c index 42fa7c915e29..ce3ab0a9c0ef 100644 --- a/fs/nfs/nfs4proc.c +++ b/fs/nfs/nfs4proc.c @@ -10383,6 +10383,7 @@ static void nfs41_free_stateid_release(void *calldata) struct nfs_free_stateid_data *data = calldata; struct nfs_client *clp = data->server->nfs_client; + nfs_sb_deactive(data->server->super); nfs_put_client(clp); kfree(calldata); } @@ -10424,6 +10425,10 @@ static int nfs41_free_stateid(struct nfs_server *server, if (!refcount_inc_not_zero(&clp->cl_count)) return -EIO; + if (!nfs_sb_active(server->super)) { + nfs_put_client(clp); + return -EIO; + } nfs4_state_protect(server->nfs_client, NFS_SP4_MACH_CRED_STATEID, &task_setup.rpc_client, &msg); diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c index 93ee57bc82ad..76316d143aaa 100644 --- a/fs/overlayfs/super.c +++ b/fs/overlayfs/super.c @@ -1345,7 +1345,8 @@ int ovl_fill_super(struct super_block *sb, struct fs_context *fc) int err; err = -EIO; - if (WARN_ON(fc->user_ns != current_user_ns())) + /* The fscontext fd may have been passed to another user namespace. */ + if (fc->user_ns != current_user_ns()) goto out_err; sb->s_d_op = &ovl_dentry_operations; diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c index bbde7180a90a..6fac7a107592 100644 --- a/fs/smb/client/sess.c +++ b/fs/smb/client/sess.c @@ -271,9 +271,9 @@ int cifs_try_adding_channels(struct cifs_ses *ses) cifs_dbg(VFS, "failed to open extra channel on iface:%pIS rc=%d\n", &iface->sockaddr, rc); - kref_put(&iface->refcount, release_iface); /* failure to add chan should increase weight */ iface->weight_fulfilled++; + kref_put(&iface->refcount, release_iface); continue; } diff --git a/fs/tracefs/event_inode.c b/fs/tracefs/event_inode.c index 36257e1092d6..4ac459c7efc3 100644 --- a/fs/tracefs/event_inode.c +++ b/fs/tracefs/event_inode.c @@ -922,7 +922,7 @@ struct eventfs_inode *eventfs_create_events_dir(const char *name, struct dentry */ static void eventfs_remove_rec(struct eventfs_inode *ei, int level) { - struct eventfs_inode *ei_child; + struct eventfs_inode *ei_child, *tmp; /* * Check recursion depth. It should never be greater than 3: @@ -935,7 +935,7 @@ static void eventfs_remove_rec(struct eventfs_inode *ei, int level) return; /* search for nested folders or files */ - list_for_each_entry(ei_child, &ei->children, list) + list_for_each_entry_safe(ei_child, tmp, &ei->children, list) eventfs_remove_rec(ei_child, level + 1); list_del_rcu(&ei->list); diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 74474139fb9a..4901f765a14d 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -301,9 +301,11 @@ struct hh_cache { * We could use other alignment values, but we must maintain the * relationship HH alignment <= LL alignment. */ -#define LL_RESERVED_SPACE(dev) \ - ((((dev)->hard_header_len + READ_ONCE((dev)->needed_headroom)) \ +#define LL_RESERVED_SPACE_EX(dev, hlen) \ + ((((hlen) + READ_ONCE((dev)->needed_headroom)) \ & ~(HH_DATA_MOD - 1)) + HH_DATA_MOD) +#define LL_RESERVED_SPACE(dev) \ + LL_RESERVED_SPACE_EX(dev, (dev)->hard_header_len) #define LL_RESERVED_SPACE_EXTRA(dev,extra) \ ((((dev)->hard_header_len + READ_ONCE((dev)->needed_headroom) + (extra)) \ & ~(HH_DATA_MOD - 1)) + HH_DATA_MOD) @@ -3197,11 +3199,6 @@ static inline bool dev_validate_header(const struct net_device *dev, if (len < dev->min_header_len) return false; - if (capable(CAP_SYS_RAWIO)) { - memset(ll_header + len, 0, dev->hard_header_len - len); - return true; - } - if (dev->header_ops && dev->header_ops->validate) return dev->header_ops->validate(ll_header, len); diff --git a/include/linux/netfilter/ipset/ip_set.h b/include/linux/netfilter/ipset/ip_set.h index b98331572ad2..cadae9b2578f 100644 --- a/include/linux/netfilter/ipset/ip_set.h +++ b/include/linux/netfilter/ipset/ip_set.h @@ -273,7 +273,7 @@ struct ip_set { /* Number of elements (vs timeout) */ u32 elements; /* Size of the dynamic extensions (vs timeout) */ - size_t ext_size; + atomic64_t ext_size; /* Element data size */ size_t dsize; /* Offsets to extensions in elements */ diff --git a/include/linux/regulator/consumer.h b/include/linux/regulator/consumer.h index 25d0684d37b3..9a2f1e9a94a8 100644 --- a/include/linux/regulator/consumer.h +++ b/include/linux/regulator/consumer.h @@ -209,6 +209,7 @@ struct regulator *__must_check devm_regulator_get_optional(struct device *dev, const char *id); int devm_regulator_get_enable(struct device *dev, const char *id); int devm_regulator_get_enable_optional(struct device *dev, const char *id); +int devm_regulator_get_enable_read_voltage(struct device *dev, const char *id); void regulator_put(struct regulator *regulator); void devm_regulator_put(struct regulator *regulator); @@ -374,6 +375,12 @@ static inline int devm_regulator_get_enable_optional(struct device *dev, return 0; } +static inline int devm_regulator_get_enable_read_voltage(struct device *dev, + const char *id) +{ + return -ENODEV; +} + static inline struct regulator *__must_check regulator_get_optional(struct device *dev, const char *id) { diff --git a/include/linux/tcp.h b/include/linux/tcp.h index e15452df9804..c38778b0baa0 100644 --- a/include/linux/tcp.h +++ b/include/linux/tcp.h @@ -445,13 +445,18 @@ struct tcp_sock { bool syn_smc; /* SYN includes SMC */ #endif -#ifdef CONFIG_TCP_MD5SIG -/* TCP AF-Specific parts; only used by MD5 Signature support so far */ +#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO) +/* TCP AF-Specific parts; only used by TCP-AO/MD5 Signature support so far */ const struct tcp_sock_af_ops *af_specific; +#ifdef CONFIG_TCP_MD5SIG /* TCP MD5 Signature Option information */ struct tcp_md5sig_info __rcu *md5sig_info; #endif +#ifdef CONFIG_TCP_AO + struct tcp_ao_info __rcu *ao_info; +#endif +#endif /* TCP fastopen related information */ struct tcp_fastopen_request *fastopen_req; diff --git a/include/net/act_api.h b/include/net/act_api.h index 56f49351fd51..b196134e4205 100644 --- a/include/net/act_api.h +++ b/include/net/act_api.h @@ -268,6 +268,25 @@ int tcf_action_check_ctrlact(int action, struct tcf_proto *tp, struct tcf_chain *tcf_action_set_ctrlact(struct tc_action *a, int action, struct tcf_chain *newchain); +/* Range check for a control action supplied by user space. + * + * This is the same test tcf_action_check_ctrlact() applies to the primary + * control action, factored out for the *fallback* control actions + * (act_gact's TCA_GACT_PROB.paction and act_police's TCA_POLICE_RESULT), + * which must not reach tcf_action_check_ctrlact() because they have no + * goto_chain to allocate. Without it, user space can store kernel-internal + * verdicts such as TC_ACT_CONSUMED, which is TC_ACT_VALUE_MAX + 1 and is + * deliberately not part of the UAPI value range. + */ +static inline bool tcf_action_valid(int action) +{ + int opcode = TC_ACT_EXT_OPCODE(action); + + if (!opcode) + return action <= TC_ACT_VALUE_MAX; + return opcode <= TC_ACT_EXT_OPCODE_MAX || action == TC_ACT_UNSPEC; +} + #ifdef CONFIG_INET DECLARE_STATIC_KEY_FALSE(tcf_frag_xmit_count); #endif diff --git a/include/net/addrconf.h b/include/net/addrconf.h index 0a9b055a8ee1..5db1d088d819 100644 --- a/include/net/addrconf.h +++ b/include/net/addrconf.h @@ -370,8 +370,8 @@ static inline struct inet6_dev *in6_dev_get(const struct net_device *dev) rcu_read_lock(); idev = rcu_dereference(dev->ip6_ptr); - if (idev) - refcount_inc(&idev->refcnt); + if (idev && !refcount_inc_not_zero(&idev->refcnt)) + idev = NULL; rcu_read_unlock(); return idev; } diff --git a/include/net/ip_vs.h b/include/net/ip_vs.h index 6935ec09af24..e0077fd28ad1 100644 --- a/include/net/ip_vs.h +++ b/include/net/ip_vs.h @@ -24,9 +24,7 @@ #include /* for union nf_inet_addr */ #include #include /* for struct ipv6hdr */ -#include #include -#include #if IS_ENABLED(CONFIG_NF_CONNTRACK) #include #endif @@ -738,10 +736,11 @@ struct ip_vs_dest { /* connection counters and thresholds */ atomic_t activeconns; /* active connections */ - atomic_t inactconns; /* inactive connections */ + atomic_t totalconns; /* total connections */ atomic_t persistconns; /* persistent connections */ __u32 u_threshold; /* upper threshold */ __u32 l_threshold; /* lower threshold */ + __u32 l_threshold_val;/* used lower threshold */ /* for destination cache */ spinlock_t dst_lock; /* lock of dst_cache */ @@ -1569,6 +1568,8 @@ static inline void ip_vs_dest_put_and_free(struct ip_vs_dest *dest) kfree(dest); } +void ip_vs_dest_update_overload(struct ip_vs_dest *dest, int mode); + /* IPVS sync daemon data and function prototypes * (from ip_vs_sync.c) */ @@ -1707,7 +1708,7 @@ static inline char ip_vs_fwd_tag(struct ip_vs_conn *cp) void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, struct ip_vs_conn *cp, int dir, unsigned int toff, - bool has_ports); + bool has_ports, struct ip_vs_iphdr *ciph); #ifdef CONFIG_IP_VS_IPV6 void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, @@ -1740,30 +1741,23 @@ static inline __wsum ip_vs_check_diff2(__be16 old, __be16 new, __wsum oldsum) return csum_partial(diff, sizeof(diff), oldsum); } -static inline bool ip_vs_checksum_needed(struct sk_buff *skb, int af) +static inline bool ip_vs_checksum_needed(struct sk_buff *skb) { /* Checksum unnecessary or already validated? */ if (skb_csum_unnecessary(skb)) return false; - /* LOCAL_OUT ? */ - if (!skb->dev || skb->dev->flags & IFF_LOOPBACK) + /* Locally generated ? */ + if (!skb->dev) return false; - /* !LOCAL_IN (FORWARD) ? */ - if (af == AF_INET6) { - if (!(dst_rt6_info(skb_dst(skb))->rt6i_flags & RTF_LOCAL)) - return false; - } else { - if (!(skb_rtable(skb)->rt_flags & RTCF_LOCAL)) - return false; - } return true; } static inline bool ip_vs_checksum_common_check(struct sk_buff *skb, int offset, int proto, int af) { - if (!ip_vs_checksum_needed(skb, af)) + if (!ip_vs_checksum_needed(skb)) return true; + /* Validate csum even for FORWARD */ return !nf_checksum(skb, NF_INET_LOCAL_IN, offset, proto, af); } @@ -1874,14 +1868,21 @@ void ip_vs_unregister_hooks(struct netns_ipvs *ipvs, unsigned int af); static inline int ip_vs_dest_conn_overhead(struct ip_vs_dest *dest) { - /* We think the overhead of processing active connections is 256 + /* We think the overhead of processing active connections is 257 * times higher than that of inactive connections in average. (This - * 256 times might not be accurate, we will change it later) We + * 257 times might not be accurate, we will change it later) We * use the following formula to estimate the overhead now: - * dest->activeconns*256 + dest->inactconns + * dest->activeconns*256 + dest->totalconns */ return (atomic_read(&dest->activeconns) << 8) + - atomic_read(&dest->inactconns); + atomic_read(&dest->totalconns); +} + +static inline int +ip_vs_dest_inactconns(const struct ip_vs_dest *dest) +{ + return max(atomic_read(&dest->totalconns) - + atomic_read(&dest->activeconns), 0); } #ifdef CONFIG_IP_VS_PROTO_TCP diff --git a/include/net/pkt_cls.h b/include/net/pkt_cls.h index ccc1c698ed00..307478c23322 100644 --- a/include/net/pkt_cls.h +++ b/include/net/pkt_cls.h @@ -72,6 +72,15 @@ static inline bool tcf_block_non_null_shared(struct tcf_block *block) return block && block->index; } +#ifdef CONFIG_NET_CLS_ACT +DECLARE_STATIC_KEY_FALSE(tcf_sw_enabled_key); + +static inline bool tcf_block_bypass_sw(struct tcf_block *block) +{ + return block && !atomic_read(&block->useswcnt); +} +#endif + static inline struct Qdisc *tcf_block_q(struct tcf_block *block) { WARN_ON(tcf_block_shared(block)); @@ -750,6 +759,15 @@ tc_cls_common_offload_init(struct flow_cls_common_offload *cls_common, cls_common->extack = extack; } +static inline void tcf_proto_update_usesw(struct tcf_proto *tp, u32 flags) +{ + if (tp->usesw) + return; + if (tc_skip_sw(flags) && tc_in_hw(flags)) + return; + tp->usesw = true; +} + #if IS_ENABLED(CONFIG_NET_TC_SKB_EXT) static inline struct tc_skb_ext *tc_skb_ext_alloc(struct sk_buff *skb) { diff --git a/include/net/route.h b/include/net/route.h index 22d8095a3a81..1d5d975c1707 100644 --- a/include/net/route.h +++ b/include/net/route.h @@ -244,6 +244,8 @@ int fib_dump_info_fnhe(struct sk_buff *skb, struct netlink_callback *cb, u32 table_id, struct fib_info *fi, int *fa_index, int fa_start, unsigned int flags); +void fnhe_update_pmtu(struct fib_nh_exception *fnhe, u32 new, u32 orig); + static inline void ip_rt_put(struct rtable *rt) { /* dst_release() accepts a NULL parameter. diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h index 385af747b0b4..8ad29d2c210a 100644 --- a/include/net/sch_generic.h +++ b/include/net/sch_generic.h @@ -100,6 +100,7 @@ struct Qdisc { struct hlist_node hash; u32 handle; u32 parent; + int depth; struct netdev_queue *dev_queue; @@ -429,6 +430,8 @@ struct tcf_proto { */ spinlock_t lock; bool deleting; + bool counted; + bool usesw; refcount_t refcnt; struct rcu_head rcu; struct hlist_node destroy_ht_node; @@ -477,6 +480,7 @@ struct tcf_block { struct flow_block flow_block; struct list_head owner_list; bool keep_dst; + atomic_t useswcnt; atomic_t offloadcnt; /* Number of oddloaded filters */ unsigned int nooffloaddevcnt; /* Number of devs unable to do offload */ unsigned int lockeddevcnt; /* Number of devs that require rtnl lock. */ diff --git a/include/net/tcp.h b/include/net/tcp.h index a6def0aab3ed..0eb9341f7488 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -37,6 +37,7 @@ #include #include #include +#include #include #include #include @@ -1681,12 +1682,7 @@ static inline void tcp_clear_all_retrans_hints(struct tcp_sock *tp) tp->retransmit_skb_hint = NULL; } -union tcp_md5_addr { - struct in_addr a4; -#if IS_ENABLED(CONFIG_IPV6) - struct in6_addr a6; -#endif -}; +#define tcp_md5_addr tcp_ao_addr /* - key database */ struct tcp_md5sig_key { @@ -1730,12 +1726,39 @@ union tcp_md5sum_block { #endif }; -/* - pool: digest algorithm, hash description and scratch buffer */ -struct tcp_md5sig_pool { - struct ahash_request *md5_req; - void *scratch; +/* + * struct tcp_sigpool - per-CPU pool of ahash_requests + * @scratch: per-CPU temporary area, that can be used between + * tcp_sigpool_start() and tcp_sigpool_end() to perform + * crypto request + * @req: pre-allocated ahash request + */ +struct tcp_sigpool { + void *scratch; + struct ahash_request *req; }; +int tcp_sigpool_alloc_ahash(const char *alg, size_t scratch_size); +void tcp_sigpool_get(unsigned int id); +void tcp_sigpool_release(unsigned int id); +int tcp_sigpool_hash_skb_data(struct tcp_sigpool *hp, + const struct sk_buff *skb, + unsigned int header_len); + +/** + * tcp_sigpool_start - disable bh and start using tcp_sigpool_ahash + * @id: tcp_sigpool that was previously allocated by tcp_sigpool_alloc_ahash() + * @c: returned tcp_sigpool for usage (uninitialized on failure) + * + * Returns 0 on success, error otherwise. + */ +int tcp_sigpool_start(unsigned int id, struct tcp_sigpool *c); +/** + * tcp_sigpool_end - enable bh and stop using tcp_sigpool + * @c: tcp_sigpool context that was returned by tcp_sigpool_start() + */ +void tcp_sigpool_end(struct tcp_sigpool *c); +size_t tcp_sigpool_algo(unsigned int id, char *buf, size_t buf_len); /* - functions */ int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key, const struct sock *sk, const struct sk_buff *skb); @@ -1791,17 +1814,12 @@ tcp_inbound_md5_hash(const struct sock *sk, const struct sk_buff *skb, #define tcp_twsk_md5_key(twsk) NULL #endif -bool tcp_alloc_md5sig_pool(void); - -struct tcp_md5sig_pool *tcp_get_md5sig_pool(void); -static inline void tcp_put_md5sig_pool(void) -{ - local_bh_enable(); -} +int tcp_md5_alloc_sigpool(void); +void tcp_md5_release_sigpool(void); +void tcp_md5_add_sigpool(void); +extern int tcp_md5_sigpool_id; -int tcp_md5_hash_skb_data(struct tcp_md5sig_pool *, const struct sk_buff *, - unsigned int header_len); -int tcp_md5_hash_key(struct tcp_md5sig_pool *hp, +int tcp_md5_hash_key(struct tcp_sigpool *hp, const struct tcp_md5sig_key *key); /* From tcp_fastopen.c */ diff --git a/include/net/tcp_ao.h b/include/net/tcp_ao.h new file mode 100644 index 000000000000..af76e1c47bea --- /dev/null +++ b/include/net/tcp_ao.h @@ -0,0 +1,90 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +#ifndef _TCP_AO_H +#define _TCP_AO_H + +#define TCP_AO_KEY_ALIGN 1 +#define __tcp_ao_key_align __aligned(TCP_AO_KEY_ALIGN) + +union tcp_ao_addr { + struct in_addr a4; +#if IS_ENABLED(CONFIG_IPV6) + struct in6_addr a6; +#endif +}; + +struct tcp_ao_hdr { + u8 kind; + u8 length; + u8 keyid; + u8 rnext_keyid; +}; + +struct tcp_ao_key { + struct hlist_node node; + union tcp_ao_addr addr; + u8 key[TCP_AO_MAXKEYLEN] __tcp_ao_key_align; + unsigned int tcp_sigpool_id; + unsigned int digest_size; + u8 prefixlen; + u8 family; + u8 keylen; + u8 keyflags; + u8 sndid; + u8 rcvid; + u8 maclen; + struct rcu_head rcu; + u8 traffic_keys[]; +}; + +static inline u8 *rcv_other_key(struct tcp_ao_key *key) +{ + return key->traffic_keys; +} + +static inline u8 *snd_other_key(struct tcp_ao_key *key) +{ + return key->traffic_keys + key->digest_size; +} + +static inline int tcp_ao_maclen(const struct tcp_ao_key *key) +{ + return key->maclen; +} + +static inline int tcp_ao_len(const struct tcp_ao_key *key) +{ + return tcp_ao_maclen(key) + sizeof(struct tcp_ao_hdr); +} + +static inline unsigned int tcp_ao_digest_size(struct tcp_ao_key *key) +{ + return key->digest_size; +} + +static inline int tcp_ao_sizeof_key(const struct tcp_ao_key *key) +{ + return sizeof(struct tcp_ao_key) + (key->digest_size << 1); +} + +struct tcp_ao_info { + /* List of tcp_ao_key's */ + struct hlist_head head; + /* current_key and rnext_key aren't maintained on listen sockets. + * Their purpose is to cache keys on established connections, + * saving needless lookups. Never dereference any of them from + * listen sockets. + * ::current_key may change in RX to the key that was requested by + * the peer, please use READ_ONCE()/WRITE_ONCE() in order to avoid + * load/store tearing. + * Do the same for ::rnext_key, if you don't hold socket lock + * (it's changed only by userspace request in setsockopt()). + */ + struct tcp_ao_key *current_key; + struct tcp_ao_key *rnext_key; + u32 flags; + __be32 lisn; + __be32 risn; + struct rcu_head rcu; +}; + +#endif /* _TCP_AO_H */ diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h index d1d08da6331a..bf93a80809d6 100644 --- a/include/uapi/linux/tcp.h +++ b/include/uapi/linux/tcp.h @@ -360,6 +360,8 @@ struct tcp_diag_md5sig { __u8 tcpm_key[TCP_MD5SIG_MAXKEYLEN]; }; +#define TCP_AO_MAXKEYLEN 80 + /* setsockopt(fd, IPPROTO_TCP, TCP_ZEROCOPY_RECEIVE, ...) */ #define TCP_RECEIVE_ZEROCOPY_FLAG_TLB_CLEAN_HINT 0x1 diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5e094c12fc94..ad679bc42427 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12657,11 +12657,12 @@ static int adjust_ptr_min_max_vals(struct bpf_verifier_env *env, break; } - /* In case of 'scalar += pointer', dst_reg inherits pointer type and id. - * The id may be overwritten later if we create a new variable offset. + /* For 'scalar += pointer', dst_reg inherits the complete pointer + * register state. Individual fields may be adjusted later by pointer + * arithmetic. Callers guarantee that below does not overwrite off_reg. */ - dst_reg->type = ptr_reg->type; - dst_reg->id = ptr_reg->id; + if (dst_reg != ptr_reg) + *dst_reg = *ptr_reg; if (!check_reg_sane_offset(env, off_reg, ptr_reg->type) || !check_reg_sane_offset(env, ptr_reg, ptr_reg->type)) @@ -12729,7 +12730,7 @@ static int adjust_ptr_min_max_vals(struct bpf_verifier_env *env, } break; case BPF_SUB: - if (dst_reg == off_reg) { + if (dst_reg != ptr_reg) { /* scalar -= pointer. Creates an unknown scalar */ verbose(env, "R%d tried to subtract pointer from scalar\n", dst); @@ -13588,8 +13589,8 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, err = mark_chain_precision(env, insn->dst_reg); if (err) return err; - return adjust_ptr_min_max_vals(env, insn, - src_reg, dst_reg); + off_reg = *dst_reg; + return adjust_ptr_min_max_vals(env, insn, src_reg, &off_reg); } } else if (ptr_reg) { /* pointer += scalar */ diff --git a/kernel/futex/core.c b/kernel/futex/core.c index f30a93e50f65..660d7061740b 100644 --- a/kernel/futex/core.c +++ b/kernel/futex/core.c @@ -660,8 +660,11 @@ static int handle_futex_death(u32 __user *uaddr, struct task_struct *curr, return -1; /* - * Special case for regular (non PI) futexes. The unlock path in - * user space has two race scenarios: + * Special case for regular (non PI) futexes. Ordinarily, we do + * not perform any processing here unless the current thread was + * the owner of the futex (by the TID check below). + * + * However, the unlock path has three race scenarios: * * 1. The unlock path releases the user space futex value and * before it can execute the futex() syscall to wake up @@ -670,41 +673,68 @@ static int handle_futex_death(u32 __user *uaddr, struct task_struct *curr, * 2. A woken up waiter is killed before it can acquire the * futex in user space. * - * In the second case, the wake up notification could be generated - * by the unlock path in user space after setting the futex value - * to zero or by the kernel after setting the OWNER_DIED bit below. + * 3. A woken up waiter is killed in user space after another + * thread has acquired the futex, but before it can set + * FUTEX_WAITERS. + * + * Note that, if userspace uses the FUTEX_ROBUST_UNLOCK flag, we + * will not see case 1 here. + * + * In the second and third case, the wake up notification could + * be generated from any of: + * + * i. An ordinary futex wakeup after unlock (with or + * without FUTEX_ROBUST_UNLOCK) + * ii. A robust wakeup from another thread's death + * iii. A previous round through this special case + * + * As a result, the futex world will be in one of four states: * - * In both cases the TID validation below prevents a wakeup of - * potential waiters which can cause these waiters to block - * forever. + * A. The futex word is 0 (unlocked) + * B. The futex word is owned by another thread + * (FUTEX_WAITERS is not set) + * C. The futex word is owned by another thread + * (FUTEX_WAITERS set) + * D. The futex's owner died and OWNER_DIED is set + * (the owner part of the word is 0) * - * In both cases the following conditions are met: + * The key issue is that the kernel usually (at least from + * sources ii. and iii. or when so requested by userspace from + * source i.) only ever wakes *one* waiter at a time. If this + * waiter dies before acquiring the futex (or setting the + * FUTEX_WAITERS bit), the kernel *must* still wake the next + * waiter down the line to uphold the futex invariants and + * avoid lost wakeups. Note we do not need to handle state C, + * as it does not matter to us whether *we* successfully set + * the bit or a third thread did so in the meantime. * - * 1) task->robust_list->list_op_pending != NULL - * @pending_op == true - * 2) The owner part of user space futex value == 0 + * Therefore, in these cases we must issue an additional + * futex_wake(). Note however that we *must not* set OWNER_DIED + * here. Our thread is *not* the owner of the futex. + * + * Thus to summarize, the conditions for needing the additional + * futex_wake() are: + * + * 1) @pending_op == true (the thread has not finished the + * mutex operation) + * 2) The futex word is in one of the states A, B or D * 3) Regular futex: @pi == false * - * If these conditions are met, it is safe to attempt waking up a - * potential waiter without touching the user space futex value and - * trying to set the OWNER_DIED bit. If the futex value is zero, - * the rest of the user space mutex state is consistent, so a woken - * waiter will just take over the uncontended futex. Setting the - * OWNER_DIED bit would create inconsistent state and malfunction - * of the user space owner died handling. Otherwise, the OWNER_DIED - * bit is already set, and the woken waiter is expected to deal with - * this. + * Note in particular that in all of the states A-D the owner + * portion of the futex word differs from our thread's TID + * (unless the actual owner has the same TID in another PID + * namespace, but we cannot currently distinguish that + * scenario), so this can be a special-case wakeup in the bail + * path of the ordinary TID check. */ owner = uval & FUTEX_TID_MASK; - if (pending_op && !pi && !owner) { - futex_wake(uaddr, 1, 1, FUTEX_BITSET_MATCH_ANY); + if (owner != task_pid_vnr(curr)) { + if (pending_op && !pi && (!owner || !(uval & FUTEX_WAITERS))) + futex_wake(uaddr, 1, 1, FUTEX_BITSET_MATCH_ANY); return 0; } - if (owner != task_pid_vnr(curr)) - return 0; - /* * Ok, this dying thread is truly holding a futex * of interest. Set the OWNER_DIED bit atomically diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c index 08a9a9f909d5..b7a2fd895eb5 100644 --- a/kernel/sched/psi.c +++ b/kernel/sched/psi.c @@ -1138,6 +1138,12 @@ void psi_cgroup_free(struct cgroup *cgroup) return; cancel_delayed_work_sync(&cgroup->psi->avgs_work); + /* + * A psi_schedule_rtpoll_work() call racing the last trigger's + * destruction may have re-armed the timer after psi_trigger_destroy() + * deleted it. Spurious firing while the group is alive is harmless. + */ + timer_shutdown_sync(&cgroup->psi->rtpoll_timer); free_percpu(cgroup->psi->pcpu); /* All triggers must be removed by now */ WARN_ONCE(cgroup->psi->rtpoll_states, "psi: trigger leak\n"); diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 0f761bca5135..0b2ed121741e 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -6090,7 +6090,7 @@ static __init int test_ringbuffer(void) out_free: for_each_online_cpu(cpu) { - if (!rb_threads[cpu]) + if (IS_ERR_OR_NULL(rb_threads[cpu])) break; kthread_stop(rb_threads[cpu]); } diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c index 9588344ef8f7..bd8167d006f9 100644 --- a/kernel/trace/trace_events.c +++ b/kernel/trace/trace_events.c @@ -3038,6 +3038,7 @@ void trace_event_eval_update(struct trace_eval_map **map, int len) int last_i; int i; + mutex_lock(&event_mutex); down_write(&trace_event_sem); list_for_each_entry_safe(call, p, &ftrace_events, list) { /* events are usually grouped together with systems */ @@ -3071,6 +3072,7 @@ void trace_event_eval_update(struct trace_eval_map **map, int len) cond_resched(); } up_write(&trace_event_sem); + mutex_unlock(&event_mutex); } static bool event_in_systems(struct trace_event_call *call, diff --git a/lib/.gitignore b/lib/.gitignore index 54596b634ecb..101a4aa92fb5 100644 --- a/lib/.gitignore +++ b/lib/.gitignore @@ -5,5 +5,3 @@ /gen_crc32table /gen_crc64table /oid_registry_data.c -/test_fortify.log -/test_fortify/*.log diff --git a/lib/Makefile b/lib/Makefile index b9d2577fbbe1..62737ad5aa74 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -407,36 +407,4 @@ CFLAGS_longest_symbol_kunit.o += $(call cc-disable-warning, missing-prototypes) obj-$(CONFIG_GENERIC_LIB_DEVMEM_IS_ALLOWED) += devmem_is_allowed.o -# FORTIFY_SOURCE compile-time behavior tests -TEST_FORTIFY_SRCS = $(wildcard $(srctree)/$(src)/test_fortify/*-*.c) -TEST_FORTIFY_LOGS = $(patsubst $(srctree)/$(src)/%.c, %.log, $(TEST_FORTIFY_SRCS)) -TEST_FORTIFY_LOG = test_fortify.log - -quiet_cmd_test_fortify = TEST $@ - cmd_test_fortify = $(CONFIG_SHELL) $(srctree)/scripts/test_fortify.sh \ - $< $@ "$(NM)" $(CC) $(c_flags) \ - $(call cc-disable-warning,fortify-source) \ - -DKBUILD_EXTRA_WARN1 - -targets += $(TEST_FORTIFY_LOGS) -clean-files += $(TEST_FORTIFY_LOGS) -clean-files += $(addsuffix .o, $(TEST_FORTIFY_LOGS)) -$(obj)/test_fortify/%.log: $(src)/test_fortify/%.c \ - $(src)/test_fortify/test_fortify.h \ - $(srctree)/include/linux/fortify-string.h \ - $(srctree)/scripts/test_fortify.sh \ - FORCE - $(call if_changed,test_fortify) - -quiet_cmd_gen_fortify_log = GEN $@ - cmd_gen_fortify_log = cat /dev/null > $@ || true - -targets += $(TEST_FORTIFY_LOG) -clean-files += $(TEST_FORTIFY_LOG) -$(obj)/$(TEST_FORTIFY_LOG): $(addprefix $(obj)/, $(TEST_FORTIFY_LOGS)) FORCE - $(call if_changed,gen_fortify_log) - -# Fake dependency to trigger the fortify tests. -ifeq ($(CONFIG_FORTIFY_SOURCE),y) -$(obj)/string.o: $(obj)/$(TEST_FORTIFY_LOG) -endif +subdir-$(CONFIG_FORTIFY_SOURCE) += test_fortify diff --git a/lib/test_fortify/.gitignore b/lib/test_fortify/.gitignore new file mode 100644 index 000000000000..c1ba37d14b50 --- /dev/null +++ b/lib/test_fortify/.gitignore @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0-only +/*.log diff --git a/lib/test_fortify/Makefile b/lib/test_fortify/Makefile new file mode 100644 index 000000000000..eba2ba0faeb6 --- /dev/null +++ b/lib/test_fortify/Makefile @@ -0,0 +1,29 @@ +# SPDX-License-Identifier: GPL-2.0 + +ccflags-y := $(call cc-disable-warning,fortify-source) +ccflags-y += $(call cc-disable-warning,stringop-overread) + +quiet_cmd_test_fortify = TEST $@ + cmd_test_fortify = $(CONFIG_SHELL) $(srctree)/scripts/test_fortify.sh \ + $< $@ "$(NM)" $(CC) $(c_flags) -DKBUILD_EXTRA_WARN1 + +$(obj)/%.log: $(src)/%.c $(srctree)/scripts/test_fortify.sh \ + $(src)/test_fortify.h \ + $(srctree)/include/linux/fortify-string.h \ + FORCE + $(call if_changed,test_fortify) + +logs = $(patsubst $(srctree)/$(src)/%.c, %.log, $(wildcard $(srctree)/$(src)/*-*.c)) +targets += $(logs) + +quiet_cmd_gen_fortify_log = CAT $@ + cmd_gen_fortify_log = cat $(or $(real-prereqs),/dev/null) > $@ + +$(obj)/test_fortify.log: $(addprefix $(obj)/, $(logs)) FORCE + $(call if_changed,gen_fortify_log) + +always-y += test_fortify.log + +# Some architectures define __NO_FORTIFY if __SANITIZE_ADDRESS__ is undefined. +# Pass CFLAGS_KASAN to avoid warnings. +KASAN_SANITIZE := y diff --git a/net/atm/common.c b/net/atm/common.c index 96f680a45e30..f3f68c231935 100644 --- a/net/atm/common.c +++ b/net/atm/common.c @@ -760,7 +760,7 @@ int vcc_setsockopt(struct socket *sock, int level, int optname, sockptr_t optval, unsigned int optlen) { struct atm_vcc *vcc; - unsigned long value; + int value; int error; if (__SO_LEVEL_MATCH(optname, level) && optlen != __SO_SIZE(optname)) @@ -772,8 +772,10 @@ int vcc_setsockopt(struct socket *sock, int level, int optname, { struct atm_qos qos; - if (copy_from_sockptr(&qos, optval, sizeof(qos))) - return -EFAULT; + error = copy_safe_from_sockptr(&qos, sizeof(qos), optval, + optlen); + if (error) + return error; error = check_qos(&qos); if (error) return error; @@ -786,8 +788,10 @@ int vcc_setsockopt(struct socket *sock, int level, int optname, return 0; } case SO_SETCLP: - if (copy_from_sockptr(&value, optval, sizeof(value))) - return -EFAULT; + error = copy_safe_from_sockptr(&value, sizeof(value), optval, + optlen); + if (error) + return error; if (value) vcc->atm_options |= ATM_ATMOPT_CLP; else diff --git a/net/bridge/br_mrp.c b/net/bridge/br_mrp.c index 5fd22bb4f5b6..a837ff7d6f36 100644 --- a/net/bridge/br_mrp.c +++ b/net/bridge/br_mrp.c @@ -224,11 +224,9 @@ static struct sk_buff *br_mrp_alloc_test_skb(struct br_mrp *mrp, sub_opt = skb_put(skb, sizeof(*sub_opt)); memset(sub_opt, 0x0, sizeof(*sub_opt)); - sub_tlv = skb_put(skb, sizeof(*sub_tlv)); - sub_tlv->type = BR_MRP_SUB_TLV_HEADER_TEST_AUTO_MGR; - /* 32 bit alligment shall be ensured therefore add 2 bytes */ - skb_put(skb, MRP_OPT_PADDING); + sub_tlv = skb_put_zero(skb, sizeof(*sub_tlv) + MRP_OPT_PADDING); + sub_tlv->type = BR_MRP_SUB_TLV_HEADER_TEST_AUTO_MGR; } br_mrp_skb_tlv(skb, BR_MRP_TLV_HEADER_END, 0x0); diff --git a/net/bridge/netfilter/ebt_nflog.c b/net/bridge/netfilter/ebt_nflog.c index 61bf8f4465ab..426f8adc912c 100644 --- a/net/bridge/netfilter/ebt_nflog.c +++ b/net/bridge/netfilter/ebt_nflog.c @@ -41,11 +41,25 @@ ebt_nflog_tg(struct sk_buff *skb, const struct xt_action_param *par) static int ebt_nflog_tg_check(const struct xt_tgchk_param *par) { struct ebt_nflog_info *info = par->targinfo; + int ret; if (info->flags & ~EBT_NFLOG_MASK) return -EINVAL; info->prefix[EBT_NFLOG_PREFIX_SIZE - 1] = '\0'; - return 0; + + ret = nf_logger_find_get(par->family, NF_LOG_TYPE_ULOG); + if (ret != 0 && !par->nft_compat) { + request_module("%s", "nfnetlink_log"); + + ret = nf_logger_find_get(par->family, NF_LOG_TYPE_ULOG); + } + + return ret; +} + +static void ebt_nflog_tg_destroy(const struct xt_tgdtor_param *par) +{ + nf_logger_put(par->family, NF_LOG_TYPE_ULOG); } static struct xt_target ebt_nflog_tg_reg __read_mostly = { @@ -54,6 +68,7 @@ static struct xt_target ebt_nflog_tg_reg __read_mostly = { .family = NFPROTO_BRIDGE, .target = ebt_nflog_tg, .checkentry = ebt_nflog_tg_check, + .destroy = ebt_nflog_tg_destroy, .targetsize = sizeof(struct ebt_nflog_info), .me = THIS_MODULE, }; diff --git a/net/bridge/netfilter/nf_conntrack_bridge.c b/net/bridge/netfilter/nf_conntrack_bridge.c index 4b4a396d9722..8834a45f697b 100644 --- a/net/bridge/netfilter/nf_conntrack_bridge.c +++ b/net/bridge/netfilter/nf_conntrack_bridge.c @@ -281,6 +281,7 @@ static unsigned int nf_ct_bridge_pre(void *priv, struct sk_buff *skb, ret = nf_ct_br_defrag6(skb, &bridge_state); break; default: + nf_reset_ct(skb); nf_ct_set(skb, NULL, IP_CT_UNTRACKED); return NF_ACCEPT; } diff --git a/net/core/dev.c b/net/core/dev.c index 0cc03e6c6fb1..0fa5431de2cc 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2146,6 +2146,11 @@ void net_dec_egress_queue(void) EXPORT_SYMBOL_GPL(net_dec_egress_queue); #endif +#ifdef CONFIG_NET_CLS_ACT +DEFINE_STATIC_KEY_FALSE(tcf_sw_enabled_key); +EXPORT_SYMBOL(tcf_sw_enabled_key); +#endif + DEFINE_STATIC_KEY_FALSE(netstamp_needed_key); EXPORT_SYMBOL(netstamp_needed_key); #ifdef CONFIG_JUMP_LABEL @@ -4030,6 +4035,14 @@ static int tc_run(struct tcx_entry *entry, struct sk_buff *skb) if (!miniq) return ret; + /* Global bypass */ + if (!static_branch_likely(&tcf_sw_enabled_key)) + return ret; + + /* Block-wise bypass */ + if (tcf_block_bypass_sw(miniq->block)) + return ret; + tc_skb_cb(skb)->mru = 0; tc_skb_cb(skb)->post_ct = false; diff --git a/net/core/sock.c b/net/core/sock.c index 87e6060c8bca..846e95805c19 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -776,7 +776,6 @@ bool sk_mc_loop(struct sock *sk) return inet6_sk(sk)->mc_loop; #endif } - WARN_ON_ONCE(1); return true; } EXPORT_SYMBOL(sk_mc_loop); diff --git a/net/core/xdp.c b/net/core/xdp.c index 5ee3f8f165e5..f640576090f2 100644 --- a/net/core/xdp.c +++ b/net/core/xdp.c @@ -674,7 +674,7 @@ struct xdp_frame *xdpf_clone(struct xdp_frame *xdpf) headroom = xdpf->headroom + sizeof(*xdpf); totalsize = headroom + xdpf->len; - if (unlikely(totalsize > PAGE_SIZE)) + if (unlikely(totalsize > SKB_WITH_OVERHEAD(PAGE_SIZE))) return NULL; page = dev_alloc_page(); if (!page) diff --git a/net/devlink/dev.c b/net/devlink/dev.c index bba4ace7d22b..c70d4dc599b8 100644 --- a/net/devlink/dev.c +++ b/net/devlink/dev.c @@ -505,6 +505,7 @@ int devlink_nl_cmd_reload(struct sk_buff *skb, struct genl_info *info) action != DEVLINK_RELOAD_ACTION_DRIVER_REINIT) { NL_SET_ERR_MSG_MOD(info->extack, "Changing namespace is only supported for reinit action"); + put_net(dest_net); return -EOPNOTSUPP; } } diff --git a/net/ipv4/Kconfig b/net/ipv4/Kconfig index 2dfb12230f08..8e94ed7c56a0 100644 --- a/net/ipv4/Kconfig +++ b/net/ipv4/Kconfig @@ -741,10 +741,27 @@ config DEFAULT_TCP_CONG default "bbr" if DEFAULT_BBR default "cubic" +config TCP_SIGPOOL + tristate + +config TCP_AO + bool "TCP: Authentication Option (RFC5925)" + select CRYPTO + select TCP_SIGPOOL + depends on 64BIT && IPV6 != m # seq-number extension needs WRITE_ONCE(u64) + help + TCP-AO specifies the use of stronger Message Authentication Codes (MACs), + protects against replays for long-lived TCP connections, and + provides more details on the association of security with TCP + connections than TCP MD5 (See RFC5925) + + If unsure, say N. + config TCP_MD5SIG bool "TCP: MD5 Signature Option support (RFC2385)" select CRYPTO select CRYPTO_MD5 + select TCP_SIGPOOL help RFC2385 specifies a method of giving MD5 protection to TCP sessions. Its main (only?) use is to protect BGP sessions between core routers diff --git a/net/ipv4/Makefile b/net/ipv4/Makefile index b18ba8ef93ad..cd760793cfcb 100644 --- a/net/ipv4/Makefile +++ b/net/ipv4/Makefile @@ -62,6 +62,7 @@ obj-$(CONFIG_TCP_CONG_SCALABLE) += tcp_scalable.o obj-$(CONFIG_TCP_CONG_LP) += tcp_lp.o obj-$(CONFIG_TCP_CONG_YEAH) += tcp_yeah.o obj-$(CONFIG_TCP_CONG_ILLINOIS) += tcp_illinois.o +obj-$(CONFIG_TCP_SIGPOOL) += tcp_sigpool.o obj-$(CONFIG_NET_SOCK_MSG) += tcp_bpf.o obj-$(CONFIG_BPF_SYSCALL) += udp_bpf.o obj-$(CONFIG_NETLABEL) += cipso_ipv4.o diff --git a/net/ipv4/fib_semantics.c b/net/ipv4/fib_semantics.c index 233d9d0437c2..facbda1ff51d 100644 --- a/net/ipv4/fib_semantics.c +++ b/net/ipv4/fib_semantics.c @@ -462,6 +462,34 @@ int ip_fib_check_default(__be32 gw, struct net_device *dev) return -1; } +static size_t fib_nexthop_nlmsg_size(const struct fib_nh_common *nhc, + bool skip_oif) +{ + size_t nhsize = 0; + + switch (nhc->nhc_gw_family) { + case AF_INET: + nhsize += nla_total_size(4); /* RTA_GATEWAY */ + break; + case AF_INET6: + nhsize += nla_total_size(sizeof(struct rtvia) + + sizeof(struct in6_addr)); + break; + } + + if (!skip_oif && nhc->nhc_dev) + nhsize += nla_total_size(4); /* RTA_OIF */ + + if (nhc->nhc_lwtstate) { + /* RTA_ENCAP */ + nhsize += lwtunnel_get_encap_size(nhc->nhc_lwtstate); + /* RTA_ENCAP_TYPE */ + nhsize += nla_total_size(2); + } + + return nhsize; +} + size_t fib_nlmsg_size(struct fib_info *fi) { size_t payload = NLMSG_ALIGN(sizeof(struct rtmsg)) @@ -479,32 +507,35 @@ size_t fib_nlmsg_size(struct fib_info *fi) payload += nla_total_size(4); /* RTA_NH_ID */ if (nhs) { - size_t nh_encapsize = 0; - /* Also handles the special case nhs == 1 */ - - /* each nexthop is packed in an attribute */ - size_t nhsize = nla_total_size(sizeof(struct rtnexthop)); + size_t mpsize = 0; unsigned int i; - /* may contain flow and gateway attribute */ - nhsize += 2 * nla_total_size(4); - - /* grab encap info */ for (i = 0; i < fib_info_num_path(fi); i++) { struct fib_nh_common *nhc = fib_info_nhc(fi, i); + size_t nhsize; + + nhsize = fib_nexthop_nlmsg_size(nhc, nhs != 1); + + if (nhs != 1) + nhsize += NLA_ALIGN(sizeof(struct rtnexthop)); + +#ifdef CONFIG_IP_ROUTE_CLASSID + if (nhc->nhc_family == AF_INET) { + struct fib_nh *nh; - if (nhc->nhc_lwtstate) { - /* RTA_ENCAP_TYPE */ - nh_encapsize += lwtunnel_get_encap_size( - nhc->nhc_lwtstate); - /* RTA_ENCAP */ - nh_encapsize += nla_total_size(2); + nh = container_of(nhc, struct fib_nh, nh_common); + if (nh->nh_tclassid) + nhsize += nla_total_size(4); } +#endif + if (nhs == 1) + payload += nhsize; + else + mpsize += nhsize; } - /* all nexthops are packed in a nested attribute */ - payload += nla_total_size((nhs * nhsize) + nh_encapsize); - + if (nhs != 1) + payload += nla_total_size(mpsize); } return payload; @@ -1926,42 +1957,30 @@ static int call_fib_nh_notifiers(struct fib_nh *nh, return NOTIFY_DONE; } -/* Update the PMTU of exceptions when: - * - the new MTU of the first hop becomes smaller than the PMTU - * - the old MTU was the same as the PMTU, and it limited discovery of - * larger MTUs on the path. With that limit raised, we can now - * discover larger MTUs - * A special case is locked exceptions, for which the PMTU is smaller - * than the minimal accepted PMTU: - * - if the new MTU is greater than the PMTU, don't make any change - * - otherwise, unlock and set PMTU +/* Walk the exceptions of a nexthop after its first hop MTU changed. The + * chain is RCU protected here, while fnhe_update_pmtu() takes fnhe_lock + * for the update of each entry. */ void fib_nhc_update_mtu(struct fib_nh_common *nhc, u32 new, u32 orig) { struct fnhe_hash_bucket *bucket; int i; - bucket = rcu_dereference_protected(nhc->nhc_exceptions, 1); + rcu_read_lock(); + bucket = rcu_dereference(nhc->nhc_exceptions); if (!bucket) - return; + goto out; for (i = 0; i < FNHE_HASH_SIZE; i++) { struct fib_nh_exception *fnhe; - for (fnhe = rcu_dereference_protected(bucket[i].chain, 1); + for (fnhe = rcu_dereference(bucket[i].chain); fnhe; - fnhe = rcu_dereference_protected(fnhe->fnhe_next, 1)) { - if (fnhe->fnhe_mtu_locked) { - if (new <= fnhe->fnhe_pmtu) { - fnhe->fnhe_pmtu = new; - fnhe->fnhe_mtu_locked = false; - } - } else if (new < fnhe->fnhe_pmtu || - orig == fnhe->fnhe_pmtu) { - fnhe->fnhe_pmtu = new; - } - } + fnhe = rcu_dereference(fnhe->fnhe_next)) + fnhe_update_pmtu(fnhe, new, orig); } +out: + rcu_read_unlock(); } void fib_sync_mtu(struct net_device *dev, u32 orig_mtu) diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c index c7a1f763e464..208f8b173a05 100644 --- a/net/ipv4/inet_connection_sock.c +++ b/net/ipv4/inet_connection_sock.c @@ -930,11 +930,23 @@ static struct request_sock *inet_reqsk_clone(struct request_sock *req, nreq->rsk_listener = sk; - /* We need not acquire fastopenq->lock - * because the child socket is locked in inet_csk_listen_stop(). - */ - if (sk->sk_protocol == IPPROTO_TCP && tcp_rsk(nreq)->tfo_listener) + if (sk->sk_protocol == IPPROTO_TCP && tcp_rsk(nreq)->tfo_listener) { + struct fastopen_queue *fastopenq; + + /* reqsk_fastopen_remove() will uncharge nreq->rsk_listener, + * that is @sk, so charge it here. Unlike the listener + * being closed, @sk is live and needs its lock. + */ + fastopenq = &inet_csk(sk)->icsk_accept_queue.fastopenq; + spin_lock_bh(&fastopenq->lock); + fastopenq->qlen++; + spin_unlock_bh(&fastopenq->lock); + + /* We need not acquire fastopenq->lock + * because the child socket is locked in inet_csk_listen_stop(). + */ rcu_assign_pointer(tcp_sk(nreq->sk)->fastopen_rsk, nreq); + } return nreq; } diff --git a/net/ipv4/route.c b/net/ipv4/route.c index 87eaae03e618..789953096c80 100644 --- a/net/ipv4/route.c +++ b/net/ipv4/route.c @@ -751,6 +751,35 @@ static void update_or_create_fnhe(struct fib_nh_common *nhc, __be32 daddr, spin_unlock_bh(&fnhe_lock); } +/* Update the PMTU of an exception when: + * - the new MTU of the first hop becomes smaller than the PMTU + * - the old MTU was the same as the PMTU, and it limited discovery of + * larger MTUs on the path. With that limit raised, we can now + * discover larger MTUs + * A special case is locked exceptions, for which the PMTU is smaller + * than the minimal accepted PMTU: + * - if the new MTU is greater than the PMTU, don't make any change + * - otherwise, unlock and set PMTU + * + * fnhe_lock keeps fnhe_pmtu and fnhe_mtu_locked consistent against + * update_or_create_fnhe(), which sets both under the same lock. + */ +void fnhe_update_pmtu(struct fib_nh_exception *fnhe, u32 new, u32 orig) +{ + spin_lock_bh(&fnhe_lock); + + if (fnhe->fnhe_mtu_locked) { + if (new <= fnhe->fnhe_pmtu) { + fnhe->fnhe_pmtu = new; + fnhe->fnhe_mtu_locked = false; + } + } else if (new < fnhe->fnhe_pmtu || orig == fnhe->fnhe_pmtu) { + fnhe->fnhe_pmtu = new; + } + + spin_unlock_bh(&fnhe_lock); +} + static void __ip_do_redirect(struct rtable *rt, struct sk_buff *skb, struct flowi4 *fl4, bool kill_route) { diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 5b1fbb0ca2ff..f05d1699683f 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -4365,141 +4365,52 @@ int tcp_getsockopt(struct sock *sk, int level, int optname, char __user *optval, EXPORT_SYMBOL(tcp_getsockopt); #ifdef CONFIG_TCP_MD5SIG -static DEFINE_PER_CPU(struct tcp_md5sig_pool, tcp_md5sig_pool); -static DEFINE_MUTEX(tcp_md5sig_mutex); -static bool tcp_md5sig_pool_populated = false; +int tcp_md5_sigpool_id = -1; +EXPORT_SYMBOL_GPL(tcp_md5_sigpool_id); -static void __tcp_alloc_md5sig_pool(void) +int tcp_md5_alloc_sigpool(void) { - struct crypto_ahash *hash; - int cpu; - - hash = crypto_alloc_ahash("md5", 0, CRYPTO_ALG_ASYNC); - if (IS_ERR(hash)) - return; - - for_each_possible_cpu(cpu) { - void *scratch = per_cpu(tcp_md5sig_pool, cpu).scratch; - struct ahash_request *req; - - if (!scratch) { - scratch = kmalloc_node(sizeof(union tcp_md5sum_block) + - sizeof(struct tcphdr), - GFP_KERNEL, - cpu_to_node(cpu)); - if (!scratch) - return; - per_cpu(tcp_md5sig_pool, cpu).scratch = scratch; - } - if (per_cpu(tcp_md5sig_pool, cpu).md5_req) - continue; - - req = ahash_request_alloc(hash, GFP_KERNEL); - if (!req) - return; - - ahash_request_set_callback(req, 0, NULL, NULL); - - per_cpu(tcp_md5sig_pool, cpu).md5_req = req; - } - /* before setting tcp_md5sig_pool_populated, we must commit all writes - * to memory. See smp_rmb() in tcp_get_md5sig_pool() - */ - smp_wmb(); - /* Paired with READ_ONCE() from tcp_alloc_md5sig_pool() - * and tcp_get_md5sig_pool(). - */ - WRITE_ONCE(tcp_md5sig_pool_populated, true); -} - -bool tcp_alloc_md5sig_pool(void) -{ - /* Paired with WRITE_ONCE() from __tcp_alloc_md5sig_pool() */ - if (unlikely(!READ_ONCE(tcp_md5sig_pool_populated))) { - mutex_lock(&tcp_md5sig_mutex); - - if (!tcp_md5sig_pool_populated) - __tcp_alloc_md5sig_pool(); + size_t scratch_size; + int ret; - mutex_unlock(&tcp_md5sig_mutex); + scratch_size = sizeof(union tcp_md5sum_block) + sizeof(struct tcphdr); + ret = tcp_sigpool_alloc_ahash("md5", scratch_size); + if (ret >= 0) { + /* As long as any md5 sigpool was allocated, the return + * id would stay the same. Re-write the id only for the case + * when previously all MD5 keys were deleted and this call + * allocates the first MD5 key, which may return a different + * sigpool id than was used previously. + */ + WRITE_ONCE(tcp_md5_sigpool_id, ret); /* Avoids the compiler potentially being smart here */ + return 0; } - /* Paired with WRITE_ONCE() from __tcp_alloc_md5sig_pool() */ - return READ_ONCE(tcp_md5sig_pool_populated); + return ret; } -EXPORT_SYMBOL(tcp_alloc_md5sig_pool); - -/** - * tcp_get_md5sig_pool - get md5sig_pool for this user - * - * We use percpu structure, so if we succeed, we exit with preemption - * and BH disabled, to make sure another thread or softirq handling - * wont try to get same context. - */ -struct tcp_md5sig_pool *tcp_get_md5sig_pool(void) +void tcp_md5_release_sigpool(void) { - local_bh_disable(); - - /* Paired with WRITE_ONCE() from __tcp_alloc_md5sig_pool() */ - if (READ_ONCE(tcp_md5sig_pool_populated)) { - /* coupled with smp_wmb() in __tcp_alloc_md5sig_pool() */ - smp_rmb(); - return this_cpu_ptr(&tcp_md5sig_pool); - } - local_bh_enable(); - return NULL; + tcp_sigpool_release(READ_ONCE(tcp_md5_sigpool_id)); } -EXPORT_SYMBOL(tcp_get_md5sig_pool); -int tcp_md5_hash_skb_data(struct tcp_md5sig_pool *hp, - const struct sk_buff *skb, unsigned int header_len) +void tcp_md5_add_sigpool(void) { - struct scatterlist sg; - const struct tcphdr *tp = tcp_hdr(skb); - struct ahash_request *req = hp->md5_req; - unsigned int i; - const unsigned int head_data_len = skb_headlen(skb) > header_len ? - skb_headlen(skb) - header_len : 0; - const struct skb_shared_info *shi = skb_shinfo(skb); - struct sk_buff *frag_iter; - - sg_init_table(&sg, 1); - - sg_set_buf(&sg, ((u8 *) tp) + header_len, head_data_len); - ahash_request_set_crypt(req, &sg, NULL, head_data_len); - if (crypto_ahash_update(req)) - return 1; - - for (i = 0; i < shi->nr_frags; ++i) { - const skb_frag_t *f = &shi->frags[i]; - unsigned int offset = skb_frag_off(f); - struct page *page = skb_frag_page(f) + (offset >> PAGE_SHIFT); - - sg_set_page(&sg, page, skb_frag_size(f), - offset_in_page(offset)); - ahash_request_set_crypt(req, &sg, NULL, skb_frag_size(f)); - if (crypto_ahash_update(req)) - return 1; - } - - skb_walk_frags(skb, frag_iter) - if (tcp_md5_hash_skb_data(hp, frag_iter, 0)) - return 1; - - return 0; + tcp_sigpool_get(READ_ONCE(tcp_md5_sigpool_id)); } -EXPORT_SYMBOL(tcp_md5_hash_skb_data); -int tcp_md5_hash_key(struct tcp_md5sig_pool *hp, const struct tcp_md5sig_key *key) +int tcp_md5_hash_key(struct tcp_sigpool *hp, + const struct tcp_md5sig_key *key) { u8 keylen = READ_ONCE(key->keylen); /* paired with WRITE_ONCE() in tcp_md5_do_add */ struct scatterlist sg; sg_init_one(&sg, key->key, keylen); - ahash_request_set_crypt(hp->md5_req, &sg, NULL, keylen); + ahash_request_set_crypt(hp->req, &sg, NULL, keylen); - /* We use data_race() because tcp_md5_do_add() might change key->key under us */ - return data_race(crypto_ahash_update(hp->md5_req)); + /* We use data_race() because tcp_md5_do_add() might change + * key->key under us + */ + return data_race(crypto_ahash_update(hp->req)); } EXPORT_SYMBOL(tcp_md5_hash_key); diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c index 337f6edd4757..a4a4434171a0 100644 --- a/net/ipv4/tcp_bpf.c +++ b/net/ipv4/tcp_bpf.c @@ -455,6 +455,7 @@ static int tcp_bpf_send_verdict(struct sock *sk, struct sk_psock *psock, case __SK_REDIRECT: redir_ingress = psock->redir_ingress; sk_redir = psock->sk_redir; + sock_hold(sk_redir); sk_msg_apply_bytes(psock, tosend); if (!psock->apply_bytes) { /* Clean up before releasing the sock lock. */ @@ -475,6 +476,7 @@ static int tcp_bpf_send_verdict(struct sock *sk, struct sk_psock *psock, if (eval == __SK_REDIRECT) sock_put(sk_redir); + sock_put(sk_redir); lock_sock(sk); sk_mem_uncharge(sk, sent); diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index eb1bf5863474..9efb84658a82 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -252,7 +252,7 @@ static void tcp_measure_rcv_mss(struct sock *sk, const struct sk_buff *skb) struct tcp_sock *tp = tcp_sk(sk); val = tcp_win_from_space(sk, sk->sk_rcvbuf); - tcp_set_window_clamp(sk, val); + WRITE_ONCE(tp->window_clamp, val); if (tp->window_clamp < tp->rcvq_space.space) tp->rcvq_space.space = tp->window_clamp; diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index c8d35f1c0ece..3f9e1cfde008 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -58,6 +58,7 @@ #include #include #include +#include #include #include @@ -1226,10 +1227,6 @@ static int __tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr, key = sock_kmalloc(sk, sizeof(*key), gfp | __GFP_ZERO); if (!key) return -ENOMEM; - if (!tcp_alloc_md5sig_pool()) { - sock_kfree_s(sk, key, sizeof(*key)); - return -ENOMEM; - } memcpy(key->key, newkey, newkeylen); key->keylen = newkeylen; @@ -1251,15 +1248,21 @@ int tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr, struct tcp_sock *tp = tcp_sk(sk); if (!rcu_dereference_protected(tp->md5sig_info, lockdep_sock_is_held(sk))) { - if (tcp_md5sig_info_add(sk, GFP_KERNEL)) + if (tcp_md5_alloc_sigpool()) return -ENOMEM; + if (tcp_md5sig_info_add(sk, GFP_KERNEL)) { + tcp_md5_release_sigpool(); + return -ENOMEM; + } + if (!static_branch_inc(&tcp_md5_needed.key)) { struct tcp_md5sig_info *md5sig; md5sig = rcu_dereference_protected(tp->md5sig_info, lockdep_sock_is_held(sk)); rcu_assign_pointer(tp->md5sig_info, NULL); kfree_rcu(md5sig, rcu); + tcp_md5_release_sigpool(); return -EUSERS; } } @@ -1276,8 +1279,12 @@ int tcp_md5_key_copy(struct sock *sk, const union tcp_md5_addr *addr, struct tcp_sock *tp = tcp_sk(sk); if (!rcu_dereference_protected(tp->md5sig_info, lockdep_sock_is_held(sk))) { - if (tcp_md5sig_info_add(sk, sk_gfp_mask(sk, GFP_ATOMIC))) + tcp_md5_add_sigpool(); + + if (tcp_md5sig_info_add(sk, sk_gfp_mask(sk, GFP_ATOMIC))) { + tcp_md5_release_sigpool(); return -ENOMEM; + } if (!static_key_fast_inc_not_disabled(&tcp_md5_needed.key.key)) { struct tcp_md5sig_info *md5sig; @@ -1286,6 +1293,7 @@ int tcp_md5_key_copy(struct sock *sk, const union tcp_md5_addr *addr, net_warn_ratelimited("Too many TCP-MD5 keys in the system\n"); rcu_assign_pointer(tp->md5sig_info, NULL); kfree_rcu(md5sig, rcu); + tcp_md5_release_sigpool(); return -EUSERS; } } @@ -1385,7 +1393,7 @@ static int tcp_v4_parse_md5_keys(struct sock *sk, int optname, cmd.tcpm_key, cmd.tcpm_keylen); } -static int tcp_v4_md5_hash_headers(struct tcp_md5sig_pool *hp, +static int tcp_v4_md5_hash_headers(struct tcp_sigpool *hp, __be32 daddr, __be32 saddr, const struct tcphdr *th, int nbytes) { @@ -1405,38 +1413,35 @@ static int tcp_v4_md5_hash_headers(struct tcp_md5sig_pool *hp, _th->check = 0; sg_init_one(&sg, bp, sizeof(*bp) + sizeof(*th)); - ahash_request_set_crypt(hp->md5_req, &sg, NULL, + ahash_request_set_crypt(hp->req, &sg, NULL, sizeof(*bp) + sizeof(*th)); - return crypto_ahash_update(hp->md5_req); + return crypto_ahash_update(hp->req); } static int tcp_v4_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key, __be32 daddr, __be32 saddr, const struct tcphdr *th) { - struct tcp_md5sig_pool *hp; - struct ahash_request *req; + struct tcp_sigpool hp; - hp = tcp_get_md5sig_pool(); - if (!hp) - goto clear_hash_noput; - req = hp->md5_req; + if (tcp_sigpool_start(tcp_md5_sigpool_id, &hp)) + goto clear_hash_nostart; - if (crypto_ahash_init(req)) + if (crypto_ahash_init(hp.req)) goto clear_hash; - if (tcp_v4_md5_hash_headers(hp, daddr, saddr, th, th->doff << 2)) + if (tcp_v4_md5_hash_headers(&hp, daddr, saddr, th, th->doff << 2)) goto clear_hash; - if (tcp_md5_hash_key(hp, key)) + if (tcp_md5_hash_key(&hp, key)) goto clear_hash; - ahash_request_set_crypt(req, NULL, md5_hash, 0); - if (crypto_ahash_final(req)) + ahash_request_set_crypt(hp.req, NULL, md5_hash, 0); + if (crypto_ahash_final(hp.req)) goto clear_hash; - tcp_put_md5sig_pool(); + tcp_sigpool_end(&hp); return 0; clear_hash: - tcp_put_md5sig_pool(); -clear_hash_noput: + tcp_sigpool_end(&hp); +clear_hash_nostart: memset(md5_hash, 0, 16); return 1; } @@ -1445,9 +1450,8 @@ int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key, const struct sock *sk, const struct sk_buff *skb) { - struct tcp_md5sig_pool *hp; - struct ahash_request *req; const struct tcphdr *th = tcp_hdr(skb); + struct tcp_sigpool hp; __be32 saddr, daddr; if (sk) { /* valid for establish/request sockets */ @@ -1459,30 +1463,28 @@ int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key, daddr = iph->daddr; } - hp = tcp_get_md5sig_pool(); - if (!hp) - goto clear_hash_noput; - req = hp->md5_req; + if (tcp_sigpool_start(tcp_md5_sigpool_id, &hp)) + goto clear_hash_nostart; - if (crypto_ahash_init(req)) + if (crypto_ahash_init(hp.req)) goto clear_hash; - if (tcp_v4_md5_hash_headers(hp, daddr, saddr, th, skb->len)) + if (tcp_v4_md5_hash_headers(&hp, daddr, saddr, th, skb->len)) goto clear_hash; - if (tcp_md5_hash_skb_data(hp, skb, th->doff << 2)) + if (tcp_sigpool_hash_skb_data(&hp, skb, th->doff << 2)) goto clear_hash; - if (tcp_md5_hash_key(hp, key)) + if (tcp_md5_hash_key(&hp, key)) goto clear_hash; - ahash_request_set_crypt(req, NULL, md5_hash, 0); - if (crypto_ahash_final(req)) + ahash_request_set_crypt(hp.req, NULL, md5_hash, 0); + if (crypto_ahash_final(hp.req)) goto clear_hash; - tcp_put_md5sig_pool(); + tcp_sigpool_end(&hp); return 0; clear_hash: - tcp_put_md5sig_pool(); -clear_hash_noput: + tcp_sigpool_end(&hp); +clear_hash_nostart: memset(md5_hash, 0, 16); return 1; } @@ -2310,6 +2312,18 @@ static int tcp_v4_init_sock(struct sock *sk) return 0; } +#ifdef CONFIG_TCP_MD5SIG +static void tcp_md5sig_info_free_rcu(struct rcu_head *head) +{ + struct tcp_md5sig_info *md5sig; + + md5sig = container_of(head, struct tcp_md5sig_info, rcu); + kfree(md5sig); + static_branch_slow_dec_deferred(&tcp_md5_needed); + tcp_md5_release_sigpool(); +} +#endif + void tcp_v4_destroy_sock(struct sock *sk) { struct tcp_sock *tp = tcp_sk(sk); @@ -2334,10 +2348,12 @@ void tcp_v4_destroy_sock(struct sock *sk) #ifdef CONFIG_TCP_MD5SIG /* Clean up the MD5 key list, if any */ if (tp->md5sig_info) { + struct tcp_md5sig_info *md5sig; + + md5sig = rcu_dereference_protected(tp->md5sig_info, 1); tcp_clear_md5_list(sk); - kfree_rcu(rcu_dereference_protected(tp->md5sig_info, 1), rcu); - tp->md5sig_info = NULL; - static_branch_slow_dec_deferred(&tcp_md5_needed); + call_rcu(&md5sig->rcu, tcp_md5sig_info_free_rcu); + rcu_assign_pointer(tp->md5sig_info, NULL); } #endif @@ -2776,13 +2792,17 @@ static int tcp4_seq_show(struct seq_file *seq, void *v) } #ifdef CONFIG_BPF_SYSCALL +union bpf_tcp_iter_batch_item { + struct sock *sk; + __u64 cookie; +}; + struct bpf_tcp_iter_state { struct tcp_iter_state state; unsigned int cur_sk; unsigned int end_sk; unsigned int max_sk; - struct sock **batch; - bool st_bucket_done; + union bpf_tcp_iter_batch_item *batch; }; struct bpf_iter__tcp { @@ -2805,21 +2825,32 @@ static int tcp_prog_seq_show(struct bpf_prog *prog, struct bpf_iter_meta *meta, static void bpf_iter_tcp_put_batch(struct bpf_tcp_iter_state *iter) { - while (iter->cur_sk < iter->end_sk) - sock_gen_put(iter->batch[iter->cur_sk++]); + union bpf_tcp_iter_batch_item *item; + unsigned int cur_sk = iter->cur_sk; + __u64 cookie; + + /* Remember the cookies of the sockets we haven't seen yet, so we can + * pick up where we left off next time around. + */ + while (cur_sk < iter->end_sk) { + item = &iter->batch[cur_sk++]; + cookie = sock_gen_cookie(item->sk); + sock_gen_put(item->sk); + item->cookie = cookie; + } } static int bpf_iter_tcp_realloc_batch(struct bpf_tcp_iter_state *iter, - unsigned int new_batch_sz) + unsigned int new_batch_sz, gfp_t flags) { - struct sock **new_batch; + union bpf_tcp_iter_batch_item *new_batch; new_batch = kvmalloc(sizeof(*new_batch) * new_batch_sz, - GFP_USER | __GFP_NOWARN); + flags | __GFP_NOWARN); if (!new_batch) return -ENOMEM; - bpf_iter_tcp_put_batch(iter); + memcpy(new_batch, iter->batch, sizeof(*iter->batch) * iter->end_sk); kvfree(iter->batch); iter->batch = new_batch; iter->max_sk = new_batch_sz; @@ -2827,112 +2858,242 @@ static int bpf_iter_tcp_realloc_batch(struct bpf_tcp_iter_state *iter, return 0; } -static unsigned int bpf_iter_tcp_listening_batch(struct seq_file *seq, - struct sock *start_sk) +static struct sock *bpf_iter_tcp_resume_bucket(struct sock *first_sk, + union bpf_tcp_iter_batch_item *cookies, + int n_cookies) +{ + struct hlist_nulls_node *node; + struct sock *sk; + int i; + + for (i = 0; i < n_cookies; i++) { + sk = first_sk; + sk_nulls_for_each_from(sk, node) + if (cookies[i].cookie == atomic64_read(&sk->sk_cookie)) + return sk; + } + + return NULL; +} + +static struct sock *bpf_iter_tcp_resume_listening(struct seq_file *seq) { struct inet_hashinfo *hinfo = seq_file_net(seq)->ipv4.tcp_death_row.hashinfo; struct bpf_tcp_iter_state *iter = seq->private; struct tcp_iter_state *st = &iter->state; - struct hlist_nulls_node *node; - unsigned int expected = 1; + unsigned int find_cookie = iter->cur_sk; + unsigned int end_cookie = iter->end_sk; + int resume_bucket = st->bucket; struct sock *sk; - sock_hold(start_sk); - iter->batch[iter->end_sk++] = start_sk; + if (end_cookie && find_cookie == end_cookie) + ++st->bucket; - sk = sk_nulls_next(start_sk); - sk_nulls_for_each_from(sk, node) { - if (seq_sk_match(seq, sk)) { - if (iter->end_sk < iter->max_sk) { - sock_hold(sk); - iter->batch[iter->end_sk++] = sk; - } - expected++; + sk = listening_get_first(seq); + iter->cur_sk = 0; + iter->end_sk = 0; + + if (sk && st->bucket == resume_bucket && end_cookie) { + sk = bpf_iter_tcp_resume_bucket(sk, &iter->batch[find_cookie], + end_cookie - find_cookie); + if (!sk) { + spin_unlock(&hinfo->lhash2[st->bucket].lock); + ++st->bucket; + sk = listening_get_first(seq); } } - spin_unlock(&hinfo->lhash2[st->bucket].lock); - return expected; + return sk; } -static unsigned int bpf_iter_tcp_established_batch(struct seq_file *seq, - struct sock *start_sk) +static struct sock *bpf_iter_tcp_resume_established(struct seq_file *seq) { struct inet_hashinfo *hinfo = seq_file_net(seq)->ipv4.tcp_death_row.hashinfo; struct bpf_tcp_iter_state *iter = seq->private; struct tcp_iter_state *st = &iter->state; + unsigned int find_cookie = iter->cur_sk; + unsigned int end_cookie = iter->end_sk; + int resume_bucket = st->bucket; + struct sock *sk; + + if (end_cookie && find_cookie == end_cookie) + ++st->bucket; + + sk = established_get_first(seq); + iter->cur_sk = 0; + iter->end_sk = 0; + + if (sk && st->bucket == resume_bucket && end_cookie) { + sk = bpf_iter_tcp_resume_bucket(sk, &iter->batch[find_cookie], + end_cookie - find_cookie); + if (!sk) { + spin_unlock_bh(inet_ehash_lockp(hinfo, st->bucket)); + ++st->bucket; + sk = established_get_first(seq); + } + } + + return sk; +} + +static struct sock *bpf_iter_tcp_resume(struct seq_file *seq) +{ + struct bpf_tcp_iter_state *iter = seq->private; + struct tcp_iter_state *st = &iter->state; + struct sock *sk = NULL; + + switch (st->state) { + case TCP_SEQ_STATE_LISTENING: + sk = bpf_iter_tcp_resume_listening(seq); + if (sk) + break; + st->bucket = 0; + st->state = TCP_SEQ_STATE_ESTABLISHED; + fallthrough; + case TCP_SEQ_STATE_ESTABLISHED: + sk = bpf_iter_tcp_resume_established(seq); + break; + } + + return sk; +} + +static unsigned int bpf_iter_tcp_listening_batch(struct seq_file *seq, + struct sock **start_sk) +{ + struct bpf_tcp_iter_state *iter = seq->private; struct hlist_nulls_node *node; unsigned int expected = 1; struct sock *sk; - sock_hold(start_sk); - iter->batch[iter->end_sk++] = start_sk; + sock_hold(*start_sk); + iter->batch[iter->end_sk++].sk = *start_sk; - sk = sk_nulls_next(start_sk); + sk = sk_nulls_next(*start_sk); + *start_sk = NULL; sk_nulls_for_each_from(sk, node) { if (seq_sk_match(seq, sk)) { if (iter->end_sk < iter->max_sk) { sock_hold(sk); - iter->batch[iter->end_sk++] = sk; + iter->batch[iter->end_sk++].sk = sk; + } else if (!*start_sk) { + /* Remember where we left off. */ + *start_sk = sk; } expected++; } } - spin_unlock_bh(inet_ehash_lockp(hinfo, st->bucket)); return expected; } -static struct sock *bpf_iter_tcp_batch(struct seq_file *seq) +static unsigned int bpf_iter_tcp_established_batch(struct seq_file *seq, + struct sock **start_sk) +{ + struct bpf_tcp_iter_state *iter = seq->private; + struct hlist_nulls_node *node; + struct sock *sk = *start_sk; + unsigned int expected = 0; + + *start_sk = NULL; + sk_nulls_for_each_from(sk, node) { + if (!seq_sk_match(seq, sk)) + continue; + expected++; + if (iter->end_sk < iter->max_sk) { + /* reqsk_queue_hash_req() inserts with sk_refcnt == 0 + * and refcount_set()s it after the bucket lock drops. + */ + if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + continue; + iter->batch[iter->end_sk++].sk = sk; + } else if (!*start_sk) { + /* Remember where we left off. */ + *start_sk = sk; + } + } + + return expected; +} + +static unsigned int bpf_iter_fill_batch(struct seq_file *seq, + struct sock **start_sk) +{ + struct bpf_tcp_iter_state *iter = seq->private; + struct tcp_iter_state *st = &iter->state; + + if (st->state == TCP_SEQ_STATE_LISTENING) + return bpf_iter_tcp_listening_batch(seq, start_sk); + else + return bpf_iter_tcp_established_batch(seq, start_sk); +} + +static void bpf_iter_tcp_unlock_bucket(struct seq_file *seq) { struct inet_hashinfo *hinfo = seq_file_net(seq)->ipv4.tcp_death_row.hashinfo; struct bpf_tcp_iter_state *iter = seq->private; struct tcp_iter_state *st = &iter->state; + + if (st->state == TCP_SEQ_STATE_LISTENING) + spin_unlock(&hinfo->lhash2[st->bucket].lock); + else + spin_unlock_bh(inet_ehash_lockp(hinfo, st->bucket)); +} + +static struct sock *bpf_iter_tcp_batch(struct seq_file *seq) +{ + struct bpf_tcp_iter_state *iter = seq->private; unsigned int expected; - bool resized = false; struct sock *sk; - - /* The st->bucket is done. Directly advance to the next - * bucket instead of having the tcp_seek_last_pos() to skip - * one by one in the current bucket and eventually find out - * it has to advance to the next bucket. - */ - if (iter->st_bucket_done) { - st->offset = 0; - st->bucket++; - if (st->state == TCP_SEQ_STATE_LISTENING && - st->bucket > hinfo->lhash2_mask) { - st->state = TCP_SEQ_STATE_ESTABLISHED; - st->bucket = 0; - } - } + int err; again: - /* Get a new batch */ - iter->cur_sk = 0; - iter->end_sk = 0; - iter->st_bucket_done = false; + sk = bpf_iter_tcp_resume(seq); + if (!sk) + return NULL; /* Done */ + + expected = bpf_iter_fill_batch(seq, &sk); + if (likely(!sk)) + goto done; - sk = tcp_seek_last_pos(seq); + /* Batch size was too small. */ + bpf_iter_tcp_unlock_bucket(seq); + bpf_iter_tcp_put_batch(iter); + err = bpf_iter_tcp_realloc_batch(iter, expected * 3 / 2, + GFP_USER); + if (err) { + iter->cur_sk = 0; + iter->end_sk = 0; + return ERR_PTR(err); + } + + sk = bpf_iter_tcp_resume(seq); if (!sk) return NULL; /* Done */ - if (st->state == TCP_SEQ_STATE_LISTENING) - expected = bpf_iter_tcp_listening_batch(seq, sk); - else - expected = bpf_iter_tcp_established_batch(seq, sk); + expected = bpf_iter_fill_batch(seq, &sk); + if (likely(!sk)) + goto done; - if (iter->end_sk == expected) { - iter->st_bucket_done = true; - return sk; + /* Batch size was still too small. Hold onto the lock while we try + * again with a larger batch to make sure the current bucket's size + * does not change in the meantime. + */ + err = bpf_iter_tcp_realloc_batch(iter, expected, GFP_NOWAIT); + if (err) { + bpf_iter_tcp_unlock_bucket(seq); + return ERR_PTR(err); } - if (!resized && !bpf_iter_tcp_realloc_batch(iter, expected * 3 / 2)) { - resized = true; + bpf_iter_fill_batch(seq, &sk); + WARN_ON_ONCE(sk); +done: + bpf_iter_tcp_unlock_bucket(seq); + if (unlikely(!iter->end_sk)) { + ++iter->state.bucket; goto again; } - - return sk; + return iter->batch[0].sk; } static void *bpf_iter_tcp_seq_start(struct seq_file *seq, loff_t *pos) @@ -2962,16 +3123,11 @@ static void *bpf_iter_tcp_seq_next(struct seq_file *seq, void *v, loff_t *pos) * meta.seq_num is used instead. */ st->num++; - /* Move st->offset to the next sk in the bucket such that - * the future start() will resume at st->offset in - * st->bucket. See tcp_seek_last_pos(). - */ - st->offset++; - sock_gen_put(iter->batch[iter->cur_sk++]); + sock_gen_put(iter->batch[iter->cur_sk++].sk); } if (iter->cur_sk < iter->end_sk) - sk = iter->batch[iter->cur_sk]; + sk = iter->batch[iter->cur_sk].sk; else sk = bpf_iter_tcp_batch(seq); @@ -3037,10 +3193,8 @@ static void bpf_iter_tcp_seq_stop(struct seq_file *seq, void *v) (void)tcp_prog_seq_show(prog, &meta, v, 0); } - if (iter->cur_sk < iter->end_sk) { + if (iter->cur_sk < iter->end_sk) bpf_iter_tcp_put_batch(iter); - iter->st_bucket_done = false; - } } static const struct seq_operations bpf_iter_tcp_seq_ops = { @@ -3355,7 +3509,7 @@ static int bpf_iter_init_tcp(void *priv_data, struct bpf_iter_aux_info *aux) if (err) return err; - err = bpf_iter_tcp_realloc_batch(iter, INIT_BATCH_SZ); + err = bpf_iter_tcp_realloc_batch(iter, INIT_BATCH_SZ, GFP_USER); if (err) { bpf_iter_fini_seq_net(priv_data); return err; diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c index 2eea9672ca01..c3f5e4fc7b21 100644 --- a/net/ipv4/tcp_minisocks.c +++ b/net/ipv4/tcp_minisocks.c @@ -261,10 +261,9 @@ static void tcp_time_wait_init(struct sock *sk, struct tcp_timewait_sock *tcptw) tcptw->tw_md5_key = kmemdup(key, sizeof(*key), GFP_ATOMIC); if (!tcptw->tw_md5_key) return; - if (!tcp_alloc_md5sig_pool()) - goto out_free; if (!static_key_fast_inc_not_disabled(&tcp_md5_needed.key.key)) goto out_free; + tcp_md5_add_sigpool(); } return; out_free: @@ -348,16 +347,26 @@ void tcp_time_wait(struct sock *sk, int state, int timeo) } EXPORT_SYMBOL(tcp_time_wait); +#ifdef CONFIG_TCP_MD5SIG +static void tcp_md5_twsk_free_rcu(struct rcu_head *head) +{ + struct tcp_md5sig_key *key; + + key = container_of(head, struct tcp_md5sig_key, rcu); + kfree(key); + static_branch_slow_dec_deferred(&tcp_md5_needed); + tcp_md5_release_sigpool(); +} +#endif + void tcp_twsk_destructor(struct sock *sk) { #ifdef CONFIG_TCP_MD5SIG if (static_branch_unlikely(&tcp_md5_needed.key)) { struct tcp_timewait_sock *twsk = tcp_twsk(sk); - if (twsk->tw_md5_key) { - kfree_rcu(twsk->tw_md5_key, rcu); - static_branch_slow_dec_deferred(&tcp_md5_needed); - } + if (twsk->tw_md5_key) + call_rcu(&twsk->tw_md5_key->rcu, tcp_md5_twsk_free_rcu); } #endif } diff --git a/net/ipv4/tcp_sigpool.c b/net/ipv4/tcp_sigpool.c new file mode 100644 index 000000000000..8512cb09ebc0 --- /dev/null +++ b/net/ipv4/tcp_sigpool.c @@ -0,0 +1,357 @@ +// SPDX-License-Identifier: GPL-2.0-or-later + +#include +#include +#include +#include +#include +#include +#include +#include + +static size_t __scratch_size; +static DEFINE_PER_CPU(void __rcu *, sigpool_scratch); + +struct sigpool_entry { + struct crypto_ahash *hash; + const char *alg; + struct kref kref; + uint16_t needs_key:1, + reserved:15; +}; + +#define CPOOL_SIZE (PAGE_SIZE / sizeof(struct sigpool_entry)) +static struct sigpool_entry cpool[CPOOL_SIZE]; +static unsigned int cpool_populated; +static DEFINE_MUTEX(cpool_mutex); + +/* Slow-path */ +struct scratches_to_free { + struct rcu_head rcu; + unsigned int cnt; + void *scratches[]; +}; + +static void free_old_scratches(struct rcu_head *head) +{ + struct scratches_to_free *stf; + + stf = container_of(head, struct scratches_to_free, rcu); + while (stf->cnt--) + kfree(stf->scratches[stf->cnt]); + kfree(stf); +} + +/** + * sigpool_reserve_scratch - re-allocates scratch buffer, slow-path + * @size: request size for the scratch/temp buffer + */ +static int sigpool_reserve_scratch(size_t size) +{ + struct scratches_to_free *stf; + size_t stf_sz = struct_size(stf, scratches, num_possible_cpus()); + int cpu, err = 0; + + lockdep_assert_held(&cpool_mutex); + if (__scratch_size >= size) + return 0; + + stf = kmalloc(stf_sz, GFP_KERNEL); + if (!stf) + return -ENOMEM; + stf->cnt = 0; + + size = max(size, __scratch_size); + cpus_read_lock(); + for_each_possible_cpu(cpu) { + void *scratch, *old_scratch; + + scratch = kmalloc_node(size, GFP_KERNEL, cpu_to_node(cpu)); + if (!scratch) { + err = -ENOMEM; + break; + } + + old_scratch = rcu_replace_pointer(per_cpu(sigpool_scratch, cpu), + scratch, lockdep_is_held(&cpool_mutex)); + if (!cpu_online(cpu) || !old_scratch) { + kfree(old_scratch); + continue; + } + stf->scratches[stf->cnt++] = old_scratch; + } + cpus_read_unlock(); + if (!err) + __scratch_size = size; + + call_rcu(&stf->rcu, free_old_scratches); + return err; +} + +static void sigpool_scratch_free(void) +{ + int cpu; + + for_each_possible_cpu(cpu) + kfree(rcu_replace_pointer(per_cpu(sigpool_scratch, cpu), + NULL, lockdep_is_held(&cpool_mutex))); + __scratch_size = 0; +} + +static int __cpool_try_clone(struct crypto_ahash *hash) +{ + struct crypto_ahash *tmp; + + tmp = crypto_clone_ahash(hash); + if (IS_ERR(tmp)) + return PTR_ERR(tmp); + + crypto_free_ahash(tmp); + return 0; +} + +static int __cpool_alloc_ahash(struct sigpool_entry *e, const char *alg) +{ + struct crypto_ahash *cpu0_hash; + int ret; + + e->alg = kstrdup(alg, GFP_KERNEL); + if (!e->alg) + return -ENOMEM; + + cpu0_hash = crypto_alloc_ahash(alg, 0, CRYPTO_ALG_ASYNC); + if (IS_ERR(cpu0_hash)) { + ret = PTR_ERR(cpu0_hash); + goto out_free_alg; + } + + e->needs_key = crypto_ahash_get_flags(cpu0_hash) & CRYPTO_TFM_NEED_KEY; + + ret = __cpool_try_clone(cpu0_hash); + if (ret) + goto out_free_cpu0_hash; + e->hash = cpu0_hash; + kref_init(&e->kref); + return 0; + +out_free_cpu0_hash: + crypto_free_ahash(cpu0_hash); +out_free_alg: + kfree(e->alg); + e->alg = NULL; + return ret; +} + +/** + * tcp_sigpool_alloc_ahash - allocates pool for ahash requests + * @alg: name of async hash algorithm + * @scratch_size: reserve a tcp_sigpool::scratch buffer of this size + */ +int tcp_sigpool_alloc_ahash(const char *alg, size_t scratch_size) +{ + int i, ret; + + /* slow-path */ + mutex_lock(&cpool_mutex); + ret = sigpool_reserve_scratch(scratch_size); + if (ret) + goto out; + for (i = 0; i < cpool_populated; i++) { + if (!cpool[i].alg) + continue; + if (strcmp(cpool[i].alg, alg)) + continue; + + /* pairs with tcp_sigpool_release() */ + if (!kref_get_unless_zero(&cpool[i].kref)) + kref_init(&cpool[i].kref); + ret = i; + goto out; + } + + for (i = 0; i < cpool_populated; i++) { + if (!cpool[i].alg) + break; + } + if (i >= CPOOL_SIZE) { + ret = -ENOSPC; + goto out; + } + + ret = __cpool_alloc_ahash(&cpool[i], alg); + if (!ret) { + ret = i; + if (i == cpool_populated) + cpool_populated++; + } +out: + mutex_unlock(&cpool_mutex); + return ret; +} +EXPORT_SYMBOL_GPL(tcp_sigpool_alloc_ahash); + +static void __cpool_free_entry(struct sigpool_entry *e) +{ + crypto_free_ahash(e->hash); + kfree(e->alg); + memset(e, 0, sizeof(*e)); +} + +static void cpool_cleanup_work_cb(struct work_struct *work) +{ + bool free_scratch = true; + unsigned int i; + + mutex_lock(&cpool_mutex); + for (i = 0; i < cpool_populated; i++) { + if (kref_read(&cpool[i].kref) > 0) { + free_scratch = false; + continue; + } + if (!cpool[i].alg) + continue; + __cpool_free_entry(&cpool[i]); + } + if (free_scratch) + sigpool_scratch_free(); + mutex_unlock(&cpool_mutex); +} + +static DECLARE_WORK(cpool_cleanup_work, cpool_cleanup_work_cb); +static void cpool_schedule_cleanup(struct kref *kref) +{ + schedule_work(&cpool_cleanup_work); +} + +/** + * tcp_sigpool_release - decreases number of users for a pool. If it was + * the last user of the pool, releases any memory that was consumed. + * @id: tcp_sigpool that was previously allocated by tcp_sigpool_alloc_ahash() + */ +void tcp_sigpool_release(unsigned int id) +{ + if (WARN_ON_ONCE(id >= cpool_populated || !cpool[id].alg)) + return; + + /* slow-path */ + kref_put(&cpool[id].kref, cpool_schedule_cleanup); +} +EXPORT_SYMBOL_GPL(tcp_sigpool_release); + +/** + * tcp_sigpool_get - increases number of users (refcounter) for a pool + * @id: tcp_sigpool that was previously allocated by tcp_sigpool_alloc_ahash() + */ +void tcp_sigpool_get(unsigned int id) +{ + if (WARN_ON_ONCE(id >= cpool_populated || !cpool[id].alg)) + return; + kref_get(&cpool[id].kref); +} +EXPORT_SYMBOL_GPL(tcp_sigpool_get); + +int tcp_sigpool_start(unsigned int id, struct tcp_sigpool *c) __cond_acquires(RCU_BH) +{ + struct crypto_ahash *hash; + + rcu_read_lock_bh(); + if (WARN_ON_ONCE(id >= cpool_populated || !cpool[id].alg)) { + rcu_read_unlock_bh(); + return -EINVAL; + } + + hash = crypto_clone_ahash(cpool[id].hash); + if (IS_ERR(hash)) { + rcu_read_unlock_bh(); + return PTR_ERR(hash); + } + + c->req = ahash_request_alloc(hash, GFP_ATOMIC); + if (!c->req) { + crypto_free_ahash(hash); + rcu_read_unlock_bh(); + return -ENOMEM; + } + ahash_request_set_callback(c->req, 0, NULL, NULL); + + /* Pairs with tcp_sigpool_reserve_scratch(), scratch area is + * valid (allocated) until tcp_sigpool_end(). + */ + c->scratch = rcu_dereference_bh(*this_cpu_ptr(&sigpool_scratch)); + return 0; +} +EXPORT_SYMBOL_GPL(tcp_sigpool_start); + +void tcp_sigpool_end(struct tcp_sigpool *c) __releases(RCU_BH) +{ + struct crypto_ahash *hash = crypto_ahash_reqtfm(c->req); + + rcu_read_unlock_bh(); + ahash_request_free(c->req); + crypto_free_ahash(hash); +} +EXPORT_SYMBOL_GPL(tcp_sigpool_end); + +/** + * tcp_sigpool_algo - return algorithm of tcp_sigpool + * @id: tcp_sigpool that was previously allocated by tcp_sigpool_alloc_ahash() + * @buf: buffer to return name of algorithm + * @buf_len: size of @buf + */ +size_t tcp_sigpool_algo(unsigned int id, char *buf, size_t buf_len) +{ + if (WARN_ON_ONCE(id >= cpool_populated || !cpool[id].alg)) + return -EINVAL; + + return strscpy(buf, cpool[id].alg, buf_len); +} +EXPORT_SYMBOL_GPL(tcp_sigpool_algo); + +/** + * tcp_sigpool_hash_skb_data - hash data in skb with initialized tcp_sigpool + * @hp: tcp_sigpool pointer + * @skb: buffer to add sign for + * @header_len: TCP header length for this segment + */ +int tcp_sigpool_hash_skb_data(struct tcp_sigpool *hp, + const struct sk_buff *skb, + unsigned int header_len) +{ + const unsigned int head_data_len = skb_headlen(skb) > header_len ? + skb_headlen(skb) - header_len : 0; + const struct skb_shared_info *shi = skb_shinfo(skb); + const struct tcphdr *tp = tcp_hdr(skb); + struct ahash_request *req = hp->req; + struct sk_buff *frag_iter; + struct scatterlist sg; + unsigned int i; + + sg_init_table(&sg, 1); + + sg_set_buf(&sg, ((u8 *)tp) + header_len, head_data_len); + ahash_request_set_crypt(req, &sg, NULL, head_data_len); + if (crypto_ahash_update(req)) + return 1; + + for (i = 0; i < shi->nr_frags; ++i) { + const skb_frag_t *f = &shi->frags[i]; + unsigned int offset = skb_frag_off(f); + struct page *page; + + page = skb_frag_page(f) + (offset >> PAGE_SHIFT); + sg_set_page(&sg, page, skb_frag_size(f), offset_in_page(offset)); + ahash_request_set_crypt(req, &sg, NULL, skb_frag_size(f)); + if (crypto_ahash_update(req)) + return 1; + } + + skb_walk_frags(skb, frag_iter) + if (tcp_sigpool_hash_skb_data(hp, frag_iter, 0)) + return 1; + + return 0; +} +EXPORT_SYMBOL(tcp_sigpool_hash_skb_data); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Per-CPU pool of crypto requests"); diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c index 84ae2759ff19..b9b21e5af1a7 100644 --- a/net/ipv4/udp_offload.c +++ b/net/ipv4/udp_offload.c @@ -22,17 +22,19 @@ static struct sk_buff *__skb_udp_tunnel_segment(struct sk_buff *skb, int tnl_hlen = skb_inner_mac_header(skb) - skb_transport_header(skb); bool remcsum, need_csum, offload_csum, gso_partial; struct sk_buff *segs = ERR_PTR(-EINVAL); - struct udphdr *uh = udp_hdr(skb); u16 mac_offset = skb->mac_header; __be16 protocol = skb->protocol; u16 mac_len = skb->mac_len; int udp_offset, outer_hlen; + struct udphdr *uh; __wsum partial; bool need_ipsec; if (unlikely(!pskb_may_pull(skb, tnl_hlen))) goto out; + uh = udp_hdr(skb); + /* Adjust partial header checksum to negate old length. * We cannot rely on the value contained in uh->len as it is * possible that the actual value exceeds the boundaries of the diff --git a/net/ipv6/ip6_tunnel.c b/net/ipv6/ip6_tunnel.c index 825d40b505cd..dfabd9969b3f 100644 --- a/net/ipv6/ip6_tunnel.c +++ b/net/ipv6/ip6_tunnel.c @@ -675,6 +675,9 @@ ip6ip6_err(struct sk_buff *skb, struct inet6_skb_parm *opt, if (!skb2) return 0; + /* Remove debris left by outer IPv6 stack. */ + memset(IP6CB(skb2), 0, sizeof(*IP6CB(skb2))); + skb_dst_drop(skb2); skb_pull(skb2, offset); skb_reset_network_header(skb2); diff --git a/net/ipv6/route.c b/net/ipv6/route.c index e6f1a22bc2be..24d097290dbc 100644 --- a/net/ipv6/route.c +++ b/net/ipv6/route.c @@ -989,13 +989,13 @@ int rt6_route_rcv(struct net_device *dev, u8 *opt, int len, } else if (rinfo->prefix_len > 128) { return -EINVAL; } else if (rinfo->prefix_len > 64) { - if (rinfo->length < 2) { + /* RFC 4191: Length MUST be 3 when Prefix Length > 64 */ + if (rinfo->length < 3) return -EINVAL; - } } else if (rinfo->prefix_len > 0) { - if (rinfo->length < 1) { + /* RFC 4191: Length MUST be 2 or 3 when Prefix Length > 0 */ + if (rinfo->length < 2) return -EINVAL; - } } pref = rinfo->route_pref; diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c index 2c579868fe81..689c0b383ebf 100644 --- a/net/ipv6/tcp_ipv6.c +++ b/net/ipv6/tcp_ipv6.c @@ -665,7 +665,7 @@ static int tcp_v6_parse_md5_keys(struct sock *sk, int optname, cmd.tcpm_key, cmd.tcpm_keylen); } -static int tcp_v6_md5_hash_headers(struct tcp_md5sig_pool *hp, +static int tcp_v6_md5_hash_headers(struct tcp_sigpool *hp, const struct in6_addr *daddr, const struct in6_addr *saddr, const struct tcphdr *th, int nbytes) @@ -686,39 +686,36 @@ static int tcp_v6_md5_hash_headers(struct tcp_md5sig_pool *hp, _th->check = 0; sg_init_one(&sg, bp, sizeof(*bp) + sizeof(*th)); - ahash_request_set_crypt(hp->md5_req, &sg, NULL, + ahash_request_set_crypt(hp->req, &sg, NULL, sizeof(*bp) + sizeof(*th)); - return crypto_ahash_update(hp->md5_req); + return crypto_ahash_update(hp->req); } static int tcp_v6_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key, const struct in6_addr *daddr, struct in6_addr *saddr, const struct tcphdr *th) { - struct tcp_md5sig_pool *hp; - struct ahash_request *req; + struct tcp_sigpool hp; - hp = tcp_get_md5sig_pool(); - if (!hp) - goto clear_hash_noput; - req = hp->md5_req; + if (tcp_sigpool_start(tcp_md5_sigpool_id, &hp)) + goto clear_hash_nostart; - if (crypto_ahash_init(req)) + if (crypto_ahash_init(hp.req)) goto clear_hash; - if (tcp_v6_md5_hash_headers(hp, daddr, saddr, th, th->doff << 2)) + if (tcp_v6_md5_hash_headers(&hp, daddr, saddr, th, th->doff << 2)) goto clear_hash; - if (tcp_md5_hash_key(hp, key)) + if (tcp_md5_hash_key(&hp, key)) goto clear_hash; - ahash_request_set_crypt(req, NULL, md5_hash, 0); - if (crypto_ahash_final(req)) + ahash_request_set_crypt(hp.req, NULL, md5_hash, 0); + if (crypto_ahash_final(hp.req)) goto clear_hash; - tcp_put_md5sig_pool(); + tcp_sigpool_end(&hp); return 0; clear_hash: - tcp_put_md5sig_pool(); -clear_hash_noput: + tcp_sigpool_end(&hp); +clear_hash_nostart: memset(md5_hash, 0, 16); return 1; } @@ -728,10 +725,9 @@ static int tcp_v6_md5_hash_skb(char *md5_hash, const struct sock *sk, const struct sk_buff *skb) { - const struct in6_addr *saddr, *daddr; - struct tcp_md5sig_pool *hp; - struct ahash_request *req; const struct tcphdr *th = tcp_hdr(skb); + const struct in6_addr *saddr, *daddr; + struct tcp_sigpool hp; if (sk) { /* valid for establish/request sockets */ saddr = &sk->sk_v6_rcv_saddr; @@ -742,30 +738,28 @@ static int tcp_v6_md5_hash_skb(char *md5_hash, daddr = &ip6h->daddr; } - hp = tcp_get_md5sig_pool(); - if (!hp) - goto clear_hash_noput; - req = hp->md5_req; + if (tcp_sigpool_start(tcp_md5_sigpool_id, &hp)) + goto clear_hash_nostart; - if (crypto_ahash_init(req)) + if (crypto_ahash_init(hp.req)) goto clear_hash; - if (tcp_v6_md5_hash_headers(hp, daddr, saddr, th, skb->len)) + if (tcp_v6_md5_hash_headers(&hp, daddr, saddr, th, skb->len)) goto clear_hash; - if (tcp_md5_hash_skb_data(hp, skb, th->doff << 2)) + if (tcp_sigpool_hash_skb_data(&hp, skb, th->doff << 2)) goto clear_hash; - if (tcp_md5_hash_key(hp, key)) + if (tcp_md5_hash_key(&hp, key)) goto clear_hash; - ahash_request_set_crypt(req, NULL, md5_hash, 0); - if (crypto_ahash_final(req)) + ahash_request_set_crypt(hp.req, NULL, md5_hash, 0); + if (crypto_ahash_final(hp.req)) goto clear_hash; - tcp_put_md5sig_pool(); + tcp_sigpool_end(&hp); return 0; clear_hash: - tcp_put_md5sig_pool(); -clear_hash_noput: + tcp_sigpool_end(&hp); +clear_hash_nostart: memset(md5_hash, 0, 16); return 1; } diff --git a/net/mac802154/scan.c b/net/mac802154/scan.c index 087e32582dbb..c8f4309f9c24 100644 --- a/net/mac802154/scan.c +++ b/net/mac802154/scan.c @@ -415,6 +415,7 @@ void mac802154_beacon_worker(struct work_struct *work) container_of(work, struct ieee802154_local, beacon_work.work); struct cfg802154_beacon_request *beacon_req; struct ieee802154_sub_if_data *sdata; + netdevice_tracker dev_tracker; struct wpan_dev *wpan_dev; u8 interval; int ret; @@ -427,12 +428,14 @@ void mac802154_beacon_worker(struct work_struct *work) } sdata = IEEE802154_WPAN_DEV_TO_SUB_IF(beacon_req->wpan_dev); + netdev_hold(sdata->dev, &dev_tracker, GFP_ATOMIC); /* Wait an arbitrary amount of time in case we cannot use the device */ if (local->suspended || !ieee802154_sdata_running(sdata)) { rcu_read_unlock(); queue_delayed_work(local->mac_wq, &local->beacon_work, msecs_to_jiffies(1000)); + netdev_put(sdata->dev, &dev_tracker); return; } @@ -450,6 +453,7 @@ void mac802154_beacon_worker(struct work_struct *work) if (interval < IEEE802154_ACTIVE_SCAN_DURATION) queue_delayed_work(local->mac_wq, &local->beacon_work, local->beacon_interval); + netdev_put(sdata->dev, &dev_tracker); } int mac802154_stop_beacons_locked(struct ieee802154_local *local, diff --git a/net/ncsi/ncsi-netlink.c b/net/ncsi/ncsi-netlink.c index 2f872d064396..8cc538358f6a 100644 --- a/net/ncsi/ncsi-netlink.c +++ b/net/ncsi/ncsi-netlink.c @@ -461,6 +461,10 @@ static int ncsi_send_cmd_nl(struct sk_buff *msg, struct genl_info *info) nca.req_flags = NCSI_REQ_FLAG_NETLINK_DRIVEN; nca.info = info; nca.payload = ntohs(hdr->length); + if (nca.payload > len - sizeof(*hdr)) { + ret = -EINVAL; + goto out_netlink; + } nca.data = data + sizeof(*hdr); ret = ncsi_xmit_cmd(&nca); diff --git a/net/netfilter/ipset/ip_set_bitmap_gen.h b/net/netfilter/ipset/ip_set_bitmap_gen.h index 9523104a90da..40f0383883f9 100644 --- a/net/netfilter/ipset/ip_set_bitmap_gen.h +++ b/net/netfilter/ipset/ip_set_bitmap_gen.h @@ -75,7 +75,7 @@ mtype_flush(struct ip_set *set) mtype_ext_cleanup(set); bitmap_zero(map->members, map->elements); set->elements = 0; - set->ext_size = 0; + atomic64_set(&set->ext_size, 0); } /* Calculate the actual memory size of the set data */ @@ -91,7 +91,7 @@ mtype_head(struct ip_set *set, struct sk_buff *skb) { const struct mtype *map = set->data; struct nlattr *nested; - size_t memsize = mtype_memsize(map, set->dsize) + set->ext_size; + size_t memsize = mtype_memsize(map, set->dsize) + atomic64_read(&set->ext_size); nested = nla_nest_start(skb, IPSET_ATTR_DATA); if (!nested) diff --git a/net/netfilter/ipset/ip_set_core.c b/net/netfilter/ipset/ip_set_core.c index f51a1af31513..29bf5ee74fe3 100644 --- a/net/netfilter/ipset/ip_set_core.c +++ b/net/netfilter/ipset/ip_set_core.c @@ -350,7 +350,7 @@ ip_set_init_comment(struct ip_set *set, struct ip_set_comment *comment, size_t len = ext->comment ? strlen(ext->comment) : 0; if (unlikely(c)) { - set->ext_size -= sizeof(*c) + strlen(c->str) + 1; + atomic64_sub(sizeof(*c) + strlen(c->str) + 1, &set->ext_size); rcu_assign_pointer(comment->c, NULL); kfree_rcu(c, rcu); } @@ -362,7 +362,7 @@ ip_set_init_comment(struct ip_set *set, struct ip_set_comment *comment, if (unlikely(!c)) return; strscpy(c->str, ext->comment, len + 1); - set->ext_size += sizeof(*c) + strlen(c->str) + 1; + atomic64_add(sizeof(*c) + strlen(c->str) + 1, &set->ext_size); rcu_assign_pointer(comment->c, c); } EXPORT_SYMBOL_GPL(ip_set_init_comment); @@ -392,7 +392,7 @@ ip_set_comment_free(struct ip_set *set, void *ptr) c = rcu_dereference_protected(comment->c, 1); if (unlikely(!c)) return; - set->ext_size -= sizeof(*c) + strlen(c->str) + 1; + atomic64_sub(sizeof(*c) + strlen(c->str) + 1, &set->ext_size); rcu_assign_pointer(comment->c, NULL); kfree_rcu(c, rcu); } diff --git a/net/netfilter/ipset/ip_set_hash_gen.h b/net/netfilter/ipset/ip_set_hash_gen.h index 81ad6747bc7a..53b65f013594 100644 --- a/net/netfilter/ipset/ip_set_hash_gen.h +++ b/net/netfilter/ipset/ip_set_hash_gen.h @@ -1288,7 +1288,7 @@ mtype_head(struct ip_set *set, struct sk_buff *skb) rcu_read_lock_bh(); t = rcu_dereference_bh(h->table); mtype_ext_size(set, &elements, &ext_size); - memsize = mtype_ahash_memsize(h, t) + ext_size + set->ext_size; + memsize = mtype_ahash_memsize(h, t) + ext_size + atomic64_read(&set->ext_size); htable_bits = t->htable_bits; rcu_read_unlock_bh(); diff --git a/net/netfilter/ipset/ip_set_list_set.c b/net/netfilter/ipset/ip_set_list_set.c index 83e1fdcc752d..9d6ab69ca1a3 100644 --- a/net/netfilter/ipset/ip_set_list_set.c +++ b/net/netfilter/ipset/ip_set_list_set.c @@ -421,7 +421,7 @@ list_set_flush(struct ip_set *set) list_for_each_entry_safe(e, n, &map->members, list) list_set_del(set, e); set->elements = 0; - set->ext_size = 0; + atomic64_set(&set->ext_size, 0); } static void @@ -455,7 +455,7 @@ list_set_head(struct ip_set *set, struct sk_buff *skb) { const struct list_set *map = set->data; struct nlattr *nested; - size_t memsize = list_set_memsize(map, set->dsize) + set->ext_size; + size_t memsize = list_set_memsize(map, set->dsize) + atomic64_read(&set->ext_size); nested = nla_nest_start(skb, IPSET_ATTR_DATA); if (!nested) diff --git a/net/netfilter/ipvs/ip_vs_conn.c b/net/netfilter/ipvs/ip_vs_conn.c index 811552669dac..b2b74ce67490 100644 --- a/net/netfilter/ipvs/ip_vs_conn.c +++ b/net/netfilter/ipvs/ip_vs_conn.c @@ -569,12 +569,6 @@ static inline void ip_vs_bind_xmit_v6(struct ip_vs_conn *cp) #endif -static inline int ip_vs_dest_totalconns(struct ip_vs_dest *dest) -{ - return atomic_read(&dest->activeconns) - + atomic_read(&dest->inactconns); -} - /* * Bind a connection entry with a virtual service destination * Called just after a new connection entry is created. @@ -626,23 +620,22 @@ ip_vs_bind_dest(struct ip_vs_conn *cp, struct ip_vs_dest *dest) /* Update the connection counters */ if (!(flags & IP_VS_CONN_F_TEMPLATE)) { + int tc; + /* It is a normal connection, so modify the counters * according to the flags, later the protocol can * update them on state change */ if (!(flags & IP_VS_CONN_F_INACTIVE)) atomic_inc(&dest->activeconns); - else - atomic_inc(&dest->inactconns); + tc = atomic_inc_return(&dest->totalconns); + if (tc == READ_ONCE(dest->u_threshold)) + ip_vs_dest_update_overload(dest, 1); } else { /* It is a persistent connection/template, so increase the persistent connection counter */ atomic_inc(&dest->persistconns); } - - if (dest->u_threshold != 0 && - ip_vs_dest_totalconns(dest) >= dest->u_threshold) - dest->flags |= IP_VS_DEST_F_OVERLOAD; } @@ -723,30 +716,20 @@ static inline void ip_vs_unbind_dest(struct ip_vs_conn *cp) /* Update the connection counters */ if (!(cp->flags & IP_VS_CONN_F_TEMPLATE)) { - /* It is a normal connection, so decrease the inactconns - or activeconns counter */ - if (cp->flags & IP_VS_CONN_F_INACTIVE) { - atomic_dec(&dest->inactconns); - } else { + int tc; + + /* It is a normal connection, so decrease the counters */ + if (!(cp->flags & IP_VS_CONN_F_INACTIVE)) atomic_dec(&dest->activeconns); - } + tc = atomic_fetch_dec(&dest->totalconns); + if (tc == READ_ONCE(dest->l_threshold_val)) + ip_vs_dest_update_overload(dest, -1); } else { /* It is a persistent connection/template, so decrease the persistent connection counter */ atomic_dec(&dest->persistconns); } - if (dest->l_threshold != 0) { - if (ip_vs_dest_totalconns(dest) < dest->l_threshold) - dest->flags &= ~IP_VS_DEST_F_OVERLOAD; - } else if (dest->u_threshold != 0) { - if (ip_vs_dest_totalconns(dest) * 4 < dest->u_threshold * 3) - dest->flags &= ~IP_VS_DEST_F_OVERLOAD; - } else { - if (dest->flags & IP_VS_DEST_F_OVERLOAD) - dest->flags &= ~IP_VS_DEST_F_OVERLOAD; - } - ip_vs_dest_put(dest); } diff --git a/net/netfilter/ipvs/ip_vs_core.c b/net/netfilter/ipvs/ip_vs_core.c index e49a4840effb..a722bacce461 100644 --- a/net/netfilter/ipvs/ip_vs_core.c +++ b/net/netfilter/ipvs/ip_vs_core.c @@ -747,28 +747,27 @@ static int ip_vs_route_me_harder(struct netns_ipvs *ipvs, int af, */ void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, struct ip_vs_conn *cp, int inout, unsigned int toff, - bool has_ports) + bool has_ports, struct ip_vs_iphdr *ciph) { struct iphdr *iph = ip_hdr(skb); struct icmphdr *icmph = (struct icmphdr *)(skb->data + toff); - struct iphdr *ciph = (struct iphdr *)(icmph + 1); - unsigned int coff __maybe_unused = toff + sizeof(struct icmphdr); + struct iphdr *cih = (struct iphdr *)(icmph + 1); if (inout) { iph->saddr = cp->vaddr.ip; ip_send_check(iph); - ciph->daddr = cp->vaddr.ip; - ip_send_check(ciph); + cih->daddr = cp->vaddr.ip; + ip_send_check(cih); } else { iph->daddr = cp->daddr.ip; ip_send_check(iph); - ciph->saddr = cp->daddr.ip; - ip_send_check(ciph); + cih->saddr = cp->daddr.ip; + ip_send_check(cih); } /* the TCP/UDP/SCTP port */ if (has_ports) { - __be16 *ports = (void *)ciph + ciph->ihl*4; + __be16 *ports = (void *)(skb->data + ciph->len); if (inout) ports[1] = cp->vport; @@ -782,10 +781,10 @@ void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, skb->ip_summed = CHECKSUM_UNNECESSARY; if (inout) - IP_VS_DBG_PKT(11, AF_INET, pp, skb, coff, + IP_VS_DBG_PKT(11, AF_INET, pp, skb, ciph->off, "Forwarding altered outgoing ICMP"); else - IP_VS_DBG_PKT(11, AF_INET, pp, skb, coff, + IP_VS_DBG_PKT(11, AF_INET, pp, skb, ciph->off, "Forwarding altered incoming ICMP"); } @@ -878,7 +877,7 @@ static int handle_response_icmp(int af, struct sk_buff *skb, ip_vs_nat_icmp_v6(skb, pp, cp, 1, toff, has_ports, ciph); else #endif - ip_vs_nat_icmp(skb, pp, cp, 1, toff, has_ports); + ip_vs_nat_icmp(skb, pp, cp, 1, toff, has_ports, ciph); if (ip_vs_route_me_harder(cp->ipvs, af, skb, hooknum)) goto out; @@ -914,7 +913,7 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, struct ip_vs_iphdr ciph; struct ip_vs_conn *cp; struct ip_vs_protocol *pp; - unsigned int offset, ihl; + unsigned int offset; union nf_inet_addr snet; *related = 1; @@ -927,7 +926,6 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, return NF_ACCEPT; } - ihl = ipvsh->len; offset = ipvsh->len; ic = skb_header_pointer(skb, offset, sizeof(_icmph), &_icmph); if (ic == NULL) @@ -953,11 +951,15 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, /* Now find the contained IP header */ offset += sizeof(_icmph); + if (!ip_vs_fill_iph_skb_icmp(AF_INET, skb, offset, true, &ciph)) + return NF_ACCEPT; /* The packet looks wrong, ignore */ + cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph); - if (!(cih && cih->version == 4 && cih->ihl >= 5)) + if (!(cih && cih->version == 4 && + ciph.len - ciph.off >= sizeof(struct iphdr))) return NF_ACCEPT; /* The packet looks wrong, ignore */ - pp = ip_vs_proto_get(cih->protocol); + pp = ip_vs_proto_get(ciph.protocol); if (!pp) return NF_ACCEPT; @@ -968,8 +970,6 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, IP_VS_DBG_PKT(11, AF_INET, pp, skb, offset, "Checking outgoing ICMP for"); - ip_vs_fill_iph_skb_icmp(AF_INET, skb, offset, true, &ciph); - /* The embedded headers contain source and dest in reverse order */ cp = INDIRECT_CALL_1(pp->conn_out_get, ip_vs_conn_out_get_proto, ipvs, AF_INET, skb, &ciph); @@ -977,8 +977,8 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, return NF_ACCEPT; snet.ip = ipvsh->saddr.ip; - return handle_response_icmp(AF_INET, skb, &snet, cp, pp, &ciph, ihl, - hooknum); + return handle_response_icmp(AF_INET, skb, &snet, cp, pp, &ciph, + ipvsh->len, hooknum); } #ifdef CONFIG_IP_VS_IPV6 @@ -1621,10 +1621,12 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, /* Now find the contained IP header */ offset += sizeof(_icmph); cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph); - if (!(cih && cih->version == 4 && cih->ihl >= 5)) + if (!cih) return NF_ACCEPT; /* The packet looks wrong, ignore */ - raddr = (union nf_inet_addr *)&cih->daddr; hlen_ipip = cih->ihl * 4; + if (!(cih->version == 4 && hlen_ipip >= sizeof(struct iphdr))) + return NF_ACCEPT; /* The packet looks wrong, ignore */ + raddr = (union nf_inet_addr *)&cih->daddr; /* Special case for errors for IPIP/UDP/GRE tunnel packets */ tunnel = false; @@ -1641,9 +1643,6 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, if (!dest || dest->tun_type != IP_VS_CONN_F_TUNNEL_TYPE_IPIP) return NF_ACCEPT; offset += hlen_ipip; - cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph); - if (!(cih && cih->version == 4 && cih->ihl >= 5)) - return NF_ACCEPT; /* The packet looks wrong, ignore */ tunnel = true; } else if ((cih->protocol == IPPROTO_UDP || /* Can be UDP encap */ cih->protocol == IPPROTO_GRE) && /* Can be GRE encap */ @@ -1668,21 +1667,25 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, /* Skip IP and UDP/GRE tunnel headers */ offset = offset2 + ulen; /* Now we should be at the original IP header */ - cih = skb_header_pointer(skb, offset, sizeof(_ciph), - &_ciph); - if (cih && cih->version == 4 && cih->ihl >= 5 && - iproto == IPPROTO_IPIP) + if (iproto == IPPROTO_IPIP) tunnel = true; else return NF_ACCEPT; } } - pd = ip_vs_proto_data_get(ipvs, cih->protocol); + if (!ip_vs_fill_iph_skb_icmp(AF_INET, skb, offset, !tunnel, &ciph)) + return NF_ACCEPT; + pd = ip_vs_proto_data_get(ipvs, ciph.protocol); if (!pd) return NF_ACCEPT; pp = pd->pp; + cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph); + if (!(cih && cih->version == 4 && + ciph.len - ciph.off >= sizeof(struct iphdr))) + return NF_ACCEPT; /* The packet looks wrong, ignore */ + /* Is the embedded protocol header present? */ if (unlikely(cih->frag_off & htons(IP_OFFSET) && !pp->dont_defrag)) return NF_ACCEPT; @@ -1690,9 +1693,6 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, IP_VS_DBG_PKT(11, AF_INET, pp, skb, offset, "Checking incoming ICMP for"); - offset2 = offset; - ip_vs_fill_iph_skb_icmp(AF_INET, skb, offset, !tunnel, &ciph); - /* The embedded headers contain source and dest in reverse order. * For IPIP/UDP/GRE tunnel this is error for request, not for reply. */ @@ -1722,11 +1722,12 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, } if (tunnel) { - unsigned int hlen_orig = cih->ihl * 4; + unsigned int hlen_orig = ciph.len - ciph.off; __be32 info = ic->un.gateway; __u8 type = ic->type; __u8 code = ic->code; + offset2 = offset; /* Update the MTU */ if (ic->type == ICMP_DEST_UNREACH && ic->code == ICMP_FRAG_NEEDED) { @@ -1767,6 +1768,7 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, if (pskb_pull(skb, offset2) == NULL) goto ignore_tunnel; skb_reset_network_header(skb); + memset(&(IPCB(skb)->opt), 0, sizeof(IPCB(skb)->opt)); /* Ensure the IP header is present in headroom */ if (!pskb_may_pull(skb, hlen_orig)) goto ignore_tunnel; diff --git a/net/netfilter/ipvs/ip_vs_ctl.c b/net/netfilter/ipvs/ip_vs_ctl.c index 27b7552b1e24..c98bdcea8b2f 100644 --- a/net/netfilter/ipvs/ip_vs_ctl.c +++ b/net/netfilter/ipvs/ip_vs_ctl.c @@ -960,6 +960,40 @@ void ip_vs_stats_free(struct ip_vs_stats *stats) } } +/* Update overload flag based on number of dest conns and lower/upper + * connection thresholds: + * - conns reach u_threshold and exceed it: set the flag + * - conns go below l_threshold (or 75% of u_threshold): clear the flag + */ +static void __ip_vs_dest_update_overload(struct ip_vs_dest *dest, int mode) +{ + int conns; + u32 l, u; + + lockdep_assert_held(&dest->dst_lock); + u = READ_ONCE(dest->u_threshold); + if (!u) + goto unset; + l = READ_ONCE(dest->l_threshold_val); + conns = atomic_read(&dest->totalconns); + if (conns >= (mode > 0 ? l : u)) { + dest->flags |= IP_VS_DEST_F_OVERLOAD; + return; + } + if (conns >= (mode < 0 ? u : l)) + return; + +unset: + dest->flags &= ~IP_VS_DEST_F_OVERLOAD; +} + +void ip_vs_dest_update_overload(struct ip_vs_dest *dest, int mode) +{ + spin_lock_bh(&dest->dst_lock); + __ip_vs_dest_update_overload(dest, mode); + spin_unlock_bh(&dest->dst_lock); +} + /* * Update a destination in the given service */ @@ -1026,10 +1060,19 @@ __ip_vs_update_dest(struct ip_vs_service *svc, struct ip_vs_dest *dest, /* set the dest status flags */ dest->flags |= IP_VS_DEST_F_AVAILABLE; - if (udest->u_threshold == 0 || udest->u_threshold > dest->u_threshold) - dest->flags &= ~IP_VS_DEST_F_OVERLOAD; - dest->u_threshold = udest->u_threshold; - dest->l_threshold = udest->l_threshold; + if (READ_ONCE(dest->u_threshold) != udest->u_threshold || + READ_ONCE(dest->l_threshold) != udest->l_threshold) { + spin_lock_bh(&dest->dst_lock); + WRITE_ONCE(dest->u_threshold, udest->u_threshold); + WRITE_ONCE(dest->l_threshold, udest->l_threshold); + /* Low threshold defaults to 75% of upper threshold */ + WRITE_ONCE(dest->l_threshold_val, + udest->l_threshold ? : + (udest->u_threshold - + (udest->u_threshold >> 2))); + __ip_vs_dest_update_overload(dest, 0); + spin_unlock_bh(&dest->dst_lock); + } dest->af = udest->af; @@ -1101,7 +1144,7 @@ ip_vs_new_dest(struct ip_vs_service *svc, struct ip_vs_dest_user_kern *udest) dest->port = udest->port; atomic_set(&dest->activeconns, 0); - atomic_set(&dest->inactconns, 0); + atomic_set(&dest->totalconns, 0); atomic_set(&dest->persistconns, 0); refcount_set(&dest->refcnt, 1); @@ -1142,6 +1185,9 @@ ip_vs_add_dest(struct ip_vs_service *svc, struct ip_vs_dest_user_kern *udest) return -ERANGE; } + if (udest->u_threshold > INT_MAX) + return -EINVAL; + if (udest->tun_type == IP_VS_CONN_F_TUNNEL_TYPE_GUE) { if (udest->tun_port == 0) { pr_err("%s(): tunnel port is zero\n", __func__); @@ -1212,6 +1258,9 @@ ip_vs_edit_dest(struct ip_vs_service *svc, struct ip_vs_dest_user_kern *udest) return -ERANGE; } + if (udest->u_threshold > INT_MAX) + return -EINVAL; + if (udest->tun_type == IP_VS_CONN_F_TUNNEL_TYPE_GUE) { if (udest->tun_port == 0) { pr_err("%s(): tunnel port is zero\n", __func__); @@ -2456,7 +2505,7 @@ static int ip_vs_info_seq_show(struct seq_file *seq, void *v) ip_vs_fwd_name(atomic_read(&dest->conn_flags)), atomic_read(&dest->weight), atomic_read(&dest->activeconns), - atomic_read(&dest->inactconns)); + ip_vs_dest_inactconns(dest)); else #endif seq_printf(seq, @@ -2467,7 +2516,7 @@ static int ip_vs_info_seq_show(struct seq_file *seq, void *v) ip_vs_fwd_name(atomic_read(&dest->conn_flags)), atomic_read(&dest->weight), atomic_read(&dest->activeconns), - atomic_read(&dest->inactconns)); + ip_vs_dest_inactconns(dest)); } } @@ -2950,10 +2999,10 @@ __ip_vs_get_dest_entries(struct netns_ipvs *ipvs, const struct ip_vs_get_dests * entry.port = dest->port; entry.conn_flags = atomic_read(&dest->conn_flags); entry.weight = atomic_read(&dest->weight); - entry.u_threshold = dest->u_threshold; - entry.l_threshold = dest->l_threshold; + entry.u_threshold = READ_ONCE(dest->u_threshold); + entry.l_threshold = READ_ONCE(dest->l_threshold); entry.activeconns = atomic_read(&dest->activeconns); - entry.inactconns = atomic_read(&dest->inactconns); + entry.inactconns = ip_vs_dest_inactconns(dest); entry.persistconns = atomic_read(&dest->persistconns); ip_vs_copy_stats(&kstats, &dest->stats); ip_vs_export_stats_user(&entry.stats, &kstats); @@ -3552,12 +3601,14 @@ static int ip_vs_genl_fill_dest(struct sk_buff *skb, struct ip_vs_dest *dest) dest->tun_port) || nla_put_u16(skb, IPVS_DEST_ATTR_TUN_FLAGS, dest->tun_flags) || - nla_put_u32(skb, IPVS_DEST_ATTR_U_THRESH, dest->u_threshold) || - nla_put_u32(skb, IPVS_DEST_ATTR_L_THRESH, dest->l_threshold) || + nla_put_u32(skb, IPVS_DEST_ATTR_U_THRESH, + READ_ONCE(dest->u_threshold)) || + nla_put_u32(skb, IPVS_DEST_ATTR_L_THRESH, + READ_ONCE(dest->l_threshold)) || nla_put_u32(skb, IPVS_DEST_ATTR_ACTIVE_CONNS, atomic_read(&dest->activeconns)) || nla_put_u32(skb, IPVS_DEST_ATTR_INACT_CONNS, - atomic_read(&dest->inactconns)) || + ip_vs_dest_inactconns(dest)) || nla_put_u32(skb, IPVS_DEST_ATTR_PERSIST_CONNS, atomic_read(&dest->persistconns)) || nla_put_u16(skb, IPVS_DEST_ATTR_ADDR_FAMILY, dest->af)) diff --git a/net/netfilter/ipvs/ip_vs_est.c b/net/netfilter/ipvs/ip_vs_est.c index b48d3a09e217..dc12da2a5f38 100644 --- a/net/netfilter/ipvs/ip_vs_est.c +++ b/net/netfilter/ipvs/ip_vs_est.c @@ -186,8 +186,11 @@ static int ip_vs_estimation_kthread(void *data) } /* kthread 0 will handle the calc phase */ - if (ipvs->est_calc_phase) + if (ipvs->est_calc_phase) { ip_vs_est_calc_phase(ipvs); + if (kthread_should_stop() || !READ_ONCE(ipvs->enable)) + return 0; + } } while (1) { @@ -262,6 +265,7 @@ int ip_vs_est_kthread_start(struct netns_ipvs *ipvs, kd->task = NULL; goto out; } + get_task_struct(kd->task); set_user_nice(kd->task, sysctl_est_nice(ipvs)); set_cpus_allowed_ptr(kd->task, sysctl_est_cpulist(ipvs)); @@ -277,7 +281,7 @@ void ip_vs_est_kthread_stop(struct ip_vs_est_kt_data *kd) { if (kd->task) { pr_info("stopping estimator thread %d...\n", kd->id); - kthread_stop(kd->task); + kthread_stop_put(kd->task); kd->task = NULL; } } @@ -509,7 +513,7 @@ static void ip_vs_est_kthread_destroy(struct ip_vs_est_kt_data *kd) if (kd) { if (kd->task) { pr_info("stop unused estimator thread %d...\n", kd->id); - kthread_stop(kd->task); + kthread_stop_put(kd->task); } ip_vs_stats_free(kd->calc_stats); kfree(kd); diff --git a/net/netfilter/ipvs/ip_vs_lc.c b/net/netfilter/ipvs/ip_vs_lc.c index 9d34d81fc6f1..534db3c4a016 100644 --- a/net/netfilter/ipvs/ip_vs_lc.c +++ b/net/netfilter/ipvs/ip_vs_lc.c @@ -31,7 +31,7 @@ ip_vs_lc_schedule(struct ip_vs_service *svc, const struct sk_buff *skb, /* * Simply select the server with the least number of - * (activeconns<<5) + inactconns + * (activeconns*256) + totalconns * Except whose weight is equal to zero. * If the weight is equal to zero, it means that the server is * quiesced, the existing connections to the server still get @@ -57,7 +57,7 @@ ip_vs_lc_schedule(struct ip_vs_service *svc, const struct sk_buff *skb, IP_VS_DBG_ADDR(least->af, &least->addr), ntohs(least->port), atomic_read(&least->activeconns), - atomic_read(&least->inactconns)); + ip_vs_dest_inactconns(least)); return least; } diff --git a/net/netfilter/ipvs/ip_vs_proto_sctp.c b/net/netfilter/ipvs/ip_vs_proto_sctp.c index 3dbd3096e163..fb8af6b15a39 100644 --- a/net/netfilter/ipvs/ip_vs_proto_sctp.c +++ b/net/netfilter/ipvs/ip_vs_proto_sctp.c @@ -193,7 +193,7 @@ sctp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, struct sctphdr *sh; __le32 cmp, val; - if (!ip_vs_checksum_needed(skb, af)) + if (!ip_vs_checksum_needed(skb)) return 1; sh = (struct sctphdr *)(skb->data + sctphoff); cmp = sh->checksum; @@ -446,12 +446,10 @@ set_sctp_state(struct ip_vs_proto_data *pd, struct ip_vs_conn *cp, if (!(cp->flags & IP_VS_CONN_F_INACTIVE) && (next_state != IP_VS_SCTP_S_ESTABLISHED)) { atomic_dec(&dest->activeconns); - atomic_inc(&dest->inactconns); cp->flags |= IP_VS_CONN_F_INACTIVE; } else if ((cp->flags & IP_VS_CONN_F_INACTIVE) && (next_state == IP_VS_SCTP_S_ESTABLISHED)) { atomic_inc(&dest->activeconns); - atomic_dec(&dest->inactconns); cp->flags &= ~IP_VS_CONN_F_INACTIVE; } } diff --git a/net/netfilter/ipvs/ip_vs_proto_tcp.c b/net/netfilter/ipvs/ip_vs_proto_tcp.c index 1ac9c233537d..944efd34290c 100644 --- a/net/netfilter/ipvs/ip_vs_proto_tcp.c +++ b/net/netfilter/ipvs/ip_vs_proto_tcp.c @@ -527,12 +527,10 @@ set_tcp_state(struct ip_vs_proto_data *pd, struct ip_vs_conn *cp, if (!(cp->flags & IP_VS_CONN_F_INACTIVE) && !tcp_state_active(new_state)) { atomic_dec(&dest->activeconns); - atomic_inc(&dest->inactconns); cp->flags |= IP_VS_CONN_F_INACTIVE; } else if ((cp->flags & IP_VS_CONN_F_INACTIVE) && tcp_state_active(new_state)) { atomic_inc(&dest->activeconns); - atomic_dec(&dest->inactconns); cp->flags &= ~IP_VS_CONN_F_INACTIVE; } } diff --git a/net/netfilter/ipvs/ip_vs_sync.c b/net/netfilter/ipvs/ip_vs_sync.c index 4174076c66fa..515346bf915a 100644 --- a/net/netfilter/ipvs/ip_vs_sync.c +++ b/net/netfilter/ipvs/ip_vs_sync.c @@ -879,13 +879,10 @@ static void ip_vs_proc_conn(struct netns_ipvs *ipvs, struct ip_vs_conn_param *pa spin_lock_bh(&cp->lock); if ((cp->flags ^ flags) & IP_VS_CONN_F_INACTIVE && !(flags & IP_VS_CONN_F_TEMPLATE) && dest) { - if (flags & IP_VS_CONN_F_INACTIVE) { + if (flags & IP_VS_CONN_F_INACTIVE) atomic_dec(&dest->activeconns); - atomic_inc(&dest->inactconns); - } else { + else atomic_inc(&dest->activeconns); - atomic_dec(&dest->inactconns); - } } flags &= IP_VS_CONN_F_BACKUP_UPD_MASK; flags |= cp->flags & ~IP_VS_CONN_F_BACKUP_UPD_MASK; diff --git a/net/netfilter/ipvs/ip_vs_xmit.c b/net/netfilter/ipvs/ip_vs_xmit.c index c214e5d05524..5b37e92df02c 100644 --- a/net/netfilter/ipvs/ip_vs_xmit.c +++ b/net/netfilter/ipvs/ip_vs_xmit.c @@ -1551,7 +1551,7 @@ ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, if (skb_cow(skb, rt->dst.dev->hard_header_len)) goto tx_error; - ip_vs_nat_icmp(skb, pp, cp, 0, toff, has_ports); + ip_vs_nat_icmp(skb, pp, cp, 0, toff, has_ports, ciph); /* Another hack: avoid icmp_send in ip_fragment */ skb->ignore_df = 1; diff --git a/net/netfilter/nf_conntrack_proto.c b/net/netfilter/nf_conntrack_proto.c index c928ff63b10e..aaf97483a037 100644 --- a/net/netfilter/nf_conntrack_proto.c +++ b/net/netfilter/nf_conntrack_proto.c @@ -79,6 +79,12 @@ void nf_ct_l4proto_log_invalid(const struct sk_buff *skb, struct net *net; va_list args; + /* nfnetlink_log may re-enter conntrack attribute dumping and try to + * take ct->lock again via helpers such as tcp_to_nlattr(), so invalid + * conntrack logs must only be emitted after dropping ct->lock. + */ + lockdep_assert_not_held(&ct->lock); + net = nf_ct_net(ct); if (likely(net->ct.sysctl_log_invalid == 0)) return; diff --git a/net/netfilter/nf_conntrack_proto_sctp.c b/net/netfilter/nf_conntrack_proto_sctp.c index 0dd55d3fba38..ef51f0babf94 100644 --- a/net/netfilter/nf_conntrack_proto_sctp.c +++ b/net/netfilter/nf_conntrack_proto_sctp.c @@ -341,10 +341,12 @@ int nf_conntrack_sctp_packet(struct nf_conn *ct, struct sctphdr _sctph; const struct sctp_chunkhdr *sch; struct sctp_chunkhdr _sch; + bool log_invalid = false; u_int32_t offset, count; unsigned int *timeouts; unsigned long map[256 / sizeof(unsigned long)] = { 0 }; bool ignore = false; + u8 invalid_type = 0; if (sctp_error(skb, dataoff, state)) return -NF_ACCEPT; @@ -456,10 +458,8 @@ int nf_conntrack_sctp_packet(struct nf_conn *ct, /* Invalid */ if (new_state == SCTP_CONNTRACK_MAX) { - nf_ct_l4proto_log_invalid(skb, ct, state, - "Invalid, old_state %d, dir %d, type %d", - old_state, dir, sch->type); - + log_invalid = true; + invalid_type = sch->type; goto out_unlock; } @@ -534,6 +534,10 @@ int nf_conntrack_sctp_packet(struct nf_conn *ct, out_unlock: spin_unlock_bh(&ct->lock); + if (log_invalid) + nf_ct_l4proto_log_invalid(skb, ct, state, + "Invalid, old_state %d, dir %d, type %d", + old_state, dir, invalid_type); out: return -NF_ACCEPT; } diff --git a/net/netfilter/nf_conntrack_proto_tcp.c b/net/netfilter/nf_conntrack_proto_tcp.c index 228005a66d52..2d39421dcf9d 100644 --- a/net/netfilter/nf_conntrack_proto_tcp.c +++ b/net/netfilter/nf_conntrack_proto_tcp.c @@ -480,37 +480,81 @@ static void tcp_init_sender(struct ip_ct_tcp_state *sender, } } -__printf(6, 7) -static enum nf_ct_tcp_action nf_tcp_log_invalid(const struct sk_buff *skb, - const struct nf_conn *ct, - const struct nf_hook_state *state, - const struct ip_ct_tcp_state *sender, - enum nf_ct_tcp_action ret, - const char *fmt, ...) +enum nf_tcp_invalid_log_type { + NF_TCP_LOG_NONE, + NF_TCP_LOG_OVERSHOT, + NF_TCP_LOG_SEQ_OVER, + NF_TCP_LOG_ACK_OVER, + NF_TCP_LOG_SEQ_UNDER, + NF_TCP_LOG_ACK_UNDER, +}; + +struct nf_tcp_invalid_log { + enum nf_tcp_invalid_log_type type; + u32 value; +}; + +static enum nf_ct_tcp_action +nf_tcp_store_invalid(const struct nf_conn *ct, + const struct ip_ct_tcp_state *sender, + struct nf_tcp_invalid_log *log, + enum nf_ct_tcp_action ret, + enum nf_tcp_invalid_log_type type, + u32 value) { const struct nf_tcp_net *tn = nf_tcp_pernet(nf_ct_net(ct)); - struct va_format vaf; - va_list args; bool be_liberal; be_liberal = sender->flags & IP_CT_TCP_FLAG_BE_LIBERAL || tn->tcp_be_liberal; if (be_liberal) return NFCT_TCP_ACCEPT; - va_start(args, fmt); - vaf.fmt = fmt; - vaf.va = &args; - nf_ct_l4proto_log_invalid(skb, ct, state, "%pV", &vaf); - va_end(args); - + log->type = type; + log->value = value; return ret; } +static void nf_tcp_log_invalid(const struct sk_buff *skb, + const struct nf_conn *ct, + const struct nf_hook_state *state, + const struct nf_tcp_invalid_log *log) +{ + switch (log->type) { + case NF_TCP_LOG_OVERSHOT: + nf_ct_l4proto_log_invalid(skb, ct, state, + "%u bytes more than expected", + log->value); + break; + case NF_TCP_LOG_SEQ_OVER: + nf_ct_l4proto_log_invalid(skb, ct, state, + "SEQ is over upper bound %u (over the window of the receiver)", + log->value); + break; + case NF_TCP_LOG_ACK_OVER: + nf_ct_l4proto_log_invalid(skb, ct, state, + "ACK is over upper bound %u (ACKed data not seen yet)", + log->value); + break; + case NF_TCP_LOG_SEQ_UNDER: + nf_ct_l4proto_log_invalid(skb, ct, state, + "SEQ is under lower bound %u (already ACKed data retransmitted)", + log->value); + break; + case NF_TCP_LOG_ACK_UNDER: + nf_ct_l4proto_log_invalid(skb, ct, state, + "ignored ACK under lower bound %u (possible overly delayed)", + log->value); + break; + case NF_TCP_LOG_NONE: + break; + } +} + static enum nf_ct_tcp_action tcp_in_window(struct nf_conn *ct, enum ip_conntrack_dir dir, unsigned int index, const struct sk_buff *skb, unsigned int dataoff, const struct tcphdr *tcph, - const struct nf_hook_state *hook_state) + struct nf_tcp_invalid_log *log) { struct ip_ct_tcp *state = &ct->proto.tcp; struct ip_ct_tcp_state *sender = &state->seen[dir]; @@ -640,31 +684,29 @@ tcp_in_window(struct nf_conn *ct, enum ip_conntrack_dir dir, sender->td_end = end; sender->flags |= IP_CT_TCP_FLAG_DATA_UNACKNOWLEDGED; - return nf_tcp_log_invalid(skb, ct, hook_state, sender, NFCT_TCP_IGNORE, - "%u bytes more than expected", overshot); + return nf_tcp_store_invalid(ct, sender, log, NFCT_TCP_IGNORE, + NF_TCP_LOG_OVERSHOT, overshot); } - return nf_tcp_log_invalid(skb, ct, hook_state, sender, NFCT_TCP_INVALID, - "SEQ is over upper bound %u (over the window of the receiver)", - sender->td_maxend + 1); + return nf_tcp_store_invalid(ct, sender, log, NFCT_TCP_INVALID, + NF_TCP_LOG_SEQ_OVER, sender->td_maxend + 1); } if (!before(sack, receiver->td_end + 1)) - return nf_tcp_log_invalid(skb, ct, hook_state, sender, NFCT_TCP_INVALID, - "ACK is over upper bound %u (ACKed data not seen yet)", - receiver->td_end + 1); + return nf_tcp_store_invalid(ct, sender, log, NFCT_TCP_INVALID, + NF_TCP_LOG_ACK_OVER, receiver->td_end + 1); /* Is the ending sequence in the receive window (if available)? */ in_recv_win = !receiver->td_maxwin || after(end, sender->td_end - receiver->td_maxwin - 1); if (!in_recv_win) - return nf_tcp_log_invalid(skb, ct, hook_state, sender, NFCT_TCP_IGNORE, - "SEQ is under lower bound %u (already ACKed data retransmitted)", - sender->td_end - receiver->td_maxwin - 1); + return nf_tcp_store_invalid(ct, sender, log, NFCT_TCP_IGNORE, + NF_TCP_LOG_SEQ_UNDER, + sender->td_end - receiver->td_maxwin - 1); if (!after(sack, receiver->td_end - MAXACKWINDOW(sender) - 1)) - return nf_tcp_log_invalid(skb, ct, hook_state, sender, NFCT_TCP_IGNORE, - "ignored ACK under lower bound %u (possible overly delayed)", - receiver->td_end - MAXACKWINDOW(sender) - 1); + return nf_tcp_store_invalid(ct, sender, log, NFCT_TCP_IGNORE, + NF_TCP_LOG_ACK_UNDER, + receiver->td_end - MAXACKWINDOW(sender) - 1); /* Take into account window scaling (RFC 1323). */ if (!tcph->syn) @@ -719,11 +761,8 @@ tcp_in_window(struct nf_conn *ct, enum ip_conntrack_dir dir, return NFCT_TCP_ACCEPT; } -static void __cold nf_tcp_handle_invalid(struct nf_conn *ct, - enum ip_conntrack_dir dir, - int index, - const struct sk_buff *skb, - const struct nf_hook_state *hook_state) +static bool __cold +nf_tcp_handle_invalid(struct nf_conn *ct, enum ip_conntrack_dir dir, int index) { const unsigned int *timeouts; const struct nf_tcp_net *tn; @@ -732,7 +771,7 @@ static void __cold nf_tcp_handle_invalid(struct nf_conn *ct, if (!test_bit(IPS_ASSURED_BIT, &ct->status) || test_bit(IPS_FIXED_TIMEOUT_BIT, &ct->status)) - return; + return false; /* We don't want to have connections hanging around in ESTABLISHED * state for long time 'just because' conntrack deemed a FIN/RST @@ -747,7 +786,7 @@ static void __cold nf_tcp_handle_invalid(struct nf_conn *ct, case TCP_FIN_SET: break; default: - return; + return false; } if (ct->proto.tcp.last_dir != dir && @@ -755,7 +794,7 @@ static void __cold nf_tcp_handle_invalid(struct nf_conn *ct, ct->proto.tcp.last_index == TCP_RST_SET)) { expires = nf_ct_expires(ct); if (expires < 120 * HZ) - return; + return false; tn = nf_tcp_pernet(nf_ct_net(ct)); timeouts = nf_ct_timeout_lookup(ct); @@ -764,16 +803,15 @@ static void __cold nf_tcp_handle_invalid(struct nf_conn *ct, timeout = READ_ONCE(timeouts[TCP_CONNTRACK_UNACK]); if (expires > timeout) { - nf_ct_l4proto_log_invalid(skb, ct, hook_state, - "packet (index %d, dir %d) response for index %d lower timeout to %u", - index, dir, ct->proto.tcp.last_index, timeout); - WRITE_ONCE(ct->timeout, timeout + nfct_time_stamp); + return true; } } else { ct->proto.tcp.last_index = index; ct->proto.tcp.last_dir = dir; } + + return false; } /* table of valid flag combinations - PUSH, ECE and CWR are always valid */ @@ -968,7 +1006,9 @@ int nf_conntrack_tcp_packet(struct nf_conn *ct, struct net *net = nf_ct_net(ct); struct nf_tcp_net *tn = nf_tcp_pernet(net); enum tcp_conntrack new_state, old_state; + struct nf_tcp_invalid_log log = {}; unsigned int index, *timeouts; + bool lowered_timeout = false; enum nf_ct_tcp_action res; enum ip_conntrack_dir dir; const struct tcphdr *th; @@ -1251,14 +1291,18 @@ int nf_conntrack_tcp_packet(struct nf_conn *ct, } res = tcp_in_window(ct, dir, index, - skb, dataoff, th, state); + skb, dataoff, th, &log); switch (res) { case NFCT_TCP_IGNORE: spin_unlock_bh(&ct->lock); + nf_tcp_log_invalid(skb, ct, state, &log); return NF_ACCEPT; case NFCT_TCP_INVALID: - nf_tcp_handle_invalid(ct, dir, index, skb, state); + lowered_timeout = nf_tcp_handle_invalid(ct, dir, index); spin_unlock_bh(&ct->lock); + nf_tcp_log_invalid(skb, ct, state, &log); + if (lowered_timeout) + nf_ct_l4proto_log_invalid(skb, ct, state, "lowered timeout to UNACK"); return -NF_ACCEPT; case NFCT_TCP_ACCEPT: break; diff --git a/net/openvswitch/datapath.c b/net/openvswitch/datapath.c index fb5b72700d82..5706f2393fb7 100644 --- a/net/openvswitch/datapath.c +++ b/net/openvswitch/datapath.c @@ -1084,9 +1084,8 @@ static int ovs_flow_cmd_new(struct sk_buff *skb, struct genl_info *info) error = -EEXIST; goto err_unlock_ovs; } - /* The flow identifier has to be the same for flow updates. - * Look for any overlapping flow. - */ + + /* Look for any overlapping flow. */ if (unlikely(!ovs_flow_cmp(flow, &match))) { if (ovs_identifier_is_key(&flow->id)) flow = ovs_flow_tbl_lookup_exact(&dp->table, @@ -1098,6 +1097,30 @@ static int ovs_flow_cmd_new(struct sk_buff *skb, struct genl_info *info) goto err_unlock_ovs; } } + + if (unlikely(reply)) { + size_t cur, req; + + cur = ovs_flow_cmd_msg_size(acts, &new_flow->id, + ufid_flags); + req = ovs_flow_cmd_msg_size(acts, &flow->id, + ufid_flags); + if (cur < req) { + struct sk_buff *resized; + + resized = ovs_flow_cmd_alloc_info(acts, + &flow->id, + info, false, + ufid_flags); + if (IS_ERR(resized)) { + error = PTR_ERR(resized); + goto err_unlock_ovs; + } + kfree_skb(reply); + reply = resized; + } + } + /* Update actions. */ old_acts = ovsl_dereference(flow->sf_acts); rcu_assign_pointer(flow->sf_acts, acts); diff --git a/net/openvswitch/flow.c b/net/openvswitch/flow.c index b80bd3a90773..52e261ce91e8 100644 --- a/net/openvswitch/flow.c +++ b/net/openvswitch/flow.c @@ -893,8 +893,6 @@ static int key_extract_l3l4(struct sk_buff *skb, struct sw_flow_key *key) * Ethernet header * @key: output flow key * - * The caller must ensure that skb->len >= ETH_HLEN. - * * Initializes @skb header fields as follows: * * - skb->mac_header: the L2 header. @@ -914,8 +912,6 @@ static int key_extract_l3l4(struct sk_buff *skb, struct sw_flow_key *key) */ static int key_extract(struct sk_buff *skb, struct sw_flow_key *key) { - struct ethhdr *eth; - /* Flags are always used as part of stats */ key->tp.flags = 0; @@ -930,6 +926,13 @@ static int key_extract(struct sk_buff *skb, struct sw_flow_key *key) skb_reset_network_header(skb); key->eth.type = skb->protocol; } else { + struct ethhdr *eth; + int err; + + err = check_header(skb, ETH_HLEN); + if (unlikely(err)) + return err; + eth = eth_hdr(skb); ether_addr_copy(key->eth.src, eth->h_source); ether_addr_copy(key->eth.dst, eth->h_dest); diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c index d435a48bc0e3..493a167a7c85 100644 --- a/net/packet/af_packet.c +++ b/net/packet/af_packet.c @@ -1367,13 +1367,25 @@ static int packet_rcv_has_room(struct packet_sock *po, struct sk_buff *skb) return ret; } -static void packet_rcv_try_clear_pressure(struct packet_sock *po) +static void __packet_rcv_try_clear_pressure(struct packet_sock *po) { if (packet_sock_flag(po, PACKET_SOCK_PRESSURE) && __packet_rcv_has_room(po, NULL) == ROOM_NORMAL) packet_sock_flag_set(po, PACKET_SOCK_PRESSURE, false); } +static void packet_rcv_try_clear_pressure(struct packet_sock *po) +{ + struct sock *sk = &po->sk; + + if (!packet_sock_flag(po, PACKET_SOCK_PRESSURE)) + return; + + spin_lock_bh(&sk->sk_receive_queue.lock); + __packet_rcv_try_clear_pressure(po); + spin_unlock_bh(&sk->sk_receive_queue.lock); +} + static void packet_sock_destruct(struct sock *sk) { skb_queue_purge(&sk->sk_error_queue); @@ -1976,11 +1988,12 @@ static void packet_parse_headers(struct sk_buff *skb, struct socket *sock) { int depth; + /* On TX skb->data is the L2 header; anchor it for all socket types. */ + skb_reset_mac_header(skb); + if ((!skb->protocol || skb->protocol == htons(ETH_P_ALL)) && - sock->type == SOCK_RAW) { - skb_reset_mac_header(skb); + sock->type == SOCK_RAW) skb->protocol = dev_parse_header_protocol(skb); - } /* Move network header to the right position for VLAN tagged packets */ if (likely(skb->dev->type == ARPHRD_ETHER) && @@ -2005,8 +2018,9 @@ static int packet_sendmsg_spkt(struct socket *sock, struct msghdr *msg, struct net_device *dev; struct sockcm_cookie sockc; __be16 proto = 0; - int err; + int hard_header_len; int extra_len = 0; + int err; /* * Get and verify the address. @@ -2049,14 +2063,18 @@ static int packet_sendmsg_spkt(struct socket *sock, struct msghdr *msg, extra_len = 4; /* We're doing our own CRC */ } + /* Keep the allocation-time header length across retry. */ + if (!skb) + hard_header_len = READ_ONCE(dev->hard_header_len); + err = -EMSGSIZE; - if (len > dev->mtu + dev->hard_header_len + VLAN_HLEN + extra_len) + if (len > dev->mtu + hard_header_len + VLAN_HLEN + extra_len) goto out_unlock; if (!skb) { - size_t reserved = LL_RESERVED_SPACE(dev); + size_t reserved = LL_RESERVED_SPACE_EX(dev, hard_header_len); int tlen = dev->needed_tailroom; - unsigned int hhlen = dev->header_ops ? dev->hard_header_len : 0; + unsigned int hhlen = dev->header_ops ? hard_header_len : 0; rcu_read_unlock(); skb = sock_wmalloc(sk, len + reserved + tlen, 0, GFP_KERNEL); @@ -2086,7 +2104,7 @@ static int packet_sendmsg_spkt(struct socket *sock, struct msghdr *msg, err = -EINVAL; goto out_unlock; } - if (len > (dev->mtu + dev->hard_header_len + extra_len) && + if (len > (dev->mtu + hard_header_len + extra_len) && !packet_extra_vlan_len_allowed(dev, skb)) { err = -EMSGSIZE; goto out_unlock; @@ -2628,6 +2646,7 @@ static int packet_snd_vnet_parse(struct msghdr *msg, size_t *len, static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb, void *frame, struct net_device *dev, void *data, int tp_len, __be16 proto, unsigned char *addr, int hlen, int copylen, + int hard_header_len, const struct sockcm_cookie *sockc) { union tpacket_uhdr ph; @@ -2659,8 +2678,8 @@ static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb, } else if (copylen) { int hdrlen = min_t(int, copylen, tp_len); - skb_push(skb, dev->hard_header_len); - skb_put(skb, copylen - dev->hard_header_len); + skb_push(skb, hard_header_len); + skb_put(skb, copylen - hard_header_len); err = skb_store_bits(skb, 0, data, hdrlen); if (unlikely(err)) return err; @@ -2791,7 +2810,7 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg) void *data; int len_sum = 0; int status = TP_STATUS_AVAILABLE; - int hlen, tlen, copylen = 0; + int hard_header_len, hlen, tlen, copylen = 0; long timeo; mutex_lock(&po->pg_vec_lock); @@ -2838,8 +2857,9 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg) goto out_put; } + hard_header_len = READ_ONCE(dev->hard_header_len); if (po->sk.sk_socket->type == SOCK_RAW) - reserve = dev->hard_header_len; + reserve = hard_header_len; size_max = po->tx_ring.frame_size - (po->tp_hdrlen - sizeof(struct sockaddr_ll)); @@ -2876,7 +2896,7 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg) goto tpacket_error; status = TP_STATUS_SEND_REQUEST; - hlen = LL_RESERVED_SPACE(dev); + hlen = LL_RESERVED_SPACE_EX(dev, hard_header_len); tlen = dev->needed_tailroom; if (vnet_hdr_sz) { data += vnet_hdr_sz; @@ -2894,10 +2914,10 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg) vnet_hdr.hdr_len); has_vnet_hdr = true; } - copylen = max_t(int, copylen, dev->hard_header_len); + copylen = max_t(int, copylen, hard_header_len); skb = sock_alloc_send_skb(&po->sk, hlen + tlen + sizeof(struct sockaddr_ll) + - (copylen - dev->hard_header_len), + (copylen - hard_header_len), !need_wait, &err); if (unlikely(skb == NULL)) { @@ -2907,7 +2927,8 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg) goto out_status; } tp_len = tpacket_fill_skb(po, skb, ph, dev, data, tp_len, proto, - addr, hlen, copylen, &sockc); + addr, hlen, copylen, hard_header_len, + &sockc); if (likely(tp_len >= 0) && tp_len > dev->mtu + reserve && !vnet_hdr_sz && @@ -3015,7 +3036,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) int offset = 0; struct packet_sock *po = pkt_sk(sk); int vnet_hdr_sz = READ_ONCE(po->vnet_hdr_sz); - int hlen, tlen, linear; + int hard_header_len, hlen, tlen, linear; int extra_len = 0; /* @@ -3056,8 +3077,9 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) goto out_unlock; } + hard_header_len = READ_ONCE(dev->hard_header_len); if (sock->type == SOCK_RAW) - reserve = dev->hard_header_len; + reserve = hard_header_len; if (vnet_hdr_sz) { err = packet_snd_vnet_parse(msg, &len, &vnet_hdr, vnet_hdr_sz); if (err) @@ -3078,10 +3100,10 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) goto out_unlock; err = -ENOBUFS; - hlen = LL_RESERVED_SPACE(dev); + hlen = LL_RESERVED_SPACE_EX(dev, hard_header_len); tlen = dev->needed_tailroom; linear = __virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len); - linear = max(linear, min_t(int, len, dev->hard_header_len)); + linear = max(linear, min_t(int, len, hard_header_len)); skb = packet_alloc_skb(sk, hlen + tlen, hlen, len, linear, msg->msg_flags & MSG_DONTWAIT, &err); if (skb == NULL) @@ -3097,7 +3119,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) } else if (reserve) { skb_reserve(skb, -reserve); if (len < reserve + sizeof(struct ipv6hdr) && - dev->min_header_len != dev->hard_header_len) + dev->min_header_len != hard_header_len) skb_reset_network_header(skb); } @@ -4351,7 +4373,7 @@ static __poll_t packet_poll(struct file *file, struct socket *sock, TP_STATUS_KERNEL)) mask |= EPOLLIN | EPOLLRDNORM; } - packet_rcv_try_clear_pressure(po); + __packet_rcv_try_clear_pressure(po); spin_unlock_bh(&sk->sk_receive_queue.lock); spin_lock_bh(&sk->sk_write_queue.lock); if (po->tx_ring.pg_vec) { @@ -4591,14 +4613,14 @@ static int packet_set_ring(struct sock *sk, union tpacket_req_u *req_u, rb->frame_max = (req->tp_frame_nr - 1); rb->head = 0; rb->frame_size = req->tp_frame_size; + po->prot_hook.func = (po->rx_ring.pg_vec) ? + tpacket_rcv : packet_rcv; spin_unlock_bh(&rb_queue->lock); swap(rb->pg_vec_order, order); swap(rb->pg_vec_len, req->tp_block_nr); rb->pg_vec_pages = req->tp_block_size/PAGE_SIZE; - po->prot_hook.func = (po->rx_ring.pg_vec) ? - tpacket_rcv : packet_rcv; skb_queue_purge(rb_queue); if (atomic_long_read(&po->mapped)) pr_err("packet_mmap: vma is busy: %ld\n", diff --git a/net/sched/act_ct.c b/net/sched/act_ct.c index 287e25593412..7a0dff24c5d1 100644 --- a/net/sched/act_ct.c +++ b/net/sched/act_ct.c @@ -841,8 +841,15 @@ static int tcf_ct_ipv6_is_fragment(struct sk_buff *skb, bool *frag) return 0; } +/* On error, tells the caller whether it still owns @skb and must free it + * itself. @skb is ours only when the header checks below reject the packet + * before it is handed to the defragmentation engine; once nf_ct_handle_ + * fragments() has been called the skb is either queued (-EINPROGRESS) or has + * already been freed by it. + */ static int tcf_ct_handle_fragments(struct net *net, struct sk_buff *skb, - u8 family, u16 zone, bool *defrag) + u8 family, u16 zone, bool *defrag, + bool *skb_is_ours) { enum ip_conntrack_info ctinfo; struct tc_skb_cb cb; @@ -860,8 +867,12 @@ static int tcf_ct_handle_fragments(struct net *net, struct sk_buff *skb, err = tcf_ct_ipv4_is_fragment(skb, &frag); else err = tcf_ct_ipv6_is_fragment(skb, &frag); - if (err || !frag) + if (err) { + *skb_is_ours = true; return err; + } + if (!frag) + return 0; cb = *tc_skb_cb(skb); err = nf_ct_handle_fragments(net, skb, zone, family, &proto, &cb.mru); @@ -971,6 +982,7 @@ TC_INDIRECT_SCOPE int tcf_ct_act(struct sk_buff *skb, const struct tc_action *a, int nh_ofs, err, retval; struct tcf_ct_params *p; bool add_helper = false; + bool skb_is_ours = false; bool skip_add = false; bool defrag = false; struct nf_conn *ct; @@ -1006,9 +1018,18 @@ TC_INDIRECT_SCOPE int tcf_ct_act(struct sk_buff *skb, const struct tc_action *a, */ nh_ofs = skb_network_offset(skb); skb_pull_rcsum(skb, nh_ofs); - err = tcf_ct_handle_fragments(net, skb, family, p->zone, &defrag); - if (err) + err = tcf_ct_handle_fragments(net, skb, family, p->zone, &defrag, + &skb_is_ours); + if (err) { + /* The skb is still ours only when the header checks rejected + * it; returning TC_ACT_CONSUMED for such a packet would leak + * it, since no caller frees an skb it was told it no longer + * owns. + */ + if (skb_is_ours) + goto drop; goto out_frag; + } err = nf_ct_skb_network_trim(skb, family); if (err) diff --git a/net/sched/act_gact.c b/net/sched/act_gact.c index 904ab3d457ef..13de33761e06 100644 --- a/net/sched/act_gact.c +++ b/net/sched/act_gact.c @@ -89,6 +89,11 @@ static int tcf_gact_init(struct net *net, struct nlattr *nla, p_parm = nla_data(tb[TCA_GACT_PROB]); if (p_parm->ptype >= MAX_RAND) return -EINVAL; + if (!tcf_action_valid(p_parm->paction)) { + NL_SET_ERR_MSG(extack, + "invalid fallback control action"); + return -EINVAL; + } if (TC_ACT_EXT_CMP(p_parm->paction, TC_ACT_GOTO_CHAIN)) { NL_SET_ERR_MSG(extack, "goto chain not allowed on fallback"); diff --git a/net/sched/act_police.c b/net/sched/act_police.c index f3121c5a85e9..7fc57d7fe66d 100644 --- a/net/sched/act_police.c +++ b/net/sched/act_police.c @@ -128,6 +128,12 @@ static int tcf_police_init(struct net *net, struct nlattr *nla, if (tb[TCA_POLICE_RESULT]) { tcfp_result = nla_get_u32(tb[TCA_POLICE_RESULT]); + if (!tcf_action_valid(tcfp_result)) { + NL_SET_ERR_MSG(extack, + "invalid fallback control action"); + err = -EINVAL; + goto failure; + } if (TC_ACT_EXT_CMP(tcfp_result, TC_ACT_GOTO_CHAIN)) { NL_SET_ERR_MSG(extack, "goto chain not allowed on fallback"); diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c index e259ec6ed145..ff6af03cb855 100644 --- a/net/sched/cls_api.c +++ b/net/sched/cls_api.c @@ -390,6 +390,7 @@ static struct tcf_proto *tcf_proto_create(const char *kind, u32 protocol, tp->protocol = protocol; tp->prio = prio; tp->chain = chain; + tp->usesw = !tp->ops->reoffload; spin_lock_init(&tp->lock); refcount_set(&tp->refcnt, 1); @@ -410,12 +411,55 @@ static void tcf_proto_get(struct tcf_proto *tp) refcount_inc(&tp->refcnt); } +static void tcf_proto_count_usesw(struct tcf_proto *tp, bool add) +{ +#ifdef CONFIG_NET_CLS_ACT + struct tcf_block *block = tp->chain->block; + bool counted = false; + + if (!add) { + if (tp->usesw && tp->counted) { + if (!atomic_dec_return(&block->useswcnt)) + static_branch_dec(&tcf_sw_enabled_key); + tp->counted = false; + } + return; + } + + spin_lock(&tp->lock); + if (tp->usesw && !tp->counted) { + counted = true; + tp->counted = true; + } + spin_unlock(&tp->lock); + + if (counted && atomic_inc_return(&block->useswcnt) == 1) + static_branch_inc(&tcf_sw_enabled_key); +#endif +} + static void tcf_chain_put(struct tcf_chain *chain); static void tcf_proto_destroy(struct tcf_proto *tp, bool rtnl_held, bool sig_destroy, struct netlink_ext_ack *extack) { - tp->ops->destroy(tp, rtnl_held, extack); + /* A locked classifier's destroy callback (e.g. u32_destroy) uses + * rtnl_dereference() and mutates shared structures (e.g. the + * tc_u_common hash list) that are only safe under rtnl_lock. When an + * unlocked classifier's request (e.g. flower on ingress) loses the + * tcf_chain_tp_insert_unique() race and ends up dropping the last + * reference on a locked classifier's proto, destroy() would run + * without rtnl held. Take it here in that case. + */ + bool not_lockless = !rtnl_held && + !(tp->ops->flags & TCF_PROTO_OPS_DOIT_UNLOCKED); + + if (not_lockless) + rtnl_lock(); + tp->ops->destroy(tp, rtnl_held || not_lockless, extack); + if (not_lockless) + rtnl_unlock(); + tcf_proto_count_usesw(tp, false); if (sig_destroy) tcf_proto_signal_destroyed(tp->chain, tp); tcf_chain_put(tp->chain); @@ -2357,6 +2401,7 @@ static int tc_new_tfilter(struct sk_buff *skb, struct nlmsghdr *n, tfilter_notify(net, skb, n, tp, block, q, parent, fh, RTM_NEWTFILTER, false, rtnl_held, extack); tfilter_put(tp, fh); + tcf_proto_count_usesw(tp, true); /* q pointer is NULL for shared blocks */ if (q) q->flags &= ~TCQ_F_CAN_BYPASS; diff --git a/net/sched/cls_bpf.c b/net/sched/cls_bpf.c index d5a5dffcd6f9..cede21257d27 100644 --- a/net/sched/cls_bpf.c +++ b/net/sched/cls_bpf.c @@ -509,6 +509,8 @@ static int cls_bpf_change(struct net *net, struct sk_buff *in_skb, if (!tc_in_hw(prog->gen_flags)) prog->gen_flags |= TCA_CLS_FLAGS_NOT_IN_HW; + tcf_proto_update_usesw(tp, prog->gen_flags); + if (oldprog) { idr_replace(&head->handle_idr, prog, handle); list_replace_rcu(&oldprog->link, &prog->link); diff --git a/net/sched/cls_flower.c b/net/sched/cls_flower.c index b00e491e8130..4fc59af6696e 100644 --- a/net/sched/cls_flower.c +++ b/net/sched/cls_flower.c @@ -2374,6 +2374,8 @@ static int fl_change(struct net *net, struct sk_buff *in_skb, if (!tc_in_hw(fnew->flags)) fnew->flags |= TCA_CLS_FLAGS_NOT_IN_HW; + tcf_proto_update_usesw(tp, fnew->flags); + spin_lock(&tp->lock); /* tp was deleted concurrently. -EAGAIN will cause caller to lookup diff --git a/net/sched/cls_matchall.c b/net/sched/cls_matchall.c index c4ed11df6254..e8353b27c8c4 100644 --- a/net/sched/cls_matchall.c +++ b/net/sched/cls_matchall.c @@ -228,6 +228,8 @@ static int mall_change(struct net *net, struct sk_buff *in_skb, if (!tc_in_hw(new->flags)) new->flags |= TCA_CLS_FLAGS_NOT_IN_HW; + tcf_proto_update_usesw(tp, new->flags); + *arg = head; rcu_assign_pointer(tp->root, new); return 0; diff --git a/net/sched/cls_route.c b/net/sched/cls_route.c index 1e20bbd687f1..fd72ab9d19ea 100644 --- a/net/sched/cls_route.c +++ b/net/sched/cls_route.c @@ -52,6 +52,7 @@ struct route4_filter { struct tcf_result res; struct tcf_exts exts; u32 handle; + bool dying; struct route4_bucket *bkt; struct tcf_proto *tp; struct rcu_work rwork; @@ -66,9 +67,11 @@ static inline int route4_fastmap_hash(u32 id, int iif) static DEFINE_SPINLOCK(fastmap_lock); static void -route4_reset_fastmap(struct route4_head *head) +route4_reset_fastmap(struct route4_head *head, struct route4_filter *f) { spin_lock_bh(&fastmap_lock); + if (f) + f->dying = true; memset(head->fastmap, 0, sizeof(head->fastmap)); spin_unlock_bh(&fastmap_lock); } @@ -81,9 +84,11 @@ route4_set_fastmap(struct route4_head *head, u32 id, int iif, /* fastmap updates must look atomic to aling id, iff, filter */ spin_lock_bh(&fastmap_lock); - head->fastmap[h].id = id; - head->fastmap[h].iif = iif; - head->fastmap[h].filter = f; + if (f == ROUTE4_FAILURE || !f->dying) { + head->fastmap[h].id = id; + head->fastmap[h].iif = iif; + head->fastmap[h].filter = f; + } spin_unlock_bh(&fastmap_lock); } @@ -297,6 +302,13 @@ static void route4_destroy(struct tcf_proto *tp, bool rtnl_held, next = rtnl_dereference(f->next); RCU_INIT_POINTER(b->ht[h2], next); tcf_unbind_filter(tp, &f->res); + /* Mark the filter dying under fastmap_lock so + * any in-flight reader that still holds it + * will skip the republish in route4_set_fastmap(). + */ + spin_lock_bh(&fastmap_lock); + f->dying = true; + spin_unlock_bh(&fastmap_lock); if (tcf_exts_get_net(&f->exts)) route4_queue_work(f); else @@ -307,6 +319,11 @@ static void route4_destroy(struct tcf_proto *tp, bool rtnl_held, kfree_rcu(b, rcu); } } + + /* All filters are unlinked and marked dying, so no in-flight + * reader can republish a stale entry after this reset. + */ + route4_reset_fastmap(head, NULL); kfree_rcu(head, rcu); } @@ -334,11 +351,11 @@ static int route4_delete(struct tcf_proto *tp, void *arg, bool *last, /* unlink it */ RCU_INIT_POINTER(*fp, rtnl_dereference(f->next)); - /* Remove any fastmap lookups that might ref filter - * notice we unlink'd the filter so we can't get it - * back in the fastmap. + /* Clear any fastmap entries that may ref this filter and + * mark it dying so in-flight readers can't republish it + * after the reset. */ - route4_reset_fastmap(head); + route4_reset_fastmap(head, f); /* Delete it */ tcf_unbind_filter(tp, &f->res); @@ -553,7 +570,7 @@ static int route4_change(struct net *net, struct sk_buff *in_skb, } } - route4_reset_fastmap(head); + route4_reset_fastmap(head, fold); *arg = f; if (fold) { tcf_unbind_filter(tp, &fold->res); diff --git a/net/sched/cls_u32.c b/net/sched/cls_u32.c index 1338d9b4c03a..9829df127d05 100644 --- a/net/sched/cls_u32.c +++ b/net/sched/cls_u32.c @@ -950,6 +950,8 @@ static int u32_change(struct net *net, struct sk_buff *in_skb, if (!tc_in_hw(new->flags)) new->flags |= TCA_CLS_FLAGS_NOT_IN_HW; + tcf_proto_update_usesw(tp, new->flags); + u32_replace_knode(tp, tp_c, new); tcf_unbind_filter(tp, &n->res); tcf_exts_get_net(&n->exts); @@ -1163,6 +1165,8 @@ static int u32_change(struct net *net, struct sk_buff *in_skb, if (!tc_in_hw(n->flags)) n->flags |= TCA_CLS_FLAGS_NOT_IN_HW; + tcf_proto_update_usesw(tp, n->flags); + ins = &ht->ht[TC_U32_HASH(handle)]; for (pins = rtnl_dereference(*ins); pins; ins = &pins->next, pins = rtnl_dereference(*ins)) diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c index b20dc987b907..2047b9c1ae3f 100644 --- a/net/sched/sch_api.c +++ b/net/sched/sch_api.c @@ -1066,6 +1066,9 @@ static int qdisc_graft(struct net_device *dev, struct Qdisc *parent, unsigned int i, num_q, ingress; struct netdev_queue *dev_queue; + if (new) + new->depth = 0; + ingress = 0; num_q = dev->num_tx_queues; if ((q && q->flags & TCQ_F_INGRESS) || @@ -1163,9 +1166,15 @@ static int qdisc_graft(struct net_device *dev, struct Qdisc *parent, NL_SET_ERR_MSG(extack, "STAB not supported on a non root"); return -EINVAL; } + if (new && parent->depth >= 7) { + NL_SET_ERR_MSG(extack, "Qdisc hierarchy is too deep"); + return -E2BIG; + } err = cops->graft(parent, cl, new, &old, extack); if (err) return err; + if (new) + new->depth = parent->depth + 1; notify_and_destroy(net, skb, n, classid, old, new, extack); } return 0; diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c index ce9ee43e9ee2..4331097f813e 100644 --- a/net/sched/sch_cake.c +++ b/net/sched/sch_cake.c @@ -1282,7 +1282,6 @@ static struct sk_buff *cake_ack_filter(struct cake_sched_data *q, seglen = ntohs(ipv6h_check->payload_len); } else { - WARN_ON(1); /* shouldn't happen */ continue; } diff --git a/net/sctp/associola.c b/net/sctp/associola.c index 44f06c5f1f25..f409b9f6b228 100644 --- a/net/sctp/associola.c +++ b/net/sctp/associola.c @@ -543,6 +543,9 @@ void sctp_assoc_rm_peer(struct sctp_association *asoc, asoc->addip_last_asconf->transport == peer) asoc->addip_last_asconf->transport = NULL; + if (asoc->new_transport == peer) + asoc->new_transport = NULL; + /* If we have something on the transmitted list, we have to * save it off. The best place is the active path. */ @@ -573,6 +576,10 @@ void sctp_assoc_rm_peer(struct sctp_association *asoc, if (ch->transport == peer) ch->transport = NULL; + list_for_each_entry(ch, &asoc->outqueue.control_chunk_list, list) + if (ch->transport == peer) + ch->transport = NULL; + asoc->peer.transport_count--; sctp_ulpevent_notify_peer_addr_change(peer, SCTP_ADDR_REMOVED, 0); @@ -1727,6 +1734,8 @@ void sctp_asconf_queue_teardown(struct sctp_association *asoc) sctp_assoc_free_asconf_queue(asoc); /* Free any cached ASCONF chunk. */ - if (asoc->addip_last_asconf) + if (asoc->addip_last_asconf) { sctp_chunk_free(asoc->addip_last_asconf); + asoc->addip_last_asconf = NULL; + } } diff --git a/net/sctp/outqueue.c b/net/sctp/outqueue.c index 0dc6b8ab9963..242282a26425 100644 --- a/net/sctp/outqueue.c +++ b/net/sctp/outqueue.c @@ -650,6 +650,7 @@ static int __sctp_outq_flush_rtx(struct sctp_outq *q, struct sctp_packet *pkt, if (chunk->tsn_gap_acked) { list_move_tail(&chunk->transmitted_list, &transport->transmitted); + chunk->transport = transport; continue; } diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c index 0f03560fcab4..1a614560223e 100644 --- a/net/sctp/sm_make_chunk.c +++ b/net/sctp/sm_make_chunk.c @@ -3354,12 +3354,11 @@ struct sctp_chunk *sctp_process_asconf(struct sctp_association *asoc, goto done; } done: - asoc->peer.addip_serial++; - /* If we are sending a new ASCONF_ACK hold a reference to it in assoc * after freeing the reference to old asconf ack if any. */ if (asconf_ack) { + asoc->peer.addip_serial++; sctp_chunk_hold(asconf_ack); list_add_tail(&asconf_ack->transmitted_list, &asoc->asconf_ack_list); diff --git a/net/sctp/sm_statefuns.c b/net/sctp/sm_statefuns.c index ce0b5d6b4c52..0a44a581900a 100644 --- a/net/sctp/sm_statefuns.c +++ b/net/sctp/sm_statefuns.c @@ -6109,8 +6109,12 @@ enum sctp_disposition sctp_sf_t4_timer_expire( struct sctp_cmd_seq *commands) { struct sctp_chunk *chunk = asoc->addip_last_asconf; - struct sctp_transport *transport = chunk->transport; + struct sctp_transport *transport; + + if (!chunk) + return SCTP_DISPOSITION_CONSUME; + transport = chunk->transport; SCTP_INC_STATS(net, SCTP_MIB_T4_RTO_EXPIREDS); /* ADDIP 4.1 B1) Increment the error counters and perform path failure diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index e5e07160e571..087102ff3c64 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -1921,11 +1921,12 @@ static void smc_listen_out(struct smc_sock *new_smc) atomic_dec(&lsmc->queued_smc_hs); release_sock(newsmcsk); /* lock in smc_listen_work() */ + lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING); if (lsmc->sk.sk_state == SMC_LISTEN) { - lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING); smc_accept_enqueue(&lsmc->sk, newsmcsk); release_sock(&lsmc->sk); } else { /* no longer listening */ + release_sock(&lsmc->sk); smc_close_non_accepted(newsmcsk); } diff --git a/net/smc/smc_llc.c b/net/smc/smc_llc.c index 018ce8133b02..149d63cff667 100644 --- a/net/smc/smc_llc.c +++ b/net/smc/smc_llc.c @@ -1918,7 +1918,8 @@ static void smc_llc_event_handler(struct smc_llc_qentry *qentry) return; case SMC_LLC_CONFIRM_LINK: case SMC_LLC_ADD_LINK_CONT: - if (lgr->llc_flow_lcl.type != SMC_LLC_FLOW_NONE) { + if (lgr->llc_flow_lcl.type != SMC_LLC_FLOW_NONE && + !lgr->llc_flow_lcl.qentry) { /* a flow is waiting for this message */ smc_llc_flow_qentry_set(&lgr->llc_flow_lcl, qentry); wake_up(&lgr->llc_msg_waiter); diff --git a/net/smc/smc_rx.c b/net/smc/smc_rx.c index a48f8c93f4b6..05229a0a07e4 100644 --- a/net/smc/smc_rx.c +++ b/net/smc/smc_rx.c @@ -150,7 +150,12 @@ static const struct pipe_buf_operations smc_pipe_ops = { static void smc_rx_spd_release(struct splice_pipe_desc *spd, unsigned int i) { + struct smc_spd_priv *priv = (struct smc_spd_priv *)spd->partial[i].private; + struct sock *sk = &priv->smc->sk; + + kfree(priv); put_page(spd->pages[i]); + sock_put(sk); } static int smc_rx_splice(struct pipe_inode_info *pipe, char *src, size_t len, @@ -209,6 +214,10 @@ static int smc_rx_splice(struct pipe_inode_info *pipe, char *src, size_t len, offset = 0; } } + for (i = 0; i < nr_pages; i++) { + get_page(pages[i]); + sock_hold(&smc->sk); + } spd.nr_pages_max = nr_pages; spd.nr_pages = nr_pages; spd.pages = pages; @@ -217,16 +226,8 @@ static int smc_rx_splice(struct pipe_inode_info *pipe, char *src, size_t len, spd.spd_release = smc_rx_spd_release; bytes = splice_to_pipe(pipe, &spd); - if (bytes > 0) { - sock_hold(&smc->sk); - if (!lgr->is_smcd && smc->conn.rmb_desc->is_vm) { - for (i = 0; i < PAGE_ALIGN(bytes + offset) / PAGE_SIZE; i++) - get_page(pages[i]); - } else { - get_page(smc->conn.rmb_desc->pages); - } + if (bytes > 0) atomic_add(bytes, &smc->conn.splice_pending); - } kfree(priv); kfree(partial); kfree(pages); diff --git a/net/tipc/node.c b/net/tipc/node.c index 97b5ce4c678f..f4a25ae7dc1b 100644 --- a/net/tipc/node.c +++ b/net/tipc/node.c @@ -1063,18 +1063,23 @@ static void __tipc_node_link_down(struct tipc_node *n, int *bearer_id, static void tipc_node_link_down(struct tipc_node *n, int bearer_id, bool delete) { - struct tipc_link_entry *le = &n->links[bearer_id]; struct tipc_media_addr *maddr = NULL; - struct tipc_link *l = le->link; int old_bearer_id = bearer_id; + struct tipc_link_entry *le; struct sk_buff_head xmitq; - - if (!l) - return; + struct tipc_link *l; __skb_queue_head_init(&xmitq); + /* Synchronize the link lookup with bearer teardown. */ tipc_node_write_lock(n); + le = &n->links[bearer_id]; + l = le->link; + if (!l) { + tipc_node_write_unlock_fast(n); + return; + } + if (!tipc_link_is_establishing(l)) { __tipc_node_link_down(n, &bearer_id, &xmitq, &maddr); } else { diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index 39021fab3c59..ac168ccd1fcb 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -458,7 +458,7 @@ int tls_tx_records(struct sock *sk, int flags) } tx_err: - if (rc < 0 && rc != -EAGAIN) + if (rc < 0 && rc != -EAGAIN && rc != -EINTR && rc != -ERESTARTSYS) tls_err_abort(sk, rc); return rc; @@ -1116,6 +1116,14 @@ static int tls_sw_sendmsg_locked(struct sock *sk, struct msghdr *msg, if (!sk_stream_memory_free(sk)) goto wait_for_sndbuf; + /* open record may be full if we couldn't push it in the last sendmsg call */ + if (sk_msg_full(msg_pl)) { + full_record = true; + sk_msg_trim(sk, msg_en, + msg_pl->sg.size + prot->overhead_size); + goto copied; + } + alloc_encrypted: ret = tls_alloc_encrypted_msg(sk, required_size); if (ret) { @@ -1216,6 +1224,12 @@ static int tls_sw_sendmsg_locked(struct sock *sk, struct msghdr *msg, msg_pl, try_to_copy); if (ret < 0) goto trim_sgl; + + if (sk_msg_full(msg_pl)) { + full_record = true; + sk_msg_trim(sk, msg_en, + msg_pl->sg.size + prot->overhead_size); + } } /* Open records defined only if successfully copied, otherwise diff --git a/net/vmw_vsock/virtio_transport.c b/net/vmw_vsock/virtio_transport.c index e1d7ce8dac08..70db737c52da 100644 --- a/net/vmw_vsock/virtio_transport.c +++ b/net/vmw_vsock/virtio_transport.c @@ -257,12 +257,13 @@ static void virtio_transport_tx_work(struct work_struct *work) struct virtqueue *vq; bool added = false; - vq = vsock->vqs[VSOCK_VQ_TX]; mutex_lock(&vsock->tx_lock); if (!vsock->tx_run) goto out; + vq = vsock->vqs[VSOCK_VQ_TX]; + do { struct sk_buff *skb; unsigned int len; @@ -362,13 +363,13 @@ static void virtio_transport_event_work(struct work_struct *work) container_of(work, struct virtio_vsock, event_work); struct virtqueue *vq; - vq = vsock->vqs[VSOCK_VQ_EVENT]; - mutex_lock(&vsock->event_lock); if (!vsock->event_run) goto out; + vq = vsock->vqs[VSOCK_VQ_EVENT]; + do { struct virtio_vsock_event *event; unsigned int len; @@ -487,12 +488,12 @@ static void virtio_transport_rx_work(struct work_struct *work) container_of(work, struct virtio_vsock, rx_work); struct virtqueue *vq; - vq = vsock->vqs[VSOCK_VQ_RX]; - mutex_lock(&vsock->rx_lock); if (!vsock->rx_run) - goto out; + goto out_nofill; + + vq = vsock->vqs[VSOCK_VQ_RX]; do { virtqueue_disable_cb(vq); @@ -538,6 +539,7 @@ static void virtio_transport_rx_work(struct work_struct *work) out: if (vsock->rx_buf_nr < vsock->rx_buf_max_nr / 2) virtio_vsock_rx_fill(vsock); +out_nofill: mutex_unlock(&vsock->rx_lock); } diff --git a/scripts/remove-stale-files b/scripts/remove-stale-files index 8b1a636f8543..38eb84eb605b 100755 --- a/scripts/remove-stale-files +++ b/scripts/remove-stale-files @@ -39,3 +39,5 @@ rm -rf include/ksym find . -name '*.usyms' | xargs rm -f rm -f binkernel.spec + +rm -f lib/test_fortify.log diff --git a/security/integrity/ima/ima_appraise.c b/security/integrity/ima/ima_appraise.c index 0acf7d6baa37..919133dc8ef2 100644 --- a/security/integrity/ima/ima_appraise.c +++ b/security/integrity/ima/ima_appraise.c @@ -304,8 +304,13 @@ static int xattr_verify(enum ima_hooks func, struct integrity_iint_cache *iint, } else { set_bit(IMA_DIGSIG, &iint->atomic_flags); } - if (xattr_len - sizeof(xattr_value->type) - hash_start >= - iint->ima_hash->length) + /* + * Use addition, not subtraction: sizeof() forces unsigned + * math and a short xattr_len would wrap around, bypassing + * this bounds check. + */ + if (xattr_len >= (int)sizeof(xattr_value->type) + hash_start + + (int)iint->ima_hash->length) /* * xattr length may be longer. md5 hash in previous * version occupied 20 bytes in xattr, instead of 16 diff --git a/sound/usb/endpoint.c b/sound/usb/endpoint.c index 59e3584b24a0..c393a9125b9b 100644 --- a/sound/usb/endpoint.c +++ b/sound/usb/endpoint.c @@ -388,13 +388,15 @@ static int prepare_inbound_urb(struct snd_usb_endpoint *ep, case SND_USB_ENDPOINT_TYPE_DATA: offs = 0; for (i = 0; i < urb_ctx->packets; i++) { + if (offs + ep->curpacksize > urb_ctx->buffer_size) + break; urb->iso_frame_desc[i].offset = offs; urb->iso_frame_desc[i].length = ep->curpacksize; offs += ep->curpacksize; } urb->transfer_buffer_length = offs; - urb->number_of_packets = urb_ctx->packets; + urb->number_of_packets = i; break; case SND_USB_ENDPOINT_TYPE_SYNC: @@ -1260,10 +1262,10 @@ static int data_ep_set_params(struct snd_usb_endpoint *ep) u->index = i; u->ep = ep; u->packets = urb_packs; - u->buffer_size = maxsize * u->packets; if (fmt->fmt_type == UAC_FORMAT_TYPE_II) u->packets++; /* for transfer delimiter */ + u->buffer_size = maxsize * u->packets; u->urb = usb_alloc_urb(u->packets, GFP_KERNEL); if (!u->urb) goto out_of_memory; diff --git a/sound/usb/midi2.c b/sound/usb/midi2.c index f70e5ec66edb..f8653958563b 100644 --- a/sound/usb/midi2.c +++ b/sound/usb/midi2.c @@ -695,6 +695,14 @@ static int parse_midi_2_0_endpoints(struct snd_usb_midi2_interface *umidi) return 0; } +static void free_ump_private_data(struct snd_ump_endpoint *ump) +{ + struct snd_usb_midi2_ump *rmidi = ump->private_data; + + if (rmidi) + rmidi->ump = NULL; +} + static void free_all_midi2_umps(struct snd_usb_midi2_interface *umidi) { struct snd_usb_midi2_ump *rmidi; @@ -745,6 +753,7 @@ static int create_midi2_ump(struct snd_usb_midi2_interface *umidi, ump->private_data = rmidi; ump->ops = &snd_usb_midi_v2_ump_ops; + ump->private_free = free_ump_private_data; rmidi->eps[STR_IN] = ep_in; rmidi->eps[STR_OUT] = ep_out; diff --git a/sound/usb/usx2y/usX2Yhwdep.c b/sound/usb/usx2y/usX2Yhwdep.c index 4937ede0b5d7..34421b85838a 100644 --- a/sound/usb/usx2y/usX2Yhwdep.c +++ b/sound/usb/usx2y/usX2Yhwdep.c @@ -29,6 +29,8 @@ static vm_fault_t snd_us428ctls_vm_fault(struct vm_fault *vmf) vmf->pgoff); offset = vmf->pgoff << PAGE_SHIFT; + if (offset >= US428_SHAREDMEM_PAGES) + return VM_FAULT_SIGBUS; vaddr = (char *)((struct usx2ydev *)vmf->vma->vm_private_data)->us428ctls_sharedmem + offset; page = virt_to_page(vaddr); get_page(page); diff --git a/sound/usb/usx2y/usx2yhwdeppcm.c b/sound/usb/usx2y/usx2yhwdeppcm.c index 36f2e31168fb..54290caa9fad 100644 --- a/sound/usb/usx2y/usx2yhwdeppcm.c +++ b/sound/usb/usx2y/usx2yhwdeppcm.c @@ -676,6 +676,8 @@ static vm_fault_t snd_usx2y_hwdep_pcm_vm_fault(struct vm_fault *vmf) void *vaddr; offset = vmf->pgoff << PAGE_SHIFT; + if (offset >= USX2Y_HWDEP_PCM_PAGES) + return VM_FAULT_SIGBUS; vaddr = (char *)((struct usx2ydev *)vmf->vma->vm_private_data)->hwdep_pcm_shm + offset; vmf->page = virt_to_page(vaddr); get_page(vmf->page); diff --git a/tools/testing/selftests/bpf/prog_tests/sockmap_listen.c b/tools/testing/selftests/bpf/prog_tests/sockmap_listen.c index a6d8aa8c2a9a..6386d6069b9d 100644 --- a/tools/testing/selftests/bpf/prog_tests/sockmap_listen.c +++ b/tools/testing/selftests/bpf/prog_tests/sockmap_listen.c @@ -51,8 +51,8 @@ static void test_insert_opened(struct test_sockmap_listen *skel __always_unused, int family, int sotype, int mapfd) { u32 key = 0; - u64 value; int err, s; + u64 value; s = xsocket(family, sotype, 0); if (s == -1) @@ -61,11 +61,8 @@ static void test_insert_opened(struct test_sockmap_listen *skel __always_unused, errno = 0; value = s; err = bpf_map_update_elem(mapfd, &key, &value, BPF_NOEXIST); - if (sotype == SOCK_STREAM) { - if (!err || errno != EOPNOTSUPP) - FAIL_ERRNO("map_update: expected EOPNOTSUPP"); - } else if (err) - FAIL_ERRNO("map_update: expected success"); + ASSERT_ERR(err, "map_update"); + ASSERT_EQ(errno, EOPNOTSUPP, "errno"); xclose(s); } @@ -75,8 +72,8 @@ static void test_insert_bound(struct test_sockmap_listen *skel __always_unused, struct sockaddr_storage addr; socklen_t len; u32 key = 0; - u64 value; int err, s; + u64 value; init_addr_loopback(family, &addr, &len); @@ -91,8 +88,12 @@ static void test_insert_bound(struct test_sockmap_listen *skel __always_unused, errno = 0; value = s; err = bpf_map_update_elem(mapfd, &key, &value, BPF_NOEXIST); - if (!err || errno != EOPNOTSUPP) - FAIL_ERRNO("map_update: expected EOPNOTSUPP"); + if (sotype == SOCK_STREAM) { + ASSERT_ERR(err, "map_update"); + ASSERT_EQ(errno, EOPNOTSUPP, "errno"); + } else { + ASSERT_OK(err, "map_update"); + } close: xclose(s); } @@ -1261,7 +1262,7 @@ static void test_ops(struct test_sockmap_listen *skel, struct bpf_map *map, /* insert */ TEST(test_insert_invalid), TEST(test_insert_opened), - TEST(test_insert_bound, SOCK_STREAM), + TEST(test_insert_bound), TEST(test_insert), /* delete */ TEST(test_delete_after_insert), diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c index e0dd101c9f2b..f2a27143edc3 100644 --- a/tools/testing/selftests/bpf/test_maps.c +++ b/tools/testing/selftests/bpf/test_maps.c @@ -752,16 +752,15 @@ static void test_sockmap(unsigned int tasks, void *data) goto out_sockmap; } - /* Test update with unsupported UDP socket */ + /* Test update with unsupported unbound UDP socket */ udp = socket(AF_INET, SOCK_DGRAM, 0); - i = 0; - err = bpf_map_update_elem(fd, &i, &udp, BPF_ANY); - if (err) { - printf("Failed socket update SOCK_DGRAM '%i:%i'\n", - i, udp); + CHECK(udp < 0, "socket(AF_INET, SOCK_DGRAM)", "errno:%d\n", errno); + err = bpf_map_update_elem(fd, &(int){0}, &udp, BPF_ANY); + close(udp); + if (!err) { + printf("Unexpectedly succeeded unbound UDP update '0:%i'\n", udp); goto out_sockmap; } - close(udp); /* Test update without programs */ for (i = 0; i < 6; i++) { diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_eprobe.tc b/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_eprobe.tc index c300eb020262..e2322693d0c3 100644 --- a/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_eprobe.tc +++ b/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_eprobe.tc @@ -1,16 +1,16 @@ #!/bin/sh # SPDX-License-Identifier: GPL-2.0 # description: Generic dynamic event - add/remove eprobe events -# requires: dynamic_events events/syscalls/sys_enter_openat ". []":README +# requires: dynamic_events events/syscalls/sys_enter_chdir ". []":README echo 0 > events/enable clear_dynamic_events SYSTEM="syscalls" -EVENT="sys_enter_openat" +EVENT="sys_enter_chdir" FIELD="filename" -EPROBE="eprobe_open" +EPROBE="eprobe_chdir" OPTIONS="file=+0(\$filename):ustring" echo "e:$EPROBE $SYSTEM/$EVENT $OPTIONS" >> dynamic_events @@ -18,20 +18,14 @@ grep -q "$EPROBE" dynamic_events test -d events/eprobes/$EPROBE echo 1 > events/eprobes/$EPROBE/enable -ls +cd /sys/kernel/tracing echo 0 > events/eprobes/$EPROBE/enable -content=`grep '^ *ls-' trace | grep 'file='` -nocontent=`grep '^ *ls-' trace | grep 'file=' | grep -v -e '"/' -e '"."' -e '(fault)' ` || true - +content=`grep -e 'file="/sys/kernel/tracing"\|(fault)' trace` if [ -z "$content" ]; then exit_fail fi -if [ ! -z "$nocontent" ]; then - exit_fail -fi - echo "-:$EPROBE" >> dynamic_events ! grep -q "$EPROBE" dynamic_events