From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 31F913F4128 for ; Fri, 26 Jun 2026 11:27:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782473239; cv=none; b=TBPOiIsYDTr8GjhKqHuXSS/vMZy2343VFF2PzvWNVValNvp1sLb7xpufQ8cacqfW+WLwIu5oJE59e+4NCks3BjnBLgIaP5NVHJBlDTH93RSjMKINQPbB7FC7tIayQatbI05To2/P3MkPJDSr0KY8UxxVONF2AJIC8fC1PKsvYls= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782473239; c=relaxed/simple; bh=XKqcae9+Don1b55+R5JA19jiHll4kjMqKowe4S6YavQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MbhW+eh7+kixiG9wcRxBbrD9q4axq6qiMSjm+Mt/3QptpX5ANYQGICJi32dRXDyTmfiSyfJ/6kLM/zngx2n4XVv46/DZOP43TE/M9DH4w9wgNXNkJbWbjNj3E2meAQSgsEezIUYTv7WaVEk7DfY9DAg/Og3oZX/SAtFPE4zpVMw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=PLMLm8RN; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=LTL2aVBs; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="PLMLm8RN"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="LTL2aVBs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782473237; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lWvS5GCqW407fjbCc5yPHkkipazXCaeN7akeVks2xPw=; b=PLMLm8RNPqdTnMLX4iFeXWWDp7p0R6oO1evRCqI0ujalnK/rO+hV5MMKcaBqpx/xp9I4wJ moPQVUy/RNKt4OxMwJhnUYr5dKSsXiRP4jVSr9zQFHtwYlQQmdZ1JUPbd2SJG8wb1I8u0W pFak36SONzigFt5rgY2iVcJZAv5SCkM= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-622-NAze35D4P46JbRnja5d_SA-1; Fri, 26 Jun 2026 07:27:14 -0400 X-MC-Unique: NAze35D4P46JbRnja5d_SA-1 X-Mimecast-MFC-AGG-ID: NAze35D4P46JbRnja5d_SA_1782473233 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-46315372c3eso638203f8f.0 for ; Fri, 26 Jun 2026 04:27:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1782473233; x=1783078033; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lWvS5GCqW407fjbCc5yPHkkipazXCaeN7akeVks2xPw=; b=LTL2aVBs5W0DwEzl0aN4NpqDRswLaBUsw0/aDK4ywv/AHBBJxNA8yg5iCOCrc3d2/t MW42PKYZ09vzLQQ5yaO1UCIlUO19I6FcsC8MCzfWTrah0jl1tiljIT/FvrTZYEvWWKg4 rUDm2iHXK5tP6Ng2JE5bzFzAq4sQn3t2xI3BjHmWjC8R5BbM4f2wqI8ky6HBJHAecs8J H7CC9RG5ID87ARPtwWq/r9ktivfrEdkJtNJRmZKG58FzamRjgxB+RhzF+7DQGgoAtrrW K0ul7JKATaVINDkMqRpN1P0/vgZodwt64gUhRQrI/o8VJ7doTXfAejxVXXj3Ko1bKW+q dShQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782473233; x=1783078033; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=lWvS5GCqW407fjbCc5yPHkkipazXCaeN7akeVks2xPw=; b=MB+Z4MGPPQ6kS+x+00p1T7/9RcD9iFeN5GqO+/Wou+x0wRbwTo4ZaMNSWesQOUVw6E w9G5UudhFkvLmsfp3/HK4G3LjN38z8qw1Ds81G9geZ6dofdRV+vYNF8/ogrzw+t0R4Pz +XCM5o1xQlfh3MsEEYjwiufUKzAfzpg1NGaJFFznyzOSRAltT999nzU5b7Tyqt8eVdk2 WtoHhLyoiovXTMtHFUWT1Y77iie6iy34UOAWIatTtCW4lr4agtu+XdqrUdVtwhWX5p1I aiy7a0T2NxCOW8bfvHS+5uz/P6cNOWvP+LgLZ3CFD+JsNSCKyYBvpzn6nEdXvnLvhJO6 04lA== X-Gm-Message-State: AOJu0Yzes/sXkyIOFfS+HxLcY+oT6W01JpyGIMleWVgTxrdLYymxo/Rx lE7vkXjr5ISTcAgtrQ1j6UybKfJxyW0SNoxxDN/oNJWJ3h6z5xiNYnlE8xUSfWEEjc4M8uzbyCQ BbXm++YgTlRLDmUmcExsksALm8lWZeCHfkGefz78PZZhbJKMAmkKzU8lCJHYAa/K5oXrxOEZ474 xSTgf7GHQK0CP4iD/K1Exr9ePRWpmTyKNcJ+xB6Ua/OmvUkZBwJQ== X-Gm-Gg: AfdE7cmM3zjvZ8em/XePFQUJVRz6Nsv0TlyZrbFQv65Vd3IA9NirDtrMgcyMAWmiCQ1 MOXCICLJ81fF5c3afQESNVTgN9LLYeiJefjX+GykoT6KpR+oItBC6Z0U7G8MhWQUnfIZfUOC78p FP39tGEAvWIR0qaIDnKpwxSUMVMzj2SRA/7PMKP0MVahUjGTeXslrKZ/8uNsztHYe2FRSDbsyAB BK/RoTElOAm2pgvY6rpg/nSPw0Tev9V3jpz/Y49VSz0X0mOz6LaL9JdT+tO8xoL9+tuk2ioClY6 uCDc4L1gytM2Lakz9CSWsP1R8YeVOmDB3Hl6GgFCCpHSoqqTh/ApYnTrN2LvfjRft7dCjEQI56T 7CPuDpQsRxjMTUEQUZBdQ+N16LUJDpPO2Cf/H/78lJPD3GvgcqOPrWBuF7Zcz8b2kGLgkFNtK2c HP4jc5bHP8DBbcN52C X-Received: by 2002:a5d:5f4c:0:b0:441:1e8e:d8fd with SMTP id ffacd0b85a97d-46dc12dfa46mr9183567f8f.29.1782473232723; Fri, 26 Jun 2026 04:27:12 -0700 (PDT) X-Received: by 2002:a5d:5f4c:0:b0:441:1e8e:d8fd with SMTP id ffacd0b85a97d-46dc12dfa46mr9183481f8f.29.1782473232001; Fri, 26 Jun 2026 04:27:12 -0700 (PDT) Received: from [192.168.10.48] ([151.95.124.208]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-46c221d998esm23376169f8f.24.2026.06.26.04.27.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jun 2026 04:27:10 -0700 (PDT) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, stable@vger.kernel.org Cc: Sean Christopherson Subject: [PATCH 5.10.y 13/17] KVM: x86/mmu: pull call to drop_large_spte() into __link_shadow_page() Date: Fri, 26 Jun 2026 13:26:30 +0200 Message-ID: <20260626112634.1778506-14-pbonzini@redhat.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260626112634.1778506-1-pbonzini@redhat.com> References: <20260626112634.1778506-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit commit 0cd8dc739833080aa0813cbd94d907a93e3a14c3 upstream. Before allocating a child shadow page table, all callers check whether the parent already points to a huge page and, if so, they drop that SPTE. This is done by drop_large_spte(). However, dropping the large SPTE is really only necessary before the sp is installed. While the sp is returned by kvm_mmu_get_child_sp(), installing it happens later in __link_shadow_page(). Move the call there instead of having it in each and every caller. To ensure that the shadow page is not linked twice if it was present, do _not_ opportunistically make kvm_mmu_get_child_sp() idempotent: instead, return an error value if the shadow page already existed. This is a bit more verbose, but clearer than NULL. Finally, now that the drop_large_spte() name is not taken anymore, remove the two underscores in front of __drop_large_spte(). Reviewed-by: Sean Christopherson Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 51 +++++++++++++++++++--------------- arch/x86/kvm/mmu/paging_tmpl.h | 29 +++++++++---------- 2 files changed, 41 insertions(+), 39 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5df1cd5bff1b..47c5c3613b68 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -1067,27 +1067,17 @@ static void drop_spte(struct kvm *kvm, u64 *sptep) rmap_remove(kvm, sptep); } - -static bool __drop_large_spte(struct kvm *kvm, u64 *sptep) +static void drop_large_spte(struct kvm *kvm, u64 *sptep) { - if (is_large_pte(*sptep)) { - WARN_ON(sptep_to_sp(sptep)->role.level == PG_LEVEL_4K); - drop_spte(kvm, sptep); - --kvm->stat.lpages; - return true; - } + struct kvm_mmu_page *sp; - return false; -} + sp = sptep_to_sp(sptep); + WARN_ON(sp->role.level == PG_LEVEL_4K); -static void drop_large_spte(struct kvm_vcpu *vcpu, u64 *sptep) -{ - if (__drop_large_spte(vcpu->kvm, sptep)) { - struct kvm_mmu_page *sp = sptep_to_sp(sptep); - - kvm_flush_remote_tlbs_with_address(vcpu->kvm, sp->gfn, + drop_spte(kvm, sptep); + --kvm->stat.lpages; + kvm_flush_remote_tlbs_with_address(kvm, sp->gfn, KVM_PAGES_PER_HPAGE(sp->role.level)); - } } /* @@ -2141,6 +2131,9 @@ static struct kvm_mmu_page *kvm_mmu_get_child_sp(struct kvm_vcpu *vcpu, { union kvm_mmu_page_role role; + if (is_shadow_present_pte(*sptep) && !is_large_pte(*sptep)) + return ERR_PTR(-EEXIST); + role = kvm_mmu_child_role(sptep, direct, access); return kvm_mmu_get_page(vcpu, gfn, role); } @@ -2208,13 +2201,21 @@ static void shadow_walk_next(struct kvm_shadow_walk_iterator *iterator) __shadow_walk_next(iterator, *iterator->sptep); } -static void link_shadow_page(struct kvm_vcpu *vcpu, u64 *sptep, - struct kvm_mmu_page *sp) +static void __link_shadow_page(struct kvm_vcpu *vcpu, + struct kvm_mmu_memory_cache *cache, u64 *sptep, + struct kvm_mmu_page *sp) { u64 spte; BUILD_BUG_ON(VMX_EPT_WRITABLE_MASK != PT_WRITABLE_MASK); + /* + * If an SPTE is present already, it must be a leaf and therefore + * a large one. Drop it and flush the TLB before installing sp. + */ + if (is_shadow_present_pte(*sptep)) + drop_large_spte(vcpu->kvm, sptep); + spte = make_nonleaf_spte(sp->spt, sp_ad_disabled(sp)); mmu_spte_set(sptep, spte); @@ -2225,6 +2226,12 @@ static void link_shadow_page(struct kvm_vcpu *vcpu, u64 *sptep, mark_unsync(sptep); } +static void link_shadow_page(struct kvm_vcpu *vcpu, u64 *sptep, + struct kvm_mmu_page *sp) +{ + __link_shadow_page(vcpu, &vcpu->arch.mmu_pte_list_desc_cache, sptep, sp); +} + static void validate_direct_spte(struct kvm_vcpu *vcpu, u64 *sptep, unsigned direct_access) { @@ -2923,11 +2930,9 @@ static int __direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code, if (it.level == level) break; - drop_large_spte(vcpu, it.sptep); - if (is_shadow_present_pte(*it.sptep)) - continue; - sp = kvm_mmu_get_child_sp(vcpu, it.sptep, base_gfn, true, ACC_ALL); + if (sp == ERR_PTR(-EEXIST)) + continue; link_shadow_page(vcpu, it.sptep, sp); if (is_tdp && huge_page_disallowed && diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index d5facee0db60..250b4b0f6b68 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -661,15 +661,13 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, gpa_t addr, gfn_t table_gfn; clear_sp_write_flooding_count(it.sptep); - drop_large_spte(vcpu, it.sptep); - sp = NULL; - if (!is_shadow_present_pte(*it.sptep)) { - table_gfn = gw->table_gfn[it.level - 2]; - access = gw->pt_access[it.level - 2]; - sp = kvm_mmu_get_child_sp(vcpu, it.sptep, table_gfn, - false, access); + table_gfn = gw->table_gfn[it.level - 2]; + access = gw->pt_access[it.level - 2]; + sp = kvm_mmu_get_child_sp(vcpu, it.sptep, table_gfn, + false, access); + if (sp != ERR_PTR(-EEXIST)) { /* * We must synchronize the pagetable before linking it * because the guest doesn't need to flush tlb when @@ -698,7 +696,7 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, gpa_t addr, if (FNAME(gpte_changed)(vcpu, gw, it.level - 1)) goto out_gpte_changed; - if (sp) + if (sp != ERR_PTR(-EEXIST)) link_shadow_page(vcpu, it.sptep, sp); } @@ -724,15 +722,14 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, gpa_t addr, validate_direct_spte(vcpu, it.sptep, direct_access); - drop_large_spte(vcpu, it.sptep); + sp = kvm_mmu_get_child_sp(vcpu, it.sptep, base_gfn, + true, direct_access); + if (sp == ERR_PTR(-EEXIST)) + continue; - if (!is_shadow_present_pte(*it.sptep)) { - sp = kvm_mmu_get_child_sp(vcpu, it.sptep, base_gfn, - true, direct_access); - link_shadow_page(vcpu, it.sptep, sp); - if (huge_page_disallowed && req_level >= it.level) - account_huge_nx_page(vcpu->kvm, sp); - } + link_shadow_page(vcpu, it.sptep, sp); + if (huge_page_disallowed && req_level >= it.level) + account_huge_nx_page(vcpu->kvm, sp); } ret = mmu_set_spte(vcpu, it.sptep, gw->pte_access, write_fault, -- 2.54.0