[PATCH v2 0/1] KVM: arm64: Resume stage-2 unmap where the walk stopped

[Haseeb Ashraf <haseebashraf091@gmail.com>]

Unmapping the whole IPA space of a shadow stage-2, as emulating TLBI
VMALLS12E1IS and TLBI ALLE1IS does, walks it one block-sized chunk at a
time. The walker steps over an invalid entry in one go, but the next
walk starts at the following chunk, so an invalid entry larger than a
chunk is visited once for every chunk it covers. Each walk also ends
with a range TLBI when FEAT_TLBIRANGE and FWB are available.

This version follows the approach of Fuad's series [2]: the walk reports
the address it reached, and the unmap loop resumes from there. The walk
already handles every entry it steps over, so nothing has to be looked
up beforehand.

An example shows what changes. Take 4K pages and a 48-bit IPA space.
The root is a level-0 table with 512 entries of 512GB each, and a chunk
is 1GB:

  Empty shadow stage-2
    before: 2^18 = 262144 walks. Each of the 512 level-0 entries is
            visited once for every one of the 512 chunks under it.
    after:  512 walks. The first walk under an invalid entry returns
            the end of that entry, and the loop resumes there.

  One populated 512GB region (RAM and devices low in the IPA space)
    before: 262144 walks.
    after:  at most 512 + 511 = 1023 walks: 512 chunks inside the
            region, and one step over each of the 511 other entries.

The example is derived from the page-table layout. Measurements follow.
The machine is an AmpereOne A192-32X (192 CPUs, bare metal, 4K pages,
48-bit IPA, FWB and TLB range maintenance present) running Xen as a KVM
guest. The workload is one create and destroy cycle of an 8GB guest
under Xen. bpftrace kprobes and kretprobes recorded every full-range
unmap (start 0, size 2^48), its walks, its range TLBIs and its duration.
"Before" is plain 6.18.29. "After" is 6.18.29 with this patch, adapted
to the older unmap loop, and with seven commits that are already in
mainline backported: 33346f8960c7 and e5843f4effaa (nested_mmus life
cycle), and a6ca6cba5959, c2778d14cd2f, 5491980c1a41, e6084aecaeef and
2064f15b0848 (guest stage-2 tracking).

                                    before          after
  full-range unmaps                    192            128
  walks per unmap                   262144     512 - 1024
  range TLBIs per unmap             262144  same as walks
  unmap duration, mean            907.8 ms        3.77 ms
  unmap duration, median          909.3 ms        2.46 ms
  unmap duration, min             815.5 ms        2.06 ms
  unmap duration, max             937.6 ms       27.8  ms
  VMALLS12E1IS trap, mean         957.0 ms        3.78 ms
  VMALLS12E1IS trap, max         1848.6 ms       27.9  ms

Walks per unmap after the change: 512 (10 unmaps), 534 (1), 546 (78),
1023 (38) and 1024 (1), mean 689. The page-table occupancy was not
recorded, so I cannot tie each count to a layout. 1024 is one more than
a single populated region gives. Before the change nine of the 177 traps
unmapped two shadow stage-2s and took about twice as long; after it
every trap unmapped one.

Patch 1 adds kvm_pgtable_stage2_unmap_range(), which returns the address
its walk reached, and has the unmap loop resume from there.
kvm_pgtable_stage2_unmap() keeps its return value for its callers at
EL2. The unmap of a shadow stage-2 also drops the guest stage-2 records
of the whole span the walk covered. The TLBI VMALLS12E1IS handler,
kvm_nested_s2_unmap() and the pending unmap of a recycled shadow stage-2
need no change.

Changes since v1 [1]:
  - Reworked as Oliver suggested: the walk reports the next IPA to the
    outer loop, as in Fuad's series [2]. This replaces
    kvm_stage2_unmap_all() and its kvm_pgtable_get_leaf() lookup.
  - No extra VMID-wide TLBI: each walk keeps its own TLB maintenance.
  - Measurements added.
  - Based on Linux 7.4-rc1, the same base as [2]. It does not depend on
    [2]; it applies with and without it.

[1] https://lore.kernel.org/r/cfa77cdbf14d9fd24c22ab1ef9b940a0d7411699.1794229714.git.haseeb.ashraf@siemens.com
[2] https://lore.kernel.org/r/20261106133000.453240-1-fuad.tabba@linux.dev

Tooling: the patch was drafted with AI assistance (SiemensGPT). The
statistics were collected with bpftrace, using kprobes at function entry
and kretprobes at function return.

Testing: the backport of this patch, together with the seven commits
listed above, was booted as 6.18.29 on the machine above, and the
measurements are from that kernel. The patch for 7.4-rc1 itself was only
compile-checked in isolation.