From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0976B342C8B for ; Mon, 29 Jun 2026 21:47:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.175 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782769638; cv=none; b=KG+6P7ua0PcsblqAyRha4ktiIhZtOB6x9EvRcHSMJepnZBviYuhna1lZxA9OLwepNl204iYv7V6Ed1udlr5TzL4ktcJguu6q+t5IPJBMtt1n4xIA0xE4yU4vXchE3rHQD/LH61cvtdYA3Cdmlhbi/pfGrbowIcAG0inevZht38w= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782769638; c=relaxed/simple; bh=AQDh0sR4xqQtaGY4gYXpI6mSYdSDC0X8Kw37xoC8foU=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=MP5g4jmZ0aR+59aX4AQoRdfJ2ezb8wntojdlnw5Gmcj40ujH91IvLjcAf6BYkmUNpCl6nk2GGxg26S4CL9GR3Cw3REVi2lTkgLiAC1dlPbBomfJLgUnxChlSNnaHscHRhEpZ/AZi/zqh/zUPWdyUBC87F7d4/JhtyupcQMVDkug= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=asu.edu; spf=pass smtp.mailfrom=asu.edu; dkim=pass (2048-bit key) header.d=asu.edu header.i=@asu.edu header.b=FQ4W9DAJ; arc=none smtp.client-ip=209.85.210.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=asu.edu Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=asu.edu Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=asu.edu header.i=@asu.edu header.b="FQ4W9DAJ" Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-846970f0acaso1377004b3a.3 for ; Mon, 29 Jun 2026 14:47:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=asu.edu; s=google; t=1782769636; x=1783374436; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=BdIjkiFaOfGY07XYoiVOjCOSv8Q05CUjEKu6WlWNo1A=; b=FQ4W9DAJNaWTdaQm64P0K15Tf7nzMhW9rcakuZXV/vPNky96cMYM8PFBL8jXeYmn1i EgCX0oFEXa8ltn03y/FBNVno7bPEvDvkKPt8ss23yO1VWwBMfR1OpUOBg4YjW7tsc/V2 UtEgqcTYf9gn83M75E5gqoGcgTA86IPZOMilLvlTdnf0ScXs/vFuNxjZn8hlVTh8XTWO bhG2UQSkBPYFhhiGCzFQgNqNIfbFSoAxmKMWqvFr6NK89qNrxZTc2ebm5KRyMWalHzLn 8vDYO6ECHCso/sqlcw/aG1XKToi8QkSDPdR0Dy2SLSigd92n2eD+Fp6/sjsV/mbBooBU jG9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782769636; x=1783374436; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=BdIjkiFaOfGY07XYoiVOjCOSv8Q05CUjEKu6WlWNo1A=; b=S6dtaJ/HDqyxSHZPWo4ufeQYxDCpOPUCfTSNNVAYhYhuqnOy3fSZ94ZlmW6CNs5JYx 9ViICCTXdEAMe0xWSMRmpIAVu7GFu6PPLrG/afGiqmbDweVSF0bLy+hp+4T8wNSt8td0 xxAKQjgyJ0YfHFzXl6Hh4sLQh7zsl+rWrhMIKC+/Jq2t9O7V6lJF689JgNzf3SL2icnf IxcGT/sNz2Hb6M0qogcNXv4VU6xifartdlAvUwuZcIy7XuUd0y8yoZ893FWktK6OvuPI 4wLG7H4XXikaM1YjLO4O+ZISgK+wikQ0CdNO07d9+NnFiVHR9wmirOeFoEWqCWS6UG5/ Dqbg== X-Forwarded-Encrypted: i=1; AHgh+RqtMu4BJfkKFEQyK2Hq0i1/Kx+4hA7Lxot7K6syl5kafgdnvaVZoZCL5G+/EAZ7rds2UbtxlxsFBjkXMBk=@vger.kernel.org X-Gm-Message-State: AOJu0YzIVXSQCQDLPFLWNw81cduIDICb/rHidA4sh81a2+zjd+CUQBmB zBOuUU7Uv7/BD614fgGXvO9SKoIXqq7RhVxvtnMgEeWQ717TsTrfLJsr65NMtP+1Jw== X-Gm-Gg: AfdE7clSyiw6adZxAUnov4r5oRwzruXWUGlu+YQFFHWNREQVKMeygHljHVWWEgrnlnP N3cRMI3RX7dzbl3/s0QoxHHpH18qr3r6i9GBHscZ8ViseAnPtceIrQh5p/FABWNWQe5zqTWOGME dWgxS7SEaVpANvB7IeTEIo6VhbrT4qpQGK1B7cX2Db1o/q7HDdfisfxkKxIKZ7pewkZhKNXjjIZ kSmoKJ0/u4JtvvvdETEI4lre88zMMXs6giT7K1uXHpK2shzX7CWLw/iaMYLaVG8Duf9v7m5OKdb qYAgNYh6eL2H1TkJ/t3Ech+u73VmbJpim0i++YCSg/29ItLpbrdvb6D6Sk1nqdexZz4PmqW8Q37 rzDBBwsiXtmvLuxpaQ/F4pHV1nOMxPaVlWAZX3t02hpdwH7JlIa5QFwFPh7qxJZY61tdLZDIOxw == X-Received: by 2002:aa7:9a83:0:b0:847:82db:9046 with SMTP id d2e1a72fcca58-8479f1698a3mr736747b3a.26.1782769636217; Mon, 29 Jun 2026 14:47:16 -0700 (PDT) Received: from p1.. ([2607:fb91:151e:4b96:1640:8d7c:ffc5:b8d9]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-8479ff8c781sm436430b3a.9.2026.06.29.14.47.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jun 2026 14:47:15 -0700 (PDT) From: Xiang Mei To: Kees Cook , Andrew Morton , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, linux-hardening@vger.kernel.org Cc: Uladzislau Rezki , "Gustavo A . R . Silva" , "H . Peter Anvin" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Jennifer Miller , Tiffany Bao , Ruoyu Wang , Adam Doupe , Kyle Zeng , Yan Shoshitaishvili , Xiang Mei Subject: [PATCH v2] mm/vmalloc: widen guard region to defeat ENTER-based stack pivot Date: Mon, 29 Jun 2026 14:47:12 -0700 Message-ID: <20260629214712.1198680-1-xmei5@asu.edu> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit With CONFIG_VMAP_STACK, kernel stacks are allocated in the vmalloc area, which an unprivileged user can surround with attacker-controlled data by spraying vmap allocations adjacent to a target stack (for example via XDP_UMEM_REG, though other vmalloc spray paths work too). Today each guarded vmalloc allocation is followed by a single unmapped guard page. A single guard page is not enough to contain the x86_64 ENTER instruction used as a one-instruction stack pivot. ENTER imm16, imm8 builds a stack frame and lowers RSP by: imm16 + 8 * (L + 1), L = imm8 & 0x1f imm16 is an unsigned 16-bit operand (ENTER never raises RSP), and L is in [0, 31], so the maximum displacement of a single ENTER is: 0xffff + 8 * 0x20 = 0x100ff bytes That is more than enough to step off the current stack, across the one-page guard, and into the adjacent sprayed pages. When those pages contain a return sled feeding a ROP chain, reaching any ENTER gadget (opcode 0xc8, abundant as both intended and unintended gadgets) turns a control-flow hijack into full ROP execution without any register control at the hijack site, making it a one-gadget-style primitive that significantly eases exploitation. The pivot happens after the control transfer, so it is not constrained by CFI (kCFI/FineIBT). Widen the guard region from one page to VMAP_GUARD_PAGES (0x11 pages, 0x11000 bytes), which is the smallest whole-page span exceeding the 0x100ff-byte maximum single-ENTER pivot. A pivot off the top of the stack now lands in the unmapped guard and faults, instead of in mapped, attacker-controlled memory. RANDOMIZE_KSTACK_OFFSET only perturbs RSP by a sub-page amount, so it does not change the required width. Introduce a VMAP_GUARD_PAGES knob that defaults to a single page (no change for current architectures) and can be overridden per arch via asm/vmalloc.h, and set it to 0x11 on x86_64. This is deliberately scoped to x86_64: the 0x100ff bound is a property of the ENTER opcode, and ENTER is also a one-byte opcode (0xc8) that appears as abundant unintended gadgets. Other architectures (e.g. arm64) have no equivalent single-instruction, immediate-controlled pivot reachable as an unaligned unintended gadget, so they keep the one-page guard and pay no cost. The override is gated on CONFIG_X86_64 rather than applying to all of x86: VMAP_STACK is selected only on x86_64, so 32-bit kernel stacks are not in the vmalloc area and the technique does not apply there. 32-bit x86 also has a far smaller vmalloc window, where widening every guarded area by 16 pages would needlessly pressure the address space. The guard pages are never populated, so there is no extra physical memory and no additional page-table population beyond the larger virtual span; the cost is virtual address space and vmap_area bookkeeping, which is negligible against the 64-bit vmalloc window. get_vm_area_size() is adjusted by the same VMAP_GUARD_SIZE so the usable size reported to callers is unchanged. On x86 this widens the guard for all guarded vmap areas, not only thread stacks. ret2enter targets the stack specifically, so a narrower alternative is to apply the wider guard only on the thread-stack allocation path via a dedicated VM_ flag; we kept the change in the common path as defense in depth for any vmalloc-adjacent pivot target, but are happy to scope it to stacks if maintainers prefer. While widening the guard, also mark percpu vmap areas VM_NO_GUARD. pcpu_get_vm_areas() and pcpu_page_first_chunk() size each area exactly and reserve no guard, so get_vm_area_size() would subtract a guard that was never added and underflow if an area were smaller than the guard. This is a latent correctness fix only: on x86_64 percpu areas are megabyte-scale, far larger than the guard. Signed-off-by: Xiang Mei Signed-off-by: Jennifer Miller --- v2: VM_NO_GUARD for percpu vmap areas arch/x86/include/asm/vmalloc.h | 21 +++++++++++++++++++++ include/linux/vmalloc.h | 16 ++++++++++++++-- mm/percpu.c | 2 +- mm/vmalloc.c | 4 ++-- 4 files changed, 38 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/vmalloc.h b/arch/x86/include/asm/vmalloc.h index 49ce331f3ac6..2c341f398227 100644 --- a/arch/x86/include/asm/vmalloc.h +++ b/arch/x86/include/asm/vmalloc.h @@ -5,6 +5,27 @@ #include #include +/* + * The x86 ENTER instruction can be used as a one-instruction stack pivot: + * ENTER imm16, imm8 lowers RSP by imm16 + 8 * (L + 1), L = imm8 & 0x1f. + * imm16 is an unsigned 16-bit operand (ENTER never raises RSP) and L is in + * [0, 31], so a single ENTER can lower RSP by at most + * 0xffff + 8 * 0x20 = 0x100ff bytes. With CONFIG_VMAP_STACK the kernel + * stack lives in the vmalloc area, where an unprivileged user can spray + * adjacent allocations; a single-page guard is too small to contain such a + * pivot. Use 0x11 guard pages (0x11000 bytes), the smallest whole-page + * span exceeding 0x100ff, so the pivot faults in the guard instead of + * landing in attacker-controlled memory. + * + * Restrict this to 64-bit: VMAP_STACK is selected only on x86_64, so 32-bit + * kernel stacks are not in the vmalloc area and the technique does not apply. + * 32-bit also has a far smaller vmalloc window, where a 16-page-per-area + * widening would needlessly pressure the address space. + */ +#ifdef CONFIG_X86_64 +#define VMAP_GUARD_PAGES 0x11 +#endif + #ifdef CONFIG_HAVE_ARCH_HUGE_VMAP #ifdef CONFIG_X86_64 diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 3b02c0c6b371..b8546e519deb 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -49,6 +49,18 @@ struct iov_iter; /* in uio.h */ #define IOREMAP_MAX_ORDER (7 + PAGE_SHIFT) /* 128 pages */ #endif +/* + * Number of unmapped guard pages appended to each guarded vmalloc + * allocation. The default is a single page; an architecture may override + * VMAP_GUARD_PAGES (via asm/vmalloc.h) when a wider guard is needed to + * contain a worst-case single-instruction stack pivot into an adjacent, + * attacker-controlled vmap allocation (see arch/x86 for the ENTER case). + */ +#ifndef VMAP_GUARD_PAGES +#define VMAP_GUARD_PAGES 1 +#endif +#define VMAP_GUARD_SIZE (VMAP_GUARD_PAGES * PAGE_SIZE) + struct vm_struct { union { struct vm_struct *next; /* Early registration of vm_areas. */ @@ -236,8 +248,8 @@ int vmap_pages_range(unsigned long addr, unsigned long end, pgprot_t prot, static inline size_t get_vm_area_size(const struct vm_struct *area) { if (!(area->flags & VM_NO_GUARD)) - /* return actual size without guard page */ - return area->size - PAGE_SIZE; + /* return actual size without guard region */ + return area->size - VMAP_GUARD_SIZE; else return area->size; diff --git a/mm/percpu.c b/mm/percpu.c index b0676b8054ed..9f7262228be1 100644 --- a/mm/percpu.c +++ b/mm/percpu.c @@ -3243,7 +3243,7 @@ int __init pcpu_page_first_chunk(size_t reserved_size, pcpu_fc_cpu_to_node_fn_t } /* allocate vm area, map the pages and copy static data */ - vm.flags = VM_ALLOC; + vm.flags = VM_ALLOC | VM_NO_GUARD; vm.size = num_possible_cpus() * ai->unit_size; vm_area_register_early(&vm, PAGE_SIZE); diff --git a/mm/vmalloc.c b/mm/vmalloc.c index bb6ae08d18f5..fc435054f640 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -3217,7 +3217,7 @@ struct vm_struct *__get_vm_area_node(unsigned long size, return NULL; if (!(flags & VM_NO_GUARD)) - size += PAGE_SIZE; + size += VMAP_GUARD_SIZE; area->flags = flags; area->caller = caller; @@ -5027,7 +5027,7 @@ struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets, spin_lock(&vn->busy.lock); insert_vmap_area(vas[area], &vn->busy.root, &vn->busy.head); - setup_vmalloc_vm(vms[area], vas[area], VM_ALLOC, + setup_vmalloc_vm(vms[area], vas[area], VM_ALLOC | VM_NO_GUARD, pcpu_get_vm_areas); spin_unlock(&vn->busy.lock); } -- 2.43.0