From: "Hou Wenlong" <houwenlong.hwl@antgroup.com>
To: Ard Biesheuvel <ardb@kernel.org>
Cc: <linux-kernel@vger.kernel.org>,
"Lai Jiangshan" <jiangshan.ljs@antgroup.com>,
"Kees Cook" <keescook@chromium.org>,
"Thomas Gleixner" <tglx@linutronix.de>,
"Ingo Molnar" <mingo@redhat.com>,
"Borislav Petkov" <bp@alien8.de>,
"Dave Hansen" <dave.hansen@linux.intel.com>, <x86@kernel.org>,
"H. Peter Anvin" <hpa@zytor.com>,
"Peter Zijlstra" <peterz@infradead.org>,
"Petr Mladek" <pmladek@suse.com>,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
"Jason A. Donenfeld" <Jason@zx2c4.com>,
"Song Liu" <song@kernel.org>,
"Julian Pidancet" <julian.pidancet@oracle.com>
Subject: Re: [PATCH RFC 31/43] x86/modules: Adapt module loading for PIE support
Date: Mon, 08 May 2023 16:32:23 +0800 [thread overview]
Message-ID: <20230508083223.GA116442@k08j02272.eu95sqa> (raw)
In-Reply-To: <CAMj1kXH8E7AgvmJ-Spa+EwHSz7kcqd+AMGvtxRUo3jFD3PAWsQ@mail.gmail.com>
On Sat, Apr 29, 2023 at 03:29:32AM +0800, Ard Biesheuvel wrote:
> On Fri, 28 Apr 2023 at 10:53, Hou Wenlong <houwenlong.hwl@antgroup.com> wrote:
> >
> > Adapt module loading to support PIE relocations. No GOT is generared for
> > module, all the GOT entry of got references in module should exist in
> > kernel GOT. Currently, there is only one usable got reference for
> > __fentry__().
> >
>
> I don't think this is the right approach. We should permit GOTPCREL
> relocations properly, which means making them point to a location in
> memory that carries the absolute address of the symbol. There are
> several ways to go about that, but perhaps the simplest way is to make
> the symbol address in ksymtab a 64-bit absolute value (but retain the
> PC32 references for the symbol name and the symbol namespace name).
> That way, you can always resolve such GOTPCREL relocations by pointing
> it to the ksymtab entry. Another option would be to take inspiration
> from the PLT code we have on ARM and arm64 (and other architectures,
> surely) and to count the GOT based relocations, allocate some extra
> r/o module space for each, and allocate slots and populate them with
> the right value as you fix up the relocations.
>
> Then, many such relocations can be relaxed at module load time if the
> symbol is in range. IIUC, the module and kernel will still be inside
> the same 2G window even after widening the KASLR range to 512G, so
> most GOT loads can be converted into RIP relative LEA instructions.
>
> Note that this will also permit you to do things like
>
> #define PV_VCPU_PREEMPTED_ASM \
> "leaq __per_cpu_offset(%rip), %rax \n\t" \
> "movq (%rax,%rdi,8), %rax \n\t" \
> "addq steal_time@GOTPCREL(%rip), %rax \n\t" \
> "cmpb $0, " __stringify(KVM_STEAL_TIME_preempted) "(%rax) \n\t" \
> "setne %al\n\t"
>
> or
>
> +#ifdef CONFIG_X86_PIE
> + " pushq arch_rethook_trampoline@GOTPCREL(%rip)\n"
> +#else
> " pushq $arch_rethook_trampoline\n"
> +#endif
>
> instead of having these kludgy push/pop sequences to free up temp registers.
>
> (FYI I have looked into this PIE linking just a few weeks ago [0] so
> this is all rather fresh in my memory)
>
>
>
>
> [0] https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=x86-pie
>
>
Hi Ard,
Thanks for providing the link, it has been very helpful for me as I am
new to the topic of compilers. One key difference I noticed is that you
linked the kernel with "-pie" instead of "--emit-reloc". I also noticed
that Thomas' initial patchset[0] used "-pie", but in RFC v3 [1], it
switched to "--emit-reloc" in order to reduce dynamic relocation space
on mapped memory.
The another issue is that it requires the addition of the
"-mrelax-relocations=no" option to support older compilers and linkers.
R_X86_64_GOTPCRELX and R_X86_64_REX_GOTPCRELX relocations are supported
in binutils 2.26 and later, but the mini version required for the kernel
is 2.25. This option disables relocation relaxation, which makes GOT not
empty. I also noticed this option in arch/x86/boot/compressed/Makefile
with the reason given in [2]. Without relocation relaxation, GOT
references would increase the size of GOT. Therefore, I do not want to
use GOT reference in assembly directly. However, I realized that the
compiler could still generate GOT references in some cases such as
"fentry" calls and stack canary references.
Regarding module loading, I agree that we should support GOT reference
for the module itself. I will refactor it according to your suggestion.
Thanks.
[0] https://yhbt.net/lore/all/20170718223333.110371-20-thgarnie@google.com
[1] https://yhbt.net/lore/all/20171004212003.28296-1-thgarnie@google.com
[2] https://lore.kernel.org/all/20200903203053.3411268-2-samitolvanen@google.com/
> > Signed-off-by: Hou Wenlong <houwenlong.hwl@antgroup.com>
> > Cc: Thomas Garnier <thgarnie@chromium.org>
> > Cc: Lai Jiangshan <jiangshan.ljs@antgroup.com>
> > Cc: Kees Cook <keescook@chromium.org>
> > ---
> > arch/x86/include/asm/sections.h | 5 +++++
> > arch/x86/kernel/module.c | 27 +++++++++++++++++++++++++++
> > 2 files changed, 32 insertions(+)
> >
> > diff --git a/arch/x86/include/asm/sections.h b/arch/x86/include/asm/sections.h
> > index a6e8373a5170..dc1c2b08ec48 100644
> > --- a/arch/x86/include/asm/sections.h
> > +++ b/arch/x86/include/asm/sections.h
> > @@ -12,6 +12,11 @@ extern char __end_rodata_aligned[];
> >
> > #if defined(CONFIG_X86_64)
> > extern char __end_rodata_hpage_align[];
> > +
> > +#ifdef CONFIG_X86_PIE
> > +extern char __start_got[], __end_got[];
> > +#endif
> > +
> > #endif
> >
> > extern char __end_of_kernel_reserve[];
> > diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c
> > index 84ad0e61ba6e..051f88e6884e 100644
> > --- a/arch/x86/kernel/module.c
> > +++ b/arch/x86/kernel/module.c
> > @@ -129,6 +129,18 @@ int apply_relocate(Elf32_Shdr *sechdrs,
> > return 0;
> > }
> > #else /*X86_64*/
> > +#ifdef CONFIG_X86_PIE
> > +static u64 find_got_kernel_entry(Elf64_Sym *sym, const Elf64_Rela *rela)
> > +{
> > + u64 *pos;
> > +
> > + for (pos = (u64 *)__start_got; pos < (u64 *)__end_got; pos++)
> > + if (*pos == sym->st_value)
> > + return (u64)pos + rela->r_addend;
> > + return 0;
> > +}
> > +#endif
> > +
> > static int __write_relocate_add(Elf64_Shdr *sechdrs,
> > const char *strtab,
> > unsigned int symindex,
> > @@ -171,6 +183,7 @@ static int __write_relocate_add(Elf64_Shdr *sechdrs,
> > case R_X86_64_64:
> > size = 8;
> > break;
> > +#ifndef CONFIG_X86_PIE
> > case R_X86_64_32:
> > if (val != *(u32 *)&val)
> > goto overflow;
> > @@ -181,6 +194,13 @@ static int __write_relocate_add(Elf64_Shdr *sechdrs,
> > goto overflow;
> > size = 4;
> > break;
> > +#else
> > + case R_X86_64_GOTPCREL:
> > + val = find_got_kernel_entry(sym, rel);
> > + if (!val)
> > + goto unexpected_got_reference;
> > + fallthrough;
> > +#endif
> > case R_X86_64_PC32:
> > case R_X86_64_PLT32:
> > val -= (u64)loc;
> > @@ -214,11 +234,18 @@ static int __write_relocate_add(Elf64_Shdr *sechdrs,
> > }
> > return 0;
> >
> > +#ifdef CONFIG_X86_PIE
> > +unexpected_got_reference:
> > + pr_err("Target got entry doesn't exist in kernel got, loc %p\n", loc);
> > + return -ENOEXEC;
> > +#else
> > overflow:
> > pr_err("overflow in relocation type %d val %Lx\n",
> > (int)ELF64_R_TYPE(rel[i].r_info), val);
> > pr_err("`%s' likely not compiled with -mcmodel=kernel\n",
> > me->name);
> > +#endif
> > +
> > return -ENOEXEC;
> > }
> >
> > --
> > 2.31.1
> >
next prev parent reply other threads:[~2023-05-08 8:32 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-28 9:50 [PATCH RFC 00/43] x86/pie: Make kernel image's virtual address flexible Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 01/43] x86/crypto: Adapt assembly for PIE support Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 02/43] x86: Add macro to get symbol address " Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 03/43] x86: relocate_kernel - Adapt assembly " Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 04/43] x86/entry/64: " Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 05/43] x86: pm-trace: " Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 06/43] x86/CPU: " Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 07/43] x86/acpi: " Hou Wenlong
2023-04-28 11:32 ` Rafael J. Wysocki
2023-04-28 9:50 ` [PATCH RFC 08/43] x86/boot/64: " Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 09/43] x86/power/64: " Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 10/43] x86/alternatives: " Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 11/43] x86/irq: " Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 12/43] x86,rethook: " Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 13/43] x86/paravirt: Use relative reference for original instruction Hou Wenlong
2023-06-01 9:29 ` Juergen Gross
2023-06-05 6:40 ` Nadav Amit
2023-06-06 11:35 ` Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 14/43] x86/Kconfig: Introduce new Kconfig for PIE kernel building Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 15/43] x86/PVH: Use fixed_percpu_data to set up GS base Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 16/43] x86-64: Use per-cpu stack canary if supported by compiler Hou Wenlong
2023-05-01 17:27 ` Nick Desaulniers
2023-05-05 6:14 ` Hou Wenlong
2023-05-05 18:02 ` Nick Desaulniers
2023-05-05 19:06 ` Fangrui Song
2023-05-08 8:06 ` Hou Wenlong
2023-05-04 10:31 ` Juergen Gross
2023-05-05 3:09 ` Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 17/43] x86/pie: Enable stack protector only if per-cpu stack canary is supported Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 18/43] x86/percpu: Use PC-relative addressing for percpu variable references Hou Wenlong
2023-04-28 9:50 ` [PATCH RFC 19/43] x86/tools: Explicitly include autoconf.h for hostprogs Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 20/43] x86/percpu: Adapt percpu references relocation for PIE support Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 21/43] x86/ftrace: Adapt assembly " Hou Wenlong
2023-04-28 13:37 ` Steven Rostedt
2023-04-29 3:43 ` Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 22/43] x86/ftrace: Adapt ftrace nop patching " Hou Wenlong
2023-04-28 13:44 ` Steven Rostedt
2023-04-29 3:38 ` Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 23/43] x86/pie: Force hidden visibility for all symbol references Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 24/43] x86/boot/compressed: Adapt sed command to generate voffset.h when PIE is enabled Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 25/43] x86/mm: Make the x86 GOT read-only Hou Wenlong
2023-04-30 14:23 ` Ard Biesheuvel
2023-05-08 11:40 ` Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 26/43] x86/pie: Add .data.rel.* sections into link script Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 27/43] x86/relocs: Handle PIE relocations Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 28/43] KVM: x86: Adapt assembly for PIE support Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 29/43] x86/PVH: Adapt PVH booting " Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 30/43] x86/bpf: Adapt BPF_CALL JIT codegen " Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 31/43] x86/modules: Adapt module loading " Hou Wenlong
2023-04-28 19:29 ` Ard Biesheuvel
2023-05-08 8:32 ` Hou Wenlong [this message]
2023-05-08 9:16 ` Ard Biesheuvel
2023-05-08 11:40 ` Hou Wenlong
2023-05-08 17:47 ` Ard Biesheuvel
2023-05-09 9:42 ` Hou Wenlong
2023-05-09 9:52 ` Ard Biesheuvel
2023-05-09 12:35 ` Hou Wenlong
2023-05-10 7:09 ` Hou Wenlong
2023-05-10 8:15 ` Ard Biesheuvel
2023-04-28 9:51 ` [PATCH RFC 32/43] x86/boot/64: Use data relocation to get absloute address when PIE is enabled Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 33/43] objtool: Add validation for x86 PIE support Hou Wenlong
2023-04-28 10:28 ` Christophe Leroy
2023-04-28 11:43 ` Peter Zijlstra
2023-04-29 4:04 ` Hou Wenlong
2023-04-29 3:52 ` Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 34/43] objtool: Adapt indirect call of __fentry__() for " Hou Wenlong
2023-04-28 15:18 ` Peter Zijlstra
2023-04-28 9:51 ` [PATCH RFC 35/43] x86/pie: Build the kernel as PIE Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 36/43] x86/vsyscall: Don't use set_fixmap() to map vsyscall page Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 37/43] x86/xen: Pin up to VSYSCALL_ADDR when vsyscall page is out of fixmap area Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 38/43] x86/fixmap: Move vsyscall page " Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 39/43] x86/fixmap: Unify FIXADDR_TOP Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 40/43] x86/boot: Fill kernel image puds dynamically Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 41/43] x86/mm: Sort address_markers array when X86 PIE is enabled Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 42/43] x86/pie: Allow kernel image to be relocated in top 512G Hou Wenlong
2023-04-28 9:51 ` [PATCH RFC 43/43] x86/boot: Extend relocate range for PIE kernel image Hou Wenlong
2023-04-28 15:22 ` [PATCH RFC 00/43] x86/pie: Make kernel image's virtual address flexible Peter Zijlstra
2023-05-06 7:19 ` Hou Wenlong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230508083223.GA116442@k08j02272.eu95sqa \
--to=houwenlong.hwl@antgroup.com \
--cc=Jason@zx2c4.com \
--cc=ardb@kernel.org \
--cc=bp@alien8.de \
--cc=dave.hansen@linux.intel.com \
--cc=gregkh@linuxfoundation.org \
--cc=hpa@zytor.com \
--cc=jiangshan.ljs@antgroup.com \
--cc=julian.pidancet@oracle.com \
--cc=keescook@chromium.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=pmladek@suse.com \
--cc=song@kernel.org \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®