From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8844CC433E0 for ; Wed, 3 Mar 2021 16:33:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 48734614A7 for ; Wed, 3 Mar 2021 16:33:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1573284AbhCCQ3W (ORCPT ); Wed, 3 Mar 2021 11:29:22 -0500 Received: from pegase1.c-s.fr ([93.17.236.30]:58924 "EHLO pegase1.c-s.fr" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350158AbhCCLvz (ORCPT ); Wed, 3 Mar 2021 06:51:55 -0500 Received: from localhost (mailhub1-int [192.168.12.234]) by localhost (Postfix) with ESMTP id 4Dr9tR6h0Xz9tygN; Wed, 3 Mar 2021 11:57:03 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [192.168.12.234]) (amavisd-new, port 10024) with ESMTP id i-a1Nz-ZudRo; Wed, 3 Mar 2021 11:57:03 +0100 (CET) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 4Dr9tR5VLxz9tyZS; Wed, 3 Mar 2021 11:57:03 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 9A0638B7CD; Wed, 3 Mar 2021 11:56:50 +0100 (CET) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id 9UicWIe8FC3n; Wed, 3 Mar 2021 11:56:47 +0100 (CET) Received: from [192.168.4.90] (unknown [192.168.4.90]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 3B6948B7D8; Wed, 3 Mar 2021 11:56:30 +0100 (CET) Subject: Re: [RFC PATCH v1] powerpc: Enable KFENCE for PPC32 To: Marco Elver Cc: Alexander Potapenko , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Dmitry Vyukov , LKML , linuxppc-dev@lists.ozlabs.org, kasan-dev References: <51c397a23631d8bb2e2a6515c63440d88bf74afd.1614674144.git.christophe.leroy@csgroup.eu> <3abbe4c9-16ad-c168-a90f-087978ccd8f7@csgroup.eu> From: Christophe Leroy Message-ID: Date: Wed, 3 Mar 2021 11:56:28 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: fr Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Le 03/03/2021 à 11:39, Marco Elver a écrit : > On Wed, 3 Mar 2021 at 11:32, Christophe Leroy > wrote: >> >> >> >> Le 02/03/2021 à 10:53, Marco Elver a écrit : >>> On Tue, 2 Mar 2021 at 10:27, Christophe Leroy >>> wrote: >>>> Le 02/03/2021 à 10:21, Alexander Potapenko a écrit : >>>>>> [ 14.998426] BUG: KFENCE: invalid read in finish_task_switch.isra.0+0x54/0x23c >>>>>> [ 14.998426] >>>>>> [ 15.007061] Invalid read at 0x(ptrval): >>>>>> [ 15.010906] finish_task_switch.isra.0+0x54/0x23c >>>>>> [ 15.015633] kunit_try_run_case+0x5c/0xd0 >>>>>> [ 15.019682] kunit_generic_run_threadfn_adapter+0x24/0x30 >>>>>> [ 15.025099] kthread+0x15c/0x174 >>>>>> [ 15.028359] ret_from_kernel_thread+0x14/0x1c >>>>>> [ 15.032747] >>>>>> [ 15.034251] CPU: 0 PID: 111 Comm: kunit_try_catch Tainted: G B >>>>>> 5.12.0-rc1-s3k-dev-01534-g4f14ae75edf0-dirty #4674 >>>>>> [ 15.045811] ================================================================== >>>>>> [ 15.053324] # test_invalid_access: EXPECTATION FAILED at mm/kfence/kfence_test.c:636 >>>>>> [ 15.053324] Expected report_matches(&expect) to be true, but is false >>>>>> [ 15.068359] not ok 21 - test_invalid_access >>>>> >>>>> The test expects the function name to be test_invalid_access, i. e. >>>>> the first line should be "BUG: KFENCE: invalid read in >>>>> test_invalid_access". >>>>> The error reporting function unwinds the stack, skips a couple of >>>>> "uninteresting" frames >>>>> (https://elixir.bootlin.com/linux/v5.12-rc1/source/mm/kfence/report.c#L43) >>>>> and uses the first "interesting" one frame to print the report header >>>>> (https://elixir.bootlin.com/linux/v5.12-rc1/source/mm/kfence/report.c#L226). >>>>> >>>>> It's strange that test_invalid_access is missing altogether from the >>>>> stack trace - is that expected? >>>>> Can you try printing the whole stacktrace without skipping any frames >>>>> to see if that function is there? >>>>> >>>> >>>> Booting with 'no_hash_pointers" I get the following. Does it helps ? >>>> >>>> [ 16.837198] ================================================================== >>>> [ 16.848521] BUG: KFENCE: invalid read in finish_task_switch.isra.0+0x54/0x23c >>>> [ 16.848521] >>>> [ 16.857158] Invalid read at 0xdf98800a: >>>> [ 16.861004] finish_task_switch.isra.0+0x54/0x23c >>>> [ 16.865731] kunit_try_run_case+0x5c/0xd0 >>>> [ 16.869780] kunit_generic_run_threadfn_adapter+0x24/0x30 >>>> [ 16.875199] kthread+0x15c/0x174 >>>> [ 16.878460] ret_from_kernel_thread+0x14/0x1c >>>> [ 16.882847] >>>> [ 16.884351] CPU: 0 PID: 111 Comm: kunit_try_catch Tainted: G B >>>> 5.12.0-rc1-s3k-dev-01534-g4f14ae75edf0-dirty #4674 >>>> [ 16.895908] NIP: c016eb8c LR: c02f50dc CTR: c016eb38 >>>> [ 16.900963] REGS: e2449d90 TRAP: 0301 Tainted: G B >>>> (5.12.0-rc1-s3k-dev-01534-g4f14ae75edf0-dirty) >>>> [ 16.911386] MSR: 00009032 CR: 22000004 XER: 00000000 >>>> [ 16.918153] DAR: df98800a DSISR: 20000000 >>>> [ 16.918153] GPR00: c02f50dc e2449e50 c1140d00 e100dd24 c084b13c 00000008 c084b32b c016eb38 >>>> [ 16.918153] GPR08: c0850000 df988000 c0d10000 e2449eb0 22000288 >>>> [ 16.936695] NIP [c016eb8c] test_invalid_access+0x54/0x108 >>>> [ 16.942125] LR [c02f50dc] kunit_try_run_case+0x5c/0xd0 >>>> [ 16.947292] Call Trace: >>>> [ 16.949746] [e2449e50] [c005a5ec] finish_task_switch.isra.0+0x54/0x23c (unreliable) >>> >>> The "(unreliable)" might be a clue that it's related to ppc32 stack >>> unwinding. Any ppc expert know what this is about? >>> >>>> [ 16.957443] [e2449eb0] [c02f50dc] kunit_try_run_case+0x5c/0xd0 >>>> [ 16.963319] [e2449ed0] [c02f63ec] kunit_generic_run_threadfn_adapter+0x24/0x30 >>>> [ 16.970574] [e2449ef0] [c004e710] kthread+0x15c/0x174 >>>> [ 16.975670] [e2449f30] [c001317c] ret_from_kernel_thread+0x14/0x1c >>>> [ 16.981896] Instruction dump: >>>> [ 16.984879] 8129d608 38e7eb38 81020280 911f004c 39000000 995f0024 907f0028 90ff001c >>>> [ 16.992710] 3949000a 915f0020 3d40c0d1 3d00c085 <8929000a> 3908adb0 812a4b98 3d40c02f >>>> [ 17.000711] ================================================================== >>>> [ 17.008223] # test_invalid_access: EXPECTATION FAILED at mm/kfence/kfence_test.c:636 >>>> [ 17.008223] Expected report_matches(&expect) to be true, but is false >>>> [ 17.023243] not ok 21 - test_invalid_access >>> >>> On a fault in test_invalid_access, KFENCE prints the stack trace based >>> on the information in pt_regs. So we do not think there's anything we >>> can do to improve stack printing pe-se. >>> >>> What's confusing is that it's only this test, and none of the others. >>> Given that, it might be code-gen related, which results in some subtle >>> issue with stack unwinding. There are a few things to try, if you feel >>> like it: >>> >>> -- Change the unwinder, if it's possible for ppc32. >>> >>> -- Add code to test_invalid_access(), to get the compiler to emit >>> different code. E.g. add a bunch (unnecessary) function calls, or add >>> barriers, etc. >>> >>> -- Play with compiler options. We already pass >>> -fno-optimize-sibling-calls for kfence_test.o to avoid tail-call >>> optimizations that'd hide stack trace entries. But perhaps there's >>> something ppc-specific we missed? >>> >>> Well, the good thing is that KFENCE detects the bad access just fine. >>> Since, according to the test, everything works from KFENCE's side, I'd >>> be happy to give my Ack: >>> >>> Acked-by: Marco Elver >>> >> >> Thanks. >> >> For you information, I've got a pile of warnings from mm/kfence/report.o . Is that expected ? >> >> CC mm/kfence/report.o >> In file included from ./include/linux/printk.h:7, >> from ./include/linux/kernel.h:16, >> from mm/kfence/report.c:10: >> mm/kfence/report.c: In function 'kfence_report_error': >> ./include/linux/kern_levels.h:5:18: warning: format '%zd' expects argument of type 'signed size_t', >> but argument 6 has type 'ptrdiff_t' {aka 'const long int'} [-Wformat=] >> 5 | #define KERN_SOH "\001" /* ASCII Start Of Header */ >> | ^~~~~~ >> ./include/linux/kern_levels.h:11:18: note: in expansion of macro 'KERN_SOH' >> 11 | #define KERN_ERR KERN_SOH "3" /* error conditions */ >> | ^~~~~~~~ >> ./include/linux/printk.h:343:9: note: in expansion of macro 'KERN_ERR' >> 343 | printk(KERN_ERR pr_fmt(fmt), ##__VA_ARGS__) >> | ^~~~~~~~ >> mm/kfence/report.c:207:3: note: in expansion of macro 'pr_err' >> 207 | pr_err("Out-of-bounds %s at 0x%p (%luB %s of kfence-#%zd):\n", >> | ^~~~~~ >> ./include/linux/kern_levels.h:5:18: warning: format '%zd' expects argument of type 'signed size_t', >> but argument 4 has type 'ptrdiff_t' {aka 'const long int'} [-Wformat=] >> 5 | #define KERN_SOH "\001" /* ASCII Start Of Header */ >> | ^~~~~~ >> ./include/linux/kern_levels.h:11:18: note: in expansion of macro 'KERN_SOH' >> 11 | #define KERN_ERR KERN_SOH "3" /* error conditions */ >> | ^~~~~~~~ >> ./include/linux/printk.h:343:9: note: in expansion of macro 'KERN_ERR' >> 343 | printk(KERN_ERR pr_fmt(fmt), ##__VA_ARGS__) >> | ^~~~~~~~ >> mm/kfence/report.c:216:3: note: in expansion of macro 'pr_err' >> 216 | pr_err("Use-after-free %s at 0x%p (in kfence-#%zd):\n", >> | ^~~~~~ >> ./include/linux/kern_levels.h:5:18: warning: format '%zd' expects argument of type 'signed size_t', >> but argument 2 has type 'ptrdiff_t' {aka 'const long int'} [-Wformat=] >> 5 | #define KERN_SOH "\001" /* ASCII Start Of Header */ >> | ^~~~~~ >> ./include/linux/kern_levels.h:24:19: note: in expansion of macro 'KERN_SOH' >> 24 | #define KERN_CONT KERN_SOH "c" >> | ^~~~~~~~ >> ./include/linux/printk.h:385:9: note: in expansion of macro 'KERN_CONT' >> 385 | printk(KERN_CONT fmt, ##__VA_ARGS__) >> | ^~~~~~~~~ >> mm/kfence/report.c:223:3: note: in expansion of macro 'pr_cont' >> 223 | pr_cont(" (in kfence-#%zd):\n", object_index); >> | ^~~~~~~ >> ./include/linux/kern_levels.h:5:18: warning: format '%zd' expects argument of type 'signed size_t', >> but argument 3 has type 'ptrdiff_t' {aka 'const long int'} [-Wformat=] >> 5 | #define KERN_SOH "\001" /* ASCII Start Of Header */ >> | ^~~~~~ >> ./include/linux/kern_levels.h:11:18: note: in expansion of macro 'KERN_SOH' >> 11 | #define KERN_ERR KERN_SOH "3" /* error conditions */ >> | ^~~~~~~~ >> ./include/linux/printk.h:343:9: note: in expansion of macro 'KERN_ERR' >> 343 | printk(KERN_ERR pr_fmt(fmt), ##__VA_ARGS__) >> | ^~~~~~~~ >> mm/kfence/report.c:233:3: note: in expansion of macro 'pr_err' >> 233 | pr_err("Invalid free of 0x%p (in kfence-#%zd):\n", (void *)address, >> | ^~~~~~ >> >> Christophe > > No this is not expected. Is 'signed size_t' != 'long int' on ppc32? > No, it is an 'int' not a 'long int', see arch/powerpc/include/uapi/asm/posix_types.h #ifdef __powerpc64__ typedef unsigned long __kernel_old_dev_t; #define __kernel_old_dev_t __kernel_old_dev_t #else typedef unsigned int __kernel_size_t; typedef int __kernel_ssize_t; typedef long __kernel_ptrdiff_t; #define __kernel_size_t __kernel_size_t What is probably specific to powerpc is that ptrdiff_t is not same as ssize_t unlike in include/uapi/asm-generic/posix_types.h : /* * Most 32 bit architectures use "unsigned int" size_t, * and all 64 bit architectures use "unsigned long" size_t. */ #ifndef __kernel_size_t #if __BITS_PER_LONG != 64 typedef unsigned int __kernel_size_t; typedef int __kernel_ssize_t; typedef int __kernel_ptrdiff_t; #else typedef __kernel_ulong_t __kernel_size_t; typedef __kernel_long_t __kernel_ssize_t; typedef __kernel_long_t __kernel_ptrdiff_t; #endif #endif I have no warning on ppc64. Christophe