[PATCH] coredump: reduce stack usage in vfs_coredump

[PATCH] coredump: reduce stack usage in vfs_coredump

[Arnd Bergmann]

From: Arnd Bergmann <arnd@arndb.de>

A recent patch changed the coredump code in mostly harmless ways,
but after this I get a very high stack usage in one function
that was benign before:

fs/coredump.c: In function 'vfs_coredump':
fs/coredump.c:1313:1: error: the frame size of 1872 bytes is larger than 1536 bytes [-Werror=frame-larger-than=]

This happens reproducibly every couple of randconfig builds on
s390 with gcc-16 and seems to be caused by kasan checks causing
local variables to be spilled to the stack.

Marking the do_coredump function as noinline_for_stack avoids
the problem here, instead I get only 464 and 672 bytes in
the worst case for vfs_coredump and do_coredump, respectively.

Fixes: 3209535cd39e ("pidfs: record the coredump on the dumping thread's pid too")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
If anyone wants to look at it in more detail, here is the
information I have:

randconfig: https://pastebin.com/raw/bskKJixQ
assembler output: https://pastebin.com/raw/Wr1V8kRS

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
 fs/coredump.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/fs/coredump.c b/fs/coredump.c
index b25ef2ffbdee..82ef9d115999 100644
--- a/fs/coredump.c
+++ b/fs/coredump.c
@@ -1198,8 +1198,10 @@ static inline bool coredump_skip(const struct coredump_params *cprm,
 	return false;
 }
 
-static void do_coredump(struct core_name *cn, struct coredump_params *cprm,
-			size_t **argv, int *argc, const struct linux_binfmt *binfmt)
+static noinline_for_stack void do_coredump(struct core_name *cn,
+					   struct coredump_params *cprm,
+					   size_t **argv, int *argc,
+					   const struct linux_binfmt *binfmt)
 {
 	trace_coredump(cprm->siginfo->si_signo);
 
-- 
2.53.0