From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S265356AbUBBKxj (ORCPT ); Mon, 2 Feb 2004 05:53:39 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S265359AbUBBKxj (ORCPT ); Mon, 2 Feb 2004 05:53:39 -0500 Received: from sv1.valinux.co.jp ([210.128.90.2]:6089 "EHLO sv1.valinux.co.jp") by vger.kernel.org with ESMTP id S265356AbUBBKxh (ORCPT ); Mon, 2 Feb 2004 05:53:37 -0500 Date: Mon, 02 Feb 2004 19:53:35 +0900 From: IWAMOTO Toshihiro To: linux-kernel@vger.kernel.org Subject: [PATCH] fix memory leak while coredumping User-Agent: Wanderlust/2.8.1 (Something) SEMI/1.14.3 (Ushinoya) FLIM/1.14.3 (=?ISO-8859-4?Q?Unebigory=F2mae?=) APEL/10.3 Emacs/21.2 (i386-debian-linux-gnu) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.3 - "Ushinoya") Content-Type: text/plain; charset=US-ASCII Message-Id: <20040202105335.DFE88700A2@sv1.valinux.co.jp> Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Hi, with some help of coworker, I found a bug in binfmt_elf.c. The bug exists in linux-2.6.1 and linux-2.6.2-rc2-mm2. This patch fixes a memory leak that happens when a core file hits the process's resource limit. I've tested the DUMP_WRITE case only a little, and the DUMP_SEEK case is only compile tested. --- old/fs/binfmt_elf.c Fri Jan 16 12:12:24 2004 +++ new/fs/binfmt_elf.c Mon Feb 2 19:31:42 2004 @@ -1441,12 +1441,22 @@ DUMP_SEEK (file->f_pos + PAGE_SIZE); } else { if (page == ZERO_PAGE(addr)) { - DUMP_SEEK (file->f_pos + PAGE_SIZE); + if (!dump_seek(file, + file->f_pos + PAGE_SIZE)) { + page_cache_release(page); + goto end_coredump; + } } else { void *kaddr; flush_cache_page(vma, addr); kaddr = kmap(page); - DUMP_WRITE(kaddr, PAGE_SIZE); + if ((size += PAGE_SIZE) > limit || + !dump_write(file, kaddr, + PAGE_SIZE)) { + kunmap(page); + page_cache_release(page); + goto end_coredump; + } kunmap(page); } page_cache_release(page);