From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752769AbbCaWL7 (ORCPT ); Tue, 31 Mar 2015 18:11:59 -0400 Received: from userp1040.oracle.com ([156.151.31.81]:50430 "EHLO userp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751496AbbCaWL5 (ORCPT ); Tue, 31 Mar 2015 18:11:57 -0400 From: Sasha Levin To: linux-kernel@vger.kernel.org Cc: mhocko@suse.cz, Sasha Levin , Andrew Morton , Mel Gorman , Vlastimil Babka , Johannes Weiner , David Rientjes , Joonsoo Kim , linux-mm@kvack.org (open list:MEMORY MANAGEMENT) Subject: [PATCH 1/2] mm: free large amount of 0-order pages in workqueue Date: Tue, 31 Mar 2015 18:11:32 -0400 Message-Id: <1427839895-16434-1-git-send-email-sasha.levin@oracle.com> X-Mailer: git-send-email 2.1.0 X-Source-IP: userv0022.oracle.com [156.151.31.74] Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Freeing pages became a rather costly operation, specially when multiple debug options are enabled. This causes hangs when an attempt to free a large amount of 0-order is made. Two examples are vfree()ing large block of memory, and punching a hole in a shmem filesystem. To avoid that, move any free operations that involve batching pages into a list to a workqueue handler where they could be freed later. Signed-off-by: Sasha Levin --- mm/page_alloc.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 46 insertions(+), 4 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 5bd9711..812ca75 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1586,10 +1586,11 @@ out: local_irq_restore(flags); } -/* - * Free a list of 0-order pages - */ -void free_hot_cold_page_list(struct list_head *list, bool cold) +static LIST_HEAD(free_hot_page_list); +static LIST_HEAD(free_cold_page_list); +static DEFINE_SPINLOCK(free_page_lock); + +static void __free_hot_cold_page_list(struct list_head *list, bool cold) { struct page *page, *next; @@ -1599,6 +1600,47 @@ void free_hot_cold_page_list(struct list_head *list, bool cold) } } +static void free_page_lists_work(struct work_struct *work) +{ + LIST_HEAD(hot_pages); + LIST_HEAD(cold_pages); + unsigned long flags; + + spin_lock_irqsave(&free_page_lock, flags); + list_cut_position(&hot_pages, &free_hot_page_list, + free_hot_page_list.prev); + list_cut_position(&cold_pages, &free_cold_page_list, + free_cold_page_list.prev); + spin_unlock_irqrestore(&free_page_lock, flags); + + __free_hot_cold_page_list(&hot_pages, false); + __free_hot_cold_page_list(&cold_pages, true); +} + +static DECLARE_WORK(free_page_work, free_page_lists_work); + +/* + * Free a list of 0-order pages + */ +void free_hot_cold_page_list(struct list_head *list, bool cold) +{ + unsigned long flags; + + if (unlikely(!keventd_up())) { + __free_hot_cold_page_list(list, cold); + return; + } + + spin_lock_irqsave(&free_page_lock, flags); + if(cold) + list_splice_tail(list, &free_cold_page_list); + else + list_splice_tail(list, &free_hot_page_list); + spin_unlock_irqrestore(&free_page_lock, flags); + + schedule_work(&free_page_work); +} + /* * split_page takes a non-compound higher-order page, and splits it into * n (1<