From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753359AbcKHPVx (ORCPT ); Tue, 8 Nov 2016 10:21:53 -0500 Received: from mail-eopbgr00106.outbound.protection.outlook.com ([40.107.0.106]:23456 "EHLO EUR02-AM5-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751455AbcKHPVv (ORCPT ); Tue, 8 Nov 2016 10:21:51 -0500 X-Greylist: delayed 18769 seconds by postgrey-1.27 at vger.kernel.org; Tue, 08 Nov 2016 10:21:50 EST Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=aryabinin@virtuozzo.com; From: Andrey Ryabinin To: Christoph Hellwig CC: Andrew Morton , , , , "Andrey Ryabinin" , Andy Lutomirski , "Joel Fernandes" , Jisheng Zhang , "Chris Wilson" , John Dias , "Thomas Gleixner" , Ingo Molnar , "H. Peter Anvin" , Subject: [PATCH 1/3] mm/vmalloc: add vfree_atomic() Date: Tue, 8 Nov 2016 18:05:43 +0300 Message-ID: <1478617545-8443-1-git-send-email-aryabinin@virtuozzo.com> X-Mailer: git-send-email 2.7.3 In-Reply-To: <20161107150947.GA11279@lst.de> References: <20161107150947.GA11279@lst.de> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [195.214.232.10] X-ClientProxiedBy: VI1PR0501CA0007.eurprd05.prod.outlook.com (10.172.9.145) To VI1PR0801MB2062.eurprd08.prod.outlook.com (10.173.74.147) X-MS-Office365-Filtering-Correlation-Id: 324f0b50-c7fe-4d0c-2289-08d407e8b816 X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB2062;2:dAPbi66HNKBWazGWNQ7xkahkEJoRjSYNVmybOZCybpGEeVMF3bQbhqkqVZXstIn+S28cMBEmDFROsQStl7gTMK3Sq5Id4BbUd22uQ1Yb8awJh+JZ6+hodZXCK2f//pxj8hoULe4ZuKMTZw2lgwUQL1JSHtpMxMdFzTh3GPG70gRh/pC6/F07KoNIB1TvNOFmYZuvz85Jaca+J/6Bw7mv1A==;3:vWdhoQZ/vl3nFVQJIoWEehGL0QMvbVKinjl5BlCYUXt+YCfBIAJLx116r1o4l/0IJXy2E17gTcpdvKmER15U/uck0W/3Ii/gTIxIbO9FXmx0y/gOuLcjsD+mwg562GmI/Q/sMoZe80RxD/RxkCVPvQ== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:VI1PR0801MB2062; X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB2062;25:9qMf+J2LzzJeKtLfThVkRyZRI1u7Mx3KiqsDp54kfMalBJo+ezygKiFpJzuViqCevV7tchJEOWeC3DZDCg9mfxn98sIYtUyRcWDKNZekLf/kzxm3eP6+QGqTGDmOPilB3ui3Z3jSLVNPOC5e6Ug1athXg9wLMkI32v/oelPCT8s0cgtu7JG3AJPMb4hm6hOvqljE6f5xpZxQFQlucbHwp/O/aKTpsub8JKlQ81qZdIfsiJd0ezNrK8wjeTZ3TdtDVoUJQbq7oQq0OHuMuHJm6loIjTSGIzp8k1L+ZysvKm55kIy/uUi2hq8d0s6BpbhhYKpaGiNPo3on3SUfPyCeDze3hkiGlm6TYysBeMfYcrWuGHwf/YR+7NbilRZS5qUntgsOmE2DpM5g2a2fB7ilQ1YoGN0jrKJ4cOTNucU4ss0JHfALlw/Tgj2rE7YdCBfqK+lLVlTfzi+hDzXYwrAKJH/B973khGHw8KNHy/Qkig+znFUzsSX7TUBto6AEGFzlRy/5QE36deW1z0V0pWuP9CqOa5WatIr6siAD9uWUMUWTqNXJp7Y0vDjuIrt5mnKJShmkhc51vRJe1K5cT7y0dccA0VXzxJXLVZPt/zhEjOGvQZ8fogaRTRECHzC8vCOS5zQXqdkbC2YOTkaf6y8wotCz730BCedqzUmMxTlsWeC7Wrd0Ex58+k/qByCMxi8Lh2a2mXuVpU+p4pd00vnzWxmgHdyrH/FPTiS9kuGBXjkRbCwRPMX0ruZeT88tCb3lB9xA2/qhSCL96ZJJ+aPE9w== X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB2062;31:oD2bvOl4sEmqmcTl0ZqIGlCgxHwfU3qmhYjwwA81p4hwoLERUI42ub3ipV/+0Gjn/I/71AGcfaB6E6wRiqjLOSJjl3vHFmDi57X6Mt/aoZmVtBPk1ark0x3er8ojyRs/ejRwHAMLs51ckESQ8ps8zwQOseagJmKpCsISxNV3fIlDsRTexp7MTHMJ88ewNr6SAcC0jgkzADaf07h6psHTfPACiRwcgWbbEZgiQ0TKAbzuDZpbxztPEDNuFgPkcXLnfuxjRWWgGPw03VTx2jeSyw==;20:7ErAC3OiC6MrrT0pkFgux4Eli8yDi7xnX/kiMn5ZdnaZ/nBoyxcupSk1lWju9etlq2ELSZz5tf/mUk2iUANd1SbaYhL5jqsyjOwk/1b8XEUp5g/VyM+FhGHIt8KU2KZFtSjZfNLq9cRU7I9YF9rYY4K/NCI/yhzsUfvgAKAa3BSwVwXUbiHrxXHr6uWd8jm9EWB4j535+JLNDMToHZdafKZtYXee14Un/d7SHzZFphMHyEWxgDugbLQyuekA4/U6 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(211936372134217); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6043046)(6042046);SRVR:VI1PR0801MB2062;BCL:0;PCL:0;RULEID:;SRVR:VI1PR0801MB2062; X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB2062;4:j4PhSr8N/TK2/pYlsQN26TgwNa7HW1L2Xc5bq/gxz+9qTI1R9aSZDqAgxvS+y28T3Hf4nkjuPW+2cuzPnIqH1pxbwdp931bIL4MC/F2gWpS8iPgWWtCr+Tor/VIORUYxdgN8omnnxYmivkfE9r/f+0+UQOUt2vQ97pGJapT+oJwbi4Zi5A6pjw6jXPRfCTewryf+ELttkxoRGrm7MKH/zJs0yjc7jmILROP7f5Gc0QNZVqA64bBzQBcf9TVxESezESY6MYndTlUHpH6dCKo4ulqP0AYxchGP0FYPT5zLwMm0L1Nl6Bb7dAW2KBmRrN66x8gzcgLDofjKeBwo3HKjSmnc3dTlPdje9VDlVlKixjZli0v9Wavs0VsmNmdyejSKAV5Sq+XEa0TswG4k4wSAX4FQ8BIwT2yOJp8nlbVsnOjnFNWxpv+jdKyPeAW4XkbR1TJ1V7Jik66UeGTO72TOv8Ziuq3kWIgCy3MnI+dH2V0= X-Forefront-PRVS: 01208B1E18 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(6069001)(7916002)(199003)(189002)(3846002)(189998001)(50226002)(2906002)(110136003)(47776003)(66066001)(6916009)(4326007)(5660300001)(7846002)(69596002)(101416001)(36756003)(86362001)(2950100002)(575784001)(68736007)(7736002)(81166006)(50466002)(77096005)(81156014)(33646002)(5003940100001)(8676002)(48376002)(92566002)(7416002)(105586002)(53416004)(586003)(106356001)(6116002)(50986999)(229853001)(76506005)(97736004)(76176999)(305945005)(42186005);DIR:OUT;SFP:1102;SCL:1;SRVR:VI1PR0801MB2062;H:localhost.sw.ru;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;VI1PR0801MB2062;23:K50wWUqO8erwoOUxGPYE+SjQbXgUw+KcwmrSj12?= =?us-ascii?Q?TzigVjlAV07+wmnzau9oT6AtqT0DYHclyZlU7C2En6epBLoOoNK4zyWSyL0F?= =?us-ascii?Q?vDcFOxOpu6dqya291s75Y09quDDfjUBczHyNbOpUXLO1GrpbJeM1++hFW9bX?= =?us-ascii?Q?B843k1CSe5VHslpjCjLCwcRJfyXvINLu5TsGZeJPZ0iSfP7K/2bZISE2i+Tb?= =?us-ascii?Q?Anv71+XnEdcnsm9Ou1voCy3RkW6zn8xvkm6XCBSNhh310BOrLyDWrLX5SVcO?= =?us-ascii?Q?YcNmyhsygusGavRIidJzrMf3KaAqdnD6Hfkw1PoDAs4+fqFJJBcuYio8DrxT?= =?us-ascii?Q?+ZGc6YhDxqI7aJAgZng5mYeULz56eTZqiM4+a+ReQ/Z8V5c1Lh5Uohg8wXw5?= =?us-ascii?Q?DH9qzdvK74UTe7iJdK6O4iaHnXCcFhpesdIhp1rsIyyEjipCCgjGhcVKGCa2?= =?us-ascii?Q?2nZsweTW4SMowGj2MNG00ZPn8Ca/L37uG/5O7SCyeVzkAt0VVvqRPCGVxg+T?= =?us-ascii?Q?uCozERvAOp4QhcW3nMG3QesejnF91e9lNJmQ4Wk+UYpDARNeQbjKrGOspWHy?= =?us-ascii?Q?LjZoJDve57ivEw+spjXILVxyGYnHbsthZm9+FbHPwGtL94XLOD24k2rStuyc?= =?us-ascii?Q?0otUYK0V+ydx0+Vkx2F+l7k+SxhGjgY4mq8ZqCcKl0SndBaBnxhcdc0F1HZT?= =?us-ascii?Q?GOC/q/weKbDf1cod67SNxyrMkBTsNEnT4ueiP0+jaJvK9AVsm8fCyoWLhTug?= =?us-ascii?Q?gzcBWa2FeDldSH8eESu/Mh0NwsdPVBgTQ4FrvSDTrDxgrG32B3bxw9lNFYAt?= =?us-ascii?Q?LqNUcDOl+Bz+kN5AWaHy8Y/Sq9xkZnRfTllvQYMKpMieoxOMmXKR+Bf6/+ny?= =?us-ascii?Q?CVnyIb4epgiwl+HXJkjtq6E8JxVJdZVI8N95Fc5Hz1Sj3l8wDZXQiQQQWUCV?= =?us-ascii?Q?HBb4PHR0+KoeaysXxCgOP3YQbbtNV5nyQbg4UHbaCPKva35mWAn6rW2ebwpm?= =?us-ascii?Q?6iveaB9vko+8dcxaJDjSQSxyf1agkKANyFawwwD4enHvQEVcNa3SXmAXLM8P?= =?us-ascii?Q?0o4KUajz2PuNbSFeWZn9whNuLo/ACtddNl2IdfnQCi1no1JQGvrbAxx+1YnD?= =?us-ascii?Q?UMx6fHl3Er/Sgpexu4bpenuaoB5nZ7aSMrO2MYcx9Tw9UXGjESsX3tw=3D?= =?us-ascii?Q?=3D?= X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB2062;6:12Nzg9xft2AughwRzud8t/AkiotBoBaSpMiZprbhHXf61of6CJwlyHZSHh6ZooZ7d6pXbjVg5NokcoUmy0EyrD1mz76reAklPUbKPZq4BevpwcSXxCVT8DPZl699jDDmOVmViYxobyihPFenhBtejnWQfcwhreLWtKNZ8h3TL7feTLjUlMbNYfZSzZKxQZcq5sdHKFuYpAucesPFGnGzjtL/ARLO29zYBV/ManzwVLktaccb3TLCtPLFPHEZhxPtm1ywpIQgast/VQnCpxNXG/avtGIbo2mDNSjLeSf2KNCB0IV/4eJaV/8fMUnaG6FWRUcrl2kZxSRZZKJ0ya4hWQ==;5:LvrSmWYDtwhmWPD3XZa8jwfbE4pQ+mkbWQzT0MPzKtLYT4i0IQpotLz7An9Q7kLED4SoI+SI4/DKBX32Eq1TlKKusqVm/Iv1d+nk9fc2P2QIdlT0uWrLtQUZDKY0zPMUVq61x1Lte6NOq+GOtuYOLw==;24:brKkd1mmR3IsJfCU7MCPFaHQdWj1AlAHwB5YQpyDMQes0qKAN3lCyqcj1/66RhAdJb68R9yU/uYeUGgwjbF+KpssuHsJJJfrsMcXUStfk+I= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB2062;7:D/ivsI5mx5XusjRn9UO5fEkBMUmcH569CEbkY3vfse7mHCbydk8aXBbrTfRrakqgh2uymYOrg8ljpxrGwkhxfEpIs8e0TjVyDPeejH7DtwIZKlLqz0Xul6OK0fsfCkNP/odKisL5X9WE1PFm+0KvZ49NURlGVg7OZbctSlseUIQYUjog9b42vSifuZtNgIWS2MzXXr8bM7FvYbJLZgZ24jf3BRj5mQW3mZKvmCAMuKdAV4cebvOLIE8httE5C19TqytjZwoeJMLl1F4eS/8EYY+Knk0yxIxRX0K/UF2m6KmYAKGFRdjzu4toWFWVLI6rw9EGVUvV6YMUC4CeMygeE6LNLxhON6QiNbENUiMc+w0=;20:iy0YFfago/2TwJAbjgDK4ChNDeMma9g6h5ojBbkjSxr7N4RJc9KnwQxrDRYLfG11Zkv5QkmtqmZlfsrMkjV4i0Z2LTV7smbmIKZfD0EnNFuuuxdceSg71lubk7KPG0G1sTHeg/eFmZMUDk6ssRk8SsuYJPLa6Rv5dwT3MPU9hUs= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2016 15:05:45.7188 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB2062 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We are going to use sleeping lock for freeing vmap. However some vfree() users want to free memory from atomic (but not from interrupt) context. For this we add vfree_atomic() - deferred variation of vfree() which can be used in any atomic context (except NMIs). Signed-off-by: Andrey Ryabinin Cc: Andy Lutomirski Cc: Joel Fernandes Cc: Christoph Hellwig Cc: Jisheng Zhang Cc: Chris Wilson Cc: John Dias Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: x86@kernel.org --- include/linux/vmalloc.h | 1 + mm/vmalloc.c | 36 ++++++++++++++++++++++++++++++------ 2 files changed, 31 insertions(+), 6 deletions(-) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 3d9d786..d68edff 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -82,6 +82,7 @@ extern void *__vmalloc_node_range(unsigned long size, unsigned long align, const void *caller); extern void vfree(const void *addr); +extern void vfree_atomic(const void *addr); extern void *vmap(struct page **pages, unsigned int count, unsigned long flags, pgprot_t prot); diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 719ced3..b0edc67 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -1471,7 +1471,33 @@ static void __vunmap(const void *addr, int deallocate_pages) kfree(area); return; } - + +static inline void __vfree_deferred(const void *addr) +{ + struct vfree_deferred *p = this_cpu_ptr(&vfree_deferred); + + if (llist_add((struct llist_node *)addr, &p->list)) + schedule_work(&p->wq); +} + +/** + * vfree_atomic - release memory allocated by vmalloc() + * @addr: memory base address + * + * This one is just like vfree() but can be called in any atomic context + * except NMIs. + */ +void vfree_atomic(const void *addr) +{ + BUG_ON(in_nmi()); + + kmemleak_free(addr); + + if (!addr) + return; + __vfree_deferred(addr); +} + /** * vfree - release memory allocated by vmalloc() * @addr: memory base address @@ -1494,11 +1520,9 @@ void vfree(const void *addr) if (!addr) return; - if (unlikely(in_interrupt())) { - struct vfree_deferred *p = this_cpu_ptr(&vfree_deferred); - if (llist_add((struct llist_node *)addr, &p->list)) - schedule_work(&p->wq); - } else + if (unlikely(in_interrupt())) + __vfree_deferred(addr); + else __vunmap(addr, 1); } EXPORT_SYMBOL(vfree); -- 2.7.3