From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 88A5A2DB780 for ; Fri, 31 Jul 2026 04:27:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785472029; cv=none; b=LJ2z/HAFzxOREifHtaXMxt+2Bq6nOarI1Czf5ySiPbualcyylQR5vndyNPk4a/a7+PaEIX+FEfZqp6pCDIZBuqe9qUqVhigkB5TAotVYfeeFtnCuV1hAr3O09yabYFNjzhCUW4QajDWiSdLMloTmRHuGZu/yWGAf+tMqhqGAato= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785472029; c=relaxed/simple; bh=wagyUDn5onTRQ/P0/h9h5V3M/XxZgRG/XQKQ3XgotQ4=; h=Date:From:To:cc:Subject:In-Reply-To:Message-ID:References: MIME-Version:Content-Type; b=F5ief9RyIRozioErd/fVNfJgkcD0Z4pN3t0lpaZBzXH9NOfhrlStZa1WfQfZZ9/IaZ7/ujO1NYv/Sl9Pv4/pTNBVAZN3rEE1sXXYnujpkVSa1q70zvlIHrblFUrPhB9OZOW/NWaXBq58/wJUMNO1N7gC3ikoWCFNBRg2fi5L4Ao= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Ti9zFRAY; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Ti9zFRAY" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-2cede6375caso58795ad.0 for ; Thu, 30 Jul 2026 21:27:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1785472027; x=1786076827; darn=vger.kernel.org; h=content-type:mime-version:references:message-id:in-reply-to:subject :cc:to:from:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=U6jZw2r6wUXuqZBKIGC1dePei5VoJhsmtK5wvBubPfM=; b=Ti9zFRAYWehgzJDNiBqBY4oGlFMoIA3yEYIihW73SRqUOb4mUGqUvroY4mncNIbBku N0Wkp6lN6NcIQIGz6nHgLYcc0NDqk7yH4ALVbBDdCcpnRaC5cMFJF18K5zAl+KSkRHMe DMi8lua/T+p/WldthutsmNZY+4VreZc1E8bobGdh8Q16PJf1zyvMazFz8qb/21Hdk1zY VW+HepnzGGnW0DGShDOLuVpyDa7C8o8UAWgzm4D+02PvkH0L29qZDMYNSm6Cjgu+1wNt ueaHc9t1WUBGtPZX7l6OJWTfFoBHFcmswNg2fjNG2+ds4EJkpF/cwkJpFc/7MUXZpSjQ zWNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785472027; x=1786076827; h=content-type:mime-version:references:message-id:in-reply-to:subject :cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=U6jZw2r6wUXuqZBKIGC1dePei5VoJhsmtK5wvBubPfM=; b=ixJkHRSFAGHjPwMYJfph8ccDsQQSX07/zUQyn8yxjg1hbqC/hJe28RtbuTc1S+Iget m/3xLlvY2eu9hR6c8gXX04Sy/JGJeFy9ELJZNxiB15ol6RaKSxvF6E0BJi2+7Pe9J9PA jxHLQvqw3o3wI2MQ1cdIRcl6UuatMVoEfofl26lM0wluakhPQH/iRwVbAT8Wi3blwE54 yNZ1IhZuYQPMPhVbwtXqY+3emnuHKv4e2Rlr1f4yi+qY38njnfz9PkmufLxfjHB3Imwp Q5UxQMjmJwLIJIiii0Z6hOzvz57epM/TcomuOXhnuNnnvTS5j1u1nPgijzE4n3VHJWu9 GUpQ== X-Forwarded-Encrypted: i=1; AHgh+RoMlbKj0q3hBYeNd0/KGGFldQ3G0yYJazifWZvuHH7W0OGCSe6fUUCt9wZOMVGtuR/jzOh5iea2/YO/AUo=@vger.kernel.org X-Gm-Message-State: AOJu0YzZEPn/xJKqqUScAs4fxwCre7Pfh1FG3ZmEC4Nve9kzlFvb9/Ak NUzmQz/cG9eMafOZwmO+B4K87jauBht3KnQoxVoUOH2gbaQ3fnMu4efUZJHltJhNgQ== X-Gm-Gg: AR+sD10P7URSkrouGxXTdoU9FLDGOEbr2ahpfWkeoTX3Tf+uwLcX6HQ5Xtd3IU0Qja4 aEH9Wau686ncGE2eIIjEVpSCP4ORBBoUolJ3wxzDHpG4XCvsyjP9o7ADvEB91464J1v6H13JTi2 5zIzA9BZr54fOLWF4d+cZBCzgJrIj1yX8N0mjr6WCl1+jLeOQ1h3QP9PcCbZ09keUZEnnhi5ZzG Vfs6kJ2kGrvMf5PR7pAfAB8kcfHyhAFJt/PB9I1FmjJesIDDBH6HB+QTLVEJZf4xFuvFgUT5UE2 guJGiVr/u3E8RrPkJM2efS6NV4OdHe7uvIcSmGeBDYPSouebI1gWQiA6AtM2BP930aqPRF3SOSk 8qe95FqRSML8ciYf2KTce/8I+eqK5TaZD5PPyWVLMaVaq3HDwbexGOIez1ym/jVvlhIcHC4chl6 XuzjoaFg/Q3oM/tvQQjLqbHlcTfdyfU95zwJ+gzVKjBXyl+Ub1RpUTNLdJ5Whoad6/MdfLKjx08 wSopgU/tVYvD1c+ErPFw7p9JAZ2azmzAm0As80BpBGUhTY4IGYxRbWEjOdaAigAZDc/ZDgGRsJn A6LSKCoJ4OKW79TU1E9Zi8B7iw== X-Received: by 2002:a17:903:2c0f:b0:2cf:9ec1:ea6e with SMTP id d9443c01a7336-2d046e9a9f7mr1527675ad.8.1785472025985; Thu, 30 Jul 2026 21:27:05 -0700 (PDT) Received: from [2a00:79e0:2eb4:9:a061:9e19:11a8:45b6] ([2a00:79e0:2eb4:9:a061:9e19:11a8:45b6]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84ed5e30634sm193651b3a.56.2026.07.30.21.27.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Jul 2026 21:27:05 -0700 (PDT) Date: Thu, 30 Jul 2026 21:27:04 -0700 (PDT) From: David Rientjes To: Andrew Morton , David Hildenbrand , Christoph Lameter cc: Vlastimil Babka , Mathieu Desnoyers , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [patch 3/3] lib: test_pagealloc: add synthetic benchmark for page allocator In-Reply-To: Message-ID: <36750300-0785-6a4d-ec18-b3f549962301@google.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII From: Christoph Lameter Add a synthetic benchmark that can be used to measure performance of the buddy allocator, including repeatedly allocating and freeing memory as well as concurrent and remote allocations. The test is run by inserting the module: modprobe test_pagealloc The module insertion will always fail so that it is automatically unloaded; the results are in the kernel log. Signed-off-by: Christoph Lameter Signed-off-by: David Rientjes --- lib/Kconfig.debug | 10 ++ lib/Makefile | 1 + lib/test_pagealloc.c | 337 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 348 insertions(+) create mode 100644 lib/test_pagealloc.c diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index a7e738e6afa8..70873aac675f 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -3447,6 +3447,16 @@ config TEST_SLAB If unsure, say N. +config TEST_PAGEALLOC + tristate "Page Allocator Benchmark" + default n + help + A benchmark measuring the performance of the page allocator, useful + to analyze any improvements or regressions when repeatedly allocating + and freeing memory as well as concurrent and remote allocations. + + If unsure, say N. + config RATELIMIT_KUNIT_TEST tristate "KUnit Test for correctness and stress of ratelimit" if !KUNIT_ALL_TESTS depends on KUNIT diff --git a/lib/Makefile b/lib/Makefile index fe54690d130c..17c164e145ad 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -107,6 +107,7 @@ obj-$(CONFIG_TEST_OBJPOOL) += test_objpool.o obj-$(CONFIG_TEST_KEXEC_HANDOVER) += test_kho.o obj-$(CONFIG_TEST_VMSTAT) += test_vmstat.o obj-$(CONFIG_TEST_SLAB) += test_slab.o +obj-$(CONFIG_TEST_PAGEALLOC) += test_pagealloc.o obj-$(CONFIG_TEST_FPU) += test_fpu.o test_fpu-y := test_fpu_glue.o test_fpu_impl.o diff --git a/lib/test_pagealloc.c b/lib/test_pagealloc.c new file mode 100644 index 000000000000..30514df52c5b --- /dev/null +++ b/lib/test_pagealloc.c @@ -0,0 +1,337 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Test module for in kernel synthetic page allocator testing. + * + * Compiled as a module. The module needs to be loaded to run. + * + * (C) 2009 Linux Foundation, Christoph Lameter + */ + +#include +#include +#include +#include +#include +#include + +#define TEST_COUNT 1000 + +#define CONCURRENT_MAX_ORDER 6 + +#ifdef CONFIG_SMP +#include +#include +#include +#include + +static struct test_struct { + struct task_struct *task; + int cpu; + int order; + int count; + struct page **v; + void (*test_p1)(struct test_struct *t); + void (*test_p2)(struct test_struct *t); + unsigned long start1; + unsigned long stop1; + unsigned long start2; + unsigned long stop2; +} test[NR_CPUS]; + +/* + * Allocate TEST_COUNT objects on cpus > 0 and then all the + * objects later on cpu 0 + */ +static void remote_free_test_p1(struct test_struct *t) +{ + int i; + + /* Perform no allocations on cpu 0 */ + for (i = 0; i < t->count; i++) { + struct page *p; + + if (smp_processor_id()) { + p = alloc_pages(GFP_KERNEL | __GFP_COMP, t->order); + /* Use object */ + memset(page_address(p), 17, 4); + } else + p = NULL; + t->v[i] = p; + } +} + +static void remote_free_test_p2(struct test_struct *t) +{ + int i; + int cpu; + + /* All frees are completed on cpu zero */ + if (smp_processor_id()) + return; + + for_each_online_cpu(cpu) + for (i = 0; i < t->count; i++) { + struct page *p = test[cpu].v[i]; + + if (!p) + continue; + + __free_pages(p, t->order); + } +} + +/* + * Allocate TEST_COUNT objects and later free them all again + */ +static void alloc_then_free_test_p1(struct test_struct *t) +{ + int i; + + for (i = 0; i < t->count; i++) { + struct page *p = alloc_pages(GFP_KERNEL | __GFP_COMP, t->order); + + memset(page_address(p), 14, 4); + t->v[i] = p; + } +} + +static void alloc_then_free_test_p2(struct test_struct *t) +{ + int i; + + for (i = 0; i < t->count; i++) { + struct page *p = t->v[i]; + + __free_pages(p, t->order); + } +} + +/* + * Allocate TEST_COUNT objects. Free them immediately. + */ +static void alloc_free_test_p1(struct test_struct *t) +{ + int i; + + for (i = 0; i < TEST_COUNT; i++) { + struct page *p = alloc_pages(GFP_KERNEL | __GFP_COMP, t->order); + + memset(page_address(p), 12, 4); + __free_pages(p, t->order); + } +} + +static atomic_t tests_running; +static atomic_t phase1_complete; +static DECLARE_COMPLETION(completion1); +static DECLARE_COMPLETION(completion2); +static DECLARE_COMPLETION(completion3); + +static int test_func(void *private) +{ + struct test_struct *t = private; + cpumask_t newmask = CPU_MASK_NONE; + + cpumask_set_cpu(t->cpu, &newmask); + set_cpus_allowed_ptr(current, &newmask); + t->v = kmalloc_array(t->count, sizeof(struct page *), GFP_KERNEL); + + atomic_inc(&tests_running); + wait_for_completion(&completion1); + t->start1 = get_cycles(); + t->test_p1(t); + t->stop1 = get_cycles(); + atomic_inc(&phase1_complete); + wait_for_completion(&completion2); + t->start2 = get_cycles(); + if (t->test_p2) + t->test_p2(t); + t->stop2 = get_cycles(); + atomic_dec(&tests_running); + wait_for_completion(&completion3); + kfree(t->v); + while (!kthread_should_stop()) { + set_current_state(TASK_UNINTERRUPTIBLE); + schedule_timeout(10); + } + return 0; +} + +static void do_concurrent_test(void (*p1)(struct test_struct *), + void (*p2)(struct test_struct *), + int order, const char *name) +{ + int cpu; + unsigned long time1 = 0; + unsigned long time2 = 0; + unsigned long sum1 = 0; + unsigned long sum2 = 0; + + atomic_set(&tests_running, 0); + atomic_set(&phase1_complete, 0); + init_completion(&completion1); + init_completion(&completion2); + init_completion(&completion3); + + for_each_online_cpu(cpu) { + struct test_struct *t = &test[cpu]; + + t->cpu = cpu; + t->count = TEST_COUNT; + t->test_p1 = p1; + t->test_p2 = p2; + t->order = order; + t->task = kthread_run(test_func, t, "test%d", cpu); + if (IS_ERR(t->task)) { + pr_err("Failed to start test func\n"); + return; + } + } + + /* Wait till all processes are running */ + while (atomic_read(&tests_running) < num_online_cpus()) { + set_current_state(TASK_UNINTERRUPTIBLE); + schedule_timeout(10); + } + complete_all(&completion1); + + /* Wait till all processes have completed phase 1 */ + while (atomic_read(&phase1_complete) < num_online_cpus()) { + set_current_state(TASK_UNINTERRUPTIBLE); + schedule_timeout(10); + } + complete_all(&completion2); + + /* Wait till all processes have completed phase 2 */ + while (atomic_read(&tests_running)) { + set_current_state(TASK_UNINTERRUPTIBLE); + schedule_timeout(10); + } + complete_all(&completion3); + + for_each_online_cpu(cpu) + kthread_stop(test[cpu].task); + + pr_alert("%s(%d):", name, order); + for_each_online_cpu(cpu) { + struct test_struct *t = &test[cpu]; + + time1 = t->stop1 - t->start1; + time2 = t->stop2 - t->start2; + sum1 += time1; + sum2 += time2; + pr_cont(" %d=%lu", cpu, time1 / TEST_COUNT); + if (p2) + pr_cont("/%lu", time2 / TEST_COUNT); + } + pr_cont(" Average=%lu", sum1 / num_online_cpus() / TEST_COUNT); + if (p2) + pr_cont("/%lu", sum2 / num_online_cpus() / TEST_COUNT); + pr_cont("\n"); + schedule_timeout(200); +} +#endif + +static int pagealloc_test_init(void) +{ + void **v = kmalloc_array(TEST_COUNT, sizeof(void *), GFP_KERNEL); + unsigned int i; + cycles_t time1, time2, time; + int rem; + int order; + + pr_alert("test init\n"); + + pr_alert("Single thread testing\n"); + pr_alert("=====================\n"); + pr_alert("1. Repeatedly allocate then free test\n"); + for (order = 0; order <= MAX_PAGE_ORDER; order++) { + time1 = get_cycles(); + for (i = 0; i < TEST_COUNT; i++) { + struct page *p = alloc_pages(GFP_KERNEL | __GFP_COMP, + order); + + if (!p) { + pr_err("Cannot allocate order=%d\n", order); + break; + } + + /* Touch page */ + memset(page_address(p), 22, 4); + v[i] = p; + } + time2 = get_cycles(); + time = time2 - time1; + + pr_alert("%i times alloc_page(,%d) ", i, order); + time = div_u64_rem(time, TEST_COUNT, &rem); + pr_cont("-> %llu cycles ", (unsigned long long) time); + + time1 = get_cycles(); + for (i = 0; i < TEST_COUNT; i++) { + struct page *p = v[i]; + + __free_pages(p, order); + } + time2 = get_cycles(); + time = time2 - time1; + + pr_cont("__free_pages(,%d)", order); + time = div_u64_rem(time, TEST_COUNT, &rem); + pr_cont("-> %llu cycles\n", (unsigned long long) time); + } + + pr_alert("2. alloc/free test\n"); + for (order = 0; order <= MAX_PAGE_ORDER; order++) { + time1 = get_cycles(); + for (i = 0; i < TEST_COUNT; i++) { + struct page *p = alloc_pages(GFP_KERNEL | __GFP_COMP, + order); + __free_pages(p, order); + } + time2 = get_cycles(); + time = time2 - time1; + + pr_alert("%i times alloc( ,%d)/free ", i, order); + time = div_u64_rem(time, TEST_COUNT, &rem); + pr_cont("-> %llu cycles\n", (unsigned long long) time); + } + kfree(v); +#ifdef CONFIG_SMP + pr_info("Concurrent allocs\n"); + pr_info("=================\n"); + for (order = 0; order < CONCURRENT_MAX_ORDER; order++) { + do_concurrent_test(alloc_then_free_test_p1, + alloc_then_free_test_p2, + order, "Page alloc N*alloc N*free"); + } + pr_info("----Fastpath---\n"); + for (order = 0; order < CONCURRENT_MAX_ORDER; order++) { + do_concurrent_test(alloc_free_test_p1, NULL, + order, "Page N*(alloc free)"); + } + + pr_info("Remote free test\n"); + pr_info("================\n"); + for (order = 0; order < CONCURRENT_MAX_ORDER; order++) { + do_concurrent_test(remote_free_test_p1, + remote_free_test_p2, + order, "N*remote free"); + } + +#endif + + return -EAGAIN; /* Fail will directly unload the module */ +} + +static void pagealloc_test_exit(void) +{ + pr_alert("test exit\n"); +} + +module_init(pagealloc_test_init) +module_exit(pagealloc_test_exit) + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Christoph Lameter"); +MODULE_DESCRIPTION("page allocator performance test");