From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A67D91DF75A for ; Sun, 19 Jan 2025 18:06:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737309979; cv=none; b=fjnHTArqUXoARthXWReqqhcOVYMXLeB8pstL/1fA54lKhg3KKn8SJWkCgsZGypYBKCv66o16gueQFlDnwrPnw78gMb1tNUvWw5U5KPYFQu7rdjmCgP8eP1DEGnvg/k15I6P26icYgKLhQD7CPdH4z60H3AfGeRi8eRNzqbMGq44= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737309979; c=relaxed/simple; bh=sHOZfog1tpd6FIBQYremyQ/uAJ2+gAVCcFFLeRFmhMQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=nJgC237YGCeB+v31hI4EqBWZcM7g/x5MKKWydD9rnBGz2PdfghUcCUK1xmttUIcbnpHLmU+oG8mcXH3AuR42YQsDD7E+okpC03i5T6QQtsIrJBKnVXt9yiPBzdXIPohpJXhctQyFz4J9n+uQPPNeNOVV/PYr5EpCgl/94FXsxB8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jiaqiyan.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WKYTyj1V; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jiaqiyan.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WKYTyj1V" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2efa0eb9cfeso7140065a91.0 for ; Sun, 19 Jan 2025 10:06:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1737309977; x=1737914777; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=NKJABtBkaO86V03jjBLlHT4N0854yHcZTt6nQ8d1Z1A=; b=WKYTyj1VlPEEfNwDxENjFVf1yEaPj9tOcplQdnJnE9UXtqUm7DT2wOJySI06zdC9gY 0VOqmtAOQB76Wqp7xn6o7QyRy8xmzyp3m/Sw127gZXkhB7t4gtnZvSV+9JH8KM9Y/n30 fPWiCdX/eFZ0XQBQhzklzKQDVV7026lvG/7WDLU/ITxm1/VjLBqb0cMAmrhccJzzq0Vf 6ZjwIjIGYyW2gpG/fxLhS8jKmrKxTfRKvMRJ2Nk3ufnXCsirtTPq+gkyjqD8HYhXOrSp PdoyK4h+c6r2RnOTzOLHP/1xvQi13hpDINue5yxUeSkk2JljWLnR7BnFlCLytdC79dNS Ykmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737309977; x=1737914777; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NKJABtBkaO86V03jjBLlHT4N0854yHcZTt6nQ8d1Z1A=; b=rueSd+GVzNlTMr9aIc74RyE3br6wfzp3033MSNt/DXl3mJ1EBLOZdl1tl6jNYw4pTT w9t87B+eqvMwxcQp/xGYMe8fQDGVUHA8xvoxniUnGwB563f4luvbAduBVAc1haicF/yi UNMaV2Hqnvw+L10twDm6IskKZSwL3V6Hp6/pXKZ4lCaamROBifAU8+g+aqr1LTrTOXo+ 7AN1EvrIhNL7a6WYAbHdlEQmBMk1l7ksJhJ5kyuHkKju8P3i6gasY563hadnkH4j32Vy gUqoeMjQ2hwn6Q3bl7UVvHj2l7W4mOwSXh5A38Ie2g5ggJ6OoQN7OVdQTyU7ulkU5Xvf JCqw== X-Forwarded-Encrypted: i=1; AJvYcCVwQLRAScl3fMFU/uX8k3Smqj9y5XqZuqwqd6+w504BTZPJ5UugNF5qm9tIzqDKSIefsTA2nCh7rclygVo=@vger.kernel.org X-Gm-Message-State: AOJu0Yx51A4NKjGLoAi6oWfzrtsXSjZ1+DDUN6M0Tu/3N04Qtavt1+m6 3XaRzE9+dNJ0tBTDjjxdT/shvVcLOF6QWUAFt5dd1FTvXOlfytfmJWtC3ObWh4T8XXndigJm+/U ZBIo0JzuY4Q== X-Google-Smtp-Source: AGHT+IE0EsAHOx0uRDMT/3GiFXAeoNZBpraudT+237mAXqRVcZkX5PO08oN7uYZXDI5uoXbE/QjkSMRHWMVvoA== X-Received: from pjot3.prod.google.com ([2002:a17:90a:9503:b0:2ea:9d23:79a0]) (user=jiaqiyan job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4d01:b0:2ee:5a82:433a with SMTP id 98e67ed59e1d1-2f782d1f5ddmr14382391a91.17.1737309976904; Sun, 19 Jan 2025 10:06:16 -0800 (PST) Date: Sun, 19 Jan 2025 18:06:07 +0000 In-Reply-To: <20250119180608.2132296-1-jiaqiyan@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250119180608.2132296-1-jiaqiyan@google.com> X-Mailer: git-send-email 2.48.0.rc2.279.g1de40edade-goog Message-ID: <20250119180608.2132296-2-jiaqiyan@google.com> Subject: [RFC PATCH v1 1/2] selftest/mm: test HWPoison hugetlb truncation behavior From: Jiaqi Yan To: nao.horiguchi@gmail.com, linmiaohe@huawei.com, sidhartha.kumar@oracle.com, muchun.song@linux.dev Cc: jane.chu@oracle.com, akpm@linux-foundation.org, osalvador@suse.de, rientjes@google.com, jthoughton@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Jiaqi Yan Content-Type: text/plain; charset="UTF-8" Test based on my understanding of the memory failure recovery behavior for HugeTLB file system, especially after file is truncated/closed. Signed-off-by: Jiaqi Yan --- tools/testing/selftests/mm/Makefile | 1 + tools/testing/selftests/mm/hugetlb-mfr-base.c | 240 ++++++++++++++++++ 2 files changed, 241 insertions(+) create mode 100644 tools/testing/selftests/mm/hugetlb-mfr-base.c diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile index 63ce39d024bb5..576626c93ccab 100644 --- a/tools/testing/selftests/mm/Makefile +++ b/tools/testing/selftests/mm/Makefile @@ -62,6 +62,7 @@ TEST_GEN_FILES += hmm-tests TEST_GEN_FILES += hugetlb-madvise TEST_GEN_FILES += hugetlb-read-hwpoison TEST_GEN_FILES += hugetlb-soft-offline +TEST_GEN_FILES += hugetlb-mfr-base TEST_GEN_FILES += hugepage-mmap TEST_GEN_FILES += hugepage-mremap TEST_GEN_FILES += hugepage-shm diff --git a/tools/testing/selftests/mm/hugetlb-mfr-base.c b/tools/testing/selftests/mm/hugetlb-mfr-base.c new file mode 100644 index 0000000000000..b8eee071babe6 --- /dev/null +++ b/tools/testing/selftests/mm/hugetlb-mfr-base.c @@ -0,0 +1,240 @@ +// SPDX-License-Identifier: GPL-2.0 + +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +#include "../kselftest.h" +#include "vm_util.h" + +#define EPREFIX " !!! " +#define BYTE_LENTH_IN_1G 0x40000000 +#define HUGETLB_FILL 0xab + +static void *sigbus_addr; +static int sigbus_addr_lsb; +static bool expecting_sigbus; +static bool got_sigbus; +static bool was_mceerr; + +static int create_hugetlbfs_file(struct statfs *file_stat) +{ + int fd; + int flags = MFD_HUGETLB | MFD_HUGE_1GB; + + fd = memfd_create("hugetlb_tmp", flags); + if (fd < 0) + ksft_exit_fail_perror("Failed to memfd_create"); + + memset(file_stat, 0, sizeof(*file_stat)); + if (fstatfs(fd, file_stat)) { + close(fd); + ksft_exit_fail_perror("Failed to fstatfs"); + } + if (file_stat->f_type != HUGETLBFS_MAGIC) { + close(fd); + ksft_exit_fail_msg("Not hugetlbfs file"); + } + + ksft_print_msg("Created hugetlb_tmp file\n"); + ksft_print_msg("hugepagesize=%#lx\n", file_stat->f_bsize); + if (file_stat->f_bsize != BYTE_LENTH_IN_1G) + ksft_exit_fail_msg("Hugepage size is not 1G"); + + return fd; +} + +/* + * SIGBUS handler for "do_hwpoison" thread that mapped and MADV_HWPOISON + */ +static void sigbus_handler(int signo, siginfo_t *info, void *context) +{ + if (!expecting_sigbus) + ksft_exit_fail_msg("unexpected sigbus with addr=%p", + info->si_addr); + + got_sigbus = true; + was_mceerr = (info->si_code == BUS_MCEERR_AO || + info->si_code == BUS_MCEERR_AR); + sigbus_addr = info->si_addr; + sigbus_addr_lsb = info->si_addr_lsb; +} + +static void *do_hwpoison(void *hwpoison_addr) +{ + int hwpoison_size = getpagesize(); + + ksft_print_msg("MADV_HWPOISON hwpoison_addr=%p, len=%d\n", + hwpoison_addr, hwpoison_size); + if (madvise(hwpoison_addr, hwpoison_size, MADV_HWPOISON) < 0) + ksft_exit_fail_perror("Failed to MADV_HWPOISON"); + + pthread_exit(NULL); +} + +static void test_hwpoison_multiple_pages(unsigned char *start_addr) +{ + pthread_t pthread; + int ret; + unsigned char *hwpoison_addr; + unsigned long offsets[] = {0x200000}; + + for (size_t i = 0; i < ARRAY_SIZE(offsets); ++i) { + sigbus_addr = (void *)0xBADBADBAD; + sigbus_addr_lsb = 0; + was_mceerr = false; + got_sigbus = false; + expecting_sigbus = true; + hwpoison_addr = start_addr + offsets[i]; + + ret = pthread_create(&pthread, NULL, &do_hwpoison, hwpoison_addr); + if (ret) + ksft_exit_fail_perror("Failed to create hwpoison thread"); + + ksft_print_msg("Created thread to hwpoison and access hwpoison_addr=%p\n", + hwpoison_addr); + + pthread_join(pthread, NULL); + + if (!got_sigbus) + ksft_test_result_fail("Didn't get a SIGBUS\n"); + if (!was_mceerr) + ksft_test_result_fail("Didn't get a BUS_MCEERR_A(R|O)\n"); + if (sigbus_addr != hwpoison_addr) + ksft_test_result_fail("Incorrect address: got=%p, expected=%p\n", + sigbus_addr, hwpoison_addr); + if (sigbus_addr_lsb != 30) + ksft_test_result_fail("Incorrect address LSB: got=%d, expected=%d\n", + sigbus_addr_lsb, pshift()); + + ksft_print_msg("Received expected and correct SIGBUS\n"); + } +} + +static int read_nr_hugepages(unsigned long hugepage_size, + unsigned long *nr_hugepages) +{ + char buffer[256] = {0}; + char cmd[256] = {0}; + + sprintf(cmd, "cat /sys/kernel/mm/hugepages/hugepages-%ldkB/nr_hugepages", + hugepage_size); + FILE *cmdfile = popen(cmd, "r"); + + if (cmdfile == NULL) { + ksft_perror(EPREFIX "failed to popen nr_hugepages"); + return -1; + } + + if (!fgets(buffer, sizeof(buffer), cmdfile)) { + ksft_perror(EPREFIX "failed to read nr_hugepages"); + pclose(cmdfile); + return -1; + } + + *nr_hugepages = atoll(buffer); + pclose(cmdfile); + return 0; +} + +/* + * Main thread that drives the test. + */ +static void test_main(int fd, size_t len) +{ + unsigned char *map; + struct sigaction new, old; + const unsigned long hugepagesize_kb = BYTE_LENTH_IN_1G / 1024; + unsigned long nr_hugepages_before = 0; + unsigned long nr_hugepages_after = 0; + + if (read_nr_hugepages(hugepagesize_kb, &nr_hugepages_before) != 0) { + close(fd); + ksft_exit_fail_msg("Failed to read nr_hugepages\n"); + } + ksft_print_msg("NR hugepages before MADV_HWPOISON is %ld\n", nr_hugepages_before); + + if (ftruncate(fd, len) < 0) + ksft_exit_fail_perror("Failed to ftruncate"); + + ksft_print_msg("Allocated %#lx bytes to HugeTLB file\n", len); + + map = mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); + if (map == MAP_FAILED) + ksft_exit_fail_msg("Failed to mmap"); + + ksft_print_msg("Created HugeTLB mapping: %p\n", map); + + memset(map, HUGETLB_FILL, len); + ksft_print_msg("Memset every byte to 0xab\n"); + + new.sa_sigaction = &sigbus_handler; + new.sa_flags = SA_SIGINFO; + if (sigaction(SIGBUS, &new, &old) < 0) + ksft_exit_fail_msg("Failed to setup SIGBUS handler"); + + ksft_print_msg("Setup SIGBUS handler successfully\n"); + + test_hwpoison_multiple_pages(map); + + if (read_nr_hugepages(hugepagesize_kb, &nr_hugepages_after) != 0) { + close(fd); + ksft_exit_fail_msg("Failed to read nr_hugepages\n"); + } + + /* + * After MADV_HWPOISON, hugepage should still be in HugeTLB pool. + */ + ksft_print_msg("NR hugepages after MADV_HWPOISON is %ld\n", nr_hugepages_after); + if (nr_hugepages_before != nr_hugepages_after) + ksft_test_result_fail("NR hugepages reduced by %ld after MADV_HWPOISON\n", + nr_hugepages_before - nr_hugepages_after); + + /* End of the lifetime of the created HugeTLB memfd. */ + if (ftruncate(fd, 0) < 0) + ksft_exit_fail_perror("Failed to ftruncate to 0"); + munmap(map, len); + close(fd); + + /* + * After freed by userspace, MADV_HWPOISON-ed hugepage should be + * dissolved into raw pages and removed from HugeTLB pool. + */ + if (read_nr_hugepages(hugepagesize_kb, &nr_hugepages_after) != 0) { + close(fd); + ksft_exit_fail_msg("Failed to read nr_hugepages\n"); + } + ksft_print_msg("NR hugepages after truncate is %ld\n", nr_hugepages_after); + if (nr_hugepages_before != nr_hugepages_after + 1) + ksft_test_result_fail("NR hugepages is not reduced after truncate memfd\n"); + + ksft_test_result_pass("All done\n"); +} + +int main(int argc, char **argv) +{ + int fd; + struct statfs file_stat; + size_t len = BYTE_LENTH_IN_1G; + + ksft_print_header(); + ksft_set_plan(1); + + fd = create_hugetlbfs_file(&file_stat); + test_main(fd, len); + + ksft_finished(); +} -- 2.48.0.rc2.279.g1de40edade-goog