From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B10BB1DE4E0 for ; Wed, 27 May 2026 15:45:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.42 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779896761; cv=none; b=hjSNLUmizD34c3Jd1oCcndTJCxx8wESaoA66TtNyfEMTJ1RWQi9FvwRg9/b0tzZjF8bOXtKojeKdlMUGFiemhQZ/MzmRmLRhp3woD2MpCwmY0MT7Ix01D62BhK6lYuAfHO327NOE6wq3MbSWf+HnCsoXWcT+VoUnYdM+JN9jkkc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779896761; c=relaxed/simple; bh=CELVVqgRn7T4Qoy1L9CmAaEsHICE+smQQ/1xwQU6xLg=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=K0qKBytrjk2kvspcCQ9+3WViCBOwrBVtcKSIVnc5c+z1hkPwdq+4AvatTvXwD6LSvhG7X1lbPGsShUsTjzgd8LUA59x0ZEfTGqNDtfLv+fPyn7gSCFVwMKuN1YCJ/5TQOiPE6O0pepi+vimxif3aj+96NV0o5OV1fDJmkVjE/zA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=q5+uLel0; arc=none smtp.client-ip=209.85.128.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="q5+uLel0" Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-49041fb8c23so44217475e9.0 for ; Wed, 27 May 2026 08:45:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1779896758; x=1780501558; darn=vger.kernel.org; h=content-disposition:mime-version:message-id:subject:cc:to:from:date :from:to:cc:subject:date:message-id:reply-to; bh=lKvMOV5uJkkVkdu/ydV3n4EuIqaTCZ91Ux72HYHzFZQ=; b=q5+uLel0KPxv8Ry/DyWAay5Sw3j/5iBPn1oC/1nLFYWI4cvro2GEODLlh0UPCAurmQ xaDxZD7bXyFd3tQpemFIl3qsd1bquMrbdh6lTPN4v/uei+gSxIrv+tFcWUSgWTxfEy4B 5rrR/0QgKfK9BNyqNSqT0fJbATG2ufOXDlgkreE4MK07iQesxKUwPoHQhIfBPhKKkC5B 71IA3Hc9YFOBRxPitTg5EX7LlaU+i8o6zk/8RyJXcvt3il0W01ewEmVXkCXZAOAnOu/k SONG+m4Fp4vlBe9bNJYSvHnpYZUr752s3VTSW+FyJ4y/kIGSsWT3UsTnPUceH1/kr4z1 TvpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779896758; x=1780501558; h=content-disposition:mime-version:message-id:subject:cc:to:from:date :x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=lKvMOV5uJkkVkdu/ydV3n4EuIqaTCZ91Ux72HYHzFZQ=; b=LUm2VuJTB7b0I41y4RvgEnoMeqUgPYz14o+nQaIAtQv+FtpojQzh+I+bABf+BH1oGH xBilR3Kk6nQBcpeJSL+QUZiBnInVSss1VHNZc8aRchpFVVG4WLHSc8ZEcUUlDfCCR/s3 Rp6PXbQt08I2xSDVOEbHVTnDCRVJfsYyWvczo/x6IMABSNCoexy810PEG2imHWyioyPK IIiczz/nOzfLOzE67pe1FYFTj5J2OaoCBqe0ZpKBvfCFw2Yfsn4AptjV2GP0sMb41XG8 YT9mEcZceSqGqxb9uHbQLHyTadWdk2LRHPVpCX7RV5AezTnZVnKCq4147NiAhoWPz5mo FCCw== X-Forwarded-Encrypted: i=1; AFNElJ/Nl3hSBBuL/Oq+u0xNmJhqLtua8fqg8u2Ilm64f5qbLQzDZTJ9Bx1DTsCl1uDCbUp/EAAejbf5vq3/Irg=@vger.kernel.org X-Gm-Message-State: AOJu0YwaozHYvdPW3UQWKLatnv+Tl5t9yrrU/zVWMrNkeJDy9FmH9GV1 ZR595xjXR/oDHjUhv4J4pZf8WOp9BX6CONg0+2h6T4Q5M2k0zrLn2Fg8 X-Gm-Gg: Acq92OFpc4d5q65dpZaY/0yTg0z1507fFGmJiwrNmkVid0lb7TNd3Lx/Jdi7xcxYmbU bnKNhY4uFK1zkZFMlxoMiIHxDeVlulTmGl9BLZzvDVx0PGUubCuDtiB+O5F7eNYG71RsePWcyAQ enLSB7VvkT7ixdOXSguklMSBgIX0UfTIVU7C2Su4vnrDph0cmV3CFMvnqr+J+SdGLtmHjYK78jf VAeHioLm+k97G6O7fs4/g/R6YAC3x6Rm7fqu5E9kXfGWqz8UaKpLZ4lw3fs3WowcI38hUlu80fV VizI9VTHFEi4MlcdOIkxyM3H9nnGaaBZq2UB2UY2AvNSRC+fhWEbOfk/CAACOHLqI4wYnAFoX52 LkBxrxc4II/0wxvsUQVslsA1aLlC4S6aNTXJEY8nsAtf6Tb8uhoI8iYpHmNh1QXHjzsOAYPmWFl wyIUbxgifgAhwbfuSxKL+Fwm49G/AZcHdMFDQuq5u6egtkVcAgJf1BSv1WJg== X-Received: by 2002:a05:600c:a30d:b0:48e:5d91:cfe3 with SMTP id 5b1f17b1804b1-490424ac7bcmr294290885e9.1.1779896757854; Wed, 27 May 2026 08:45:57 -0700 (PDT) Received: from localhost (dhcp-91-156.inf.ed.ac.uk. [129.215.91.156]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45edb5b2a2asm7023605f8f.28.2026.05.27.08.45.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 May 2026 08:45:57 -0700 (PDT) Date: Wed, 27 May 2026 16:45:56 +0100 From: Karim Manaouil To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: k.manaouil@gmail.com Subject: Memory fragmentation and compaction test and evaluation Message-ID: <20260527154556.chak2cd7woi3gagy@wrangler> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi folks, With mTHP/THP, large block sizes, and increasing adoption of larger pages in various subsystems (e.g. drm/ttm [1]), there is an increasing pressure on physical memory contiguity and hence anti-fragmentation, defragmentation and compaction. This brings attention to revisiting existing mechanisms in the kernel, especially with the rapidly changing hardware landscape in terms of memory technology, memory bandwidth, CPU microarchitecture, CPU core count, interconnects and other factors. However, it turns out it's extremely hard to evaluate fragmentation, compaction and huge page availability. The main challenge rests on reliable reproduction. We need tools and workloads. Specifically, we need tools that: 1. Reliably reproduce fragmentation on a system. Preferably with varying degrees and some control over movable vs unmovable. 2. We need tools/workloads that continously stress the system and continously introduce fragmentation and challenge large block allocation while other workloads are running, preferably without interference between the two. For memory fragmentation, a famous synthetic way that is widely adopted in previous patches is allocating massive amout of anonymous/file memory while an I/O metadata intensive workload is running on paralle to interleave unmovable slab allocations from the filesystem with movable allocations, for example by running "find /" or running fio in parallel as is the case with mmtests/thpchallenge. This addresses point (1) to some extent. Point (2) remains entirely un-addressed. It is also important to have real world workloads. I would love to see how other people address these challenges and how people here test and evaluate fragmentation and compaction, especially under sustained pressure. We can always start from a fragmented system (for example mmtests/thpchallenge way) and run a workload to completion, but that only tests how the system recovers initially and not how it copes with continous stress. Please share you experiences in this thread and share any workloads and ideas for tools that you think could help in these sorts of development. [1] https://lore.kernel.org/linux-mm/20260506033300.3534883-1-matthew.brost@intel.com/ Thanks -- ~karim