From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 20C1E3BBE8 for ; Fri, 24 May 2024 20:00:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716580813; cv=none; b=hCN7rEzN80d11fb4uJsdQ6BnLjh+B7ILVCpSC7lBOQq8Cfqj+qTU7lFKhwa/HSkyJbqA57gbSF2j+tLPCo3X0MIlpkJbxKHwHwVYwppH6l4qgiDZqCbdBp+KBQAnVSjBoxB57B1vOJeBmzvzPiqGg0JcWYpq9QKK2tvRgTmpHaM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716580813; c=relaxed/simple; bh=Gv43iCvXawnlGLRmctGhbdkb5WAxXv1tZ9waG3dYXuw=; h=Date:From:To:cc:Subject:In-Reply-To:Message-ID:References: MIME-Version:Content-Type; b=m8OftY93OyuYh0PIW2pLLhKDCnUrThvt6VylI5GsF4JqfH6ga/eepwDQsQ39EB3GJl+aIs2SEKvBTZGqyDOoQwTmUe0sWdqPoLgkl44o1UYTybatopo5E92dVP+hSQOYdkb/CYxQTtnK5hoTNdJJBz2hiQFQCorv2Al2LUuUdFU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=luY7Yz16; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="luY7Yz16" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1f337d0e382so33185ad.0 for ; Fri, 24 May 2024 13:00:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716580810; x=1717185610; darn=vger.kernel.org; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=J03u/azUiuyJ/7gPFey8suUIJZ5rc1p0eW52hXcyf1w=; b=luY7Yz16J6LPMLhLqtyuR+ZV1IxJQv9r1b0wa+/3z1fQhLwtAOmyP8UGuyOQ5DNYcL mX+KwRFwb0ojhQCxG7rZwyIXcBDc6sX2F815JmFgNbiNARUW2DgMCwyPpy15/mE2Ip65 ErBN2ch8jOg0QdzNQrNacUhZZ9IEwdrfHHks5icZPgtkyDzhPPXDkLlCyXjEkXvuAXrX 9kz1V8zR/66zL/w+W99dTrb/ynvjBpzfduy3+x0OV4jZeYud5LE4sIc1uvGPtnJlWWeM tGEvla9idzN3dK4Ylaeu+D5b0ko1nTiJh7aRbxxb5tfCnGSlKYtHkv29nRNGy00o/Uv8 WboQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716580810; x=1717185610; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=J03u/azUiuyJ/7gPFey8suUIJZ5rc1p0eW52hXcyf1w=; b=o2UJPeeSBVmgqvnWtmIktDYVKQ1xGPMAo9bO5dUKeqtiFM3+mRwUfVcCiQ7k7a/Dks 7jy6VAVcEEGhkd3fhgAY2tRVPkkuoh6O7uUvvi/8afjwaUZSkq8g73oLidTLe3dePgo0 PrQAFdesgkKGMfjCaXJ8SBUNzi6McuqAtoet1EtSUDP6Iti6M3Chv7+BHax9KKEjk15s DVMB/pwU76xNkQsSpcXnH1ceKgkzWPOha4C0ZAfopV8XiDb8YwNqAlTQXpj/xkFgEASc RurSJ6+x1YA4p4A8Ay1TLSKBatCUhv3SdIcNpA2zKDXdVf6zVlHNIlh229JX7sEH6EFN KcKg== X-Forwarded-Encrypted: i=1; AJvYcCXaIkyiNn+mco9l0anYPHIROgRW8jmBoJ/dQtAJMwUslldhfQ93Q59fy1f5+kDeVVXQgNbRj3l1bY2YjS11XwLqIpeEUcRVgteOKeqR X-Gm-Message-State: AOJu0Yzk5HP4h/eavmTJ//kxU6Un9osgM0lEVSb9N2SSqnaKRtB5enJY fi1O3y0zWGs46SQML0R8JsTTJv6A5WbNBXbpq25ZCSw5LLh/YAf57RrNjsCEBg== X-Google-Smtp-Source: AGHT+IGCIXQDdrO6BGUEBPI0rbMRLEFS1n4FDhL/LvduVdqztKRsVCv6w1yspdPe5J3ICBr0hoDTAg== X-Received: by 2002:a17:902:d902:b0:1f4:50b4:a50b with SMTP id d9443c01a7336-1f466ce2270mr381615ad.18.1716580808226; Fri, 24 May 2024 13:00:08 -0700 (PDT) Received: from [2620:0:1008:15:c92f:57d0:1ea4:5439] ([2620:0:1008:15:c92f:57d0:1ea4:5439]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-6f8fc05e08bsm1456208b3a.52.2024.05.24.13.00.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 May 2024 13:00:07 -0700 (PDT) Date: Fri, 24 May 2024 13:00:06 -0700 (PDT) From: David Rientjes To: Sourav Panda cc: corbet@lwn.net, gregkh@linuxfoundation.org, rafael@kernel.org, Andrew Morton , mike.kravetz@oracle.com, muchun.song@linux.dev, rppt@kernel.org, david@redhat.com, rdunlap@infradead.org, chenlinxuan@uniontech.com, yang.yang29@zte.com.cn, tomas.mudrunka@gmail.com, bhelgaas@google.com, ivan@cloudflare.com, pasha.tatashin@soleen.com, yosryahmed@google.com, hannes@cmpxchg.org, shakeelb@google.com, kirill.shutemov@linux.intel.com, wangkefeng.wang@huawei.com, adobriyan@gmail.com, Vlastimil Babka , "Liam R. Howlett" , surenb@google.com, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, Matthew Wilcox , weixugc@google.com Subject: Re: [PATCH v12] mm: report per-page metadata information In-Reply-To: <45fb4c94-dd77-94c3-f08f-81bf31e6d6d2@google.com> Message-ID: <1e1b89af-3b7a-7400-cfd7-d22a101955de@google.com> References: <20240512010611.290464-1-souravpanda@google.com> <45fb4c94-dd77-94c3-f08f-81bf31e6d6d2@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII On Sat, 18 May 2024, David Rientjes wrote: > On Sun, 12 May 2024, Sourav Panda wrote: > > > Today, we do not have any observability of per-page metadata > > and how much it takes away from the machine capacity. Thus, > > we want to describe the amount of memory that is going towards > > per-page metadata, which can vary depending on build > > configuration, machine architecture, and system use. > > > > This patch adds 2 fields to /proc/vmstat that can used as shown > > below: > > > > Accounting per-page metadata allocated by boot-allocator: > > /proc/vmstat:nr_memmap_boot * PAGE_SIZE > > > > Accounting per-page metadata allocated by buddy-allocator: > > /proc/vmstat:nr_memmap * PAGE_SIZE > > > > Accounting total Perpage metadata allocated on the machine: > > (/proc/vmstat:nr_memmap_boot + > > /proc/vmstat:nr_memmap) * PAGE_SIZE > > > > Utility for userspace: > > > > Observability: Describe the amount of memory overhead that is > > going to per-page metadata on the system at any given time since > > this overhead is not currently observable. > > > > Debugging: Tracking the changes or absolute value in struct pages > > can help detect anomalies as they can be correlated with other > > metrics in the machine (e.g., memtotal, number of huge pages, > > etc). > > > > page_ext overheads: Some kernel features such as page_owner > > page_table_check that use page_ext can be optionally enabled via > > kernel parameters. Having the total per-page metadata information > > helps users precisely measure impact. Furthermore, page-metadata > > metrics will reflect the amount of struct pages reliquished > > (or overhead reduced) when hugetlbfs pages are reserved which > > will vary depending on whether hugetlb vmemmap optimization is > > enabled or not. > > > > For background and results see: > > lore.kernel.org/all/20240220214558.3377482-1-souravpanda@google.com > > > > Signed-off-by: Sourav Panda > > Reviewed-by: Pasha Tatashin > > Acked-by: David Rientjes > This would be a very useful extension to be able to provide observability of per-page metadata overhead and the impact of things like HVO on the overall footprint. Today, we don't have observability for this memory overhead. Andrew, anything else that can be addressed before this is eligible for staging in MM unstable?