From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CBD714B959 for ; Fri, 27 Dec 2024 23:15:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735341347; cv=none; b=Xd4NkqYXCG8lknVmNXGUNSlg2RJ9opMiwrTH4AxFqm0iOp8Ij2CDTi7DkVt65H8GNsZTFc8CZpBzObnRCzMSy0rlv5KoZwsRVqy07YjNjT7xPeg265MZVY/EuF6kmQsyGBW+eCUvZDqj/QoEgHOkb5fJAguGhRh6jn45hEirKF4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735341347; c=relaxed/simple; bh=i1QBFjgTS0T6ShVTSrbJwyEnTv6x/n8xJ1vOd3nwAZU=; h=Date:In-Reply-To:Mime-Version:Message-ID:Subject:From:To:Cc: Content-Type; b=JriPn8ZwWCxdrU/w5MZ4OixA0fsRR08GircWR7Bev5QQ6gxXxYBj23LsHwxmk7r8MauMf51GMvsQCV6zkepI9kWN33UzK1bnDqrvjYGG5Ei2RlDcxVEr9LHnJgsYFVdCXJrcgaMvQWb4FhtO9yyjiBUk6TjZ4EeHcMHGhdmF5uw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ackerleytng.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3TvHvrW8; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ackerleytng.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3TvHvrW8" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ef9dbeb848so9784235a91.0 for ; Fri, 27 Dec 2024 15:15:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1735341345; x=1735946145; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:in-reply-to:date:from:to :cc:subject:date:message-id:reply-to; bh=pYN9ztGBOKvJNFzrcQMWsC4rhR3Yt3k6FRUdi+tvDCY=; b=3TvHvrW8JfIPyoce8mgJQ6uCnojZ2FM96SOITfzVSs0e2Iup4SMcez/yTTJW3vy3gc Lo5YCTr+3FmIPZhepFi4zceEu9tVg+HDCriZQLPE2kSDQoXqtyl7EYzDQgFyxJBEw2Ql 7RtKB+9H/m4DX1mhSz9yrE6sceNEyEiqW2dg3b09Olzr/ECTz1q/jx5f7CIIbbGmVIqe gmgBTXbaL4UoqlfTT0LRsNbGYe6mg6lbxXwPsWo/UQUYMmwddeKfgRgNFdjKKgNRdE1h ve6a9ag0UzLIe407j7yGqzKrWAByxzkn9u8I1spcaEwbEIrBUY7DsZjUoxDrfa0w6BK+ YSRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735341345; x=1735946145; h=cc:to:from:subject:message-id:mime-version:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=pYN9ztGBOKvJNFzrcQMWsC4rhR3Yt3k6FRUdi+tvDCY=; b=BX4yNk2o2cLyqBrSfm+lvuUYv6URufRcgisALkOqX4HUyvl49j3gPjX31Q07sSuGKK 9gCXvq33c3gOjgxZQC5HqLL6n9MjXRme7Rne4hA3NkYq/KPD5aFDawmKj9cVw84XlI3N jZD/crTRpT5uF4jr3lmFiXKZ+taI8rxJhVP4Rx50Nsoh2iIgDgBoaTogyaYlk7HqwYJE iaiOvCfyKETg3U0O/DJyWZQb8DtUQ1zFTE4uasi2VWn2DhreAvBSo8C+BXho/XvjPMbF tr7sp90dSbua2jLQEGpxgsZL4gT/xFOAt8lGg5V8nYeM2pAmvcv8Aeb6nH2M0lLVPBc9 ZiMA== X-Forwarded-Encrypted: i=1; AJvYcCXAkq3AGTRaU+8ZnYaACJfWlscvCASnqE+5lIo3lRU8zd6e9MX4si8VnTru4i7eZcJV60FPYCJa+MhnccU=@vger.kernel.org X-Gm-Message-State: AOJu0Yzlj4tKiK4VnL/8IoyCV1HH9Tql/0IlPqPvgj/7VXEeEKxrbvX1 cYaXywj7Z9bdrDYojEg0pQO+Su2G/IybwhMuB60DGXY3lBXD7WQ2Nizf2KOfWv1LYlC5N5ULeWC w/qAdXPeDKEVt6zpL5y+VoA== X-Google-Smtp-Source: AGHT+IGYAECDHJ4KsM8Z6EpTiSh3tBGqcEn9YutEFxQ53dQTWNAXX4rILEOQyv0ufEIHQKSHvStzVsVLF+tXsGrdJA== X-Received: from pfwy41.prod.google.com ([2002:a05:6a00:1ca9:b0:724:f17d:ebd7]) (user=ackerleytng job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:2c86:b0:725:cfa3:bc76 with SMTP id d2e1a72fcca58-72abdd4f2a7mr46778692b3a.4.1735341345609; Fri, 27 Dec 2024 15:15:45 -0800 (PST) Date: Fri, 27 Dec 2024 23:15:44 +0000 In-Reply-To: (message from Ackerley Tng on Wed, 18 Dec 2024 14:33:43 +0000) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 Message-ID: Subject: Re: [PATCH 1/7] mm/hugetlb: Fix avoid_reserve to allow taking folio from subpool From: Ackerley Tng To: Ackerley Tng Cc: peterx@redhat.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, riel@surriel.com, leitao@debian.org, akpm@linux-foundation.org, muchun.song@linux.dev, osalvador@suse.de, roman.gushchin@linux.dev, nao.horiguchi@gmail.com, stable@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Ackerley Tng writes: > > > I'll go over the rest of your patches and dig into the meaning of `avoid_reserve`. Yes, after looking into this more deeply, I agree that avoid_reserve means avoiding the reservations in the resv_map rather than reservations in the subpool or hstate. Here's more detail of what's going on in the reproducer that I wrote as I reviewed Peter's patch: 1. On fallocate(), allocate page A 2. On mmap(), set up a vma without VM_MAYSHARE since MAP_PRIVATE was requested 3. On faulting *buf = 1, allocate a new page B, copy A to B because the mmap request was MAP_PRIVATE 4. On fork, prep for COW by marking page as read only. Both parent and child share B. 5. On faulting *buf = 2 (write fault), allocate page C, copy B to C + B belongs to the child, C belongs to the parent + C is owned by the parent 6. Child exits, B is freed 7. On munmap(), C is freed 8. On unlink(), A is freed When C was allocated in the parent (owns MAP_PRIVATE page, doing a copy on write), spool->rsv_hpages was decreased but h->resv_huge_pages was not. This is the root of the bug. We should decrement h->resv_huge_pages if a reserved page from the subpool was used, instead of whether avoid_reserve or vma_has_reserves() is set. If avoid_reserve is set, the subpool shouldn't be checked for a reservation, so we won't be decrementing h->resv_huge_pages anyway. I agree with Peter's fix as a whole (the entire patch series). Reviewed-by: Ackerley Tng Tested-by: Ackerley Tng --- Some definitions which might be helpful: + h->resv_huge_pages indicates number of reserved pages globally. + This number increases when pages are reserved + This number decreases when reserved pages are allocated, or when pages are unreserved + spool->rsv_hpages indicates number of reserved pages in this subpool. + This number increases when pages are reserved + This number decreases when reserved pages are allocated, or when pages are unreserved + h->resv_huge_pages should be the sum of all subpools' spool->rsv_hpages. More details on the flow in alloc_hugetlb_folio() which might be helpful: hugepage_subpool_get_pages() returns "the number of pages by which the global pools must be adjusted (upward)". This return value is never negative other than errors. (hugepage_subpool_get_pages() always gets called with a positive delta). Specifically in alloc_hugetlb_folio(), the return value is either 0 or 1 (other than errors). If the return value is 0, the subpool had enough reservations and so we should decrement h->resv_huge_pages. If the return value is 1, it means that this subpool did not have any more reserved hugepages, and we need to get a page from the global hstate. dequeue_hugetlb_folio_vma() will get us a page that was already allocated. In dequeue_hugetlb_folio_vma(), if the vma doesn't have enough reserves for 1 page, and there are no available_huge_pages() left, we quit dequeueing since we will need to allocate a new page. If we want to avoid_reserve, that means we don't want to use the vma's reserves in resv_map, we also check available_huge_pages(). If there are available_huge_pages(), we go on to dequeue a page. Then, we determine whether to decrement h->resv_huge_pages. We should decrement if a reserved page from the subpool was used, instead of whether avoid_reserve or vma_has_reserves() is set. In the case where a surplus page needs to be allocated, the surplus page isn't and doesn't need to be associated with a subpool, so no subpool hugepage number tracking updates are required. h->resv_huge_pages still has to be updated... is this where h->resv_huge_pages can go negative?