From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55256EB64DA for ; Wed, 28 Jun 2023 08:37:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235515AbjF1IhJ (ORCPT ); Wed, 28 Jun 2023 04:37:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234445AbjF1Ic7 (ORCPT ); Wed, 28 Jun 2023 04:32:59 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 957572D78 for ; Wed, 28 Jun 2023 01:25:11 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-c38c87ea9ceso34999276.1 for ; Wed, 28 Jun 2023 01:25:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687940711; x=1690532711; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=gPRmZ26IBsiMsbW2DFKvtKQmDaUZH2fHIbrEB332iB8=; b=DUmaXirbxVAivZHig+mjQqvfHcbOAHAnca1Z72aGuEntgSr/iBnDcYpQgUMwRDEnvL mvVm/wjUKLqf4xCSzIzklR+aizu1YOUMXJeKLjG7jK9tB3/EF5vxoAHu2hB+InPJUlHq LIrlxKFhydvQMZ9P1yHUmsqgWV2QL+1Hl2QKWHZ2iNqnjU20S/J5tlpRsPM3xpLxDpTU WKtj34tLVgkhOVKe7kbqC2RIEGJXtURP/8lruoUYpJaHFlCf+tHZARjhDLEXyhzaeuAX e4RfJq29jVDofVaNTmJ9F84/2qGOHNlXTN1Pn4YHrgzLwkvI+M+M9aURQuA0XnXhLV9I 9TXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687940711; x=1690532711; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=gPRmZ26IBsiMsbW2DFKvtKQmDaUZH2fHIbrEB332iB8=; b=TJ9AnSG/GHI33MXGJNuDM1Jnm/IKITCtZ2phT1Sa6knFyUhnG4JxWX/lT5rxa6Na7p sT56Z7VLMLrY5dnmTBzMS8JdiZKy5EVXNp2gOpY4gW+2YKDLsG7nh4bp9ELxWw5MLZp1 qMUOaO8IrAiggzqStatNCf3i15ciAnzrN7x67BEULFr1DYyfVf+GJPXRMfIR9IK0XnIq uezLZHmXGn/fHnRHWDSjA5h5b7Gd+FxTneUkqLwkMJSU0gD4raRLRmrikwvU3obYFgtf MEp3qg6tUWTcENAwU22t6J8wHEvjvPDlsy8YGFHUF/M8lmetBNlXtDMW2f9LlSuz5sO0 IzhQ== X-Gm-Message-State: ABy/qLafcfK3PdSVW86b7w65ChHoPeHlEYQ93rT58sDAdXuG2MomnwEa GdVxRc3YEuYqYXf0rkTOaWSrd7xecq0= X-Google-Smtp-Source: APBJJlEpYSp8he1DXAXVpI2wIb6u5ggZR99ctstsOjVLxfgRvzgDR0H3VpIWbi4McUwp9/ln4tUqhFGOd9A= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:6664:8bd3:57fd:c83a]) (user=surenb job=sendgmr) by 2002:a05:690c:2f82:b0:56d:5db:2f07 with SMTP id ew2-20020a05690c2f8200b0056d05db2f07mr7658ywb.5.1687936684846; Wed, 28 Jun 2023 00:18:04 -0700 (PDT) Date: Wed, 28 Jun 2023 00:17:54 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230628071800.544800-1-surenb@google.com> Subject: [PATCH v4 0/6] Per-VMA lock support for swap and userfaults From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: willy@infradead.org, hannes@cmpxchg.org, mhocko@suse.com, josef@toxicpanda.com, jack@suse.cz, ldufour@linux.ibm.com, laurent.dufour@fr.ibm.com, michel@lespinasse.org, liam.howlett@oracle.com, jglisse@google.com, vbabka@suse.cz, minchan@google.com, dave@stgolabs.net, punit.agrawal@bytedance.com, lstoakes@gmail.com, hdanton@sina.com, apopple@nvidia.com, peterx@redhat.com, ying.huang@intel.com, david@redhat.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, viro@zeniv.linux.org.uk, brauner@kernel.org, pasha.tatashin@soleen.com, surenb@google.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When per-VMA locks were introduced in [1] several types of page faults would still fall back to mmap_lock to keep the patchset simple. Among them are swap and userfault pages. The main reason for skipping those cases was the fact that mmap_lock could be dropped while handling these faults and that required additional logic to be implemented. Implement the mechanism to allow per-VMA locks to be dropped for these cases. First, change handle_mm_fault to drop per-VMA locks when returning VM_FAULT_RETRY or VM_FAULT_COMPLETED to be consistent with the way mmap_lock is handled. Then change folio_lock_or_retry to accept vm_fault and return vm_fault_t which simplifies later patches. Finally allow swap and uffd page faults to be handled under per-VMA locks by dropping per-VMA and retrying, the same way it's done under mmap_lock. Naturally, once VMA lock is dropped that VMA should be assumed unstable and can't be used. Changes since v3 posted at [2] - Renamed folio_lock_or_retry back to folio_lock_fault, per Peter Xu - Moved per-VMA lock release to where VM_FAULT_RETRY is returned, per Peter Xu - Dropped FAULT_FLAG_LOCK_DROPPED usage, per Peter Xu - Introduced release_fault_lock() helper function, per Peter Xu - Dropped the patch releasing per-VMA lock before migration_entry_wait, per Peter Xu - Introduced assert_fault_locked() helper function, per Peter Xu - Added BUG_ON to prevent FAULT_FLAG_RETRY_NOWAIT usage with per-VMA locks Note: patch 3/8 will cause a trivial merge conflict in arch/arm64/mm/fault.c when applied over mm-unstable branch due to a patch from ARM64 tree [3] which is missing in mm-unstable. [1] https://lore.kernel.org/all/20230227173632.3292573-1-surenb@google.com/ [2] https://lore.kernel.org/all/20230627042321.1763765-1-surenb@google.com/ [3] https://lore.kernel.org/all/20230524131305.2808-1-jszhang@kernel.org/ Suren Baghdasaryan (6): swap: remove remnants of polling from read_swap_cache_async mm: add missing VM_FAULT_RESULT_TRACE name for VM_FAULT_COMPLETED mm: drop per-VMA lock when returning VM_FAULT_RETRY or VM_FAULT_COMPLETED mm: change folio_lock_or_retry to use vm_fault directly mm: handle swap page faults under per-VMA lock mm: handle userfaults under VMA lock arch/arm64/mm/fault.c | 3 ++- arch/powerpc/mm/fault.c | 3 ++- arch/s390/mm/fault.c | 3 ++- arch/x86/mm/fault.c | 3 ++- fs/userfaultfd.c | 39 ++++++++++++++++++--------------------- include/linux/mm.h | 39 +++++++++++++++++++++++++++++++++++++++ include/linux/mm_types.h | 3 ++- include/linux/pagemap.h | 9 ++++----- mm/filemap.c | 37 +++++++++++++++++++------------------ mm/madvise.c | 4 ++-- mm/memory.c | 38 ++++++++++++++++---------------------- mm/swap.h | 1 - mm/swap_state.c | 12 +++++------- 13 files changed, 113 insertions(+), 81 deletions(-) -- 2.41.0.162.gfafddb0af9-goog