From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27A573328FC for ; Thu, 27 Nov 2025 14:11:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.43 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764252702; cv=none; b=he/LYGQQKYf1nRHWq2iWYdSlmx8lAFkRkdl5HURyVNtKQaQdSvjigw5lyiiZgAIiFlBh19OJjyo9qNwaclUmnYNaecimXBtIKvLi/++Lva5tT4hFtLTfPbiDn8y471y+lG3A4O0/LMd4mfkba761FE/D90E/WIxCvPFbwbLVgf8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764252702; c=relaxed/simple; bh=BkugkW0ls1hsxp2vPtkZ656cs1RuwcdSxB7Cfbd/jhA=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=DZdsa91qyah1e4/bdcUztSDMpj3+Zk9s3XYRbmersmXXQ/P0/Kop6KEsO6kbXSOEnZ9T0kDU/thXPmYl8HDorMck1HvVMJJSqOurpu3QzZk+jTB3TeosAvw856gNtwIAkF9IlUf6+fLyaNrCS5Kc1VZmzuulYBnh2JmIcQ+bE5s= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=h4yjz1dR; arc=none smtp.client-ip=209.85.216.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="h4yjz1dR" Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-3436cbb723fso734265a91.2 for ; Thu, 27 Nov 2025 06:11:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1764252699; x=1764857499; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=u3WXNKq2AX75j2bmfv330sc+Ngjbx9cqOj8cPkF805A=; b=h4yjz1dRJRZGYroXEAk2QXNHAS0c13pBd6196WOv2yuD8st+1na7GX6vr7moBrK83b mNcwFwdI6DeiNgo2sSmSXMHTEfTj32O2WQDThORupSySz5iiIWA/omvuqbSLhzmK2efm XHv4IRnT3W7BCAQXGbDWXzNwfO7LHGKjf83b1bhiBlzCrhTDeXea+WJqQviOEPvjHTUM Wt2Nw7KZGLEcs3Qkt7euPpnTOZYSf1dCcTZSauSOK8BTHPIeyR4vxyvvky/fmg+jPr9r ZRh+5ELVeUubFNmrAv2wPiQrq+EMbthWa9iBKshyBQUePik65nEbviErYBr8ef96pGN6 tzyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764252699; x=1764857499; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=u3WXNKq2AX75j2bmfv330sc+Ngjbx9cqOj8cPkF805A=; b=AP45iL2kj6qttGUDwzmaPKb2c/WSuDDt7yBvQucPZHAkmxm7MAU7zP0ORHzSKI5++S U/Oy5ME1m+ZwCYMoaubS2BTQF0LpKH4xgMaPCqqAAhQbN7wxtLkgJfrH7QUNZLRww9pO QMdIOdeXpHrRetk6yKXx4XW17drip2GW40j1y2wFuYGkeNvsCB9zF+5mxE84JVXwvd0z X44Lfm+XLE6DYqkyvLo9wLi0bVbJ2ELAE2u6z8jx3KDyFqWE7c2u7fQIkqsht1tshVib s0oZllxR1vscdI0F38faczxqxbgm6ce2O4vp/3CDn6PC8/O0lUy5/Az88ifbmxteaxzm MfRg== X-Forwarded-Encrypted: i=1; AJvYcCUEH9ohCUssPW85jrNK2Uf2iQsezHZkhGjqCYSTESCDwe+JavPnvaEtMEKNh3M0uFvaDL4z4lkXvfatnSY=@vger.kernel.org X-Gm-Message-State: AOJu0Yw1Yn39/OFFPdWxxLlBM+l5VXVVimhLDHTeGyRnKOCCGewL0Yml TQHuLD8om4diO9AoQpZnR529x6HdmmyVpXlYp2WSqxIbeanoOyCVVYD9+dIjqGVN3gQ= X-Gm-Gg: ASbGncv3kOKDehymCUrkkO78y//hXG4xRkM+yV18NkKzTgQDN2vfyd3GTgph0t0fFxx pEOB8Kj7sAM210O7X7lW33YdTJpOMBe6TJg/yB2/Nhmd/q0N9divY/J7Yp2oFPDtljRhC1A4uNK ecnw/5JY15bJFQG0j6+xjM1jwWB+seL95u5S/CRFKAMWsiTcSoI1ZE+Vvz8GaGevv2o+1o4Rzpl UE3VyKGnmswQ0Iz2ORwFWUTL5xPgi9112qHIXGfW7IwNF3S4hiy5Jjl5Psd0eascH6L3pTfQ1wF peaR+2PT/WL7k56hwARphCHe+amIywnCzyBCJUtLXDUoWGy6kSCJHZTYWwAr5TtC6YVXlq83tYR m2Eqvc5cVcJKD/1K933mJDUxEgJ/fwZ8Zap5Klan2WiCkI0poRzpGbQUe6G+rNYKhylvhLIt+tM /gnCut0JJx9ZBpN8dHcJ2p3OdWpP9iJOZoqyYA2pcMSqeif0UoUsd5O2GdmnNa8xTCP2wfRHfyJ T09I7g1p5D7 X-Google-Smtp-Source: AGHT+IGqqZbiAaSC9ZX+pYvJ1IbIc0rETHK1OBZOJFFlIrrfxJbsAOd/0yhH41xHJLHj1PsrjPP9Tw== X-Received: by 2002:a17:90b:54cd:b0:32e:3552:8c79 with SMTP id 98e67ed59e1d1-3475ed6a4f9mr10948253a91.29.1764252699207; Thu, 27 Nov 2025 06:11:39 -0800 (PST) Received: from J9GPGXL7NT.bytedance.net ([61.213.176.58]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3477b7341d2sm2030249a91.11.2025.11.27.06.11.32 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 27 Nov 2025 06:11:38 -0800 (PST) From: Xu Lu To: pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, alex@ghiti.fr, kees@kernel.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, akpm@linux-foundation.org, david@redhat.com, apatel@ventanamicro.com, guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Xu Lu Subject: [RFC PATCH v2 0/9] riscv: mm: Introduce lazy tlb flush Date: Thu, 27 Nov 2025 22:11:08 +0800 Message-ID: <20251127141117.87420-1-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.50.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit This patch series introduces a lazy tlb flush mechanism for riscv. This mechanism is based on two insights: 1) Since each CPU has limited TLB entries, there exist limited active ASIDs in each CPU's TLB at the same time. When a mm has not been used for enough long time (or, after enough switch_mm times), we can assume its TLB entries are all evicted out. Then we can clear current CPU in its mm_cpumask so that next time when the memory mapping of this mm is modified, no IPI will be sent to current CPU. 2) When memory mapping of a mm is modified, instead of sending IPI to all CPUs recorded in its mm_cpumask, we check whether each target CPU is using this mm right now. If not, we just store the TLB Flush information in target CPU's percpu buffer, avoiding the IPI. Next time when the target CPU switch_mm to this mm, it can check the percpu buffer and perform TLB Flush itself, without IPI involvement either. Using this mechanism, we significantly reduced the number of IPI due to TLB Flush: * ltp - mmapstress01 Before: ~108k After: ~17k * ltp - hackbench Before: ~385k After: ~2k Thanks Guo Ren for his advice on memory access latency test via lmbench. We are unable to test it now due to lack of real machines. We will supply this test and adjust our mechanism according to it as soon as possible. Xu Lu (9): riscv: Introduce RISCV_LAZY_TLB_FLUSH config riscv: mm: Apply a threshold to the number of active ASIDs on each CPU riscv: mm: Grab mm_count to avoid mm getting released fork: Add arch override for do_shoot_lazy_tlb() riscv: mm: Introduce arch_do_shoot_lazy_tlb riscv: mm: Introduce percpu TLB Flush queue riscv: mm: Defer the TLB Flush to switch_mm riscv: mm: Clear mm_cpumask during local_flush_tlb_all_asid() riscv: mm: Clear mm_cpumask during local_flush_tlb_all() arch/riscv/Kconfig | 12 ++ arch/riscv/include/asm/mmu.h | 4 + arch/riscv/include/asm/mmu_context.h | 5 + arch/riscv/include/asm/tlbflush.h | 63 ++++++ arch/riscv/mm/context.c | 24 ++- arch/riscv/mm/tlbflush.c | 302 +++++++++++++++++++++++++-- kernel/fork.c | 6 +- 7 files changed, 394 insertions(+), 22 deletions(-) -- 2.20.1