[PATCH v3] mm/memory: drop PROCESS_PAGES_NON_PREEMPT_BATCH

[PATCH v3] mm/memory: drop PROCESS_PAGES_NON_PREEMPT_BATCH

[Shivank Garg]

clear_contig_highpages() clears the whole range at once under
preemptible models. Otherwise it clears PROCESS_PAGES_NON_PREEMPT_BATCH
pages at a time, with a cond_resched() in between. The batch is 32MB if
the architecture defines clear_pages(), and one page otherwise.

Since commit 7dadeaa6e851 ("sched: Further restrict the preemption
modes"), architectures that support lazy preemption only offer the
full, lazy and RT preemption models, also with PREEMPT_DYNAMIC. x86 is
one of them, and it is the only architecture that defines
clear_pages(). So the 32MB batch is never used: whenever
clear_contig_highpages() runs under a cooperative model, the batch is
one page.

Make that explicit. Clear the whole range when CONFIG_PREEMPTION is
set, and one page at a time with cond_resched() in between otherwise.
Drop PROCESS_PAGES_NON_PREEMPT_BATCH, which has no other users.

PREEMPT_DYNAMIC kernels also have CONFIG_PREEMPTION set, and
cond_resched() does nothing on any of these kernels. So a build-time
CONFIG_PREEMPTION check is enough, and preempt_model_preemptible() is
not needed.

No functional change intended.

Suggested-by: David Hildenbrand <david@kernel.org>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Signed-off-by: Shivank Garg <shivankg@amd.com>
---
This came up in review of the copy_mc_highpages() series [1]. Pedro
pointed out that most architecture run with full or lazy preempt, so mm
does not need to check the preemption model at runtime or batch the copy
for cond_resched(), and a simple per-page fallback is enough for
non-preemptible models.
David suggested doing the same for the zeroing path, and this patch
does that.

[1] https://lore.kernel.org/all/20261019-batpagecopy-v2-1-d75e7d0dfaae@amd.com 
---
Changes in v3:
- Use clear_user_highpage() in the per-page loop (Pedro).
- rebase on mm/core.git/for-next.
- Link to v2: https://lore.kernel.org/r/20261109-clear-pages-batch-v2-1-2e62c4f84f94@amd.com

Changes in v2:
- Rebased on mm/linux.git/for-next, and drop RFC tag.
- Link to v1: https://lore.kernel.org/r/20261022-clear-pages-batch-v1-1-13d3bd0b876e@amd.com
---
 include/linux/mm.h | 28 +---------------------------
 mm/memory.c        | 29 ++++++++++++++++-------------
 2 files changed, 17 insertions(+), 40 deletions(-)

diff --git a/include/linux/mm.h b/include/linux/mm.h
index e6da76fbe8e8..8d14e45fcf24 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -5250,7 +5250,7 @@ static inline void clear_page_guard(struct zone *zone, struct page *page,
  *
  * When running under preemptible models this is not a problem. Under
  * cooperatively scheduled models, however, the caller is expected to
- * limit @npages to no more than PROCESS_PAGES_NON_PREEMPT_BATCH.
+ * clear one page at a time and call cond_resched() in between.
  */
 static inline void clear_pages(void *addr, unsigned int npages)
 {
@@ -5261,32 +5261,6 @@ static inline void clear_pages(void *addr, unsigned int npages)
 }
 #endif
 
-#ifndef PROCESS_PAGES_NON_PREEMPT_BATCH
-#ifdef clear_pages
-/*
- * The architecture defines clear_pages(), and we assume that it is
- * generally "fast". So choose a batch size large enough to allow the processor
- * headroom for optimizing the operation and yet small enough that we see
- * reasonable preemption latency for when this optimization is not possible
- * (ex. slow microarchitectures, memory bandwidth saturation.)
- *
- * With a value of 32MB and assuming a memory bandwidth of ~10GBps, this should
- * result in worst case preemption latency of around 3ms when clearing pages.
- *
- * (See comment above clear_pages() for why preemption latency is a concern
- * here.)
- */
-#define PROCESS_PAGES_NON_PREEMPT_BATCH		(SZ_32M >> PAGE_SHIFT)
-#else /* !clear_pages */
-/*
- * The architecture does not provide a clear_pages() implementation. Assume
- * that clear_page() -- which clear_pages() will fallback to -- is relatively
- * slow and choose a small value for PROCESS_PAGES_NON_PREEMPT_BATCH.
- */
-#define PROCESS_PAGES_NON_PREEMPT_BATCH		1
-#endif
-#endif
-
 #ifdef __HAVE_ARCH_GATE_AREA
 extern struct vm_area_struct *get_gate_vma(struct mm_struct *mm);
 extern int in_gate_area_no_mm(unsigned long addr);
diff --git a/mm/memory.c b/mm/memory.c
index c9d64170e47a..fcc7ea2a00d1 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -7650,27 +7650,30 @@ static inline int process_huge_page(
 }
 
 static void clear_contig_highpages(struct page *page, unsigned long addr,
-				   unsigned int nr_pages)
+		unsigned int nr_pages)
 {
-	unsigned int i, count;
+	unsigned int i;
+
+	might_sleep();
+
 	/*
 	 * When clearing we want to operate on the largest extent possible to
 	 * allow for architecture specific extent based optimizations.
 	 *
-	 * However, since clear_user_highpages() (and primitives clear_user_pages(),
-	 * clear_pages()), do not call cond_resched(), limit the unit size when
-	 * running under non-preemptible scheduling models.
+	 * clear_user_highpages() (and primitives clear_user_pages(),
+	 * clear_pages()) do not call cond_resched(). That is fine on
+	 * preemptible kernels, where the scheduler can preempt the clearing
+	 * at any point, so clear the whole range at once. Otherwise, clear
+	 * one page at a time and reschedule in between.
 	 */
-	const unsigned int unit = preempt_model_preemptible() ?
-				   nr_pages : PROCESS_PAGES_NON_PREEMPT_BATCH;
-
-	might_sleep();
+	if (IS_ENABLED(CONFIG_PREEMPTION)) {
+		clear_user_highpages(page, addr, nr_pages);
+		return;
+	}
 
-	for (i = 0; i < nr_pages; i += count) {
+	for (i = 0; i < nr_pages; i++) {
 		cond_resched();
-
-		count = min(unit, nr_pages - i);
-		clear_user_highpages(page + i, addr + i * PAGE_SIZE, count);
+		clear_user_highpage(page + i, addr + i * PAGE_SIZE);
 	}
 }
 

---
base-commit: 2683df3b7eac1d91380194955ad0c9751454610c
change-id: 20261021-clear-pages-batch-77f72ef808e4

Best regards,
-- 
Shivank Garg <shivankg@amd.com>

Re: [PATCH v3] mm/memory: drop PROCESS_PAGES_NON_PREEMPT_BATCH

[David Hildenbrand (Arm)]

On 11/9/26 20:15, Shivank Garg wrote:
> clear_contig_highpages() clears the whole range at once under
> preemptible models. Otherwise it clears PROCESS_PAGES_NON_PREEMPT_BATCH
> pages at a time, with a cond_resched() in between. The batch is 32MB if
> the architecture defines clear_pages(), and one page otherwise.
> 
> Since commit 7dadeaa6e851 ("sched: Further restrict the preemption
> modes"), architectures that support lazy preemption only offer the
> full, lazy and RT preemption models, also with PREEMPT_DYNAMIC. x86 is
> one of them, and it is the only architecture that defines
> clear_pages(). So the 32MB batch is never used: whenever
> clear_contig_highpages() runs under a cooperative model, the batch is
> one page.
> 
> Make that explicit. Clear the whole range when CONFIG_PREEMPTION is
> set, and one page at a time with cond_resched() in between otherwise.
> Drop PROCESS_PAGES_NON_PREEMPT_BATCH, which has no other users.
> 
> PREEMPT_DYNAMIC kernels also have CONFIG_PREEMPTION set, and
> cond_resched() does nothing on any of these kernels. So a build-time
> CONFIG_PREEMPTION check is enough, and preempt_model_preemptible() is
> not needed.
> 
> No functional change intended.
> 
> Suggested-by: David Hildenbrand <david@kernel.org>
> Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> Signed-off-by: Shivank Garg <shivankg@amd.com>
> ---
>

Thanks, I'll wait a bit for more review before applying.

Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>

-- 
Cheers,

David

Re: [PATCH v3] mm/memory: drop PROCESS_PAGES_NON_PREEMPT_BATCH

[Garg, Shivank]

On Mon, 2026-11-09 at 23:00 +0100, David Hildenbrand (Arm) wrote:
> On 11/9/26 20:15, Shivank Garg wrote:
> > clear_contig_highpages() clears the whole range at once under
> > preemptible models. Otherwise it clears PROCESS_PAGES_NON_PREEMPT_BATCH
> > pages at a time, with a cond_resched() in between. The batch is 32MB if
> > the architecture defines clear_pages(), and one page otherwise.
> > 
> > Since commit 7dadeaa6e851 ("sched: Further restrict the preemption
> > modes"), architectures that support lazy preemption only offer the
> > full, lazy and RT preemption models, also with PREEMPT_DYNAMIC. x86 is
> > one of them, and it is the only architecture that defines
> > clear_pages(). So the 32MB batch is never used: whenever
> > clear_contig_highpages() runs under a cooperative model, the batch is
> > one page.
> > 
> > Make that explicit. Clear the whole range when CONFIG_PREEMPTION is
> > set, and one page at a time with cond_resched() in between otherwise.
> > Drop PROCESS_PAGES_NON_PREEMPT_BATCH, which has no other users.
> > 
> > PREEMPT_DYNAMIC kernels also have CONFIG_PREEMPTION set, and
> > cond_resched() does nothing on any of these kernels. So a build-time
> > CONFIG_PREEMPTION check is enough, and preempt_model_preemptible() is
> > not needed.
> > 
> > No functional change intended.
> > 
> > Suggested-by: David Hildenbrand <david@kernel.org>
> > Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> > Signed-off-by: Shivank Garg <shivankg@amd.com>
> > ---
> > 
> 
> Thanks, I'll wait a bit for more review before applying.
> 
> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>


Thanks, David, for the review.

Once this is final, I'll work on the next version of the batch copy
in folio_mc_copy() [1]. :)

[1] https://lore.kernel.org/linux-mm/20261019-batpagecopy-v2-1-d75e7d0dfaae@amd.com/

Regards,
Shivank

Re: [PATCH v3] mm/memory: drop PROCESS_PAGES_NON_PREEMPT_BATCH

[Lorenzo Stoakes (ARM)]

On Mon, Nov 09, 2026 at 07:15:51PM +0000, Shivank Garg wrote:
> clear_contig_highpages() clears the whole range at once under
> preemptible models. Otherwise it clears PROCESS_PAGES_NON_PREEMPT_BATCH
> pages at a time, with a cond_resched() in between. The batch is 32MB if
> the architecture defines clear_pages(), and one page otherwise.
>
> Since commit 7dadeaa6e851 ("sched: Further restrict the preemption
> modes"), architectures that support lazy preemption only offer the
> full, lazy and RT preemption models, also with PREEMPT_DYNAMIC. x86 is
> one of them, and it is the only architecture that defines
> clear_pages(). So the 32MB batch is never used: whenever
> clear_contig_highpages() runs under a cooperative model, the batch is
> one page.
>
> Make that explicit. Clear the whole range when CONFIG_PREEMPTION is
> set, and one page at a time with cond_resched() in between otherwise.
> Drop PROCESS_PAGES_NON_PREEMPT_BATCH, which has no other users.
>
> PREEMPT_DYNAMIC kernels also have CONFIG_PREEMPTION set, and
> cond_resched() does nothing on any of these kernels. So a build-time
> CONFIG_PREEMPTION check is enough, and preempt_model_preemptible() is
> not needed.
>
> No functional change intended.
>
> Suggested-by: David Hildenbrand <david@kernel.org>
> Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> Signed-off-by: Shivank Garg <shivankg@amd.com>

Nice cleanup! LGTM so:

Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>

> ---
> This came up in review of the copy_mc_highpages() series [1]. Pedro
> pointed out that most architecture run with full or lazy preempt, so mm
> does not need to check the preemption model at runtime or batch the copy
> for cond_resched(), and a simple per-page fallback is enough for
> non-preemptible models.
> David suggested doing the same for the zeroing path, and this patch
> does that.
>
> [1] https://lore.kernel.org/all/20261019-batpagecopy-v2-1-d75e7d0dfaae@amd.com
> ---
> Changes in v3:
> - Use clear_user_highpage() in the per-page loop (Pedro).
> - rebase on mm/core.git/for-next.
> - Link to v2: https://lore.kernel.org/r/20261109-clear-pages-batch-v2-1-2e62c4f84f94@amd.com
>
> Changes in v2:
> - Rebased on mm/linux.git/for-next, and drop RFC tag.
> - Link to v1: https://lore.kernel.org/r/20261022-clear-pages-batch-v1-1-13d3bd0b876e@amd.com
> ---
>  include/linux/mm.h | 28 +---------------------------
>  mm/memory.c        | 29 ++++++++++++++++-------------
>  2 files changed, 17 insertions(+), 40 deletions(-)
>
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index e6da76fbe8e8..8d14e45fcf24 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -5250,7 +5250,7 @@ static inline void clear_page_guard(struct zone *zone, struct page *page,
>   *
>   * When running under preemptible models this is not a problem. Under
>   * cooperatively scheduled models, however, the caller is expected to
> - * limit @npages to no more than PROCESS_PAGES_NON_PREEMPT_BATCH.
> + * clear one page at a time and call cond_resched() in between.
>   */
>  static inline void clear_pages(void *addr, unsigned int npages)
>  {
> @@ -5261,32 +5261,6 @@ static inline void clear_pages(void *addr, unsigned int npages)
>  }
>  #endif
>
> -#ifndef PROCESS_PAGES_NON_PREEMPT_BATCH
> -#ifdef clear_pages
> -/*
> - * The architecture defines clear_pages(), and we assume that it is
> - * generally "fast". So choose a batch size large enough to allow the processor
> - * headroom for optimizing the operation and yet small enough that we see
> - * reasonable preemption latency for when this optimization is not possible
> - * (ex. slow microarchitectures, memory bandwidth saturation.)
> - *
> - * With a value of 32MB and assuming a memory bandwidth of ~10GBps, this should
> - * result in worst case preemption latency of around 3ms when clearing pages.
> - *
> - * (See comment above clear_pages() for why preemption latency is a concern
> - * here.)
> - */
> -#define PROCESS_PAGES_NON_PREEMPT_BATCH		(SZ_32M >> PAGE_SHIFT)
> -#else /* !clear_pages */
> -/*
> - * The architecture does not provide a clear_pages() implementation. Assume
> - * that clear_page() -- which clear_pages() will fallback to -- is relatively
> - * slow and choose a small value for PROCESS_PAGES_NON_PREEMPT_BATCH.
> - */
> -#define PROCESS_PAGES_NON_PREEMPT_BATCH		1
> -#endif
> -#endif
> -

Nice to remove this :)

>  #ifdef __HAVE_ARCH_GATE_AREA
>  extern struct vm_area_struct *get_gate_vma(struct mm_struct *mm);
>  extern int in_gate_area_no_mm(unsigned long addr);
> diff --git a/mm/memory.c b/mm/memory.c
> index c9d64170e47a..fcc7ea2a00d1 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -7650,27 +7650,30 @@ static inline int process_huge_page(
>  }
>
>  static void clear_contig_highpages(struct page *page, unsigned long addr,
> -				   unsigned int nr_pages)
> +		unsigned int nr_pages)
>  {
> -	unsigned int i, count;
> +	unsigned int i;
> +
> +	might_sleep();
> +
>  	/*
>  	 * When clearing we want to operate on the largest extent possible to
>  	 * allow for architecture specific extent based optimizations.
>  	 *
> -	 * However, since clear_user_highpages() (and primitives clear_user_pages(),
> -	 * clear_pages()), do not call cond_resched(), limit the unit size when
> -	 * running under non-preemptible scheduling models.
> +	 * clear_user_highpages() (and primitives clear_user_pages(),
> +	 * clear_pages()) do not call cond_resched(). That is fine on
> +	 * preemptible kernels, where the scheduler can preempt the clearing
> +	 * at any point, so clear the whole range at once. Otherwise, clear
> +	 * one page at a time and reschedule in between.
>  	 */
> -	const unsigned int unit = preempt_model_preemptible() ?
> -				   nr_pages : PROCESS_PAGES_NON_PREEMPT_BATCH;
> -
> -	might_sleep();
> +	if (IS_ENABLED(CONFIG_PREEMPTION)) {
> +		clear_user_highpages(page, addr, nr_pages);
> +		return;
> +	}
>
> -	for (i = 0; i < nr_pages; i += count) {
> +	for (i = 0; i < nr_pages; i++) {
>  		cond_resched();
> -
> -		count = min(unit, nr_pages - i);
> -		clear_user_highpages(page + i, addr + i * PAGE_SIZE, count);
> +		clear_user_highpage(page + i, addr + i * PAGE_SIZE);
>  	}

Very lovely :)

>  }
>
>
> ---
> base-commit: 2683df3b7eac1d91380194955ad0c9751454610c
> change-id: 20261021-clear-pages-batch-77f72ef808e4
>
> Best regards,
> --
> Shivank Garg <shivankg@amd.com>
>

--
Cheers, Lorenzo

Re: [PATCH v3] mm/memory: drop PROCESS_PAGES_NON_PREEMPT_BATCH

[David Hildenbrand]

On Mon, 09 Nov 2026 19:15:51 +0000, Shivank Garg wrote:
> clear_contig_highpages() clears the whole range at once under
> preemptible models. Otherwise it clears PROCESS_PAGES_NON_PREEMPT_BATCH
> pages at a time, with a cond_resched() in between. The batch is 32MB if
> the architecture defines clear_pages(), and one page otherwise.
> 
> Since commit 7dadeaa6e851 ("sched: Further restrict the preemption
> modes"), architectures that support lazy preemption only offer the
> full, lazy and RT preemption models, also with PREEMPT_DYNAMIC. x86 is
> one of them, and it is the only architecture that defines
> clear_pages(). So the 32MB batch is never used: whenever
> clear_contig_highpages() runs under a cooperative model, the batch is
> one page.
> 
> [...]

Applied, thanks!

tree: mm/core.git
branch: core-7.5.misc

Applied patches will appear soon in mm-next, followed by linux-next.

Please report any outstanding bugs that were missed during review, so we
can either drop the patch series or decide how to fix it up.

It is encouraged to provide Acked-bys, Reviewed-bys, and Tested-bys even
though the patches have already been applied. If possible, patch trailers
will be updated.

Note that commit hashes shown below are subject to change due to rebase,
trailer updates or similar. Further, this is not a guarantee that the
patches will go upstream. If in doubt, please check the listed branch.

[1/1] mm/memory: drop PROCESS_PAGES_NON_PREEMPT_BATCH
      https://git.kernel.org/mm/core/c/6df321a9

-- 
Cheers,

David

Re: [PATCH v3] mm/memory: drop PROCESS_PAGES_NON_PREEMPT_BATCH

[Peter Zijlstra]

On Mon, Nov 09, 2026 at 07:15:51PM +0000, Shivank Garg wrote:
>  static void clear_contig_highpages(struct page *page, unsigned long addr,
> +		unsigned int nr_pages)
>  {
> +	unsigned int i;
> +
> +	might_sleep();
> +
>  	/*
>  	 * When clearing we want to operate on the largest extent possible to
>  	 * allow for architecture specific extent based optimizations.
>  	 *
> +	 * clear_user_highpages() (and primitives clear_user_pages(),
> +	 * clear_pages()) do not call cond_resched(). That is fine on
> +	 * preemptible kernels, where the scheduler can preempt the clearing
> +	 * at any point, so clear the whole range at once. Otherwise, clear
> +	 * one page at a time and reschedule in between.
>  	 */
> +	if (IS_ENABLED(CONFIG_PREEMPTION)) {
> +		clear_user_highpages(page, addr, nr_pages);
> +		return;
> +	}
>  
> +	for (i = 0; i < nr_pages; i++) {

FWIW one could do:

	for (int i = 0; i < nr_pages; i++) {

and forgo the earlier variable declaration.

>  		cond_resched();
> +		clear_user_highpage(page + i, addr + i * PAGE_SIZE);
>  	}
>  }

Re: [PATCH v3] mm/memory: drop PROCESS_PAGES_NON_PREEMPT_BATCH

[Lorenzo Stoakes (ARM)]

On Wed, Nov 11, 2026 at 01:40:06PM +0100, Peter Zijlstra wrote:
> FWIW one could do:
>
> 	for (int i = 0; i < nr_pages; i++) {
>
> and forgo the earlier variable declaration.

I guess we all got used to the existing patterns and I for one only
recently found out (doh) that this was legal in kernel style (because I saw
it used in KVM)...

--
Cheers, Lorenzo

Re: [PATCH v3] mm/memory: drop PROCESS_PAGES_NON_PREEMPT_BATCH

[Peter Zijlstra]

On Wed, Nov 11, 2026 at 01:12:06PM +0000, Lorenzo Stoakes (ARM) wrote:
> On Wed, Nov 11, 2026 at 01:40:06PM +0100, Peter Zijlstra wrote:
> > FWIW one could do:
> >
> > 	for (int i = 0; i < nr_pages; i++) {
> >
> > and forgo the earlier variable declaration.
> 
> I guess we all got used to the existing patterns and I for one only
> recently found out (doh) that this was legal in kernel style (because I saw
> it used in KVM)...

Right; which is why I keep reminding people this is now 'legal' :-)

Re: [PATCH v3] mm/memory: drop PROCESS_PAGES_NON_PREEMPT_BATCH

[Garg, Shivank]

On Wed, 2026-11-11 at 14:21 +0100, Peter Zijlstra wrote:
> On Wed, Nov 11, 2026 at 01:12:06PM +0000, Lorenzo Stoakes (ARM) wrote:
> > On Wed, Nov 11, 2026 at 01:40:06PM +0100, Peter Zijlstra wrote:
> > > FWIW one could do:
> > > 
> > > 	for (int i = 0; i < nr_pages; i++) {
> > > 
> > > and forgo the earlier variable declaration.
> > 
> > I guess we all got used to the existing patterns and I for one only
> > recently found out (doh) that this was legal in kernel style (because I saw
> > it used in KVM)...
> 
> Right; which is why I keep reminding people this is now 'legal' :-)


Thank you, Peter.
Good to know this and I'll keep in mind for next time.

btw the patch is already in mm/core.git. I don't have a strong
opinion on this, but I'm happy to send a V4 if you'd prefer.

Thanks,
Shivank

Re: [PATCH v3] mm/memory: drop PROCESS_PAGES_NON_PREEMPT_BATCH

[David Hildenbrand (Arm)]

On 11/11/26 20:07, Garg, Shivank wrote:
> On Wed, 2026-11-11 at 14:21 +0100, Peter Zijlstra wrote:
>> On Wed, Nov 11, 2026 at 01:12:06PM +0000, Lorenzo Stoakes (ARM) wrote:
>>>
>>> I guess we all got used to the existing patterns and I for one only
>>> recently found out (doh) that this was legal in kernel style (because I saw
>>> it used in KVM)...
>>
>> Right; which is why I keep reminding people this is now 'legal' :-)
> 
> 
> Thank you, Peter.
> Good to know this and I'll keep in mind for next time.
> 
> btw the patch is already in mm/core.git. I don't have a strong
> opinion on this, but I'm happy to send a V4 if you'd prefer.

I can fix it up, but we can also just leave it like that. (I tend to prefer the
old way of doing it, but in the end I don't care :) )

-- 
Cheers,

David