* [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall()
@ 2026-05-18 9:49 Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 1/6] vdso/datastore: Rename data pages variable Thomas Weißschuh
` (6 more replies)
0 siblings, 7 replies; 12+ messages in thread
From: Thomas Weißschuh @ 2026-05-18 9:49 UTC (permalink / raw)
To: Anna-Maria Behnsen, Frederic Weisbecker, Andy Lutomirski,
Vincenzo Frascino, Thomas Gleixner
Cc: Nam Cao, Sebastian Andrzej Siewior, linux-kernel, Andrew Morton,
David Hildenbrand, Jason Gunthorpe, John Hubbard, Peter Xu,
linux-mm, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka,
Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Thomas Weißschuh
While mlockall() is meant to lock page *memory*, effectively it will
also create and lock the corresponding page table entries.
Latency-sensitive applications expect not to experience any pagefaults
after calling mlockall(). However mlockall() ignores VM_IO mappings,
which is used by the generic vDSO datastore.
While the fault handler itself is very fast, going through the full
pagefault exception handling is much slower, on the order of 20us in a
test machine.
Since the memory behind the datastore mappings is always present and
accessible it is not necessary to use VM_IO for them.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
---
Changes in v3:
- Rebase on v7.1-rc1
- Stop using zero page, which can not be mapped in this way
- Link to v2: https://lore.kernel.org/r/20250901-vdso-mlockall-v2-0-68f5a6f03345@linutronix.de
Changes in v2:
- Stop using nth_page() which is being removed
- Link to v1: https://lore.kernel.org/r/20250812-vdso-mlockall-v1-0-2f49ba7cf819@linutronix.de
---
Thomas Weißschuh (6):
vdso/datastore: Rename data pages variable
vdso/datastore: Map pages in terms of the faults pgoff
vdso/datastore: Map zeroed pages for unavailable data
vdso/datastore: Explicitly prevent remote access to timens vvar page
vdso/datastore: Allow prefaulting by mlockall()
vdso/datastore: Simplify the mapping logic for VDSO_TIME_PAGE_OFFSET
kernel/time/namespace_vdso.c | 7 ++---
lib/vdso/datastore.c | 65 ++++++++++++++++++++------------------------
2 files changed, 32 insertions(+), 40 deletions(-)
---
base-commit: b0f603a5e7c076c7e424b5a621022b3bec16225b
change-id: 20250721-vdso-mlockall-461bb33205b1
Best regards,
--
Thomas Weißschuh <thomas.weissschuh@linutronix.de>
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v3 1/6] vdso/datastore: Rename data pages variable
2026-05-18 9:49 [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall() Thomas Weißschuh
@ 2026-05-18 9:49 ` Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 2/6] vdso/datastore: Map pages in terms of the faults pgoff Thomas Weißschuh
` (5 subsequent siblings)
6 siblings, 0 replies; 12+ messages in thread
From: Thomas Weißschuh @ 2026-05-18 9:49 UTC (permalink / raw)
To: Anna-Maria Behnsen, Frederic Weisbecker, Andy Lutomirski,
Vincenzo Frascino, Thomas Gleixner
Cc: Nam Cao, Sebastian Andrzej Siewior, linux-kernel, Andrew Morton,
David Hildenbrand, Jason Gunthorpe, John Hubbard, Peter Xu,
linux-mm, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka,
Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Thomas Weißschuh
An upcoming change will make this a file-scoped variable, for which it
should have a clearer name.
Rename the variable to prepare for that.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Tested-by: Nam Cao <namcao@linutronix.de>
---
lib/vdso/datastore.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/lib/vdso/datastore.c b/lib/vdso/datastore.c
index cf5d784a4a5a..9d66ac2fdb8d 100644
--- a/lib/vdso/datastore.c
+++ b/lib/vdso/datastore.c
@@ -32,7 +32,7 @@ struct vdso_arch_data *vdso_k_arch_data __refdata =
void __init vdso_setup_data_pages(void)
{
unsigned int order = get_order(VDSO_NR_PAGES * PAGE_SIZE);
- struct page *pages;
+ struct page *vdso_data_pages;
/*
* Allocate the data pages dynamically. SPARC does not support mapping
@@ -42,24 +42,24 @@ void __init vdso_setup_data_pages(void)
* Do not use folios. In time namespaces the pages are mapped in a different order
* to userspace, which is not handled by the folio optimizations in finish_fault().
*/
- pages = alloc_pages(GFP_KERNEL, order);
- if (!pages)
+ vdso_data_pages = alloc_pages(GFP_KERNEL, order);
+ if (!vdso_data_pages)
panic("Unable to allocate VDSO storage pages");
/* The pages are mapped one-by-one into userspace and each one needs to be refcounted. */
- split_page(pages, order);
+ split_page(vdso_data_pages, order);
/* Move the data already written by other subsystems to the new pages */
- memcpy(page_address(pages), vdso_initdata, VDSO_NR_PAGES * PAGE_SIZE);
+ memcpy(page_address(vdso_data_pages), vdso_initdata, VDSO_NR_PAGES * PAGE_SIZE);
if (IS_ENABLED(CONFIG_GENERIC_GETTIMEOFDAY))
- vdso_k_time_data = page_address(pages + VDSO_TIME_PAGE_OFFSET);
+ vdso_k_time_data = page_address(vdso_data_pages + VDSO_TIME_PAGE_OFFSET);
if (IS_ENABLED(CONFIG_VDSO_GETRANDOM))
- vdso_k_rng_data = page_address(pages + VDSO_RNG_PAGE_OFFSET);
+ vdso_k_rng_data = page_address(vdso_data_pages + VDSO_RNG_PAGE_OFFSET);
if (IS_ENABLED(CONFIG_ARCH_HAS_VDSO_ARCH_DATA))
- vdso_k_arch_data = page_address(pages + VDSO_ARCH_PAGES_START);
+ vdso_k_arch_data = page_address(vdso_data_pages + VDSO_ARCH_PAGES_START);
}
static vm_fault_t vvar_fault(const struct vm_special_mapping *sm,
--
2.53.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v3 2/6] vdso/datastore: Map pages in terms of the faults pgoff
2026-05-18 9:49 [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall() Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 1/6] vdso/datastore: Rename data pages variable Thomas Weißschuh
@ 2026-05-18 9:49 ` Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 3/6] vdso/datastore: Map zeroed pages for unavailable data Thomas Weißschuh
` (4 subsequent siblings)
6 siblings, 0 replies; 12+ messages in thread
From: Thomas Weißschuh @ 2026-05-18 9:49 UTC (permalink / raw)
To: Anna-Maria Behnsen, Frederic Weisbecker, Andy Lutomirski,
Vincenzo Frascino, Thomas Gleixner
Cc: Nam Cao, Sebastian Andrzej Siewior, linux-kernel, Andrew Morton,
David Hildenbrand, Jason Gunthorpe, John Hubbard, Peter Xu,
linux-mm, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka,
Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Thomas Weißschuh
To support mlockall() on the datapages the VMA can have no holes where
inner pages return VM_FAULT_SIGBUS. An upcoming change will avoid these
holes by mapping a zeroed pages into these holes. That logic will be
simpler when the mapping logic is based on vmf->pgoff instead of the
vdso_k_ symbols.
Switch to the equivalent vmf->pgoff logic.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Tested-by: Nam Cao <namcao@linutronix.de>
---
lib/vdso/datastore.c | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/lib/vdso/datastore.c b/lib/vdso/datastore.c
index 9d66ac2fdb8d..8aabb289a7b5 100644
--- a/lib/vdso/datastore.c
+++ b/lib/vdso/datastore.c
@@ -29,10 +29,11 @@ struct vdso_arch_data *vdso_k_arch_data __refdata =
(void *)&vdso_initdata[VDSO_ARCH_PAGES_START * PAGE_SIZE];
#endif /* CONFIG_ARCH_HAS_VDSO_ARCH_DATA */
+static struct page *vdso_data_pages __ro_after_init;
+
void __init vdso_setup_data_pages(void)
{
unsigned int order = get_order(VDSO_NR_PAGES * PAGE_SIZE);
- struct page *vdso_data_pages;
/*
* Allocate the data pages dynamically. SPARC does not support mapping
@@ -67,13 +68,13 @@ static vm_fault_t vvar_fault(const struct vm_special_mapping *sm,
{
struct page *page, *timens_page;
+ page = vdso_data_pages + vmf->pgoff;
timens_page = find_timens_vvar_page(vma);
switch (vmf->pgoff) {
case VDSO_TIME_PAGE_OFFSET:
if (!IS_ENABLED(CONFIG_GENERIC_GETTIMEOFDAY))
return VM_FAULT_SIGBUS;
- page = virt_to_page(vdso_k_time_data);
if (timens_page) {
/*
* Fault in VVAR page too, since it will be accessed
@@ -99,17 +100,15 @@ static vm_fault_t vvar_fault(const struct vm_special_mapping *sm,
*/
if (!IS_ENABLED(CONFIG_TIME_NS) || !timens_page)
return VM_FAULT_SIGBUS;
- page = virt_to_page(vdso_k_time_data);
+ page = vdso_data_pages + VDSO_TIME_PAGE_OFFSET;
break;
case VDSO_RNG_PAGE_OFFSET:
if (!IS_ENABLED(CONFIG_VDSO_GETRANDOM))
return VM_FAULT_SIGBUS;
- page = virt_to_page(vdso_k_rng_data);
break;
case VDSO_ARCH_PAGES_START ... VDSO_ARCH_PAGES_END:
if (!IS_ENABLED(CONFIG_ARCH_HAS_VDSO_ARCH_DATA))
return VM_FAULT_SIGBUS;
- page = virt_to_page(vdso_k_arch_data) + vmf->pgoff - VDSO_ARCH_PAGES_START;
break;
default:
return VM_FAULT_SIGBUS;
--
2.53.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v3 3/6] vdso/datastore: Map zeroed pages for unavailable data
2026-05-18 9:49 [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall() Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 1/6] vdso/datastore: Rename data pages variable Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 2/6] vdso/datastore: Map pages in terms of the faults pgoff Thomas Weißschuh
@ 2026-05-18 9:49 ` Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 4/6] vdso/datastore: Explicitly prevent remote access to timens vvar page Thomas Weißschuh
` (3 subsequent siblings)
6 siblings, 0 replies; 12+ messages in thread
From: Thomas Weißschuh @ 2026-05-18 9:49 UTC (permalink / raw)
To: Anna-Maria Behnsen, Frederic Weisbecker, Andy Lutomirski,
Vincenzo Frascino, Thomas Gleixner
Cc: Nam Cao, Sebastian Andrzej Siewior, linux-kernel, Andrew Morton,
David Hildenbrand, Jason Gunthorpe, John Hubbard, Peter Xu,
linux-mm, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka,
Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Thomas Weißschuh
mlockall() stops if a page in a VMA is unmappable. As the datastore VMA
can contain holes, mlockall() would not process all data pages.
Replace the mapping error VM_FAULT_SIGBUS by just mapping the underlying
unused and zeroed-out data page. The vDSO will not access these pages in
any case and for other userspace these pages have undefined contents.
This will allow mlockall() to process all pages within the VMA as soon
as VM_IO is removed from the VMA.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Tested-by: Nam Cao <namcao@linutronix.de>
---
lib/vdso/datastore.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/lib/vdso/datastore.c b/lib/vdso/datastore.c
index 8aabb289a7b5..f9a1f0fb7ccd 100644
--- a/lib/vdso/datastore.c
+++ b/lib/vdso/datastore.c
@@ -74,7 +74,7 @@ static vm_fault_t vvar_fault(const struct vm_special_mapping *sm,
switch (vmf->pgoff) {
case VDSO_TIME_PAGE_OFFSET:
if (!IS_ENABLED(CONFIG_GENERIC_GETTIMEOFDAY))
- return VM_FAULT_SIGBUS;
+ break;
if (timens_page) {
/*
* Fault in VVAR page too, since it will be accessed
@@ -99,16 +99,11 @@ static vm_fault_t vvar_fault(const struct vm_special_mapping *sm,
* See also the comment near timens_setup_vdso_data().
*/
if (!IS_ENABLED(CONFIG_TIME_NS) || !timens_page)
- return VM_FAULT_SIGBUS;
+ break;
page = vdso_data_pages + VDSO_TIME_PAGE_OFFSET;
break;
case VDSO_RNG_PAGE_OFFSET:
- if (!IS_ENABLED(CONFIG_VDSO_GETRANDOM))
- return VM_FAULT_SIGBUS;
- break;
case VDSO_ARCH_PAGES_START ... VDSO_ARCH_PAGES_END:
- if (!IS_ENABLED(CONFIG_ARCH_HAS_VDSO_ARCH_DATA))
- return VM_FAULT_SIGBUS;
break;
default:
return VM_FAULT_SIGBUS;
--
2.53.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v3 4/6] vdso/datastore: Explicitly prevent remote access to timens vvar page
2026-05-18 9:49 [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall() Thomas Weißschuh
` (2 preceding siblings ...)
2026-05-18 9:49 ` [PATCH v3 3/6] vdso/datastore: Map zeroed pages for unavailable data Thomas Weißschuh
@ 2026-05-18 9:49 ` Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 5/6] vdso/datastore: Allow prefaulting by mlockall() Thomas Weißschuh
` (2 subsequent siblings)
6 siblings, 0 replies; 12+ messages in thread
From: Thomas Weißschuh @ 2026-05-18 9:49 UTC (permalink / raw)
To: Anna-Maria Behnsen, Frederic Weisbecker, Andy Lutomirski,
Vincenzo Frascino, Thomas Gleixner
Cc: Nam Cao, Sebastian Andrzej Siewior, linux-kernel, Andrew Morton,
David Hildenbrand, Jason Gunthorpe, John Hubbard, Peter Xu,
linux-mm, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka,
Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Thomas Weißschuh
The fault handler for the timens page does not have access to the target
task and therefore can not be invoked remotely.
Currently the handler relies on the fact that the vvar mapping is marked as
VM_IO and VM_PFNMAP for which the mm core always prevents remote access.
However the VM_IO and VM_PFNMAP flags are going to be removed.
Add an explicit check to prevent remote access to the mapping.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Tested-by: Nam Cao <namcao@linutronix.de>
---
kernel/time/namespace_vdso.c | 7 ++-----
lib/vdso/datastore.c | 3 +++
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/kernel/time/namespace_vdso.c b/kernel/time/namespace_vdso.c
index 0d74d160eec9..5ac7b6a6d3a8 100644
--- a/kernel/time/namespace_vdso.c
+++ b/kernel/time/namespace_vdso.c
@@ -74,11 +74,8 @@ struct page *find_timens_vvar_page(struct vm_area_struct *vma)
return current->nsproxy->time_ns->vvar_page;
/*
- * VM_PFNMAP | VM_IO protect .fault() handler from being called
- * through interfaces like /proc/$pid/mem or
- * process_vm_{readv,writev}() as long as there's no .access()
- * in special_mapping_vmops().
- * For more details check_vma_flags() and __access_remote_vm()
+ * vvar_fault() protects this from being called through remote interfaces like
+ * /proc/$pid/mem or process_vm_{readv,writev}().
*/
WARN(1, "vvar_page accessed remotely");
diff --git a/lib/vdso/datastore.c b/lib/vdso/datastore.c
index f9a1f0fb7ccd..3381ccf9edb1 100644
--- a/lib/vdso/datastore.c
+++ b/lib/vdso/datastore.c
@@ -68,6 +68,9 @@ static vm_fault_t vvar_fault(const struct vm_special_mapping *sm,
{
struct page *page, *timens_page;
+ if (unlikely(vmf->flags & FAULT_FLAG_REMOTE))
+ return VM_FAULT_SIGBUS;
+
page = vdso_data_pages + vmf->pgoff;
timens_page = find_timens_vvar_page(vma);
--
2.53.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v3 5/6] vdso/datastore: Allow prefaulting by mlockall()
2026-05-18 9:49 [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall() Thomas Weißschuh
` (3 preceding siblings ...)
2026-05-18 9:49 ` [PATCH v3 4/6] vdso/datastore: Explicitly prevent remote access to timens vvar page Thomas Weißschuh
@ 2026-05-18 9:49 ` Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 6/6] vdso/datastore: Simplify the mapping logic for VDSO_TIME_PAGE_OFFSET Thomas Weißschuh
2026-06-02 18:41 ` [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall() David Hildenbrand (Arm)
6 siblings, 0 replies; 12+ messages in thread
From: Thomas Weißschuh @ 2026-05-18 9:49 UTC (permalink / raw)
To: Anna-Maria Behnsen, Frederic Weisbecker, Andy Lutomirski,
Vincenzo Frascino, Thomas Gleixner
Cc: Nam Cao, Sebastian Andrzej Siewior, linux-kernel, Andrew Morton,
David Hildenbrand, Jason Gunthorpe, John Hubbard, Peter Xu,
linux-mm, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka,
Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Thomas Weißschuh
While mlockall() is meant to lock page *memory*, effectively it will
also create and lock the corresponding page table entries.
Latency-sensitive applications expect not to experience any pagefaults
after calling mlockall(). However mlockall() ignores VM_IO mappings,
which is used by the generic vDSO datastore.
While the fault handler itself is very fast, going through the full
pagefault exception handling is much slower, on the order of 20us in a
test machine.
Since the memory behind the datastore mappings is always present and
accessible it is not necessary to use VM_IO for them.
The data page mapping is now also aligned with the architecture-specific
code pages. Some architecture-specific data pages, like the x86 VCLOCK
pages, continue to use VM_IO as they are not always mappable. They will
require their own special handling later when the general aproach has
been agreed upon.
Regular mlock() would also work, but userspace does not know the boundaries
of the vDSO.
Reported-by: Nam Cao <namcao@linutronix.de>
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Tested-by: Nam Cao <namcao@linutronix.de>
---
lib/vdso/datastore.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/vdso/datastore.c b/lib/vdso/datastore.c
index 3381ccf9edb1..3f837cb6a886 100644
--- a/lib/vdso/datastore.c
+++ b/lib/vdso/datastore.c
@@ -125,7 +125,7 @@ const struct vm_special_mapping vdso_vvar_mapping = {
struct vm_area_struct *vdso_install_vvar_mapping(struct mm_struct *mm, unsigned long addr)
{
return _install_special_mapping(mm, addr, VDSO_NR_PAGES * PAGE_SIZE,
- VM_READ | VM_MAYREAD | VM_IO | VM_DONTDUMP |
+ VM_READ | VM_MAYREAD | VM_DONTDUMP |
VM_MIXEDMAP | VM_SEALED_SYSMAP,
&vdso_vvar_mapping);
}
--
2.53.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v3 6/6] vdso/datastore: Simplify the mapping logic for VDSO_TIME_PAGE_OFFSET
2026-05-18 9:49 [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall() Thomas Weißschuh
` (4 preceding siblings ...)
2026-05-18 9:49 ` [PATCH v3 5/6] vdso/datastore: Allow prefaulting by mlockall() Thomas Weißschuh
@ 2026-05-18 9:49 ` Thomas Weißschuh
2026-06-02 18:41 ` [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall() David Hildenbrand (Arm)
6 siblings, 0 replies; 12+ messages in thread
From: Thomas Weißschuh @ 2026-05-18 9:49 UTC (permalink / raw)
To: Anna-Maria Behnsen, Frederic Weisbecker, Andy Lutomirski,
Vincenzo Frascino, Thomas Gleixner
Cc: Nam Cao, Sebastian Andrzej Siewior, linux-kernel, Andrew Morton,
David Hildenbrand, Jason Gunthorpe, John Hubbard, Peter Xu,
linux-mm, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka,
Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
Thomas Weißschuh
The logic for CONFIG_GENERIC_GETTIMEOFDAY=n and !timens_page is
identical now.
Use this to simplify the logic a bit.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
---
lib/vdso/datastore.c | 28 +++++++++++++---------------
1 file changed, 13 insertions(+), 15 deletions(-)
diff --git a/lib/vdso/datastore.c b/lib/vdso/datastore.c
index 3f837cb6a886..21f09fa5e254 100644
--- a/lib/vdso/datastore.c
+++ b/lib/vdso/datastore.c
@@ -76,22 +76,20 @@ static vm_fault_t vvar_fault(const struct vm_special_mapping *sm,
switch (vmf->pgoff) {
case VDSO_TIME_PAGE_OFFSET:
- if (!IS_ENABLED(CONFIG_GENERIC_GETTIMEOFDAY))
+ if (!IS_ENABLED(CONFIG_GENERIC_GETTIMEOFDAY) || !timens_page)
break;
- if (timens_page) {
- /*
- * Fault in VVAR page too, since it will be accessed
- * to get clock data anyway.
- */
- unsigned long addr;
- vm_fault_t err;
-
- addr = vmf->address + VDSO_TIMENS_PAGE_OFFSET * PAGE_SIZE;
- err = vmf_insert_page(vma, addr, page);
- if (unlikely(err & VM_FAULT_ERROR))
- return err;
- page = timens_page;
- }
+ /*
+ * Fault in VVAR page too, since it will be accessed
+ * to get clock data anyway.
+ */
+ unsigned long addr;
+ vm_fault_t err;
+
+ addr = vmf->address + VDSO_TIMENS_PAGE_OFFSET * PAGE_SIZE;
+ err = vmf_insert_page(vma, addr, page);
+ if (unlikely(err & VM_FAULT_ERROR))
+ return err;
+ page = timens_page;
break;
case VDSO_TIMENS_PAGE_OFFSET:
/*
--
2.53.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall()
2026-05-18 9:49 [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall() Thomas Weißschuh
` (5 preceding siblings ...)
2026-05-18 9:49 ` [PATCH v3 6/6] vdso/datastore: Simplify the mapping logic for VDSO_TIME_PAGE_OFFSET Thomas Weißschuh
@ 2026-06-02 18:41 ` David Hildenbrand (Arm)
2026-06-03 10:10 ` Thomas Weißschuh
6 siblings, 1 reply; 12+ messages in thread
From: David Hildenbrand (Arm) @ 2026-06-02 18:41 UTC (permalink / raw)
To: Thomas Weißschuh, Anna-Maria Behnsen, Frederic Weisbecker,
Andy Lutomirski, Vincenzo Frascino, Thomas Gleixner
Cc: Nam Cao, Sebastian Andrzej Siewior, linux-kernel, Andrew Morton,
Jason Gunthorpe, John Hubbard, Peter Xu, linux-mm,
Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko
On 5/18/26 11:49, Thomas Weißschuh wrote:
> While mlockall() is meant to lock page *memory*, effectively it will
> also create and lock the corresponding page table entries.
> Latency-sensitive applications expect not to experience any pagefaults
> after calling mlockall(). However mlockall() ignores VM_IO mappings,
> which is used by the generic vDSO datastore.
> While the fault handler itself is very fast, going through the full
> pagefault exception handling is much slower, on the order of 20us in a
> test machine.
>
> Since the memory behind the datastore mappings is always present and
> accessible it is not necessary to use VM_IO for them.
GUP also refuses VM_IO, so this change unlocks GUP'ing these pages and using
them for DIRECT_IO etc.
Well, I assume that GUP-fast (no VMA available) would be able to GUP them
already. But mechanisms that don't use GUP-fast would get reliably blocked for now.
I've been wondering for a while, whether the VM_IO for GUP is still required.
IIRC, we want to disallow read/write access that could have side effects through
ptrace, including process_vm_*(). For memory-mapped I/O that might certainly the
case.
With your change, you'd also unlock get_dump_page() on vdso pages.
Is that all ok and desired?
--
Cheers,
David
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall()
2026-06-02 18:41 ` [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall() David Hildenbrand (Arm)
@ 2026-06-03 10:10 ` Thomas Weißschuh
2026-06-03 11:35 ` David Hildenbrand (Arm)
0 siblings, 1 reply; 12+ messages in thread
From: Thomas Weißschuh @ 2026-06-03 10:10 UTC (permalink / raw)
To: David Hildenbrand (Arm)
Cc: Anna-Maria Behnsen, Frederic Weisbecker, Andy Lutomirski,
Vincenzo Frascino, Thomas Gleixner, Nam Cao,
Sebastian Andrzej Siewior, linux-kernel, Andrew Morton,
Jason Gunthorpe, John Hubbard, Peter Xu, linux-mm,
Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko
On Tue, Jun 02, 2026 at 08:41:31PM +0200, David Hildenbrand (Arm) wrote:
> On 5/18/26 11:49, Thomas Weißschuh wrote:
> > While mlockall() is meant to lock page *memory*, effectively it will
> > also create and lock the corresponding page table entries.
> > Latency-sensitive applications expect not to experience any pagefaults
> > after calling mlockall(). However mlockall() ignores VM_IO mappings,
> > which is used by the generic vDSO datastore.
> > While the fault handler itself is very fast, going through the full
> > pagefault exception handling is much slower, on the order of 20us in a
> > test machine.
> >
> > Since the memory behind the datastore mappings is always present and
> > accessible it is not necessary to use VM_IO for them.
>
> GUP also refuses VM_IO, so this change unlocks GUP'ing these pages and using
> them for DIRECT_IO etc.
>
> Well, I assume that GUP-fast (no VMA available) would be able to GUP them
> already. But mechanisms that don't use GUP-fast would get reliably blocked for now.
I am not sure about all of the side-effects of GUP.
Userspace is allowed to read this data in any way it wants. It is *not* allowed
to interpret it however, which is obviously not enforcable anyways.
*All* writes from userspace need to be rejected. But those are forbidden via
VMA flags which as far as I can see are respected by GUP.
> I've been wondering for a while, whether the VM_IO for GUP is still required.
If that restriction goes away, mlockall() would work automatically for the vDSO
data pages, no?
> IIRC, we want to disallow read/write access that could have side effects through
> ptrace, including process_vm_*(). For memory-mapped I/O that might certainly the
> case.
There are no side-effects, this is just global shared memory.
(But as mentioned before, writes can not be allowed)
Accesses from other tasks need to be prevented, which is what
patch 4 tries to do based on FAULT_FLAG_REMOTE.
The only data in these pages which is not globally shared and accessible
are the time namespace offsets. But /proc/$PID/timens_offsets is
world-readable anyways.
> With your change, you'd also unlock get_dump_page() on vdso pages.
We use VM_DONTDUMP on the VMA, shouldn't that prevent dumping?
Dumping these pages is not a (security) issue, just pointless.
> Is that all ok and desired?
It is not necessarily desired, but shouldn't hurt.
Thomas
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall()
2026-06-03 10:10 ` Thomas Weißschuh
@ 2026-06-03 11:35 ` David Hildenbrand (Arm)
2026-06-03 12:47 ` Thomas Weißschuh
0 siblings, 1 reply; 12+ messages in thread
From: David Hildenbrand (Arm) @ 2026-06-03 11:35 UTC (permalink / raw)
To: Thomas Weißschuh
Cc: Anna-Maria Behnsen, Frederic Weisbecker, Andy Lutomirski,
Vincenzo Frascino, Thomas Gleixner, Nam Cao,
Sebastian Andrzej Siewior, linux-kernel, Andrew Morton,
Jason Gunthorpe, John Hubbard, Peter Xu, linux-mm,
Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko
On 6/3/26 12:10, Thomas Weißschuh wrote:
> On Tue, Jun 02, 2026 at 08:41:31PM +0200, David Hildenbrand (Arm) wrote:
>> On 5/18/26 11:49, Thomas Weißschuh wrote:
>>> While mlockall() is meant to lock page *memory*, effectively it will
>>> also create and lock the corresponding page table entries.
>>> Latency-sensitive applications expect not to experience any pagefaults
>>> after calling mlockall(). However mlockall() ignores VM_IO mappings,
>>> which is used by the generic vDSO datastore.
>>> While the fault handler itself is very fast, going through the full
>>> pagefault exception handling is much slower, on the order of 20us in a
>>> test machine.
>>>
>>> Since the memory behind the datastore mappings is always present and
>>> accessible it is not necessary to use VM_IO for them.
>>
>> GUP also refuses VM_IO, so this change unlocks GUP'ing these pages and using
>> them for DIRECT_IO etc.
>>
>> Well, I assume that GUP-fast (no VMA available) would be able to GUP them
>> already. But mechanisms that don't use GUP-fast would get reliably blocked for now.
>
> I am not sure about all of the side-effects of GUP.
> Userspace is allowed to read this data in any way it wants.
Ok, good!
> It is *not* allowed
> to interpret it however, which is obviously not enforcable anyways.
> *All* writes from userspace need to be rejected. But those are forbidden via
> VMA flags which as far as I can see are respected by GUP.
Yes, even for FOLL_FORCE (see below).
>
>> I've been wondering for a while, whether the VM_IO for GUP is still required.
>
> If that restriction goes away, mlockall() would work automatically for the vDSO
> data pages, no?
Yes. But I guess there is this "debugger should not read/write memory with
side-effects thing.
Again, the GUP-fast interaction is a bit confusing, because that should still
succeed.
But debuggers don't use that. So the VM_IO protects that for now.
>
>> IIRC, we want to disallow read/write access that could have side effects through
>> ptrace, including process_vm_*(). For memory-mapped I/O that might certainly the
>> case.
>
> There are no side-effects, this is just global shared memory.
> (But as mentioned before, writes can not be allowed)
> Accesses from other tasks need to be prevented, which is what
> patch 4 tries to do based on FAULT_FLAG_REMOTE.
>
> The only data in these pages which is not globally shared and accessible
> are the time namespace offsets. But /proc/$PID/timens_offsets is
> world-readable anyways.
>
>> With your change, you'd also unlock get_dump_page() on vdso pages.
>
> We use VM_DONTDUMP on the VMA, shouldn't that prevent dumping?
> Dumping these pages is not a (security) issue, just pointless.
Ah, right.
So we still have
VM_READ | VM_MAYREAD | VM_DONTDUMP | VM_MIXEDMAP ...
So even FOLL_FORCE cannot write to it.
It might be good to document the change (pages can not be GUP'ed, including
being read by ptrace etc). But I agree that it should likely be fine.
--
Cheers,
David
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall()
2026-06-03 11:35 ` David Hildenbrand (Arm)
@ 2026-06-03 12:47 ` Thomas Weißschuh
2026-06-03 12:57 ` David Hildenbrand (Arm)
0 siblings, 1 reply; 12+ messages in thread
From: Thomas Weißschuh @ 2026-06-03 12:47 UTC (permalink / raw)
To: David Hildenbrand (Arm)
Cc: Anna-Maria Behnsen, Frederic Weisbecker, Andy Lutomirski,
Vincenzo Frascino, Thomas Gleixner, Nam Cao,
Sebastian Andrzej Siewior, linux-kernel, Andrew Morton,
Jason Gunthorpe, John Hubbard, Peter Xu, linux-mm,
Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko
On Wed, Jun 03, 2026 at 01:35:19PM +0200, David Hildenbrand (Arm) wrote:
> On 6/3/26 12:10, Thomas Weißschuh wrote:
> > On Tue, Jun 02, 2026 at 08:41:31PM +0200, David Hildenbrand (Arm) wrote:
> >> On 5/18/26 11:49, Thomas Weißschuh wrote:
(...)
> >> IIRC, we want to disallow read/write access that could have side effects through
> >> ptrace, including process_vm_*(). For memory-mapped I/O that might certainly the
> >> case.
> >
> > There are no side-effects, this is just global shared memory.
> > (But as mentioned before, writes can not be allowed)
> > Accesses from other tasks need to be prevented, which is what
> > patch 4 tries to do based on FAULT_FLAG_REMOTE.
> >
> > The only data in these pages which is not globally shared and accessible
> > are the time namespace offsets. But /proc/$PID/timens_offsets is
> > world-readable anyways.
> >
> >> With your change, you'd also unlock get_dump_page() on vdso pages.
> >
> > We use VM_DONTDUMP on the VMA, shouldn't that prevent dumping?
> > Dumping these pages is not a (security) issue, just pointless.
>
> Ah, right.
>
> So we still have
>
> VM_READ | VM_MAYREAD | VM_DONTDUMP | VM_MIXEDMAP ...
>
> So even FOLL_FORCE cannot write to it.
>
> It might be good to document the change (pages can not be GUP'ed, including
> being read by ptrace etc). But I agree that it should likely be fine.
Should this "not" have been a "now"?
I'll extend the descriptions and will resend the patches next cycle.
Thanks for taking a look!
Thomas
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall()
2026-06-03 12:47 ` Thomas Weißschuh
@ 2026-06-03 12:57 ` David Hildenbrand (Arm)
0 siblings, 0 replies; 12+ messages in thread
From: David Hildenbrand (Arm) @ 2026-06-03 12:57 UTC (permalink / raw)
To: Thomas Weißschuh
Cc: Anna-Maria Behnsen, Frederic Weisbecker, Andy Lutomirski,
Vincenzo Frascino, Thomas Gleixner, Nam Cao,
Sebastian Andrzej Siewior, linux-kernel, Andrew Morton,
Jason Gunthorpe, John Hubbard, Peter Xu, linux-mm,
Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko
On 6/3/26 14:47, Thomas Weißschuh wrote:
> On Wed, Jun 03, 2026 at 01:35:19PM +0200, David Hildenbrand (Arm) wrote:
>> On 6/3/26 12:10, Thomas Weißschuh wrote:
>
> (...)
>
>>>
>>> There are no side-effects, this is just global shared memory.
>>> (But as mentioned before, writes can not be allowed)
>>> Accesses from other tasks need to be prevented, which is what
>>> patch 4 tries to do based on FAULT_FLAG_REMOTE.
>>>
>>> The only data in these pages which is not globally shared and accessible
>>> are the time namespace offsets. But /proc/$PID/timens_offsets is
>>> world-readable anyways.
>>>
>>>
>>> We use VM_DONTDUMP on the VMA, shouldn't that prevent dumping?
>>> Dumping these pages is not a (security) issue, just pointless.
>>
>> Ah, right.
>>
>> So we still have
>>
>> VM_READ | VM_MAYREAD | VM_DONTDUMP | VM_MIXEDMAP ...
>>
>> So even FOLL_FORCE cannot write to it.
>>
>> It might be good to document the change (pages can not be GUP'ed, including
>> being read by ptrace etc). But I agree that it should likely be fine.
>
> Should this "not" have been a "now"?
Yes! :)
--
Cheers,
David
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2026-06-03 12:57 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-05-18 9:49 [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall() Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 1/6] vdso/datastore: Rename data pages variable Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 2/6] vdso/datastore: Map pages in terms of the faults pgoff Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 3/6] vdso/datastore: Map zeroed pages for unavailable data Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 4/6] vdso/datastore: Explicitly prevent remote access to timens vvar page Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 5/6] vdso/datastore: Allow prefaulting by mlockall() Thomas Weißschuh
2026-05-18 9:49 ` [PATCH v3 6/6] vdso/datastore: Simplify the mapping logic for VDSO_TIME_PAGE_OFFSET Thomas Weißschuh
2026-06-02 18:41 ` [PATCH v3 0/6] vdso/datastore: Allow prefaulting by mlockall() David Hildenbrand (Arm)
2026-06-03 10:10 ` Thomas Weißschuh
2026-06-03 11:35 ` David Hildenbrand (Arm)
2026-06-03 12:47 ` Thomas Weißschuh
2026-06-03 12:57 ` David Hildenbrand (Arm)
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®