[PATCH 0/3] Fix flush_dcache_page() for usage from irq context on ARM, NIOS2 and PARISC

[PATCH 0/3] Fix flush_dcache_page() for usage from irq context on ARM, NIOS2 and PARISC

[Helge Deller]

ARM, NIOS2 and PARISC unintentionally re-enable IRQs in
flush_dcache_page() when called from irq context.

This series fixes it and should go to stable series to at
least kernel 6.1.

Helge

Helge Deller (3):
  arm: Fix flush_dcache_page() for usage from irq context
  nios2: Fix flush_dcache_page() for usage from irq context
  parisc: Fix flush_dcache_page() for usage from irq context

 arch/arm/include/asm/cacheflush.h    | 4 ++++
 arch/arm/mm/flush.c                  | 5 +++--
 arch/nios2/include/asm/cacheflush.h  | 4 ++++
 arch/nios2/mm/cacheflush.c           | 5 +++--
 arch/parisc/include/asm/cacheflush.h | 4 ++++
 arch/parisc/kernel/cache.c           | 5 +++--
 6 files changed, 21 insertions(+), 6 deletions(-)

--
2.38.1

[PATCH 1/3] arm: Fix flush_dcache_page() for usage from irq context

[Helge Deller]

Since at least kernel 6.1, flush_dcache_page() is called with IRQs
disabled, e.g. from aio_complete().

But the current implementation for flush_dcache_page() on ARM
unintentionally re-enables IRQs, which may lead to deadlocks.

Fix it by using xa_lock_irqsave() and xa_unlock_irqrestore()
for the flush_dcache_mmap_*lock() macros instead.

Cc: Russell King (Oracle) <linux@armlinux.org.uk>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: linux-arm-kernel@lists.infradead.org
Signed-off-by: Helge Deller <deller@gmx.de>
---
 arch/arm/include/asm/cacheflush.h | 4 ++++
 arch/arm/mm/flush.c               | 5 +++--
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/arch/arm/include/asm/cacheflush.h b/arch/arm/include/asm/cacheflush.h
index a094f964c869..5b8a1ef0dc50 100644
--- a/arch/arm/include/asm/cacheflush.h
+++ b/arch/arm/include/asm/cacheflush.h
@@ -315,6 +315,10 @@ static inline void flush_anon_page(struct vm_area_struct *vma,

 #define flush_dcache_mmap_lock(mapping)		xa_lock_irq(&mapping->i_pages)
 #define flush_dcache_mmap_unlock(mapping)	xa_unlock_irq(&mapping->i_pages)
+#define flush_dcache_mmap_lock_irqsave(mapping, flags)		\
+		xa_lock_irqsave(&mapping->i_pages, flags)
+#define flush_dcache_mmap_unlock_irqrestore(mapping, flags)	\
+		xa_unlock_irqrestore(&mapping->i_pages, flags)

 /*
  * We don't appear to need to do anything here.  In fact, if we did, we'd
diff --git a/arch/arm/mm/flush.c b/arch/arm/mm/flush.c
index 7ff9feea13a6..d57ec9165520 100644
--- a/arch/arm/mm/flush.c
+++ b/arch/arm/mm/flush.c
@@ -238,6 +238,7 @@ static void __flush_dcache_aliases(struct address_space *mapping, struct page *p
 {
 	struct mm_struct *mm = current->active_mm;
 	struct vm_area_struct *mpnt;
+	unsigned long flags;
 	pgoff_t pgoff;

 	/*
@@ -248,7 +249,7 @@ static void __flush_dcache_aliases(struct address_space *mapping, struct page *p
 	 */
 	pgoff = page->index;

-	flush_dcache_mmap_lock(mapping);
+	flush_dcache_mmap_lock_irqsave(mapping, flags);
 	vma_interval_tree_foreach(mpnt, &mapping->i_mmap, pgoff, pgoff) {
 		unsigned long offset;

@@ -262,7 +263,7 @@ static void __flush_dcache_aliases(struct address_space *mapping, struct page *p
 		offset = (pgoff - mpnt->vm_pgoff) << PAGE_SHIFT;
 		flush_cache_page(mpnt, mpnt->vm_start + offset, page_to_pfn(page));
 	}
-	flush_dcache_mmap_unlock(mapping);
+	flush_dcache_mmap_unlock_irqrestore(mapping, flags);
 }

 #if __LINUX_ARM_ARCH__ >= 6
--
2.38.1

Re: [PATCH 1/3] arm: Fix flush_dcache_page() for usage from irq context

[Arnd Bergmann]

On Wed, May 24, 2023, at 17:26, Helge Deller wrote:
> Since at least kernel 6.1, flush_dcache_page() is called with IRQs
> disabled, e.g. from aio_complete().
>
> But the current implementation for flush_dcache_page() on ARM
> unintentionally re-enables IRQs, which may lead to deadlocks.
>
> Fix it by using xa_lock_irqsave() and xa_unlock_irqrestore()
> for the flush_dcache_mmap_*lock() macros instead.
>
> Cc: Russell King (Oracle) <linux@armlinux.org.uk>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: linux-arm-kernel@lists.infradead.org
> Signed-off-by: Helge Deller <deller@gmx.de>

Cc: stable@vger.kernel.org
Reviewed-by: Arnd Bergmann <arnd@arndb.de>

From what I can tell, the behavior in aio_complete has been
there for over 10 years, since 21b40200cfe96 ("aio: use
flush_dcache_page()"). Others may have done the same already
back then.

I also see you sent patches for nios2 and parisc, but not
for csky, which appears to need the same thing.

     Arnd

Re: [PATCH 1/3] arm: Fix flush_dcache_page() for usage from irq context

[Helge Deller]

On 5/24/23 22:00, Arnd Bergmann wrote:
> On Wed, May 24, 2023, at 17:26, Helge Deller wrote:
>> Since at least kernel 6.1, flush_dcache_page() is called with IRQs
>> disabled, e.g. from aio_complete().
>>
>> But the current implementation for flush_dcache_page() on ARM
>> unintentionally re-enables IRQs, which may lead to deadlocks.
>>
>> Fix it by using xa_lock_irqsave() and xa_unlock_irqrestore()
>> for the flush_dcache_mmap_*lock() macros instead.
>>
>> Cc: Russell King (Oracle) <linux@armlinux.org.uk>
>> Cc: Arnd Bergmann <arnd@arndb.de>
>> Cc: linux-arm-kernel@lists.infradead.org
>> Signed-off-by: Helge Deller <deller@gmx.de>
>
> Cc: stable@vger.kernel.org
> Reviewed-by: Arnd Bergmann <arnd@arndb.de>

Thanks!
I assume it's picked up in the arm git tree then.

>  From what I can tell, the behavior in aio_complete has been
> there for over 10 years, since 21b40200cfe96 ("aio: use
> flush_dcache_page()").

Oh, then those arches are broken since then.

> Others may have done the same already back then.
>
> I also see you sent patches for nios2 and parisc, but not
> for csky, which appears to need the same thing.

csky doesn't use flush_dcache_mmap_lock() inside it's
flush_dcache_page() function, so I think it's not affected.

Helge

Re: [PATCH 1/3] arm: Fix flush_dcache_page() for usage from irq context

[Helge Deller]

On 5/24/23 22:00, Arnd Bergmann wrote:
> On Wed, May 24, 2023, at 17:26, Helge Deller wrote:
>> Since at least kernel 6.1, flush_dcache_page() is called with IRQs
>> disabled, e.g. from aio_complete().
>>
>> But the current implementation for flush_dcache_page() on ARM
>> unintentionally re-enables IRQs, which may lead to deadlocks.
>>
>> Fix it by using xa_lock_irqsave() and xa_unlock_irqrestore()
>> for the flush_dcache_mmap_*lock() macros instead.
>>
>> Cc: Russell King (Oracle) <linux@armlinux.org.uk>
>> Cc: Arnd Bergmann <arnd@arndb.de>
>> Cc: linux-arm-kernel@lists.infradead.org
>> Signed-off-by: Helge Deller <deller@gmx.de>
>
> Cc: stable@vger.kernel.org
> Reviewed-by: Arnd Bergmann <arnd@arndb.de>
>
>  From what I can tell, the behavior in aio_complete has been
> there for over 10 years, since 21b40200cfe96 ("aio: use
> flush_dcache_page()"). Others may have done the same already
> back then.

gentle ping...
I think this patch hasn't been picked up yet for arm.

Helge

[PATCH 2/3] nios2: Fix flush_dcache_page() for usage from irq context

[Helge Deller]

Since at least kernel 6.1, flush_dcache_page() is called with IRQs
disabled, e.g. from aio_complete().

But the current implementation for flush_dcache_page() on NIOS2
unintentionally re-enables IRQs, which may lead to deadlocks.

Fix it by using xa_lock_irqsave() and xa_unlock_irqrestore()
for the flush_dcache_mmap_*lock() macros instead.

Cc: Dinh Nguyen <dinguyen@kernel.org>
Signed-off-by: Helge Deller <deller@gmx.de>
---
 arch/nios2/include/asm/cacheflush.h | 4 ++++
 arch/nios2/mm/cacheflush.c          | 5 +++--
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/arch/nios2/include/asm/cacheflush.h b/arch/nios2/include/asm/cacheflush.h
index d0b71dd71287..a37242662809 100644
--- a/arch/nios2/include/asm/cacheflush.h
+++ b/arch/nios2/include/asm/cacheflush.h
@@ -48,5 +48,9 @@ extern void invalidate_dcache_range(unsigned long start, unsigned long end);

 #define flush_dcache_mmap_lock(mapping)		xa_lock_irq(&mapping->i_pages)
 #define flush_dcache_mmap_unlock(mapping)	xa_unlock_irq(&mapping->i_pages)
+#define flush_dcache_mmap_lock_irqsave(mapping, flags)		\
+		xa_lock_irqsave(&mapping->i_pages, flags)
+#define flush_dcache_mmap_unlock_irqrestore(mapping, flags)	\
+		xa_unlock_irqrestore(&mapping->i_pages, flags)

 #endif /* _ASM_NIOS2_CACHEFLUSH_H */
diff --git a/arch/nios2/mm/cacheflush.c b/arch/nios2/mm/cacheflush.c
index 6aa9257c3ede..35f3b599187f 100644
--- a/arch/nios2/mm/cacheflush.c
+++ b/arch/nios2/mm/cacheflush.c
@@ -75,11 +75,12 @@ static void flush_aliases(struct address_space *mapping, struct page *page)
 {
 	struct mm_struct *mm = current->active_mm;
 	struct vm_area_struct *mpnt;
+	unsigned long flags;
 	pgoff_t pgoff;

 	pgoff = page->index;

-	flush_dcache_mmap_lock(mapping);
+	flush_dcache_mmap_lock_irqsave(mapping, flags);
 	vma_interval_tree_foreach(mpnt, &mapping->i_mmap, pgoff, pgoff) {
 		unsigned long offset;

@@ -92,7 +93,7 @@ static void flush_aliases(struct address_space *mapping, struct page *page)
 		flush_cache_page(mpnt, mpnt->vm_start + offset,
 			page_to_pfn(page));
 	}
-	flush_dcache_mmap_unlock(mapping);
+	flush_dcache_mmap_unlock_irqrestore(mapping, flags);
 }

 void flush_cache_all(void)
--
2.38.1

Re: [PATCH 2/3] nios2: Fix flush_dcache_page() for usage from irq context

[Dinh Nguyen]

Hi Helge,

Thanks for the patch. Does it need a Fixes tag?

Dinh

On 5/24/23 10:26, Helge Deller wrote:
> Since at least kernel 6.1, flush_dcache_page() is called with IRQs
> disabled, e.g. from aio_complete().
> 
> But the current implementation for flush_dcache_page() on NIOS2
> unintentionally re-enables IRQs, which may lead to deadlocks.
> 
> Fix it by using xa_lock_irqsave() and xa_unlock_irqrestore()
> for the flush_dcache_mmap_*lock() macros instead.
> 
> Cc: Dinh Nguyen <dinguyen@kernel.org>
> Signed-off-by: Helge Deller <deller@gmx.de>
> ---
>   arch/nios2/include/asm/cacheflush.h | 4 ++++
>   arch/nios2/mm/cacheflush.c          | 5 +++--
>   2 files changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/nios2/include/asm/cacheflush.h b/arch/nios2/include/asm/cacheflush.h
> index d0b71dd71287..a37242662809 100644
> --- a/arch/nios2/include/asm/cacheflush.h
> +++ b/arch/nios2/include/asm/cacheflush.h
> @@ -48,5 +48,9 @@ extern void invalidate_dcache_range(unsigned long start, unsigned long end);
> 
>   #define flush_dcache_mmap_lock(mapping)		xa_lock_irq(&mapping->i_pages)
>   #define flush_dcache_mmap_unlock(mapping)	xa_unlock_irq(&mapping->i_pages)
> +#define flush_dcache_mmap_lock_irqsave(mapping, flags)		\
> +		xa_lock_irqsave(&mapping->i_pages, flags)
> +#define flush_dcache_mmap_unlock_irqrestore(mapping, flags)	\
> +		xa_unlock_irqrestore(&mapping->i_pages, flags)
> 
>   #endif /* _ASM_NIOS2_CACHEFLUSH_H */
> diff --git a/arch/nios2/mm/cacheflush.c b/arch/nios2/mm/cacheflush.c
> index 6aa9257c3ede..35f3b599187f 100644
> --- a/arch/nios2/mm/cacheflush.c
> +++ b/arch/nios2/mm/cacheflush.c
> @@ -75,11 +75,12 @@ static void flush_aliases(struct address_space *mapping, struct page *page)
>   {
>   	struct mm_struct *mm = current->active_mm;
>   	struct vm_area_struct *mpnt;
> +	unsigned long flags;
>   	pgoff_t pgoff;
> 
>   	pgoff = page->index;
> 
> -	flush_dcache_mmap_lock(mapping);
> +	flush_dcache_mmap_lock_irqsave(mapping, flags);
>   	vma_interval_tree_foreach(mpnt, &mapping->i_mmap, pgoff, pgoff) {
>   		unsigned long offset;
> 
> @@ -92,7 +93,7 @@ static void flush_aliases(struct address_space *mapping, struct page *page)
>   		flush_cache_page(mpnt, mpnt->vm_start + offset,
>   			page_to_pfn(page));
>   	}
> -	flush_dcache_mmap_unlock(mapping);
> +	flush_dcache_mmap_unlock_irqrestore(mapping, flags);
>   }
> 
>   void flush_cache_all(void)
> --
> 2.38.1
> 

Re: [PATCH 2/3] nios2: Fix flush_dcache_page() for usage from irq context

[Helge Deller]

Hi Dinh,

On 6/14/23 00:39, Dinh Nguyen wrote:
> Thanks for the patch. Does it need a Fixes tag?

I did not add a fixes tag for the parisc or arm version.

The code was originally correct, but later patches then suddenly started
using cache flushes from irq context (e.g. 21b40200cfe96 ("aio: use
flush_dcache_page()")) which then triggers the bug.
So, it's hard to say that it fixes one specific commit.

I suggest you backport it as far as possible.

Helge

> Dinh
>
> On 5/24/23 10:26, Helge Deller wrote:
>> Since at least kernel 6.1, flush_dcache_page() is called with IRQs
>> disabled, e.g. from aio_complete().
>>
>> But the current implementation for flush_dcache_page() on NIOS2
>> unintentionally re-enables IRQs, which may lead to deadlocks.
>>
>> Fix it by using xa_lock_irqsave() and xa_unlock_irqrestore()
>> for the flush_dcache_mmap_*lock() macros instead.
>>
>> Cc: Dinh Nguyen <dinguyen@kernel.org>
>> Signed-off-by: Helge Deller <deller@gmx.de>
>> ---
>>   arch/nios2/include/asm/cacheflush.h | 4 ++++
>>   arch/nios2/mm/cacheflush.c          | 5 +++--
>>   2 files changed, 7 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/nios2/include/asm/cacheflush.h b/arch/nios2/include/asm/cacheflush.h
>> index d0b71dd71287..a37242662809 100644
>> --- a/arch/nios2/include/asm/cacheflush.h
>> +++ b/arch/nios2/include/asm/cacheflush.h
>> @@ -48,5 +48,9 @@ extern void invalidate_dcache_range(unsigned long start, unsigned long end);
>>
>>   #define flush_dcache_mmap_lock(mapping)        xa_lock_irq(&mapping->i_pages)
>>   #define flush_dcache_mmap_unlock(mapping)    xa_unlock_irq(&mapping->i_pages)
>> +#define flush_dcache_mmap_lock_irqsave(mapping, flags)        \
>> +        xa_lock_irqsave(&mapping->i_pages, flags)
>> +#define flush_dcache_mmap_unlock_irqrestore(mapping, flags)    \
>> +        xa_unlock_irqrestore(&mapping->i_pages, flags)
>>
>>   #endif /* _ASM_NIOS2_CACHEFLUSH_H */
>> diff --git a/arch/nios2/mm/cacheflush.c b/arch/nios2/mm/cacheflush.c
>> index 6aa9257c3ede..35f3b599187f 100644
>> --- a/arch/nios2/mm/cacheflush.c
>> +++ b/arch/nios2/mm/cacheflush.c
>> @@ -75,11 +75,12 @@ static void flush_aliases(struct address_space *mapping, struct page *page)
>>   {
>>       struct mm_struct *mm = current->active_mm;
>>       struct vm_area_struct *mpnt;
>> +    unsigned long flags;
>>       pgoff_t pgoff;
>>
>>       pgoff = page->index;
>>
>> -    flush_dcache_mmap_lock(mapping);
>> +    flush_dcache_mmap_lock_irqsave(mapping, flags);
>>       vma_interval_tree_foreach(mpnt, &mapping->i_mmap, pgoff, pgoff) {
>>           unsigned long offset;
>>
>> @@ -92,7 +93,7 @@ static void flush_aliases(struct address_space *mapping, struct page *page)
>>           flush_cache_page(mpnt, mpnt->vm_start + offset,
>>               page_to_pfn(page));
>>       }
>> -    flush_dcache_mmap_unlock(mapping);
>> +    flush_dcache_mmap_unlock_irqrestore(mapping, flags);
>>   }
>>
>>   void flush_cache_all(void)
>> --
>> 2.38.1
>>

[PATCH 3/3] parisc: Fix flush_dcache_page() for usage from irq context

[Helge Deller]

Since at least kernel 6.1, flush_dcache_page() is called with IRQs
disabled, e.g. from aio_complete().

But the current implementation for flush_dcache_page() on parisc
unintentionally re-enables IRQs, which may lead to deadlocks.

Fix it by using xa_lock_irqsave() and xa_unlock_irqrestore()
for the flush_dcache_mmap_*lock() macros instead.

Cc: linux-parisc@vger.kernel.org
Signed-off-by: Helge Deller <deller@gmx.de>
---
 arch/parisc/include/asm/cacheflush.h | 4 ++++
 arch/parisc/kernel/cache.c           | 5 +++--
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/arch/parisc/include/asm/cacheflush.h b/arch/parisc/include/asm/cacheflush.h
index 0bdee6724132..c8b6928cee1e 100644
--- a/arch/parisc/include/asm/cacheflush.h
+++ b/arch/parisc/include/asm/cacheflush.h
@@ -48,6 +48,10 @@ void flush_dcache_page(struct page *page);

 #define flush_dcache_mmap_lock(mapping)		xa_lock_irq(&mapping->i_pages)
 #define flush_dcache_mmap_unlock(mapping)	xa_unlock_irq(&mapping->i_pages)
+#define flush_dcache_mmap_lock_irqsave(mapping, flags)		\
+		xa_lock_irqsave(&mapping->i_pages, flags)
+#define flush_dcache_mmap_unlock_irqrestore(mapping, flags)	\
+		xa_unlock_irqrestore(&mapping->i_pages, flags)

 #define flush_icache_page(vma,page)	do { 		\
 	flush_kernel_dcache_page_addr(page_address(page)); \
diff --git a/arch/parisc/kernel/cache.c b/arch/parisc/kernel/cache.c
index 1d3b8bc8a623..ca4a302d4365 100644
--- a/arch/parisc/kernel/cache.c
+++ b/arch/parisc/kernel/cache.c
@@ -399,6 +399,7 @@ void flush_dcache_page(struct page *page)
 	unsigned long offset;
 	unsigned long addr, old_addr = 0;
 	unsigned long count = 0;
+	unsigned long flags;
 	pgoff_t pgoff;

 	if (mapping && !mapping_mapped(mapping)) {
@@ -420,7 +421,7 @@ void flush_dcache_page(struct page *page)
 	 * to flush one address here for them all to become coherent
 	 * on machines that support equivalent aliasing
 	 */
-	flush_dcache_mmap_lock(mapping);
+	flush_dcache_mmap_lock_irqsave(mapping, flags);
 	vma_interval_tree_foreach(mpnt, &mapping->i_mmap, pgoff, pgoff) {
 		offset = (pgoff - mpnt->vm_pgoff) << PAGE_SHIFT;
 		addr = mpnt->vm_start + offset;
@@ -460,7 +461,7 @@ void flush_dcache_page(struct page *page)
 		}
 		WARN_ON(++count == 4096);
 	}
-	flush_dcache_mmap_unlock(mapping);
+	flush_dcache_mmap_unlock_irqrestore(mapping, flags);
 }
 EXPORT_SYMBOL(flush_dcache_page);

--
2.38.1