[PATCH] iommu/vt-d: Do not dump pasid table entries in kdump kernel

[PATCH] iommu/vt-d: Do not dump pasid table entries in kdump kernel

[Zelin Deng]

In kdump kernel PASID translations won't be copied from previous kernel
even if scalable-mode is enabled, so pages of PASID translations are
non-present in kdump kernel. Attempt to access those address will cause
PF fault:

[   13.396476] DMAR: DRHD: handling fault status reg 3
[   13.396478] DMAR: [DMA Read NO_PASID] Request device [81:00.0] fault addr 0xffffd000 [fault reason 0x59] SM: Present bit in PA$
[   13.396480] DMAR: Dump dmar5 table entries for IOVA 0xffffd000
[   13.396481] DMAR: scalable mode root entry: hi 0x0000000000000000, low 0x00000000460d1001
[   13.396482] DMAR: context entry: hi 0x0000000000000008, low 0x00000010c4237401
[   13.396485] BUG: unable to handle page fault for address: ff110010c4237000
[   13.396486] #PF: supervisor read access in kernel mode
[   13.396487] #PF: error_code(0x0000) - not-present page
[   13.396488] PGD 5d201067 P4D 5d202067 PUD 0
[   13.396490] Oops: 0000 [#1] PREEMPT SMP NOPTI
[   13.396491] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.16.0-rc6-next-20211224+ #6
[   13.396493] Hardware name: Intel Corporation EAGLESTREAM/EAGLESTREAM, BIOS EGSDCRB1.86B.0067.D12.2110190950 10/19/2021
[   13.396494] RIP: 0010:dmar_fault_dump_ptes+0x13b/0x295

Hence skip dumping pasid table entries if in kdump kernel.

Fixes: 914ff7719e8a (“iommu/vt-d: Dump DMAR translation structure when DMA fault occurs”)
Signed-off-by: Zelin Deng <zelin.deng@linux.alibaba.com>
---
 drivers/iommu/intel/iommu.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 92fea3fb..f0134cf 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -1074,6 +1074,12 @@ void dmar_fault_dump_ptes(struct intel_iommu *iommu, u16 source_id,
 	if (!sm_supported(iommu))
 		goto pgtable_walk;
 
+	/* PASID translations is not copied, skip dumping pasid table entries
+	 * otherwise non-present page will be accessed.
+	 */
+	if (is_kdump_kernel())
+		goto pgtable_walk;
+
 	/* get the pointer to pasid directory entry */
 	dir = phys_to_virt(ctx_entry->lo & VTD_PAGE_MASK);
 	if (!dir) {
-- 
1.8.3.1

Re: [PATCH] iommu/vt-d: Do not dump pasid table entries in kdump kernel

[Lu Baolu]

On 1/19/22 5:07 PM, Zelin Deng wrote:
> In kdump kernel PASID translations won't be copied from previous kernel
> even if scalable-mode is enabled, so pages of PASID translations are

Yes. The copy table support for scalable mode is still in my task list.

> non-present in kdump kernel. Attempt to access those address will cause
> PF fault:
> 
> [   13.396476] DMAR: DRHD: handling fault status reg 3
> [   13.396478] DMAR: [DMA Read NO_PASID] Request device [81:00.0] fault addr 0xffffd000 [fault reason 0x59] SM: Present bit in PA$
> [   13.396480] DMAR: Dump dmar5 table entries for IOVA 0xffffd000
> [   13.396481] DMAR: scalable mode root entry: hi 0x0000000000000000, low 0x00000000460d1001
> [   13.396482] DMAR: context entry: hi 0x0000000000000008, low 0x00000010c4237401
> [   13.396485] BUG: unable to handle page fault for address: ff110010c4237000
> [   13.396486] #PF: supervisor read access in kernel mode
> [   13.396487] #PF: error_code(0x0000) - not-present page
> [   13.396488] PGD 5d201067 P4D 5d202067 PUD 0
> [   13.396490] Oops: 0000 [#1] PREEMPT SMP NOPTI
> [   13.396491] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.16.0-rc6-next-20211224+ #6
> [   13.396493] Hardware name: Intel Corporation EAGLESTREAM/EAGLESTREAM, BIOS EGSDCRB1.86B.0067.D12.2110190950 10/19/2021
> [   13.396494] RIP: 0010:dmar_fault_dump_ptes+0x13b/0x295
> 
> Hence skip dumping pasid table entries if in kdump kernel.

This just asks dmar_fault_dump_ptes() to keep silent. The problem is
that the context entry is mis-configured. Perhaps we should disable
copy table for scalable mode for now. How about below change?

--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -3337,10 +3337,11 @@ static int __init init_dmars(void)

                 init_translation_status(iommu);

-               if (translation_pre_enabled(iommu) && !is_kdump_kernel()) {
+               if (translation_pre_enabled(iommu) &&
+                   (!is_kdump_kernel() || sm_supported(iommu))) {
                         iommu_disable_translation(iommu);
                         clear_translation_pre_enabled(iommu);
-                       pr_warn("Translation was enabled for %s but we 
are not in kdump mode\n",
+                       pr_warn("Translation was enabled for %s but we 
are not in kdump mode or copy table not supported\n",
                                 iommu->name);
                 }

> 
> Fixes: 914ff7719e8a (“iommu/vt-d: Dump DMAR translation structure when DMA fault occurs”)
> Signed-off-by: Zelin Deng <zelin.deng@linux.alibaba.com>
> ---
>   drivers/iommu/intel/iommu.c | 6 ++++++
>   1 file changed, 6 insertions(+)
> 
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index 92fea3fb..f0134cf 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -1074,6 +1074,12 @@ void dmar_fault_dump_ptes(struct intel_iommu *iommu, u16 source_id,
>   	if (!sm_supported(iommu))
>   		goto pgtable_walk;
>   
> +	/* PASID translations is not copied, skip dumping pasid table entries
> +	 * otherwise non-present page will be accessed.
> +	 */
> +	if (is_kdump_kernel())
> +		goto pgtable_walk;
> +
>   	/* get the pointer to pasid directory entry */
>   	dir = phys_to_virt(ctx_entry->lo & VTD_PAGE_MASK);
>   	if (!dir) {
> 

Best regards,
baolu

Re: [PATCH] iommu/vt-d: Do not dump pasid table entries in kdump kernel

[zelin deng]

在 2022/1/20 上午10:58, Lu Baolu 写道:
> On 1/19/22 5:07 PM, Zelin Deng wrote:
>> In kdump kernel PASID translations won't be copied from previous kernel
>> even if scalable-mode is enabled, so pages of PASID translations are
>
> Yes. The copy table support for scalable mode is still in my task list.
>
>> non-present in kdump kernel. Attempt to access those address will cause
>> PF fault:
>>
>> [   13.396476] DMAR: DRHD: handling fault status reg 3
>> [   13.396478] DMAR: [DMA Read NO_PASID] Request device [81:00.0] 
>> fault addr 0xffffd000 [fault reason 0x59] SM: Present bit in PA$
>> [   13.396480] DMAR: Dump dmar5 table entries for IOVA 0xffffd000
>> [   13.396481] DMAR: scalable mode root entry: hi 0x0000000000000000, 
>> low 0x00000000460d1001
>> [   13.396482] DMAR: context entry: hi 0x0000000000000008, low 
>> 0x00000010c4237401
>> [   13.396485] BUG: unable to handle page fault for address: 
>> ff110010c4237000
>> [   13.396486] #PF: supervisor read access in kernel mode
>> [   13.396487] #PF: error_code(0x0000) - not-present page
>> [   13.396488] PGD 5d201067 P4D 5d202067 PUD 0
>> [   13.396490] Oops: 0000 [#1] PREEMPT SMP NOPTI
>> [   13.396491] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 
>> 5.16.0-rc6-next-20211224+ #6
>> [   13.396493] Hardware name: Intel Corporation 
>> EAGLESTREAM/EAGLESTREAM, BIOS EGSDCRB1.86B.0067.D12.2110190950 
>> 10/19/2021
>> [   13.396494] RIP: 0010:dmar_fault_dump_ptes+0x13b/0x295
>>
>> Hence skip dumping pasid table entries if in kdump kernel.
>
> This just asks dmar_fault_dump_ptes() to keep silent. The problem is
> that the context entry is mis-configured. Perhaps we should disable
> copy table for scalable mode for now. How about below change?

Yep.  The change looks good to me.

Actually I had encountered another issue which had blocked virtio-net 
device when scalable mode is enabled in kdump kernel so that I had made 
the same change as yours -- 'to disable translation if sm_on in kdump 
kernel' in our internal tree.

I only observe this issue on our dragonfly baremetal server with 
virtio-net device inside, I did't send the fix to upstream as I am not 
sure if it is reasonable to disable translation in kdump kernel.


>
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -3337,10 +3337,11 @@ static int __init init_dmars(void)
>
>                 init_translation_status(iommu);
>
> -               if (translation_pre_enabled(iommu) && 
> !is_kdump_kernel()) {
> +               if (translation_pre_enabled(iommu) &&
> +                   (!is_kdump_kernel() || sm_supported(iommu))) {
>                         iommu_disable_translation(iommu);
>                         clear_translation_pre_enabled(iommu);
> -                       pr_warn("Translation was enabled for %s but we 
> are not in kdump mode\n",
> +                       pr_warn("Translation was enabled for %s but we 
> are not in kdump mode or copy table not supported\n",
>                                 iommu->name);
>                 }
>
>>
>> Fixes: 914ff7719e8a (“iommu/vt-d: Dump DMAR translation structure 
>> when DMA fault occurs”)
>> Signed-off-by: Zelin Deng <zelin.deng@linux.alibaba.com>
>> ---
>>   drivers/iommu/intel/iommu.c | 6 ++++++
>>   1 file changed, 6 insertions(+)
>>
>> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
>> index 92fea3fb..f0134cf 100644
>> --- a/drivers/iommu/intel/iommu.c
>> +++ b/drivers/iommu/intel/iommu.c
>> @@ -1074,6 +1074,12 @@ void dmar_fault_dump_ptes(struct intel_iommu 
>> *iommu, u16 source_id,
>>       if (!sm_supported(iommu))
>>           goto pgtable_walk;
>>   +    /* PASID translations is not copied, skip dumping pasid table 
>> entries
>> +     * otherwise non-present page will be accessed.
>> +     */
>> +    if (is_kdump_kernel())
>> +        goto pgtable_walk;
>> +
>>       /* get the pointer to pasid directory entry */
>>       dir = phys_to_virt(ctx_entry->lo & VTD_PAGE_MASK);
>>       if (!dir) {
>>
>
> Best regards,
> baolu