From: fangyu.yu@linux.alibaba.com
To: gsh517025@gmail.com
Cc: alex@ghiti.fr, andrew.jones@linux.dev, anup@brainfault.org,
aou@eecs.berkeley.edu, atish.patra@linux.dev,
baolu.lu@linux.intel.com, fangyu.yu@linux.alibaba.com,
gong.shuai@sanechips.com.cn, guoren@kernel.org,
iommu@lists.linux.dev, jgg@nvidia.com, jgg@ziepe.ca,
joerg.roedel@amd.com, joro@8bytes.org, jroedel@suse.de,
kevin.tian@intel.com, kvm-riscv@lists.infradead.org,
linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org,
nutty.liu@hotmail.com, palmer@dabbelt.com, pjw@kernel.org,
robin.murphy@arm.com, skhawaja@google.com,
tanchi25@mails.ucas.ac.cn, tomasz.jeznach@linux.dev,
vasant.hegde@amd.com, will@kernel.org, zong.li@sifive.com
Subject: Re: [PATCH v7 05/10] iommu/riscv: support GSCID and GVMA invalidation command
Date: Sat, 21 Nov 2026 14:15:32 +0800 [thread overview]
Message-ID: <20261121061532.23534-1-fangyu.yu@linux.alibaba.com> (raw)
In-Reply-To: <23b033cb-1b98-48cf-a089-4e710dfe3e15@gmail.com>
>>>Hi Fangyu,
>>>
>>>> From: Zong Li <zong.li@sifive.com>
>>>>
>>>> This patch adds a ID Allocator for GSCID and a wrap for setting up
>>>> GSCID in IOTLB invalidation command.
>>>>
>>>> Set up iohgatp to enable second stage table and flush stage-2 table if
>>>> the GSCID is set.
>>>>
>>>> The GSCID of domain should be freed when release domain. GSCID will be
>>>> allocated for parent domain in nested IOMMU process.
>>>>
>>>> Signed-off-by: Zong Li <zong.li@sifive.com>
>>>> Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
>>>> Reviewed-by: Nutty Liu <nutty.liu@hotmail.com>
>>>> ---
>>>> drivers/iommu/riscv/iommu-bits.h | 7 ++++
>>>> drivers/iommu/riscv/iommu.c | 62 ++++++++++++++++++++++++++------
>>>> 2 files changed, 58 insertions(+), 11 deletions(-)
>>>>
>>>> diff --git a/drivers/iommu/riscv/iommu-bits.h b/drivers/iommu/riscv/iommu-bits.h
>>>> index f2ef9bd3cde9..65d1f2250011 100644
>>>> --- a/drivers/iommu/riscv/iommu-bits.h
>>>> +++ b/drivers/iommu/riscv/iommu-bits.h
>>>> @@ -720,6 +720,13 @@ static inline void riscv_iommu_cmd_inval_vma(struct riscv_iommu_command *cmd)
>>>> cmd->dword1 = 0;
>>>> }
>>>>
>>>> +static inline void riscv_iommu_cmd_inval_gvma(struct riscv_iommu_command *cmd)
>>>> +{
>>>> + cmd->dword0 = FIELD_PREP(RISCV_IOMMU_CMD0_OPCODE, RISCV_IOMMU_CMD_IOTINVAL_OPCODE) |
>>>> + FIELD_PREP(RISCV_IOMMU_CMD0_FUNC, RISCV_IOMMU_CMD_IOTINVAL_FUNC_GVMA);
>>>> + cmd->dword1 = 0;
>>>> +}
>>>> +
>>>> static inline void riscv_iommu_cmd_inval_set_addr(struct riscv_iommu_command *cmd,
>>>> u64 addr)
>>>> {
>>>> diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c
>>>> index 918fce4aa8c4..5546dfcd976c 100644
>>>> --- a/drivers/iommu/riscv/iommu.c
>>>> +++ b/drivers/iommu/riscv/iommu.c
>>>> @@ -49,6 +49,10 @@
>>>> static DEFINE_IDA(riscv_iommu_pscids);
>>>> #define RISCV_IOMMU_MAX_PSCID (BIT(20) - 1)
>>>>
>>>> +/* IOMMU GSCID allocation namespace. */
>>>> +static DEFINE_IDA(riscv_iommu_gscids);
>>>> +#define RISCV_IOMMU_MAX_GSCID (BIT(16) - 1)
>>>> +
>>>> /* Device resource-managed allocations */
>>>> struct riscv_iommu_devres {
>>>> void *addr;
>>>> @@ -842,6 +846,7 @@ struct riscv_iommu_domain {
>>>> struct list_head bonds;
>>>> spinlock_t lock; /* protect bonds list updates. */
>>>> int pscid;
>>>> + int gscid;
>>>> };
>>>> PT_IOMMU_CHECK_DOMAIN(struct riscv_iommu_domain, riscvpt.iommu, domain);
>>>>
>>>> @@ -1004,7 +1009,7 @@ static void riscv_iommu_tlbi_calc(struct riscv_iommu_tlbi *tlbi,
>>>> }
>>>>
>>>> static void riscv_iommu_iotlb_inval_iommu(struct riscv_iommu_device *iommu,
>>>> - int pscid,
>>>> + int pscid, int gscid,
>>>> struct riscv_iommu_tlbi *tlbi)
>>>> {
>>>> bool use_nl = tlbi->non_leaf &&
>>>> @@ -1012,8 +1017,13 @@ static void riscv_iommu_iotlb_inval_iommu(struct riscv_iommu_device *iommu,
>>>> struct riscv_iommu_command cmd;
>>>> unsigned int i;
>>>>
>>>> - riscv_iommu_cmd_inval_vma(&cmd);
>>>> - riscv_iommu_cmd_inval_set_pscid(&cmd, pscid);
>>>> + if (gscid) {
>>>> + riscv_iommu_cmd_inval_gvma(&cmd);
>>>> + riscv_iommu_cmd_inval_set_gscid(&cmd, gscid);
>>>> + } else {
>>>> + riscv_iommu_cmd_inval_vma(&cmd);
>>>> + riscv_iommu_cmd_inval_set_pscid(&cmd, pscid);
>>>> + }
>>>
>>>As I understand it, the same choice is needed in
>>>riscv_iommu_bond_unlink().
>>>
>>>The missing GVMA command does not break anything today, because every
>>>caller of bond_unlink() runs iodir_update() just before it, and that
>>>already sends the GVMA for the old GSCID. But depending on that
>>>ordering seems fragile, and the comment in bond_unlink(), "invalidate
>>>all cached entries for domain's PSCID", is no longer true now that we
>>>have S2 domains.
>>>
>>
>> There is no functional issue with the current code, since
>> iodir_update() already performs the required invalidation.
>>
>> That said, as this series adds S2 domain support,
>> bond_unlink() should indeed be updated accordingly.
>> Good catch.
>
>Hi Fangyu,
>
>I've included this code in patch 10 of the v3 nested series [1].
>
Hi Shuai,
Thanks for the pointer. If possible, I'd suggest keeping this
change in the current patch rather than splitting it out --
it makes the functionality look more complete, and splitting
it into two patches seems a bit odd.
>BTW, if you have time, would you mind reviewing the nested series?
>I'd greatly appreciate your help.
>
Regarding the nested series, it looks like a solid set of
patches that already provides the basic nested IOMMU
functionality. I'd be glad to help with the review and hope
I can be of some help.
Thanks,
Fangyu
>[1] https://lore.kernel.org/linux-riscv/20261119132744.396426-11-gong.shuai@sanechips.com.cn/
>
>Thanks,
>Shaui
>
>>
>> Thanks,
>> Fangyu
>>
>>>The change, if you think it is needed, fits better in the last patch,
>>>"iommu/riscv: Add IOTINVAL.GVMA invalidations". I am replying here
>>>only because the code is very similar.
>>>
>>>Thanks,
>>>Shuai
>>>
next prev parent reply other threads:[~2026-11-21 6:15 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-03 12:02 [PATCH v7 00/10] iommu/riscv: Add hardware dirty tracking for second-stage domains fangyu.yu
2026-11-03 12:02 ` [PATCH v7 01/10] iommu/generic_pt: Fix the RISC-V supported feature mask fangyu.yu
2026-11-03 12:02 ` [PATCH v7 02/10] iommupt: Add RISC-V Second-stage (iohgatp) page table support fangyu.yu
2026-11-03 12:02 ` [PATCH v7 03/10] iommupt: Add RISC-V dirty tracking PTE ops fangyu.yu
2026-11-03 12:02 ` [PATCH v7 04/10] iommu/riscv: use data structure instead of individual values fangyu.yu
2026-11-03 12:02 ` [PATCH v7 05/10] iommu/riscv: support GSCID and GVMA invalidation command fangyu.yu
2026-11-04 9:21 ` Gong Shuai
2026-11-04 11:27 ` fangyu.yu
2026-11-19 18:02 ` Gong Shuai
2026-11-21 6:15 ` fangyu.yu [this message]
2026-11-03 12:02 ` [PATCH v7 06/10] RISC-V: KVM: Enable KVM_VFIO interfaces on RISC-V arch fangyu.yu
2026-11-04 6:45 ` Gong Shuai
2026-11-04 10:59 ` fangyu.yu
2026-11-03 12:02 ` [PATCH v7 07/10] iommu/riscv: Add domain_alloc_paging_flags for second-stage domain fangyu.yu
2026-11-03 12:02 ` [PATCH v7 08/10] iommu/riscv: Pre-enable GADE for second-stage domains fangyu.yu
2026-11-03 12:02 ` [PATCH v7 09/10] iommu/riscv: Add dirty tracking support " fangyu.yu
2026-11-04 8:45 ` Gong Shuai
2026-11-04 11:50 ` fangyu.yu
2026-11-03 12:02 ` [PATCH v7 10/10] iommu/riscv: Add IOTINVAL.GVMA invalidations fangyu.yu
2026-11-11 15:02 ` [PATCH v7 00/10] iommu/riscv: Add hardware dirty tracking for second-stage domains Tan Chi
2026-11-12 2:14 ` fangyu.yu
2026-11-18 6:18 ` Gong Shuai
2026-11-19 13:52 ` fangyu.yu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261121061532.23534-1-fangyu.yu@linux.alibaba.com \
--to=fangyu.yu@linux.alibaba.com \
--cc=alex@ghiti.fr \
--cc=andrew.jones@linux.dev \
--cc=anup@brainfault.org \
--cc=aou@eecs.berkeley.edu \
--cc=atish.patra@linux.dev \
--cc=baolu.lu@linux.intel.com \
--cc=gong.shuai@sanechips.com.cn \
--cc=gsh517025@gmail.com \
--cc=guoren@kernel.org \
--cc=iommu@lists.linux.dev \
--cc=jgg@nvidia.com \
--cc=jgg@ziepe.ca \
--cc=joerg.roedel@amd.com \
--cc=joro@8bytes.org \
--cc=jroedel@suse.de \
--cc=kevin.tian@intel.com \
--cc=kvm-riscv@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-riscv@lists.infradead.org \
--cc=nutty.liu@hotmail.com \
--cc=palmer@dabbelt.com \
--cc=pjw@kernel.org \
--cc=robin.murphy@arm.com \
--cc=skhawaja@google.com \
--cc=tanchi25@mails.ucas.ac.cn \
--cc=tomasz.jeznach@linux.dev \
--cc=vasant.hegde@amd.com \
--cc=will@kernel.org \
--cc=zong.li@sifive.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®