Re: [PATCH v2 0/2] iommu/arm-smmu-v3: Add support for ECMDQ register mode

[Nicolin Chen <nicolinc@nvidia.com>]

On Wed, Aug 09, 2023 at 07:18:36PM -0700, Leizhen (ThunderTown) wrote:
> On 2023/8/9 21:56, Will Deacon wrote:
> > On Wed, Aug 09, 2023 at 09:13:01PM +0800, thunder.leizhen@huaweicloud.com wrote:
> >> From: Zhen Lei <thunder.leizhen@huawei.com>
> >>
> >> v1 --> v2:
> >
> > Jason previously asked about performance numbers for ECMDQ:
> >
> > https://lore.kernel.org/r/ZL6n3f01yV7tc4yH@ziepe.ca
> >
> > Do you have any?
> 
> I asked my colleagues in the chip department, and they said that the chip
> was not commercially available and the specific data could not be disclosed.
> However, to be sure, the performance has improved, but not by much, the
> public benchmark is only about 5%. Your optimization patch was so perfect
> that it ruined our jobs.
> 
> However, since Marvell also implements ECMDQ, there are at least two users.
> Do we think about making it available first?

I have seen something similar (~5%) with VCMDQ on NVIDIA Grace,
when running, in host OS, TLB flush benchmark tests concurrently
on different CPUs.

Although VCMDQ could be slightly different from ECMDQ, both have
a multi-queue feature. And the amount of improvement in my case
came from a reduction of congestion at issueing commands to the
multi queues vs. a single queue. And I guess ECMDQ might benefit
its 5% from that too.

If we decide to move ECMDQ forward, perhaps we can converge some
of the functions to support both :)

Thanks
Nicolin