From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AAD0EE49A5 for ; Tue, 22 Aug 2023 05:52:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232906AbjHVFwd (ORCPT ); Tue, 22 Aug 2023 01:52:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232898AbjHVFwb (ORCPT ); Tue, 22 Aug 2023 01:52:31 -0400 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ADF5F1A5 for ; Mon, 21 Aug 2023 22:52:05 -0700 (PDT) Received: by mail-pl1-x642.google.com with SMTP id d9443c01a7336-1bf1935f6c2so27509885ad.1 for ; Mon, 21 Aug 2023 22:52:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1692683525; x=1693288325; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=pX6r3WX3NvSr18ZoSAOxDmvOMvYtId2NY9DB6/FvmJw=; b=A5/+3Q3Hyn/lH9dYFfXb928gLVZSQE6OlxNPDiZQhprTlxgVutY1N2jFlGCAOzK3IQ AOFrdh1kgz4mDfmBe5QcNK4x2A0zw2bs3z7yQTSMnxpajPU2DNxu9yaaqhnaeU/Yub22 qcqwOOQ41zMiHtW21Vy/cxX8jojKejbQoxNNS/8oUfpGpqy33SS0C21imrUR4girrMy5 dcv03Rns271ypsH8Fb1aoEjLZCsY3Z2Xt1JTeOMl9m6QhDX/5pCrsImxxwBuxco4UX7U PemA/SoCWe/uhjsGYpi2g/ncwn91JOIccizpXz3WuSNr8KzN+8eN7A5HWS7DvS5HwE1e FOqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692683525; x=1693288325; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=pX6r3WX3NvSr18ZoSAOxDmvOMvYtId2NY9DB6/FvmJw=; b=F+7oj1i+xXjEUssiztZsxTiRTfJAeR6Fp4zxesmPyaCBNrrVlQ2xdEfdywKdzGJ7y/ QY32jjcmwmGjAa6z0xu8xwU6zITlpqLrVAGHNXp4U28nRyKUjtcoRIjVJcGG4f13JLrL RRWPQh63IV3TK0iIGrXJNc12q6Q0RXxM4HZAGGQtDh4iX5kqlAxOI1rkv+NpjhBiiRSs dIq5DahvVmxrAQ6dn5WcJIDda0KsfQfV+LvlX1cVxe8NZJBZTUikaRiRP3wUwkv6XIGK SpPqobSK5p0EJacW0Ofi/4EbWC8iyMEUvm0gqI1cCfMVKRI5rh6hT89rUVxPNVmXH3HL zAWQ== X-Gm-Message-State: AOJu0Yw5hhXeGQKDNVU98caidXllEUSoClmfVKl/i+3VgSkKtRWdMe5T J/cdYX6lI04RY5H48hwZyEG1ww== X-Google-Smtp-Source: AGHT+IHoStOkuAAck1PdJ7uJtA2xFxVwA0h3uFvR/EarNEv49cmAKimiDq0JXFxErYjSOQNtqhqqFA== X-Received: by 2002:a17:903:2304:b0:1b8:8682:62fb with SMTP id d4-20020a170903230400b001b8868262fbmr12007273plh.4.1692683525104; Mon, 21 Aug 2023 22:52:05 -0700 (PDT) Received: from [10.3.220.88] ([61.213.176.9]) by smtp.gmail.com with ESMTPSA id q6-20020a170902a3c600b001bbd8cf6b57sm8210654plb.230.2023.08.21.22.52.02 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 21 Aug 2023 22:52:04 -0700 (PDT) Message-ID: Date: Tue, 22 Aug 2023 11:51:30 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC] KVM: arm/arm64: optimize vSGI injection performance To: Marc Zyngier Cc: pbonzini@redhat.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, zhouyibo@bytedance.com, zhouliang.001@bytedance.com, Oliver Upton , kvmarm@lists.linux.dev, Mark Rutland References: <20230818104704.7651-1-zhaoxu.35@bytedance.com> <86msykg0ox.wl-maz@kernel.org> From: zhaoxu In-Reply-To: <86msykg0ox.wl-maz@kernel.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org hi marc, thanks for reviewing. On 2023/8/21 18:16, Marc Zyngier wrote: >>> This work is based on v5.4, and here is test data: > > This is a 4 year old kernel. I'm afraid you'll have to provide > something that is relevant to a current (e.i. v6.5) kernel. > In fact, the core vCPU search algorithm remains the same in the latest kernel: iterate all vCPUs, if mpidr matches, inject. next version will based on latest kernel. >>> Based on the test results, the performance of vm with less than 16 cores remains almost the same, >>> while significant improvement can be observed with more than 16 >>> cores. > > This triggers multiple questions: > > - what is the test being used? on what hardware? how can I reproduce > this data? > 1. I utilized the ipi_benchmark (https://patchwork.kernel.org/project/linux-arm-kernel/patch/20171211141600.24401-1-ynorov@caviumnetworks.com/) with a modification to the Normal IPI target in the following manner: smp_call_function_single(31, handle_ipi, &time, 1). 2. On kunpeng 920 platform. 3. Using ipi_benchmark but change the target cpu in Normal IPI case, and use bcc or bpftrace to measuret the execution time of vgic_v3_dispatch_sgi. > - which current guest OS *currently* make use of broadcast or 1:N > SGIs? Linux doesn't and overall SGI multicasting is pretty useless > to an OS. > > [...] Yes, arm64 linux almost never send broadcast ipi. I will use another test data to prove performence improvement > >>> /* >>> - * Compare a given affinity (level 1-3 and a level 0 mask, from the SGI >>> - * generation register ICC_SGI1R_EL1) with a given VCPU. >>> - * If the VCPU's MPIDR matches, return the level0 affinity, otherwise >>> - * return -1. >>> + * Get affinity routing index from ICC_SGI_* register >>> + * format: >>> + * aff3 aff2 aff1 aff0 >>> + * |- 8 bits -|- 8 bits -|- 8 bits -|- 4 bits or 8bits -| > > OK, so you are implementing RSS support: > > - Why isn't that mentioned anywhere in the commit log? > > - Given that KVM actively limits the MPIDR to 4 bits at Aff0, how does > it even work the first place? > > - How is that advertised to the guest? > > - How can the guest enable RSS support? > thanks to mention that, I also checked the relevant code, guest can't enable RSS, it was my oversight. This part has removed in next version. > This is not following the GICv3 architecture, and I'm sceptical that > it actually works as is (I strongly suspect that you have additional > patches...). there are nothing left, all the patchs are here > > M. > With regards Xu