From: Dexuan Cui <decui@microsoft.com>
To: Vitaly Kuznetsov <vkuznets@redhat.com>,
KY Srinivasan <kys@microsoft.com>
Cc: Haiyang Zhang <haiyangz@microsoft.com>,
"devel@linuxdriverproject.org" <devel@linuxdriverproject.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH 5/6] Drivers: hv: vmbus: distribute subchannels among all vcpus
Date: Fri, 24 Apr 2015 08:40:40 +0000 [thread overview]
Message-ID: <091d0fd6321f4dd490e61a574d5b5b50@SIXPR30MB031.064d.mgd.msft.net> (raw)
In-Reply-To: <1429626460-7947-6-git-send-email-vkuznets@redhat.com>
> -----Original Message-----
> From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com]
> Sent: Tuesday, April 21, 2015 22:28
> To: KY Srinivasan
> Cc: Haiyang Zhang; devel@linuxdriverproject.org; linux-
> kernel@vger.kernel.org; Dexuan Cui
> Subject: [PATCH 5/6] Drivers: hv: vmbus: distribute subchannels among all
> vcpus
>
> Primary channels are distributed evenly across all vcpus we have. When the
> host asks us to create subchannels it usually makes us num_cpus-1 offers
Hi Vitaly,
AFAIK, in the VSP of storvsc, the number of subchannel is
(the_number_of_vcpus - 1) / 4.
This means for a 8-vCPU guest, there is only 1 subchannel.
Your new algorithm tends to make the vCPUs with small-number busier:
e.g., in the 8-vCPU case, assuming we have 4 SCSI controllers:
vCPU0: scsi0's PrimaryChannel (P)
vCPU1: scsi0's SubChannel (S) + scsi1's P
vCPU2: scsi1's S + scsi2's P
vCPU3: scsi2's S + scsi3's P
vCPU4: scsi3's S
vCPU5, 6 and 7 are idle.
In this special case, the existing algorithm is better. :-)
However, I do like this idea in your patch, that is, making sure a device's
primary/sub channels are assigned to differents vCPUs.
I'm just wondering if we should use an even better (and complex) algorithm :-)
PS, yeah, for netvsc(HV_NIC_GUID), the number of SC is indeed
the_number_vcpus -1. I'm not sure about the upcoming HV_ND_GUID --
maybe it's the same as HV_NIC_GUID.
Thanks,
-- Dexuan
> and we are supposed to distribute the work evenly among the channel
> itself and all its subchannels. Make sure they are all assigned to
> different vcpus.
>
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
> drivers/hv/channel_mgmt.c | 29 ++++++++++++++++++++++++++++-
> 1 file changed, 28 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
> index 8f2761f..daa6417 100644
> --- a/drivers/hv/channel_mgmt.c
> +++ b/drivers/hv/channel_mgmt.c
> @@ -270,6 +270,8 @@ static void init_vp_index(struct vmbus_channel
> *channel,
> int i;
> bool perf_chn = false;
> u32 max_cpus = num_online_cpus();
> + struct vmbus_channel *primary = channel->primary_channel, *prev;
> + unsigned long flags;
>
> for (i = IDE; i < MAX_PERF_CHN; i++) {
> if (!memcmp(type_guid->b, hp_devs[i].guid,
> @@ -290,7 +292,32 @@ static void init_vp_index(struct vmbus_channel
> *channel,
> channel->target_vp = 0;
> return;
> }
> - cur_cpu = (++next_vp % max_cpus);
> +
> + /*
> + * Primary channels are distributed evenly across all vcpus we have.
> + * When the host asks us to create subchannels it usually makes us
> + * num_cpus-1 offers and we are supposed to distribute the work
> evenly
> + * among the channel itself and all its subchannels. Make sure they
> are
> + * all assigned to different vcpus.
> + */
> + if (!primary)
> + cur_cpu = (++next_vp % max_cpus);
> + else {
> + /*
> + * Let's assign the first subchannel of a channel to the
> + * primary->target_cpu+1 and all the subsequent channels
> to
> + * the prev->target_cpu+1.
> + */
> + spin_lock_irqsave(&primary->lock, flags);
> + if (primary->num_sc == 1)
> + cur_cpu = (primary->target_cpu + 1) % max_cpus;
> + else {
> + prev = list_prev_entry(channel, sc_list);
> + cur_cpu = (prev->target_cpu + 1) % max_cpus;
> + }
> + spin_unlock_irqrestore(&primary->lock, flags);
> + }
> +
> channel->target_cpu = cur_cpu;
> channel->target_vp = hv_context.vp_index[cur_cpu];
> }
> --
> 1.9.3
next prev parent reply other threads:[~2015-04-24 8:55 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-04-21 14:27 [PATCH 0/6] Drivers: hv: vmbus: fair round robin algorithm for vmbus_get_outgoing_channel() Vitaly Kuznetsov
2015-04-21 14:27 ` [PATCH 1/6] Drivers: hv: vmbus: unify calls to percpu_channel_enq() Vitaly Kuznetsov
2015-04-21 14:27 ` [PATCH 2/6] Drivers: hv: vmbus: briefly comment num_sc and next_oc Vitaly Kuznetsov
2015-04-21 14:27 ` [PATCH 3/6] Drivers: hv: vmbus: decrease num_sc on subchannel removal Vitaly Kuznetsov
2015-04-24 7:02 ` Dexuan Cui
2015-04-24 8:40 ` Vitaly Kuznetsov
2015-04-21 14:27 ` [PATCH 4/6] Drivers: hv: vmbus: move init_vp_index() call to vmbus_process_offer() Vitaly Kuznetsov
2015-04-24 8:38 ` Dexuan Cui
2015-04-24 8:46 ` Vitaly Kuznetsov
2015-04-21 14:27 ` [PATCH 5/6] Drivers: hv: vmbus: distribute subchannels among all vcpus Vitaly Kuznetsov
2015-04-24 8:40 ` Dexuan Cui [this message]
2015-04-24 9:05 ` Vitaly Kuznetsov
2015-04-24 16:46 ` KY Srinivasan
2015-04-27 13:30 ` Vitaly Kuznetsov
2015-04-27 18:09 ` KY Srinivasan
2015-04-21 14:27 ` [PATCH 6/6] Drivers: hv: vmbus: do a fair round robin when selecting an outgoing channel Vitaly Kuznetsov
2015-04-24 8:42 ` Dexuan Cui
2015-04-24 8:59 ` Vitaly Kuznetsov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=091d0fd6321f4dd490e61a574d5b5b50@SIXPR30MB031.064d.mgd.msft.net \
--to=decui@microsoft.com \
--cc=devel@linuxdriverproject.org \
--cc=haiyangz@microsoft.com \
--cc=kys@microsoft.com \
--cc=linux-kernel@vger.kernel.org \
--cc=vkuznets@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®