From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759565AbcAUNje (ORCPT ); Thu, 21 Jan 2016 08:39:34 -0500 Received: from mx1.redhat.com ([209.132.183.28]:44361 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1759403AbcAUNja (ORCPT ); Thu, 21 Jan 2016 08:39:30 -0500 Date: Thu, 21 Jan 2016 15:39:26 +0200 From: "Michael S. Tsirkin" To: virtio@lists.oasis-open.org Cc: virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org, qemu-devel@nongnu.org, dev@dpdk.org, virtualization@lists.linux-foundation.org, "Xie, Huawei" , linux-kernel@vger.kernel.org Subject: virtio ring layout changes for optimal single-stream performance Message-ID: <20160121145418-mutt-send-email-mst@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi all! I have been experimenting with alternative virtio ring layouts, in order to speed up single stream performance. I have just posted a benchmark I wrote for the purpose, and a (partial) alternative layout implementation. This achieves 20-40% reduction in virtio overhead in the (default) polling mode. http://article.gmane.org/gmane.linux.kernel.virtualization/26889 The layout is trying to be as simple as possible, to reduce the number of cache lines bouncing between CPUs. For benchmarking, the idea is to emulate virtio in user-space, artificially adding overhead for e.g. signalling to match what happens in case of a VM. I'd be very curious to get feedback on this, in particular, some people discussed using vectored operations to format virtio ring - would it conflict with this work? You are all welcome to post enhancements or more layout alternatives as patches. TODO: - documentation+discussion of interaction with CPU caching - thorough benchmarking of different configurations/hosts - experiment with event index replacements - better emulate vmexit/vmentry cost overhead - virtio spec proposal Thanks! -- MST