From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1754635AbcAHKgG (ORCPT <rfc822;w@1wt.eu>);
	Fri, 8 Jan 2016 05:36:06 -0500
Received: from smtp-out6.electric.net ([192.162.217.190]:50022 "EHLO
	smtp-out6.electric.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1754147AbcAHKgE (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Fri, 8 Jan 2016 05:36:04 -0500
From: David Laight <David.Laight@ACULAB.COM>
To: "'Hans Westgaard Ry'" <hans.westgaard.ry@oracle.com>,
        "David S. Miller" <davem@davemloft.net>
CC: Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>, James Morris <jmorris@namei.org>,
        Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>,
        Patrick McHardy <kaber@trash.net>,
        Alexei Starovoitov <ast@plumgrid.com>, Jiri Pirko <jiri@mellanox.com>,
        Eric Dumazet <edumazet@google.com>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Nicolas Dichtel <nicolas.dichtel@6wind.com>,
        "Eric W. Biederman " <ebiederm@xmission.com>,
        Salam Noureddine <noureddine@arista.com>,
        Jarod Wilson <jarod@redhat.com>,
        Toshiaki Makita <makita.toshiaki@lab.ntt.co.jp>,
        Julian Anastasov <ja@ssi.bg>, Ying Xue <ying.xue@windriver.com>,
        Craig Gallek <kraig@google.com>,
        Mel Gorman <mgorman@techsingularity.net>,
        "\" hannes\"@stressinduktion.org " <hannes@stressinduktion.org>,
        Edward Jee <edjee@google.com>, Julia Lawall <julia.lawall@lip6.fr>,
        "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Haakon Bugge <haakon.bugge@oracle.com>,
        Knut Omang <knut.omang@oracle.com>,
        Wei Lin Guay <wei.lin.guay@oracle.com>,
        Santosh Shilimkar <santosh.shilimkar@oracle.com>,
        Yuval Shaia <yuval.shaia@oracle.com>
Subject: RE: [PATCH] net: add per device sg_max_frags for skb
Thread-Topic: [PATCH] net: add per device sg_max_frags for skb
Thread-Index: AQHRSISSMwvRT8zmnE6xY3PnxvoTm57ugndggALiCQCAAAkwwA==
Date: Fri, 8 Jan 2016 10:33:11 +0000
Message-ID: <063D6719AE5E284EB5DD2968C1650D6D1CCC00C7@AcuExch.aculab.com>
References: <1452086182-26748-1-git-send-email-hans.westgaard.ry@oracle.com>
 <063D6719AE5E284EB5DD2968C1650D6D1CCBE5AA@AcuExch.aculab.com>
 <568F87AC.60405@oracle.com>
In-Reply-To: <568F87AC.60405@oracle.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.202.99.200]
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
X-Outbound-IP: 213.249.233.130
X-Env-From: David.Laight@ACULAB.COM
X-PolicySMART: 3396946, 3397078
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id u08AaCP0016740

From: Hans Westgaard
> Sent: 08 January 2016 09:56
...
> >> The patch allows the device to limit the maximum number fragments used
> >> in one skb.
> >
> > This doesn't seem to me to be the correct way to fix this.
> > Anything that adds an extra fragment (in this case IPoIB) should allow
> > for the skb already having the maximum number of fragments.
> > Fully linearising the skb is overkill, but I think the first fragment
> > can be added to the linear part of the skb.
> >
> > 	David
> >
> >
> When IpoIB handles a skb-request it converts fragments to SGEs to
> be handled by a HCA.
> The problem arises when the HCA have a limited number of SGEs less than
> MAX_SKB_FRAGS.
> (it gets a little worse since IPoIB need to yet another segment)
> I have not found any easy way of fixing this with currenct codebase.

I think one of the xen ethernet interfaces had a similar problem.

Just reduce the number of fragments by copying two (or more) of them
into a single fragment.
In effect, anything that reduces the number of fragments will do a copy.

	David