From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760978AbaCUNdI (ORCPT ); Fri, 21 Mar 2014 09:33:08 -0400 Received: from mail-pd0-f172.google.com ([209.85.192.172]:34706 "EHLO mail-pd0-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751322AbaCUNdB (ORCPT ); Fri, 21 Mar 2014 09:33:01 -0400 Message-ID: <1395408778.6441.2.camel@edumazet-glaptop2.roam.corp.google.com> Subject: Re: [RFC] csum experts, csum_replace2() is too expensive From: Eric Dumazet To: Andi Kleen Cc: "H. Peter Anvin" , Patrick McHardy , Herbert Xu , "H.K. Jerry Chu" , Michael Dalton , netdev , "linux-kernel@vger.kernel.org" Date: Fri, 21 Mar 2014 06:32:58 -0700 In-Reply-To: <1395406250.9114.142.camel@edumazet-glaptop2.roam.corp.google.com> References: <1395341341.9114.93.camel@edumazet-glaptop2.roam.corp.google.com> <87a9cknwk4.fsf@tassilo.jf.intel.com> <1395406250.9114.142.camel@edumazet-glaptop2.roam.corp.google.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.3-0ubuntu6 Content-Transfer-Encoding: 7bit Mime-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 2014-03-21 at 05:50 -0700, Eric Dumazet wrote: > Or the fact that we mix 16 bit stores and 32bit loads ? > > iph->tot_len = newlen; > iph->check = 0; > iph->check = ip_fast_csum(iph, 5); Yep definitely. Using 16 bit loads in ip_fast_csum() totally removes the stall. I no longer see inet_gro_complete() in perf top... + if (__builtin_constant_p(ihl) && ihl == 5) { + asm(" movw (%[iph]), %w[sum]\n" /* ihl/version/tos */ + " addw 2(%[iph]), %w[sum]\n" /* tot_len */ + " adcw 8(%[iph]), %w[sum]\n" /* ttl/protocol */ + " adcw 10(%[iph]), %w[sum]\n" /* check */ + " adcl 4(%[iph]), %[sum]\n" /* id/frag_off */ + " adcl 12(%[iph]), %[sum]\n" /* saddr */ + " adcl 16(%[iph]), %[sum]\n" /* daddr */ + " adcl $0, %[sum]\n" + : [sum] "=r" (sum) + : [iph] "r" (iph) + ); + return csum_fold(sum);