mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: "Nivedita Singhvi" <nivedita@us.ibm.com>
To: ron.flory@adtran.com
Cc: linux-kernel@vger.kernel.org
Subject: Re: Kernel 2.4.6 and 2.4.7 networking performance: Seeing serious delays
Date: Wed, 15 Aug 2001 10:30:58 -0700	[thread overview]
Message-ID: <OF643DA24A.2BEBB0B0-ON88256AA9.0059AF34@boulder.ibm.com> (raw)


> [1.] One line summary of the problem:
>
>  Depending upon length of TCP socket write requests, I'm seeing
> extremely long TCP delays, limiting throughput to only 25 datagrams
> per second. Data blocks above a 'magic' length have a much higher
> throughput than shorter blocks, which is counter-intuitive.

This is correct (but unfortunate) operation of TCP. Youre just
hitting a well known phenomenon: the worst case interaction of
delayed acks and Nagle at a boundary condition.

Note that in the first case, your  write is 1447 bytes, which is
1 byte less than an MSS, i.e. subject to Nagle if there is an
outstanding packet not acked.

Whats happening, we suspect, is that the ack for the 8 byte request
(Frame 10) is delayed once delayed acks kick in (no longer in quick
ack mode). Delayed ack timeout is 40ms, and youre seeing around 36ms.

Since the other side has bytes outstanding and has to send a small
packet (1447 bytes, 1 less than MSS), it will delay the send (Nagle)
until it gets the ack.

> # Time Src Dst Port Info
>  8 0.000111 .100 .101 41359 > 4180 [PSH, ACK] Seq=4186558043
>       Ack=124141127 Win=5840 Len=8
>  9 0.000062 .101 .100 4180 > 41359 [PSH, ACK] Seq=124141127
>       Ack=4186558051 Win=5792 Len=8
> 10 0.036399 .100 .101 41359 > 4180 [ACK] Seq=4186558051
>       Ack=124141135 Win=5840 Len=0 <- DELAY
> 11 0.000074 .101 .100 4180 > 41359 [PSH, ACK] Seq=124141135
>       Ack=4186558051 Win=5792 Len=1447
> 12 0.000362 .100 .101 41359 > 4180 [ACK] Seq=4186558051
>       Ack=124142582 Win=8682 Len=0
> 13 0.000279 .100 .101 41359 > 4180 [FIN, ACK] Seq=4186558051
>       Ack=124142582 Win=8682 Len=0
> 14 0.000073 .101 .100 4180 > 41359 [FIN, ACK] Seq=124142582
>       Ack=4186558052 Win=5792 Len=0

In the next session, when youre sending the 1448 bytes, (exactly MSS),
the ack is delayed, but the write isnt delayed since we have a big enough
packet to send. The other end sends the ack as soon as it receives the
1447 bytes. (Acks are delayed unless we have more than one MSS
outstanding),
so it doesnt need to wait the full delayed ack timeout.

>  # Time Src Dst Port Info
>  8 0.000122 .100 .101 41363 > 4180 [PSH, ACK] Seq=4268751138
>       Ack=189815010 Win=5840 Len=8
>  9 0.000061 .101 .100 4180 > 41363 [PSH, ACK] Seq=189815010
>       Ack=4268751146 Win=5792 Len=8
> 10 0.000034 .101 .100 4180 > 41363 [ACK] Seq=189815018
>       Ack=4268751146 Win=5792 Len=1448
> 11 0.000351 .100 .101 41363 > 4180 [ACK] Seq=4268751146
>       Ack=189816466 Win=8688 Len=0
> 12 0.000520 .100 .101 41363 > 4180 [FIN, ACK] Seq=4268751146
>       Ack=189816466 Win=8688 Len=0

> Of particular interest is line #10 of each above.
>
> For some reason, in the case of the 'short' packet, the server machine
> waits for a TCP ACK from the client machine (delaying 36 millisec),
> which does not happen if the packet is just 1-byte longer, in which
> case the server immediately follows the 8-byte block with the
> 1448-byte payload.
[ snip ]

Hope that helps. Its a consequence of sending repeated small packets.
You can turn off Nagle (delayed sends) by setting the TCP socket option
TCP_NODELAY, or alter your application sequence of writes, if possible.

I'm still wondering why the next ack isnt delayed as well, though.
I dont believe we should still be in quickack mode, and
so I would have expected subsequent acks (there's just one, the FIN comes
down as a result of the close, which might be a factor, masked by ethereal
somehow) to be delayed as well.

Thanks to Sridhar Samudrala, Dave Stevens, Shirley Ma who got interested
in the question and contributed in shedding light on whats happening..

thanks,
Nivedita
----
Nivedita Singhvi                            (503) 578-4580
Linux Technology Center           nivedita@us.ibm.com
IBM Beaverton, OR                      nivedita@sequent.com


             reply	other threads:[~2001-08-15 17:27 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2001-08-15 17:30 Nivedita Singhvi [this message]
2001-08-16 23:06 ` Alexey Kuznetsov
2001-08-21  4:00 Nivedita Singhvi

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=OF643DA24A.2BEBB0B0-ON88256AA9.0059AF34@boulder.ibm.com \
    --to=nivedita@us.ibm.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=ron.flory@adtran.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®