Re: FreeBSD 4.x and OS-X tcp performance

From: Charles Sprickman (spork_at_fasttrackmonkey.com)
Date: 03/08/05

  • Next message: Daniel Hartmeier: "Re: FreeBSD 4.x and OS-X tcp performance"
    Date: Tue, 8 Mar 2005 02:04:20 -0500 (EST)
    To: Daniel Hartmeier <daniel@benzedrine.cx>
    
    

    Hello all,

    As promised, I have 4 tcpdump traces (saved with the "-w" option
    per request of another poster). Since these are a little too big to be
    broadcast to everyone on the list, I'm posting them here:

    http://home.manymonkeys.com/tcpdump/

    For all tests I used os-x's command line ftp program and uploaded a 1.7MB
    file to each system.

    tcpdump.osx-fbsd - tcpdump ran on os-x box while transferring to FBSD
    tcpdump.fbsd-osx - tcpdump ran on FBSD box while transferring from os-x

    tcpdump.osx-obsd - tcpdump ran on os-x box while transferring to OBSD
    tcpdump.obsd-osx - tcpdump ran on OBSD box while transferring from os-x

    So there's a view of a single file transfer on each run of the test.

    Observing this showed that the os-x to fbsd transfer went at about 200KB/s
    and the os-x to obsd transfer went at about 2.6MB/s.

    Again I'd like to note that the OpenBSD box is at 3.3, which may prove
    important since it's quite dated.

    If there's anything else I can provide, let me know.

    Thanks,

    Charles

    On Mon, 7 Mar 2005, Daniel Hartmeier wrote:

    > On Mon, Mar 07, 2005 at 02:04:01PM -0500, Charles Sprickman wrote:
    >
    >> Very interesting, thank you for that read of the tcpdump output. If you
    >> have the time, could you post back a few lines of the tcpdump with
    >> comments so that I might learn a little about what's going on? I don't
    >> have the best understanding of the intricacies of tcp...
    >
    > Data is flowing only in one direction (from client to server), so we
    > see the client sending payload and the server acknowledging it.
    >
    > As long as no packets are lost and the server keeps acknowledging, the
    > client could send one continuous stream of maximum sized payload packets.
    > However, if the server stops acknowledging, the client must stop sending
    > more data once it has filled up the window advertised by the server and
    > wait for the server to either acknowledge further data or increase its
    > window.
    >
    > Nothing exciting happens until timestamp 16:41:29.008909 in your log.
    >
    > 16:41:29.008672 client 321457:322905(1448)
    > 16:41:29.008751 server ack 322905 win 7240
    > 16:41:29.008909 client 324353:325801(1448)
    >
    > Since the log represents the server's view, the lack of segment
    > 324353:322905 in the log means that this packet from the client was lost
    > in transit. Now the server keeps acknowledging 322905, and we expect the
    > client to notice and retransmit the lost segment.
    >
    > 16:41:29.009064 server ack 322905 win 7400
    > 16:41:29.009146 client 325801:327249(1448)
    > 16:41:29.009233 server ack 322905 win 8424
    > 16:41:29.009409 client 328697:330145(1448)
    > 16:41:29.009521 server ack 322905 win 9448
    >
    > The client hasn't noticed yet and is still sending further segments past
    > the gap. The server is advertising increasing window sizes, probably
    > because it's still draining the data up to the gap.
    >
    > Notice that there's a new gap created as 327249:328697(1448) was lost.
    >
    > 16:41:29.011250 client 335937:337385(1448)
    > 16:41:29.011331 server ack 322905 win 14568
    >
    > Now the client has filled up the window. If it doesn't retransmit now,
    > it has to stall. It can't send anymore higher segments because the
    > window is full.
    >
    > 16:41:29.011758 client 322905:324353(1448)
    >
    > Finally, the retransmit.
    >
    > 16:41:29.011919 server ack 327249 win 10224
    >
    > Now the server acknowledges up the the second gap.
    >
    > 16:41:29.021620 server ack 327249 win 13296
    > ...
    > 16:41:29.024911 server ack 327249 win 56304
    >
    > And keeps acknowledging the same, while the window grows again. But
    > nothing from the client.
    >
    > 16:41:29.146218 192.168.0.40.58347 > home.manymonkeys.com.ftp: . ack 376
    > win 65535 <nop,nop,timestamp 38 05665564 543875729> (DF) [tos 0x10]
    >
    > Odd, an acknowledgment on the FTP control connection just at this time.
    > I don't know what that means, but it's not a coincidence, I'd bet.
    >
    > 16:41:30.462822 client 327249:328697(1448)
    >
    > Now the client finally retransmitted the segment from the second gap.
    > But look at the timestamp, about 1.4 seconds have passed, which is much
    > too long.
    >
    > After that, things go back to normal.
    >
    > The same thing repeats nine more times throughout the connection. You
    > can spot the places by comparing subsequent timestamps. You'll find the
    > spots where there are >1s delays in between two packets.
    >
    >> The tcpdump was run on the server (FBSD). Later today I will gladly do
    >> this again with a dump from each side.
    >
    > Before we blame the client, let's look at the dump from the client.
    > Maybe the client is trying to retransmit earlier, but those packets get
    > lost. It's just odd that this would happen for more than a full second.
    >
    > Daniel
    >
    _______________________________________________
    freebsd-net@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-net
    To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"


  • Next message: Daniel Hartmeier: "Re: FreeBSD 4.x and OS-X tcp performance"

    Relevant Pages

    • Strange network related data corruption
      ... I am encountering some strange data corruption when transferring ... When I have the server do the sha1sum of the file locally the problem ... /dev/zero and generate the sha1sum on the client. ... self-compiled kernel 2.6.23-rc9. ...
      (Linux-Kernel)
    • Re: FreeBSD 4.x and OS-X tcp performance
      ... It's a tcpdump of both hosts transferring a 1MB tarfile. ... Client 192.168.0.40 is uploading one file of about 1.6MB to ... server home.manymonkeys.com. ...
      (freebsd-net)
    • PROBLEM: NFS data corruption
      ... When transferring from a 2.6.16 nfs client to a 2.6.16.1 server I get ... There is no indication of any problems in any log on ...
      (Linux-Kernel)
    • Re: Servers losing sync or not syncing at all.
      ... > I have two problems with my NTP server/client. ... > The server & client are linux servers on a private lan. ... > The bd checksum can be ignored as tcpdump is getting the packet before the ...
      (comp.protocols.time.ntp)
    • Re: FreeBSD 4.x and OS-X tcp performance
      ... It's a tcpdump of both hosts transferring a 1MB tarfile. ... > server home.manymonkeys.com. ... > only retransmits the lost segment after a full 1.4 seconds. ... I think the client should retransmit earlier. ...
      (freebsd-net)