Re: TCP & R'cev Window Size Questions

ChrisS wrote:
I'm looking for some TCP information. To start with, I am not a TCP
expert. Just a Solaris Admin. And sorry for the book, but details do
count.

We have had issues FTPing files across a dedicated DS-3 for months now.
Part of the problem was a firewall in the path. That's temporarily
taken out and things have greatly improved. The TCP Receive window
size is not negotiating correctly all the time. Rather, it's loosing
its negotiation and dropping to 0.

We GET large files day-in and day-out. These files range in size from
400 MB to 4 GB. We're currently using the only good GUI FTP tool for
Solaris. (gFTP). We'll get upwards of 150 files a day! (They are a
binary file). Also, if we perform MGETS with the command line FTP this
problem also occurs.

While transferring files, we'll run a "netstat -f inet" on the FTP
client and we see the FTP transfer in progress with an expected TCP
Receive Window size. In this case, we've tried many different NDD
settings for the receive & send windows. Including Congestion buffer,
max buffer, and scaling set to 1. We've tried Win. sizes between 500
KB to 4 MB. Results are always the same - loss of a good negotiated
WinSize. Oh, both client and server are set to the same send/receive
window size durning any given moment.

What happens is the user will grab many files at once within gFTP
2.0.18 and with a netstat running, we see it negotiates nicely with the
Solaris FTP server on the distant end, and at some apparently random
point during the transfer, the R'cev Window Size on the FTP CLIENT
machine drops to 0 or near zero (something like 1385 I think). If that
occurs more than several times in a row, the "netstat -f inet"
indicates the two machines (the client & server) are no longer
negotiating correctly and it sits near 0. Though watching it for every
10 seconds (netstat that is) shows it's attempting to re-negotiate for
a higher window, but can only jump up a marginal percentage, then it's
back down to zero.

The FTP server is a Sun Fire 6800 running Solaris 8 and the client is a
Sun Fire 15K domain with 12 CPUs running Solaris 9 with pretty
up-to-date patches, including the latest CE ethernet patch and kernel
for Sol 9. There's not other apparent pattern to if and when the
WinSize will drop. The only fix is to wait for that transfer to
complete, or to stop and restart the transfer and hope the TCP gods are
going to play nicer.

Does anyone have a clue as to what we can look at next to find the
problem? We may resort to testing with Solaris 10 shortly to see if
this is a problem with Solaris 8 or 9. Especially since S10 has a new
TCP stack.

Could it be we're pulling more than 4 GB of data at once in the FTP
session and it's blowing its top off of TCP?

Could the rollover of the TCP sequence numbers have anything to do with
it, when considering any lost packets that may occur would possibly be
using the re-cycled sequence numbers? After reaching 2 to the 32 -1,
this number will wrap around to 0.

Thanks for any suggestions.

Not sure if this will help, but you might
try using "route" to express your preference for the
large tcp window between these two hosts, for example:

on the 6800...
# route change <route-to-15K> -sendpipe 403456
# route change <route-to-15K> -recvpipe 403456
# route get <route-to-15K>

on the 15K...
# route change <route-to-6800> -sendpipe 403456
# route change <route-to-6800> -recvpipe 403456
# route get <route-to-6800>

Calculate the window more or less as:

(44*1024*1024*.070)/8 = 403456
------------ --- - ------
| | | |
| | | ------- window size in bytes
| | ------------- bits/byte
| ----------------- delay (rtt)
--------------------------- ~bandwidth of ds3 (bits/sec)

(don't know if bandwidth should be expressed as powers of 2 or 10)
but 44Mb/s is the right ball park for ds3. Perhaps someone else
can clarify.

check the values for these using ndd.
/dev/tcp tcp_max_buf
/dev/tcp tcp_wscale_always
/dev/tcp tcp_tstamp_if_wscale
/dev/tcp tcp_deferred_acks_max
/dev/tcp tcp_compression_enabled

Window scaling is negotiated only in the 3-way handshake so
far as I know. The window size may change, but scaling should not
once the connection is established. you might try increasing
tcp_deferred_acks_max in case exceeding this causes the window to
shrink. Also just an sysadmin. Good luck.
.

Relevant Pages

  • TCP & Rcev Window Size Questions
    ... I'm looking for some TCP information. ... Just a Solaris Admin. ... if we perform MGETS with the command line FTP this ... Receive Window size. ...
    (comp.unix.solaris)
  • Solaris 8 long-haul FTP performance problem
    ... The slowdown seems to have been coincident with an upgrade to Solaris 8 on ... during a transfer shows that larger window size is in use. ... The ftp client used is Sun's Solaris 8, and the servers are all WU-ftpd ...
    (comp.unix.solaris)
  • Solaris 8 long-haul FTP performance problem
    ... The slowdown seems to have been coincident with an upgrade to Solaris 8 on ... during a transfer shows that larger window size is in use. ... The ftp client used is Sun's Solaris 8, and the servers are all WU-ftpd ...
    (comp.sys.sun.admin)
  • Solaris 8 long-haul FTP performance problem
    ... The slowdown seems to have been coincident with an upgrade to Solaris 8 on ... during a transfer shows that larger window size is in use. ... The ftp client used is Sun's Solaris 8, and the servers are all WU-ftpd ...
    (comp.sys.sun.hardware)
  • Re: Re, Re : set prompt
    ... >> open multiple SmarTerm sessions in multiple SmarTerm windows. ... >> for FTP I open a DOS window and use its FTP! ... Only 3 lines of command output. ... window, I just type ASCII or BIN and I'm done with it. ...
    (comp.os.vms)