Re: jumbo frames slowed Oracle/Tivoli backups

From: Rick Jones (foo_at_bar.baz.invalid)
Date: 02/25/05 

Date: Fri, 25 Feb 2005 21:52:05 GMT

jthvedt <jthvedt@yahoo.com> wrote:
> rick jones wrote:
>> What size sends does the backup application make, and is it rather
>> "chatty?" - that is is it rather request-response in nature?

> I don't know. I don't think so, but I'm not basing that on anything
> other than some MRTG charts that show tons of output at the AIX end and
> little input, and vice-versa on the Windows end. Any ITSMers listening
> in?

Might try taking a system call trace. Or you could try a packet trace
(eg tcpdump)

>> What was the window size of the backup connection(s)?

> On Windows, 128K. On AIX, I don't know. How do I tell? I thought "no
> -a" would tell me, but either it doesn't, or I can't read it.

One thing would be to see if there are setsockopt() calls by the
backup software and see what values they set - those should affect the
window size. Or, in a packet trace you can look at the SYNchronize
segments and their window and window scaling values.

[ Although just as the old Ginsu knife commercial said, this method
  does not always work with a tomato - in this case the tomato being
  Linux which in sufficiently contemporary releases seems to want to
  start with a small window and grow it for you automagically because of
  course the stack knows best... but I digress :) ]

>> Was there much in the way of packet loss - TCP retransmissions?

> No.

That makes me think the backup software may have been somewhat chatty
and may have followed the rather poor practice of:

write
write
read

where the data in the two write calls were logically associated and
the other end didn't reply until both had arrived. When you set the
MTU to 9000 bytes, the TCP MSS increases to ~9000-40 bytes which
becomes (generally) the new value against which the Nagle algorithm
checks. If, in particular, that second write were > 1460 bytes but
less than 8960 bytes it would become (properly) delayed by Nagle in
the 9000 byte MTU case.

The proper fix in such a situation is to get the software to use a
single, perhaps gathering, write such as writev() or sendmsg() so it
can place logically associated data into the socket at one time. This
will (should) give it to the stack at one time and thus _not_
necessarily run afoul of Nagle. (There are as always other details -
searching in groups.google.com or the like for "nagle logically
associated" will probably find some of the previous discussions -
generally in comp.protocols.tcp-ip)

>> Sometimes what different vendors consider "Jumbo" differs - are you
>> _certain_ that _all_ the deviced between the systems involved
>> supported full 9000 byte MTU jumbo frames?

> You mean really, REALLY certain? I think I've proven my ignorance in
> two short posts, so "no." But there are only three devices here, so I'm
> sort of confident.

You might also try a netperf TCP_STREAM or TCP_MAERTS test between the
two machines. That will see what the not to exceed bandwidth between
the two might be. http://www.netperf.org/

rick jones 

-- 
a wide gulf separates "what if" from "if only"
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to raj in cup.hp.com  but NOT BOTH...

Relevant Pages

  • Re: large tcp window
    ... >> TcpWindowSize or sliding window is size of buffer peer can send without ... >> waiting for ACK. ... >> Nagle algorithm. ... TCP over UDP even in such case where ...
    (microsoft.public.win32.programmer.networks)
  • Re: Corrupted DB, now problems entering data
    ... Well I found a backup that was onlythree months old. ... >> users type anything in that field, they get the wrong ResultID (which is ... That new window shows ...
    (microsoft.public.access.gettingstarted)
  • Re: Corrupted DB, now problems entering data
    ... The most recent DB backup is from six months ago (told you I was a DB noob). ... >> users type anything in that field, they get the wrong ResultID (which is ... That new window shows ...
    (microsoft.public.access.gettingstarted)
  • Re: 1
    ... However, once a weekly backup is made, it will be ... I meant a hundred different documents where "anti matter" could ... Time Machine makes a new backup. ... it will show a window containing the 12 midnight backup. ...
    (comp.sys.mac.system)
  • Re: Restore Win98 file
    ... the window didn't show any files on the CD". ... so the "window" (I'm assuming his XP "restore" window) ... will not be able to read W98 backup files ... ... if you install such software on Windows ...
    (microsoft.public.windowsxp.basics)