Re: Automatic TCP send socker buffer sizing



Hello,

I've tested this out and had a few comments.

1) I've seen in production that some sockets get large very quickly during periods of high latency (eg: when sending to a user downloading from a cablemodem and their headend gets temporarily saturated and has large buffers, which raises the RTT under saturation, which increases the bandwidth delay product), but then as there isn't any code to shrink the buffers. This would probably need to be in the timers to notice the case of the sender temporarily stopping sending - eg in a keepalive http socket (a separate, but related issue).

2) In our code, and I believe in other code, we watch the tcp send buffer drain before closing the sockets, which in our implementation works by checking with kqueue how many bytes we can send to that socket. If the buffer size keeps changing, we would now have to call getsockopt() periodically to get the current size, which would obviously have a polling-type performance problem. Is it possible to make a kqueue hook to let us know when the socket buffer size changes?

3) There is a potential for a DoS here as you could have a large number of clients simulate a very large bandwidth delay product (eg: ipfw delay 100,000 ms) and start up a lot of downloads. This could cause your total socket buffers to rapidly grow much larger than normal and eat up all the available kernel memory. Perhaps having another tunable for a soft limit on total socket size past which we get more conservative on buffer sizing, and reduce the size of the largest buffers in exchange for making the smallest buffers larger.

4) What happens if you call setsockopt() to set the size? Perhaps we can have a new setsockopt() flag to disable automatic sizing on a socket if we have a specific size we want a buffer to be and don't want it dynamically sized.


After some refining, this does have some very good potential to be very useful for us.

-P
_______________________________________________
freebsd-net@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@xxxxxxxxxxx"



Relevant Pages

  • Re: Sockets debugging tools
    ... This is perfectly possible by programming your peer not ... system buffers are fairly large ... and the socket just remains in a permanent would-block state? ... > returned by recv or send calls. ...
    (comp.unix.programmer)
  • Re: Socket switch delay
    ... buffers are queued and will be later sent. ... socket, not sure about a blocking socket. ... >>(and I noticed you don't do overlapped receive at the server!) ... > optional in the client too). ...
    (microsoft.public.win32.programmer.networks)
  • [PATCH 2.6.23-rc6 Resending] NETWORKING : Edge Triggered EPOLLOUT events get missed for TCP sock
    ... SOCK_NOSPACE bit is set in the socket flags. ... EPOLLOUT events to be missed for TCP sockets, ... No free buffers (all are dispatched to sendfile and are ...
    (Linux-Kernel)
  • Re: Fixing socket.makefile()
    ... Donn Cave wrote: ... > in Python as in the original C, with socket file descriptors ... > in place of socket objects and stdio file pointers in place ... buffers of buffers. ...
    (comp.lang.python)
  • Re: To Block or not to Block
    ... > non-blocking sockets and most of the time it works fine, ... > non-blocking socket to want to block. ... after socket declared that its buffers are full. ... man setsockopt, section on SO_SNDBUF: ...
    (comp.unix.programmer)