Re: BSD sockets: recv with MSG_WAITALL should return EWOULDBLOCK?

"Rainer Weikusat" <rainer.weikusat@xxxxxxxxx> wrote in message
news:87ac1ha7uz.fsf@xxxxxxxxxxxxxxxxxxxx
"Alex Fraser" <me@xxxxxxxxxxx> writes:
"Rainer Weikusat" <rainer.weikusat@xxxxxxxxx> wrote:
"Alex Fraser" <me@xxxxxxxxxxx> writes:
It seems to me that MSG_WAITALL does not make sense on non-blocking
sockets. (I am not convinced it is much use for blocking sockets
either.)

It is useful for bulk downloads.

...but not much else.

For a start, either you must be expecting the sender to send at least as
many bytes as the size argument to recv() or to close the connection
after sending. So MSG_WAITALL has strictly limited applicability.

This sounds a lot like 'it is only useful if you specified the flag
because it would be useful and it wasn't, for instance, generated by a
call to random(3)'. Which seems pretty obvious to me.

I said I wasn't convinced MSG_WAITALL was much use. Part of that is the fact
that you can't possibly use it in many cases. Yes, that is obvious; I was
just being explicit.

Even if the above is satisfied, MSG_WAITALL can harm throughput if the
size passed to recv() is "too large" and you are performing some kind of
stream processing (for instance, decompressing it).

And this sounds like 'if you are using a buffer size that doesn't
match some unspecified restrictions some postprocessing code may have
[...]

I think you completely missed the point here. Suppose you have a simple loop
like this:

while ((n = recv(s, buf, len, MSG_WAITALL)) > 0)
consume(buf, n);

Where consume() implements an algorithm which runs in O(n) time. In
practice, of course, there will be some per-call overhead. Assume consume()
can process data faster than it can possibly need to (eg it runs at 20MB/s
but the input is coming over a 100Mbit/s link).

Clearly, the greater the value of len, the better the efficiency (ie
clocks/byte) will be because there are fewer kernel/userspace transitions
due to fewer recv() calls, and of course, similar applies to the per-call
overhead from consume().

But beyond some value of len, you will eventually reach a (system-dependent)
point where throughput starts to reduce, because the kernel buffers will
fill up while consume() is running. That is, len can be "too large". But how
large is too large?

Assuming your application processes data fast enough, you can expect
recv to return for each 'link-layer PDU' (eg ethernet frame) received
unless the kernel handles TCP PSH somehow (at least Linux doesn't).

The above is only true if the delay between when the thread blocked in
recv() becomes ready to run (the relevant receive buffer becomes
non-empty) and actually running (a CPU becomes available) is always
small.

Technically, this is not correct. It doesn't matter if the delay is
small or large, the process must just be able to process incoming
frames in real time.

You snipped the important bit, perhaps because once again you missed the
point. This is roughly what I think could (depending on various factors)
happen if the delay is large:

1. The application calls recv() supplying a large buffer but without
specifying MSG_WAITALL. No data is available so the thread sleeps.
2. A frame arrives causing data to be queued to the socket buffer. The
thread becomes ready-to-run.
3. Another frame arrives and more data is queued to the socket buffer. (And
perhaps another, etc.)
4. The thread runs; all available data is copied to the user buffer, then
recv() returns.

This is getting closer to what would happen if you used MSG_WAITALL, ie the
relative gain in efficiency is reduced on a loaded system.

I haven't made any new points in this post, just - hopefully - clarified the
points I was trying to make before. Unless you actually address them I think
we'll have to call it a day and agree to disagree :).

Alex


.

Relevant Pages

  • unix(7) and MSG_TRUNC semantics
    ... I use unix domain datagram sockets. ... The man page recv(2) tells me about the flags argument to a recv ... message truncation due to insufficient buffer size. ...
    (Linux-Kernel)
  • Problem with unix sockets: SOCK_DGRAM ignores MSG_TRUNC
    ... I use unix domain datagram sockets for IPC, ... "man 2 recv" tells me about the flags argument to a recvcall, ... message truncation due to insufficient buffer size. ...
    (Linux-Kernel)
  • Re: BSD sockets: recv with MSG_WAITALL should return EWOULDBLOCK?
    ... due to fewer recv() calls, and of course, similar applies to the per-call ... that the application needs to use a sensible buffer size. ... Another frame arrives and more data is queued to the socket buffer. ...
    (comp.unix.programmer)
  • Honoring SO_RCVLOWAT in proto_ops.poll methods
    ... I have a need for select/poll/epoll_wait to block on sockets which have ... looking for specific keyword and value from buffer. ... If buffer is too short call setsockopt() with SO_RCVLOWAT parameter set ... greater than what recv() with MSG_PEEK returned. ...
    (Linux-Kernel)
  • Socket Programming Question
    ... Ok, so I'm not good with sockets, but I need some help. ... when I call recv() to get the message sent by a client I wrote, ... characters to the buffer than was indicated by the return value of recv. ...
    (comp.lang.c)