Re: sendfile(2) SF_NOPUSH flag proposal

From: Igor Sysoev (is_at_rambler-co.ru)
Date: 05/27/03

  • Next message: Igor Sysoev: "Re: sendfile(2) SF_NOPUSH flag proposal" 
    Date: Tue, 27 May 2003 14:49:35 +0400 (MSD)
To: Terry Lambert <tlambert2@mindspring.com>

    On Tue, 27 May 2003, Terry Lambert wrote:

    > Igor Sysoev wrote:
    > > > I don't see this as being terrifically useful; small files
    > > > should probably just be mapped and written; the copy expense
    > > > is still there for the headers and trailers, no matter what,
    > > > and the file size itself is very small overhead, relatively
    > > > speaking, for files small enough for this to be an issue.
    > >
    > > FreeBSD 4.x has no zero_copy(9) so mmap()ed files would be copied.
    > > sendfile() allows to avoid this copy.
    >
    > It's actually "one copy", from an external mbuf reference
    > to a VM buffer to the card, vs. two copies, from the mapped
    > file in the process address space to mbufs, to the card.

    I know what "zero copy" means.

    > My point, though was that fr files small enough to fit in a
    > packet, you are probably going to spend more on the headers
    > and trailers copyin and the mbuf list assembly and the system
    > call overhead, than you'll end up spending one the extra data
    > copy for the very small file contents.
    >
    > I would be really surprised if you were able to demonstrate a
    > measuarble performance difference which was above the noise.

    I hope I will demonstrate at least CPU usage in near future.

    > > By the way what do you call by small files ? 4K, 30K or 100K ?
    >
    > You were talking about the file and the header living in the
    > same packet.

    I mean that if you have 230 bytes header then sendfile() will send it
    in separate packet nevertheless the size of header and of the file.
    Something like this - 230, 1460, 1460, ...

    > > > I also think your headers and trailers are very small, if
    > > > they are fitting with the file contents in a single packet.
    > > > I think this is atypical.
    > >
    > > If I ask Google:
    > > ----
    > > HEAD /images/hp0.gif HTTP/1.0
    > > Host: www.google.com
    > >
    > > ----
    > > it will return me 230 bytes:
    >
    > The "HEAD" is atypical, compared to the "GET"; the full Google
    > front page is larger than that, and consists of multiple files;
    > assuming you support HTTP/1.1 and pipelining, it's going to be
    > a back-to-back transfer involving multiple sendfile() calls.

    I use HEAD to show you the size of the HTTP header.
    The HEAD is atypical but such small HTTP header is typical.

    > > > BTW: if you go ahead with this, you should verify that it
    > > > also works for the trailers, etc., and you should probably
    > > > skip it if you headers > transmit queue depth, or file size
    > > > > transmit queue depth, or trailers > transmit queue depth.
    > >
    > > Currently sendfile() can send the file in not full packets even
    > > file is bigger then the transmit queue depth. It can send
    > > it in 2 x 1460 + 1176 or 5 x 1460 + 892 packets.
    >
    > 3 packets vs. 6. And using HTTP/1.0, there's also the three
    > handshake packets, SYN/SYN-ACK/ACK, and the tear-down three
    > teardown packets, FIN/FIN-ACK/ACK (or 4), plus the ACK's for
    > the packets you sent (should be one ACK, since that's below
    > the TCP window size).

    Actually 6 vs. 6 for this 8K file. But I said about another thing.
    Let's see 48K file and 250 bytes header. sendfile() usually sends
    it as 4K or 8K hunks so there are 48/8 * 6 + 1 (header) = 37 packets.
    But (48K + 250) / 1460 = 33 * 1460 + 1270 i.e. 34 packets.
    It's 8% decrease of data packets. Add here the possible retransmitions.

    > Really: it's in the noise. Unless you are paying by packet
    > count, you probably shouldn't care.

    So do you consider that IP fragmentation is the good thing ?

    Igor Sysoev
    http://sysoev.ru/en/

    _______________________________________________
    freebsd-arch@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-arch
    To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org"

  • Next message: Igor Sysoev: "Re: sendfile(2) SF_NOPUSH flag proposal"

    Relevant Pages

    • sendfile(2) SF_NOPUSH flag proposal
      ... packets even their sizes allow to place all them in one packet. ... For example the typical HTTP response header is less then an ethernet ... When I turn TCP_NOPUSH on just before sendfile() then it sends the header ... try the trailer) on FreeBSD 4.2 and 4.3 without src/sys/netinet/tcp_usrreq.c ...
      (freebsd-arch)
    • RFC: Sendfile patch
      ... The attached patch: ... modifies sendfile to ensure that the header is sent in the same packet as ... it if someone could verify that I have the old sendfile compatibility part ...
      (freebsd-arch)
    • RE: Transfer a sending packet to upper TCP/IP protocol layer in IM
      ... If the IPv6 address can be resolved, ... IPv4 header will be larger than the MTU. ... After prepending IPv4 header and UDP header to the original IPv6 packet, ...
      (microsoft.public.development.device.drivers)
    • Re: sendfile(2) SF_NOPUSH flag proposal
      ... limiting factor to be PCI bus bandwidth first, memory second, ... > in separate packet nevertheless the size of header and of the file. ... So when a retransmit, if any, is necessary, the packet stream ...
      (freebsd-arch)
    • RE: Transfer a sending packet to upper TCP/IP protocol layer in IM
      ... The original MAC header is INVALID because my IM driver will assign a new ... After prepending IPv4 header and UDP header to the original IPv6 packet, ...
      (microsoft.public.development.device.drivers)