Update: Debox sendfile modifications

From: Mike Silbersack (silby_at_silby.com)
Date: 11/04/03

  • Next message: Vivek Pai: "Re: Update: Debox sendfile modifications"
    Date: Tue, 4 Nov 2003 11:02:35 -0600 (CST)
    To: Vivek Pai <vivek@CS.Princeton.EDU>
    
    

    Ok, I've reread the debox paper, looked over the patch, and talked to Alan
    Cox about his present and upcoming work on the vm system.

    The debox patch does three basic things (if I'm understanding everything
    correctly.):

    1. It ensures that the header is sent in the same packet as the first
    part of the data, fixing performance with small files.

    - This part of the patch needs a little cleanup, but that's easy enough.
    I will try to integrate it next week.

    2. The patch merges sendfile buffers so that when the same page is sent
    to multiple connections, kernel address space is not wasted.

    - While this is the part of the patch with the widest benefit, it will be
    the most difficult to integrate. In order to support 64-bit architectures
    better, Alan has refactored the sendfile code, meaning that the patch
    would have to be rewritten to fit this new layout.

    3. The patch returns a new error when sendfile realizes that it will have
    to block on disk I/O, thereby allowing Flash to have a helper do the
    blocking call.

    - While this change could be made easily enough, I'm not sure that it
    would benefit anything other than Flash, so I'm not certain if we should
    do it. However, based on what you learned with Flash, I have an alternate
    idea:

    ---
    Suppose that sendfile is called to send to a non-blocking socket, and that
    it detects that the page(s) required are not in memory, and that disk I/O
    will be necessary.  Instead of blocking, sendfile would call a sendfile
    helper kernel thread (either by calling kthread_create, or by having a
    preexisting pool.)  After dispatching this thread, sendfile would return
    EWOULDBLOCK to the caller.  Note that only a limited number of threads
    would exist (perhaps 8?), so, if all threads were busy, sendfile would
    have to block like it does at present.
    Once the I/O was complete, the thread would call sowakeup (or whatever is
    called typically when a thread is now ready for writing) for the socket in
    question.  The application would call sendfile, like normal, but this time
    everything would succeed because the page would be in memory.
    ---
    If such a feature were implemented, it might have the same increased
    performance effect that your new return value does, except that it would
    require no modification for a non-blocking sendfile based application to
    take advantage of it.
    Alan, would this be possible from the VM system's perspective?  Is it safe
    to assume that once the page in question was in the page cache that it
    would hang around long enough for the second sendfile call to access it
    before it is paged back out again?
    Thanks,
    Mike "Silby" Silbersack
    _______________________________________________
    freebsd-hackers@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
    To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
    

  • Next message: Vivek Pai: "Re: Update: Debox sendfile modifications"

    Relevant Pages

    • Re: Update: Debox sendfile modifications
      ... > Ok, I've reread the debox paper, looked over the patch, and talked to Alan ... > better, Alan has refactored the sendfile code, meaning that the patch ... > would benefit anything other than Flash, so I'm not certain if we should ... > it detects that the pagerequired are not in memory, ...
      (freebsd-hackers)
    • Re: sendfile and page usage statistics
      ... PG_REFERENCED is never set when sendfile() gets cached vm_page. ... As I understand the following patch against uipc_syscalls.c ... You are missing opening/closing braces on your ifstatement with this ... I'm going to test the patch on machine where all files takes more memory ...
      (freebsd-current)
    • Re: [PATCH] A generic_file_sendpage()
      ... Your patch is *much* smaller than mine. ... depend on the struct file* passed to generic_file_sendpage. ... If you actually open, sendfile() and close, you end up ... send the line "unsubscribe linux-kernel" in ...
      (Linux-Kernel)
    • Re: sendfile and page usage statistics
      ... so sendfile() should set it in advance. ... I'm going to test the patch on machine where all files takes more memory ... user process does not see this mmap. ...
      (freebsd-current)
    • Re: [i2c] [PATCH] i2c: Push ioctl BKL down into the i2c code
      ... > Hi Alan, ... > Description of what the patch does and why it is needed, ... to do with these patches and they don't follow lkml. ... that we can work on getting rid of the big kernel lock for good. ...
      (Linux-Kernel)