Re: kqueue giant-locking (&kq_Giant, locking)

From: Mark Terribile (materribile_at_yahoo.com)
Date: 05/14/04

  • Next message: Yar Tikhiy: "Interoperation of flock(2), fcntl(2), and lockf(3)" 
    Date: Fri, 14 May 2004 01:30:27 -0700 (PDT)
To: arch@freebsd.org

    Sirs,

    Please excuse my coming in late on this; I don't ordinarily follow this list.

    Brian F. Feldman writes

    > I can't imagine a well-designed applications has kqueues of kqueues. I
    > didn't remove the file descriptor polling interface, I removed the file
    > descriptor kqueue interface.

    I took over a server to which I added a kqueue of kqueues. It used separate
    kqueues as priority queues, with controls on starvation. I found this
    essential for getting full performance out of the box. It was a caching web
    proxy running on a 1 GHz PIII, serving about 1,500 requests per second. That
    actually meant about 7,200 stimuli/second, including disk completion, TCP
    connects, TCP disconnects, TCP read data available, and TCP write buffer
    space available. It did its own async DNS, with caching. We often ran
    with 40,000 file descriptors open. After tuning of IP buffer space and
    connect queues, it withstood a 20 percent overload with graceful degradation,
    load shedding, and no lockup.

    Without multiple kqueues, this would have been far more difficult--or
    impossible.

    Roughly 70% of the processor was spent in the kernel, and measurements
    with different load mixes indicated the bulk of that was in the TCP stack.
    This means that the average service time at user level was about 40 usec.
    FreeBSD and recursive kqueue deserve a lot of the performance credit.

    I consider the server well-designed. When I was doing this, other developers
    at other companies were getting similar results from similar hardware, some
    of them using FreeBSD and kqueue. I have reason to believe that they were
    using similar priority schemes.

    I recommend allowing kqueues of kqueues.

         Mark Terribile

            
                    
    __________________________________
    Do you Yahoo!?
    SBC Yahoo! - Internet access at a great low price.
    http://promo.yahoo.com/sbc/
    _______________________________________________
    freebsd-arch@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-arch
    To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org"

  • Next message: Yar Tikhiy: "Interoperation of flock(2), fcntl(2), and lockf(3)"