Re: API change for bus_dma

From: Andrew Gallatin (gallatin_at_cs.duke.edu)
Date: 06/27/03

  • Next message: Justin T. Gibbs: "Re: API change for bus_dma" 
    Date: Fri, 27 Jun 2003 17:52:29 -0400 (EDT)
To: Scott Long <scott_long@btc.adaptec.com>

    Scott Long writes:
    > >
    > > Unfortunately, in our application we must assume coherency in some
    > > situations. We have kernel memory mmap'ed into user space for
    > > zero-copy io of small messages. Doing a system call to force the dma
    > > sync would add unacceptable latency. (we're talking sub 10us latencies
    > > here, without syscalls).
    > >
    >
    > The bus_dmamap_sync() isn't done from a separate system call. The flow
    > is this:
    >
    >
    > bus_dmamap_load();
    > driver_callback()
    > {
    > set up S/G list;
    > bus_dmamap_sync(PREWRITE);
    > tell hardware that DMA is ready;
    > }
    >
    > The callback gets called immediately as long as conditions are met, as
    > we have discuss prior.
    >
    > Then:
    >
    > driver_intr()
    > {
    > see that hardware has DMA'd data to us;
    > bus_dmamap_sync(POSTREAD);
    > bus_dmamap_unload();
    > }

    In our application, millions of separate DMAs can happen with no kernel
    intervention at all.

    We do the bus_dmamap_load() in the context of an ioctl which allocates
    some kernel memory, and pokes the DMA addresses for the kernel memory
    down onto the board. The user then mmaps the kernel memory, and also
    mmaps a page of SRAM on our board.

    When the user wants to initate a dma, he writes to the device's SRAM
    indicating an offset in the mmaped kernel memory. The board then
    initates the transfer (after doing bounds checking..).

    When the user is done, or the process exits, the dma addresses are
    cleared from the nic, and the kernel memory is freed.

    This is OS-bypass networking.

    > As I understand it, it is possible to set the pycho bridge to use
    > a coherent address range, but FreeBSD doesn't take advantage of that
    > yet.
    >

    Yes, that's what solaris does..

    Drew
    _______________________________________________
    freebsd-arch@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-arch
    To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org"

  • Next message: Justin T. Gibbs: "Re: API change for bus_dma"