iSCSI, was Re: My project wish-list for the next 12 months

From: Danny Braniss (danny_at_cs.huji.ac.il)
Date: 12/15/04

  • Next message: Dmitry A. Bondareff: "5.3 IPFW bug"
    To: "Scott M. Ferris" <sferris@gmail.com>
    Date: Wed, 15 Dec 2004 10:05:10 +0200
    
    

    let me start by a local saying:
            Q- What's a camel
            A- It's a horse designed by a committee.
    BTW, the camel is a very efficient piece of equipment.

    If I would plan a 24/7 life support system I would not use iSCSI.
    (having to rely on packets traveling the Internet, DOS, etc, is not good for
    your health)

    Having said that, I can see the attractive sides of iSCSI, specially if there
    are vendors trying to peddle cheap disk boxes that speak iSCSI.
    At the moment, they only work under MS or Linux (BTW we tested them
    under Linux and so far we were not impressed - as HRH would say :-), so
    any kind of implementation is most welcome.

    > How do you plan on handling cases where the user program blocks and
    > can't login again (because of a page fault for code or data,
    > allocating a new socket in the kernel, allocating a new socket buffer
    > in the kernel, etc)?

    Let me see, if a user cannot login, and that is my main service, then probably
    not being able to login to my iSCSI disk is the least of my problems.

    Seriuosly, I understand your missgivings, but so many other, more simple
    problems might cause a meltdown.

    As others have pointed out, NFS/GEOM/etc rely on the kernel for resources,
    so should the iSCSI.

            danny

    >
    > One of the major problems any software-only iSCSI initiator has is
    > dealing with memory deadlocks. The OS may try to write out one or
    > more pages in order to free up memory. If the iSCSI initiator needs
    > to allocate memory (directly or indirectly in the TCP stack) in order
    > to complete that write, you've got a circular dependency where in
    > order to get free memory you need to have free memory.
    > Hardware-based iSCSI HBAs solve this by having their own memory and
    > TCP stack separate from the OS. Software-only iSCSI initiators such
    > as linux-iscsi usually just hope it doesn't happen, and that's why I
    > don't usually recommend software-only iSCSI initiators to anyone.
    >
    > Having a user program login is fine for SendTargets discovery and
    > testing, but if you're planning on getting a reliable iSCSI initiator,
    > you'll probably need to do the login from within the kernel, and make
    > sure you have all of the resources needed to do that reserved (wire
    > pages into RAM, etc). The hard part of that is making sure you have
    > all the resources needed to send and receive TCP packets reserved, as
    > well as the resources to establish new connections in case the
    > existing connections is closed or aborted. The linux-iscsi initiator
    > doesn't even attempt this, and just hopes deadlocks don't happen very
    > often.
    >
    > I used to be the main (and for most of that time, only) developer for
    > the linux-iscsi initiator. When I first took over the initiator, it
    > used the approach of doing iSCSI session login from a daemon, and
    > passing the socket down into a kernel module. I had to take that out
    > because it deadlocked too easily. I wouldn't recommend that approach.
    >
    > --
    > Scott M. Ferris

    _______________________________________________
    freebsd-hackers@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
    To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"


  • Next message: Dmitry A. Bondareff: "5.3 IPFW bug"

    Relevant Pages

    • Re: [RFC][PATCH 1/1] cxgb3i: cxgb3 iSCSI initiator
      ... driver to ask the iSCSI initiator where PDU boundaries are so it can ... I get how you can optimize "flows", but "flows" are a fancy name for a key that looks into a TCAM to get the "information" necessary to do header prediction. ...
      (Linux-Kernel)
    • Re: My project wish-list for the next 12 months
      ... You could preallocate, but the hard part would be getting the OS ... network stack to allocate from the iSCSI driver's preallocated memory ... pool, instead of calling the usual kernel memory allocator (or more ...
      (freebsd-hackers)
    • Re: [RFC][PATCH 2/9] deadlock prevention core
      ... Networked Block devices (NBD, iSCSI, AoE) can deadlock in the following ... deplete normal memory because of memory pressure; ... writeout over network, ... Mark some sockets with SOCK_MEMALLOC; which is essentially a promise to ...
      (Linux-Kernel)
    • Re: iSCSI initiator and media changer devices
      ... For iSCSI bugs, ... DEVICEMAP issue! ... >> This is not the only problem with MS iSCSI initiator. ... >> a situation when iSCSI target is mapping physical hardware as iSCSI. ...
      (microsoft.public.development.device.drivers)
    • [ANNOUNCE 0/6] Open-iSCSI High-Performance Initiator for Linux
      ... This is to announce Open-iSCSI project: High-Performance iSCSI Initiator ... ever-growing control plane code, including but not limited to: ...
      (Linux-Kernel)