RE: em0, polling performance, P4 2.8ghz FSB 800mhz

From: Don Bowman (don_at_sandvine.com)
Date: 02/29/04

  • Next message: Deepak Jain: "Re: em0, polling performance, P4 2.8ghz FSB 800mhz" 
    To: 'Deepak Jain' <deepak@ai.net>
Date: Sat, 28 Feb 2004 23:17:44 -0500

    > It was kindly pointed out that I didn't including the symptoms of the
    > problem:
    >
    >
    > Without polling on, I get 70+% interrupt load, and I get live lock.
    >
    > With polling on, I start getting huge amounts of input errors, packet
    > loss, and general unresponsiveness to the network. The web
    > server on it
    > doesn't respond though it occassionally will open the
    > connection, just
    > not respond. accept_filter on/off makes no difference. I have
    > read other
    > posts that say em systems can more >200kpps without serious incident.
    >
    > Thanks in advance,
    >
    > DJ

    You may need to increase the MAX_RXD inside your em driver to e.g. 512.

    With similar system, em can handle ~800Kpps of bridging.

    Your earlier email showed a very large number of RST messages,
    which makes me suspect the blackhole actually wasn't enabled.

    Not exactly sure what you're trying to do here. It sounds like
    you are trying to generate a SYN flood on port 80, and your listen
    queue is backing up. You've increased kern.ipc.somaxconn? Does your
    application specify a fixed listen queue depth? Could it be increased?
    Are you using apache as the server? Could you use a kqueue-enabled
    one like thttpd?

    Have you checked net.inet.ip.intr_queue_drops?
    If its showing >0, check net.inet.ip.intr_queue_maxlen, perhaps
    increase it.

    Have you sufficient mbufs and clusters? netstat -m.

    If you want to spend more time in kernel, perhaps change
    kern.polling.user_frac to 10?

    I might have HZ @ 2500 as well.

    You could use ipfw to limit the damage of a syn flood, e.g.
    a keep-state rule with a limit of ~2-5 per source IP, lower the
    timeouts, increase the hash buckets in ipfw, etc. This would
    use a mask on src-ip of all bits.
    something like:
    allow tcp from any to any setup limit src-addr 2

    this would only allow 2 concurrent TCP sessions per unique
    source address. Depends on the syn flood you are expecting
    to experience. You could also use dummynet to shape syn
    traffic to a fixed level i suppose.

    now... this will switch the DoS condition to elsewhere in
    the kernel, and it might not win you anything.
    net.inet.ip.fw.dyn_buckets=16384
    net.inet.ip.fw.dyn_syn_lifetime=5
    net.inet.ip.fw.dyn_max=32000

    might be called for if you try that approach.

    _______________________________________________
    freebsd-hackers@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
    To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"

  • Next message: Deepak Jain: "Re: em0, polling performance, P4 2.8ghz FSB 800mhz"

    Relevant Pages

    • FreeBSD 7, bridge, PF and syn flood = very bad performance
      ... Does anyone try to see PF with "keep state" in action when under syn flood attack? ... I tried to get some help in freebsd-pf@, because the test firewall, that ... The problem is that a quad core bridge firewall running freebsd 7 amd64 with PF is near useless and can't handle "small" SYN ddos. ... It seems that device polling helps a lot in this situation, and at least the bridge firewall is accessible. ...
      (freebsd-current)
    • RE: em0, polling performance, P4 2.8ghz FSB 800mhz
      ... > use a mask on src-ip of all bits. ... > this would only allow 2 concurrent TCP sessions per unique ... Depends on the syn flood you are expecting ...
      (freebsd-hackers)
    • Re: em0, polling performance, P4 2.8ghz FSB 800mhz
      ... >>You could use ipfw to limit the damage of a syn flood, ... >>timeouts, increase the hash buckets in ipfw, etc. ... You could also use dummynet to shape syn ...
      (freebsd-hackers)