802.1Q vlan performance.

Hi

While doing some experimentation and work on ipfw to see where I
could improve performance for our virtualised firewall I came across
the following comment in sys/net/if_vlan.c:

* The VLAN_ARRAY substitutes the dynamic hash with a static array
* with 4096 entries. In theory this can give a boots(sic) in processing,
* however on practice it does not. Probably this is because array
* is too big to fit into CPU cache.

Being curious and having determined the main throughput bottleneck
to be the vlan driver, I thought that I'd test the assertion. I
have have 506 vlans on this machine.

With VLAN_ARRAY unset, ipfw disabled, fastforwarding enabled,
vlanhwtag enabled on the interface, the fastest forwarding rate I
could get was 278kpps (This was a steady decrease from 440kpps with
24 vlans linearly proportional to the number of vlans).

With exactly the same configuration, but the vlan driver compiled
with VLAN_ARRAY defined, the forwarding rate of the system is back
at 440kpps.

The testbed looks like this:

|pkt gen | | router | | pkt rec |
| host |vlan2 vlan2 | |vlan1002 vlan1002 | host |
|netperf |----------->| |------------------->| netserver|
| |em0 em0 | |em1 em0 | |

The router has vlan2 to vlan264 and vlan1002 through vlan1264 in
22 blocks of 23 vlan groups (a consequence of 24 port switches to
to tag/untag for customers). The pkt gen and recieve host both
have 253 vlans.

Can anyone suggest a good reason not to turn this option on by
default. It looks to me like it dramatically improves performance.

Ian

--
Ian Freislich
_______________________________________________
freebsd-current@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: 5.4 -- bridging, ipfw, dot1q
    ... On Fri, 12 Aug 2005, Luigi Rizzo wrote: ... I'd personally just be happy if ipfw was smart enough to know that if I ... I'd also really like it if non-bridged interfaces kept their arp table ... also happens to be one of the vlans within the dot1q trunk. ...
    (freebsd-hackers)
  • Re: 802.1Q vlan performance.
    ... that on moderate number of vlans it would be a regression. ... I> to be the vlan driver, I thought that I'd test the assertion. ... I> vlanhwtag enabled on the interface, ... the array to determine which VLAN does this frame belong to. ...
    (freebsd-current)