Re: IPFW update frequency

Luigi Rizzo wrote:
On Sat, Mar 31, 2007 at 10:21:02AM +0200, Andre Oppermann wrote:
Julian Elischer wrote:
Luigi Rizzo wrote:
On Fri, Mar 30, 2007 at 01:40:46PM -0700, Julian Elischer wrote:
I have been looking at the IPFW code recently, especially with respect to locking.
There are some things that could be done to improve IPFW's behaviour
...
The locking overhead per packet in ipfw is by no means its limiting

i think you and Julian are looking at different issues.
if i understand julian's comment, the problem is that the list
is protected by a single lock, so no hope of parallelising
the work, and if one kernel thread is busy processing a packet
in the filter, others might be blocked for a long time
(in your case, the set of 30 rules is 765ns for ipfw and 1198ns
for pf).

Your tests presumably have little if any contention on the lock.

Specifically, if you compute the difference of the inverses
of those pps rates you see the following:

+pfil_pass 45.3 ns per packet

+ipfw_allow +253.4 ns/packet (setup and first rule)
+ipfw_30 +17.67 ns/(packet * extra rule)

+pf_pass +376.9 ns/packet (setup and first rule)
+pf_30 +28.34 ns/(packet * extra rule)


the lock acquisition cost is in the 'setup' part but i cannot tell
how expensive it is.
Julian's suggested change (and surely the one i described)
replaces the lock/unlock pair on the rule list with a refcount add/dec
pair (with uncontested locks the cost should be similar), but especially
makes the operation non-blocking allowing running the input and
output paths in parallel.

-current now used a read-write lock so this is theoretically already possible..
trouble is that the lock is held whioch produces LORs when you
try access something that is further up the stack e.g. the UID rules.


factor. Actually it's a very small part and pretty much any work on
it is lost love. It would be much better spent time to optimize the
main rule loop of ipfw to speed things up. I was profiling ipfw early
last year with an Agilent packet generator and hwpmc. In the meantime
the packet forwarding path (w/o ipfw) has been improved but relative
to each other the number are still correct.

actually your numbers show that at least the rule setup (and the
processing of simple rules) is significantly faster (50% or so) in
ipfw2 than in pf.

I know that the setup time is expensive, but i am not sure that
one can save much - in both cases, you need to fetching a lot
of information, which is scattered in variable locations in
the mbuf and packet headers.

cheers
luigi
_______________________________________________
freebsd-ipfw@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@xxxxxxxxxxx"

_______________________________________________
freebsd-net@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: IPFW update frequency
    ... is protected by a single lock, ... Haveing been involved in the hacks that went in and out of ipfw and pfil ... locking over the last few years and the problems that went along with it, ... early last year with an Agilent packet generator and hwpmc. ...
    (freebsd-net)
  • Re: IPFW update frequency
    ... Your tests presumably have little if any contention on the lock. ... the lock acquisition cost is in the 'setup' part but i cannot tell ... main rule loop of ipfw to speed things up. ... last year with an Agilent packet generator and hwpmc. ...
    (freebsd-net)
  • Re: IPFW update frequency
    ... Luigi Rizzo wrote: ... There are some things that could be done to improve IPFW's behaviour when processing packets, but some of these take a ... If you can tell us what is your plan to make the list lock free ... done with the packet. ...
    (freebsd-net)
  • Re: IPFW update frequency
    ... There are some things that could be done to improve IPFW's behaviour when processing packets, but some of these take a ... certainly ipfw was not designed with SMP in mind. ... If you can tell us what is your plan to make the list lock free ... done with the packet. ...
    (freebsd-net)
  • Re: ipfw: reset tcp
    ... > Luigi Rizzo wrote: ... I suspect it does not enter ipfw itself, ... to reprocess internally-generated packet, because you would rely ... but i don't think the generated packet did reenter the firewall. ...
    (freebsd-net)