Re: [fbsd] Network performance in a dual CPU system


On Thu, 27 Apr 2006, Jeremie Le Hen wrote:

PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND
60 root 1 -44 -163 0K 8K WAIT 355.6H 72.17% swi1:
net
39 root 1 -68 -187 0K 8K WAIT 52.3H 5.22% irq28:
bge0
40 root 1 -68 -187 0K 8K WAIT 28.3H 2.25% irq29:
bge1
11 root 1 171 52 0K 8K RUN 166.6H 0.00% idle
63 root 1 -16 0 0K 8K - 121:55 0.00% yarrow
61 root 1 -32 -151 0K 8K WAIT 46:21 0.00% swi4:
clock sio
[...]

Does anyone know whether a dual CPU system can help us improve the situation? I was wondering if the software interrupt threads would be divided between the two processors.

I am a few weeks late, I just saw this very interesting thread. What solution did you finally employ to circumvent your high interrupt load ?

I missed the original thread, but in answer to the question: if you set net.isr.direct=1, then FreeBSD 6.x will run the netisr code in the ithread of the network device driver. This will allow the IP forwarding and related paths in two threads instead of one, potentially allowing greater parallelism. Of course, you also potentially contend more locks, you may increase the time it takes for the ithread to respond to new interrupts, etc, so it's not quite cut and dry, but with a workload like the one shown above, it might make quite a difference.

Robert N M Watson
_______________________________________________
freebsd-net@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: kernel: return from interrupt
    ... >> happening with PREEMPTION compiled in? ... > AFAIK this is the only return path from an interrupt. ... > another return path for the interrupts, the scheduler is not invoked on ... or call ithread_schedule() to schedule the ithread. ...
    (freebsd-current)
  • Re: Potential source of interrupt aliasing
    ... I can think of two solutions that avoid masking: ... > interrupt, then change it back to level triggered to unmask. ... But, certainly, changing the trigger mode can't be ... ithread come along later and process the data. ...
    (freebsd-current)
  • RE: serious networking (em) performance (ggate and NFS) problem
    ... > in the network layer that debug.mpsafenet=0 might correct. ... > noticed that our setup here has that setup, ... own ithread. ... generating a higher interrupt load. ...
    (freebsd-current)
  • RE: serious networking (em) performance (ggate and NFS) problem
    ... > in the network layer that debug.mpsafenet=0 might correct. ... > noticed that our setup here has that setup, ... own ithread. ... generating a higher interrupt load. ...
    (freebsd-stable)
  • Re: Potential source of interrupt aliasing
    ... > not one interrupt aliasing issue. ... > What I/O APIC chipset and stepping does Doug have on that motherboard? ... work at all on 4.x is that the interrupts on the second and third APICs ... - uhci ithread scheduled ...
    (freebsd-current)