Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp]

I can't reproduce the 580kpps maximum that I saw when I first compiled for some reason, I don't understand, the max I get even with ULE and preemption
is now about 530 and it dips to 480 a lot.. The first time I tried it it was at 580 and dipped to 520...what the?.. (kernel config attached at end)
* noticed that SOMETIMES the em0 taskq jumps around cpus and doesn't use 100% of any one cpu
* noticed that the netstat packets per second rate varies explicitly with the CPU usage of em0 taskq
(top output with ULE/PREEMPTION compiled in):
PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
10 root 171 ki31 0K 16K RUN 3 64:12 94.09% idle: cpu3
36 root -68 - 0K 16K CPU1 1 5:43 89.75% em0 taskq
13 root 171 ki31 0K 16K CPU0 0 63:21 87.30% idle: cpu0
12 root 171 ki31 0K 16K RUN 1 62:44 66.75% idle: cpu1
11 root 171 ki31 0K 16K CPU2 2 62:17 56.49% idle: cpu2
39 root -68 - 0K 16K - 0 0:54 10.64% em3 taskq

this is about 480-500kpps rate.........
now I wait a minute and

PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND
10 root 171 ki31 0K 16K CPU3 3 64:56 100.00% idle: cpu3
36 root -68 - 0K 16K CPU2 2 6:21 94.14% em0 taskq
13 root 171 ki31 0K 16K RUN 0 63:55 80.18% idle: cpu0
11 root 171 ki31 0K 16K RUN 2 62:48 67.38% idle: cpu2
12 root 171 ki31 0K 16K CPU1 1 63:04 58.40% idle: cpu1
39 root -68 - 0K 16K - 1 1:00 10.21% em3 taskq


530kpps rate.......


drops to 85%.. 480kpps rate
goes back up to 95% 530kpps

it keeps flopping like this...........

none of the CPUs are 100% use and none of the cpus add up , like the cpu time of em0 taskq is 94% so one of the cpus should be 6% idle but it's not.
This is with ULE/PREEMPTION.. I see different behavior without preemption and with 4bsd..
and I also see different behavior depending on the time of day lol :)
Figure that one out

I'll post back without preemption and with 4bsd in a min
then i'll move on to the 32 bit platform tests


_______________________________________________
freebsd-net@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@xxxxxxxxxxx"

Relevant Pages