Re: strange performance receiving with ixgbe/82599

On Mon, 2011-05-23 at 19:20 +0200, Luigi Rizzo wrote:
Jack,
i am doing some experiments with a modified ixgbe driver, and i am
seeing some strange performance numbers which I'd like to explain.

My card uses the 82599 chip, monted on an x16 slot on an Asus
motherboard. CPU is i7-870 @ 2.93GHz. Both -head and RELENG_8
exhibit the same behaviour. I am using a modified driver which
essentially accesses the rings from userspace, sending or
receiving frames as soon as there is room.

On the transmit side, using 64 byte packets (60+4 CRC) i can saturate
the link (measured 14.87-14.88 Mpps) with no problem with just one
core at 1.33 GHz and 2 or 4 queues (one queue tops at 12Mpps,
but that's not a big issue).

On the receive side, i can sustain 14.2Mpps (on 2-4 queues) if the
sender generates 68-byte packets (i.e. 64+4CRC), but as soon as i
go to 63+4 or less, the receive rate drops to 11.5-11.6Mpps.
CPU cycles abound, because the speed is unchanged down to 1.45GHz

Any reason to explain that suddend drop with short (but legal)
packet sizes ?
[...]

With 64-bit wide RAM, a DMA write of 64 bytes with 8-byte alignment
covers exactly 8 memory words. However a DMA write of 63 bytes with
8-byte alignment at the start covers 7 full and one partial memory word.
So a read-modify-write operation is required for the last word.

Ben.

--
Ben Hutchings, Senior Software Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

_______________________________________________
freebsd-net@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscribe@xxxxxxxxxxx"