Re: network performance



Stefan Lambrev wrote:
Hello,

Kris Kennaway wrote:
Stefan Lambrev wrote:

Thanks for investigating this. One thing to note is that ip flows from
the same connection always go down the same interface, this is because
Ethernet is not allowed to reorder frames. The hash uses
src-mac, dst-mac, src-ip and dst-ip (see lagg_hashmbuf), make sure when
performance testing that your traffic varies in these values. Adding
tcp/udp ports to the hashing may help.
The traffic, that I generate is with random/spoofed src part, so it is split between interfaces for sure :)

Here you can find results when under load from hwpmc and lock_profiling:
http://89.186.204.158/lock_profiling-lagg.txt

OK, this shows the following major problems:

39 22375065 1500649 5690741 3 0 119007 712359 /usr/src/sys/net/route.c:147 (sleep mutex:radix node head)
21 3012732 1905704 1896914 1 1 14102 496427 /usr/src/sys/netinet/ip_output.c:594 (sleep mutex:rtentry)
22 120 2073128 47 2 44109 0 3 /usr/src/sys/modules/if_lagg/../../net/ieee8023ad_lacp.c:503 (rw:if_lagg rwlock)
39 17857439 4262576 5690740 3 0 95072 1484738 /usr/src/sys/net/route.c:197 (sleep mutex:rtentry)

It looks like the if_lagg one has been fixed already in 8.0, it could probably be backported but requires some other infrastructure that might not be in 7.0.

The others are to do with concurrent transmission of packets (it is doing silly things with route lookups). kmacy has a WIP that fixes this. If you are interested in testing an 8.0 kernel with the fixes let me know.
Well those servers are only for tests so I can test everything, but at some point I'll have to make final decision what to use in production :)

http://www.freebsd.org/~kris/p4-net.tbz is a sys/ tarball from my p4 branch, which includes these and other optimizations.

http://89.186.204.158/lagg-gprof.txt

http://89.186.204.158/lagg2-gprof.txt I forget this file :)

I found that MD5Transform aways uses ~14% (with rx/txcsum enabled or disabled).

Yeah, these don't have anything to do with MD5.
Well I didn't find from where MD5Transform() is called, so I guess it's a some 'magic', that I still do not understand ;)

MD5Transform is an internal function called by other MD5* functions. Check netinet/tcp_syncache.c

It is probably from the syncache. You could disable it (net.inet.tcp.syncookies_only) if you don't need strong protection against SYN flooding.

Kris
How the server perform during SYN flooding is exactly what I test at the moment :)
So I can't disable this.

I thought this trace was on the machine you are transmitting the SYNs from, perhaps I misunderstood.

Just for information, if someone is interested - I looked how linux (2.6.22-14-generic ubuntu) perform in the same situation .. by default it doesn't perform at all - it hardly replays to 100-200 packets/s,
with syncookies enabled it can handle up to 70-90,000 pps (250-270,000 compared to freebsd), but the server is very loaded and not very responsible.
Of course this doesn't mean that FreeBSD can't perform better ;)

What do you mean "compared to freebsd"?

Kris

_______________________________________________
freebsd-performance@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@xxxxxxxxxxx"



Relevant Pages

  • Re: network performance
    ... the same connection always go down the same interface, ... You could disable it if you don't need strong protection against SYN flooding. ... with syncookies enabled it can handle up to 70-90,000 pps (250-270,000 compared to freebsd), but the server is very loaded and not very responsible. ...
    (freebsd-performance)
  • Re: Question: Why aint I getting gigabit speed?
    ... I have an re interface on my FreeBSD router and it connects at 1000baseT = ... In the case of connecting direct to my other server, ... RELEASE with an on-motherboard Nvidia gigabit ethernet interface, ...
    (freebsd-net)
  • Re: What doesnt lend itself to OO?
    ... The whole idea that a subsystem is just ... > The first line exists in the server. ... objects between client and server i.e. as far as the client code is ... > external interface is the traditional input interface whose ...
    (comp.object)
  • Re: More Get-IPlayer Questions
    ... to use with mutt mail client. ... antinat - 0.90-4 - Antinat is a flexible SOCKS server and client ... protocol for Sybase or MS SQL Server. ... ifstat - 1.1-1 - InterFace STATistics Monitoring ...
    (uk.comp.os.linux)
  • Re: Brian Kernighan, maybe Im not worthy, maybe Im scum
    ... If you think I'm giving a verbose pseud write access to my server, ... interface appears in C Sharp as consisting of sbyte arrays. ... failed to see that a for loop is not just a while loop because a while ... chilling effect on freedom of speech. ...
    (comp.programming)