Re: Abnormal network errors?

From: adp (dap99_at_i-55.com)
Date: 05/05/04

  • Next message: adp: "Re: Abnormal network errors?"
    To: "Charles Swiger" <cswiger@mac.com>
    Date: Wed, 5 May 2004 16:34:28 -0500
    
    

    ----- Original Message -----
    From: "Charles Swiger" <cswiger@mac.com>

    > On May 5, 2004, at 2:27 PM, adp wrote:
    > > On this server I'm thinking I need two things:
    > >
    > > 1. More sockets available.
    > > 2. Larger sockbufs for send and recv.
    > >
    > > Is this an accurate assessment?
    >
    > Given the application of this system, you might want to up the value of
    > kern.ipc.nmbclusters by a factor of four or so (it's NBMCLUSTERS in the
    > kernel config file). However, it's not essential-- your "netstat -m"
    > is OK, and your TCP send and receive windows are reasonably sized as-is
    > by default.

    Several problems. First, we are hosting a DNS server on this box. The DNS
    resolves domains we are auth. for very fast, or anything in its cache very
    fast, but anything else is SLOW or times-out. Also, our www server (another
    box) is responding slowly in general (4-6 seconds).

    > > What is "2432320 packets for unknown/unsupported protocol"? What
    > > specifically does this mean? (In other words, what should I do to
    > > resolve
    > > this?)
    >
    > It means machines are sending non-IP traffic on your network, which is
    > normal if you have Windows protocols (NetBEUI, SPX/IPX) or Macs
    > (AppleTalk) around. Or chatty network devices like some printers....

    What is 802.1d? I am getting a lot of this:

    16:21:16.788617 802.1d config 8000.00:04:27:d1:cb:d3.8019 root
    8000.00:03:6c:51:a2:a7 pathcost 8 age 2 max 20 hello 2 fdelay 15

    And this:

    16:21:15.424508 CDP v2, ttl=180s DevID 'Six2' Addr (1): IPv4 10.2.254.62
    PortID 'FastEthernet0/12' CAP 0x0a[|cdp]

    I'm at a colo.

    > See /usr/include/net/ethernet.h for an idea, or maybe "tcpdump not ip"
    > might give some idea of what's going by.
    >
    > > What about "921363 calls to icmp_error"?
    >
    > ICMP messages like responding to a ping, or people sending traffic with
    > RFC-1918 unroutable addresses (gives "dest unreachable")...

    That's weird. I tried 'tcpdump icmp' and see a few errors right off the bat:

    # tcpdump -n icmp
    tcpdump: listening on rl0
    16:27:46.633262 192.168.42.71 > 192.168.42.76: icmp: 192.168.42.71 udp port
    1249 unreachable
    16:27:53.639237 192.168.42.71 > 192.168.42.76: icmp: 192.168.42.71 udp port
    1280 unreachable
    16:28:02.579417 192.168.42.71 > 192.168.42.76: icmp: 192.168.42.71 udp port
    1204 unreachable
    16:28:07.716527 192.168.42.71 > 192.168.42.76: icmp: 192.168.42.71 udp port
    1510 unreachable
    16:28:08.589910 192.168.42.71 > 192.168.42.76: icmp: 192.168.42.71 udp port
    1218 unreachable
    16:28:15.668697 192.168.42.71 > 192.168.42.76: icmp: 192.168.42.71 udp port
    1327 unreachable
    16:28:33.581427 192.168.42.71 > 192.168.42.76: icmp: 192.168.42.71 udp port
    1355 unreachable

    Hmm. I am running a DNS server in a jail on the NFS server. We have been
    getting very slow responses times from it. Seems related.

    > > Under tcp I have "481930 embryonic connections dropped". I assume that
    > > means
    > > I don't have enough sockets available for when this server gets loaded.
    > > Correct?
    >
    > More likely, these are someone doing a port scan and leaving half-open
    > connections lying around to get cleaned up.

    We are behind a managed NetScreen firewall. I can't see how anyone is
    port-scanning us, unless they are just scanning the few ports we have open
    to the world.

    > It might be helpful if you gave us some idea as to what the performance
    > problem you were seeing was? Is NFS access slow, or some such? Are
    > you seeing errors or collisions in netstat -i or in whatever statistics
    > your switch keeps per port?
    >
    > The following areas struck me as being relevant:
    >
    > > # ifconfig rl0
    >
    > First, consider upgrading to a fxp or dc-based NIC.

    Noted.

    > > udp:
    > > 272987897 datagrams received
    > > [ ... ]
    > > 19976574 dropped due to full socket buffers
    >
    > This is high enough to represent a concern, agreed.

    How do I fix this then? I assume I don't have enough sockets available. Is
    there a way to see where I am peaking?

    I'm thinking adding more memory will increase the system-set defaults (also
    read 'man tuning').

    > > ip:
    > > 578001924 total packets received
    > [ ... ]
    > > 4899083 fragments received
    > > 4 fragments dropped (dup or out of space)
    > > 750 fragments dropped after timeout
    > > 842689 packets reassembled ok
    > [ ... ]
    > > 609745425 packets sent from this host
    > > 1914687 output datagrams fragmented
    > > 10496350 fragments created
    >
    > Second, you're fragmenting a relatively large number of packets going
    > by, you ought to see what's going on with your MTU and pMTU discovery.
    > I suppose if you're using large UDP datagrams with NFS, that might be
    > it.
    >
    > [ The machines I've got around with comparible traffic volume might
    > have 400 frags received, and 10 transmitted, or some such. ]

    Could this be related to my switch or anything else?

    _______________________________________________
    freebsd-performance@freebsd.org mailing list
    http://lists.freebsd.org/mailman/listinfo/freebsd-performance
    To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org"


  • Next message: adp: "Re: Abnormal network errors?"

    Relevant Pages

    • V210 BGE0@1000FDX (Adam Tomkinson)
      ... sunmanagers Digest, Vol 31, Issue 28 ... When connecting a server to a Gig interface you need to enable autoneg ... Blocked port after process kill ... NFS oddity ...
      (SunManagers)
    • V210 BGE0@1000FDX
      ... When connecting a server to a Gig interface you need to enable autoneg ... Blocked port after process kill ... NFS oddity ... where hostname is the name of the NFS client which will automount the ...
      (SunManagers)
    • Re: Abnormal network errors?
      ... we are hosting a DNS server on this box. ... I tried 'tcpdump icmp' and see a few errors right off the bat: ... I am running a DNS server in a jail on the NFS server. ... these are someone doing a port scan and leaving half-open ...
      (freebsd-questions)
    • Re: FC8 and NFS service
      ... The NFS server is doing some kind of evil security check which was ... Since the port works against the FC1 server, ... Changing clients isn't going to happen, ...
      (Fedora)
    • RC1 dump hangs - sometimes
      ... nfs send error 35 for server backup:/back ... K8-class CPU) ... port may not be enabled ...
      (freebsd-current)