sendto: No buffer space available

From: Haesu (haesu_at_towardex.com)
Date: 07/30/03

  • Next message: Schalk Erasmus: "Re: DVD question." 
    Date: Wed, 30 Jul 2003 12:32:02 -0400
To: freebsd-isp@freebsd.org, freebsd-questions@freebsd.org

    Hello,

    We have a FreeBSD box here that we use to route some GRE tunnels and ipv6 gif tunnels. We use zebra for dynamic routing running zebra, bgpd, ospfd, and ospf6d.

    We have about 12 FreeBSD boxes with exact same configuration, the only
    difference is just the IP address of each interface.

    None of them fail but this one box...

    Everyday, this box stops all networking. I can still console in and stuff.. When
    I typed 'ping 127.0.0.1' at the console after networking locked up, it says:
    ping: sendto: No buffer space available

    The only solution seems to be rebooting it everyday... It happens every 12 hours
    or so...

    This is not related with mbuf, etc either, as netstat -m doesn't show any
    issues.

    The box has one IP address and IPv6 address in addition to 127.0.0.1 on lo0
    interface. It also has a ds0 interface with 10.5.5.5/30 assigned to ds0.
    This is exact same configuration on all other boxes, and none of them fail but
    this one.

    I've swapped out NICs with different vendors 3 times (tried, xl, dc, and now rl)

    I've also swapped out the whole box, and also swapped out the whole hard drive
    and did full reinstall. And problem still persists and it's definately not
    hardware as I swapped everything out... (unless the 3 NIC vendors above are all
    exhibiting same issue)

    I tried to look on Google but nothing useful that corelates to this particular
    issue..

    Any help would be very appreciated :)

    Thanks,
    -hc

    The box is running FreeBSD 4.8-STABLE
    FreeBSD necsis 4.8-STABLE FreeBSD 4.8-STABLE #0: Tue Jul 29 13:10:11 GMT 2003 root@necsis:/usr/src/sys/compile/router i386

    Following is output of netstat -s AFTER the networking locks up with no buffer
    space available error:

    tcp:
            30115 packets sent
                    17167 data packets (1232152 bytes)
                    301 data packets (54320 bytes) retransmitted
                    0 resends initiated by MTU discovery
                    12416 ack-only packets (10931 delayed)
                    0 URG only packets
                    0 window probe packets
                    41 window update packets
                    280 control packets
            28010 packets received
                    16762 acks (for 1236693 bytes)
                    140 duplicate acks
                    0 acks for unsent data
                    13205 packets (567038 bytes) received in-sequence
                    43 completely duplicate packets (818 bytes)
                    0 old duplicate packets
                    2 packets with some dup. data (38 bytes duped)
                    9 out-of-order packets (240 bytes)
                    0 packets (0 bytes) of data after window
                    0 window probes
                    31 window update packets
                    0 packets received after close
                    0 discarded for bad checksums
                    0 discarded for bad header offset fields
                    0 discarded because packet too short
            252 connection requests
            18 connection accepts
            6 bad connection attempts
            0 listen queue overflows
            30 connections established (including accepts)
            288 connections closed (including 10 drops)
                    23 connections updated cached RTT on close
                    23 connections updated cached RTT variance on close
                    11 connections updated cached ssthresh on close
            164 embryonic connections dropped
            16643 segments updated rtt (of 16929 attempts)
            1566 retransmit timeouts
                    10 connections dropped by rexmit timeout
            0 persist timeouts
                    0 connections dropped by persist timeout
            161 keepalive timeouts
                    0 keepalive probes sent
                    161 connections dropped by keepalive
            96 correct ACK header predictions
            10392 correct data packet header predictions
            19 syncache entries added
                    6 retransmitted
                    2 dupsyn
                    0 dropped
                    18 completed
                    0 bucket overflow
                    0 cache overflow
                    0 reset
                    0 stale
                    0 aborted
                    0 badack
                    1 unreach
                    0 zone failures
            0 cookies sent
            0 cookies received
    udp:
            196 datagrams received
            0 with incomplete header
            0 with bad data length field
            0 with bad checksum
            1 with no checksum
            61 dropped due to no socket
            3 broadcast/multicast datagrams dropped due to no socket
            0 dropped due to full socket buffers
            0 not for hashed pcb
            132 delivered
            132 datagrams output
    ip:
            2154646 total packets received
            0 bad header checksums
            0 with size smaller than minimum
            0 with data size < data length
            0 with ip length > max ip packet size
            0 with header length < data size
            0 with data length < header length
            0 with bad options
            0 with incorrect version number
            366 fragments received
            0 fragments dropped (dup or out of space)
            0 fragments dropped after timeout
            183 packets reassembled ok
            1118217 packets for this host
            11 packets for unknown/unsupported protocol
            1031566 packets forwarded (0 packets fast forwarded)
            265 packets not forwardable
            3573 packets received for unknown multicast group
            0 redirects sent
            772758 packets sent from this host
            13697 packets sent with fabricated ip header
            0 output packets dropped due to no bufs, etc.
            0 output packets discarded due to no route
            0 output datagrams fragmented
            0 fragments created
            366 datagrams that can't be fragmented
            0 tunneling packets that can't find gif
            0 datagrams with bad address in header
    icmp:
            1075 calls to icmp_error
            0 errors not generated 'cuz old message was icmp
            Output histogram:
                    echo reply: 28
                    destination unreachable: 244
                    time exceeded: 831
            0 messages with bad code fields
            0 messages < minimum length
            0 bad checksums
            0 messages with bad length
            0 multicast echo requests ignored
            0 multicast timestamp requests ignored
            Input histogram:
                    destination unreachable: 21
                    echo: 28
                    time exceeded: 111
            28 message responses generated
            1 invalid return address
            1 no return route
            ICMP address mask responses are disabled
    igmp:
            0 messages received
            0 messages received with too few bytes
            0 messages received with bad checksum
            0 membership queries received
            0 membership queries received with invalid field(s)
            0 membership reports received
            0 membership reports received with invalid field(s)
            0 membership reports received for groups to which we belong
            9 membership reports sent
    ip6:
            26952 total packets received
            0 with size smaller than minimum
            0 with data size < data length
            0 with bad options
            0 with incorrect version number
            0 fragments received
            0 fragments dropped (dup or out of space)
            0 fragments dropped after timeout
            0 fragments that exceeded limit
            0 packets reassembled ok
            20061 packets for this host
            35 packets forwarded
            108 packets not forwardable
            0 redirects sent
            29834 packets sent from this host
            0 packets sent with fabricated ip header
            0 output packets dropped due to no bufs, etc.
            82 output packets discarded due to no route
            0 output datagrams fragmented
            0 fragments created
            0 datagrams that can't be fragmented
            0 packets that violated scope rules
            108 multicast packets which we don't join
            Input histogram:
                    TCP: 12122
                    UDP: 12
                    ICMP6: 6761
                    OSPF: 8057
            Mbuf statistics:
                    26308 one mbuf
                    644 one ext mbuf
                    0 two or more ext mbuf
            0 packets whose headers are not continuous
            0 tunneling packets that can't find gif
            0 packets discarded due to too may headers
            0 failures of source address selection
            source addresses on an outgoing I/F
                    4 link-locals
                    44 globals
            source addresses of same scope
                    4 link-locals
                    44 globals
            9346 forward cache hit
            10034 forward cache miss
    icmp6:
            0 calls to icmp_error
            0 errors not generated because old message was icmp error or so
            0 errors not generated because rate limitation
            Output histogram:
                    multicast listener report: 49
                    multicast listener done: 4
                    neighbor solicitation: 4265
                    neighbor advertisement: 3239
            0 messages with bad code fields
            0 messages < minimum length
            0 bad checksums
            0 messages with bad length
            Input histogram:
                    unreach: 3
                    neighbor solicitation: 3239
                    neighbor advertisement: 3506
            Histogram of error messages to be generated:
                    0 no route
                    0 administratively prohibited
                    0 beyond scope
                    0 address unreachable
                    0 port unreachable
                    0 packet too big
                    0 time exceed transit
                    0 time exceed reassembly
                    0 erroneous header field
                    0 unrecognized next header
                    0 unrecognized option
                    0 redirect
                    0 unknown
            0 message responses generated
            0 messages with too many ND options
            0 messages with bad ND options
            0 bad neighbor solicitation messages
            0 bad neighbor advertisement messages
            0 bad router solicitation messages
            0 bad router advertisement messages
            0 bad redirect messages
            0 path MTU changes 

    -- 
Sincerely,
  Haesu C.
  TowardEX Technologies, Inc.
  WWW: http://www.towardex.com
  E-mail: haesu@towardex.com
  Cell: (978) 394-2867
_______________________________________________
freebsd-isp@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-isp
To unsubscribe, send any mail to "freebsd-isp-unsubscribe@freebsd.org"
  • Next message: Schalk Erasmus: "Re: DVD question."