Re: Slow NFS Writes, Network Errors

From: Jason King (jasonking_at_sbcglobal.net)
Date: 06/19/05 

Date: Sat, 18 Jun 2005 22:10:38 GMT

Bryan.Berns@gmail.com wrote:
> Thanks to all who've responded.
>
> Apparently there's debate about whether to use ndd in a startup script
> or to edit ce.conf directly; I'll investigate more and decide for
> myself.
>
> Apparently there's this whole "don't rely on autonegotiate" thing
> between Sun's cards and Cisco switches so I'll probably just set them
> statically since not much will be added to this network for the next
> decade or so.
>
> That's right, I said DECADE --- that's how often stuff changes and is
> updated around here. Let me clarify by saying GOVERNMENT.
>
> Thanks Again,
>
> - Bryan
>

One thing to consider is there's a chance (though probably small) that
using the ce.conf file might get overwritten by patches, while a custom
rc script would not.

As to the actual duplex problem, are your network switches running
spanning tree on every port?? I seem to recall that such a setup can
mess up timings with fast ethernet autonegotiation (though maybe someone
more experienced in the area can comment/correct me). I believe the
usual fix is to just run spanning tree on trunking ports between
switches, and leave it off on ports connected to machines (though I
cannot remember the exact setting for cisco switches).

The disadvantage is that you will have to trust that no one's going to
hook up a dual-ported device and turn on bridging or otherwise cause a
loop. I would argue though that in a datacenter (at least), if such
things are a possibility, it would suggest there are other more pressing
policy problems that should be addressed.

Also, is the server reasonably up to date with patches (within the last
year - 18 months)? I recall some issues early on (in Solaris 8 at
least) with ce devices that seemed to be driver related, but were fixed
with some patches. This was quite a while ago (I want to say around the
time the V440s were first released), so anything somewhat recent should
eliminate that as a potential cause.

As an aside, I know there does still seem to be a tendency for network
admins to set switches to 100mb fdx, with autonegotiation disabled,
mostly because when fast ethernet was first introduced, not all cards
did it properly. However, that's supposed to have been fixed for quite
some time. The problem is, I believe the actual IEEE spec actually
requires a fallback to 100mb HDX when no autoneg is available, so it's
quite easy to end up with a duplex mismatch. This has caused myself a
large amount of cursing (among other thigs) when trying to maintain
various serengeti servers since our network admins were quite adamant
for a very long time about not turning on autoneg, even if it was only
on specific ports. It made monitoring them as well as doing firmware
upgrades painful to say the least.