Re: Network card firmaware upgrade....
- From: The Grand Poobah <grandpoobah@xxxxxxxxxxxxxxx>
- Date: Thu, 07 Aug 2008 20:44:01 -0500
Well, allegedly, in article <64b6dc99-1737-43d8-b941-4e9a32e050be@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>, chriseas@xxxxxxxxxxxxxx <chriseas@xxxxxxxxxxxxxx> wrote:
I just spend several hours this morning trying to upgrade the firmware
on our troublsome 10Gig ethernet card.
Hi - yeah, not much fun, huh? I have the quad 1 Gig ethernet card, but
similar or same firmware, and similar procedure.
I dropper to the OpenBoot prom and was trying to follow the
instructions but was failing at the first step.
It said to do this:
{0} ok load-base dload NXGE_OBP.FC
Boot device: /pci@780/pci@0/pci@1/network@0:,NXGE_OBP.FC File and
args:
100 Mbps full duplex Link up
Requesting Internet Address for 0:3:ba:d8:e9:68
Requesting Internet Address for 0:3:ba:d8:e9:68
Requesting Internet Address for 0:3:ba:d8:e9:68
But that was not working for me - rather stupidly I did not write down
the exact errors. Ideally I would to read NXGE_OBP.FC from the local
boot disk and not from a tftp server.
You *MUST* find a way to make TFTP work. So you need to debug the
errors, probably with snoop on the TFTP server (assuming Solaris). But
if it's Linux, try tcpdump or tshark. If Windows, try Wireshark. Etc.
You need TWO components for this to work:
1) rarpd daemon that knows how to map 0:3:ba:d8:e9:68 to an IP
On Solaris, vi /etc/ethers and add line such as:
0:3:ba:d8:e9:68 temp-ip
Then add to /etc/hosts:
192.168.0.33 temp-ip
(Or whatever IP that is reachable by both the TFTP server and
the machine with the card)
Then start rarpd. On Solaris: /usr/sbin/in.rarpd -d -a
(Also start snoop on the appropriate network interface.)
You do not need to use Solaris for rarpd; I used a Linux host
for in.tftpd and in.rarpd, and worked fine.
On Solaris 10, if tcp wrappers is enabled (see 'inetadm -p'),
then you may need to temporarily add in.rarpd and in.tftpd
entries with the desired client IP to /etc/hosts.allow.
2) tftpd daemon that can serve the file via TFTP.
On Solaris 10: add an appropriate tftp line in /etc/inetd.conf like:
tftp dgram udp6 wait root /usr/sbin/in.tftpd in.tftpd -s /tftpboot
Then 'inetconv' then 'svcadm enable tftp/udp6'. On older Solaris,
just have it in inetd.conf then 'pkill -HUP inetd' to enable.
But you can also use Linux, Windows, MacOS X, etc. as the TFTP
server if you have a tftpd daemon on it.
Put the NXGE_OBP.FC and QGCLP3.12 (or whatever appropriate file)
in /tftpboot or /tftproot (or whatever directory is appropriate
for your TFTP server. Solaris in.tftpd defaults to /tftpboot,
most Linux defaults to /tftpboot but some might default to /tftproot.
Windows, depends on whatever the software's default location is
unless you change it.
Don't forget to disable rarpd/tftpd afterwards unlees you need
both running full time.
Based on your messages, you are probably still stuck at the rarpd step.
You need to set up a rarpd daemon somewhere on the same subnet or VLAN.
(And then likewise for a TFTP server.)
It does not matter what OS your rarpd and tftp server is. It just has to
be on the same subnet/VLAN as the client machine you're booting to flash
the nxge card. It could even be a laptop or another server (temporarily)
on the same network. Doesn't matter. I used Linux for this, worked fine.
Can anyone tell me either the command line I should use or what
variables I need to set to get it to read from the disk.
I don't know how to do that, and would not want to risk a bad flash by
using unsupported means. I would STRONGLY recommend sticking to the only
official Sun procedure.
Make the TFTP method work -- do whatever it takes. It only took me about
10 minutes to figure out issues because I was using rarpd's debug mode
and snoop/tshark to see the network traffic between the rarpd/tftp
server and the client doing the nxge firmware update.
The procedure (done on 4 hosts) I used was not too bad for my quad port
1Gb/sec nxge card after the first time:
- Set up rarpd server
- Set up tftp server
- Load the NXGE_OBP.FC and QGCLP3.12 files (for quad port 1 Gb/sec nxge)
on the TFTP server
- On T2000 console at OBP ok prompt, your procedure may vary a bit:
setenv auto-boot? false
load-base dload NXGE_OBP.FC
load-base 1 byte-load
load net:,QGCLP3.12
(above file is for my quad port 1Gb ethernet nxge card, may be different
for 10Gb nxge; check your upgrade documentation for right file to use)
0 15 4d ba 8b 22 set-mac-addr
(above is an example of actual MAC address printed on the card; you must
set it to whatever is printed on your card)
" /pci@7c0/pci@0/pci@8/network@0" begin-select-dev
(above is specific to my T2000)
load-base update
reset-all
show-nets
cd /pci@7c0/pci@0/pci@8/network@0
(above is specific to my T2000)
..properties
(if it shows the newer firmware then continue. If it does NOT, STOP and
figure out what went wrong -- don't continue past here!)
device-end
setenv auto-boot? true
boot
Follow your upgrade documentation for exact procedure for your card precisely.
I sure wish Sun had written some kind of tool to load this via the OS!
My colleagues were NOT impressed with this quirky procedure (nor was I),
but unfortunately, it is all we have for now.
I also recommend you have the latest nxge driver loaded and a reasonably
current system/OBP firmware. For my T2000, I had to load a newer nxge
driver before doing this flashing so that post-installation with the new
driver, it wouldn't crash at boot. Not all Solaris 10 OS versions needs
the newer driver version, so check the matrix carefully to determine if
this applies in your case.
Good luck. It's worth it; I was able to, with some tuning (including
enabling jumbo frames on the routers and switch ports involved) and
firmware upgrades, to get up to 981 Mbps out of nxge0 for a 1 Gbps port
over a 3,700 mile high speed WAN -- measured with iperf with appropriate
parameters. nxge is a pretty good performing card and driver when
appropriately and fully tuned.
Please let us know (or just send me email) when you have this working.
Thanks!!
P.S. I suggest you file an official RFE request via Sun Support; enough
RFE requests may see the nxge guys reconsider and develop a disk-based
method to do it for future firmware upgrades. I was the unlucky person
stuck with doing this upgrade on 4 machines here. :-) Don't want to do
it again; it's enough of a hassle in my very locked down environment.
P.P.S. Sun support told us that Sun's TCP/IP stack is tuned for LAN
performance. We had to work a bit to get it doing well with a WAN. But
if you're doing it for a LAN, you'll have fewer stuff needed to tune.
.
- Follow-Ups:
- Re: Network card firmaware upgrade....
- From: chriseas
- Re: Network card firmaware upgrade....
- References:
- Network card firmaware upgrade....
- From: chriseas
- Network card firmaware upgrade....
- Prev by Date: Re: S/N for T2000 from OS?
- Next by Date: Re: Solaris Mem consumption
- Previous by thread: Network card firmaware upgrade....
- Next by thread: Re: Network card firmaware upgrade....
- Index(es):
Relevant Pages
|