[HPADM] Summary: vmunix: file: table is full & oracle client ORA-12500: TNS:listener failed to start a dedicated server process
- From: sunhux G <sunhux@xxxxxxxxx>
- Date: Sat, 30 May 2009 14:39:17 +0800
Pardon me for the late and sloppy summary. This is a great list :
Firstly a one liner script (which I've yet to test it but I'll probably
won't need it anymore as I've increased nfile to about 14000 on our
PA RISC B11.11. Found on our SAP Itanium B11.23 running
Oracle, we never have this issue as the settings closely
match what Oracle recommends (as given by Andy).
sar -v 2 29|awk '($8 ~
/[0-9][0-9]*\/[0-9][0-9]/){split($4,a,"/");split($8,b,"/");if((a[1]/a[2]>0.65)||(b[1]/b[2]>0.65)){print
"mail -salert you@somewhere"}}'|sh
(script above still lack checks like we don't want to be alerted every
minute when there's a problem otherwise our mailboxes will be
filled up as Bill Hassell rightly pointed out)
ORA-12500: TNS:listener failed to start a dedicated server process
Thus my colleague increased our nfile, nproc, maxusers values
by about 30% & the current values for each of the above
parameters are now: 3329, 1620 & 2000 respectively
Far, far too small!!! NEVER change a full parameter by less than 100%.
Otherwise, you continue to get errors like file table full or proc table
full.
NFILE should be changed to 5000, NPROC set to 2000. Monitor NPROC and NFILE
with: sar –v 1 5
MAXUSERS is *not* a kernel parameter, it is a junk value used in kernel
parameter formulas. When you set it to a high value (like 2000), then a
*LOT* of parameters are increased, many unnecessarily too high.
(my colleague did not want to increase more as he said this server has only
4Gb memory and he ever encounter a case on this server that if these are
increased too high, the server can't boot up).
Yes, you can increase parameters far too large for the kernel to use – for
very old versions of HP-UX. The MAXUSERS value is probably using a lot of
memory with large kernel parameters.
After that 30% increase, things have stabilized for about a month & today
that same Oracle client message resurfaced and in the HP-UX Oracle Server,
we got the following message in syslog.log :
May 21 10:25:00 hostname vmunix: file: table is full
NFILE is way too small. Never change a full parameter with a small increase..
Always double it. You are running enough processes to exceed NPROC and more
will be likely in the future.
Q1:
What's the cause of this problem, is it our nproc, nfile and maxusers got
exceeded again? or what's the cause of this? Users login to a Windows
webserver which
retrieves data from this HP-UX oracle server and when this error occur,
users can't retrieve data
The cause is that you have too many users that are running too many
processes. Either reduce the number of users or increase the parameters to
meet the requirements. If you want to see all the processes, use ps –ef.
Every process is the result of HP-UX, Oracle, applications and users running
programs. NOTE: one user does *NOT* equal one program.
Q2:
Are my colleague's concern unfounded (ie increase too high a value on
a 4Gb server can result in the server can't boot up)?
If you change NFILE to 5000000000 and NPROC to 30000, then yes, your kernel
may exceed the RAM available. Your colleague is correct only when
the increases are unreasonable.
=============================================================
Oracle recommends the following kernel parameters for HP-UX. Try the
following:
Parameter Recommended Formula or Value
ksi_alloc_max (nproc*8)
executable_stack 0
max_thread_proc 1024
maxdsiz 1073741824 (1 GB)
maxdsiz_64bit 2147483648 (2 GB)
maxssiz 134217728 (128 MB)
maxssiz_64bit 1073741824 (1 GB)
maxswapchunks 16384
maxuprc ((nproc*9)/10)
msgmap (msgtql+2)
msgmni (nproc)
msgseg 32767
msgtql (nproc)
ncsize (ninode+1024)
nfile (15*nproc+2048)
nflocks (nproc)
ninode (8*nproc+2048)
nkthread (((nproc*7)/4)+16)
nproc 4096
semmap (semmni+2)
semmni (nproc)
semmns (semmni*2)
semmnu (nproc-4)
semvmx 32767
shmmax The size of physical memory or 1073741824 (0X40000000),
whichever is greater.
Note: To avoid performance degradation, the value should
be greater than or equal to the size of the available
memory.
shmmni 512
shmseg 120
vps_ceiling 64
--Andy
=============================================================
Shmmax has a missing word below - it should NOT be equal to or greater
than the size of physical memory.
I'm quoting from the Oracle Installation docs at:
http://download.oracle.com/docs/cd/B19306_01/install.102/b19068/pre_install.htm
Usually it should be *greater* than the physical memory. Please
elaborate more on your remark.
--Andy
=============================================================
Shmmax has a missing word below - it should NOT be equal to or greater
than the size of physical memory.
However, I wouldn't adjust shmmax at all but rather focus on maxfiles
and maxfiles_lim.
Also if adjusting the kernel just made it took longer to get the same
error it may be you have some sort of runaway process that is eating
more and more resources over time. If so no matter what you set
parameters to you'll eventually reach them. It is much better to
examine the application to see if its file handles are growing. You can
use lsof for this purpose.
You might want to take a look at this thread on the ITRC.
http://forums13.itrc.hp.com/service/forums/questionanswer.do?admit=10944
7627+1242909697116+28353475&threadId=205347
=============================================================
Unless you have reason to believe that a misbehaving app will
go berserk with nonstop shared memory requests, there's no
reason not to set shmmax to be equal to physical RAM. It
will save you headaches down the line. Setting
shmmax = phys RAM is as common as cranking maxswapchunks
to the max. It's just something you normally do when you
stand up a new server. This isn't rocket science.
Jim Turner
=================================================
Right you are, Jim, SHMMAX should be EQUAL to physical RAM!
At least on HP-Unix for Itanium.
However, the error Pit Ong is most likely to receive as a
result of SHMMAX being too small is
ORA-04030: out of process memory
Jens Kieffer-Olsen
=================================================
================Original question follows ================
On Thu, May 21, 2009 at 3:31 PM, sunhux G <sunhux@xxxxxxxxx> wrote:
Hi,
More than 5 weeks ago, we ran into this error messages in Oracle client end
:
ORA-12500: TNS:listener failed to start a dedicated server process
Thus my colleague increased our nfile, nproc, maxusers values by about
30% &
the current values for each of the above parameters are now :
3329, 1620 & 2000 respectively
(my colleague did not want to increase more as he said this server has only
4Gb memory and he ever encounter a case on this server that if these are
increased too high, the server can't boot up).
After that 30% increase, things have stabilized for about a month & today
that same Oracle client message resurfaced and in the HP-UX Oracle Server,
we got the following message in syslog.log :
May 21 10:25:00 hostname vmunix: file: table is full
Q1:
What's the cause of this problem, is it our nproc, nfile and maxusers got
exceeded
again? or what's the cause of this? Users login to a Windows webserver
which
retrieves data from this HP-UX oracle server and when this error occur,
users can't
retrieve data
Q2:
Are my colleague's concern unfounded (ie increase too high a value on a 4Gb
server can result in the server can't boot up)?
Q3:
Below is a sar output after we reboot this HP-UX B11.11 server to give
interim
relief. I read somewhere I have to look out for file-sz in "sar" output.
Which of the
values below would give me a clue that nproc, nfile is running low or got
exceeded?
# sar -v 5 5
HP-UX hostname B.11.11 U 9000/800 05/21/09
10:04:37 text-sz ov proc-sz ov inod-sz ov file-sz ov
10:04:57 N/A N/A 577/1620 0 1867/1988 0 3315/3339 0
10:05:02 N/A N/A 578/1620 0 1869/1988 0 3323/3339 22
Thanks
Pit Ong
- Prev by Date: [HPADM] Re: RE: RE: RE: RE: Re: Alternative to VERITAS Cluster File System wanted
- Previous by thread: [HPADM] Alternative to VERITAS Cluster File System wanted
- Index(es):
Relevant Pages
|
