Re: strange memory leak

hetricktessa@xxxxxxxxx wrote:

On Sep 29, 4:15 am, Mike <m...@xxxxxxxxx> wrote:
dickd wrote:
I agree with Jim. This is not something to be concerned with (yet).
Every file you touch in AIX is cached in the real memory paging.
If you need access to the file again, it will be very quick.
If memory demands increase, and your applications need more memory
these filesystem pages will be quickly harvested for the new storage
requests.

This is normal operation.

My primary tool to decide whether I have a memory leak is "svmon".
It requires root privilege to use it ... but honestly, I don't see
why,
so I always change the permissions so the "little people" can use it.

chmod 6555 /usr/sbin/svmon

The first tool I reach for is "vmstat". Watch the paging columns.
They are a quick and easy way to decide whether you have a paging
problem.

Note: I have seen customers observe this AIX behaviour, and decide
to go out and buy extra real memory, only to have that fill too.
It will always fill on AIX to the extent that there are additional
filesystem blocks that have been touched.

This behaviour is controllable in the "tuneables" area of AIX,
but you really REALLY have to have a good reason to change it.
A good reason might be ... substantially less memory than 8GB.

The other tuning exercise that might cause this is to over-tune
your database and other applications to provide memory caches
that are substantially larger than available real memory.

Of course you're going to fill them ... and then the system will
start paging, and additional real memory will help.
But so will tuning-down the memory cache behaviour of the
applications.

Funny story that is related to your customers buying memory. I won't
tell you what multinational company I work for but lets call it the
Institute of Black Magic. I once was called into an after hours call by
on of the many managers we have. Mr Customer was complaining that memory
was full and they wanted to double it. pi,po were 0. Paging space was 0.
vmstat was 99 % idle. fr and sr on vmsat were 0. I almost laughed until
I realized they were quite serious. It took almost 4 hours to convince
the manger he was wasting the customers money and then a 2 page email
describingin memory management in AIX to make them realize they were
full of ***.

I miss the old days when the only thing between you and your server was
a keyboard not the BS we have today.

HAHAHahahha!!! thats got to be a pretty common scenario, given the
incredibly obtuse and convoluted methods AIX provides for answering
such a SIMPLE question: "how much memory is the system using? does it
need a memory upgrade?"
to make matters worse in this particular case, the pSeries boxes in
question are the first unix boxes this co. has EVER had... never even
had linux or anything... NO experience w/any kind of unix at all...
strictly a windows shop up until now, and they only have a couple of
windows admins, who, of course, are used to seeing very simple memory
usage statistics, a la Windows Task Manager -> Performance tab.
trying to explain to these guys (and even worse, their managers!)
that simply checking memory usage on an AIX server requires a PhD in
CS (my brain still hurts from trying to understand it all myself, and
I've been doing unix (Solaris) for years) is not either simple or
convincing.
bottom line seems to be that there IS NO simple way to check mem.
usage, without having a firm grasp of the concepts of AIX memory
management internals (paging space.. the diff. b/w page-ins, page-
outs, paging space page-ins, paging-space page-outs, virtual memory,
page faults, pins, steals, working/consistent/user pages, frames,
buffers, filesytem caches, computational memory (i.e. process memory
(data, stack, and heap) VS. kernel memory and shared memory, etc.
etc... ) as well as understanding the cryptic output of svmon, vmstat,
ps, and some tools you end up having to install from perf.tools, and
of course the multiplicity of various flags and arguments to those
commands, which tend to have man-page explanations such as:
"%MEM
Calculated as the sum of the number of working segment and code
segment pages in memory times 4 (that is, the RSS value), divided by
the size of the real memory of the machine in KB, times 100, rounded
to the nearest full percentage point. "

so... yeah. trying to explain all that *** to somebody who's used to
just hitting "ctrl-alt-delete"-> Task Mgr. to get an answer, in about
2 seconds, that they don't have to calculate or RESEARCH... is...
well... IT SUCKS!!! seriously.

oh well.. if it was easy, i guess we'd all be out of a job. so,
thanks for "keeping it real", 'International Black Magic'!!!!!

I'm accustomed to something even more straight-forward (very small Alpha,
hobbyist machine):

$ show memory
System Memory Resources on 2-OCT-2007 21:57:38.86

Physical Memory Usage (pages): Total Free In Use Modified
Main Memory (256.00MB) 32768 23658 8340 770

Extended File Cache (Time of last reset: 24-SEP-2007 22:07:52.18)
Allocated (MBytes) 28.37 Maximum size (MBytes) 128.00
Free (MBytes) 0.71 Minimum size (MBytes) 3.12
In use (MBytes) 27.66 Percentage Read I/Os 56%
Read hit rate 66% Write hit rate 0%
Read I/O count 8643 Write I/O count 6605
Read hit count 5776 Write hit count 0
Reads bypassing cache 14 Writes bypassing cache 0
Files cached open 251 Files cached closed 164
Vols in Full XFC mode 0 Vols in VIOC Compatible mode 5
Vols in No Caching mode 0 Vols in Perm. No Caching mode 0

Granularity Hint Regions (pages): Total Free In Use Released
Execlet code region 1024 0 772 252
Execlet data region 256 0 256 0
S0/S1 Executive data region 260 0 260 0
Resident image code region 1024 0 471 553

Slot Usage (slots): Total Free Resident Swapped
Process Entry Slots 62 43 19 0
Balance Set Slots 60 43 17 0

Dynamic Memory Usage: Total Free In Use Largest
Nonpaged Dynamic Memory (MB) 1.97 0.84 1.13 0.75
Paged Dynamic Memory (MB) 1.14 0.63 0.50 0.63
Lock Manager Dyn Memory (KB) 496.00 320.50 175.50

Buffer Object Usage (pages): In Use Peak
32-bit System Space Windows (S0/S1) 0 0
64-bit System Space Windows (S2) 0 0
Physical pages locked by buffer objects 0 0

Memory Reservations (pages): Group Reserved In Use Type
Total (0 bytes reserved) 0 0

Write Bitmap (WBM) Memory Summary
Local bitmap count: 0 Local bitmap memory usage (bytes) 0.00
Master bitmap count: 0 Master bitmap memory usage (bytes) 0.00

Swap File Usage (8KB pages): Index Free Size
DISK$ALPHASYS:[SYS0.SYSEXE]SWAPFILE.SYS
1 488 488

Paging File Usage (8KB pages): Index Free Size
DISK$ALPHASYS:[SYS0.SYSEXE]PAGEFILE.SYS
254 10744 10744
Total committed paging file usage: 2204

Of the physical pages in use, 2895 pages are permanently allocated to OpenVMS.

Apologies if that wraps badly.

Wonder if there are any third-party products for taming the UN*X beast...

--
David J Dachtera
dba DJE Systems
http://www.djesys.com/

Unofficial OpenVMS Marketing Home Page
http://www.djesys.com/vms/market/

Unofficial Affordable OpenVMS Home Page:
http://www.djesys.com/vms/soho/

Unofficial OpenVMS-IA32 Home Page:
http://www.djesys.com/vms/ia32/

Unofficial OpenVMS Hobbyist Support Page:
http://www.djesys.com/vms/support/
.