Re: Virtual Address Range Prioritization
- From: info2007@xxxxxxxxx (Robert Krten)
- Date: Mon, 21 May 2007 21:39:50 -0500
Jason Bourne <j_bourne_treadstone@xxxxxxxxxxx> wrote:
Robert Krten wrote:
[snip]
There are two interpretations of the priority scheme; Hard and Soft.[snip]
The Soft priority scheme could operate much like FreeBSD's current
CPU scheduler; even though a process has a higher priority than another
process, it still doesn't get to hog 100% of the CPU. The Hard
priority scheme would be the opposite, and function more like the
scheduling on an RTOS; a higher priority process runs until it blocks
or until an even higher priority process is able to use the CPU,
potentially hogging 100% of the CPU.
Similarly with memory; a Soft priority memory region would be a
"suggestion" of importance, whereas a Hard priority memory region
would be an enforceable criteria.
Disclaimer: I'm not a kernel guy, so this may just be pie in the
sky dreaming. Any thoughts? Anything similar?
I'm a sysadmin, not either a kernel hacker nor hard core coder either. My
first impression of reading this was two things:
1. I think this is mostly what the scheduler does, albeit it is
CPU "centric"; but where the CPU goes, the memory must follow. Perhaps an
improvement to scheduling in general? Or would it be too much overhead,
context switching, etc, or some other form of performance hit which is why
it would be a waste of time to look into (and the kernel hacker types would
probably have already looked at something like this previous to this point
in time).
Hmmm... let's look at it another way. Suppose you have exactly two
applications running.
The first is what I'll call a data-centric app. It mmap()s a 10G
file into its VA space, and accesses a good chunk of it. Enough, in fact,
that it pages in (let's assume you have 2G of RAM) 2G of the file.
Now, 100% (or really close to that) of the RAM is full of the file.
Now this application goes to sleep (it's waiting for something to happen).
Meanwhile, the second application does a "find /" of the filesystem.
As I understand it, regardless of the priority of the second application,
the OS will use more and more of the RAM to cache filesystem data.
So, in my experience, the problem here is that I have no way to ask the OS,
"please keep 90% of my data in memory if you can" (Yes, there's the MADVISE
stuff, but that doesn't allow for prioritization; there are two options --
"I'm going to use this, don't throw it away", and "I don't want this any
more, kill it".)
With two processes, it's a little contrived; with multiple processes
competing for CPU and RAM, some kind of prioritization is desirable.
2. Most newer memory controllers in recent chipsets have some form
of "prefetch" capability that is constantly trying to guess where in memory
the CPU will go next and attempt to place the contents of ram into the CPU
cache. When there is a hit all is good, when a miss the CPU will stall a
few clock ticks to fetch from ram. Generally what I'm trying to get at is
some of this functionality is already covered by algorithms in hardware.
This is at too low of a level -- you're talking about things like instruction
prefetch and instruction flow prediction (and perhaps the data analogs of
these). These will apply for the next 8 bytes, or 64 bytes, or 256 bytes,
or a few k -- I'm talking about 3+ orders of magnitude difference :-)
The question I would have of such a scheme as trying to couple the scheduler
software into the hardware controller, or otherwise, may use more resources
and slow the OS down rather than providing a positive return with higher
performance because it is trying to "second guess" other already active
processes.
I don't know enough to really say anything substantive; just a couple of
impressions that floated up from the depths. :-)
Discussion is good, it helps me to clarify my points as well...
The "thing" being discussed here is the page replacement algorithm.
The part of the OS that decides which pages to throw out (by either
throwing them out, or swapping them to backing store).
Yes, this works in conjunction with the scheduler...
Cheers,
-RK
--
Robert Krten, Antique computer collector looking for PDP-8 and PDP-8/S
minicomputers; check out their "good home" at www.parse.com/~museum
.
- References:
- Virtual Address Range Prioritization
- From: Robert Krten
- Re: Virtual Address Range Prioritization
- From: Jason Bourne
- Virtual Address Range Prioritization
- Prev by Date: Re: Can I save dmesg of a boot CD?
- Next by Date: Re: Virtual Address Range Prioritization
- Previous by thread: Re: Virtual Address Range Prioritization
- Next by thread: Re: Virtual Address Range Prioritization
- Index(es):
Relevant Pages
|
