Re: Question about adding flags to mmap system call / NVIDIA amd64 driver implementation



On Tue, Apr 28, 2009 at 22:19, Julian Bangert <julidaoc@xxxxxxxxx> wrote:
Hello,

I am currently trying to work a bit on the remaining "missing feature" that
NVIDIA requires ( http://wiki.freebsd.org/NvidiaFeatureRequests  or a back
post in this ML) -  the improved mmap system call.
 For now, I am trying to extend the current system call and implementation
to add cache control ( the type of memory caching used) . This feature
inherently is very architecture specific- but it can lead to enormous
performance improvements for memmapped devices ( useful for drivers, etc). I
would do this at the user site by adding 3 flags to the mmap system call
(MEM_CACHE__ATTR1 to MEM_CACHE__ATTR3 ) which are a single octal digit
corresponding to the various caching options ( like Uncacheable,Write
Combining, etc... ) with the same numbers as the PAT_* macros from
i386/include/specialreg.h except that the value 0 ( PAT_UNCACHEABLE ) is
replaced with value 2 ( undefined), whereas value 0 ( all 3 flags cleared)
is assigned the meaning "feature not used, use default cache control".
For each cache behaviour there would of course also be a macro expanding to
the rigth combination of these flags for enhanced useability.

Hmm, I don't like that. What about using something like PAT_WC
directly for the userland? Afaik a userland app that uses stuff like
this is md anyway.

 The mmap system call would, if any of these flags are set, decode them and
get a corresponding PAT_* value, perform the mapping and then call into the
pmap module to modify the cache attributes for every page.

 My first question is if there is a more elegant way of solving that - the 3
flags would be architecture specific ( they could be used for other things
on other architectures though if need be ) and I do not know the policy on
architecture specific syscall flags, therefore I appreciate any input.

The second question goes to all those great VM/pmap gurus out there: As far
as I understand, at the moment the pmap_change_attr can only cange the cache
flags for kernel pages. Is there a particular reason why this function might
not be adapted/extended to userspace mappings? If not, I would either add a
new function to iterate over all pages and set cache flags for a particular
region or add a new member (possibly just add the 3 flags again ? ) to the
md part of vm_page_t. Or one could just keep track and return errors as soon
as someone tries to map a memory region ( cache-customized mapping is
usually done to device memory ) already mapped with  different cache
behaviour.

Do you know how other OS handle this stuff? Maybe there is some
inspiration there for a clean interface. I'm not sure if I remember
correctly but there is something in my mind that we must take care
that no virtual pages have different PAT settings for the same
physical page. Maybe I read something like this in the AMD's
documentation of PAT. Sorry I don't remember exactly but perhaps
someone else can explain it better.
_______________________________________________
freebsd-hackers@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@xxxxxxxxxxx"



Relevant Pages

  • Re: Question about adding flags to mmap system call / NVIDIA amd64 driver implementation
    ... I am currently trying to work a bit on the remaining "missing feature" that NVIDIA requires - the improved mmap system call. ... I am trying to extend the current system call and implementation to add cache control. ... I would do this at the user site by adding 3 flags to the mmap system call which are a single octal digit corresponding to the various caching options (like Uncacheable,Write Combining, etc... ...
    (freebsd-hackers)
  • Question about adding flags to mmap system call / NVIDIA amd64 driver implementation
    ... I am currently trying to work a bit on the remaining "missing feature" that NVIDIA requires - the improved mmap system call. ... I am trying to extend the current system call and implementation to add cache control. ... This feature inherently is very architecture specific- but it can lead to enormous performance improvements for memmapped devices ... For each cache behaviour there would of course also be a macro expanding to the rigth combination of these flags for enhanced useability. ...
    (freebsd-hackers)
  • Re: SMS Admin Tools You Gotta Have.
    ... Cache Cleanup feature added ... Disable SMS Components ... Show Client History ...
    (microsoft.public.sms.admin)
  • Re: [PATCHv3 1/2] virtio: support layout with avail ring before idx
    ... index, flags), and adds padding between index and flags. ... to extend avail control without incurring extra cache misses. ... * struct vring { ...
    (Linux-Kernel)
  • Re: [PATCH] virtio-blk: set QUEUE_ORDERED_DRAIN by default
    ... You'd normally have to add a feature for something like this. ... The second semantic is not useful for production, but is very useful for testing out things where you aren't worries about host crashes and you're usually rebooting the guest very often (you can't rely on guest caches, so you want the host to cache). ...
    (Linux-Kernel)