Re: [Q] giving priority to network interfaces?
From: Gavin Maltby (G_a_v_i_n.M_a_l_t_b_y_at_sun.com)
Date: 09/15/03
- Next message: Hovik Melikyan: "Re: detecting 64-bitness inside a program"
- Previous message: qyvind_Gjerstad?=: "Re: detecting 64-bitness inside a program"
- In reply to: Michal Szymanski: "[Q] giving priority to network interfaces?"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Date: Mon, 15 Sep 2003 09:40:51 +0100
Michal Szymanski wrote:
> Hi,
>
> Is it possible to give different priorities to two network interfaces on
> the same host running Solaris 8?
I don't think so, no.
> The problem is as follows: I have an Ultra 2 dual CPU Sparc equipped
> with two 100 Mbit ethernet cards. One interface connects it to the
> data acquisition computer, from which the Ultra gets data, and has to
> get it as fast as possible. The second interface connects the Ultra to
> the local network, on which there are hosts processing the data.
>
> The program receiving data from the "data" network runs in RT to get
> them AFAP.
Note that putting something into the RT class does not make it a realtime
application. It may be scheduled as such, but needs to be designed from
the ground up as an RT app. In particular, any RT application can deny
cpu time to the kernel especially if it goes into any tight processing
loops. It is usually suggested to place RT apps in a processor set of
their own, and shelter it from interrupts.
I've noticed, however, that when the "processing" host pulls
> data from the Ultra by NFS, it puts quite a big load on Ultra, so
> occasionally even the "data" transfer fails. I'm no expert but AFAIK,
> NFS server is (at least in great part) in kernel-space. That might
> explain why putting the "data server" in RT mode may be not enough to
> get all system resources when needed.
There are a nunmber of possibilities here.
In 5.8 NFS runs in the kernel in the SYS class at priority 60 (lowest
kernel priority). In 5.9 it runs in userland but still at priority 60
using the FX (fixed) scheduling class.
RT priorities are way above 60, so the pri 60 work should not
interfere with the RT. Not directly, anyway. But if your RT app
(a system call on it's behalf) wants a kernel lock currently held
by activity caused by the pri 60 NFS activity caused by your
processing clients then there will be a delay. Yes the RT
stuff wills it's priority onto the NFS threads, but it still has
to wait for the NFS operation to complete before the resource
is unlocked and it can proceed. This is an example of what I
say above - unless the app is designed for RT it may be
defeated in curious ways.
I suspect a way to avoid this is to have the incoming and processing
streams target different directories (they'll fight over fewer locks).
Have the data stream to private directories, and when all data is in
and ready to be processed rename the file into the processing
directories (within the same f/s this will be quick).
NFS also has some bias towards writers in a directory - if you are
writing a big file to a directory then other nfs operations to that
same dir lose out a bit. So if your clients are writing back to the
directory then they may be holding the RT app off a bit. Again,
try splitting into directories as above. This behaviour is less
pronounced in 5.9 and eliminated thereafter.
Gavin
> I fixed the problem by changing the way the "processing" host gets the
> data - instead of accessing them directly by NFS, it uses either "rcp"
> or even makes a "rsh" to the Ultra, invoking "back-copying", niced to
> +19. In both cases there are user-space programs running on Ultra and
> copying the data, which behave politely when confronted with a RT task.
>
> Still, I wonder whether there might be a more elegant solution. One idea
> would be to give the data network interface "higher priority", as
> compared to the other interface, that kernel would be aware of. Is it
> possible?
>
> any hints would be appreciated,
>
> regards, Michal.
>
- Next message: Hovik Melikyan: "Re: detecting 64-bitness inside a program"
- Previous message: qyvind_Gjerstad?=: "Re: detecting 64-bitness inside a program"
- In reply to: Michal Szymanski: "[Q] giving priority to network interfaces?"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Relevant Pages
|
