central logfile / real time logging

Hi,

do you have an idea how to store application generated logfiles
(i.e. apache, ... ) from 1000 Solaris "zones" running on 250
physical Sun server onto a central FreeBSD logserver ?

There are 6 strong demands, which makes this setup very special
and tricky.

After many thinking and googeling I didn't find any OpenSource or
at least "not too expensive" solution which fulfills all of these
demands:

- "reliability", no logmessage may get lost
- on the central logserver the logfiles need to stay in separate files
per virtual zone and application (can be the case the zone runs
multiple services)
- "real-time logging", as soon as an application logrecord gets written
on one of the 1000 zones this logmessage should be written immediately
onto the central logserver. So its not sufficient to make this batched
on an hourly base.
- applications may not write to NFS directly to prevent locking of apps
if NFS server is unavailable (if solution should be NFS based).
- this solution needs to be aware of logfile rotation
on the application server side (the 1000 zones)
- this logging needs to be done from the "global zone" of a Solaris server.

Some people seem to use syslog-ng for this on client and central
repository side.

But I see here some problems. I need to convert apache logs from
normal logfiles. This must happen from the Global Zones.

Then the syslog-ng on the central side has no chance (I think)
to demultiplex the logfile stream into logs separated by
the different zone and application.

On the Global Zones I need a process that tracks some logfiles
that are from different virtual zones.

I need to open one tcp session to central archive per logfile.

So ~250 global zones open ~1000 tcp sessions towards the central
FreeBSD server.

There must be a way to tell the application on central FreeBSD
Server, that this stream now should be written to a certain
subdirectory and filename, so that all is separate.

This application need to buffer on the Global zone, if the server
is not available. Or at least keep track, what hasn't been written
yet. And must be aware of logfile rotation.

Any idea ?

Andreas ///

--
Andreas Klemm - Powered by FreeBSD 6
Need a magic printfilter today ? -> http://www.apsfilter.org/
_______________________________________________
freebsd-isp@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-isp
To unsubscribe, send any mail to "freebsd-isp-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: DCDIAG DNS Failure
    ... Without advance view I have 4 forward lookup zones and 7 reverse ... My DNS server is not multihomed. ...
    (microsoft.public.windows.server.dns)
  • Re: Blocking undesirable domains using BIND
    ... server is now consuming 208 MB of ram. ... to the exact same thing it seems to be needlessly wasting memory. ... My experience, albeit with a smaller number of zones, is a bit different. ... zone file 244B in size on disk: ...
    (freebsd-questions)
  • RE: SBS2003: Intermitent Connectivity from Clients to Public websi
    ... I already had two zones set up under the Forward Zones. ... names along with their respective IP addresses per DNS name. ... Originally, I did install ISA Server, but uninstalled it. ... you can access them normally on SBS. ...
    (microsoft.public.windows.server.sbs)
  • Re: DCDIAG DNS Failure
    ... Without advance view I have 4 forward lookup zones and 7 reverse ... do you need all these reverse zones or your DNS server is multihomed ... -Your exchange server that is giving you problems, can it send Mail outside, ...
    (microsoft.public.windows.server.dns)
  • Re: DCDIAG DNS Failure
    ... also can you check if the DNS server is responding to all its IPAddress ... zone and forward zones. ... did you created MX records for the exchange domains that you're hosting? ...
    (microsoft.public.windows.server.dns)
Loading