modifying syslog-ng.conf to process syslog messages

Pit-Ong.Ong.Goh_at_reuters.com
Date: 06/30/04

  • Next message: Khan, Farooq: "Sun HighGround Storage Resource Manager"
    Date: Wed, 30 Jun 2004 15:10:10 +0800
    To: sunmanagers@sunmanagers.org
    
    

    Hello again,

    Firstly thanks to all who responded to my question on Perl script but I've
    just realized that it's the syslog-ng.conf that I need to change as the
    syslog-ng.conf would call a Perl script to convert hostnames to capitals.

    Attached below is my syslog-ng.conf which I'm hoping to modify such that
    syslog messages containing the string "Port scan" (there's a space in
    between) OR "Teardrop" (no space in between) OR say "00304"
    should not be treated as high priority & thus should not be sent to
        /logs/ocm/high-priority_log nor to /dev/ttyb

    ie the lines below extracted from syslog-ng.conf are the pertinent entries :
       destination d_hpri { file("/logs/ocm/high-priority_log"); };

       # Netscreen level warning to emergency
       log {source(s_transform); destination(d_hpri); };
       log {source(s_transform); filter(f_filter10); destination(d_ttyb); };
       log {source(s_udp); filter(f_filter10); destination(d_transnet); };

    Let me know how to filter off those excessive syslog messages (ought to be
    case-insensitive except for numbers which does not apply for case sensitivity).

    We're getting too much messages being forwarded to our monitoring system,
    making it unmanageable.

    Thanks in advance,
    Goh

    ******************************** syslog-ng.conf ************************************
    # syslog-ng configuration file.
    #
    # This should behave pretty much like the original syslog on SunOS. But
    # it could be configured a lot smarter.
    #
    # See syslog-ng(8) and syslog-ng.conf(5) for more information.
    #
    # File sync time increased to 200 to reduce filesystem operations - this does have an impact on
    # log file rotations as a higher number of messages will be lost..
    #
    #------------------------------------------------
    # Global variables
    #------------------------------------------------
    options { sync (0);
          time_reopen (10);
          log_fifo_size (1000);
          use_dns (yes);
          use_fqdn (no);
          create_dirs (yes);
          keep_hostname (yes);
          chain_hostnames (no);
            };

    #------------------------------------------------
    # Destinations
    #------------------------------------------------
    source s_sys { sun-streams ("/dev/log" door("/etc/.syslog_door")); internal(); };
    source s_udp { udp(); };
    source s_transform { pipe("/home/ocmscript/ocmpipe"); };

    destination d_cons { file("/dev/console"); };
    destination d_mesg { file("/var/adm/messages"); };
    destination d_mail { file("/var/log/syslog"); };
    destination d_auth { file("/var/log/authlog"); };
    destination d_ipf { file("/var/log/ipf"); };
    destination d_all { file("/var/log/syslog_all"); };
    destination d_udp { file("/logs/$HOST/$HOST_$YEAR$MONTH$DAY_log"); };
    destination d_smn { udp("loghost"); };
    destination d_mlop { usertty("operator"); };
    destination d_mlrt { usertty("root"); };
    destination d_mlal { usertty("*"); };
    destination d_ttyb { file("/dev/ttyb"); };
    destination d_ttyocm { file("/dev/ttyb"); };
    destination d_hpri { file("/logs/ocm/high-priority_log"); };
    destination d_transnet { program("/home/ocmscript/transform_netscreen.pl"); };
    destination d_transcis { program("/home/ocmscript/transform_cisco.pl"); };

    filter f_filter1 { level(err) or
                     (level(notice..emerg) and facility (auth, kern)); };
    filter f_filter2 { level(err) or level(alert) or level(emerg) or
                         (facility(kern) and level(debug..emerg)) or
                         (facility(daemon) and level(info..emerg)) or
                         (facility(user) and level(info..emerg)); };
    filter f_filter3 { facility(auth) and level(info..emerg); };
    filter f_filter4 { facility(mail) and level(info..emerg); };
    filter f_filter5 { facility(local0) and level(info..emerg);};
    filter f_filter6 { match("ssmond");};
    filter f_filter7 { match("needs maintenance");};
    filter f_filter8 { match("disk not responding");};
    filter f_filter9 { match("Syslog Backup FAILED");};
    filter f_filter10 { facility(local0) and level(warning..emerg);};
    filter f_filter11 { facility(local7);};
    filter f_filter12 { match("changed");};
    #filter f_filter13 { facility(local0) and level(info);};
    #filter f_filter14 { match("change");};

    log { source(s_sys); filter(f_filter1); destination(d_cons); };
    log { source(s_sys); filter(f_filter2); destination(d_mesg); };
    log { source(s_sys); filter(f_filter3); destination(d_auth); };
    log { source(s_sys); filter(f_filter4); destination(d_mail); };
    log { source(s_sys); filter(f_filter5); destination(d_ipf); };
    log { source(s_udp); filter(f_filter5); destination(d_udp); };
    log { source(s_udp); filter(f_filter11); filter(f_filter12); destination(d_udp); };

    # Disk errors go to loghost
    log { source(s_sys); filter(f_filter6); destination(d_smn); };
    log { source(s_sys); filter(f_filter7); destination(d_smn); };
    log { source(s_sys); filter(f_filter8); destination(d_smn); };

    # Backup Failures go to loghost
    log { source(s_sys); filter(f_filter9); destination(d_smn); };

    # OCM and highpriority file
    # Netscreen level warning to emergency
    log {source(s_transform); destination(d_hpri); };
    log {source(s_transform); filter(f_filter10); destination(d_ttyb); };
    log {source(s_udp); filter(f_filter10); destination(d_transnet); };

    # Cisco
    log {source(s_transform); destination(d_hpri);};
    log {source(s_transform); destination(d_ttyb);};
    log {source(s_udp); filter(f_filter11); filter(f_filter12); destination(d_transcis);};

    # Netscreen with info and word "change" in message
    #log {source(s_udp); filter(f_filter13); filter(f_filter14); destination(d_ttyb);};
    #log {source(s_udp); filter(f_filter13); filter(f_filter14); destination(d_hpri);};
    #log {source(s_udp); filter(f_filter13); filter(f_filter14); destination(d_transnet);};

    #log {source(s_transform); destination(d_hpri); };

    # Catchall rule
    log { source(s_udp); source(s_sys); filter(DEFAULT); destination(d_all); };

    *********************** transform_netscreen.pl ****************************
    ****** This is the Perl script which I was trying to modify the other day but
    ****** it's not the right thing to modify as it's called by syslog-ng.conf
    ****** to convert hostnames found in syslog to capitals; so my apologies

    #!/usr/bin/perl -T

    use IO::File;
    use IO::Handle;
    use strict;

    my $naptime = 1;

    my $namepipe = new IO::File ">/home/ocmscript/ocmpipe"
        or die "Can't open pipe /home/ocmscript/ocmpipe for writing: $!\n";

    #my $fh = IO::File->new(">> /logs/ocm/test.log")
    # or die "Couldn't open /tmp/test.log for writing: $!\n";

        while (<STDIN>)
        {
               my $hostname;
               next unless m
               {
                  (stc-nsfw[0-9]+[a-z])
                  (.*)
                  (stc-nsfw[0-9]+[a-z])
                  (.*)
                  (stc-nsfw[0-9]+[a-z])
             }gix;

               $hostname=$3;
             $hostname=~tr/a-z/A-Z/;
             #print $`.$1." ".$hostname.$4.$hostname.$';
             $namepipe->print($`.$1." ".$hostname.$4.$hostname.$');
             $namepipe->flush();

    # $fh->print( $`.$1." ".$hostname.$4.$hostname.$') ;
    # $fh->flush();

              } # or appropriate processing

        sleep $naptime;
        STDIN->clearerr(); # clear stdio error flag

    $namepipe->close();

    --------------------------------------------------------------- -
            Visit our Internet site at http://www.reuters.com

    Get closer to the financial markets with Reuters Messaging - for more
    information and to register, visit http://www.reuters.com/messaging

    Any views expressed in this message are those of the individual
    sender, except where the sender specifically states them to be
    the views of Reuters Ltd.
    _______________________________________________
    sunmanagers mailing list
    sunmanagers@sunmanagers.org
    http://www.sunmanagers.org/mailman/listinfo/sunmanagers


  • Next message: Khan, Farooq: "Sun HighGround Storage Resource Manager"

    Relevant Pages

    • RE: about access-list location?
      ... "Standard Access List" = Can only filter based on the Source Address. ... Because of this limitation, it has to be near the "Destination" host, ... without effecting the communication between Host1-Host2. ... Router1 and then create rules to allow Host1-Host2 ...
      (Security-Basics)
    • Re: RFC 1918 addresses
      ... in any case, but 224/4, being multicast, can be very useful to allow ... as a destination address. ... where I only filter 240/4 for destination addresses. ... usually given to the loopback interface. ...
      (comp.unix.bsd.openbsd.misc)
    • Re: IPSec and clusters Easy Read Filters
      ... supports IPsec on WS03 clusters so we are sailing into new waters here. ... Filter Action: Allow ... > After one or both cluster nodes are rebooted the "filter ... > Destination Port Source DNS Source Address ...
      (microsoft.public.win2000.security)
    • Re: [fw-wiz] Blocking Video/Audio Streaming
      ... The pix is not just a packet filter only. ... keeps track of sessions not just source and destination. ... content filtering then look at the fixup command which will interogate the ... protocol http and fixup protocol rtsp 80. ...
      (Firewall-Wizards)
    • Re: Block all outside traffice when using RRAS
      ... matter) filters, and choose to create a new filter, it allows me to ... select my source and destination hosts or networks and a destination ... protocol, but not a destination port number. ...
      (microsoft.public.windows.server.active_directory)