Administrator Handbook TOC

                                    

Syslog Collector

Administrator Handbook

 

 

 

 

 

 

syslog collector logo

 

 

  

 

 

Disclaimer

 

Luteus SARL makes no representations or warranties with respect to the contents or

use of this handbook, and specifically disclaims any express or implied warranties of

merchantability or fitness for any particular purpose.

Further, Luteus SARL reserves the right to revise this publication and to make

changes to its content, at any time, without obligation to notify any person or entity of

such revisions or changes.

Further, Luteus SARL makes no representations or warranties with respect to any

Luteus software, and specifically disclaims any express or implied warranties of

merchantability or fitness for any particular purpose.

Further, Luteus SARL reserves the right to make changes to any and all parts of

Luteus Software, at any time, without any obligation to notify any person or entity of

such revisions or changes.

 

Copyright 2003 Luteus SARL. All rights reserved. No part of this book may be

reproduced, photocopied, stored on a recovery system, or transmitted without

the express written consent of the publisher.

 

Luteus SARL

Bâtiment : « Le Sextant »

ZI de Moissy-Cramayel

462 rue Benjamin Delessert

BP 83

77554 Moissy-Cramayel

FRANCE

 

 

 

 

 

 

 

Table of contents

 

4      The Syslog Collector architecture. 4-11

6      Syslog Collector Manager 6-39

 


 

 

List of Figures

 

Not all illustrations, logos, and diagrams used in this manual may correspond exactly to the software, which, by its very nature, is always changing. They are only there to document and illustrate the concepts of the manual but may not under any circumstances be used for reference.

 

 

1       About this guide

 

This manual has been made as an online help file available from the software or from the website www.loriotpro.com. The order of the chapters is designed to help you to quickly configure your LoriotPro software and supervise your Information System (IS) in a typical manner.

 

1.1       Conventions used in this guide

 

This guide uses a special format to show the path to a specific menu option. Rather that specifying all menu titles, we will use the greater than sign '>' and italics.

 

Example : Programs>LoriotPro>LoriotPro

 

Links to websites and e-mail addresses are shown in blue:

 

http://www.loriotpro.com/

sales@loriotpro.com

CThis icon calls your attention to a note or a tip.

 

* This icon calls your attention to a possible trap.

1.2       How to get technical support

 

For technical support, send an e-mail directly to support@loriotpro.com. Specify your problem and the context of the problem in the e-mail, and we will try to help you as soon as possible.

 

1.3       Web site

 

There are many tips and tricks available on our website  http://www.loriotpro.com

 

.

 

2       Introduction to LoriotPro

Users accustomed to our LoriotPro product can skip this chapter and go directly to the next chapter

2.1       Overview of LoriotPro

With LoriotPro, you possess a tool with the power of a control tower for monitoring your computing resources that guarantees availability and performance to your users.
snmp control tower
Your computing resources, data and applications, servers, workstations, switches and network routers all constitute your IS's infrastructure and can be supervised thanks to the management protocol SNMP (Simple Network Management Protocol), the Internet standard. LoriotPro takes advantage of this protocol in its smallest details to help you be effective and accurate in your daily supervising tasks. LoriotPro does not stop there; it extends its control by using various protocols such as ICMP (Internet Control Message Protocol) and HTTP for WEB server monitoring, for example. Monitoring is made possible thanks to an optimized use of the Windows graphical interface as well as Internet browsers. The direct display of your computing resources and of their performance status in the form of color-coded icons and alert messages draws your attention to any fault situations.


Among its multiple features:


Directory

Helps you to manage and order your devices in the way you want. Shows devices as icons and displays in real time their current operating status. The entry point for resource inventory with advanced search capabilities and HTML reporting.

 

Fault Management

Collects, stores and presents alarm information for reporting. Can trigger actions, alarms, e-mail, process, and so on, based on advanced filtering conditions. Supports Syslog and Trap event messages. Display in one click the health of a portion of your network.
 

Configuration Management

Allows remote configuration of any IP device through the MIB (Management Information Base) and the SNMP SET command.

 

Topology maps

Draws a map with devices (IP nodes) and dynamically displays their operating status. Displays huge Networks on a condensed Map.

 

Auto Discovery

Discovers network nodes and adds them to the Directory and the Map. In-depth configuration of this module provides a total control of the discovery process.

 

Web Enabled

Supports thin client Web access, allowing network to be monitored from any location.

 

Protocol Support

Supports Simple Network Management Protocol (SNMP), Syslog, ICMP.

 

MIB Management

MIB browser and compiler, generation of reports in HTML format. MIB scripter creates SCI files that automate any set of SNMP requests.

 

Detect Device MIB Support

Detects what MIBs are supported by a device, provide the MIB file reference.

 

Scripting Language

Provides a scripting language that automates the generation of reports in HTML format.

 

IP Scanner

Scans a range of IP addresses and detects the presence of hosts and SNMP agents.

 

Database Interface

ODBC (Open Database Connectivity) interface provides a way of storing events and more. Examples with MYSQL database are described.

 

Customizable GUI

Toolbars and menus can be customized, proprietary Skins can be set.

 

Plug-In Technology

Plug-ins within LoriotPro are dedicated tasks that perform SNMP data processing for a specified host or a group of hosts. Numerous plug-ins are provided by default but this technology is also opened and allows you to develop additional modules that manage your specific needs. Wizard tools are provided for Visual C++ and multiple examples are provided with their source code.

 

Online Help

Provides access to extensive online documentation.

 

2.2       Introduction to LoriotPro Plug-In

A Plug-In is a separate code module that behaves as though it is part of the LoriotPro software. Plug-ins are used for example to perform MIB object values acquisition via SNMP on a host or a group of hosts, to process them, to display them in various ways and eventually to generate alarms.

There are three types of plug-ins:

Direct Plug-In: Used for processing the data of a specific Directory Object. Direct plug-ins are started one by one and work independently. They are used to perform one SNMP request or a set. These plug-ins have the file extension <.lp>.

Directory Plug-In: Allows you to associate an application to a Directory host. These plug-ins are loaded and saved within the Directory. They allow you to perform scheduled tasks or repetitive tasks (graphics, polling, etc.). These plug-ins extend the capacity of LoriotPro by creating an Active Directory. These plug-ins have the file extension <.slp>.

Service Plug-In: Is loaded as a complement of the LoriotPro services. Service plug-ins are not linked to a host or to any object of the Directory. They are used to extend the global features of LoriotPro.

2.3       Plug-In limitations

 

The use of plug-ins can extend the features of LoriotPro and hundreds of them can run simultaneously and process management data. The downside of that is the increasing need of CPU on the LoriotPro system and the increasing bandwidth used by the network traffic converging on the LoriotPro system. This statement is at the origin of the Collector concept that distributes the process load on multiple agents and reduces network traffic by setting the collector agent closest to the source of traffic.

 

3       Introduction to the Collector concept

 
The goal of the Collector architecture designed by LUTEUS is to provide a scalable solution for handling huge quantities of management information and to help the administrator to classify, analyze and process them. 
 
The product has been designed based on the following issues:
 
·       The main issue that system and network managers have to face is the collection and processing of huge quantities of management information.
·       They need to have detailed information of what is happening in their system and network devices by turning on in-depth logging and alarm facilities. 
·       They want to receive the critical information immediately but also want to retrieve non-critical ones at a later time.
·       They want to have a centralized system that can collect information files without using too much network bandwidth.
·       They want to manage their infrastructure from a centralized manager.
·       They want to have access security and control of the management solution.

 

The concept of Collector in the LoriotPro management system architecture is based on two components:

 

  1. The Collector agent software running on Microsoft Windows workstations. These agents are dedicated to the collection of the supervision information sent by the System and Network devices located in a predefined area.

 

  1. The Collector Manager software running on top of our LoriotPro supervision software. This manager has to manage the agents avoiding any human intervention directly on the agent.

 

Collector agents are responsible for:

 

Filter rules applied on the agent can be set on each agent either from the agent GUI or from the Collector Manager from a central location.

 

 

4       The Syslog Collector architecture

 

The Syslog Collector Solution uses the predefined concept of Collector applied to the management of Syslog messages. The Syslog system provides the transport and storage mechanisms for event notification messages, in the form of Logs. Syslog is a de-facto standard defined by RFC3164 for logging system events. It was commonly and initially used by Unix systems, later on by network devices (routers and switches) and more recently by firewalls.
The Syslog Collector architecture is built around two components: the agents and the manager. The manager is a program running exclusively on our LoriotPro supervisor solution.

 
syslog concept
 

Syslog messages are collected by agents called, in our terminology, “Syslog Collector Agents.” Agents are designed to collect a large throughput of Syslog messages and to process them according to advanced filtering rules. Filtered messages can then be displayed on a viewer, the agent taking on the role of a simple Syslog server. Messages can be stored locally in files or forwarded to the central management system. Critical messages can be sent to the centralized management system either as LoriotPro proprietary-formatted event messages or as Syslog-formatted messages. Agents can be cascaded to build a hierarchical architecture of Syslog message relays.

Agents can be used as a standalone solution and act as a Syslog server or Syslog relay. Our LoriotPro Network Management System (NMS)
and the Syslog Manager are not necessary in this case. Filtering rules can be defined from the Agent GUI and applied. Actions taken on conditions defined in
the filtering rules can be displayed in a viewer, stored in files or forwarded to another Syslog server. 
 

The Syslog Collector Manager is responsible for the management of the agents from a centralized position. Filtering rules are defined on the manager and pushed to the agent. The manager is also able to retrieve a filter rule previously loaded onto an agent. Filtering rules are stored in local text-only files.

The manager is also able to upload Syslog files previously stored on the agent. The Syslog files can be compressed on the fly during uploading, sparing precious bandwidth of WAN links or on-demand links. The manager works on top of our LoriotPro NMS as a Plug-In Service.

As we have stated previously, the messages sent by the Syslog Collector Agents can be in the LoriotPro event format. The LoriotPro Event Manager receives them and processes them. They are first displayed in the Event Log window and if necessary, they trigger actions based on predefined conditions. Actions can send messages, start programs, play sounds, etc.

 

5       Syslog Collector Agent

5.1       Agent Installation

 

5.1.1     Hardware and Software requirement

 

 

The Syslog Collector Agent should be installed on a Microsoft Windows workstation. We highly recommend using a Windows 2000 professional version. In this environment the agent can run as a service at start up.

 

5.1.2     Preliminary checking

 

The agent will hook the UDP Syslog port 514 for its operation. You must check beforehand that no other program is already using this port.

 

From a DOS session, type:

 

C:\Netstat  -na

 

Active connections

 

  Proto  Local Address         Remote Address   State

  TCP    0.0.0.0:8020           0.0.0.0:0              LISTENING

  TCP    0.0.0.0:5002           0.0.0.0:0              LISTENING

  TCP    0.0.0.0:5003           0.0.0.0:0              LISTENING

  TCP    10.33.10.130:139       0.0.0.0:0              LISTENING

  UDP    0.0.0.0:514            *:*                   

  UDP    0.0.0.0:2593           *:*                    

  UDP    0.0.0.0:5001           *:*                   

  UDP    0.0.0.0:1421           *:*                   

  UDP    0.0.0.0:1422           *:*                   

  UDP    0.0.0.0:1440           *:*                   

  UDP    10.33.10.130:137       *:*                   

  UDP    10.33.10.130:138       *:*                   

  UDP    10.33.10.130:8888      *:*                   

 

 

The line  <UDP    0.0.0.0:514            *:*>  shows you that a Syslog server is already running on that machine. You should find this program or service and get rid of it before the agent installation.

 

If you want to use the agent on a system where LoriotPro is already installed, you have to modify the  lalarm.ini  file located in the /bin directory of LoriotPro.

 

Lalarm.ini

[ALARM]

alarm_port 5001

syslog_port 514

 

Replace the  line syslog_port 514 by syslog_port 0. This will suppress the service at the next restart of LoriotPro.

 

5.1.3     Starting the installation

 

From the CD ROM or from the hard disk start the program:

 

loriotprov200b136sp0-collector-xx.exe

 

CThe xx characters in the filename specify the current release number and are subject to change.

 

syslog license

Figure 5‑1 : License agreement

syslog installation

Figure 5‑2 : Installation directory choice

* If you have LoriotPro installed on this machine, keep the same directory as where LoriotPro is installed. You must stop LoriotPro before you continue to the agent installation.

 

By clicking the Start button, files will be copied onto your disk.

 

* If you proceed to an installation of the agent over a previously installed agent you must stop the agent and/or remove it from the list of Windows services .

 

syslog agent installation

 

 

syslog collector installation

 

The installation is completed, click the OK button

 

The license file is displayed and the program icons are set in the Windows 

Syslog file

 

If LoriotPro is installed on this computer the complementary Syslog icons will be placed in the same Windows (directory).

 

syslog license agreement 

 

If the following message appears, answer NO :

 

 

This message warns you about an already running or installed agent. You must stop it and uninstall it before continuing.  

 

If your agent runs as a Windows Service, uninstall it with the MS DOS program Uninstall Syslog Collector Service. When done you can restart and proceed to the agent installation.

 

The agent software is located in the bin/collector/syslog directory as shown below:

 

syslog collector directory

 

5.2       Agent configuration

 

Agents are  by default configured to communicate with a Syslog Manager located on the same computer. In a distributed architecture these parameters have to be modified.

Use Notepad to edit the CollectorSyslogAgent.ini file located in bin/collector/syslog directory.

 

CollectorSyslogAgent.ini.

[ALARM]

syslogd_port 514

max_log_view_lines 50

collector_mode 0

hide_log_view 0

loriotpro_ip_add 127.0.0.1

loriotpro_event_send 16001

Loriotpro_event_port 5001

collector_tcp_manager_server_ip 127.0.0.1

collector_tcp_manager_server_port 5002

collector_tcp_agent_server_port 5003

collector_tcp_agent_server_timeout 5000

collector_tcp_server_password "admin"

 

 

5.2.1     Configuration of communication parameters

 

By default you should find these two lines:

 

loriotpro_ip_add 127.0.0.1

collector_tcp_manager_server_ip 127.0.0.1

 

Replace both IP address 127.0.0.1 by the IP address of your LoriotPro supervision System:

 

Example:

 

loriotpro_ip_add 193.2.2.2

collector_tcp_manager_server_ip 193.2.2.2

 

 

To exchange information, agents and the manager use TCP ports. Agents listen on TCP port 5003 and the manager listens on TCP port 5002.

CCheck by using the  DOS netstat –na command that these two TCP ports are not already hooked by another application.

 

If another application is using it, you can set another value for the agent and/or for the manager.

 

Example:

 

collector_tcp_agent_server_port 678

collector_tcp_manager_server_port 679

 

 

* Warning: If you make the modifications above, you will have to modify the manager parameters in the agent configuration too.

 

Agents and the manager use a proprietary secured protocol to communicate. A password is used to authenticate the agent to the manager and reciprocally. This password is never sent but used to generate a session key by using an MD5 hashing code.

 

The following line of the configuration file is used to declare this password:

 

collector_tcp_server_password admin

 

* Warning: If your password contains spaces, set your password between quotes.

 

Example:

 

collector_tcp_server_password “admin 001”

 

The following diagram shows the functional architecture of the Syslog Collector solution. For clarity’s sake, only one agent is represented.

syslog server and loriotpro communication

Figure 5‑3: Schema of the communication between an agent and the manager.

 

5.2.2     Syslog server daemon configuration, display options

The agent is first a Syslog Server or relay. It should be able to receive Syslog messages coming form network or system devices. The default port for receiving Syslog messages is UDP 514:

 

syslogd_port 514

 

The agent GUI has an integrated viewer of filtered Syslog messages. By default the viewer displays the last 50 filtered messages. You can change this value by modifying the configuration line:

 

max_log_view_lines 50

 

* Keep this number as low as possible; displaying messages in the viewer is CPU time consuming and could generate a loss of incoming messages in high throughput conditions. If the viewer agent is not used locally do not display messages and hide the GUI from the screen.

5.3       License installation

To work, the agent needs a registered license which is stored in the CollectorSyslogAgentLicence.ini located in the bin/collector/syslog directory. This file is provided to you when you buy a license.

By default the agent is provided with a license key limited to 30 days.

During this 30-day trial period you can modify the license number (second line) and install as many agents as you want.

 

* Warning: Each agent should have a different license number. The manager will refuse to manage agents having the same license number. The license number is used in the authentication process.

 

5.4       Starting the software agent

 

5.4.1     Running the agent as a service

 

It is possible to install the agent as a Windows service with the provided program CollectorSyslogService  located in the bin/collector/syslog directory.

Specify option  -i.

 

CollectorSyslogService –i

 

C:\Program Files\LoriotPro\bin\collector\syslog>collectorsyslogservice -i

 

LoriotPro V2.00 Beta NT/2000 service Installer v1.3

Copyright (C) 2003 Luteus SARL

====================================================================

= WARNING : This tools permit only to automatiquely start the      =

=           LoriotPro Syslog Collector like one service at startup.=

[Syntax]============================================================

= For Installing the module like a service use :                   =

=    CollectorSyslogService -i                                     =

= For UnInstalling the module use :                                =

=    CollectorSyslogService -d                                     =

====================================================================

 

 

Create [c:\run_CollectorSyslogService.bat] file Sucessfully

 

LoriotPro Syslog Collector Service Installed Sucessfully

 

Press a key to continue

CThis tool creates the file c:\run_CollectorSyslogService.bat allowing the program to run as a service.

 

CThe line hide_log_view 0 in the CollectorSyslogAgent.ini file with option 1 unhides the agent service at startup.

 

hide_log_view 1

 

Once the service is installed, you can start the Administration tools and manage the service from there.  

 

 

Figure 5‑4: NT/2000 Administration Tools

 

In the service list you will find:

 

LoriotPro Syslog Collector Agent Service

 

The service is configured by default  to start automatically at each Windows startup.

 

syslog agent

Figure 5‑5: NT/2000 Service Administration program

 

However, just after the installation process of the service either restart Windows or do a manual start form the Service Administration Windows above.

 

Select the Syslog service by using a single click and choose the Properties option on the contextual menu.

 

syslog windows service

Figure 5‑6: Control window of an NT/2000 agent service

 

Click on the Start button.

 

The service start and the agent GUI are displayed. If you selected the hide_log_view 1 in the  CollectorSyslogAgent.ini file, the agent GUI disappears after a few seconds.

Only the Syslog manager is able to unhide it with the remote control protocol.

 

syslog filter rules

Figure 5‑7: Main windows of the Syslog Collector agent GUI

 

If the message “Prob to generate one SOCK_STREAM xxx for CollectorSyslog is displayed, the agent is having problems communicating with the manager. Check the TCP port configuration on both sides, agent and manager. This message can appear if you stop and start the agent program successively and too fast.

 

 

syslog socket

Figure 5‑8:The TCP port is already used

 

Remember this TCP port number can be changed in CollectorSyslogAgent.ini file:

 

collector_tcp_agent_server_port 5003

 

*Warning: If you change the agent TCP port you have to change the setting on the manager to the same port number for that agent.

 

 

If the message “Error on bind() Syslog, error code=10048“ is displayed, you probably have another Syslog server running that hooks the UDP port 514. Stop it and make sure that it is not started by default (as a service) the next time you restart the computer.

 

Figure 5‑9: The UDP port on the Syslog is already being used.

 

5.4.2     Uninstall the agent service

 

Before uninstalling the agent service, you should stop it. Use the Service Administration tool available in the Control Panel. Select the Syslog agent service and click the Properties option of the contextual menu.

 

start syslog

Figure 5‑10: Service control windows

syslog service control

 

When done, start the program CollectorSyslogService.exe with option –d.

 

CollectorSyslogService –d

 

C:\Program Files\LoriotPro\bin\collector\syslog>collectorsyslogservice -d

 

LoriotPro V2.00 Beta NT/2000 service Installer v1.3

Copyright (C) 2003 Luteus SARL

====================================================================

= WARNING : This tools permit only to automatiquely start the      =

=           LoriotPro Syslog Collector like one service at startup.=

[Syntax]============================================================

= For Installing the module like a service use :                   =

=    CollectorSyslogService -i                                     =

= For UnInstalling the module use :                                =

=    CollectorSyslogService -d                                     =

====================================================================

 

 

LoriotPro Syslog Collector Service UnInstalled Sucessfully

 

Press a key to continue

5.4.3     Running the agent as a standard program

 

To start the agent, you simply run CollectorSyslogAgent.exe located in the  bin/collector/syslog Directory.

 

The main screen is displayed.

 

syslog message log

Figure 5‑11: Agent main screen,  the View Filtered Syslog tab

5.5       Agent control

The agent software has two tabs and five control buttons. The Button bar allows you to control agent operations.

 

5.5.1     Control buttons

 

Figure 5‑12 : Control buttons

 

Button

Explanation

Close the agent program.

Restart the agent after a stop.

Note:

All displayed Syslog messages are cleared.

Stop the agent. Incoming Syslog messages are not processed, because the agent is in sleep mode.

Restart the agent if stopped with the Pause button.

Note:

All displayed Syslog messages are not cleared.

Pause the agent and allow the  Resume action.

 

5.5.2     License Information

 

Your license number is displayed in the main window within the communication parameters. In the example below the agent is installed on the same PC as LoriotPro and  is running in evaluation mode.

Figure 5‑13: License information example

 

5.5.3     Syslog message viewer

 

view syslog

 

This window displays the messages allowed by filtering rules. According to filter rules it is possible to have the same messages with different colors

 

*Warning: All received messages are not displayed by default.

 

 

Column

Explanation

Date

The arrival date of the syslog Message

Time

The arrival time of the Syslog message in hours:minutes:seconds

Priority

The “facility“ field and  “level” of the Syslog message

Hostname

The IP address of the device that sent the Syslog message

Message

The message itself, a character string

Note

The facility filed is not displayed, information <xxx>.

 

The Facilities and Severities of the messages are numerically coded with decimal values.  Some of the operating system daemons and processes have been assigned Facility values.  Processes and daemons that have not been explicitly assigned a Facility may use any of the "local use" facilities or they may use the "user-level" Facility. Those Facilities that have been designated are shown in the following table along with their numerical code values.

 

Numerical Code               

Facility   

           0            

kernel messages

           1            

user-level messages

           2            

mail system

           3            

system daemons

           4            

security/authorization messages

           5            

messages generated internally by Syslog

           6            

line printer subsystem

           7            

network news subsystem

           8            

UUCP subsystem

           9            

clock daemon

          10            

security/authorization messages

          11            

FTP daemon

          12            

NTP subsystem

          13            

log audit

          14            

log alert

          15            

clock daemon

          16            

local use 0  (local0)

          17            

local use 1  (local1)

          18            

local use 2  (local2)

          19            

local use 3  (local3)

          20            

local use 4  (local4)

          21            

local use 5  (local5)

          22            

local use 6  (local6)

          23            

local use 7  (local7)

 

Numerical

         Severity  Code

           0

       Emergency: system is unusable

           1

       Alert: action must be taken immediately

           2

       Critical: critical conditions

           3

      Error: error conditions

           4

       Warning: warning conditions

           5

       Notice: normal but significant condition

           6

      Informational: informational messages

           7

      Debug: debug-level messages

 

The Clear button allows the administrator to clear the contents of the window.

 

 

By default the agent displays the last 50 received and filtered messages. This value can be changed in the following line of the configuration file CollectorSyslogAgent.ini:

 

max_log_view_lines 50

 

* Restart the program after modification.

 

5.5.4     Filter List Editor

 

 

This window is not used if you manage the agent from a Syslog Manager. If you have not chosen a distributed architecture and you have no LoriotPro and Syslog Manager, you have to define filter rules from there on each agent or define it on one agent and copy the filter rule file manually on each agent.

 

 

A set of buttons allows the management of rules within a filter rule list. You can add, insert, and move filter rules.

 

insert new syslog filter rule

Figure 5‑14: Filter creation and modification buttons

 

Button

Explanation

insert new syslog filter rule

Insert a new filter rule in the list above an existing selected rule.

insert new syslog filter rule

Insert a new filter rule in the list below an existing selected rule.

insert new syslog filter rule

Insert a new filter rule at the top of the list.

insert new syslog filter rule

Insert a new filter rule at the bottom of the list.

Move the selected filter rule up.

Move the selected filter rule down.

Suppress the selected filter rule.

 

Filter rules are sets of filters gathered in a filter list. Each time a Syslog message arrives, it is analyzed by each rule in the list, sequentially processing from top to bottom. A rule contains conditions and actions. If the conditions are satisfied, actions are executed.
A single Syslog message can match multiple filter rules and triggers multiple actions.
Among the possible actions, one is able to stop the walking process through the filter list and jump to the processing of the next incoming message.  

 

 

syslog filter rules

Figure 5‑15: Filter Management window

 

5.5.5     Filter rule settings

 

Parameter table

Columns

Explanation

IP Address

IP Mask

This is a condition field. The agent process checks that the source IP address of the sender matches the IP address and network mask specified here.

Example

IP (0.0.0.0) Mask  (0.0.0.0). All IP source addresses are accepted.

IP (10.0.0.0) Mask  (255.0.0.0). All IP addresses pertaining to the network 10.xxx.xxx.xxx are accepted.

IP (10.45.25.63) Mask (255.255.255.255). Only this host is accepted.

 

 

A double-click on this field in the filter list allows you to modify this parameter.

Facility

This field allows you to filter messages according to their Facility type. The Facility field is defined initially on the device that sends the Syslog message.

kernel syslog local syslog local syslog 

The  « –1 all » choice matches all types of facility values.

Level

This field allows you to filter messages according to their Facility Level value. The Facility field is defined initially on the device that sends the Syslog message.

.syslog level

The  « –1 all » choice matches all types of facility level values.

String 1

A Syslog message is a simple character string. The field “String 1” allows you to filter messages based on a match between this string and the contents of the message.

syslog message

A double-click on the field allows you to specify the search string.

 

An empty string (null) will allow any message to match this condition.

Offset

If the offset is specified the predefined string (String 1) will have to start at this precise position.

 Note:

This option could be useless because message contents could change and thus the offset is no longer viable.

And/Or

A second condition on a second string can be added. Boolean “or” and “and” operators can be applied to both strings.

String 2

This is the second string that can be defined as a condition.

Offset

Offset that can be applied on this second string. Offset specified the number of characters from the string’s beginning.

 

syslog filters

Figure 5‑16 Filter Management window

 

Column

Explanation

 Case

The case of the string is either sensitive or not. If sensitive, uppercase and lowercase characters are not the same.

Action

If all the previous conditions are satisfied then a basic action is executed.

syslog filter action

Actions

Explanation

00 none 

The message is cleared from memory, nothing happens.

01 log 

The message is saved to a file whose name is defined in the Log File column.

02 display 

The message is displayed in the View Filtered Syslog with the appropriate color as defined in the Bkg Color and Ink Color column.

03 log+display 

The message is saved to a file defined in the Log File column and displayed in the View Filtered Syslog with the appropriate color as defined in the Bkg Color and Ink Color column.

 

LoriotPro

If all the conditions are satisfied and if an IP address is defined in this field the agent will send a LoriotPro event message (proprietary format) to this address. The next fields, Event and Level, are used to build the message. However, the event number should be different from 0.

Note:

The LoriotPro message content is a copy of the Syslog message content.

Event

The event number use in the LoriotPro event format.

Level

The severity level used by the LoriotPro event format.

Syslog

If all the filtering conditions are satisfied and if an IP address is defined in this field the agent will send a Syslog message to this address.

The Threshold value sets the number of incoming messages needed to satisfy this filter rule before sending a Syslog message.

 

Syslog agent filter rules

Figure 5‑17 : Filters Management window

 

Columns

Explanation

 Threshold

Is used to trigger the sending of a LoriotPro or Syslog message upon a predefined count.

Example: If the value is set to 3, a LoriotPro and/or a Syslog message will be sent only when three incoming syslog messages of that type will be seen.

Bkg Color

Allows you to define the background color of the message displayed in the View Received Syslog window .

A double-click of the mouse in the field allows you to change this parameter.

Ink Color

Allows you to define the color of the ink use to write text in the View Received Syslog window.

 

 

A double-click of the mouse in the field allows you to change this parameter.

syslog management

Next Filter

This option allows you to stop the filter rule list processing. The next rules in the list are not processed if the NO option is selected.

 

Log File

If all the conditions are satisfied and if the action is log or log+display the message is appended to the file specified here. The final file name is built from this name and from the current date. The file follows the csv format and is text readable.

Note:

A new file is automatically created each 24 hours.

syslog csv log file

 

 

5.5.6     Apply and save filter list

 

Buttons on the left side of the filter window allow the management of filter lists.

 

syslog manager

 

Button

Explanation

Reload the active filter list and apply modifications.

Open a previously saved filter list file.

syslog filter file

 

Note:

The file filter_Syslog.fil is loaded by default at each agent startup.

Save the filter list in the default  filter_Syslog.fil file.

Save the filter list in a file with extension .fil.

Apply the current filter list to the agent filtering process. If this filter list is not saved using the Save button the change will be lost at the next agent startup.

 

6       Syslog Collector Manager

6.1       Introduction

We have seen that an agent can work alone and also act as a Syslog server or relay. When you use only a few agents this solution is well adapted, but the management of each agent becomes a cumbersome task if they are numerous and/or distributed among various geographic locations.

 

The Syslog Collector Architecture is designed to solve this problem.

 

To have the full Syslog Collector architecture, a Syslog Collector Manager and a LoriotPro supervisor are necessary.

 

The main roles of the Syslog Collector Manager are:

 

 

The main roles of the LoriotPro software in the Syslog Collector architecture are:

 

·       To support the Syslog Manager as a plug-in.

·       To provide a framework for network and system supervision.

·       To provide the directory infrastructure for the agent management.

·       To manage events or Syslog coming from the agent.

6.2       Installing the Syslog Collector Manager

The Syslog Collector Manager is installed as any service plug-in from LoriotPro. This is done from the Service tab of the Workspace within the LoriotPro main screen or from the Service menu.

 

This plug-in has the extension .sp and is loaded in the same way as other standard services of LoriotPro.

These plug-ins are not related to the Directory structure of LoriotPro. They are automatically started at each LoriotPro startup.

 

 

syslog collector manager

Figure 6‑1: LoriotPro Service Plug-In Management window

 

The service tree and its various menu options are covered in the LoriotPro administrator handbook. The following information is provided as a reminder.

 

6.3       The Syslog Collector Manager Plug-In

 

The Syslog Collector Manager Plug-In is installed with the agent software in the bin/plug-in directory of LoriotPro software. If you install the agent on the LoriotPro system the plug-in is automatically copied into the correct directory.

 

File name

Agent directory

LoriotPro (manager directory)

CollectorSyslogManager.sp

Bin/plug-in

Bin/plug-in

CollectorSyslogManager.ini

bin

Bin

CollectorSyslogManagerLicence.ini

bin

Bin

CollectorSyslogManagerMsg.txt

bin

Bin

 

6.4       Starting the Syslog Collector Manager

 

From the contextual menu of the service workspace:

 

>Add New Service

 

add syslog service

Figure 6‑2 : Installing a new service plug-in (the Syslog Collector Manager) within LoriotPro

 

Select the Syslog Collector Manager

 

syslog plug-in

Figure 6‑3: Selecting the Agent Management Service Plug-In

 

Select Open and the plug-in is loaded and displayed.

 

 

syslog window

Figure 6‑4: Main Window of the LoriotPro Syslog Collector Manager Service Plug-In

To communicate with agents, the Manager must be set up. You will first need the license key of each installed agent.

The entire configuration is done in the CollectorSyslogManagerLicence.ini file.

 

6.5       Using the Manager

6.5.1     Declare Syslog Collector agents

 

Use the Edit Agent List button.

 

add syslog agent

Figure 6‑5: Edit Agent List button

 

The CollectorSyslogManagerLicence.ini is in text format and can be modified using Notepad.  

 

syslog configuration file

Figure 6‑6: Edit Agents with the Notepad utility

For each agent, append a line containing the following information:

1)    Agent name
2)    Agent IP address
3)    License key for this agent (the same key is set on the agent side)
4)   
The TCP listening port for this agent
5)   
The password for this agent (the same password is set on the agent side)

CIf the password includes a space, the password should be specified between quotes.

 

Example:

 

“agent italien” 182.2.3.4 101101 5003 « admin secret »

 

Fields

Parameters

Agent name 

“agent italien”

Agent IP address

182.2.3.4

License key for this agent

101101

The TCP listening port of the agent

5003

The password for this agent

“admin secret”

* Warning: The manager will not work if two agents own the same license number.

 

 

 

During the evaluation period, you can change the license number set by default in the CollectorSyslogAgentLicence.ini file located in the bin/collector/Syslog directory of the agent and set the same number on the Manager side.

 

Example:

Agent 1

 

IP Adresse : 192.168.1.1

Port TCP : 5003

Password : admin

 

CollectorSyslogAgent.ini

[ALARM]

syslogd_port 514

max_log_view_lines 50

collector_mode 0

hide_log_view 0

loriotpro_ip_add 193.1.1.1

loriotpro_event_send 16001

Loriotpro_event_port 5001

collector_tcp_manager_server_ip 193.1.1.1

collector_tcp_manager_server_port 5002

collector_tcp_agent_server_port 5003

collector_tcp_agent_server_timeout 5000

collector_tcp_server_password "admin"

 

CollectorSyslogAgentLicence.ini

 

30 days Evaluation

10001

AAAA-AAAA-AAAA-AAAAA

 

Agent 2

 

IP Adresse : 194.169.1.2

Port TCP : 5003

Password : admin

 

CollectorSyslogAgent.ini

[ALARM]

syslogd_port 514

max_log_view_lines 50

collector_mode 0

hide_log_view 0

loriotpro_ip_add 193.1.1.1

loriotpro_event_send 16001

Loriotpro_event_port 5001

collector_tcp_manager_server_ip 193.1.1.1

collector_tcp_manager_server_port 5002

collector_tcp_agent_server_port 5003

collector_tcp_agent_server_timeout 5000

collector_tcp_server_password "admin"

 

CollectorSyslogAgentLicence.ini Agent 2

30 days Evaluation

10002

AAAA-AAAA-AAAA-AAAAA

 

 

 Manager

 

IP Adresse : 193.1.1.1

PORT TCP : 5002

 

CollectorSyslogManagerLicence.ini

# For each agent, Add a line with,  Agent Name, Agent IP address, Agent License_ID, Agent password

# You will find the License_ID for this agent on line two of the CollectorSyslogAgentLicence.ini file,

# located on your agent. Each agent should have a unique license

# SyslogConnectorAgent_name SyslogConnectorAgent_ip_addr license_id server_port password

LocalAgent 127.0.0.1 1000 5003 admin

Agent1 192.168.1.1 10001 5003 admin

Agent2 194.169.1.2 10002 5003 admin

 

 

 

When done, if you use the combo box of the Manager you should see the three agents.

 

syslog agent list

Figure 6‑7 : Combo box

 

Select one agent from the list and click the Get Filter button. If everything is configured properly the filter list of the agent appears in the Manager’s filter list editor.

 

The message  « Configuration File Receive OK » should appears :

syslog agent manager

Figure 6‑8: Result of a Get Filters operation on the agent “local agent.”

 

If the agent does not answer your request:

 

Verify your configuration parameters.
Do trace route or a ping to the agent to check that it is not a connectivity issue.
Do a telnet ipadd_agent :TCP_Port, if the connection is established and stop the manager to check your agent configuration (password and License number).

 

If a firewall is located between you and the agent add the following rules to it.

 

Source

Port

Destination

Port

Protocol

Action

Agent

>1023

Manager

5002

TCP

Permit

Agent

>1023

Manager

5001

UDP

Permit

Agent

>1023

Manager

514

UDP

Permit

Manager

>1023

Agent

5003

TCP

Permit

 

6.5.2     Agent remote control

 

Control

Explanation

Selection and setting of the current agent.

Uploads the filter list of the selected agent in the Filter List Editor window.

Sends to the selected agent the filter list currently displayed in the manager editor and applies it to the agent filter process.

Note:

Filters are immediately applied to the agent but are not saved in the agent default filter file.

The agent answers with an acknowledge message: “Agent Filters Send OK (delete tmp file).”

update filter list

Note:

If you ask for the agent’s current status, it should notify you that the current applied filter list is not saved.  

get agent status

 

agent status screen

 

Sends the Save command to the agent. The agent saves its current filter list into the default filter list file.

The agent status is returned .

syslog agent status window

 

Allows you to read the list of the current log files stored on the agent and to download if needed the selected files to your LoriotPro system. Select the file to download and click the Get Selected File  button.

zip syslog file

The list includes csv and gz file formats. The gz file format is archived, compressed csv files. The GZip remote file before download  checkbox allows you to force the agent to compress the file before download. The compression ratio  s approximately 15.

Warning: The interface allows you to download one file at once.

The dialog box displays asking you to specify the local directory where the log file has to be saved.

gzip syslog file

The Download process progression bar is displayed. You can cancel the transfer using the Cancel button.

upload zip syslog file

Note:

During the transfer, the LoriotPro software is totally operational for other tasks.

Once the transfer is done, the manager offers to display the Download file.

Note:

If you use the compression option, the manager software waits 60 seconds before starting the download. If this time is not enough for the agent to compress the required file, the download is cancelled. However, the agent still works on the compression of the file. The next time you open the list of remote log files you will see the new file in GZ format.

remote update filter

 

The agent can be managed remotely with a set of commands in this combo box.

Control

Function

Hide

Hides the agent GUI on the remote PC.

List

Retrieves and displays the list of log files.

Save

Saves the agent’s current applied filter list in the default filter list file on the agent.

Show

Unhides the GUI on the remote agent.

Start

Restarts the Syslog server daemon of the agent.

Status

Retrieves statistics of the agent.

Stop

Stops the Syslog server daemon of the agent.

 

6.5.3     Manager settings

 

The Manager parameters are located in the CollectorSyslogManager.ini file in the /bin directory of  LoriotPro. These parameters are similar to those used on the agent.

 

CollectorSyslogManager.ini

[ALARM]

Loriot_event_port 5001

loriot_ip_add 127.0.0.1

loriot_event_send 16001

collector_mode 1

collector_tcp_manager_server_port 5002

collector_tcp_server_password "admin"

 

Parameters are loaded when the Syslog Manager plug-in starts and cannot be modified dynamically. However, it is possible to modify the manager port from the manager GUI.

 

 

CIf you change the value you should click the Reset Manager Server button to apply this setting.

 

* Warning: If you change this parameter all agents must be reconfigured. Agents should be stopped, the CollectorSyslogAgent.ini has to be modified and agents restarted.

 

6.5.4     Managing filter list files

 

It is possible to save your filter list of each agent on the Manager using the Open Filters or Save Filters buttons.

 

 

Example:

 

You select a local filter list file, located on your local hard disk, edit it and push it to the agent.

 

Figure 6‑9: Local filter list file selection

A window informs you that the current filter list present in the Manager Editor will be cleared.

 

 

The new filter list is loaded in the editor window.

 

syslog filter list

Figure 6‑10: New filter list loaded in the editor window

 

The next step is to select the agent destination as shown in the screen capture below.

 

syslog window

Figure 6‑11 Agent Selection

Then apply Agent selection the filter list by clicking the Send/Apply button.

 

syslog list

Figure 6‑12: New filter list is applied

 

apply rules on syslog agent

Figure 6‑13: Acknowledgment by the agent of receipt of filters

 

If you look at the Filter Management status bar you see that the filter list is applied.

 

applied rules on syslog agent

Figure 6‑14: The agent has received the new filter list

The Status option is another way of checking that the filter file has been received and applied.

 

 

get remote syslog status

Figure 6‑15: The status option

 

syslog status

Figure 6‑16 : Display of agent statistics

The Save button forces the agent to save the filter list in its default file.

 

save syslog config

Figure 6‑17: Perform a save of the current agent filter list

 

The agent confirms the save operation.

 

syslog port information

Figure 6‑18 : The active filters are saved

6.5.5     Manager Filter List Editor

 

The Manager Filter List Editor has the same capabilities as the Agent Filter List Editor explained in the previous sections. Refer to the section “Agent Filter List Editor” for operations.

 

The Manager has an extended feature that allows it to use a text file called connectorsyslog-msg.txt  located in the LoriotPro /bin Directory.

 

This file contains predefined character strings allowing you to search for message strings easily. By default this file contains the set of messages sent by Cisco Pix Firewalls.

 

The syntax of this file is:

Reference :   comments

 

The colon “:” is used to separate the string from the comment.

 

To use it , simply click in the string field of the editor. A dialog box appears with available strings.

 

PIX error

 

You can manually edit the proposed examples.

 

PIX syslog error

 

The result is set in the filter rule.

 

PIX syslog

 

 

7       Syslog Files Browser

 

7.1       Introduction

 

The Syslog Collector Message Browser is dedicated to the browsing of Syslog type files. With it, you will be able to open Syslog log file generated by the Syslog Collector agent and display messages. An advanced interface allows you to filter the messages to display. The browser interface is embedded in the agent and also available on the LoriotPro console as a Plug-in

 

Among the filtering feature, you can select the messages from a date range, from a specific file, containing two specific strings of characters, by Syslog facility and/or level, by time stamp and date, by agent source name or address.

 

The list of displayed messages or a selection of them can be exported in a CSV format. A fine search could be done again on that file.

 

Overview of the user interface:

 

syslog browser

Figure 7‑1 The Syslog Message Browser Interface


7.2       Running the Browser

 

 

The Syslog Message Browser is integrated in the agent and run as a Plug-in in LoriotPro

 

To run the Syslog Message Browser on the agent select the Filter Management tab and then click on the Browser button.

 

The Syslog Message Browser is a Service Plug-in from LoriotPro. To start it, you should open the Service Tab of the LoriotPro workspace, then with the right click open the contextual menu

 

syslog manager

Figure 7‑2 Start the Syslog Message Browser Plug-In

Choose Add New Service and Select in the Following Screen the Syslog Collector Message Browser Plug-in.

syslog message browser

Figure 7‑3 The Plug-In selection window


7.3       Interface control

 

This chapter explains the role of each field and button of the interface.

 

Control

Explanation

syslog file

The File Selection Window allows selecting the file(s) on which the browse will be performed.

 

syslog file

Search For – These two field allow you to specify character strings to search for anywhere in the Syslog message text. TwThese two string can be set with operator OR and AND.

 

syslog file

Facility - Select the facility type of the Syslog message to filter. <-1 All> option remove filter on that condition.

 

syslog file

Level - Select the level of the Syslog message to filter. <-1 All> option remove filter on that condition.

 

syslog file

 

Update the File Selection list window and display new files

 

syslog file

 

Compress the selected file(s) in the gz format in the same directory. One file for each file selected file is created with the gz extension

syslog file

 

Archive the selected files compressed  in a single file with the extension zip

syslog file

Start Date – Specify the first date of the range for starting the search. If the date included in the log file name is after this date and before the End date the file will be browse.

 

syslog file

End Date - Specify the last date of the range for ending the search. If the date included in the log file name is after this date and before the End date the file will be browse.

 

syslog file

File Header - Allow you to select the file name beginning. This heading is the name of the file set in the Syslog Filter configuration. The end of the name is automatically assigned with the date by the agent.

 

syslog file

Scan File on Date Range – The scan is performed based on the files name containing a date in the specified range and display the Syslog messages matching the filtering conditions.

 

syslog file

Scan Selected File – Scan the files selected in the Files Pane and display the Syslog messages matching the filtering conditions.

 

syslog file

Scan All Files – Scan all the log files located in the current selected directory display the Syslog messages matching the filtering conditions.

 

syslog file

Stop Scan

 

 

Compress File(s) (Zip format) – Compress the selected file(s) in ZIP format

 

 

Compress File(s) (GZ format) - Compress the selected file(s) in GunZip format

 

syslog file

Export All Messages – Export all the displayed messages to a .csv formatted file.

 

syslog file

Export Selection – Export only the selected messages to a .csv formatted file.

 

 

 

Clear Messages – Clear all message from the windows

 

syslog file

Clear Selection – Clear only the selected messages from the windows.

 

 

 

 

 

The status bar display the current scanned file,syslog file the current processed line in the file and the current messages found according to the filter conditions.

 

syslog file

 

Messages Pane description

 

The message pane contains for each message the following information :

 

TimeStamp

The time at which the message has been created

Date

The date and time at which the message has been received by the Syslog Collector Agent

Agent

Agent the Agent IP address

Facility Level

The Facility and the Level of the Syslog Message

Message

The contents of the message

File

The file  which the message has been found

Line

The line number in the file where the message has been found

EventNumber

The number of the LoriotPro Event associated to that message if any.


7.4       How to use the Browser

 

Select first in the File Selection window the directory where your log file have been stored.

If you are using the Browser on LoriotPro the file are located in the directory where you uploaded them from the agent.

 

*Warning: The Browser can’t pick up files directly on the agent. You should use the Syslog Collector Manager first to retrieve file form the agent.

 

When your have selected the right directory, you have multiple options.

 

  1. You can browse all the files in the selected directory and display them
  2. You can browse only the selected file in the selected directory
  3. You can browse the file with dates included in a range. The date range is checked according to the date specified in the file name and not on the date of the file maintained by the operating system.

 

A log file is composed of a header and a date ( ex: Log_may_22_2003.csv)

 

When browsing the file the application can filter messages on multiple criteria.

 

You can specify two different strings of characters that should be in the message. It could be any field, a part of the message, an IP address, a Date…

You can select the Facility type of the SYSLOG message

You can select the Level type of the SYSLOG message

 

The screen shot below is an example of an advanced filter.

 

The browse is performed on the files located between a date ranges and heading by “log”. Within this selection, all the messages containing the “Paris-NewYork” string are displayed.

 

Figure 74 Example of Browser applied filters

After this, it is possible to select one or more message line and export them to a .csv file.

 

 

                                                          


www.loriotpro.com
Copyright © 2004 LUTEUS SARL. All rights reserved. This documentation is copyrighted by LUTEUS SARL. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying or otherwise, without the prior express written permission of LUTEUS SARL