Administrator Handbook TOC

                                    

Syslog Collector

Administrator Handbook

 

 

 

 

 

 

syslog collector logo

 

 

  

 

 

Disclaimer

 

Luteus SARL makes no representations or warranties with respect to the contents or

use of this handbook, and specifically disclaims any express or implied warranties of

merchantability or fitness for any particular purpose.

Further, Luteus SARL reserves the right to revise this publication and to make

changes to its content, at any time, without obligation to notify any person or entity of

such revisions or changes.

Further, Luteus SARL makes no representations or warranties with respect to any

Luteus software, and specifically disclaims any express or implied warranties of

merchantability or fitness for any particular purpose.

Further, Luteus SARL reserves the right to make changes to any and all parts of

Luteus Software, at any time, without any obligation to notify any person or entity of

such revisions or changes.

 

Copyright 2003 Luteus SARL. All rights reserved. No part of this book may be

reproduced, photocopied, stored on a recovery system, or transmitted without

the express written consent of the publisher.

 

Luteus SARL

Bâtiment : « Le Sextant »

ZI de Moissy-Cramayel

462 rue Benjamin Delessert

BP 83

77554 Moissy-Cramayel

FRANCE

 

 

 

 

 

 

 

Table of contents

 

4      The Syslog Collector architecture. 4-11

6      Syslog Collector Manager 6-39

 


 

 

List of Figures

 

Not all illustrations, logos, and diagrams used in this manual may correspond exactly to the software, which, by its very nature, is always changing. They are only there to document and illustrate the concepts of the manual but may not under any circumstances be used for reference.

 

 

1       About this guide

 

This manual has been made as an online help file available from the software or from the website www.loriotpro.com. The order of the chapters is designed to help you to quickly configure your LoriotPro software and supervise your Information System (IS) in a typical manner.

 

1.1       Conventions used in this guide

 

This guide uses a special format to show the path to a specific menu option. Rather that specifying all menu titles, we will use the greater than sign '>' and italics.

 

Example : Programs>LoriotPro>LoriotPro

 

Links to websites and e-mail addresses are shown in blue:

 

http://www.loriotpro.com/

sales@loriotpro.com

CThis icon calls your attention to a note or a tip.

 

* This icon calls your attention to a possible trap.

1.2       How to get technical support

 

For technical support, send an e-mail directly to support@loriotpro.com. Specify your problem and the context of the problem in the e-mail, and we will try to help you as soon as possible.

 

1.3       Web site

 

There are many tips and tricks available on our website  http://www.loriotpro.com

 

.

 

2       Introduction to LoriotPro

Users accustomed to our LoriotPro product can skip this chapter and go directly to the next chapter

2.1       Overview of LoriotPro

With LoriotPro, you possess a tool with the power of a control tower for monitoring your computing resources that guarantees availability and performance to your users.
snmp control tower
Your computing resources, data and applications, servers, workstations, switches and network routers all constitute your IS's infrastructure and can be supervised thanks to the management protocol SNMP (Simple Network Management Protocol), the Internet standard. LoriotPro takes advantage of this protocol in its smallest details to help you be effective and accurate in your daily supervising tasks. LoriotPro does not stop there; it extends its control by using various protocols such as ICMP (Internet Control Message Protocol) and HTTP for WEB server monitoring, for example. Monitoring is made possible thanks to an optimized use of the Windows graphical interface as well as Internet browsers. The direct display of your computing resources and of their performance status in the form of color-coded icons and alert messages draws your attention to any fault situations.


Among its multiple features:


Directory

Helps you to manage and order your devices in the way you want. Shows devices as icons and displays in real time their current operating status. The entry point for resource inventory with advanced search capabilities and HTML reporting.

 

Fault Management

Collects, stores and presents alarm information for reporting. Can trigger actions, alarms, e-mail, process, and so on, based on advanced filtering conditions. Supports Syslog and Trap event messages. Display in one click the health of a portion of your network.
 

Configuration Management

Allows remote configuration of any IP device through the MIB (Management Information Base) and the SNMP SET command.

 

Topology maps

Draws a map with devices (IP nodes) and dynamically displays their operating status. Displays huge Networks on a condensed Map.

 

Auto Discovery

Discovers network nodes and adds them to the Directory and the Map. In-depth configuration of this module provides a total control of the discovery process.

 

Web Enabled

Supports thin client Web access, allowing network to be monitored from any location.

 

Protocol Support

Supports Simple Network Management Protocol (SNMP), Syslog, ICMP.

 

MIB Management

MIB browser and compiler, generation of reports in HTML format. MIB scripter creates SCI files that automate any set of SNMP requests.

 

Detect Device MIB Support

Detects what MIBs are supported by a device, provide the MIB file reference.

 

Scripting Language

Provides a scripting language that automates the generation of reports in HTML format.

 

IP Scanner

Scans a range of IP addresses and detects the presence of hosts and SNMP agents.

 

Database Interface

ODBC (Open Database Connectivity) interface provides a way of storing events and more. Examples with MYSQL database are described.

 

Customizable GUI

Toolbars and menus can be customized, proprietary Skins can be set.

 

Plug-In Technology

Plug-ins within LoriotPro are dedicated tasks that perform SNMP data processing for a specified host or a group of hosts. Numerous plug-ins are provided by default but this technology is also opened and allows you to develop additional modules that manage your specific needs. Wizard tools are provided for Visual C++ and multiple examples are provided with their source code.

 

Online Help

Provides access to extensive online documentation.

 

2.2       Introduction to LoriotPro Plug-In

A Plug-In is a separate code module that behaves as though it is part of the LoriotPro software. Plug-ins are used for example to perform MIB object values acquisition via SNMP on a host or a group of hosts, to process them, to display them in various ways and eventually to generate alarms.

There are three types of plug-ins:

Direct Plug-In: Used for processing the data of a specific Directory Object. Direct plug-ins are started one by one and work independently. They are used to perform one SNMP request or a set. These plug-ins have the file extension <.lp>.

Directory Plug-In: Allows you to associate an application to a Directory host. These plug-ins are loaded and saved within the Directory. They allow you to perform scheduled tasks or repetitive tasks (graphics, polling, etc.). These plug-ins extend the capacity of LoriotPro by creating an Active Directory. These plug-ins have the file extension <.slp>.

Service Plug-In: Is loaded as a complement of the LoriotPro services. Service plug-ins are not linked to a host or to any object of the Directory. They are used to extend the global features of LoriotPro.

2.3       Plug-In limitations

 

The use of plug-ins can extend the features of LoriotPro and hundreds of them can run simultaneously and process management data. The downside of that is the increasing need of CPU on the LoriotPro system and the increasing bandwidth used by the network traffic converging on the LoriotPro system. This statement is at the origin of the Collector concept that distributes the process load on multiple agents and reduces network traffic by setting the collector agent closest to the source of traffic.

 

3       Introduction to the Collector concept

 
The goal of the Collector architecture designed by LUTEUS is to provide a scalable solution for handling huge quantities of management information and to help the administrator to classify, analyze and process them. 
 
The product has been designed based on the following issues:
 
·       The main issue that system and network managers have to face is the collection and processing of huge quantities of management information.
·       They need to have detailed information of what is happening in their system and network devices by turning on in-depth logging and alarm facilities. 
·       They want to receive the critical information immediately but also want to retrieve non-critical ones at a later time.
·       They want to have a centralized system that can collect information files without using too much network bandwidth.
·       They want to manage their infrastructure from a centralized manager.
·       They want to have access security and control of the management solution.

 

The concept of Collector in the LoriotPro management system architecture is based on two components:

 

  1. The Collector agent software running on Microsoft Windows workstations. These agents are dedicated to the collection of the supervision information sent by the System and Network devices located in a predefined area.

 

  1. The Collector Manager software running on top of our LoriotPro supervision software. This manager has to manage the agents avoiding any human intervention directly on the agent.

 

Collector agents are responsible for:

 

Filter rules applied on the agent can be set on each agent either from the agent GUI or from the Collector Manager from a central location.

 

 

4       The Syslog Collector architecture

 

The Syslog Collector Solution uses the predefined concept of Collector applied to the management of Syslog messages. The Syslog system provides the transport and storage mechanisms for event notification messages, in the form of Logs. Syslog is a de-facto standard defined by RFC3164 for logging system events. It was commonly and initially used by Unix systems, later on by network devices (routers and switches) and more recently by firewalls.
The Syslog Collector architecture is built around two components: the agents and the manager. The manager is a program running exclusively on our LoriotPro supervisor solution.

 
syslog concept
 

Syslog messages are collected by agents called, in our terminology, “Syslog Collector Agents.” Agents are designed to collect a large throughput of Syslog messages and to process them according to advanced filtering rules. Filtered messages can then be displayed on a viewer, the agent taking on the role of a simple Syslog server. Messages can be stored locally in files or forwarded to the central management system. Critical messages can be sent to the centralized management system either as LoriotPro proprietary-formatted event messages or as Syslog-formatted messages. Agents can be cascaded to build a hierarchical architecture of Syslog message relays.

Agents can be used as a standalone solution and act as a Syslog server or Syslog relay. Our LoriotPro Network Management System (NMS)
and the Syslog Manager are not necessary in this case. Filtering rules can be defined from the Agent GUI and applied. Actions taken on conditions defined in
the filtering rules can be displayed in a viewer, stored in files or forwarded to another Syslog server. 
 

The Syslog Collector Manager is responsible for the management of the agents from a centralized position. Filtering rules are defined on the manager and pushed to the agent. The manager is also able to retrieve a filter rule previously loaded onto an agent. Filtering rules are stored in local text-only files.

The manager is also able to upload Syslog files previously stored on the agent. The Syslog files can be compressed on the fly during uploading, sparing precious bandwidth of WAN links or on-demand links. The manager works on top of our LoriotPro NMS as a Plug-In Service.

As we have stated previously, the messages sent by the Syslog Collector Agents can be in the LoriotPro event format. The LoriotPro Event Manager receives them and processes them. They are first displayed in the Event Log window and if necessary, they trigger actions based on predefined conditions. Actions can send messages, start programs, play sounds, etc.

 

5       Syslog Collector Agent

5.1       Agent Installation

 

5.1.1     Hardware and Software requirement

 

 

The Syslog Collector Agent should be installed on a Microsoft Windows workstation. We highly recommend using a Windows 2000 professional version. In this environment the agent can run as a service at start up.

 

5.1.2     Preliminary checking

 

The agent will hook the UDP Syslog port 514 for its operation. You must check beforehand that no other program is already using this port.

 

From a DOS session, type:

 

C:\Netstat  -na

 

Active connections

 

  Proto  Local Address         Remote Address   State

  TCP    0.0.0.0:8020           0.0.0.0:0              LISTENING

  TCP    0.0.0.0:5002           0.0.0.0:0              LISTENING

  TCP    0.0.0.0:5003           0.0.0.0:0              LISTENING

  TCP    10.33.10.130:139       0.0.0.0:0              LISTENING

  UDP    0.0.0.0:514            *:*                   

  UDP    0.0.0.0:2593           *:*                    

  UDP    0.0.0.0:5001           *:*                   

  UDP    0.0.0.0:1421           *:*                   

  UDP    0.0.0.0:1422           *:*                   

  UDP    0.0.0.0:1440           *:*                   

  UDP    10.33.10.130:137       *:*                   

  UDP    10.33.10.130:138       *:*                   

  UDP    10.33.10.130:8888      *:*                   

 

 

The line  <UDP    0.0.0.0:514            *:*>  shows you that a Syslog server is already running on that machine. You should find this program or service and get rid of it before the agent installation.

 

If you want to use the agent on a system where LoriotPro is already installed, you have to modify the  lalarm.ini  file located in the /bin directory of LoriotPro.

 

Lalarm.ini

[ALARM]

alarm_port 5001

syslog_port 514

 

Replace the  line syslog_port 514 by syslog_port 0. This will suppress the service at the next restart of LoriotPro.

 

5.1.3     Starting the installation

 

From the CD ROM or from the hard disk start the program:

 

loriotprov200b136sp0-collector-xx.exe

 

CThe xx characters in the filename specify the current release number and are subject to change.

 

syslog license

Figure 5‑1 : License agreement

syslog installation

Figure 5‑2 : Installation directory choice

* If you have LoriotPro installed on this machine, keep the same directory as where LoriotPro is installed. You must stop LoriotPro before you continue to the agent installation.

 

By clicking the Start button, files will be copied onto your disk.

 

* If you proceed to an installation of the agent over a previously installed agent you must stop the agent and/or remove it from the list of Windows services .

 

syslog agent installation

 

 

syslog collector installation

 

The installation is completed, click the OK button

 

The license file is displayed and the program icons are set in the Windows 

Syslog file

 

If LoriotPro is installed on this computer the complementary Syslog icons will be placed in the same Windows (directory).

 

syslog license agreement 

 

If the following message appears, answer NO :

 

 

This message warns you about an already running or installed agent. You must stop it and uninstall it before continuing.  

 

If your agent runs as a Windows Service, uninstall it with the MS DOS program Uninstall Syslog Collector Service. When done you can restart and proceed to the agent installation.

 

The agent software is located in the bin/collector/syslog directory as shown below:

 

syslog collector directory

 

5.2       Agent configuration

 

Agents are  by default configured to communicate with a Syslog Manager located on the same computer. In a distributed architecture these parameters have to be modified.

Use Notepad to edit the CollectorSyslogAgent.ini file located in bin/collector/syslog directory.

 

CollectorSyslogAgent.ini.

[ALARM]

syslogd_port 514

max_log_view_lines 50

collector_mode 0

hide_log_view 0

loriotpro_ip_add 127.0.0.1

loriotpro_event_send 16001

Loriotpro_event_port 5001

collector_tcp_manager_server_ip 127.0.0.1

collector_tcp_manager_server_port 5002

collector_tcp_agent_server_port 5003

collector_tcp_agent_server_timeout 5000

collector_tcp_server_password "admin"

 

 

5.2.1     Configuration of communication parameters

 

By default you should find these two lines:

 

loriotpro_ip_add 127.0.0.1

collector_tcp_manager_server_ip 127.0.0.1

 

Replace both IP address 127.0.0.1 by the IP address of your LoriotPro supervision System:

 

Example:

 

loriotpro_ip_add 193.2.2.2

collector_tcp_manager_server_ip 193.2.2.2

 

 

To exchange information, agents and the manager use TCP ports. Agents listen on TCP port 5003 and the manager listens on TCP port 5002.

CCheck by using the  DOS netstat –na command that these two TCP ports are not already hooked by another application.

 

If another application is using it, you can set another value for the agent and/or for the manager.

 

Example:

 

collector_tcp_agent_server_port 678

collector_tcp_manager_server_port 679

 

 

* Warning: If you make the modifications above, you will have to modify the manager parameters in the agent configuration too.

 

Agents and the manager use a proprietary secured protocol to communicate. A password is used to authenticate the agent to the manager and reciprocally. This password is never sent but used to generate a session key by using an MD5 hashing code.

 

The following line of the configuration file is used to declare this password:

 

collector_tcp_server_password admin

 

* Warning: If your password contains spaces, set your password between quotes.

 

Example:

 

collector_tcp_server_password “admin 001”

 

The following diagram shows the functional architecture of the Syslog Collector solution. For clarity’s sake, only one agent is represented.

syslog server and loriotpro communication

Figure 5‑3: Schema of the communication between an agent and the manager.

 

5.2.2     Syslog server daemon configuration, display options

The agent is first a Syslog Server or relay. It should be able to receive Syslog messages coming form network or system devices. The default port for receiving Syslog messages is UDP 514:

 

syslogd_port 514

 

The agent GUI has an integrated viewer of filtered Syslog messages. By default the viewer displays the last 50 filtered messages. You can change this value by modifying the configuration line:

 

max_log_view_lines 50

 

* Keep this number as low as possible; displaying messages in the viewer is CPU time consuming and could generate a loss of incoming messages in high throughput conditions. If the viewer agent is not used locally do not display messages and hide the GUI from the screen.

5.3       License installation

To work, the agent needs a registered license which is stored in the CollectorSyslogAgentLicence.ini located in the bin/collector/syslog directory. This file is provided to you when you buy a license.

By default the agent is provided with a license key limited to 30 days.

During this 30-day trial period you can modify the license number (second line) and install as many agents as you want.

 

* Warning: Each agent should have a different license number. The manager will refuse to manage agents having the same license number. The license number is used in the authentication process.

 

5.4       Starting the software agent

 

5.4.1     Running the agent as a service

 

It is possible to install the agent as a Windows service with the provided program CollectorSyslogService  located in the bin/collector/syslog directory.

Specify option  -i.

 

CollectorSyslogService –i

 

C:\Program Files\LoriotPro\bin\collector\syslog>collectorsyslogservice -i

 

LoriotPro V2.00 Beta NT/2000 service Installer v1.3

Copyright (C) 2003 Luteus SARL

====================================================================

= WARNING : This tools permit only to automatiquely start the      =

=           LoriotPro Syslog Collector like one service at startup.=

[Syntax]============================================================

= For Installing the module like a service use :                   =

=    CollectorSyslogService -i                                     =

= For UnInstalling the module use :                                =

=    CollectorSyslogService -d                                     =

====================================================================

 

 

Create [c:\run_CollectorSyslogService.bat] file Sucessfully

 

LoriotPro Syslog Collector Service Installed Sucessfully

 

Press a key to continue

CThis tool creates the file c:\run_CollectorSyslogService.bat allowing the program to run as a service.

 

CThe line hide_log_view 0 in the CollectorSyslogAgent.ini file with option 1 unhides the agent service at startup.

 

hide_log_view 1

 

Once the service is installed, you can start the Administration tools and manage the service from there.  

 

 

Figure 5‑4: NT/2000 Administration Tools

 

In the service list you will find:

 

LoriotPro Syslog Collector Agent Service

 

The service is configured by default  to start automatically at each Windows startup.

 

syslog agent

Figure 5‑5: NT/2000 Service Administration program

 

However, just after the installation process of the service either restart Windows or do a manual start form the Service Administration Windows above.

 

Select the Syslog service by using a single click and choose the Properties option on the contextual menu.

 

syslog windows service

Figure 5‑6: Control window of an NT/2000 agent service

 

Click on the Start button.

 

The service start and the agent GUI are displayed. If you selected the hide_log_view 1 in the  CollectorSyslogAgent.ini file, the agent GUI disappears after a few seconds.

Only the Syslog manager is able to unhide it with the remote control protocol.

 

syslog filter rules

Figure 5‑7: Main windows of the Syslog Collector agent GUI

 

If the message “Prob to generate one SOCK_STREAM xxx for CollectorSyslog is displayed, the agent is having problems communicating with the manager. Check the TCP port configuration on both sides, agent and manager. This message can appear if you stop and start the agent program successively and too fast.

 

 

syslog socket

Figure 5‑8:The TCP port is already used

 

Remember this TCP port number can be changed in CollectorSyslogAgent.ini file:

 

collector_tcp_agent_server_port 5003

 

*Warning: If you change the agent TCP port you have to change the setting on the manager to the same port number for that agent.

 

 

If the message “Error on bind() Syslog, error code=10048“ is displayed, you probably have another Syslog server running that hooks the UDP port 514. Stop it and make sure that it is not started by default (as a service) the next time you restart the computer.

 

Figure 5‑9: The UDP port on the Syslog is already being used.

 

5.4.2     Uninstall the agent service

 

Before uninstalling the agent service, you should stop it. Use the Service Administration tool available in the Control Panel. Select the Syslog agent service and click the Properties option of the contextual menu.

 

start syslog

Figure 5‑10: Service control windows

syslog service control

 

When done, start the program CollectorSyslogService.exe with option –d.

 

CollectorSyslogService –d

 

C:\Program Files\LoriotPro\bin\collector\syslog>collectorsyslogservice -d

 

LoriotPro V2.00 Beta NT/2000 service Installer v1.3

Copyright (C) 2003 Luteus SARL

====================================================================

= WARNING : This tools permit only to automatiquely start the      =

=           LoriotPro Syslog Collector like one service at startup.=

[Syntax]============================================================

= For Installing the module like a service use :                   =

=    CollectorSyslogService -i                                     =

= For UnInstalling the module use :                                =

=    CollectorSyslogService -d                                     =

====================================================================

 

 

LoriotPro Syslog Collector Service UnInstalled Sucessfully

 

Press a key to continue

5.4.3     Running the agent as a standard program

 

To start the agent, you simply run CollectorSyslogAgent.exe located in the  bin/collector/syslog Directory.

 

The main screen is displayed.

 

syslog message log

Figure 5‑11: Agent main screen,  the View Filtered Syslog tab

5.5       Agent control

The agent software has two tabs and five control buttons. The Button bar allows you to control agent operations.

 

5.5.1     Control buttons

 

Figure 5‑12 : Control buttons

 

Button

Explanation

Close the agent program.

Restart the agent after a stop.

Note:

All displayed Syslog messages are cleared.

Stop the agent. Incoming Syslog messages are not processed, because the agent is in sleep mode.

Restart the agent if stopped with the Pause button.

Note:

All displayed Syslog messages are not cleared.

Pause the agent and allow the  Resume action.

 

5.5.2     License Information

 

Your license number is displayed in the main window within the communication parameters. In the example below the agent is installed on the same PC as LoriotPro and  is running in evaluation mode.

Figure 5‑13: License information example

 

5.5.3     Syslog message viewer

 

view syslog

 

This window displays the messages allowed by filtering rules. According to filter rules it is possible to have the same messages with different colors

 

*Warning: All received messages are not displayed by default.

 

 

Column

Explanation

Date

The arrival date of the syslog Message

Time

The arrival time of the Syslog message in hours:minutes:seconds

Priority

The “facility“ field and  “level” of the Syslog message

Hostname

The IP address of the device that sent the Syslog message

Message

The message itself, a character string

Note

The facility filed is not displayed, information <xxx>.

 

The Facilities and Severities of the messages are numerically coded with decimal values.  Some of the operating system daemons and processes have been assigned Facility values.  Processes and daemons that have not been explicitly assigned a Facility may use any of the "local use" facilities or they may use the "user-level" Facility. Those Facilities that have been designated are shown in the following table along with their numerical code values.

 

Numerical Code               

Facility   

           0            

kernel messages

           1            

user-level messages

           2            

mail system

           3            

system daemons

           4            

security/authorization messages

           5            

messages generated internally by Syslog

           6            

line printer subsystem

           7            

network news subsystem

           8            

UUCP subsystem

           9            

clock daemon

          10            

security/authorization messages

          11