Take the S(ys)LOG out of logging !

Regularly I have a requirement to provide adhoc monitoring of a device or application and the tool of choice is normally Syslog. Indeed some hardware providers and other produce their own tailored syslog tools to give a more useful interpretation of the information output by their product. A case in point is Draytek – with a syslog daemon for the Vigor range of adsl , cable and sdsl routers. It breaks the syslog messages down and displays them in categories which are of use in analysing performance or functional problems.

However what I’m writing about today is the use of a syslog daemon for generic syslogging and integrating information from more than one source with a final destination of an odbc database.

Some background on syslog though first. If you are interested in in-depth details about Syslog, I would strongly suggest you to go through RFC: 3164.

Syslog is a protocol that allows a machine to send event notification messages across IP networks to event message collectors – also known as Syslog Servers (services) or Syslog Daemons. In other words, a machine or a device can be configured in such a way that it generates a Syslog Message and forwards it to a specific Syslog Daemon (Server).

Syslog messages are based on the User Datagram Protocol (UDP) type of Internet Protocol (IP) communications. Syslog messages are received on UDP port 514. Syslog message text is generally no more than 1024 bytes in length. Since the UDP type of communication is connectionless, the sending or receiving host has no knowledge receipt for retransmission. If a UDP packet gets lost due to congestion on the network or due to resource unavailability, it will simply get lost – nobody would know – you have been warned !

What is Syslog Daemon or Service ?
A Syslog Daemon or Server is an entity (program) that would listen to the Syslog messages that are sent to it. You cannot configure a Syslog Daemon to ask a specific device to send it Syslog Messages. If a specific device has no ability to generate Syslog Messages, then a Syslog Daemon cannot do anything about it.

So what do I do to get syslog working ?

First select a syslog server or daemon for the platform you will run monitoring on and select some hardware or software that can generate syslog compliant messages. In this case I’m going to talk about a specific syslog server – Kiwi Syslog (www.kiwisyslog.com) where you will find a free version as well as a paid for version. The application that I’m going to use is INM from www.intellipool.com which I use to monitor a range of environmental and server based metrics such as disk space. The first thing to do is install the Kiwi Syslog server and start the service. You can use the inbuilt test tool to ensure that the server is receiving the syslog messages before you activate an application to send messages.

Once you have been able to send the test message – activating INM to send syslog messages is simply a matter of activating the option under program settings , give it the IP address of the syslog server and the port number if changed from the default 514 and then waiting for status messages to appear in syslog.

The next stage is to set up an action on the syslog server which is going to insert the records into an odbc data source. You need to set up the data source as a system dsn and I would suggest that the easiest would be MS Access if you have that install on the machine that is syslogging. If you don’t have that – MYSQL is a little more complicated but will do if you require something more substantial (www.mysql.com for a free version). Assuming that you have Access – create the data source then open the setup menu in Kiwi Syslog. Click on actions – add an action and after selecting Log to ODBC Database as the type of action – enter the DSN name you gave to ODBC datasource (in the form dsn=yourdatasourcename;) – choose Kiwi Access format – pick a name for the table – and as a nice usuability feature – select create table to have Kiwi Syslog create that table with the appropriate field layout for you !! Apply and ok to save the action. Wait until some more syslog messages have been processed – or again use the File / Send test to localhost option and open the Access database to see your logged messages.

This gives you an overview of this process – Kiwi Syslog has a vast range of features which are listed below. Another article will take us deeper into processing the raw syslog messages into something more meaningful and useful.

Outline of Kiwi Syslog features (not exhaustive)

Kiwi Syslog Daemon receives syslog messages from network devices, and displays them in real-time. Actions can be performed on received messages and messages can be filtered by host name, host IP address, priority, message text or time of day.

Syslog messages can then be processed using events like:

Display the message in the scrolling window
Logging the message to a text file
Forward the message to another syslog daemon
Log to an ODBC database
Log to the NT Application Event Log
E-mail the message to someone via SMTP
Triggering a sound alarm
Running an external program
Send an SNMP Trap message
Page someone using NotePager Pro