How to set up Log.io?

Logs are very useful as it monitors and provides important information about the program such as activities, executions, real-time data etc.

Log.io is a log management tool that monitors the system and provides real-time information on the system logs. It is a simple and effective application built on top of Node.js and Socket.io. Log.io is not used to store, archive or index log messages, or provide information on the historical or aggregate analysis.

Now, let’s install Log.io and see how it works. Before the installation, let’s first understand the architecture of Log.io

Harvesters monitors the log files and look for new changes,  sends messages about the new log files to the server, which are then broadcasted to the web browsers. Log messages are tagged with a stream, node, and log level information based on user configuration.

Log.io has no persistence layer i.e. file changes are informed via inotify, and log messages flow from harvester to server to web browser via TCP and Socket.io, respectively.

Steps to install and getting started with Log.io

1. Install latest version of Node.js

   [js]
   sudo add-apt-repository ppa:chris-lea/node
   sudo apt-get update
   sudo apt-get install nodejs
   [/js]

2. Install rpm

   [js]
   curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash –
   [/js]

3. Installing Log.io

   [js]
   sudo npm config set unsafe-perm true
   sudo npm install -g –prefix=/usr/local log.io
   [/js]

4. All the files will be stored inside /usr/local/lib/node_modules/log.io
5. Now going inside ~/.log.io directory
   • 3 new conf. files would be present
     • harvester.conf configuring server name and log files path

       [js]
       exports.config = {
       nodeName: "application_server",
       logStreams: {
       syslogs: ["/var/log/syslog"]
       },
       server: {
       host: ‘0.0.0.0’,
       port: 28777
       }
       }
       [/js]

     • log_server.conf

       [js]
       exports.config = {
       host: ‘0.0.0.0’, //Listens to every machine
       port: 28777
       }
       [/js]

     • web_server.conf

       [js]
       exports.config = {
       host: ‘0.0.0.0’,
       port: 28778,

       /*
       // Enable HTTP Basic Authentication
       auth: {
       user: "admin",
       pass: "1234"
       },
       */

       /*
       // Enable HTTPS/SSL
       ssl: {
       key: ‘/path/to/privatekey.pem’,
       cert: ‘/path/to/certificate.pem’
       },
       */

       /*
       // Restrict access to websocket (socket.io)
       // Uses socket.io ‘origins’ syntax
       restrictSocket: ‘*:*’,
       */

       /*
       // Restrict access to http server (express)
       restrictHTTP: [
       "192.168.29.39",
       "10.0.*"
       ]
       */
       }
       [/js]

6. Now we can start log.io-server and log.io-harvester

   [js]
   nohup log.io-server &
   nohup log.io-harvester &
   [/js]

7. Now our log.io-server will run on 28778 and can be accessed by browsing localhost:28778The above steps will help you to set-up a colorful logging dashboard for real-time current logs within 5 minutes.

Here is another blog about using Amazon Elasticsearch with Amazon Cloudwatch to setup centralized logs, which can be done by using Amazon ElasticSearch and CloudWatch services.