Logstash

Logstash is the open source version of splunk, using ElasticSearch as its search engine.

Installation

For detailed information, consult logstash's tutorial at http://logstash.net/docs/1.1.10/tutorials/getting-started-centralized

Elastic Search

Download and extract the archive. Ensure Java is installed, then run elastic search:

wget jre-7u21-linux-x64.rpm
rpm -ivh jre-7u21-linux-x64.rpm

~/elasticsearch-0.20.6//bin/elasticsearch -f

Redis

This may not be necessary if we are going to be using syslog to log everything

Download and compile redis:

yum install make gcc

wget redis-2.6.12.tar.gz
tar -xzf redis-2.6.12.tar.gz
cd redis-2.6.12
make
# You can run `make install`, or just run the binary from the src directory
# like `./src/redis-server` if you decide to not run the following two lines:
make install
redis-server

Logstash

Download logstash

wget http://logstash.objects.dreamhost.com/release/logstash-1.1.10-flatjar.jar

note: (for 1.1.10) if you want to use logstash's web ui, you need to use the monolothic jar file at http://build.logstash.net/job/logstash.jar.daily/237/artifact/build/logstash-1.1.10-monolithic.jar note2: if you want to use what that google groups person said, you also need grok. see https://github.com/jordansissel/grok/blob/master/INSTALL

You will then need to configure logstash by creating a config file, then running the jar file with -f config.cfg

Kibana

Kibana is the nice looking web UI that looks similar to splunk.

More info at http://kibana.org/intro.html Get it at https://github.com/rashidkpc/Kibana.git

Integration with Clients

https://groups.google.com/forum/#!msg/logstash-users/X6kNHU0alBg/j95HZkTLo-EJ