Install Elasticsearch, Kibana 4 , fluentd (Opensource splunk) with syslog clients

So used splunk some times but it has its limit (money) so now Im testing

1. Java

first install java on your server. Get java from here

 yum localinstall jdk-8u25-linux-x64.rpm

And install it on your server.

2. Elasticsearch

Get it from here I installed the rpm and run
yum localinstall elasticsearch-1.4.0.Beta1.noarch.rpm

I hade to make some settings in this file my vps only hade 512m

vi /etc/sysconfig/elasticsearch
/etc/init.d/elasticsearch start

So moving on

3. Kibana 4

Download kibana from here

cd /var/www/html
tar zxvf kibana-4.0.0-BETA1.1.tar.gz
mv kibana-4.0.0-BETA1.1 kibana
chown apache:apache -R kibana

4. Install fluentd

curl -L | sh

Install gems needed

yum install libcurl-devel
/usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-elasticsearch

open this file and have only this in the file

vi /etc/td-agent/td-agent.conf
<match td.*.*>
 type tdlog
 apikey YOUR_API_KEY
buffer_type file
buffer_path /var/log/td-agent/buffer/td
 type syslog
 port 42185
 tag syslog
type forward
<match syslog.**>
type elasticsearch
logstash_format true
flush_interval 10s # for testing


Restart the agent

/etc/init.d/td-agent restart

Time for sending some logs to the server

5. Client

in rsyslog open the file /etc/rsyslog.conf and add at the buttom

*.* @

  3 comments for “Install Elasticsearch, Kibana 4 , fluentd (Opensource splunk) with syslog clients

  1. April 5, 2014 at 12:07 am

    Nice Post, I was comparing logstash with fluentd as a replacement to my Splunk install and your post helped. Thanx!

  2. Sarveshwar Singh
    May 13, 2016 at 5:37 am

    In my early days of Linux(10 days), your post helped me a lot. Thanks!

    I tried to install step [4. Install fluentd] at first. Got following error

    Building native extensions. This could take a while…
    ERROR: Error installing fluent-plugin-elasticsearch:
    ERROR: Failed to build gem native extension.

    gcc was also required,

    yum install libcurl-devel

    yum -y install libcurl-devel gcc

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.