Beside monitor topic, Log also is a important issue we need to concern. In this post, I just mention the way how to centralize Docker Logs using FluentD, Elasticsearch and Kibana
Try not to become a man of success, but rather try to become a man of value
Secenario
We will install FluentD, ElasticSearch and Kibana in the same machine.
- FluentD : Collect and Transfer log data to Elasticseach
- Elasticsearch: Store and indexing log data to support searching/filtering log data
- Kibana: A web view supports you search/filter and virtualize the log data
Prerequisites
- We have a machine installed Ubuntu 14.04 with IP 192.168.1.191
- We already installed Docker, Wget
Now, I will show you step by step to get stated to centralize the log data with FluentD
Elasticsearch
Download:
|
1 |
wget https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.1.1/elasticsearch-2.1.1.tar.gz |
You should check the latest version at https://www.elastic.co/downloads/elasticsearch
Uncompress:
|
1 |
tar xvxf elasticsearch-2.1.1.tar.gz |
Run:
|
1 2 |
cd elasticsearch-2.1.1 sudo bin/elasticsearch |
Or run as daemon:
|
1 |
sudo bin/elasticsearch -d |
Now, we have Elasticsearch run on port 9200.
FluentD
Add the lines below to /etc/security/limits file:
|
1 2 3 4 |
root soft nofile 65536 root hard nofile 65536 * soft nofile 65536 * hard nofile 65536 |
Open new terminal and type command below, make sure the output is correct:
|
1 2 |
$ ulimit -n 65535 |
Install FluentD (uses Treasure Data):
|
1 |
curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-trusty-td-agent2.sh | sh |
For other Ubuntu version, please read: http://docs.fluentd.org/articles/install-by-deb
Now, we need to install Elasticsearch Plugin for FluentD:
|
1 2 3 |
sudo apt-get install make libcurl4-gnutls-dev --yes sudo td-agent-gem install fluent-plugin-elasticsearch sudo td-agent-gem install fluent-plugin-record-reformer |
Add the content below to /etc/td-agent/td-agent.conf to setup Fluentd transfer all docker logs to Elasticsearch :
|
1 2 3 4 5 6 7 8 |
<match docker.*> @type elasticsearch logstash_format true host 192.168.1.191 port 9200 index_name fluentd-docker type_name fluentd-docker </match> |
And restart FluentD:
|
1 |
sudo /etc/init.d/td-agent restart |
Docker
Now, we change docker configuration file to use Fluent as a Log Driver. Open /etc/default/docker, and add the line below:
|
1 |
DOCKER_OPTS="--log-driver=fluentd --log-opt fluentd-address=localhost:24224" |
Add restart docker to apply the change:
|
1 |
sudo service docker restart |
Kibana
We will run Kibana in Docker Container with command:
|
1 |
docker run --name kibana -e ELASTICSEARCH_URL=http://192.168.1.191:9200 -p 5601:5601 -d kibana:4.1.4 |
Now, you can access http://192.168.1.191:5601 to see Docker Logs in Kibana.
Tips
Delete a index in Elasticsearch:
|
1 |
curl -XDELETE 'http://localhost:9200/twitter/' |
List all Indexes in Elasticsearch:
|
1 |
curl 'localhost:9200/_cat/indices?v' |
