Monitor Nginx Response Time with FluentD, Kibana and Elasticsearch

In my previous post, I showed you How to centralize Nginx logs.  So now, I will use FluentD, Kibana and ElasticSearch to collect Nginx Response Time.

To implement it, we have to change Nginx Log Format. Because, in the default, Nginx does not store Response Time to access.log file. So we change nginx.conf as below:

After reload Nginx, you can try to tail access.log. The result should be as below:

Now, we will create a regex string that matches with the log format above:

And Insert to td-agent.conf as below:

After restarting td-agent. We can wait a minute and then view kibana. The outfit should be:

2016-02-26_1800

Monitor Nginx Response Time with FluentD, Kibana and Elasticsearch

For basic installation, please refer to https://sonnguyen.ws/centralize-docker-logs-with-fluentd-elasticsearch-and-kibana/

Centralize Nginx Logs with FluentD, Kibana and Elasticsearch

As you know, FluentD is a great tool to collect the logs, ElasticSearch supports store and search the log data and Kibana helps you to view and search logs in web interface.

Nginx Logs is a good thing help you monitor, debug, and troubleshoot your application. So in this post, I will show  you how to Centralize Nginx Logs with FluentD, Kibana and Elasticsearch.

Before reading this post, please read Centralize Docker Logs with FluentD, Kibana and Elasticsearch to know how to install Fluent, Kibana and Elasticsearch in Ubuntu 14.04.

In the next step, we add content below to /etc/td-agent/td-agent.conf file:

And Restart FluentD:

For Debugging:

If you got an error:

We just need add td-agent user to adm group:

Finally, access Kibana to view the logs as image below:

2016-02-16_1310

Centralize Nginx Logs with FluentD, Kibana and Elasticsearch