In my previous post, I showed you How to centralize Nginx logs. So now, I will use FluentD, Kibana and ElasticSearch to collect Nginx Response Time.
To implement it, we have to change Nginx Log Format. Because, in the default, Nginx does not store Response Time to access.log file. So we change nginx.conf as below:
|
1 2 |
log_format upstreamlog '[$time_local] $remote_addr - $remote_user - $server_name to: $upstream_addr $request $status upstream_response_time $upstream_response_time msec $msec request_time $request_time'; access_log /var/log/nginx/access.log upstreamlog; |
After reload Nginx, you can try to tail access.log. The result should be as below:
|
1 |
[26/Feb/2016:06:42:49 +0000] 118.70.190.253 - - - example.com to: 172.31.27.195:33687 GET /api/v1/post/31856?access_token=r12dx4whN147sm0YC8IYTyn-bfkyWg52-F08_BI_pfKBOo1JCliMwhI_g8qtffAm9QUoRjNiF8TULGpjw8sgbQ&limit=15 HTTP/1.1 200 upstream_response_time 0.140 msec 1456468969.606 request_time 0.259 |
Now, we will create a regex string that matches with the log format above:
|
1 |
/^\[(?([^ ]* [^\]]*))\] (?[^ ]*) - ([^ ]*) - ([^ ]*) to: (?[^ ]*) (?[^ ]*) (?[^ ]*) ([^ ]*) (?<code>[^ ]*) upstream_response_time (?[^ ]*) msec ([^ ]*) request_time ([^ ]*)/ |
And Insert to td-agent.conf as below:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
<source> @type tail path /var/log/nginx/access.log #...or where you placed your Apache access log pos_file /var/log/td-agent/nginx-access.log.pos # This is where you record file position tag nginx.access time_format %d/%b/%Y:%H:%M:%S %z format /^\[(?<time>([^ ]* [^\]]*))\] (?<remote>[^ ]*) - ([^ ]*) - ([^ ]*) to: (?<host>[^ ]*) (?<method>[^ ]*) (?<path>[^ ]*) ([^ ]*) (?<code>[^ ]*) upstream_response_time (?<response_time>[^ ]*) msec ([^ ]*) request_time ([^ ]*)/ </source> <source> @type tail path /var/log/nginx/error.log pos_file /var/log/td-agent/nginx-error.log.pos tag nginx.error format /^(?<time>[^ ]+ [^ ]+) \[(?<log_level>.*)\] (?<pid>\d*).(?<tid>[^:]*): (?<message>.*)$/ </source> <source> @type tail path /build/clotify/logs/*/*.log pos_file /var/log/td-agent/clotify.log.pos tag clotify format /^(?<time>([^ ]* [^ ]*)) (?<log_level>[^ ]*): (?<service>[^ ]*) (?<message>.*)\[in (?<filename>[^ ]*):(?<lineno>[^ ]*)\]/ </source> <match nginx.*> @type elasticsearch logstash_format true host 172.31.18.133 port 9200 index_name fluentd-aws-dev-nginx type_name fluentd-aws-dev-nginx </match> |
After restarting td-agent. We can wait a minute and then view kibana. The outfit should be:
Monitor Nginx Response Time with FluentD, Kibana and Elasticsearch
For basic installation, please refer to https://sonnguyen.ws/centralize-docker-logs-with-fluentd-elasticsearch-and-kibana/
