If you are using Elasticsearch to centralize your log data, that is a great solution. However, after few months, you will have huge log data store in your server hard disk. You have to clean the old log data that you are sure that you will not use it in future.
To delete the 10-day-ago log data, we can use the script below:
|
1 2 3 |
#/bin/bash d=$(date --date="10 day ago" +%Y.%m.%d) curl -XDELETE http://localhost:9200/logstash-${d}?pretty |
So we should run the script above every night to delete data. So we use cronjob:
|
1 |
30 11 * * * bash /home/ubuntu/backend/scripts/es_retention.sh > /dev/null 2>&1 |
Thanks for your reading
Software is like sex t-shirt
Related Posts
How to HOT Backup Database (MongoDB, MySQL, ES …) to AWS S3
Monitor Nginx Response Time with FluentD, Kibana and Elasticsearch
Centralize Nginx Logs with FluentD, Kibana and Elasticsearch
Centralize Docker logs with FluentD, ElasticSearch and Kibana
