How to HOT Backup Database (MongoDB, MySQL, ES …) to AWS S3

Actually, there are many way to backup your database. You can using RSYNC, MongoDump for Mongo, S3 Backup Plugin for ElasticSearch. However, this post will show you the way I used in my project. Maybe, It is not perfect for all case. But in my case, it is really perfect.

Docker Ironman T-shirt

Docker Ironman T-shirt

I am running a project with Microservice Architecture. All Databases and Services are running in Docker Container.  In my plan, I have to backup all databases every night.

At the beginning, I tried to use tar command to compress the data, and then I use command aws s3 copy to copy backup data to S3. It seems work. But tar command makes MongoDB stop working. I tried to google to solve the problem. I found the solution is rsync command.

The backup process should be implement in three steps:

  • Use rsync command to copy the data to other location
  • Compress the data by tar command
  • Move the compressed data to AWS S3

The script should be:

Thanks for your reading.

Backup Postgres 9.4 to S3 with WAL-E in Ubuntu 14.04

If you are using Postgres 9.4 Database for your project. I think that you are thinking about backup Backup Postgres everyday. So in this post I will show you how to  backup Backup Postgres 9.4 to S3.


A sign of a good leader is not how many followers you have, but how many leaders you create.

Install Dependencies:

Using PIP to install WAL-E:

Using PIP to upgrade Request:

Using PIP to upgrade Six:

If you not upgrade them,  maybe you will meet an error as below when you run WAL-E Backup:

And we should change permission for PIP packet so that postgres user is able to use them:

Edit postgresql.conf to do backup with wall-push command:

Now, we restart postgres to apply the changes:

Backup Everyday

Assume that you created a bucket on S3, and you have AWS credentials. So you should push them to config file with commands:

Now, we will try to backup to S3 in the first time. At the first, change to postgres user:

Run backup command:

The output should be:

Finally, we add the command to crontab to backup 5 AM everyday:

Finished your works now!