Microservice with Docker Swarm and Consul – Part 2

In the previous part, I showed you step by step install Node Gateway and Node Agent-One. In this post, we will continue to install Node Agent-Two. After finishing this step, you will understand how to scale or replicate you service in Docker Swarm, and how to Nginx load-balance your services with Consul and Consul-Template.

Nginx Load-balancer, Docker Swarm Cluster

Nginx Load-balancer, Docker Swarm Cluster

Node Agent-Two

In this node, we also need to install Consul and Swarm. Similar with the last post,  we run Consul with command:

And run Registrator:

Restart Docker Daemon:

And run Swarm node to join Swarm Cluster:

Now, the node is ready for running you microservices:

After command above, you are scale your services from one container to two containers running in Docker Swarm

You can check with consul members command:

Scale

To scale your service (in my case is angular-admin-seed service), you should access either Node Agent-One or Agent-Two with root privilege, and type commands:

And check with docker ps command, the output should be:

And check /etc/nginx/sites-available/default at node API Gateway:

It means that Nginx is loading balance between three running services.

Conclusions

After reading, the series of post, we know that there are many open sources support us implement Microservices. However, we can not own a microservices system with one step at this time. We need to deeply understand about tools, open source and Linux as well to implement it.

Microservices with Docker Swarm and Consul – Part 1

In this post, I will show you step by step install Node Gateway and Node Agent One.  After finishing your installation, you will understand the way how to Consul, Swarm, Consul-Template, and Registrator work together.

Prerequisites

  • Three Nodes at address:
    • 172.20.20.10 named Node Gateway
    • 172.20.20.11 named Node Agent One
    • 172.20.20.12 named Node Agent Two
  • Three Nodes with configuration Ram 2G, 2 CPUs are installed Ubuntu 14.04
  • Node Agent One and Node Agent Two are installed Docker and Docker Swarm
  • Three Nodes connected each others
Microservice with Docker Swarm and Consul

Microservice with Docker Swarm and Consul

Node Gateway

As my scenario, we will run consul, consul-template and Nginx on this node.

For downloading and installing Consul, Consul-Template on Ubuntu 14.04, please see the link: Install Consul and Consul Template in Ubuntu 14.04

And it is easy to install Nginx:

Now, you are ready to run Consul and Consul-Template in Node Gateway

Run Consul as a master node:

Now, we create a template file for Consul-Template at /opt/consul-template directory

And create file /opt/consul-template/nginx.ctmpl with the content:

Next step, run Consul-Template:

The guide above, I just show you the way to run Consul, Consul-Template. In fact, we should run them in background and on startup. To do that please see link: Setup Consul, Consul-Template run in background

To verify Consul, you should access http://172.20.20.10:8500 to see Consul Web Interface

Now, you are done with Node Gateway. Do the next step with Node Agent-One

Node Agent-One

With Node Agent-One, we need to run Docker, Swarm Joiner, Swarm Manager, Consul Agent, Registrator, (cAdvisor for monitor as needed)

At the first, We install Consul on this node (It is same as you do with Node Gateway). After that, we run Consul Agent with command:

Next step, we need to restart Docker Daemon to run on address tcp://0.0.0.0:2375. So we run command:

After running Docker Daamon, we should run Registrator.

And you should run cAdvisor to monitor this node:

Now, It is time to run Swarm Manager and Swarm Joiner (Node). But first of all, we need to have a token for Swarm Cluster, you can get it by command:

So we can run Swarm Joiner:

And run Swarm Manager:

Swarm is ready to use for now.  If you need to run Docker Swarm in background, you should read this post: Run Docker Swarm with Upstart

And finally, I will run a service on this node. Try command:

Now you can take a rest and test your work by access link: http://172.20.20.10

clotify_microservice (6)

Nginx, Consul, Consul-Template, Docker Swarm

In the next post, I will show you how to scale a service by using Docker Swarm, Nginx and Consul

 

Microservices with Docker Swarm and Consul

Recently, perhaps you hear too much about Microservices includes the concepts, cons, pros … I sure that you will ask yourself that how to implement it for your project? It is not easy to answer your question. In this post, i will tell you a part of my story after six month implement a social network project.

The story I would like to tell you is how to deploy a microservices with Docker Swarm and Consul  (step by step instruction). With , my scenario, I have three nodes, installed Ubuntu 14.04 with 2G Ram and 2 CPUs. I used Vagrant and Virtualbox to build it.

Microservice with Docker Swarm and Consul

Microservice with Docker Swarm and Consul

As the image above, We will install Consul in all nodes. With, Node Gateway, It is installed Nginx to load-balance and used as a API Gateway for your services. And we also install Consul-Template to help you change Nginx configuration once there is an new service is on or down via Consul.

With Node Agent-One and Node Agent Two, we will use it as Swarm Clusters. So we will install Swarm, and run Swarm Join for all of them. Beside that, we have to install Consul and Registrator. Consul is a Service Discovery, and Registrator will help you announce for Consul know if there is any service is up or down.

There are many steps we need to do to build this system. So I deviced my post to three parts:

I also publish all scripts to build this system to Github. You can check out it microservices-swarm-consul

So you just can download it and run:

Note: Make sure you installed Virtualbox and Vagrant in your machine. And if there is any issue in my code. Please report me. Thank you

 

Before you reading my series of the posts, you should read about the concept of the tools we will use for deploy microservices

Docker & Docker Swarm

Docker is a platform for developers and sysadmins to develop, ship, and run applications. Docker lets you quickly assemble applications from components and eliminates the friction that can come when shipping code. Docker lets you get your code tested and deployed into production as fast as possible.

Docker Swarm is native clustering for Docker. It allows you create and access to a pool of Docker hosts using the full suite of Docker tools. Because Docker Swarm serves the standard Docker API, any tool that already communicates with a Docker daemon can use Swarm to transparently scale to multiple hosts

Consul

  • Service Discovery
  • Key/Value Storage
  • Failure Detection
  • Multiple Datacenter

Consul-Template

  • Listen/Query the updates from Consul
  • Updates Configuration File by the templates provided
  • Executes commands

Registrator

Registrator automatically registers and deregisters services for any Docker container by inspecting containers as they come online. Registrator supports pluggable service registries, which currently includes Consul, etcd and SkyDNS 2.

In next step, please go to Install Load Balance, API Gateway and Docker Swarm

Setup Consul, Consul-Template run in background with Ubuntu 14.04

Before reading this post, please take a look Install Consul and Consul Template in Ubuntu 14.04 to know how to installing Consul and Consul-Template. And now, this post will help you run them in background.

With Ubuntu 14.04, you should use Upstart to make your services run in background and on startup.

You create consul-server.conf file  at /etc/init/ directory with content:

And create consul-template.conf file at /etc/init/ directory with content:

Now, we run them:

You can troubleshoot your service by command tail -f upstart log files:

 

 

Install Consul and Consul Template in Ubuntu 14.04

As you know, Consul and Consul-Template are great tools to help you manage nodes, services and configuration files. This post will show you the way how to install Consul and Consul Template in Ubuntu 14.04.

To download a release of Consul and Consul-Template, please see the link: https://releases.hashicorp.com/

Consul

At the first, down load a Consul’s release:

And extract the archive:

And move to bin directory to use Consul anywhere

The next step, download Consul UI:

Extracting the archive to directory /opt/consul-ui:

Creating configuration directory for Consul:

Now, you can try to run Consul (with -server option) by command:

For now, you can access http://YOUR_IP:8500 to see Consul UI

Consul-Template

Download and Unzip Consul-Template:

Run Consul-Template:

That’s all.

 

Run Docker Swarm with Upstart

In Ubuntu 14.04, to run a job in background an on startup, you should use Upstart. In this post, I will show you the way to run Swarm with Upstart.  I think that it is more complicated that SystemD used by CoreOS or Ubuntu 15.04.

At the first, we need to open /etc/default/docker file and add a line:

And restart Docker:

We need to do this step so that Docker Daemon run at all interfaces and on port 2375. Now, you ready to run Swarm on the machine.

To run Swarm Join, you create file /etc/init/swarm-join.conf with content:

Now, run Swarm Join job:

To run Swarm Manage, you create file /etc/init/swarm-manage.conf with content:

Now run Swarm Manage job:

Done