#microservices

AngularJS, Docker, NodeJS

Starting a Miroservices Project with NodeJS and Docker

8

If you are planning to build a project with Microservices pattern and DevOps culture, you can use the source code here.

In this source code, I built two services, one for Frontend AngularJS and Bootstrap, one for Backend ExpressJS and Mongoose. All services in this project is dockerized and pushed to Docker Hub. You can read Dockerfile in each service for further details. To create a new service, you just create a new directory, writing source code for that service and update docker-compose.yml file.

Run

The Orchestration of Project is written in docker-compose.yml file. So it is so easily to understand and run the project.

1
docker-compose up

Using Vagrant and Virtualbox

We also start from Virtual Machine layer with Vagrant.

1
vagrant up

After that, you can access the application via link: http://172.20.20.20

You can read Vagrantfile to understand what we need to prepare for VMs.

System Architecture

With the source code, we will have a system architecture as below

Microservices DevOps Starter Project

Microservices DevOps Starter Project

 

Monitor & Logs

This starter project also fully supports monitor by using Telegraf, InfluxDB, Grafana and Kapacitor. Supports centralizing Logs with fluentd, Kibana and Elasticsearch.

Contents

Part Title Git Tag
1 Starting with ExpressJS 1.0
2 Logger with Winston 1.1
4 Config Management with Node-Config 1.2
5 Building Create User API 2.1
6 Adding Swagger Documents 2.2
7 Building Login API 2.3
8 Building Get User List/Detail API 2.4
9 Authorization all APIs 2.5
10 Unit Test 2.6
11 Building Config API 3.0
12 Using Cache 3.1
13 Using Queue 3.2
14 Starting AngularJS with Yeoman 4.0
15 Config Management for AngularJS 4.1
16 Building Login Page 4.2
17 Building List User Page 4.2
18 Pagination with AngularJS and Bootstrap 4.3
19 Multiple Languages 4.4
20 AngularJS Unit Test 4.5
21 Dockerize Aplication 5.0
22 Orchestration with Docker Compose 5.1

Nguyen Sy Thanh Son

8

Docker, Linux

Microservices with Docker Swarm and Consul – Part 1

8

In this post, I will show you step by step install Node Gateway and Node Agent One.  After finishing your installation, you will understand the way how to Consul, Swarm, Consul-Template, and Registrator work together.

Prerequisites

  • Three Nodes at address:
    • 172.20.20.10 named Node Gateway
    • 172.20.20.11 named Node Agent One
    • 172.20.20.12 named Node Agent Two
  • Three Nodes with configuration Ram 2G, 2 CPUs are installed Ubuntu 14.04
  • Node Agent One and Node Agent Two are installed Docker and Docker Swarm
  • Three Nodes connected each others
Microservice with Docker Swarm and Consul

Microservice with Docker Swarm and Consul

Node Gateway

As my scenario, we will run consul, consul-template and Nginx on this node.

For downloading and installing Consul, Consul-Template on Ubuntu 14.04, please see the link: Install Consul and Consul Template in Ubuntu 14.04

And it is easy to install Nginx:

1
sudo apt-get install nginx

Now, you are ready to run Consul and Consul-Template in Node Gateway

Run Consul as a master node:

1
2
3
4
consul agent -server -bootstrap-expect 1 \
    -data-dir /tmp/consul -node=gateway -bind=172.20.20.10
    -client=0.0.0.0 \
    -config-dir /etc/consul.d -ui-dir /opt/consul/

Now, we create a template file for Consul-Template at /opt/consul-template directory

1
2
mkdir -p /opt/consul-template
cd /opt/consul-template

And create file /opt/consul-template/nginx.ctmpl with the content:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
upstream angular-admin-seed {
    {{range service "angular-admin-seed"}}
    server {{.NodeAddress}}:{{.Port}};{{end}}
}
 
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
 
server_name localhost;
 
    location / {
        proxy_pass http://angular-admin-seed/;
    }
}

Next step, run Consul-Template:

1
2
3
4
consul-template \
    -consul 127.0.0.1:8500 \
    -template "/opt/consul-template/nginx.ctmpl:/etc/nginx/sites-available/default:service nginx reload" \
    -retry 30s

The guide above, I just show you the way to run Consul, Consul-Template. In fact, we should run them in background and on startup. To do that please see link: Setup Consul, Consul-Template run in background

To verify Consul, you should access http://172.20.20.10:8500 to see Consul Web Interface

Now, you are done with Node Gateway. Do the next step with Node Agent-One

Node Agent-One

With Node Agent-One, we need to run Docker, Swarm Joiner, Swarm Manager, Consul Agent, Registrator, (cAdvisor for monitor as needed)

At the first, We install Consul on this node (It is same as you do with Node Gateway). After that, we run Consul Agent with command:

1
2
3
4
consul agent -data-dir /tmp/consul -node=agent-one \
    -bind=172.20.20.11 -client=0.0.0.0 \
-config-dir /etc/consul.d \
    -retry-join 172.20.20.10

Next step, we need to restart Docker Daemon to run on address tcp://0.0.0.0:2375. So we run command:

1
docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock

After running Docker Daamon, we should run Registrator.

1
2
3
4
5
6
docker run -d \
    --name=registrator \
    --net=host \
    --volume=/var/run/docker.sock:/tmp/docker.sock \
    gliderlabs/registrator:latest \
    consul://172.20.20.11:8500

And you should run cAdvisor to monitor this node:

1
2
3
4
5
6
docker run --volume=/:/rootfs:ro \
    --volume=/var/run:/var/run:rw --volume=/sys:/sys:ro \
    --volume=/var/lib/docker/:/var/lib/docker:ro \
    --volume=/sys/fs/cgroup:/sys/fs/cgroup:ro \
    --publish=8080:8080 \
    --detach=true --name=cadvisor google/cadvisor:latest

Now, It is time to run Swarm Manager and Swarm Joiner (Node). But first of all, we need to have a token for Swarm Cluster, you can get it by command:

1
2
$ docker run --rm swarm create
acdb9dfa3ea6da0b0cfb2c819385fcd3 # keep it, we will use this token for our scenario

So we can run Swarm Joiner:

1
$  docker run -d swarm join --addr=172.20.20.11:2375 token://acdb9dfa3ea6da0b0cfb2c819385fcd3

And run Swarm Manager:

1
$ docker run -d -p 12375:2375 swarm manage token://acdb9dfa3ea6da0b0cfb2c819385fcd3

Swarm is ready to use for now.  If you need to run Docker Swarm in background, you should read this post: Run Docker Swarm with Upstart

And finally, I will run a service on this node. Try command:

1
2
3
4
export DOCKER_HOST=tcp://172.20.20.11:12375
docker pull thanhson1085/angular-admin-seed
docker run -d -p 80 --name angular_admin_seed_1 \
        thanhson1085/angular-admin-seed

Now you can take a rest and test your work by access link: http://172.20.20.10

clotify_microservice (6)

Nginx, Consul, Consul-Template, Docker Swarm

In the next post, I will show you how to scale a service by using Docker Swarm, Nginx and Consul

 

Nguyen Sy Thanh Son

8

NodeJS

Combines all swagger documents in microservices

2

If you are building a microservices system and using swagger as an API Document. You may have many API documents, per service an API document.

I met the same case as above, and I develop a tool to help me combine all documents to only one document. You can see it here.

Skills are cheap. Passion is priceless

Skills are cheap. Passion is priceless

Installation

1
npm install swagger-combined

Configuration

Create sample configuration file:

cp -R node_modules/swagger-combined/config .

And see config/default.json as below:

1
2
3
4
5
6
7
8
9
10
11
{
    "list_url": [
        {
            "docs": "http://petstore.swagger.io/v2/swagger.json",
            "base_path": "http://petstore.swagger.io/v2",
            "route_match": ["/user*", "/pet*", "/store*"]
        }
    ],
    "info": { "title": "Example API", "version": "1.0" },
    "port": 3000
}

 

  • docs: swagger document links
  • base_path: Proxy Target
  • route_match: Routes for proxy

Run

Change config/default.json to match your swagger links

node node_modules/swagger-combined/app.js

Please make note that you changed config/default.json to match all swagger document links you have

Test

In the default, swagger-combined run on port 3000 and included swagger-ui. So you just run http://localhost:3000 to see everything you need

Github

https://github.com/thanhson1085/swagger-combined

Demo

https://swagger-combined.sonnguyen.ws/

Nguyen Sy Thanh Son

2