If you are planning to build a project with Microservices pattern and DevOps culture, you can use the source code here.
In this source code, I built two services, one for Frontend AngularJS and Bootstrap, one for Backend ExpressJS and Mongoose. All services in this project is dockerized and pushed to Docker Hub. You can read Dockerfile in each service for further details. To create a new service, you just create a new directory, writing source code for that service and update docker-compose.yml file.
Run
The Orchestration of Project is written in docker-compose.yml file. So it is so easily to understand and run the project.
1
docker-compose up
Using Vagrant and Virtualbox
We also start from Virtual Machine layer with Vagrant.
You can read Vagrantfile to understand what we need to prepare for VMs.
System Architecture
With the source code, we will have a system architecture as below
Microservices DevOps Starter Project
Monitor & Logs
This starter project also fully supports monitor by using Telegraf, InfluxDB, Grafana and Kapacitor. Supports centralizing Logs with fluentd, Kibana and Elasticsearch.
This tutorial will show you how to install Docker and Docker Swarm on CentOS7. This tutorial is for Docker Version 1.11 and I use Vagrant and VirtualBox to create CentOS 7 in Virtual Machine.
Stop worrying about what you have to loose and start focusing on what you have to gain
Beside monitor topic, Log also is a important issue we need to concern. In this post, I just mention the way how to centralize Docker Logs using FluentD, Elasticsearch and Kibana
Try not to become a man of success, but rather try to become a man of value
Secenario
We will install FluentD, ElasticSearch and Kibana in the same machine.
FluentD : Collect and Transfer log data to Elasticseach
Elasticsearch: Store and indexing log data to support searching/filtering log data
Kibana: A web view supports you search/filter and virtualize the log data
Prerequisites
We have a machine installed Ubuntu 14.04 with IP 192.168.1.191
We already installed Docker, Wget
Now, I will show you step by step to get stated to centralize the log data with FluentD
Recently, perhaps you hear too much about Microservices includes the concepts, cons, pros … I sure that you will ask yourself that how to implement it for your project? It is not easy to answer your question. In this post, i will tell you a part of my story after six month implement a social network project.
The story I would like to tell you is how to deploy a microservices with Docker Swarm and Consul (step by step instruction). With , my scenario, I have three nodes, installed Ubuntu 14.04 with 2G Ram and 2 CPUs. I used Vagrant and Virtualbox to build it.
Microservice with Docker Swarm and Consul
As the image above, We will install Consul in all nodes. With, Node Gateway, It is installed Nginx to load-balance and used as a API Gateway for your services. And we also install Consul-Template to help you change Nginx configuration once there is an new service is on or down via Consul.
With Node Agent-One and Node Agent Two, we will use it as Swarm Clusters. So we will install Swarm, and run Swarm Join for all of them. Beside that, we have to install Consul and Registrator. Consul is a Service Discovery, and Registrator will help you announce for Consul know if there is any service is up or down.
There are many steps we need to do to build this system. So I deviced my post to three parts:
I also publish all scripts to build this system to Github. You can check out it microservices-swarm-consul
So you just can download it and run:
1
vagrant up
Note: Make sure you installed Virtualbox and Vagrant in your machine. And if there is any issue in my code. Please report me. Thank you
Before you reading my series of the posts, you should read about the concept of the tools we will use for deploy microservices
Docker & Docker Swarm
Docker is a platform for developers and sysadmins to develop, ship, and run applications. Docker lets you quickly assemble applications from components and eliminates the friction that can come when shipping code. Docker lets you get your code tested and deployed into production as fast as possible.
Docker Swarm is native clustering for Docker. It allows you create and access to a pool of Docker hosts using the full suite of Docker tools. Because Docker Swarm serves the standard Docker API, any tool that already communicates with a Docker daemon can use Swarm to transparently scale to multiple hosts
Registrator automatically registers and deregisters services for any Docker container by inspecting containers as they come online. Registrator supports pluggable service registries, which currently includes Consul, etcd and SkyDNS 2.
If you are building docker images in your machine. I sure that your hard disk is growing up day by day. To stop it, you have to remove unnecessary images to release for your hard disk by command below:
If you are learning the way how to work with RABBITMQ + CELERY. This source code may help you. I wrote a small example to upload image to web server and use Celery to generate a thumbnail.
Difficult doesn’t mean impossible. It simply means that you have to work hard
As you know, Install Hadoop is not easy. And It require a virtual machine with high configuration. In my post, I will give you the shortest way to have Hadoop in your machine. It is installation via Docker. I assume that you had Docker in your Ubuntu 14.04
If your dreams do not scare you. They are too small.
Prerequisites
Docker
Ubuntu 14.04
Ram 4G
CPU 2 core
Install
To install the docker-cloudera-quickstart from docker-hub, simply use the following command:
Docker is a platform of developer and sysadmins . With Docker, you will be able to push whole your virtual machines to cloud. So you could easily use it everywhere and every time, or share to other people. I will share you the details of Docker installation in Ubuntu 14.04