All posts by Nguyen Sy Thanh Son

I am a software developer. I follow microservices and devops

Get started Ethereum/Solidity ICO Smart Contract

The below is commands when I start to learn Ethereum/Solidity . Assume that we installed Geth and Solidity Compiler in our machine.

Run an Ethereum Node in testnet:

Run an Ethereum Node in Ropsten Testnet:

Run a private blockchain:

Open another terminal, connect to the exist node:

Mining

Strart mining

Check hashrate

Stop mining

Sync status

Remove Ethereum Database

Check balance in Ether

Send a transaction:

Send with gas, gasPrice:

 

ERC20

Specification: https://theethereum.wiki/w/index.php/ERC20_Token_Standard#Approve_And_TransferFrom_Token_Balance

Logstash Elasticsearch data retention

If you are using Elasticsearch to centralize your log data, that is a great solution. However, after few months, you will have huge log data store in your server hard disk. You have to clean the old log data that you are sure that you will not use it in future.

To delete the 10-day-ago log data, we can use the script below:

So we should run the script above every night to delete data. So we use cronjob:

Thanks for your reading

linux shirt

Software is like sex t-shirt

My First AI Application

I am not Data or AI scientist, I am just a Developer. I tried to learn AI in my free time. There are a lot of information from Internet. After reading many many articles and source codes, I still did not find the approach. I confused by many AI algorithms as CNN, Logistic Regresstion, K-Tree, N-Tree …  As you know, I am a developer, so I did not understand at all. :).

Finally, I find the solution. The thing I need just Tensorflow.

To start, I try to build AI that can classify Dog/Cat Image. I followed the steps below.

Step 0 – Installation

I installed Tensorflow on my Ubuntu 14.04 Server by commands:

To understand this, you have to understand Python, PIP and Virtualenv

Step 1 – The Samples

I search Dog/Cat Image on Google Image. After that, I resize/save them to my local storage.

Please see link: https://github.com/thanhson1085/Hello-AI/tree/master/dataset/training_set

Step 2 – Training

After having the samples, I started to research to train my AI. I found the solution here: https://github.com/thanhson1085/Hello-AI/blob/master/retrain.py

It was copy from Tensorflow Example: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/image_retraining/retrain.py

Now, I run command to train my AI by using Inception V3 Model and my Training Dataset

The most import thing is the output file output/retrained_graph.pd. This is the trained model file, I will use this file for my AI.

Step 3 – AI Service

After having the trained model file. I wrote the AI Service. I used Flask Framework to handle uploading image. When user uploads a image, my AI service will check it is CAT or DOG. Please see the source code:

For the details, see file https://github.com/thanhson1085/Hello-AI/blob/master/app.py and https://github.com/thanhson1085/Hello-AI/blob/master/catordog.py

Step 4 – Deployment

Now, everything is reading to deploy. I chose Heroku to run my AI service https://thanhson1085-hello-ai.herokuapp.com/

Finally, we have a simple AI, we understand how it works, how to train it and what is most important thing we need to build a AI.

Thanks for your reading!

Tensorflow T-shirt

Tensorflow T-shirt

How to HOT Backup Database (MongoDB, MySQL, ES …) to AWS S3

Actually, there are many way to backup your database. You can using RSYNC, MongoDump for Mongo, S3 Backup Plugin for ElasticSearch. However, this post will show you the way I used in my project. Maybe, It is not perfect for all case. But in my case, it is really perfect.

Docker Ironman T-shirt

Docker Ironman T-shirt

I am running a project with Microservice Architecture. All Databases and Services are running in Docker Container.  In my plan, I have to backup all databases every night.

At the beginning, I tried to use tar command to compress the data, and then I use command aws s3 copy to copy backup data to S3. It seems work. But tar command makes MongoDB stop working. I tried to google to solve the problem. I found the solution is rsync command.

The backup process should be implement in three steps:

  • Use rsync command to copy the data to other location
  • Compress the data by tar command
  • Move the compressed data to AWS S3

The script should be:

Thanks for your reading.

Connect USB from Virtual Machine using Vagrant and Virtual Box

Sometimes (e.g You are developing an IoT Project), you need to connect you board to Virtual Machine via USB Port. This post will show you steps to connect USB port from Virtual Machine using Vagrant and Virtual Box.

At the first, you have to make sure that the USB device was connected to your computer with command:

The output above shows the USB Device has VendorID is 0x1a86 and ProductId is 0x7523. So we add the lines below to Vagrantfile:

Finally, starting your Virtual Machine:

In your Linux Virtual Machine, type lsusb to check the result.

Fix UTF-8 Issue with Python Logger

When you work with Python Logger to STDOUT, sometimes you got error messages as the below:

The reason is STDOUT does not support UTF-8 encoder.

I try to google to solve the issue. And I got the solution. We have to create environment variable PYTHONIOENCODING=utf-8

So It works.

Compress/Split/Concat MP4 with AVCONV

This post just save some command that I used to edit my video. Hope it is helpful with you.

Compress MP4 with H264:

Split Video by time frame:

Concat/Join two video files:

Export video to PNG files

Create GIF from PNG files (ImageMagick):

 

 

I start studying Internet of Things

As you know, Internet Of Things is the hottest technical trend in the world recently. So we should study it. A old developers should learn it before becoming a backward guys. And a young developer should learn IOT to make the future better that today, or just simple to have  a good job.

I heard about IOT the last year.  From there,  I thought about it when i have free time. I prepared basic knowledge about IOT as below:

  • The Essentials of Electricity & Electronics
  • IOT Concept
  • IOT Architecture Overview
  • MQTT Protocol

I am a Software Developer, I always want to start writing source code after leaning a bit theory. But I do not know how to start.  So I joined Maker Hanoi ( A IOT Group in Hanoi Vietnam) and started study a IOT class for beginners there.

At the beginning, I bought a NodeMCU ESP8266 Dev Kit with price 9$ (200.000 VND)

NodeMCU ESP8266 Dev Kit

NodeMCU ESP8266 Dev Kit

I also need a micro USB cable to connect Development Kit to my computer.

Micro USB Cable

Micro USB Cable

After having DevKit, I took a look to ESP8266 Datasheet to understand ESP8266’s pins, functions. I also need to refer the image below to know Port Mapping between DevKit and ESP8266.

NodeMCU ESP8266 Dev Kit Pin Out

NodeMCU ESP8266 Dev Kit Pin Out

After the hardware was under my control, I went to the next step – studying software. I was suggested using Sming Framework to write firmware for ESP8266.  Sming Framework is easy to understand and use. However,  maybe you can have some issues with installation.

My first idea about IOT is pushing data from ESP8266 to an IOT Cloud Platform.  Actually, there are many solutions for IOT Platform. For example, I can build a IOT Platform on my own by using open source, or try IBM IOT Platform. However,  to simple and free, I chose Koor.IO – An IOT Platform for Developer.

With Koor.IO, I can have use Restful API, Websocket and MQTT. Koor.IO help me collect data and show on chart.

To implement my idea with Koor.IO, I just do the some steps below:

Step 1: Create Project

Create an account on Koor.IO and create my first project.

Create a IOT Project on Koor.IO

Create a IOT Project on Koor.IO

Waiting until Koor.IO finishing setup your project

2016-10-12_1700

After that, open your project, create Field to indicate what field you need KoorIO collect the data.

2016-10-12_1704

Now, you are done with Koor.IO

Step 2: Source code

In this step, you clone Koor.IO’s MQTTClient example.

You need to change configuration file include/config.h

MQTT_TOPIC should be project_domain/field. And field  is the field you created on koor.io, project_domain is the domain of your Koor.IO Project.

MQTT_HOST, you enter mqtt.koor.io

And I entered your Wifi Information in WIFI_SSID, WIFI_PWD

Step 4: Build

After change the configuration file, We just need to build the source code and load into ESP6266 by the commands:

Result

After build flash was successful, I go to the dashboard of the project on Koor.IO to see the data was pushed from the device. The chart is updated realtime as below.

Koor.IO - Chart - ESP8266

Koor.IO – Chart – ESP8266

Troubleshoot

With Linux,  in case you can not load data to ESP8266, you should check the connection between DevKit and your computer by command

Or

If you did not see the logs from ESP8266. You can use screen command:

Thank you for your reading!!!