#Python

Send SMS from Raspberry Pi with USB 3G

In Digital Marketing Strategy, email and SMS are the important tools to improve your  customer services. And Raspberry Pi is a good platform to help us to implement sending email or SMS.

In this weekend, I try to send SMS from Raspberry Pi with USB 3G, and i was successful.  I used a Raspberry Pi 2 model B, and a USB 3G (I am in Vietnam, so I bought a Viettel USB 3G).

Send SMS with Raspberry Pi and USB 3G Viettel

Send SMS with Raspberry Pi and USB 3G Viettel

At the first step, plug USB 3G in Raspberry Pi, and restart Pi.
From the terminal, type lsusb command:

And using dmesg command, find ttyUSB:

If you see the output above, it means that USB 3G connection is correct.

Now, we need to install a software to help us connect to serial port ttyUSB. That software is picocom:

After that, try to connect ttyUSB1 with command:

If you see the output has “terminal ready” and you try to type “AT” and receive the response “OK”. It means that your connection is prefect. Now, you type “AT+CMFG=1” to change to “text mode”

And try to send a SMS message with command:

Finally, press Ctrl + a + x to exit picocom terminal.

If you send SMS successfully with picocom. And now, we write a Python script to send SMS message.  You can see the source code below. It is so simple.

To build a high performance SMS sender, we should apply Redis Pub/Sub to push all messages need to send to Queue.

Good luck!

Upload and Download files from AWS S3 with Python 3

If you are trying to use S3 to store files in your project. I hope that this simple example will be helpful for you.

Install Boto3 via PIP

Please take a look to the source code at https://github.com/thanhson1085/python-s3 before reading this post.

With boto3, It is easy to push file to S3. Please make sure that you had a AWS account and created a bucket in S3 service.

And it is not difficult to save file from S3 to local:

 

Run a Flask application in Nginx uwsgi

As you know uWSGI is the most popular to deploy a Python Application.  And Nginx is a powerful web server to run website in production. So this post will show a simple way to run your application (uses Flask Framework)  with WSGI and Nginx

1007966f-b1c3-4b52-9082-5e3f2f4b85d0-medium

Peace. It does not mean to be in a place where there is no noise, trouble or hard work. It mean to be in the midst of those things and still be calm in your heart

Create wsgi.py in your root directory of the application:

(Please make note that you understand the meaning of the source code above. So you can modify it to match your situation) 

And a config file for WSGI – wsgi.ini:

Install uwsgi with PIP:

Test your uWSGI config with command below:

Make sure that WSGI works with your application properly. Now, you can config NGINX to run over socket file(your_app_name_here.sock). Please create nginx config file with content below

Enable your site in Nginx:

And restart Nginx to apply your changes:

Test your site now!!!

Create upstart script:

With content:

 

A simple examle for Python Kafka Avro

In the weekend, I try to use Python to write a producer and a consumer for Apache Kafka. I found Kafka-Python library that can help me do it easily. However, If you try to send Avro data from Producer to Consumer, it is not easy. You have to understand about them. We have enough specifications but there is no example source code. So this is a simple example to create a producer (producer.py) and a consumer (consumer.py) to stream Avro data via Kafka in Python.

The wise man never knows all, only fools know everything.

To run this source code, please make sure that you installed Kafka (https://sonnguyen.ws/install-apache-kafka-in-ubuntu-14-04/) and Python libraries (kafka-python, avro, io). And I am using Python 2.7

Create producer.py

Create consumer.py

Time for test:

I hope that this post will help you say “Hello” to Kafka, Python and Avro

Please see the details in GitHub: https://github.com/thanhson1085/python-kafka-avro

In the source code repository above, I also created consumer_bottledwater-pg.py to decode avro data that pushed from bottedwater-pg Kafka producer. This base on the question in Stackoverflow 

How to save a file to HDFS with Python

This source code is a simple example the way how to upload image and save it to HDFS. This program will connect HDFS via webhdfs. Actually, it is easier than you think. The most dificulty is preparing environment to test your source code

Help people, even when you know they can't help you back

Help people, even when you know they can’t help you back

Prerequisites:

  • python3
  • virtualenv (optional)
  • pip3
  • Flask==0.10.1
  • Flask-Swagger
  • Hadoop with webhdfs

Installation:

To install Hadoop, pleas take a look at https://github.com/thanhson1085/docker-cloudera-quickstart

I will show you the detail, just hope that it will save your time.

At the first, using Docker to clone docker-cloudera-quickstart to your local.

And run that image, and please do not forget expose webhdfs port(50070, 50075):

There is 2 ways to run my source code. I recommend you using Docker

Using Docker

Install in normal way

There is a trick here. You have to edit /etc/hosts file to your machine know where is HDFS server. You can use “docker inspect” command to get HDFS server ip address.

Finally, you have a HDFS Server with webhdfs support.

Clone this source code to your local

Run the commands below to install and create environment to run the application

Test

I integrated swagger in this app. So you can use Swagger UI to test this app. And you can find swagger schema at http://localhost:5000/docs

After upload the files. You can use hadoop command to look up in hdfs.

Go to Hadoop server:

List files in hdfs:

The output should be:

Github: https://github.com/thanhson1085/flask-webhdfs

Flask RabbitMQ Celery example

If you are learning the way how to work with RABBITMQ + CELERY. This source code may help you. I wrote a small example to upload image to web server and use Celery to generate a thumbnail.

Difficult doesn’t mean impossible. It simply means that you have to work hard

This source code supports 2 ways to run it.

You can check the source code in Github: flask-celery-rabbitmq-generate-thumbnail

And the image at docker: flask-celery-rabbitmq-example

At the first, clone this source code to your local:

Using Docker

  1. Build from Dockerfile

Or pull from Docker Repo

Run Docker image

After running the docker image, you should wait for the output as below:

Install packets normally (Ubuntu 14.04)

I will show you how to run this source code from scratch. And i am using ubuntu server 14.04, installed virtualenv, pip.

Install RabbitMQ Server:

Fix the issue of PIL

Create environment to run the application with virtualenv:

Install all packets required:

Run web server to upload files

Run the “generate thumbnail” task in Celery

Now, it is ready for testing. (http://localhost:5000). The page should be as image below:

generate thumbnail with celery rabbitmq

Generate thumbnail with celery rabbitmq