How to create a small API with a queue of tasks with Python

I recently needed a way to distribute tasks from a centralised server. It’s amazing how simple this is to achieve with less than 20 lines of code.

Installation

On most GNU/Linux distributions you should already have Python, if not you can install it with your package manager.

On Windows if you have installed the chocolatey package manager you can install it with that.

choco install python

Pip should be installed alongside Python, if it isn’t, you can manually install it with your package manager. It’s probably called python-pip. On Windows it’s always included.

Now install the required libraries.

sudo pip install flask_restful flask

Everything should be installed now.

Creating your first Flask API

Create a file with a .py extension.

At the top of the file we import the libraries that we just installed.

from flask import Flask, request
from flask_restful import Resource, Api

Now in my case I was simply trying to distribute tasks for a very low amount of clients. This is why I use a simple file as storage, you can change this to any SQL-database.

To read files, you need the os library.

import os
tasks_file = str(os.environ['HOME']) + os.pathsep + "omega_project" + os.pathsep +"... .txt"

Create a Flask object and Api.

app = Flask(__name__)
api = Api(app)

To keep track of the current position I define a global variable called task_iterator.

task_iterator = 0

I only want to load the file once into memory so I don’t do uny unnecessary IO operations. It’s also faster using only RAM. This function will only execute once when starting the App.

task_names = []

@app.before_first_request
def load_tasks_file():
    with open(tasks_file) as f:
        global task_names
        task_names = f.readlines()

Now is the time to make any function you would like, in my example I create the function get_task which returns a taskname without any arguments. It’s available by going to the hostname in combination with /task.

@app.route('/task')
def get_task():
    global task_iterator
    task = task_names[task_iterator]
    task_iterator = (task_iterator + 1) % len(task_names)
    return task

Finally run the app.

if __name__ == '__main__':
     app.run()

If you want your app only to be accessible from localhost you can simply run the app without arguments. If you want it to be externally available, give it the argument host=‘0.0.0.0’.

app.run(host='0.0.0.0')

Click here to view the resulting script.

Running the script

Just execute the app you just made with Python.

python ./flask_api.py

And then open your browser and go to http://localhost:5000/task

Deploying with Docker

For a quick and easy way to deploy your app, you can use Docker. In your current working directory you simple have to create a file called Dockerfile with the following content:

Installing Docker

To install Docker, use your package manager or follow the official install guide from Docker.

FROM python:3

WORKDIR /usr/src/app

COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

COPY . .
EXPOSE 5000
CMD [ "python", "./app.py" ]

In the same directory you also have to create a file requirements.txt with the libraries.

flask_restful
flask

Now build the Dockerfile and then you can simply run it.

docker build -t myapi .
docker run -d -p 80:5000 -v $HOME/tasks:/omega_project myapi

Ease of use with Docker Compose

If you don’t want to always remember the arguments that Docker needs or you plan on using this App in combination with a database like MySQL, then Docker Compose is ideal.

Installing Docker Compose

To install Docker Compose, don’t use your package manager. The Docker Compose binary needs to be newer or the same as Docker which is often not the case when you use the official repositories in my experience. Follow the official install guide from Docker.

The configuration file

version: '2'
services:
  api:
    build: ./docker
    ports:
     - "80:5000"
    volumes:
     - .:/root/omega_project/

And then start it.

docker-compose up -d