Your Web News in One Place

Help Webnuz

Referal links:

Sign up for GreenGeeks web hosting
October 30, 2022 06:16 pm GMT

Python Flask: Interacts with Docker Containers

Python3 Flask: Interacts with Docker Containers

Last Friday, I was talking with my co-worker at my work. Just about work, you know. He told me he had to implement a server for serving his features. Although I didn't understand what he was going to do exactly, it sounded really interesting and I wanted to try. Since I got a lot of work that make many pages, I had written a bunch of HTML things with React. It wasn't technically interesting at the moment(I love React but the work was almost the same at all). I got tired of that. While I was in there, the coworker said me what he was doing.

He had to do

  • Make an API server that gets an image file
  • Execute a docker container and pass the image file to the container and the container will create the text file that is generated from the image file
  • Execute another docker container and pass the text
  • The API Server knows that the container has done with its work and notify to another server(The backend that servers the data to users).

It sounds interesting, ha?, I decided to implement the system on the weekend. I thought it was going to be fun. But it's been really a long time since I coded with python. So, I had to learn a lot of stuff. During the weekends, I couldn't learn all the things, so, I would use the minimum skills for implementing the system. If you're a python developer or working with the docker, it might be not in your mind at some points.

Anyways, Let's get started!

Prerequisite

  • pipenv

Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world. Windows is a first-class citizen, in our world.
When I first use python, it was about 5 years ago, I used conda or venv for managing packages. While I was searching for package management, I found this. It seems like npm, I'm not sure this is the best solution, but in my opinion, it must be worth it to try.

  • docker

Docker is a platform designed to help developers build, share, and run modern applications. We handle the tedious setup, so you can focus on the code.
As I introduced at first, this is the main topic in this post

  • Flask

Flask is a web application framework written in Python. It was developed by Armin Ronacher, who led a team of international Python enthusiasts called Poocco.
To implement a simple API web server, I used Flask. there were other libraries like Fast API. For reducing my hours, I went with Flask.

Process

This is the process that I'm going to implement.

  1. The API server receives a file from the user

  2. Container A processes A file then saves it to the B File

  3. Container B processes the B File then saves it to the C File

  4. Users can see the result through API

Python Apps and Docker Images

I made two images and uploaded to my them repository in docker-hub.

A tokenizer and word-counting app.

[Tokenizer]

import sys, json, os, requestsfrom textblob import TextBlobdef extract_nouns(text):    blob = TextBlob(text)    filtered_tags = list(filter(lambda tag: tag[1] == "NN", blob.tags))    nouns = list(map(lambda tag: tag[0], filtered_tags))    return nounsdef read_file(path):    with open(path) as f:        contents = f.read()    return contentsdef save_data(path, data):    with open(path, "w") as f:        json.dump(data, f)def get_filename_from_path(path):    return os.path.splitext(os.path.basename(path))[0]def notify_done(url, file_name):    requests.get(f"{url}/docker/tokenizer_done?file_name={file_name}")if __name__ == "__main__":    if len(sys.argv) < 4:        print("You must pass file path as an argument")        print("python3 main.py [file path to read] [dir to save] [notification api]")        print("Example) python3 main.py ./test.txt ./ http://host.docker.internal:20000")        sys.exit()    api_url = sys.argv[3]    file_path = sys.argv[1]    file_name = get_filename_from_path(file_path)    target_path = os.path.join(sys.argv[2], file_name + ".json")     text = read_file(file_path)    nouns = extract_nouns(text)    save_data(target_path, {"nouns": nouns})    notify_done(api_url, file_name)    print("Done")

[word-counting]

import sys, json, os, requestsdef count_word(nouns_list):    count_dict = dict()    for noun in nouns_list:        if noun in count_dict:            count_dict[noun] += 1        else:            count_dict[noun] = 1    return count_dictdef load_data(path):    with open(path) as f:        json_data = json.load(f)    return json_datadef save_data(path, data):    with open(path, "w") as f:        json.dump(data, f)def get_filename_from_path(path):    return os.path.splitext(os.path.basename(path))[0]def notify_done(url, file_name):    requests.get(f"{url}/docker/word_count_done?file_name={file_name}")if __name__ == "__main__":    if len(sys.argv) < 4:        print("You must pass file path as an argument")        print("python3 main.py [file path to read] [dir to save] [notification api]")        print("Example) python3 main.py ./test.txt ./ http://host.docker.internal:20000")        sys.exit()    api_url = sys.argv[3]    file_path = sys.argv[1]    file_name = get_filename_from_path(file_path)    target_path = os.path.join(sys.argv[2], file_name + ".json")     json_data = load_data(file_path)    count_dict = count_word(json_data["nouns"])    save_data(target_path, {"result": count_dict})    notify_done(api_url, file_name)    print("Done")

For running the apps from the API server, I built both python files with below Dockerfiles.

[Tokenizer]

FROM python:3.9WORKDIR /appCOPY . .RUN pip install pipenvRUN pipenv installRUN pipenv run python3 -m textblob.download_corporaENTRYPOINT ["pipenv", "run", "python3", "./main.py"]

[word-counting]

FROM python:3.9WORKDIR /appCOPY . .RUN pip install pipenvRUN pipenv installENTRYPOINT ["pipenv", "run", "python3", "./main.py"]

API Server

This is the main code and it's kind of simple.

from flask import Flaskfrom dotenv import load_dotenvload_dotenv()app = Flask(__name__)import routes

Routes

[routes/docker.py]

import osfrom flask import jsonify, requestfrom server import appfrom lib import docker, jsonresult = []@app.route('/docker/tokenizer_done')def get_tokenizer_done():    file_name = request.args.get("file_name")    docker.run_word_count_container(file_name)    return "run a word_count container"@app.route('/docker/word_count_done')def get_word_count_done():    file_name = request.args.get("file_name")    json_data = json.load_data(        os.path.join(os.getenv("SHARED_VOLUME_PATH"),        "word_count_output",        f"{file_name}.json"    ))    result.append(json_data)    return "all works done"@app.route('/docker/result')def get_result():    file_name = request.args.get("file_name")    return jsonify({        "result": result    })

[routes/upload.py]

import osfrom flask import jsonify, requestfrom werkzeug.utils import secure_filenamefrom server import appfrom lib import docker@app.route("/upload", methods=["POST"])def upload_file():    f = request.files["file"]    file_name = secure_filename(f.filename)    f.save(os.path.join(os.getenv("SHARED_VOLUME_PATH"), "input", file_name))    docker.run_tokenizer_container(file_name)    return "succeed to upload"

[routes/__init__.py]

from routes import docker, upload

[lib/docker.py]

import osAPI_URL = os.getenv("API_URL")VOLUME_ROOT_PATH = os.getenv("SHARED_VOLUME_PATH")RUN_TOKENIZER_CONTAINER = 'docker run -it --add-host=host.docker.internal:host-gateway -v "' + VOLUME_ROOT_PATH + ':/shared_volume" hskcoder/tokenizer:0.2 /shared_volume/input/{FILE_NAME_WITH_EXTENSION} /shared_volume/tokenizer_output ' + API_URLRUN_WORD_COUNT_CONTAINER = 'docker run -it --add-host=host.docker.internal:host-gateway -v "' + VOLUME_ROOT_PATH + ':/shared_volume" hskcoder/word_count:0.2 /shared_volume/tokenizer_output/{FILE_NAME_WITHOUT_EXTENSION}.json /shared_volume/word_count_output ' + API_URLdef run_tokenizer_container(file_name):    print(RUN_TOKENIZER_CONTAINER.format(        FILE_NAME_WITH_EXTENSION = file_name    ))    os.popen(RUN_TOKENIZER_CONTAINER.format(        FILE_NAME_WITH_EXTENSION = file_name    ))def run_word_count_container(file_name):    os.popen(RUN_WORD_COUNT_CONTAINER.format(        FILE_NAME_WITHOUT_EXTENSION = file_name    ))

[iib/json.py]

import jsondef load_data(path):    with open(path) as f:        json_data = json.load(f)    return json_data

This app read environment variables from .env file, so you need to set up like this.
The below variables just fit my system.

API_URL=http://host.docker.internal:20000ROOT_PATH=C:\Users\hskco\OneDrive\ \stuff\docker\apiSHARED_VOLUME_PATH=C:\Users\hskco\OneDrive\ \stuff\docker\api\shared_volume

You can run the server with this script

python3 -m pipenv run flask run -h 0.0.0.0 --port 20000

Before running this command, you need to set an environment variable FLASK_APP.
Since I was developing in Windows, I ran this command in api dir.

$env:FLASK_APP = './server.py'

If you enter http://127.0.0.1/docker/result you must see this page.

Empty Result

Let's send a file to the API server and see the result.

[1]
step one

[2]
step two

[3]
result

Conclusion

It was really fun. I've learned a lot of things.
Regardless of your position, I think It would be really good to try anything you're interested in.

This example is really basic, I mean.
It should've considered like

  • Authorization
  • Communication between containers (In this example, the API server exposes all routes to the public)
  • Management containers (Containers that have done need to be deleted)
  • Deployment the API server

Without these, there must be many things that you need to consider. (I respect for backend developers) I was just focusing on implementing the system, Honestly, I didn't care much about others. If I did, I couldn't write this article. I'm going to be back to work tomorrow.

Anyways, it was fun, it's true :) I hope it'll be helpful for someone.

References

Github Source Code

Github Source Code

Python
Pipenv
Flask
Docker

Original Link: https://dev.to/lico/python-3-flask-interacts-with-docker-containers-3nce

Share this article:    Share on Facebook
View Full Article

Dev To

An online community for sharing and discovering great ideas, having debates, and making friends

More About this Source Visit Dev To