Use Make to Power Up Your Python Development

When you think of GNU Make what is the first thing that pops up to your mind? Perhaps you remember building C or C++ programs from source and automatically associate it with languages where you need to build the code before being able to run anything.

However, Make can be a really powerful tool for your other projects too. In this article I’ll show you some examples on how to utilize Make for Python development. I’ll be using the Flask web framework for demonstration purposes but these principles can be really adapted to any other project.

Note that you can’t directly copy the Makefile examples from this page as they are indented with spaces instead of tabs which is a requirement for Make.

Initialize a virtual environment

Your typical Python project might have the following structure where the main code resides in one module and test in another one. The project dependencies are defined in the requirements.txt file. Because your code probably lives in a Git repository it should also have a .gitignore file for managing files that are unwanted for source control.

.
├── .gitignore
├── app
│   ├── __init__.py
│   └── main.py
├── requirements.txt
└── tests
    ├── __init__.py
    └── test_main.py

The first thing that someone cloning your repository wants to probably do is to set up a Python virtual environment to be able to run your code.

Nowadays the recommended way to set up a virtual environment is to use the venv module that comes built in with Python 3. For example:

>> python3 -m venv venv

This creates a virtual environment venv inside the project directory. You should have it in your .gitignore file too:

venv/

Because we want to develop a Flask application we need to first define the dependency in the requirements.txt file:

Flask==1.1.1

Then the next thing to do is to activate the virtual environment and install the dependencies with pip.

>> . venv/bin/activate
>> python3 -m pip install -r requirements.txt

And now you have a virtualenv ready for running the code. Next, let’s automate the process. Create a new file called Makefile at the project root directory with these contents (make sure the file is tab indented):

.PHONY: venv
venv:
    python3 -m venv venv && . venv/bin/activate && python3 -m pip install -r requirements.txt

Now you can set up the environment with a simple call to make venv!

The second line defines the name of the Make target. If a file or directory with the same name exists in the directory make wouldn’t normally do anything.

>> make venv
make: 'venv' is up to date.

This is why I have defined it as a phony target. The .PHONY prerequisite target makes Make ignore the venv directory if it exists so that the dependencies are always updated when the target is invoked. You can read more about the usage of phony targets from here.

Finally, the shell command is just all the previous manual commands chained together.

Run a local server

Let’s create a simple application that can be tested. Create the app/main.py file and insert this code to create a super simple web service that can calculate squares of numbers.

from flask import Flask
app = Flask(__name__)


@app.route("/square/<value>")
def square(value):
    return f"The square of {value} is {(int(value)**2)}"

if __name__ == '__main__':
    app.run()

I know this isn’t that exciting example but it serves the purpose. There is no input validation or anything but the point of this tutorial is not exactly to teach how to develop with Flask.

To run the application server locally you would need to first enable the virtual environment and then run the application with python app/main.py. Let’s add this to the Makefile:

.PHONY: dev
dev: venv
    . venv/bin/activate && python3 app/main.py

Now if you run make dev the development server should start up on http://localhost:5000. Try out the square endpoint, for example http://localhost:5000/square/5 should return “The square of 5 is 25”.

The dev target uses venv as a dependency. This means that you don’t need to explicitly set up the virtual environment before running the development server but it will be created automatically instead.

Run tests

The next thing you might want to do is to run unit tests for your software. While it is quite easy to run tests in your IDE it can also be handy to have a command line version. With the unit test target you could run the tests immediately after cloning the repository to make sure that they work on your system, or they could be configured to run in your CI system.

Create this simple unit test file to tests/test_app.py. Also make sure that you have an empty __init__.py file in your tests directory as it is required for the unittest module to discover your tests.

import unittest

from app.main import app


class TestApp(unittest.TestCase):
    def setUp(self):
        app.config["TESTING"] = True
        self.app = app.test_client()

    def test_square(self):
        response = self.app.get("/square/5")
        self.assertEqual(b"The square of 5 is 25", response.get_data())

Again, I’m not going too much into the details of unit testing and this is an extremely simplified example written for this article. Nevertheless, the test should pass.

Add this to the Makefile:

.PHONY: test
test: venv
	. venv/bin/activate && python3 -m unittest discover -v

Now you should be able to use make test to run the unit tests.

Build and run a Docker image

To run the Flask application on Docker I am using this base image from the Docker Hub. It contains the uWSGI application server and an nginx proxy in front of it. The required Dockerfile is really simple:

FROM tiangolo/uwsgi-nginx-flask:python3.7

COPY ./app /app

If you don’t have Docker installed and want to follow this tutorial you can go ahead and install it with the instructions from Docker Docs. Save the above example to a file called Dockerfile at the root of your project.

Let’s add a build target to the Makefile:

.PHONY: build
build: test
    docker build -t my-app .

As you can see calling make build will first call the dependency test which runs the unit tests before building the application to a Docker image. The docker build command tags the image as my-app but you can change it to match the name of your own project.

To run the container you’ll need to use the docker run command. Let’s add it to the Makefile too.

.PHONY: run
run: build
    docker run -d --name my-container -p 8080:80 my-app

The run endpoint will now ultimately create the virtual environment, run unit tests, build the image, and finally run the container locally with the command make run. Try accessing the container again at http://localhost:8080/square/5 and you should see it respond.

To stop the running container we could have the following target:

.PHONY: stop
stop:
    docker stop my-container

Let’s create one more target to clean up the system after we’re done with the project. It should remove the virtualenv, stop and delete the container and remove the image. Here is what I came up with:

.PHONY: clean
clean:
    docker stop my-container ; docker rm my-container ; docker rmi my-app ; true
    rm -rf venv

The first line of commands calls the docker commands for stopping and removing the container and removing the image. The ; character links these commands together so that even if they fail the execution continues to the next one. Otherwise the cleanup would only work if the container was actually running. The true at the end makes sure that the execution will continue on the next line so that the virtual environment will get deleted even if the Docker image hasn’t been built.

Conclusion

I hope that this has given you some ideas on how to utilize Make in your development processes. I’ve found it to be really useful to hide away those complicated command line calls that I don’t really want to try to remember.

How do you use/intend to use Make in your development?

p.s. I know that I’ve partly reimplemented some of the funtionality found from Docker Compose. However, I think that Make is handy with single container repositories as you’re probably going to build and push images to a separate container registry instead of running the containers locally. For that you could have a deploy target that builds the container image with a tag from your version control and pushes it to the registry.