Make(file) a Python Repository

Create a Makefile for deploying Python

make
python
tools
Author

Brandon Rundquist

Releasing code requires a series of different commands that can become unruly. A Makefile can keep available commands tidy and unify how they are executed.

What is make

In 1976 there was a command line tool created called make. It was originally made to keep build files in sync for writing in C. The same principals can be used for building Python code.

The structure of make is as follows.

target: prerequisites | order only prereqs
    recipe
  • target: This is the command you will use at the command line (make target) and can be used as a prereq to other targets.
  • prerequisites: These are things that have to be run before the recipe is run.
  • order only prequisites: Only runs when the file does not exist.
  • recipe: The set of commands to run for this target.

Benefits

Creating jobs and dependent jobs is easy. The format target will run the lint target first.

lint:
    python3 -m pylint .
format: lint
    python3 -m black .

This will not run if the requirements.txt file has not changed.

install: requirements.txt
    python3 -m pip install -r requirements.txt

Create the venv folder if it does not exist and ignore if it does.

install: requirements.txt | venv 
    python3 -m pip install -r requirements.txt
venv:
    python3 -m venv venv

Gotchas

Below are a few gotchas and how to get around them.

  • Each line is executed in a different subshell. This is a problem if sourcing virtual environments and then running commands since that will deactivate the virtual environment.
    • The solution is point the Python interpreter and other required tools directly to their path.
    • $(PYTHON) := venv/bin/python $(PYTHON) -m ruff
  • Keep track of changes with a hidden file in the format .{target}.stamp.
    • .lint.stamp will keep track of when lint happened last. # Makefile for Python venv

.PHONY: install clean lint format test docker-build

# Variables
GLOBAL_PYTHON := python3

VENV_DIR := venv
PYTHON := $(VENV_DIR)/bin/python
PIP := $(VENV_DIR)/bin/pip

REQUIREMENTS := requirements.txt
DOCKERFILE := Dockerfile
PYTHON_FILES := $(shell find . -name '*.py')

LINT_STAMP := .lint.stamp
FORMAT_STAMP := .format.stamp
DOCKER_STAMP := .docker.stamp

# Targets

# Install: Set up the virtual environment and install dependencies
install: $(VENV_DIR)/bin/activate

$(VENV_DIR)/bin/activate: $(REQUIREMENTS) | $(VENV_DIR)
    python3 -m venv $(VENV_DIR)
    $(PIP) install --upgrade pip
    $(PIP) install -r $(REQUIREMENTS)
    @touch $(VENV_DIR)/bin/activate

# Ensure the venv directory exists
$(VENV_DIR):
    $(GLOBAL_PYTHON) -m venv $(VENV_DIR)

# Lint: Run flake8 only if Python files changed
lint: $(LINT_STAMP)

$(LINT_STAMP): $(PYTHON_FILES) | $(VENV_DIR)/bin/activate
    $(PIP) install flake8
    $(VENV_DIR)/bin/flake8 $(PYTHON_FILES)
    @touch $(LINT_STAMP)

# Format: Run black only if Python files changed
format: $(FORMAT_STAMP)

$(FORMAT_STAMP): $(PYTHON_FILES) | $(VENV_DIR)/bin/activate
    $(PIP) install black
    $(VENV_DIR)/bin/black $(PYTHON_FILES)
    @touch $(FORMAT_STAMP)

# Test: Run tests with pytest
test: $(VENV_DIR)/bin/activate
    $(PIP) install pytest
    $(VENV_DIR)/bin/pytest tests/

# Docker Build: Build the Docker image if Dockerfile or requirements.txt changes
docker-build: $(DOCKER_STAMP)

$(DOCKER_STAMP): $(DOCKERFILE) $(REQUIREMENTS)
    docker build -t my-python-app .
    @touch $(DOCKER_STAMP)

# Clean: Remove the virtual environment, stamp files, and any other generated files
clean:
    rm -rf $(VENV_DIR) $(LINT_STAMP) $(FORMAT_STAMP) $(DOCKER_STAMP)
    find . -type d -name "__pycache__" -exec rm -r {} +
    rm -rf .coverage coverage_html_report/