Curious about using GitLab CI for efficient Python testing? This article covers building robust CI pipelines in GitLab that streamline Python development, with essentials like unit tests, integration tests, linting, and migrations. Get ready to elevate your testing game and make your pipelines both effective and enjoyable to create!
Base Library Test
This test assumes we are not regularly building a Docker image but instead will publish a python library. The key steps are:
We need the base image defined (the python version we will use), this should be defined on the top as it ensures there are no mismatches or drift
Define the Poetry version assuming you use Poetry which you should ;)
Install Poetry with some default settings
Have an easy way to transfer the Environment variables
Combine the base and caching
image: python:3.11-buster
stages:
- test # at the moment we dont consider other steps
variables:
POETRY_VERSION: 1.8.0
.env_variables: &setup_env_template
- cp .env.example .env
- ./scripts/replace_env.sh
.install_poetry: &install_poetry
- pip install poetry
- poetry config virtualenvs.create true # this will help with caching venv
- poetry install
.test:
stage: test
before_script:
- *setup_env_template
- *install_poetry
cache: # this will speed up poetry installs drastically
paths:
- .venv
rules: # modify this to your liking
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: always
- when: never
Environment variable injection
Sometimes there are variables that you need within your project to come from the environment. You can define those as CI variables in the job and then inject them into an .env file.
#!/bin/bash
while IFS='=' read -r key value; do
if [ -n "${!key}" ]; then
sed -i "s/^$key=.*$/$key=${!key}/" .env
fi
done < .env.example
Unit Tests
Unit tests can sometimes be difficult do delineate but in this case we will just define them as tests that do not need any services (such as databases). For these tests we want to have Coverage in both the MR displayed as well as in the MR Code viewer. This is what the coverage and artifacts are.
tests:unit:
extends: .test
variables:
APP_ENV: "test"
script:
- ./scripts/unit.sh
coverage: /TOTAL.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/
artifacts:
reports:
junit: report.xml
coverage_report:
coverage_format: cobertura
path: coverage.xml
Testing with Pytest
You can ofcourse use unittest, tox or any other testing frameworks but I prefer pytest. You will need the pytest-cov and pytest-xdist dependency installed. Having this in a separate file allows you to run the same script locally which can come in handy.
#!/bin/bash
# Passed parameters are forwarded to pytest.
set -e
# Execute all remaining tests
python -m pytest \
-n 8 \ # this will do paralel tests (will produce flakyness with DBs)
--junitxml=./ci-reports/ju-report.xml \ # creates the correct cov report
--durations=15 \
--disable-warnings \
--cov \
--cov-config .coveragerc \
--cov-branch \
--cov-report term \
--cov-report xml:./ci-reports/coverage-report.xml \
"${@}"
Integration Tests
Suppose you have a codebase that will use several Databases, and you wish to test those. You would use services in the GitLab CI. Here is also where the .env file is relevant since you will need a local version but the same thing will be needed in CI. For the sake of example, we use Redis, MongoDB and Postgres in here.
test:
extends: .test
variables:
API_PG_HOST: db
API_REDIS_HOST: redis
API_MONGO_HOST: mongo
API_ENV: test
script:
- ./scripts/unit.sh # make sure you dont have paralelism enabled
coverage: /TOTAL.*?\s(?:\d+\s+\d+\s+)(100(?:\.0+)?%|[1-9]?\d(?:\.\d+)?%)/
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: coverage.xml
services: # make sure the passwords are defaults or use the variables in the env
- name: postgres:latest
alias: db
variables:
POSTGRES_DB: admin
POSTGRES_USER: admin
POSTGRES_PASSWORD: password
- name: redis:latest
alias: redis
- name: mongo:latest
alias: mongo
variables:
MONGO_INITDB_ROOT_USERNAME: admin
MONGO_INITDB_ROOT_PASSWORD: password
Migrations
In the case of SQL databases its highly advisable to run migrations as part of the testing suite so you can identify breaks. In this case, we use Alembic but any other framework will do.
NOTE: In some cases you would run the migrations prior to the integration tests, depending on your setup.
migrations:
extends: .test
variables:
API_PG_HOST: db
API_ENV: test
script:
- poetry run alembic upgrade head
services: # make sure these variables match
- name: postgres:latest
alias: db
variables:
POSTGRES_DB: fonapi
POSTGRES_USER: admin
POSTGRES_PASSWORD: password
In some cases, you need to modify the env file in the alembic folder. Since I usually put source code in /src this is needed.
# this will enable Alembic to find your classes even when you have a different IDE setup
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..', 'src')))
# Import the Settings class
from ninja_api.config import settings
from ninja_api.models import Base # Import your Base class from models
# this is the Alembic Config object, which provides access to the config
# via `config` attribute.
config = context.config
# Set the SQLAlchemy URL dynamically from settings (assume you use pydantic settings)
config.set_main_option('sqlalchemy.url', settings.pg.database_url)
Code Quality Pipelines
Besides the unit tests, we would usually analyse the codebase for quality and type checking if you are a fan of MyPy.
Linting (with combo of services)
You can combine several tools to check various aspects of the code such as code smells, import sorting, formatting, etc… Of course you will need all these dependencies installed.
#!/bin/bash
# Executes all configured linters
set -e
echo "Running dotenv checks..."
dotenv-linter .env.example
echo "Running flake8..."
flake8 .
echo "Running isort checks..."
isort --check-only .
echo "Running black checks..."
black --check .
# Don't do `docformatter --recursive --check .` because docformatter has no exclude
# feature, so that would try to format site-packages.
echo "Running docformatter..."
docformatter --recursive --check src/ scripts/ tests/
echo "Running mypy..."
mypy src scripts tests
echo "Good to go! Nice.."
Ruff
Alternatively, you can be a total Chad and use Ruff to take care of everything blazing fast.
ruff:
extends: .test
script:
- poetry run ruff check
You can specify the Ruff settings in the pyproject.toml conveniently.
[tool.ruff]
line-length = 88 # Use the same line length as Black if needed
lint.select = ["E", "F", "I", "N", "Q"] # Example: select codes for flake8, isort, etc.
exclude = ["alembic"]
[tool.ruff.lint.isort]
known-third-party = ["sqlalchemy", "fastapi", "pydantic"]
known-first-party = ["ninja_api"]
section-order = ["future", "standard-library", "third-party", "first-party", "local-folder"]
Type checking
Finally, to sleep better at night knowing all the type-related bugs will not plague your code you run a MyPy pipeline.
mypy:
extends: .test
script:
- poetry run mypy src/
And the settings can be specified in mypy.ini. With the main importance being the python version any plugins you might need and your desired strictness.
[mypy]
python_version = 3.12
plugins = sqlalchemy.ext.mypy.plugin
disallow_untyped_calls = True
disallow_untyped_defs = True
ignore_missing_imports = True
strict_optional = True
Service Tests
Suppose you are not building a library but instead have a Docker service. The main difference there is you can simply skip the poetry installation step and directly use the image as your image for the test. Modifying the test base.
.test:
stage: test
image: $CI_IMAGE # this could be the image you built in the previous step
before_script: *setup_env_template
rules: # modify these to your liking
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: always
- when: never
Conclusion
With all these, you should have an easy time keeping your codebase well-tested and kept at a high quality.