Publishing Python Libraries in GitLab

This article provides a concise guide on publishing Python libraries to the GitLab Package Registry using CI/CD tools. Discover how creating and sharing modular libraries can streamline dependencies and enhance code organization across your organization’s projects.

GitLab Package Registry

Within Gitlab we can utilize its Packages Registry to store all the created libraries, big or small. Keep them versioned and easily accessible.

Building and Publishing after the Tag

We want the job to run whenever a tag event happened, which should build the Python package and push it to the registry. We use CI Rules to establish the trigger on Tag events. We could also add specific rules on which branch and the tag structure to prevent publishin of tags in undesired situations.

In the following .gitlab-ci.yaml job snippet, we:

• ensure Poetry is installed

• we add access to any internal library repositories (this is optional if other libraries exist and are needed

• we add the dynamic versioning which will ensure the version used is based on the Git Tag

• and finally we publish the wheel using Twine

publish:
  stage: publish
  only:
    refs:
      - tags
  script:
    - pip install --upgrade pip
    - pip install twine poetry
    
    - poetry config repositories.gitlab 
      $CI_API_V4_URL/groups/XXXX/-/packages/pypi/simple
    
    - poetry config http-basic.gitlab gitlab-ci-token $CI_JOB_TOKEN
    
    # for this part to work, see "poetry-stub" file
    - pip install poetry-dynamic-versioning
    - poetry build -f sdist
    
    - TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token
      python -m twine upload --verbose --repository-url
      $CI_API_V4_URL/projects/${CI_PROJECT_ID}/packages/pypi dist/*

Creating a GitLab Release via the Tag mechanism

Creating a Git tag with the appropriate Semantic versioning will create a release in GitLab that can be tracked, with all the information included based on the tag message. This is the .gitlab-ci.yaml job which takes care of creating a release on a tag event.

release:
  stage: release
  image: registry.gitlab.com/gitlab-org/release-cli:latest
  script:
    - echo "Creating release $CI_COMMIT_TAG"
  release:
    name: '$CI_COMMIT_TAG'
    tag_name: '$CI_COMMIT_TAG'
    description: '$CI_COMMIT_TAG_MESSAGE'
  rules:
    - if: $CI_COMMIT_TAG =~ /[0-9]+\.[0-9]+\.[0-9]+/

Setting up Poetry for the Dynamic Versioning

In order to use Dynamic Versioning, we need to add the following parts to the project.toml. We need to keep the version of the library 0.0.0 as it will be dynamically changed. We add a snippet at the bottom setting up how we want the versioning to function.

[tool.poetry]
name = "python-library-name"
version = "0.0.0" # keep this version at 0.0.0 to make it dynamically changed
description = "Description for the package"
authors = ["Name <email>"]
  
[tool.poetry-dynamic-versioning]
enable = true
vcs = "git"
style = "semver"
pattern = "(?P<base>\\d+(\\.\\d+)*)"
latest-tag = true

Conclusion

With this you will be able to easily version and publish Python libraries within your organization. Next topics might be, how to create semantic versioning automatically from commits, how to manage dependency graphs for Python projects, and how to use published libraries within your projects using Poetry.