Publish a Python Package the Right Way (pyproject.toml + CI)

Quickstart

If you want the highest-impact steps first, follow this mini-path. You can do it on an existing repo or a fresh one. The goal is simple: build locally, test in CI, publish from a tag.

Overview

“Publish a Python package” sounds like a single action, but it’s really a chain of small contracts: the metadata contract (name/version), the build contract (how artifacts are created), the dependency contract (what you require), and the distribution contract (what ends up in the wheel/sdist). A modern packaging workflow makes those contracts explicit and verifiable.

This post focuses on a practical default path for pure-Python packages and small libraries. If you ship compiled extensions (C/C++/Rust), you’ll add platform wheels and more build steps, but the core ideas remain the same: explicit configuration, artifact verification, and CI-driven releases.

Core concepts

pyproject.toml: one file to describe the build and the project

pyproject.toml is the modern center of Python packaging. It can define: (1) how your project is built (the build backend), and (2) your project metadata (name, version, dependencies, entry points). The win is not “new syntax” — it’s that tools stop guessing.

Wheel vs sdist: why you should build both

When you publish to PyPI, you typically upload one or more artifacts: a wheel (pre-built, fast install) and an sdist (source distribution). A common mistake is uploading “only a wheel” and accidentally omitting source files or package data. Another is uploading “only an sdist” and forcing end users to build during install.

CI as the release gate: you want one way to publish

If releases happen manually from a developer machine, sooner or later you’ll ship a broken build: missing files, wrong Python version, stale dependency lock, or an uncommitted local change. CI fixes this by making “publish” a predictable, logged process: tests run, artifacts are built, artifacts are checked, then uploaded.

Versioning and compatibility: be intentional

Users rely on your version numbers for safety. You don’t need a complicated policy, but you do need consistency: decide how you bump versions, document what “breaking” means for your API, and declare what Python versions you support.

Step-by-step

This is a complete, practical flow to publish a Python package the right way. You can adapt it to your tooling preference, but the structure matters: layout → metadata → build → verify → CI → publish.

Step 1 — Choose a clean project layout (use src/)

A src layout prevents a classic trap: Python importing your package from the repository folder instead of the installed artifact. That trap hides missing files and broken packaging until your users hit it.

Step 2 — Write pyproject.toml (metadata + build backend)

Here’s a compact example using a modern build backend and a src layout. Treat it as a starting point: fill in your real name, URLs, classifiers, and dependencies.

[build-system]
requires = ["hatchling>=1.18"]
build-backend = "hatchling.build"

[project]
name = "your-package-name"
version = "0.1.0"
description = "Short, accurate one-liner for PyPI."
readme = "README.md"
requires-python = ">=3.10"
license = { file = "LICENSE" }
authors = [{ name = "Your Name" }]
keywords = ["python", "packaging"]
classifiers = [
  "Programming Language :: Python :: 3",
  "License :: OSI Approved :: MIT License",
  "Operating System :: OS Independent",
]

dependencies = [
  "requests>=2.31",
]

[project.urls]
Homepage = "https://github.com/your-org/your-repo"
Issues = "https://github.com/your-org/your-repo/issues"

[project.optional-dependencies]
dev = [
  "pytest>=8",
  "ruff>=0.6",
]

[project.scripts]
yourcli = "your_pkg.cli:main"

[tool.hatch.build.targets.wheel]
packages = ["src/your_pkg"]

Step 3 — Build locally and verify the artifacts

Don’t wait for CI to tell you your distribution is broken. Build once locally, check the metadata, and run a clean install test. This catches missing files, wrong packaging configuration, and README rendering issues early.

# Create a clean build environment
python -m venv .venv
source .venv/bin/activate

python -m pip install -U pip build twine

# Run your tests (optional but recommended)
python -m pip install -e ".[dev]"
pytest

# Build wheel + sdist into ./dist
python -m build

# Check distribution metadata for common problems
twine check dist/*

# Clean install test: new venv, install only the built wheel
deactivate
python -m venv .venv_install
source .venv_install/bin/activate
python -m pip install -U pip
python -m pip install dist/*.whl

# Smoke test import and CLI (adjust to your package)
python -c "import your_pkg; print('ok')"
yourcli --help

Step 4 — Add CI: test on push, build + publish on tags

Your CI should do two things reliably: (1) run tests for every change, and (2) publish only when you explicitly release. Below is a GitHub Actions workflow that runs tests and publishes to PyPI when you push a tag like v1.2.3. It uses a trusted publishing flow (OIDC) so you don’t need to store a long-lived PyPI API token in GitHub.

name: CI

on:
  push:
    branches: ["main"]
    tags: ["v*"]
  pull_request:

permissions:
  contents: read
  id-token: write  # needed for trusted publishing (OIDC) to PyPI

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install
        run: |
          python -m pip install -U pip
          python -m pip install -e ".[dev]"
      - name: Test
        run: pytest

  build-and-publish:
    needs: test
    if: startsWith(github.ref, 'refs/tags/v')
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - name: Build
        run: |
          python -m pip install -U pip build twine
          python -m build
          twine check dist/*
      - name: Publish to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1

Step 5 — Do a safe first release (TestPyPI → PyPI)

Your first release is where most “oops” moments happen: missing README, broken classifiers, wrong package name, or missing files in the wheel. TestPyPI is a low-risk rehearsal that helps you confirm the whole pipeline.

Step 6 — Maintain like a grown-up: yanks, hotfixes, and metadata hygiene

Even with a careful workflow, mistakes happen. The difference between “annoying” and “disaster” is whether you can recover cleanly. Keep releases small, automate publishing, and make it easy to ship a patch version when needed.

Common mistakes

If you’ve ever hit “ModuleNotFoundError after pip install” or “my files aren’t in the wheel”, you’re not alone. These are the packaging pitfalls that show up in real projects — plus how to fix them fast.

FAQ

Do I still need setup.py?

In most modern workflows, no. For many projects, pyproject.toml is enough, and your build backend reads everything from there. Some legacy or highly customized setups still use setup.py, but it’s increasingly a compatibility layer rather than the primary configuration.

Which build backend should I choose: setuptools, hatchling, flit, poetry?

Pick the one that matches your complexity. If you want a lightweight, standards-first workflow, a simple backend (like hatchling or flit) is great. If you need maximum ecosystem compatibility or complex build customization, setuptools remains common. If you want an opinionated tool that also manages environments, Poetry can be convenient — just ensure you can build wheel/sdist reliably in CI.

How do I include package data (templates, JSON files, etc.)?

Prefer explicit inclusion rules via your build tool configuration, then validate by installing from the wheel and verifying the file exists at runtime. Avoid relying on “it’s in the repo so it must be included” — wheel contents are defined by your build configuration, not your Git tree.

Should I publish both wheel and sdist?

Yes, by default. Wheels provide fast installs, while sdists provide a source fallback and transparency. Building both also helps catch misconfigurations early (because sdists often expose missing files or metadata issues).

How do I publish pre-releases (alpha/beta/rc)?

Use a pre-release version scheme (for example: 1.2.0rc1 or 1.2.0b1) and tag it explicitly. Many users won’t install pre-releases unless they opt in, which is exactly what you want for “try it early” builds.

What’s the safest way to publish from CI?

Publish only on tags/releases, run tests first, build artifacts in CI, run a metadata check, and then upload. If your hosting supports it, trusted publishing (OIDC) avoids storing long-lived tokens and reduces secret leakage risk.

I published a broken release — what now?

Don’t panic-delete and re-upload the same version. The clean recovery path is: fix the issue, bump a patch version, and publish again. If needed, you can discourage installs of a specific version (for example by yanking) while keeping history intact.

Cheatsheet

Keep this as your “shipping checklist” for every Python release. If you follow it, packaging becomes boring — in the best way.

Wrap-up

The “right way” to publish a Python package is the way that stays stable under pressure: a single source of truth (pyproject.toml), a build you can reproduce (CI artifacts), and a release mechanism that’s explicit (tags).

If you implement only one thing from this post, make it the clean install test from the built wheel. It’s the fastest way to align your repo reality with your users’ reality.

Related posts

• Python in 60 Minutes: From Zero to Useful
• Top 25 Python Errors (And the 1-Line Fixes)
• Python Dataclasses: Cleaner Models With Less Boilerplate
• Make Python Faster: 12 Micro-Optimizations That Matter
• Asyncio Explained With 3 Real Projects
• Build a Beautiful CLI Tool: Arguments, Colors, and TUI Basics