PyPI Package Updater

A Python package that automatically checks PyPI for new versions of your dependencies and updates your requirements files with comprehensive format support.

Features

• 🔍 Automatic Update Detection: Checks PyPI for the latest versions of all packages in your requirements files
• 📝 Universal Format Support: Handles requirements.txt, requirements.in, setup.py, and pyproject.toml files
• 🔄 Dependency-Aware Updates: Understands your requirements file hierarchy and updates in the correct order
• 🎯 Selective Updates: Update specific files or packages, or update everything at once
• 🛡️ Safe by Default: Dry-run mode and interactive confirmation before making changes
• 🔨 Integration: Automatically runs your existing compilation script after updates
• 📊 Detailed Reporting: Comprehensive summary of what was updated, failed, or skipped
• ✅ High Code Quality: 99% test coverage with comprehensive branch coverage
• 🚀 CI/CD Ready: Full GitHub Actions workflow with automated testing and deployment

Installation

Via pip (Python Package Index)

pip install pypi-package-updater

Or install with development dependencies:

pip install pypi-package-updater[dev]

Via Homebrew (macOS/Linux)

# Add the tap and install (one-time setup)
brew tap adambirds/homebrew-pypi-package-updater
brew install pypi-package-updater

# Or install directly without tapping
brew install adambirds/homebrew-pypi-package-updater/pypi-package-updater

Via APT (Debian/Ubuntu)

From the ADB APT Repository (Recommended)

# Add the custom APT repository
echo "deb https://adambirds.github.io/adb-apt-repo stable main" | sudo tee /etc/apt/sources.list.d/adb-apt-repo.list
curl -fsSL https://adambirds.github.io/adb-apt-repo/pubkey.gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/adb-apt-repo.gpg
sudo apt update
sudo apt install pypi-package-updater

Manual Installation from GitHub Releases

You can download pre-built packages from the GitHub Releases page:

• Python Wheel (.whl): For manual pip installation

  pip install pypi_package_updater-X.X.X-py3-none-any.whl

• Source Distribution (.tar.gz): For building from source

  pip install pypi_package_updater-X.X.X.tar.gz

• Debian Package (.deb): For Debian/Ubuntu systems

  sudo dpkg -i pypi-package-updater_X.X.X-1_all.deb

From Source

git clone https://github.com/adambirds/pypi-package-updater.git
cd pypi-package-updater
pip install -e .

Requirements

• Python 3.11+
• aiohttp
• packaging
• tomli (for pyproject.toml support)

Usage

Command Line Interface

Check for available updates (without making changes):

pypi-update --check-only

Update all packages interactively:

pypi-update

Update specific files:

pypi-update requirements/common.in requirements/dev.in

Dry run (see what would be updated):

pypi-update --dry-run

Non-interactive mode (update everything automatically):

pypi-update --non-interactive

Skip compilation after updates:

pypi-update --no-compile

Python API

import asyncio
from pypi_updater import PyPIUpdater

async def main():
    updater = PyPIUpdater(requirements_dir="requirements", tools_dir="tools")
    
    # Check for updates
    updates = await updater.check_for_updates()
    
    # Update packages
    summary = await updater.update_packages(
        dry_run=False,
        auto_compile=True,
        interactive=True
    )
    
    # Print summary
    updater.print_update_summary(summary)

asyncio.run(main())

Configuration

Supported File Formats

The tool automatically detects and supports multiple dependency file formats:

• requirements.txt - Standard pip requirements files
• requirements.in - pip-tools input files with -c, -e, compilation comments
• setup.py - setuptools configuration with install_requires
• pyproject.toml - Modern Python packaging format (PEP 518)

Directory Structure

The tool expects your project to follow this structure:

your-project/
├── requirements/
│   ├── common.in       # Base requirements
│   ├── dev.in          # Development requirements
│   ├── prod.in         # Production requirements
│   └── *.txt          # Compiled requirements (generated)
├── setup.py           # Optional: setuptools configuration
├── pyproject.toml     # Optional: modern Python packaging
└── tools/
    └── update-locked-requirements  # Your compilation script

Requirements File Format

The tool supports multiple dependency file formats with automatic detection:

requirements.txt / requirements.in

Standard pip requirements format with:

• Version pinning: Django==4.2.0
• Version ranges: requests>=2.25.0
• Includes: -r common.in
• Comments: # This is a comment
• Extras: django[postgres]==4.2.0

Example requirements/common.in:

# Core Dependencies
Django==4.2.0
requests>=2.25.0
celery==5.2.0

# Database
psycopg2==2.9.5

setup.py

Supports setuptools configuration:

from setuptools import setup

setup(
    name="my-package",
    install_requires=[
        "Django>=4.2.0",
        "requests>=2.25.0",
    ],
    extras_require={
        "dev": ["pytest>=6.0"],
    }
)

pyproject.toml

Modern Python packaging format (PEP 518):

[project]
dependencies = [
    "Django>=4.2.0",
    "requests>=2.25.0",
]

[project.optional-dependencies]
dev = [
    "pytest>=6.0",
    "black>=22.0",
]

# Poetry format also supported
[tool.poetry.dependencies]
python = "^3.11"
Django = "^4.2.0"

How It Works

1. Parse Requirements: Scans your .in files to extract package names and current versions
2. Check PyPI: Queries the PyPI API to get the latest version for each package
3. Calculate Updates: Compares current versions with latest versions
4. Update Files: Modifies your .in files with new versions (respects dependency order)
5. Compile Requirements: Runs your existing compilation script to generate .txt files

Advanced Features

Dependency Resolution

The tool understands your requirements hierarchy through -r includes and updates files in the correct order:

dev.in +-> prod.in +-> common.in
   |
   v
mypy.in

Files with no dependencies (like common.in) are updated first, followed by files that depend on them.

Interactive Mode

In interactive mode, you'll be prompted for each update:

Package: Django
  Current: 4.1.0
  Latest:  4.2.0
  File:    requirements/common.in
Update Django from 4.1.0 to 4.2.0? [y/N/q]:

• y: Update this package
• N: Skip this package (default)
• q: Quit the update process

Dry Run Mode

Use --dry-run to see what would be updated without making any changes:

pypi-update --dry-run

This is useful for:

• Checking what updates are available
• Testing the tool's behavior
• Generating reports for review

Error Handling

The tool handles various error conditions gracefully:

• Network issues: Retries and timeouts for PyPI API calls
• Invalid packages: Warns about packages not found on PyPI
• File permissions: Clear error messages for file access issues
• Version parsing: Handles non-standard version formats

Release and Distribution

This project uses automated CI/CD to build and distribute packages across multiple platforms:

Automated Publishing

On each release, the CI/CD pipeline automatically:

1. PyPI: Publishes Python packages (wheel and source distribution)
2. GitHub Releases: Uploads multiple package formats for manual download
3. Homebrew: Updates the Homebrew tap with the latest version

Package Formats Available

• Python Wheel (.whl): Universal Python package for pip install
• Source Distribution (.tar.gz): Python source package
• Debian Package (.deb): For Debian/Ubuntu systems
• Homebrew Formula: For macOS and Linux via Homebrew

Development

Development Setup

# Clone the repository
git clone https://github.com/adambirds/pypi-package-updater.git
cd pypi-package-updater

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install in development mode with all dependencies
pip install -e ".[dev]"

Code Quality & Testing

This project maintains high code quality standards with comprehensive tooling:

Running Tests

# Run all tests (263 tests with 99% coverage)
pytest

# Run tests with coverage report
pytest --cov=pypi_updater --cov-report=html

# Run specific test categories
pytest tests/test_basic.py                    # Basic functionality
pytest tests/test_branch_coverage.py         # Branch coverage tests
pytest tests/test_formats.py                 # File format handling
pytest tests/test_updater.py                 # Main updater logic

Code Linting & Formatting

We use automated linting with Black, isort, and mypy:

# Run all linters (check mode)
python scripts/lint.py

# Run all linters with automatic fixes
python scripts/lint.py --fix

# Individual tools
black pypi_updater/ tests/                   # Code formatting
isort pypi_updater/ tests/                   # Import sorting  
mypy pypi_updater/                           # Type checking

Test Coverage

• Current Coverage: 99% (654 statements, 262 branches)
• Statement Coverage: 100% (all lines covered)
• Branch Coverage: 99% (252/262 branches covered)
• Total Tests: 263 comprehensive tests

CI/CD Pipeline

The project includes a complete GitHub Actions workflow (.github/workflows/ci-cd.yml):

Continuous Integration

• Multi-Python Testing: Python 3.11, 3.12, 3.13
• Code Quality Checks: Black, isort, mypy validation
• Test Execution: Full test suite with coverage reporting
• Coverage Upload: Automatic codecov.io integration

Continuous Deployment

• PyPI Publishing: Automatic release on version tags
• Release Automation: GitHub releases with changelogs

GitHub Secrets Setup

To enable full CI/CD functionality, configure these repository secrets:

Required Secrets

1. CODECOV_TOKEN - For coverage reporting

   # Get from https://codecov.io/gh/adambirds/pypi-package-updater
   # Add to GitHub repo settings → Secrets and variables → Actions

2. PAT_TOKEN - Personal Access Token for updating homebrew tap

   # Create at GitHub Settings → Developer settings → Personal access tokens
   # Scope: repo (full repository access)
   # Add as repository secret named: PAT_TOKEN
   # Used to update the separate homebrew tap repository

PyPI Publishing

This project uses PyPI Trusted Publishing for secure, token-free publishing to PyPI. No PYPI_API_TOKEN is required - the publishing is authenticated automatically through GitHub's OIDC provider.

Automatically Provided

• GITHUB_TOKEN - Automatically provided by GitHub Actions

Release Process

1. Update Version: Bump version in pyproject.toml
2. Create Tag: git tag v1.0.0 && git push origin v1.0.0
3. Automatic Deployment: GitHub Actions builds and publishes to PyPI
4. GitHub Release: Automatic release creation with changelog

Configuration Files

pyproject.toml

• Tool configuration for Black, isort, mypy
• Package metadata and dependencies
• Build system configuration

codecov.yaml

• Coverage requirements (95% project, 90% patch)
• Coverage status checks configuration

scripts/lint.py

• Automated linting script with --fix mode
• Colored output and comprehensive reporting

Contributing

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Make your changes with tests
4. Run quality checks: python scripts/lint.py --fix && pytest
5. Ensure tests pass: All 263 tests should pass with maintained coverage
6. Submit a pull request

Pull Request Requirements

• ✅ All tests pass (99%+ coverage maintained)
• ✅ Code follows Black formatting
• ✅ Imports sorted with isort
• ✅ Type hints pass mypy validation
• ✅ New features include tests
• ✅ Documentation updated if needed

License

MIT License - see LICENSE file for details.

Changelog

See CHANGELOG.md for release history and details.