apko icon indicating copy to clipboard operation
apko copied to clipboard

Fix APK resolver for version constraints on provided packages

Open AmberArcadia opened this issue 4 months ago • 1 comments

Summary

This PR fixes a critical bug in the APK resolver that was causing numpy and pytorch packages to fail building in Wolfi/Chainguard. The resolver wasn't correctly handling version constraints when packages were found via their provides entries.

The Problem

When migrating from py3-numpy to versioned packages like py3.12-numpy, we hit an issue:

  1. Package structure:

    • py3.12-numpy (version 2.1.0) provides py3-numpy=2.1.0
    • py3.12-numpy (version 1.26.4) provides py3-numpy=1.26.4
    • PyTorch requires py3-numpy<2.0
  2. The bug:

    • When resolving py3-numpy<2.0, APK would find packages providing py3-numpy
    • But it incorrectly checked the package's own version (e.g., py3.12-numpy version 2.1.0) against the constraint
    • It didn't check the version specified in the provides clause (py3-numpy=2.1.0)
    • Result: APK couldn't find ANY package satisfying py3-numpy<2.0, even though py3.12-numpy-1.26.4 provides py3-numpy=1.26.4

The Solution

This PR adds context awareness to the package filtering logic:

  1. Query context tracking: Added queryName to track which name was used to find packages
  2. Proper version checking:
    • When querying py3-numpy<2.0, only checks versions relevant to py3-numpy
    • Checks the version in the provides clause when a package was found via provides
    • Only checks the package's own version when querying by its actual name
  3. Backward compatibility: For provides without explicit versions, falls back to the package's version

Why This Matters

This fix is critical for:

  1. Unblocking builds: PyTorch and other packages depending on numpy can now build
  2. Python package migration: Enables migration from py3-* to py3.XX-* versioned packages
  3. Preventing future conflicts: Each Python version can have its own package versions without conflicts
  4. Proper constraint solving: APK can now correctly resolve complex version constraints on virtual packages

Testing

  • Added comprehensive test case TestNumpyVersionConstraintWithProvides that reproduces the exact scenario
  • All existing tests pass
  • Verified the fix handles:
    • Direct version constraints on provided names
    • Transitive dependencies with version constraints
    • Provides with and without explicit versions

Impact

This fixes the current build failures in:

  • https://github.com/chainguard-dev/extra-packages/pull/3792 (pytorch pinning)
  • https://github.com/wolfi-dev/os/pull/56095 (numpy package removal issues)

And enables the planned migration away from py3-* packages as discussed in the incident.

🤖 Generated with Claude Code

Co-Authored-By: Claude [email protected]

AmberArcadia avatar Sep 16 '25 20:09 AmberArcadia

As far as I can tell the tests added here work with the current version of apko: https://github.com/chainguard-dev/apko/pull/1862

justinvreeland avatar Sep 19 '25 17:09 justinvreeland