apache
Fix: Python build dependencies installation
Improve wheel and cython dependency management in gpMgmt/bin/Makefile to handle Ubuntu 24.04's PEP 668 restrictions while maintaining compatibility with Rocky Linux and older Ubuntu versions.
Changes:
- Split wheel and cython dependency checks into separate commands
- Add fallback to --break-system-packages flag for Ubuntu 24.04+
- Only install dependencies if not already present in the system
- Maintain backward compatibility with existing build environments
This resolves build failures on Ubuntu 24.04 where pip install --user is restricted by default, while preserving the existing behavior on Rocky Linux 8/9 and Ubuntu 20.04/22.04 systems.
Fixes #1433
What does this PR do?
Type of Change
- [ ] Bug fix (non-breaking change)
- [ ] New feature (non-breaking change)
- [ ] Breaking change (fix or feature with breaking changes)
- [ ] Documentation update
Breaking Changes
Test Plan
- [ ] Unit tests added/updated
- [ ] Integration tests added/updated
- [ ] Passed
make installcheck - [ ] Passed
make -C src/test installcheck-cbdb-parallel
Impact
Performance:
User-facing changes:
Dependencies:
Checklist
- [ ] Followed contribution guide
- [ ] Added/updated documentation
- [ ] Reviewed code for security implications
- [ ] Requested review from cloudberry committers
Additional Context
CI Skip Instructions
We also need to update the installation docs to install the required dependencies before building Cloudberry.
Could you explain why we need flag --break-system-packages? Where are these py packages installed, system library path for python? Is it possible to install them under $GPHOME/lib?
Could you explain why we need flag
--break-system-packages? Where are these py packages installed, system library path for python? Is it possible to install them under$GPHOME/lib?
Hi @gfphoenix78 Thanks for the review!
The --break-system-packages flag is needed for Ubuntu 24.04 and newer, which enforce PEP 668
restrictions. This policy prevents pip install --user from modifying system-managed Python environments. Without this flag, pip will refuse to install packages like wheel or cython, causing build failures.
Regarding the installation location: when using --user, the packages are installed under the user’s local site directory (e.g., ~/.local/lib/python3.x/site-packages), not under the system library path. This avoids conflicts with system Python.
Installing them into $GPHOME/lib would require customizing PYTHONPATH and ensuring consistent Python environments across all build and runtime contexts, which might introduce complexity and compatibility issues. The current approach minimizes intrusion into the system and maintains backward compatibility with other platforms like Rocky Linux and older Ubuntu versions.
In short, --break-system-packages is only used as a fallback on Ubuntu 24.04+, while other systems continue using the existing safe paths.
We will also need to update the build docs to guide the users to install the needed packages from the software source. If these packages are detected as installed, then will skip the installation during the build.