Namespace package's `__init__.py` is corrupted when multiple libraries in it are installed
Given:
- Multiple libraries in the same namespace package (proprietary, developed by different teams)
- All those libraries have a
namespace/__init__.pyfile. This is not the recommended way to proceed with namespace packages nowadays, but this is software with a long history and it’s not easy to change. - The content of
namespace/__init__.pydiffers from library to library. Either it’s something minor like comments, or because some of them have already started a staggered migration. - A
Dockerfile, where we useuvto install packages (version 0.2.21). - CI pipeline, where we build the container and run linters and tests.
Form time to time (very rarely) we get a failed pipeline due to a syntax error in namespace/__init__.py. Investigation showed that it has the content from one library (of larger size) partially overwritten with the content from another library (of smaller size), leading to invalid code.
"Partially overwritten" is strange -- like the content is a mix of two different files...?
The content of namespace/__init__.py file in one of the linraries is:
"""Common libraries for ..........."""
__import__("pkg_resources").declare_namespace(__name__)
__version__ = "1.11.0"
the content of it in another library:
# TODO Switch to PEP-420 implicit namespace packages after moving all
# `..........` packages to `pkgutil`.
# About staged migration and interoperability tables:
# https://github.com/pypa/sample-namespace-packages/pull/22/files
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
and the resulting content I see in the container after install:
"""Common libraries for ..........."""
__import__("pkg_resources").declare_namespace(__name__)
__version__ = "1.11.0"
ged migration and interoperability tables:
# https://github.com/pypa/sample-namespace-packages/pull/22/files
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
"Partially overwritten" is my interpretation of what's going on.
That looks bad indeed! On what platform are you on, are you setting --link-mode and could you check the verbose log (-v) for messages with attempting to copy files as a fallback in it?
On what platform are you on
In docker, image is based on official python:3.12.4-slim, linux/x86_64
are you setting --link-mode
No. The actually used command is:
RUN --mount=from=uv,source=/uv,target=/bin/uv \
--mount=type=cache,target=${HOME}/.cache,uid=${UID},gid=${GID} \
uv pip install --system --prefix ${HOME}/.local -r requirements.txt --require-hashes --no-deps
could you check the verbose log (-v) for messages with attempting to copy files as a fallback in it?
Not sure how to achieve this. We switched back to pip after discovering the problem, so uv is not used in pipeline anymore. I'll try to find a way to reproduce the problem locally.