uv icon indicating copy to clipboard operation
uv copied to clipboard

Namespace package's `__init__.py` is corrupted when multiple libraries in it are installed

Open ods opened this issue 1 year ago • 3 comments

Given:

  • Multiple libraries in the same namespace package (proprietary, developed by different teams)
  • All those libraries have a namespace/__init__.py file. This is not the recommended way to proceed with namespace packages nowadays, but this is software with a long history and it’s not easy to change.
  • The content of namespace/__init__.py differs from library to library. Either it’s something minor like comments, or because some of them have already started a staggered migration.
  • A Dockerfile, where we use uv to install packages (version 0.2.21).
  • CI pipeline, where we build the container and run linters and tests.

Form time to time (very rarely) we get a failed pipeline due to a syntax error in namespace/__init__.py. Investigation showed that it has the content from one library (of larger size) partially overwritten with the content from another library (of smaller size), leading to invalid code.

ods avatar Jul 05 '24 11:07 ods

"Partially overwritten" is strange -- like the content is a mix of two different files...?

charliermarsh avatar Jul 05 '24 20:07 charliermarsh

The content of namespace/__init__.py file in one of the linraries is:

"""Common libraries for ..........."""

__import__("pkg_resources").declare_namespace(__name__)
__version__ = "1.11.0"

the content of it in another library:

# TODO Switch to PEP-420 implicit namespace packages after moving all
# `..........` packages to `pkgutil`.
# About staged migration and interoperability tables:
# https://github.com/pypa/sample-namespace-packages/pull/22/files

from pkgutil import extend_path

__path__ = extend_path(__path__, __name__)

and the resulting content I see in the container after install:

"""Common libraries for ..........."""

__import__("pkg_resources").declare_namespace(__name__)
__version__ = "1.11.0"
ged migration and interoperability tables:
# https://github.com/pypa/sample-namespace-packages/pull/22/files

from pkgutil import extend_path

__path__ = extend_path(__path__, __name__)

"Partially overwritten" is my interpretation of what's going on.

ods avatar Jul 06 '24 11:07 ods

That looks bad indeed! On what platform are you on, are you setting --link-mode and could you check the verbose log (-v) for messages with attempting to copy files as a fallback in it?

konstin avatar Jul 06 '24 20:07 konstin

On what platform are you on

In docker, image is based on official python:3.12.4-slim, linux/x86_64

are you setting --link-mode

No. The actually used command is:

RUN --mount=from=uv,source=/uv,target=/bin/uv \
    --mount=type=cache,target=${HOME}/.cache,uid=${UID},gid=${GID} \
    uv pip install --system --prefix ${HOME}/.local -r requirements.txt --require-hashes --no-deps

could you check the verbose log (-v) for messages with attempting to copy files as a fallback in it?

Not sure how to achieve this. We switched back to pip after discovering the problem, so uv is not used in pipeline anymore. I'll try to find a way to reproduce the problem locally.

ods avatar Jul 07 '24 16:07 ods