rye icon indicating copy to clipboard operation
rye copied to clipboard

PROJECT_ROOT conveted to absolute path in lockfile

Open sksat opened this issue 2 years ago • 6 comments

Steps to Reproduce

$ cd /tmp
$ rye init hoge --build-system pdm
$ rye init fuga
$ rye add --path ../fuga fuga
Added fuga @ file:///${PROJECT_ROOT}/../fuga as regular dependency
$ rye lock
$ cat requirements.lock | grep fuga

Expected Result

../fuga

Actual Result

fuga @ file:///tmp/hoge/../fuga

Version Info

rye 0.13.0
commit: 0.13.0 (8e011231e 2023-08-29)
platform: linux (x86_64)
self-python: [email protected]
symlink support: true

Stacktrace

No response

sksat avatar Sep 21 '23 19:09 sksat

I think the behavior of current is same as pdm

CharlesChen0823 avatar Sep 22 '23 05:09 CharlesChen0823

The solution is to set up a workspace, then all local dependencies will be added to requirements.lock as simply -e file:package_name.

But then you still can't install the path package (fuga in your case) in the dependent package (hoge), as rye/pip-tools will hit a conflict resolving the plain file:package_name and the file:///tmp... that you have.

What I've done is installed the path package as an optional dependency rye add --optional local --path ../fuga fuga, which does nothing to requirements.lock, and then using something like this in my Dockerfiles:

COPY hoge/pyproject.toml requirements.lock ./
RUN sed -i '/^-e file:/d' requirements.lock
RUN pip install '.[local]' --constraint requirements.lock

carderne avatar Oct 10 '23 21:10 carderne

Seriously, this. Otherwise, it can't be shared.

np-kyokyo avatar Jan 19 '24 13:01 np-kyokyo

Unfortunately the choices here are limited because the lock file "standard" is relatively restricted. Rye already prefers relative paths in lock files where possible.

mitsuhiko avatar Jan 20 '24 11:01 mitsuhiko

Thank. I wanted to install the openapi_client generated by openapi positioned in a subdirectory as a package. This issue seem to be resolved by properly setting the packageName option in the openapi generator (e.g., --additional-properties=packageName=src.openapi.generated.openapi_client). I will write more if there is anything else.

np-kyokyo avatar Jan 20 '24 11:01 np-kyokyo

I got some results with uv.

In requirements.lock file, wrote local path dependency like xxx @ file:///abc/def is related to your build backend.

here is used pyproject.toml:

[project]
name = "temp"
version = "0.1.0"
description = "Add your description here"
authors = [
    { name = "Hiroyuki Tanaka", email = "[email protected]" }
]
dependencies = [
    "core @ file:///${PROJECT_ROOT}/../core",
]
readme = "README.md"
requires-python = ">= 3.8"
classifiers = ["Private :: Do Not Upload"]

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

# [build-system]
# requires = ["pdm-backend"]
# build-backend = "pdm.backend"

# [build-system]
# requires = ["hatchling"]
# build-backend = "hatchling.build"

[tool.rye]
managed = true
dev-dependencies = []

[tool.hatch.metadata]
allow-direct-references = true

[tool.hatch.build.targets.wheel]
packages = ["src/temp"]

I checked the generated lock file of three backends and got a result:

hatch: failed to build: "ValueError: Unknown context field PROJECT_ROOT" pdm: absolute path in lockfile setuptools: "file:///${PROJECT_ROOT}/../core" in lockfile

here is a lock file generated by using setuptools backend:

# generated by rye
# use `rye lock` or `rye sync` to update this lockfile
#
# last locked with the following flags:
#   pre: false
#   features: []
#   all-features: false
#   with-sources: false

-e file:.
core @ file:///${PROJECT_ROOT}/../core
    # via temp
wheel==0.42.0
    # via core

My guess

uv seems to use local path info in wheels Requires-Dist section of .dist-info/METADATA file. if I built wheel using pdm, Requires-Dict section is absolute path, and wheel using setuptools is "core@ file:///${PROJECT_ROOT}/../core".

Possible solution

If uv could convert the path to the relative, like "file:///${PROJECT_ROOT}" in its post-process if dependencies in pyproject.toml has "file:///${PROJECT_ROOT}" like path.

roy-ht avatar Feb 27 '24 10:02 roy-ht