poetry-plugin-export icon indicating copy to clipboard operation
poetry-plugin-export copied to clipboard

Extras present in exported constraints.txt file

Open adriangonz opened this issue 2 years ago • 3 comments

After generating the constraints.txt file, I noticed that it still exported a few packages with extra dependencies. This eventually makes pip install ... --constraint constraints.txt fail with:

DEPRECATION: Constraints are only allowed to take the form of a package name and a version specifier. Other forms were originally permitted as an accident of the implementation, but were undocumented. The new implementation of the resolver no longer supports these forms. A possible replacement is replacing the constraint with a requirement. Discussion can be found at https://github.com/pypa/pip/issues/8210
ERROR: Constraints cannot have extras

This seems to be handled by #128 so it's unclear why it would still happen (perhaps a regression?). Just in case it's useful, the only packages exported with extras were VCS ones (installed from Git). Happy to provide more details if needed.

Poetry version: 1.4.2

adriangonz avatar Jun 06 '23 10:06 adriangonz

if you hope for anyone ever to look at this, provide a way to reproduce it!

dimbleby avatar Jun 06 '23 15:06 dimbleby

Good point @dimbleby !

Here's a pyproject.toml that reproduces the issue:

[tool.poetry]
name = "foo"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"
optimum = {extras = ["onnxruntime"], version = "^1.8.6"}

[tool.poetry.group.dev.dependencies]
transformers = {extras = ["sentencepiece"], git = "https://github.com/huggingface/transformers.git", branch = "main", rev = "0c65fb7cfa1258fb5946c5ae4d13f5a2a88a2f56"}


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

With that on your root folder, you can then follow these steps to reproduce the issue:

  1. Generate lock file with poetry lock

  2. Generate constraints file with poetry export --format constraints.txt -o constraints.txt

  3. On the generated constraints.txt, search for extras (i.e. search for [ or ]) which should show you something like:

    transformers[sentencepiece] @ git+https://github.com/huggingface/transformers.git@0c65fb7cfa1258fb5946c5ae4d13f5a2a88a2f56 ; python_version >= "3.10" and python_version < "4.0"
    
  4. Try to install anything with that constraints.txt file (e.g. pip install scikit-learn --constraint constraints.txt), which will return an error like the one below:

    DEPRECATION: Constraints are only allowed to take the form of a package name and a version specifier. Other forms were originally permitted as an accident of the implementation, but were undocumented. The new implementation of the resolver no longer supports these forms. A possible replacement is replacing the constraint with a requirement. Discussion can be found at https://github.com/pypa/pip/issues/8210
    ERROR: Constraints cannot have extras
    

adriangonz avatar Jun 06 '23 15:06 adriangonz

fix looks like it should be something along these lines

diff --git a/src/poetry_plugin_export/exporter.py b/src/poetry_plugin_export/exporter.py
index 4ec7cc4..bf307e7 100644
--- a/src/poetry_plugin_export/exporter.py
+++ b/src/poetry_plugin_export/exporter.py
@@ -101,11 +101,11 @@ class Exporter:
         ):
             line = ""

-            if not with_extras:
-                dependency_package = dependency_package.without_features()
-
             dependency = dependency_package.dependency
             package = dependency_package.package
+            if not with_extras:
+                dependency = dependency.without_features()
+                package = package.without_features()

             if package.develop:
                 if not allow_editable:

but I expect you'll need to write a testcase to get that merged, and that's likely to be more work

dimbleby avatar Jun 06 '23 17:06 dimbleby