poetry-plugin-export Duplicate dependency export

[x] I am on the latest Poetry version.
[x] I have searched the issues of this repo and believe that this is not a duplicate.
[x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

OS version and name: elementary OS 5.0 Juno
Poetry version:1.0.3
Link of a Gist with the contents of your pyproject.toml file:

pyproject.toml

[tool.poetry]
name = "tmpoetry"
version = "0.1.0"
description = "sdfg"
authors = ["Author <[email protected]>"]

[tool.poetry.dependencies]
python = "^3.6"
mypy = "^0.701"
pylint = "^2.3"

Issue

When a (sub)dependency has different requirements for different python versions, it appears twice in poetry.lock. In itself this is no problem, but poetry export generates duplicate entries.
In this use case, the command poetry add fails because said library has "conflifting" versions. (try poetry add astroid)

Feb 03 '20 15:02 pwoolvett

Same issue here.

[tool.poetry]
name = "hello"
version = "2020.02.27"
description = "zzzz"
authors = ["aarrr <[email protected]>"]

[tool.poetry.dependencies]
python = "~2.7 || ~3.7"

pytest = "*"
firebase-admin = "*"

Feb 27 '20 07:02 squirrel532

Anyone has a workaround for this?

I'm currently forced to manually specify the different versions in the pyproject.toml file.

@azdkj532 I have the same issue with pytest specifically, which results in dual zipp packages in the requirements file.

Currently my solution is adding this:

zipp = "^1"

poetry export output without the fix:

atomicwrites==1.3.0
attrs==19.3.0
colorama==0.4.3; sys_platform == "win32" and python_version != "3.4"
configparser==4.0.2; python_version < "3"
contextlib2==0.6.0.post1; python_version < "3.4"
funcsigs==1.0.2; python_version < "3.0"
importlib-metadata==1.6.0; python_version < "3.8"
more-itertools==5.0.0; python_version <= "2.7"
more-itertools==8.2.0; python_version > "2.7"
packaging==20.3
pathlib2==2.3.5; python_version < "3.6"
pluggy==0.13.1
py==1.8.1
pyparsing==2.4.6
pytest==4.6.9
scandir==1.10.0; python_version < "3.5"
six==1.14.0
wcwidth==0.1.9
zipp==1.2.0; python_version < "3.8"
zipp==3.1.0; python_version < "3.8"

poetry export output with the fix:

atomicwrites==1.3.0
attrs==19.3.0
colorama==0.4.3; sys_platform == "win32" and python_version != "3.4"
configparser==4.0.2; python_version < "3"
contextlib2==0.6.0.post1; python_version < "3.4"
funcsigs==1.0.2; python_version < "3.0"
importlib-metadata==1.6.0; python_version < "3.8"
more-itertools==5.0.0; python_version <= "2.7"
more-itertools==8.2.0; python_version > "2.7"
packaging==20.3
pathlib2==2.3.5; python_version < "3.6"
pluggy==0.13.1
py==1.8.1
pyparsing==2.4.6
pytest==4.6.9
scandir==1.10.0; python_version < "3.5"
six==1.14.0
wcwidth==0.1.9
zipp==1.2.0

Mar 31 '20 12:03 mcouthon

Poetry 1.0.5 fails to export its own dependencies because of this.

Reproduce:

$ git clone --branch 1.0.5 https://github.com/python-poetry/poetry
$ cd poetry
$ poetry export -f requirements.txt --without-hashes | grep idna
idna==2.8
idna==2.9

May 15 '20 03:05 DavHau

It's not just poetry export - poetry show has the same problem.

Jul 03 '20 10:07 maxgrenderjones

Today, locking and exporting the given pyproject.toml does not give any duplicates in the output.

So I suppose this is fixed:

astroid==2.3.0 ; python_version >= "3.6" and python_version < "4.0"
colorama==0.4.4 ; python_version >= "3.6" and python_version < "4.0" and sys_platform == "win32"
isort==4.3.21 ; python_version >= "3.6" and python_version < "4.0"
lazy-object-proxy==1.7.1 ; python_version >= "3.6" and python_version < "4.0"
mccabe==0.6.1 ; python_version >= "3.6" and python_version < "4.0"
mypy-extensions==0.4.3 ; python_version >= "3.6" and python_version < "4.0"
mypy==0.701 ; python_version >= "3.6" and python_version < "4.0"
pylint==2.4.4 ; python_version >= "3.6" and python_version < "4.0"
six==1.16.0 ; python_version >= "3.6" and python_version < "4.0"
typed-ast==1.3.5 ; python_version >= "3.6" and python_version < "4.0" or implementation_name == "cpython" and python_version >= "3.7" and python_version < "3.8"
wrapt==1.14.1 ; python_version >= "3.6" and python_version < "4.0"

May 28 '22 11:05 dimbleby

I don't think it's fixed: https://stackoverflow.com/q/69810820/562769

Dec 05 '22 06:12 martin-thoma

That stack overflow question has nothing to do with export, and this still is fixed

Dec 05 '22 08:12 dimbleby

I am also observing this behaviour. I am using poetry version 1.8.3. Prefect is installed on my system via pipx. I am working on WSL with Ubuntu 22.04.

See my pyproject.toml file below:

[build-system]
requires = [ "poetry-core",]
build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "ghb_process_flow"
version = "2.9.14"
description = ""
authors = ["some author <some_emaiil@some_domain.com>"]
readme = "README.md"
packages = [{include = "ifrs9_atlas_implementation"}]

[tool.poetry.dependencies]
python = "^3.11"
polars = "1.0.0"
sas7bdat-converter = {extras = ["parquet"], version = "^2.0.0"}
python-dotenv = "^1.0.0"
tqdm = "^4.66.1"
pyodbc = "^4.0.39"
sqlparse = "^0.4.4"
numpy-financial = "^1.0.0"
numpy = "1.26.4"
tabulate = "^0.9.0"
wonderwords = "^2.2.0"
prefect = "2.20.2"
slack-sdk = "^3.27.1"
psycopg2-binary = "^2.9.9"
azure-identity = "^1.16.0"

[tool.poetry.dev-dependencies]
pytest = "^7.4.0"
pytest-cov = "^4.1.0"
pytest-html = "^4.0.2"
sphinx = "^7.2.6"
sphinx-rtd-theme = "^1.3.0"
sphinx-code-include = "^1.1.1"
interrogate = "^1.5.0"

[tool.ruff]
select = [
    "D",
]

[tool.ruff.pydocstyle]
convention = "numpy"

[tool.poetry.scripts]
purgedb = "ifrs9_atlas_implementation.data.ad_hoc_db_management:purge_database_contents"
dbtocsv = "ifrs9_atlas_implementation.data.ad_hoc_db_management:write_database_contents_to_csv"
quickflow = "ifrs9_atlas_implementation.flow_management.cli:run_ecl_flow_cli"

[tool.pytest.ini_options]
markers = [
    "dependency",
]

I am using the following command on by bash shell to create a requirements.txt file without the hashes and python version number.

poetry export --without-hashes --format=requirements.txt > requirements.txt
sed -i 's/ *;.*//' requirements.txt

It produces the below requirements.txt file where pendulum is stated twice with 2 different versions.

aiosqlite==0.20.0
alembic==1.13.3
annotated-types==0.7.0
anyio==4.6.0
apprise==1.9.0
asgi-lifespan==2.1.0
async-timeout==4.0.3
asyncpg==0.29.0
attrs==24.2.0
azure-core==1.31.0
azure-identity==1.18.0
cachetools==5.5.0
certifi==2024.8.30
cffi==1.17.1
charset-normalizer==3.3.2
click==8.1.7
cloudpickle==3.0.0
colorama==0.4.6
coolname==2.2.0
croniter==2.0.7
cryptography==43.0.1
dateparser==1.2.0
dnspython==2.6.1
docker==7.1.0
email-validator==2.2.0
exceptiongroup==1.2.2
fsspec==2024.9.0
google-auth==2.35.0
graphviz==0.20.3
greenlet==3.1.1
griffe==0.47.0
h11==0.14.0
h2==4.1.0
hpack==4.0.0
httpcore==1.0.5
httpx[http2]==0.27.2
humanize==4.10.0
hyperframe==6.0.1
idna==3.10
importlib-resources==6.1.3
itsdangerous==2.2.0
jinja2-humanize-extension==0.4.0
jinja2==3.1.4
jsonpatch==1.33
jsonpointer==3.0.0
jsonschema-specifications==2023.12.1
jsonschema==4.23.0
kubernetes==29.0.0
mako==1.3.5
markdown-it-py==3.0.0
markdown==3.7
markupsafe==2.1.5
mdurl==0.1.2
msal-extensions==1.2.0
msal==1.31.0
numpy-financial==1.0.0
numpy==1.26.4
oauthlib==3.2.2
orjson==3.10.7
packaging==24.1
pandas==2.2.3
pathspec==0.12.1
**pendulum==2.1.2
pendulum==3.0.0**
polars==1.0.0
portalocker==2.10.1
prefect==2.20.2
psycopg2-binary==2.9.9
pyarrow==12.0.1
pyasn1-modules==0.4.1
pyasn1==0.6.1
pycparser==2.22
pydantic-core==2.23.4
pydantic[email]==2.9.2
pygments==2.18.0
pyjwt[crypto]==2.9.0
pyodbc==4.0.39
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
python-multipart==0.0.10
python-slugify==8.0.4
pytz==2024.2
pytzdata==2020.1
pywin32==306
pyyaml==6.0.2
readchar==4.2.0
referencing==0.35.1
regex==2024.9.11
requests-oauthlib==2.0.0
requests==2.32.3
rfc3339-validator==0.1.4
rich==13.8.1
rpds-py==0.20.0
rsa==4.9
ruamel-yaml-clib==0.2.8
ruamel-yaml==0.18.6
sas7bdat-converter[parquet]==2.0.1
shellingham==1.5.4
six==1.16.0
slack-sdk==3.33.1
sniffio==1.3.1
sqlalchemy==2.0.35
sqlalchemy[asyncio]==2.0.35
sqlparse==0.4.4
tabulate==0.9.0
text-unidecode==1.3
toml==0.10.2
tqdm==4.66.5
typer==0.12.5
typing-extensions==4.12.2
tzdata==2024.2
tzlocal==5.2
ujson==5.10.0
urllib3==2.2.3
uvicorn==0.30.6
websocket-client==1.8.0
websockets==12.0
wonderwords==2.2.0

Sep 24 '24 10:09 specialkapa

I ... create a requirements.txt file without the hashes and python version number.

of course if you mangle the requirements.txt that poetry creates you risk getting nonsense

the actual poetry output for pendulum in your case is

pendulum==2.1.2 ; python_version >= "3.11" and python_version < "3.12"
pendulum==3.0.0 ; python_version >= "3.12" and python_version < "4.0"

which is correct

this issue has been fixed at least since https://github.com/python-poetry/poetry-plugin-export/issues/28#issuecomment-1140243677 and should be closed

Sep 24 '24 11:09 dimbleby

My bad! Thanks for getting back to me David.

On Tue, 24 Sep 2024 at 12:33, David Hotham @.***> wrote:

I ... create a requirements.txt file without the hashes and python version number.

of course if you mangle the requirements.txt that poetry creates you risk getting nonsense

the actual poetry output for pendulum in your case is

pendulum==2.1.2 ; python_version >= "3.11" and python_version < "3.12" pendulum==3.0.0 ; python_version >= "3.12" and python_version < "4.0"

which is correct

this issue has been fixed at least since #28 (comment) https://github.com/python-poetry/poetry-plugin-export/issues/28#issuecomment-1140243677 and should be closed

— Reply to this email directly, view it on GitHub https://github.com/python-poetry/poetry-plugin-export/issues/28#issuecomment-2371007822, or unsubscribe https://github.com/notifications/unsubscribe-auth/AW7JCDK3URS7JTXXFDJXH2DZYFE2LAVCNFSM6AAAAABOX53LV6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNZRGAYDOOBSGI . You are receiving this because you commented.Message ID: @.***>

Sep 24 '24 11:09 specialkapa

poetry-plugin-export poetry-plugin-export copied to clipboard

Duplicate dependency export

Issue

poetry-plugin-export
poetry-plugin-export copied to clipboard