mtkennerly
pyproject.toml becomes broken after `poetry install`
https://github.com/masenf/dynamic-versioning-bug
- Clone repo and enter directory
- run
poetry install3 times - pyproject.toml is broken now:
Either [project.version] or [tool.poetry.version] is required in package mode.
masenf@minicone scratch % git clone [email protected]:masenf/dynamic-versioning-bug.git
Cloning into 'dynamic-versioning-bug'...
Enter passphrase for key '/Users/masenf/.ssh/id_rsa_github':
remote: Enumerating objects: 52, done.
remote: Counting objects: 100% (52/52), done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 52 (delta 24), reused 52 (delta 24), pack-reused 0 (from 0)
Receiving objects: 100% (52/52), 20.13 KiB | 6.71 MiB/s, done.
Resolving deltas: 100% (24/24), done.
masenf@minicone scratch % cd dynamic-versioning-bug
masenf@minicone dynamic-versioning-bug % poetry install
Ensuring that the Poetry plugins required by the project are available...
The following Poetry plugins are required by the project but are not installed in Poetry's environment:
- poetry-dynamic-versioning[plugin] (>=1.0.0,<2.0.0)
Installing Poetry plugins only for the current project...
Updating dependencies
Resolving dependencies... (0.7s)
Package operations: 4 installs, 0 updates, 0 removals
- Installing markupsafe (3.0.2)
- Installing dunamai (1.23.0): Installing...
- Installing dunamai (1.23.0)
- Installing jinja2 (3.1.6)
- Installing poetry-dynamic-versioning (1.7.1): Installing...
- Installing poetry-dynamic-versioning (1.7.1)
Writing lock file
Installing dependencies from lock file
Installing the current project: dynamic-versioning-bug (0.0.0)
masenf@minicone dynamic-versioning-bug % poetry install
Installing dependencies from lock file
Installing the current project: dynamic-versioning-bug (0.1.11)
masenf@minicone dynamic-versioning-bug % git diff
diff --git a/pyproject.toml b/pyproject.toml
index a09ee80..37e8f18 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -7,11 +7,11 @@ name = "dynamic-versioning-bug"
dynamic = ["version"]
[tool.poetry]
-version = "0.0.0"
[tool.poetry.requires-plugins]
poetry-dynamic-versioning = { version = ">=1.0.0,<2.0.0", extras = ["plugin"] }
+version = "0.0.0"
[tool.poetry-dynamic-versioning]
enable = true
masenf@minicone dynamic-versioning-bug % poetry install
The Poetry configuration is invalid:
- Either [project.version] or [tool.poetry.version] is required in package mode.
I went through a few iterations of tracking this down, and if i re-order some sections in my pyproject.toml, then i can seem to workaround it, but it would be awesome to get this fixed in the plugin itself.
Specifically, moving the [project.scripts] section above all of the [tool.] sections seems to result in no diff/breakage to pyproject.toml when installing or building.
Hi! Thanks for the detailed report. However, oddly enough, I can't seem to reproduce this - pyproject.toml stays the same after each command. Could you give me some info about your environment? This is mine:
- Windows 11
- Python 3.10.7
- Poetry 2.1.1
- No plugins installed other than
requires-plugins - Poetry installed via Pipx
- No plugins installed other than
- poetry-dynamic-versioning 1.7.1
- tomlkit 0.13.2
I was able to reproduce this issue - the key difference appears to be the version of tomlkit.
Environment:
- Windows 11
- Python 3.11.6
- Poetry 2.1.3
- Installed via Pipx 1.6.0
- Plugins: specified via
requires-plugins, but I found I had to install them withpipx inject poetry <plugin-name>rather than duringpoetry install(possibly an unrelated bug, not with poetry-dynamic-versioning - this was the first time I'd used these plugins since migrating to poetry 2)
- poetry-plugin-export 1.9.0
- poetry-dynamic-versioning 1.8.2
- tomlkit 0.12.5
When I ran any of poetry install, poetry check, poetry lock, poetry build I observed that it left changes in the pyproject.toml file. The change was either
- an additional newline case1_newline_pyproject_toml.txt; or
- re-ordering (even into different section!) of the
version = "0.0.0"line case2_reorder_pyproject_toml.txt
depending on the position of the [tool.poetry] section. In the re-ordering case this indeed broke the pyproject.toml.
Attached are stripped-back pyproject.toml files for each case. The following diffs compare (a) original pyproject.toml to (b) the pyproject.toml file after running poetry check, for case 1
poetry-test> git diff HEAD~1 HEAD .\pyproject.toml
diff --git a/poetry-test/pyproject.toml b/poetry-test/pyproject.toml
index 3f9e9de4a..909fb761b 100644
--- a/poetry-test/pyproject.toml
+++ b/poetry-test/pyproject.toml
@@ -8,8 +8,8 @@ dynamic = ["version"]
dependencies = []
[tool.poetry]
-version = "0.0.0"
+version = "0.0.0"
[build-system]
requires = [
"poetry-core>=2.0.0",
and for case 2:
poetry-test> git diff HEAD~1 HEAD .\pyproject.toml
diff --git a/poetry-test/pyproject.toml b/poetry-test/pyproject.toml
index 480f01563..860ded9d0 100644
--- a/poetry-test/pyproject.toml
+++ b/poetry-test/pyproject.toml
@@ -23,12 +23,12 @@ science = [
]
[tool.poetry]
-version = "0.0.0"
[tool.poetry.group.dev.dependencies]
pylint = ">=3.1.0,<4.0.0"
pytest = ">=8.2.0,<9.0.0"
+version = "0.0.0"
[tool.poetry-dynamic-versioning]
enable = true
Manually upgrading tomlkit to 0.13.2 using pipx runpip poetry install tomlkit --upgrade stopped the issue - there were no longer modifications to pyproject.toml after poetry commands.
However, poetry self update insists on rolling the tomlkit version back to 0.12.5.
Thanks for the info!
poetry-dynamic-versioning only requires tomlkit = ">= 0.4", so I'm not sure where the forced downgrade is coming from. Does it help if you regenerate Poetry's internal lockfile (~/AppData/Roaming/pypoetry/poetry.lock on Windows) using poetry self lock?
When I install a fresh copy of Poetry, it picks tomlkit 0.13.2:
$ pipx install poetry --suffix "-2.1"
$ pipx runpip poetry-2.1 list | grep tomlkit
tomlkit 0.13.2
$ pipx runpip poetry-2.1 install tomlkit==0.13.1
$ poetry-2.1 self update
- Updating tomlkit (0.13.1 -> 0.13.2)
Ultimately, if a fresh install of Poetry 2.1.3 is picking tomlkit 0.12.5, then I think you should open a ticket in the Poetry project to see if they can figure out why it's using an older version.
Here's an example of the issue in Ubuntu 24.04 GitHub runner: https://github.com/intercreate/smpclient/actions/runs/15367816937/job/43242970293
A workaround commit: https://github.com/intercreate/smpclient/commit/dcace1aae61910ff25d8a674fcb8351a538f95b3
And successful flow: https://github.com/intercreate/smpclient/actions/runs/15367969414/job/43243318229
@JPHutchins That's a separate issue with your custom format:
format-jinja = "{% if distance == 0 %}{{ base }}{% else %}{{ base }}-dev{{ distance }}+g{{ commit }}{% endif %}{% if dirty %}.dirty{% endif %}"
If the distance is 0 and the workspace is dirty, then you'd get {{ base }}.dirty. You probably want two different dirty checks, so that you can use +dirty instead of .dirty when the distance is 0, or just ignore dirty in that case.
@JPHutchins That's a separate issue with your custom format:
format-jinja = "{% if distance == 0 %}{{ base }}{% else %}{{ base }}-dev{{ distance }}+g{{ commit }}{% endif %}{% if dirty %}.dirty{% endif %}"If the distance is 0 and the workspace is dirty, then you'd get
{{ base }}.dirty. You probably want two different dirty checks, so that you can use+dirtyinstead of.dirtywhen the distance is 0, or just ignore dirty in that case.
I will investigate! However, the main problem is that the workspace is dirty in the first place. Dirty builds in CI are mostly useless because the build is not a reproducible version. For example, users cannot distinguish between a dirty build caused by an innocuous stale pyproject.toml, their own mistake of leaving out commits (only relevant for local builds), and a dirty build caused by a third party workflow injecting malware into the build.