Establish single source of truth for Python package versions

[kantord]

Is your feature request related to a problem? Please describe. We have several Python packages:

• librelingo_audios
• librelingo_fakes
• librelingo_json_export
• librelingo_types
• librelingo_utils
• librelingo_yaml_loader

packaging is managed by poetry and the version is configured in meta data files, for example:

https://github.com/kantord/LibreLingo/blob/main/apps/librelingo_fakes/pyproject.toml#L3

the problem is, that versions might be (are) used elsewhere. Right not, only in init.py files.

for example:

https://github.com/kantord/LibreLingo/blob/main/apps/librelingo_json_export/librelingo_json_export/init.py#L7

The problem with this is that we need to be very careful to make sure these versions are always the same. Instead of double checking it every time, we should only need to update it in one place.

Describe the solution you'd like

• The version number of each package is only explicitly specified in one place

Describe alternatives you've considered We could instead have a script that updates this, and/or verify in CI that the version numbers are in sync. I don't like that solution though, because it sounds like it would add unnecessary complication and it still requires manual work to update the version everywhere.

Additional context I found a related thread no GitHub: https://github.com/python-poetry/poetry/issues/273

[kantord]

Marked as good first issue, as it doesn't require almost any initial understanding of the project structure, nor the codebase

[szabgab]

IMHO the individual packages should have NO poetry.lock files as they are packages. There should be only one such file in the root of all the project.

[kantord]

IMHO the individual packages should have NO poetry.lock files as they are packages. There should be only one such file in the root of all the project.

I think this does not depend on us. As far as I know, poetry has no way of operating without the poetry.lock files. If poetry had built in monorepo features then I guess it could be used to get rid of these lockfiles and have only one in the project root.

Perhaps the only thing we can do is gitignore these lockfiles. I also don't think it's possible to publish those packages without the poetry.lock though. It's because poetry will create a virtualenv for the package in order to publish it.

btw why do you say that individual packages should have NO poetry.lock? I think by default all packages managed by poetry will have this file, that's just how it works. As long as poetry cannot share virtualenvs between each package I think it kind makes sense to have these files.

[szabgab]

This is the first time I use poetry so I am not sure.

IMHO In general in libraries we probably want to be flexible with the versions of your dependencies so others who use these libraries won't be stuck on specific versions of the dependencies as well. In the applications we will want to be specific, use the lock file and control the exact time when we upgrade any of the dependencies.

Here https://python-poetry.org/docs/libraries/ it says you don't have to keep the lock file.

I wonder if it wouldn't be better if the application (the main pyproject.toml file) would specify the names of the packages only without the link to the source code and let it install the source files from pypi. Though I am not sure how could we then instruct poetry to prefer to use the source code that is in the repo and not what it installed.

Alternatively we could add all the dependencies of the python packages to the main pyproject.toml as well.

[kantord]

From the poetry docs you linked:

For your library, you may commit the poetry.lock file if you want to. This can help your team to always test against the same dependency versions. However, this lock file will not have any effect on other projects that depend on it. It only has an effect on the main project.

So effectively the lock file already has no effect on the apps that depend on it, however it makes the testing environment more predictable

In the applications we will want to be specific

To some degree yes, but maybe the same case can be made as someone might use them in the same virtualenv that they share with other things. Or use them globally installed. In which case being overly particular with version can still be a nuisance. So maybe I wouldn't immediately discard this either.

I wonder if it wouldn't be better if the application (the main pyproject.toml file)

The main pyproject.toml file is not the application. That is merely for development purposes. Actually I'm not fully sure what you mean by the application. Do you mean the web app? It's not managed by poetry at all because it has no Python code at all

I think this is a fairly typical way to set up a monorepo, basically you have apps and libraries that can be released independently, however there's also another layer grouping everything together that allows for managing the monorepo more easily. For example, we only need to set up linting and testing tools once for each language etc.

But probably this should be documented better as I think the documentation does not mention this at all right now.

Though I am not sure how could we then instruct poetry to prefer to use the source code that is in the repo and not what it installed.

As far as I know right now it's not possible, although I would have also preferred it to work in a similar way as you say. At least in that the linking in development mode could be handled internally by poetry instead of having to specify it manually.

Nevertheless, in our current use case this wouldn't change much, as the "root" project is just meant for development and is not meant to be published at all.