Search more files for `__version__`?

[woodruffw]

First of all, thanks a ton for flit! It's very nice to be able to put everything in a pyproject.toml and remove setup.py/setup.cfg from my projects 🙂

This is a follow-up from some behavior that a change in pip unearthed (https://github.com/pypa/pip/issues/11110).

When configured with dynamic = ["version"], flit currently looks for a __version__ in the top level module or __init__.py of the package being built. It does this by building the AST for the file, then searching for a __version__ assignment whose RHS is a string literal: https://github.com/pypa/flit/blob/ec6e6b7c8994d5e79d0fd951d9e4c26a7c5d9977/flit_core/flit_core/common.py#L135-L141

This works really nicely when projects embed their __version__ in their top-level module, but another common pattern is this:

# ./foo/__init__.py

from .version import __version__

__all__ = ["__version__"]

# ./foo/version.py

__version__ = "1.2.3"

So, my question: would it be possible to expand flit's __version__ search to include version.py and _version.py (the standard pattern for an implementation-internal module), when present? Here are some search procedures I can envision (not necessarily exhaustive!):

1. Search __init__.py, falling back on version.py and then _version.py if they exist
2. Search __init__.py, checking __all__ if __version__ is not present and walking upwards to the originating module (if unambiguous)

The second is more generic, but IMO also overkill (since the standard pattern is to use either version.py or _version.py and nothing else.)

Thoughts?

[pradyunsg]

Hiya! Thanks for filing this issue.

Do we know of a project that can not store the version in __init__.py, but can store it in a different Python file for some reason? If so, could you elaborate on why that's the case?

While I don't think this is technically difficult, I'm not quite sure what value this provides and why someone might want to do this.

[woodruffw]

I will admit that I cannot think of a good technical reason for storing __version__ in a separate file -- IME it mostly happens on projects where two mis-patterns interact:

• The top-level __init__.py contains nontrivial imports, which in turn require either dependencies or have side effects
• The __version__ was previously extracted through some non-ideal technique, such as exec in setup.py

It's also a recommended place to put the __version__ in the Single-sourcing the package version guide in the PyPA packaging user guide, although a lot of the advice in that guide is stale at this point (like the exec technique). Less authoritatively, it's a popular answer on SO.

So the boring but not technically good reason: there's a bunch of projects out there that are probably using {_,}version.py instead of __init__.py, and support would save them a small amount of churn.

[pquentin]

For what it's worth, urllib3 hit the exact same pip bug for the same historical reason, but we're happy to remove _version.py now that we use flit.

Well, actually what's nice with _version.py is that we can protect with CODEOWNERS, but maybe we don't need to do that. I'll have to discuss this with @sethmlarson.

[edmorley]

So my use-case for needing __version__ in a file other than __init__.py is that I use the version within the library itself in order to set a User-Agent header. If I try to from . import __version__ at the top of the file that handles the client, I get:

ImportError: cannot import name '__version__' from partially initialized module 'foo' (most likely due to a circular import)

As such, I have to either: (a) lazily import __version__ within the method that uses it, instead of importing at the top of the module. (b) move __version__ to another file, then from X import __version__ inside my __init__.py so flit can find it

If there was another known location Flit checked (eg __version__.py), I could do (b) but skip having to add the import to __init__.py.

[pquentin]

Hatch supports this by requiring projects to specify where the version file is and allowing customizing the regex that searches for the version. See https://hatch.pypa.io/latest/version/

[takluyver]

@amyreese did this in #630, for files called version.py, _version.py or __version__.py. :+1:

[merwok]

I’d like to note that Python reserves all names with double underscores on both sides. At present, __init__ and __main__ are special module names. It would be best of projects did not invent more names with non-official meanings.

_version is great though!