packaging.python.org

packaging.python.org copied to clipboard

There are many approaches offered in the current documentation, but no approach seems to be "recommended" or otherwise suggested as the preferred approach to take. For a new developer looking to package their project, this can be confusing.

Looking at a few of the projects I'm familiar with, it seems that approach 3 is common.

Set the value to a __version__ global variable in a dedicated module in your project (e.g. version.py), then have setup.py read and exec the value into a variable.

Would it make sense to present this approach as more than just one among many? Or is there really no preferred way to single-source the project version, even for new projects?

Typically any attempt to recommend a preferred solution is met with debate. There really doesn't seem to be a consensus.

FWIW, if I recall the common objection to the approach you mention here is that it doesn't work properly if your project build directory isn't on sys.path (which it may not be in certain situations).

The objections to the other options seem to me to be no stronger (but no weaker either).

FWIW, if I recall the common objection to the approach you mention here is that it doesn't work properly if your project build directory isn't on sys.path (which it may not be in certain situations).

Is that a common occurrence? I'm not trying to start a debate about this now; I just ask as a total newcomer to packaging. :)

I suppose if there really is no consensus, or even no approach that we can offer as "works for most people", then we can close this issue.

I don't know, it's not my objection...

It think approach #3 is becoming more common (I use it now btw), but it's hard for me to say what is globally "more common", w/o doing some kind of analysis of what's being posted to pypi these days.

let's keep this issue open, and maybe #3 can become preferred at some point.

BTW, my explanation of the objection I've seen is wrong (if you exec the file,you don't need it on sys.path).

One real issue is getting encodings right. The execfile approach is Python 2 only. The exec version needs an open() but Python 2 doesn't support supplying an encoding to open(), and the default encoding depends on the user's PC settings. The best approach is only using ASCII in your version.py and relying on the fact that nobody's going to have a default encoding that isn't ASCII-compatible. But all it takes is a non-ASCII character in a comment and the read gets a decoding error :-(

But it's a corner case, and not likely to affect newcomers (um, unless they don't speak English as their native language, I guess...)

You could always just add myproject/_version which is just a text file, then have:

# myproject/__init__.py
import pkgutil

__version__ = pkgutil.get_data("myproject", "_version").strip()

and

# setup.py
import os.path

with open(os.path.join(os.path.dirname(__file__), "myproject", "_version")) as fp:
    version = fp.read().strip()

setup(
    ...
    version = version,
    ...
)

Also, codecs.open is like open except it supports an encoding parameter and works on 2 and 3.

You could always just add myproject/_version which is just a text file

btw, this is already presented as one of the options... #4

Note: #190 proposes changes to the document that would affect this discussion.

The problem I have with all the solutions currently listed in the document is that if your project uses Git tags to mark releases, then there isn't a single source of truth anymore: there's the version number in the code and the one in the tags.

That's correct, but I think that falls outside of the domain of what the PUG should cover. If you want your single-source version to be in Git, then you're coupling how Python versioning is done to something outside of Python. What about users who use Mercurial or SVN? Should the guide cover those as well?

As far as the PUG is concerned, I think agreeing on the best approach to having a single source for the version across your Python code is what we should shoot for. Once we have that, it's easier to make additional recommendations on how to add an additional layer of indirection to source that version from outside of Python.

So to provide an example, say the guide evolves to converge on option 3. From there, it's much easier to add tips on how to source that version in turn from a VCS.

For example:

Set the value to a __version__ global variable in a dedicated module in your project (e.g. version.py), then have setup.py read and exec the value into a variable. You can set __version__ automatically by querying your VCS. For example, in Git...

Why option 3 and not any other one ? :-) Note also that all these discussions relate to the use of distutils/setuptools as build tool.

Per the discussion earlier in this thread, option 3 seems to be the option that new projects are converging on. Do you observe otherwise?

I think the PUG serves the community best when it recognizes already-existing conventions, or when it helps nudge the community towards emerging conventions and accelerates their adoption. I don't think anyone here is trying to invent something new and impose it on people by diktat.

Personally, which option we go with is less important to me than the fact that we converge on a single "recommended" option. It's better for users when there are fewer choices on display for implementing such a basic pattern.

To expand a bit: The PUG is a guide, not an encyclopedia. Its purpose is to present common advice to users facing common problems, not to document every possible permutation that people have found useful.

At least, that's how I see the PUG. Perhaps I've misunderstood its purpose though.

Per the discussion earlier in this thread, option 3 seems to be the option that new projects are converging on. Do you observe otherwise?

I must admit I usually don't check the version hack used by the library I use but at work we tend to use something similar to pip: https://github.com/pypa/pip/blob/develop/setup.py#L34-L39 so that would be option 1.

@nchammas That sounds pretty much correct.

Since #190 has become tangled with this issue I've turned it into the following proposal:

• simplify option 1 (__version__ in __init__) and use the same coding style in option 3 (__version__ in __about__)
• move option 2 (external tools) to the end
• remove options 4 (complicated variant of option 3) and 5 (unreliable use of pkg_resources)
• remove option 6 (importing __version__ from __init__) and mention why it's not recommended (in the note of option 1)
• put the VCS tag solution in its own item, because it's not a release tool so it doesn't fit into option 2

I've kept both option 1 and option 3 because I can't find a good argument to declare that one is better than the other.

In #190 @pfmoore points out that option 1 is less simple and solid because it uses regexps. That could justify dropping it in favour of option 3. Opinions?

No, I was pointing out that your edits omitted the note that the fact that it used regexps is a factor. I don't think there's anything new in the fact that option 1 uses regexps. So -1 on dropping it "because it uses regexps".

Remember that the original question here was "Is there no preferred approach to single-sourcing the project version?" My answer to that is still "No there isn't, sorry". (But for my personal projects, I don't actually care about single-sourcing the version, having it in a couple of places isn't an issue to me).

Sorry, I didn't mean to misrepresent what you said. Your comment about the missing note made me realize that option 1 is less simple and solid.

No problem

Sorry #190 failed to solve this issue, @nchammas.

Considering the reluctance that was expressed to removing options from the page, maybe splitting the options in two sections (e.g. "recommended" and "other") would be a better approach. I won't be the one making that PR though.

As I understood it, but correct me if I'm wrong, the point of not being opinionated is to prevent the inevitable disagreements and bike-shedding. And that's why PPUG don't want to recommend one way to do it.

That being said, I still believe it should be easier for the user to pick something. Currently you'd be inclined to pick something based on ordering - so there's already some implied recommendation there ;-)

I propose to have a list of pros and cons for each option, and then the user can look over those and decide what's important for him. Cause no one has the same needs.

Belatedly revisiting this, I do agree there's value in offering folks a prescriptive "I don't care about the details, just give me a 'not wrong' approach that covers absolutely everything I need to do".

However, I also think it's problematic for the PyPA specifically to publish such a guide, since it's easy for folks to interpret "this is one good approach" as "all other possible approaches are bad" and try to use it as a lever to coerce maintainers of existing projects into changing the way they do things even when there's nothing wrong with their current approach (cf. folks wanting to retroactively apply modern versions of PEP 8 to code bases that predate those updates by many years).

Accordingly, perhaps we could take the expedient route and offer a link at the start of the Packaging & Distribution guide referencing @hynek's walk-through at https://hynek.me/articles/sharing-your-labor-of-love-pypi-quick-and-dirty/ ?

That kind of "This is a good, comprehensive, example of doing it right" endorsement is softer than actually providing such a prescriptive guide directly on packaging.python.org, but still meets the need for a step-by-step introduction that doesn't ask first-time publishers to make any decisions they may not be comfortable making yet.

One minor thing - @hynek's post says "Sorry, no Windows", which in't needed as (as far as I can see) there's basically nothing in the post that's OS-specific.

But that nit aside, I agree with the principle of pointing to articles that we endorse to break the logjam of "we don't want to say anything as it makes it too official".

If someone can confirm that everything works on Windows as described, I’ll be more than delighted to remove it. :)

I haven't run all of the commands, but I have reviewed everything and the only points I'd make are:

1. The read() function in setup.py assumes UTF-8. Windows users will have to specifically ensure that they get their text editor to save in UTF-8. But honestly, it's the only sane cross-platform choice, so that's not a problem.
2. In the "You can test whether both install properly" sections, the commands are Unix, but there's no way you could avoid some level of platform specificity in an example like this. I'd just add a comment "the examples use Unix, Windows would be similar".

(Honestly, as a Windows user, I'm used to reading articles on the net that use Unix terms, and mentally translating. I wouldn't even have thought about the issue if you hadn't explicitly mentioned it).

Just adding my 2 cents to the discussion. I stumbled over this a few days ago, as I also have 4 places where I need to change the version number in my library, and failed to do so correctly already twice.

I really like method 1, as the __version__ is defined in your main __init__.py as a string, which I would expect from most packages. It is the first place where I would look for it. If the problem is that regexes are perceived as hard / obscure for some people, why could setuptools not propose the find_version method ? It really looks like some boilerplate code that could benefit to everybody. That would hide the complexity away from the end user, and allow the setup tools maintainer to take care of the details.

About method 4, I find that saying:

An advantage with this technique is that it’s not specific to Python. Any tool can read the version

is probably not really an advantage. It's just more direct. You can probably extract that version from a .py file with any other language or tool without too much hassle.

And I think that having 7 options is way too many. Besides, at the end you end up reading this thread to get even more opinions :), which is great but ... not realistic for most of the users.

None of these is single-source, as pretty much everything is under version control these days, and people tag versions. I prefer using the VCS as source for the version using ~~get_version or versioneer~~ hatch-vcs.