make __init__.py's version a 3-tuple

[indirectlylit]

Observed behavior

Currently, __init__.py looks like this:

#: This may not be the exact version as it's subject to modification with
#: get_version() - use ``kolibri.__version__`` for the exact version string.
VERSION = (0, 8, 0, 'alpha', 0)

The fact that the fourth value ('alpha'/'beta'/'final') needs to be updated makes creating releases and managing release branches very frustrating:

• we need to do a whole PR+review+merge to set it to 'final', create a release tag, then do another PR+review+merge to set it back to either 'alpha' or 'beta'
• this creates repeated merge conflicts when syncing old releases back to develop, or even worse across multiple release branches
• there doesn't seem to be any difference between 'beta' and 'alpha'
• the fifth value (0) is completely ignored
• need to create an alpha0 tag on develop after cutting a new beta or we get the confusing error You have added a final tag without bumping kolibri.VERISON

Expected behavior

Since the alpha/beta/final status is reflected in git tags and not in source code, we shouldn't need to update these values in source. Instead, we can simply do:

#: This string does not reflect alpha/beta/final status and is modified by get_version()
#: Use ``kolibri.__version__`` for the exact version string.
VERSION = (0, 8, 0)

Then the final/beta/alpha status can be driven by git tags and the process for releasing will be greatly simplified

Final releases should be embedded in the code. Everything else can come from the git tag. Anything interim can be versioned with a git hash, because finding parents is unreliable.

If a tag is made (alpha, beta, rc, or final) that does not match the 3-tuple in __init__.py, then the code should throw an error, to prevent tags being made on the wrong branch or code.

User-facing consequences

No user-facing consequences, but this would improve developer experience for creating new releases and managing release branches.

Steps to reproduce

Go through the process of creating a patch release and then syncing those changes back to develop

[indirectlylit]

cc @rtibbles @benjaoming @aronasorman

[benjaoming]

I'm very much :-1: on re-opening anything that has to do with versioning. We spent so much time on this already, I think we just need to stick to a routine and move along.

[benjaoming]

Current problem in develop was because the 0.8 release branch had a tag 0.9.0-alpha0, I deleted that tag and bumped kolibri.VERSION. It now works.

Exception was legit.

Obtaining file:///home/travis/build/learningequality/kolibri
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/home/travis/build/learningequality/kolibri/setup.py", line 16, in <module>
        import kolibri
      File "kolibri/__init__.py", line 13, in <module>
        __version__ = str(get_version(VERSION))
      File "kolibri/utils/lru_cache.py", line 137, in wrapper
        result = user_function(*args, **kwds)
      File "kolibri/utils/version.py", line 385, in get_version
        return get_prerelease_version(version)
      File "kolibri/utils/version.py", line 297, in get_prerelease_version
        git_version,
    AssertionError: Version detected from git describe --tags, but it's inconsistent with kolibri.__version__.__version__ is: (0, 8, 0, u'beta', 0), tag says: (0, 9, 0, 'alpha', 0).

[indirectlylit]

I'm very much 👎 on re-opening anything that has to do with versioning. We spent so much time on this already, I think we just need to stick to a routine and move along.

no urgency on this. just flagging that it's a frustrating process currently

[benjaoming]

kolibri/utils/version.py has over 30 commits, together with a rather lengthy set of tests. I think it's worth giving it a long-lived chance, i.e. let the routines settle in.

The other day, I was chatting about some annoying stuff in Django, and I think we found out that it was like "opening up a can of worms... and bugs" :smile:

Let's reap this!

[rtibbles]

We've tried reaping this for 3 months now, and it continues to be an incredibly frustrating process with multiple merge conflicts, builds that are broken for no good reason, and continual busy work to keep version tuples and git tags aligned.

Final releases should be embedded in the code. Everything else can come from the git tag. Anything interim can be versioned with a git hash, because finding parents is unreliable.

[indirectlylit]

Final releases should be embedded in the code. Everything else can come from the git tag. Anything interim can be versioned with a git hash, because finding parents is unreliable.

This sounds correct to me

[benjaoming]

Hi @rtibbles @indirectlylit !

It sounds like it's time to find out where the versioning process can be relaxed.

incredibly frustrating

I'm sorry that this is causing issues. I would like to point out the nature of CI and builds: They break, and for various reasons. Waiting for something to fail for a "meta" reason and then rebuilding is not nice of course, but the version string is really important, more than just as some meta reasoning, because it generates one of the most important pieces of knowledge upon which all of subsequent support inquiries and remote debugging rely on. Better have builds break before releasing to the wider community than risking ambiguity in versions.

Especially also worth noting that every bit of extra time for assertions on our side makes holistic sense, because fellow community members may be spending a large amount of time downloading due lack or bandwidth or cannot access subsequent upgrades.

But because of striving for a "press of a button" release mechanism, we carefully put assertions of where we should be vary of human errors.

The frustration may also indicate a warning of pressure being high before releases, so in that regards there are also other factors at play.

builds that are broken for no good reason

That isn't true, they break because of cautiousness :) The assertions are there with good reasons, they implement our release policy (and guard against human or unexpected errors). If they need to be adjusted, then let's do that.

Since we are many maintainers, often working under pressure, I'd advocate staying no the cautious.

Final releases should be embedded in the code. Everything else can come from the git tag.

We also need the kolibri/VERSION file. The complexity remains in the order or version resolution:

1. If final release, don't do stuff like create git subprocesses, just assume the final version
2. If pre-release step 1: a. Derive from git --describe (if in a git repo and that repo is a kolibri repo!) and recall all the issues with branches etc, so verify the output before using it... b. Derive from kolibri/VERSION if that file exists (and is inside a reasonable threshold of kolibri.__version__ because we are/were cautious of distributing the wrong file, resolving the wrong path or whatever) c. Fall back to whatever is in kolibri/
3. If pre-release step 2: Suffix the pre-release version with .dev+git...

This version is ultimately a correct final version: (1, 2, 3, 'post', 1), should resolve to 1.2.3.post1. In case something messes up after a release (for instance a corrupt file) and it cannot be re-called and has to be re-built with minor adjustments, this notation is used. We never used it, so maybe we want to rule out that possibility.

In relation to the proposal of removing the last version component and only relying on git --describe / kolibri/VERSION, I would say: Yes, that can be done, provided that we do not want to use 1.2.3.post1 for final releases.

I can make a test case + fix for #3678 and then move onto work to ignore the latter version component, although I don't know if this is originally motivated by that very same issue..

If you feel tempted to simplify this problem out of frustrations, please have a recreational visit to utils/version.py :smile:

[rtibbles]

Fixed in #9740