Schedule stop for 3.6 support and discuss a policy going forward

[PGijsbers]

Most scientific packages already have Py3.7+ releases or are talking about it (NEP 29, scikit-learn issue). This means we should make plans on when to drop Py3.6 as well if we want to use the latest version of those packages. Additionally it allows us to use newer language features and ensures users are using a safer and faster Python. It's probably also a good idea to have a discussion on a policy going forward.

NumPy follows NEP 29:

When a project releases a new major or minor version, we recommend that they support at least all minor versions of Python introduced and released in the prior 42 months from the anticipated release date with a minimum of 2 minor versions of Python.

This would mean that we should have dropped 3.6 last year, and 3.7 in December. Since (Python) minor versions are backwards compatible we don't actually have to make code changes, but we do have to:

• update setup.py with new minimum requirement and removing a pypi classifier
• update the continuous integration Python version matrix

And as a bonus it allows us to refactor old code with new language features, where applicable. These include:

• 3.7:
  • Data classes might simplify our core OpenML entity classes, worth a closer look.
  • Module attributes could perhaps be interesting for the extensions as described here.
  • ordered dictionaries in the language specification (as opposed to CPython implementation detail)
• 3.8:
  • the notorious walrus operator.
  • positional only parameters
• 3.10 (release later this year):
  • match statement

Of course actually using any of these features would break backwards compatibility to older Python minor versions.

[PGijsbers]

I propose we bump straight to Py3.8+ before the end of this year, and from then on follow the NEP guideline or stricter (i.e. drop minor versions faster).

[mfeurer]

Sounds good to me. In addition to NEP29 we should always strive to support the python version of the latest Ubuntu LTS and Google Colab as a lot of people will be using those.

We could drop Python3.6 for the upcoming 0.13 development?

[PGijsbers]

Good points. Ubuntu 20.04 LTS features Python 3.8, Google Colab features Python 3.7 by default (as of writing). I suppose we can drop official Py3.6 support in 0.13 and then in principle drop Py3.7 support in December if Google Colab moved to Py3.8 at that time.

[PGijsbers]

I propose we leave this issue open until:

• [ ] The policy is described (or referenced) in developer documentation.
• [ ] The steps to drop Python 3.6 support for the 0.13 release are taken.

[PGijsbers]

An important reason this is not straightforward as I initially thought is the support for automated reproduction of machine learning experiments. We'll need to discuss how we can reinstantiate old models (e.g. scikit-learn 0.20, which requires <Py3.7) when the package requires newer software (e.g. Py3.7+). Simply only supporting it through old openml-python versions is complicated due to the server API changing over time.

[eddiebergman]

As far as I'm aware, the slightly insane but sane way to do this, where you need backwards compatibility but also hit a moving target (the servers), is to have a dedicated 3.6 branch in which the only part that should be updated is the API interaction end point.

[PGijsbers]

We will split off the scikit-learn extension into its own package (we can make it an optional dependency for easy install). We will then make sure that openml-python and the sklearn extension continue to work for recent versions of python and scikit-learn, but in principle will not put more effort into full compatibility with the old stack for reproducing runs. There is a lot of overhead and there are likely few, if any, users. If it turns out that there are affected users, we will look into how to best support them.

When scikit-learn is split off, openml-python can move to 3.9+ (and 3.10+ in April). Similar for the scikit-learn extension when we release a version that officially drops support for old scikit-learn versions.