RFC: Export versioning

[Keno]

As julia has grown, questions about API evolution, both in Base and in packages have become more important. One particular manifestation of this issue is that adding exports is permitted in minor versions, but that can of course be breaking when those exports start conflicting with exports in other packages (ref #42080). One solution that's been thrown around here and there is to version a package's exported API so that different importers can have different views of a package's API. Such a feature could be implemented as an extension of #54654, so I figured now would be the time to flesh out what it would look like and whether we want it.

General Overview

The general idea is that exports would (optionally) declare version ranges for which the export applies. E.g. base might do:

# atomic was added in 1.7
export vr">= 1.7": @atomic

On the flip side, imports could specify a specific version:

using Base(v"1.10")

Since 1.10 is in the version range >= 1.7, this pulls in 1.10. In particular, even if the current julia version is 1.11, any statements tagged as export vr">= 1.11" would not get pulled in by the using statement.

An export without version specification would be equivalent to a full range, preserving current behavior. An import without version specification would default to the minimum compat version declared in Project.toml.

Detailed Motivation

The motivation for this feature is two-fold. First, the addition of conflicting exports would no longer cause erros, simply because julia or a dependent package is upgraded (addressing the concern in #42080).

The second motivation is to allow more ways of non-breaking API evolution. The key thing here is that different dependent packages can have different views of the API surface of a package. For example, suppose I am interested in using a new feature in a commonly used package (let's say Graphs for argument's sake). However, let's say one of my other dependencies also depends on Graphs and the functionality it uses happens to have been renamed in the current version of Graphs. Currently, because the view of exported symbols is consistent for all packages, I would need to update my other dependency to the new Graphs version before I could use the unrelated new feature. If Graphs instead versioned its exported and still provided the old API, even in newer versions, I could start using the new feature while keeping my dependency at the old (API, but not package) version until there's a natural point to upgrade.

For completeness, I should note that a feature like this exists in UNIX dynamic linkers, though it is not used particularly frequently outside the C library itself, because it has poor language level-integration.

Concrete semantic proposal

1. All export statements gain the ability to specify an opaque version range object (Base to provide a default and corresponding string macro).
2. All import/using statements gain the ability to specify an opaque version object.
3. On import/using, the visible symbols are precisely those for which in(import_version, export_version_range) (as evaluated using the visibility/world age for in of the importing Module).
4. If not specified, import_version defaults to Base.min_compat_version(this_module, imported_module). This map is populated from Project.toml with the minimum specified compat version.
5. If not specified, export defaults to default_export_range() (as looked up in the exporting module).
6. (Optional, but probably a good idea) It is an error to try to import an API version larger than your minimum compat (smaller is allowed).

Concrete syntax proposal

One option would be to allow verion(ranges)s after very symbol. The simplest way to do that would be:

export foo vr">= 1.7",
	   bar vr">= 1.8",
	   bob(vr"1.9, 1.10") # Parentheses optional

import Base.foo v"1.8"
import Base.bar(v"1.8") # Parentheses optional

However, It's a bit weird to specify a different version for every symbol, so it probabl makes most sense to group these:

export vr">= 1.7": foo, # ...
export vr">= 1.8": bar
export vr"v1.9, 1.10": bob

or on import

import Base(v"1.8"): foo, bar

though I would expect most imports to happen implicitly via the Project.toml mechanism:

import Base: foo, bar # only works if compat julia is >= 1.8 in Project.toml

I'm open to other suggestions here.

Examples of advanced use cases

While one of the primary use cases of this is just to protect new exports from causing conflicts, this feature can be used for API evolution more broadly.

Symbol renaming/deletion

Suppose we added a new function foo in 1.6, but in 1.11 we decided bar was a better name for it, but then in 1.12 we deleted it entirely from the export set.

function bar() # Formerly foo
end
export vr"1.6-1.10": bar as foo
export vr"1.11": bar
# Deleted in >= 1.12

Packages could define their own policies for how long to continue providing old API sets, but presumably, APIs would continue being supported until at least the next major version.

API surface renaming

It is also possible to use this mechanism for changes to the API itself, e.g. addition or removal of kwargs, changes in the arguments, etc.

# Gained a mandatory `io` argument in 1.10
function hello(io::IO)
	println(io, "Hello")
end
hello_legacy() = hello(stdout)

export vr"1.6-1.9": hello_legacy as hello
export vr">= 1.10": hello

That said, one does have to be a bit careful here since these are different generic functions, so this does not work for interfaces that are expected to be extended (at least by itself).

[kimikage]

I am not sure if this is a good proposal since I do not fully understand the following two points:

• What is the "version" or "version range" here?
  • julia, package, or API (more loosely coupled with the implementation)?
• When is the version determined and when are the version ranges evaluated?
  • Depending on the timing of the evaluation, it does not seem to be very compatible with existing isdefined switches, precompilation caches, etc.

My underlying concern is that this may not be very compatible with the SemVer management of packages and stdlibs.

Edit: This seems to be a problem of my poor comprehension skills.

[bvdmitri]

As a package developer I would comment that this seems a bit too intimidating and my mental model just does not work on the provided code snippets. Not talking for everyone, but it just does not feel intuitive and instead brings some complex mental exercise. It's quite difficult to keep in mind potentially different API of one dependency.

Currently, because the view of exported symbols is consistent for all packages, I would need to update my other dependency to the new Graphs version before I could use the unrelated new feature.

Why is it a big problem? It is annoying for sure, but this proposal suggests to create a universe of madness in the API's of dependencies. Just fixing your package to a new version of some dependency seems mentally easier than tracking potentially different versions of API of the same dependency. I'm not even saying that proper IDE integration of this proposal might be quite difficult as well.

[Seelengrab]

One additional worry I have with this is that this yet again doesn't handle "small" API changes relating to a single signature/method. For an already exported name, it's impossible to communicate that a new signature is now supported API from a given version onwards. Is the intended solution to use a new name so that it can be versioned? That seems backwards to me.

[Keno]

Thinking about this again over dinner today, I think it makes more sense to move the predicate to the import side, i.e. the exporting module would tag exports with some immutable tag. These tags would form a total preference order, and then the import side would specify some predicate which gets evaluated over each tag to determine which export sets are part of the resolution set. I think that's cleaner.

[StefanKarpinski]