rebar 3.18 upgrade --all breaks build pipelines

[baronbosse]

Environment

 version 3.18.0
 generated at 2022-01-26T15:14:47+00:00
=================
Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task: 
Entered as:
  
-----------------
Operating System: x86_64-pc-linux-gnu
ERTS: Erlang/OTP 24 [erts-12.1.4] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit]
Root Directory: /home/patrik/.asdf/installs/erlang/24.1.4
Library directory: /home/patrik/.asdf/installs/erlang/24.1.4/lib
-----------------
Loaded Applications:
bbmustache: 1.12.2
certifi: 2.8.0
cf: 0.3.1
common_test: 1.21
compiler: 8.0.3
crypto: 5.0.4
cth_readable: 1.5.1
dialyzer: 4.4.2
edoc: 1.0.1
erlware_commons: 1.5.0
eunit: 2.7
eunit_formatters: 0.5.0
getopt: 1.0.1
inets: 7.4.2
kernel: 8.1.2
providers: 1.9.0
public_key: 1.11.3
relx: 4.6.0
sasl: 4.1
snmp: 5.10.1
ssl_verify_fun: 1.1.6
stdlib: 3.16.1
syntax_tools: 2.6
tools: 3.5.1

-----------------
Escript path: /home/patrik/.asdf/installs/rebar/3.18.0/bin/rebar3
Providers:
  app_discovery as clean compile compile cover ct deps dialyzer do edoc escriptize eunit fmt get-deps help install install_deps lint list lock new path pkgs release relup report repos shell state tar tree unlock update upgrade upgrade upgrade version xref 

Current behaviour

Most of our projects use a Makefile like this for building:

upgrade:
	rebar3 upgrade

compile:
	rebar3 compile

This now fails with

$ make
rebar3 upgrade
===> Verifying dependencies...
===> Specify a list of dependencies to upgrade, or --all to upgrade them all
make: *** [Makefile:4: upgrade] Error 1

Expected behaviour

rebar3 upgrade should fetch all dependencies.

Since adding --all to rebar upgrade for versions pre-3.18 also will fail, we would either have to upgrade everyone to 3.18 at the same time or go through every project and lots of checks on rebar version to either add --all or not to them.

We understand that rebar3 unlock was a bit broken, but making backward-incompatible changes like this for rebar3 upgrade which is used much much more is a bit bad.

[baronbosse]

This is also commented on in the issue for the rebar3 unlock fix here: https://github.com/erlang/rebar3/issues/2583#issuecomment-990850936

[ferd]

Well it is officially too late now, because we can't easily pull the old version back. So whatever we now publish to fix it forces to live with a broken state with the broken releases anyway. We could arguably allow a new dual mode which would ask that your pipelines still have a special version checks on which versions to ignore, but that still requires as much manual work as a regular version check to enable/disable that --all switch.

There's little to say except that we're sorry for the inconvenience.

[ferd]

Oh while I think about it, I'm a bit surprised you would have basic steps that just mandate upgrading all dependencies all the time. This mostly negates the whole purpose of the lock file if each build repeats nothing at all.

I think back then this is one of the things that made me go "Oh, it's backwards incompatible, but it shouldn't break too many CIs because you shouldn't upgrade everything all the time in a build anyway."

Ultimately my recommendations would however be:

1. Do not ignore the lock files for each build, this is a useful feature for safe builds
2. If you do want to upgrade all the time you might as well just not check the lock file into version control and remove the whole upgrade command altogether and speed up your builds.
3. If you do want to check in the lockfile but disregard it entirely, just rm -rf it I guess, that's really what this is asking for and will always be compatible, even if inadvisable from my point of view (but you know your builds better than I do).

[paulo-ferraz-oliveira]

(It also broke our CI, but yeah, we figured there was no turning back - and I kinda knew this was coming from previous comments...). As an example... Our CI is as follows (among other things):

1. unlock
2. upgrade
3. make sure all changes are committed to Git (ie. the local rebar.lock is the one pushed)

This way we make sure no rebar.lock update was missed.

Your recommendations make sense, though.

[baronbosse]

We use tags in rebar.config to lock versions. Doing a rebar3 upgrade automatically when you develop makes it super easy to switch branches and if your colleague has a new version of a dependency in his branch, compiling will just work.

Ultimately my recommendations on how to do releases on a build tool is:

1. If you want to do a breaking change, add this as an optional feature and deprecate the old way of doing stuff with a warning.
2. Let this simmer in the community for a few years giving them time to move over at their own pace. Since both ways of doing something still works, the community can safely upgrade to the new ways in a timely fashion.
3. When enough time has passed, remove the old way of doing stuff.

Ultimately my recommendation in this case is:

1. Don't fail on rebar3 upgrade for now, just give a warning.

And obligatory Linus Torvalds quote: "WE DO NOT BREAK USERSPACE!" (emphasis his)

[ferd]

If you switch branches and the deps are locked, Rebar3 should automatically fetch the proper dependencies without needing to upgrade. You don't need to call the upgrade command on all dependencies ever unless you actually want to upgrade all dependencies at once.

Here's the actual thing: this is a free project we've been maintaining unpaid for years in our free time, and there are things where the cost of NEVER BREAKING ANYTHING EVER (regardless of what Linus may favour) is way too high to actually do. We at some point maintained backwards compatibility up to 5 Erlang major releases, something the OTP team at Ericsson themselves do not do.

We try to do our best and sometimes make the decision that a small break is permissible (the Relx 4.0.0 update was much bigger) if the tradeoff for us maintaining things and offering a safer or more conceptually uniform experience is deemed worth it.

In this case, the main intent was to make plugin upgrading safer (since they do not have lock files), but in doing so we'd have made its interface inconsistent with dependency upgrading. So in making plugins safer to upgrade we applied the same change to dependencies (since they use a similar mechanism). I judged the change safe because, again, you should not really need to upgrade all dependencies all the time in an automated fashion, or so I thought.

Ultimately my recommendation is:

• We'll keep doing things that make maintainership workload sustainable; we have to look out for our own workload if we want to keep maintaining things in the long run and make it easier for contributors to come lend a hand.
• We can't guarantee a Linux-like level of support between the two of us, part time, for free
• You're gonna have to sometimes deal with our mistakes and decisions, unfortunately

[tsloughter]

Working when switching branches is something from the early days we wanted to make sure worked when starting on rebar3. It should not require running rebar3 upgrade --all and if it does it possibly a bug.