bin/START upgrade VERSION failed when using sys.config.src and vm.args.src

[qingchuwudi]

Environment

Rebar3 report
 version 3.13.0
 generated at 2020-01-20T07:05:13+00:00
=================
Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task:
Entered as:

-----------------
Operating System: x86_64-apple-darwin19.0.0
ERTS: Erlang/OTP 22 [erts-10.6] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] [dtrace]
Root Directory: /usr/local/Cellar/erlang/22.2/lib/erlang
Library directory: /usr/local/Cellar/erlang/22.2/lib/erlang/lib
-----------------
Loaded Applications:
bbmustache: 1.8.0
certifi: 2.5.1
cf: 0.2.2
common_test: 1.18.1
compiler: 7.5
crypto: 4.6.3
cth_readable: 1.4.6
dialyzer: 4.1.1
edoc: 0.11
erlware_commons: 1.3.1
eunit: 2.4
eunit_formatters: 0.5.0
getopt: 1.0.1
hipe: 3.19.2
inets: 7.1.2
kernel: 6.5.1
providers: 1.8.1
public_key: 1.7.1
relx: 3.33.0
sasl: 3.4.1
snmp: 5.4.4
ssl_verify_fun: 1.1.5
stdlib: 3.11
syntax_tools: 2.2.1
tools: 3.3

-----------------
Escript path: /usr/local/bin/rebar3
Providers:
  app_discovery as clean compile compile cover ct deps dialyzer do edoc escriptize eunit get-deps help install install_deps list lock new path pkgs release relup report repos shell state tar tree unlock update upgrade upgrade upgrade version xref

Current behaviour

I'm using sys.config.src and vm.args.src ( Dynamic Configuration ). This future is every useful. But there is a problem when I want to use appup. Erlang node will crash when I upgrade application.

• Two versions of an application: 1.4.0 and 1.4.1
• Upgrade application from 1.4.0 to 1.4.1

Config:

{plugins, [
	{rebar3_appup_plugin, "2.2.1"},
	{rebar3_gpb_plugin, "2.4.2"}
]}.
{relx, [
	{
		release, {account, "1.4.1"}, [
                ... ...
                , ts_node, 
                ... ...
		account
	]},
	{dev_mode, true},
	{include_erts, true},
	{system_libs, true},
	{include_src, false},
	{generate_start_script, true},
	{extended_start_script, true},

	{sys_config, "./config/sys.config"},
	{vm_args, "./config/vm.args"}
]}.
{profiles, [
	{dev, [
		{relx, [
			{dev_mode, false},
			{sys_config_src, "./config/dev/sys.config.src"},
			{vm_args_src, "./config/dev/vm.args.src"}
		]}
	]}
]}.

config └── dev    ├── sys.config.src    └── vm.args.src

Operations:

1. release 1.4.0 and run:

   $ git checkout 1.4.0
   $ rebar3 as dev
   ... ...
   $ mkdir /data/test_appup
   $ tar xzf account-1.4.0.tar.gz -C /data/test_appup/
   $ /data/test_appup/bin/account console
   

   Everything is ok.

2. appup 1.4.1:

   $ git checkout 1.4.1
   $ rebar3 as dev release
   $ rebar3 as dev appup generate --previous_version 1.4.0
   $ rebar3 as dev relup tar
   ... ...
   $ cp account-1.4.1.tar.gz /data/test_appup/releases
   

3. install 1.4.1:

   $ /data/test_appup/bin/account install 1.4.1  ##crashed after this operation
   

log:

Warning: "/data/account/test_appup/releases/1.4.1/sys.config" missing (optional)
2020-01-20T14:34:23.449+08:00 [notice] <0.1056.0> application_controller:info_exited:1938 [{application,recon},{exited,stopped},{type,permanent}] {application_controller,exit}
2020-01-20T14:34:23.483+08:00 [notice] <0.1056.0> application_controller:info_exited:1938 [{application,observer_cli},{exited,stopped},{type,permanent}] {application_controller,exit}
2020-01-20T14:34:23.853+08:00 [error] <0.1175.0> gen_server:error_info:889 {state} "{{badmatch,undefined},[{ts_node,master,0,[{file,"/data/Dragon/account/_build/default/lib/ts_node/src/ts_node.erl"},{line,41}]},{tsn_connect_process,handle_info,2,[{file,"/data/Dragon/account/_build/default/lib/ts_node/src/tsn_connect_process.erl"},{line,124}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}" tsn_connect_process  CHECK_CLUSTER {gen_server,terminate} undefined

ts_node is my private project, and ts_node:master/0:

-spec(master()-> true| node()).
master() ->
    %% Crash here. 
    %% Got unexpected return value ‘undefined’
    {ok, Master} = application:get_env(ts_node, master), 
    ... ...

Finally, I found it that sys.config.src is not generated in the latest version of my application.

$ ls /data/test_appup/releases/1.4.1
relup          start.boot     sys.config.src vm.args.src

Expected behaviour

sys.config and vm.args can be generated with sys.config.src and vm.args.src, when using rebar3 install|uninstall|upgrade

---

[tsloughter]

I thought all the sys.config.src conversion stuff happened before any command was run, but this must be related to those commands either not being pointed to the correct sys.config or the process of converting from the .src never running.

This would be an issue with the extended_start_script in relx.

If you have a test repo I can pull down that'd help a lot to get started.

[qingchuwudi]

This is the test repo rebar3_2208.

[tsloughter]

@qingchuwudi thanks. I hope to fix this before releasing relx-4.0 but it is a tough one. The install command calls runs an escript install_upgrade_script which unpacks the tarball and then installs it. The conversion of sys.config.src to sys.config must come in between those 2 steps, which is a problem since the conversion is done in the shell script and has no way to currently call only that functionality.

It may be that install_upgrade_script should be replaced with more shell script and use erl_call for running the release_handler functions. I don't know yet if that will be doable.

[qingchuwudi]

Thank you all the same. I will avoid using sys.config.src and install/upgrade at the same time.