rebar3 Unable to build tar with relname different to app name

I'm busy upgrading rebar3 from a much older version.

When building a tar with a --relname different to the application name the building of the tar fails as it's looking at the wrong path.

Config snippet

{relx, [{release, {smd, "3.1.0"},
         [{smd, "3.1.0"}, .....],
         [{overlay_vars, "rel/vars/build_date.config"},
          {overlay, [{template, "rel/files/vm.args", "releases/{{release_version}}/vm.args"},
                     {template, "rel/files/smd.config", "releases/{{release_version}}/sys.config"}]},
          {extended_start_script, true}]},

        {release, {smd_node, "3.1.0"},
         [{smd, "3.1.0"}, .....],
         [{overlay_vars, "rel/vars/build_date.config"},
          {overlay, [{template, "rel/files/vm.args", "releases/{{release_version}}/vm.args"},
                     {template, "rel/files/smd_node.config", "releases/{{release_version}}/sys.config"}]},
          {extended_start_script, true}]}
    ]}.
{profiles, [{local,
             [{relx, [{dev_mode, true}
                     , {include_erts, true}
                     , {overlay_vars, "rel/vars/local.config"} ]}]},
            {local_node,
             [{relx, [{dev_mode, true}
                     , {include_erts, true}
                     , {overlay_vars, "rel/vars/local.config"} ]}]},
            {local_core,
             [{relx, [{dev_mode, true}
                     , {include_erts, true}
                     , {overlay_vars, "rel/vars/local.config"} ]}]},
            {prod_core,
             [{relx, [{overlay_vars, "rel/vars/prod_core.config"}
                     , {include_erts, true}
                     , {include_src, false}
                     , {dev_mode, false}
                     , {debug_info, strip}
                     ]}]},
            {prod_node,
             [{relx, [{overlay_vars, "rel/vars/prod_node.config"}
                     , {include_erts, true}
                     , {include_src, false}
                     , {dev_mode, false}
                     , {debug_info, strip}
                     ]}]}
           ]}.

When running rebar3 as prod_node release -n smd_node works fine and the smd app is built and ends up at _build/prod_node/rel/smd_node

When running

leonard@captmarvel:/opt/smd$ rebar3 as prod_node tar --relname=smd_node
===> Verifying dependencies...
===> Analyzing applications...
===> Compiling smd
===> /opt/smd/_build/prod_node/rel/smd/releases/start_erl.data is missing

It appears this is due to commit 71e497146 where command_args and command_parsed_args are being removed from the state.

I can understand the reasoning, but this makes it impossible to use relname for tars where the application name and the release name do not match.

Would it make sense to not remove relname from the command args?

Aug 04 '23 18:08 leonardb

Hm, there's something odd here because you're absolutely not relying on hooks anywhere in the config, so I'm a bit confused as to why hook arguments have any bearing there.

Particularly, keeping the argument list would likely fix things if tar was seen as anything except a post-hook of release, or unless tar specifies release as a pre-hook (and would break things if tar became a post-hook of say, running tests—which is why this fix isn't ideal), but that's not the way the bug is surfacing here.

Checking things step by step:

Tar doesn't seem to rely on release and does not specify hooks.
The opt spec list carries the term so the provider should get it from the command line.
The hooks are run but don't save their state so the omission should not impact the relup task.
The tar task is called directly in relx
- The picking of the release seems to assume the call sets the relname and version
- The name should be chosen in a common path

So there's something missing here in how the issue is reported. I tried reproducing it:

λ [vps] /tmp → rebar3 new release aaa
===> Writing aaa/apps/aaa/src/aaa_app.erl
===> Writing aaa/apps/aaa/src/aaa_sup.erl
===> Writing aaa/apps/aaa/src/aaa.app.src
===> Writing aaa/rebar.config
===> Writing aaa/config/sys.config
===> Writing aaa/config/vm.args
===> Writing aaa/.gitignore
===> Writing aaa/LICENSE.md
===> Writing aaa/README.md
λ [vps] /tmp → cd aaa
λ [vps] aaa → vim rebar.config
λ [vps] aaa → cat rebar.config
{relx, [{release, {aaa, "0.1.0"},
         [aaa, sasl]},
        {release, {bbb, "0.1.0"},
         [aaa, sasl]}]}.
λ [vps] aaa → rebar3 do release --relname bbb, tar --relname bbb
===> Verifying dependencies...
===> Analyzing applications...
===> Compiling aaa
===> Assembling release bbb-0.1.0...
===> Release successfully assembled: _build/default/rel/bbb
===> Verifying dependencies...
===> Analyzing applications...
===> Compiling aaa
===> Assembling release bbb-0.1.0...
===> Release successfully assembled: _build/default/rel/bbb
===> Building release tarball bbb-0.1.0.tar.gz...
===> Tarball successfully created: _build/default/rel/bbb/bbb-0.1.0.tar.gz

So at least, the command works exactly as advertised.

Can you provide at least your hook specification to match here?

Specifically though, if we want to fix it for hooks (which is a valid concern), the fix needs to be a bit more careful. For one, rather than looking only at the hook's provider (tar in your fix, which makes me believe you're using it as a post-hook to release), we would likely want to check that the tar post-hook is paired with the release Command, and that the release pre-hook is paired with the tar Command. This will at least prevent crashes when trying to run commands with different configuration sets.

Aug 09 '23 14:08 ferd

So it appears you're correct.

We have

{provider_hooks,
 [{pre, [{tar, {appup, tar}}]},
  {post, [{compile, {appup, compile}},
          {clean, {appup, clean}}]}]}.

It appears that the pre hook there is the issue. Will have to do some research as to why this is here.

On Wed, Aug 9, 2023 at 10:32 AM Fred Hebert @.***> wrote:

Hm, there's something odd here because you're absolutely not relying on hooks anywhere in the config, so I'm a bit confused as to why hook arguments have any bearing there.

Particularly, keeping the argument list would likely fix things if tar was seen as anything except a post-hook of release, or unless tar specifies release as a pre-hook (and would break things if tar became a post-hook of say, running tests—which is why this fix isn't ideal), but that's not the way the bug is surfacing here.

Checking things step by step:

Tar doesn't seem to rely on release https://github.com/erlang/rebar3/blob/9ae99e9c5d94a5a3dd37fa6eff22d9a49f3748ca/apps/rebar/src/rebar_prv_tar.erl#L14-L15 and does not specify hooks https://github.com/erlang/rebar3/blob/9ae99e9c5d94a5a3dd37fa6eff22d9a49f3748ca/apps/rebar/src/rebar_prv_tar.erl#L23-L30 .

The opt spec list carries the term https://github.com/erlang/rebar3/blob/9ae99e9c5d94a5a3dd37fa6eff22d9a49f3748ca/apps/rebar/src/rebar_relx.erl#L277-L278 so the provider should get it from the command line.

The hooks are run but don't save their state https://github.com/erlang/rebar3/blob/9ae99e9c5d94a5a3dd37fa6eff22d9a49f3748ca/apps/rebar/src/rebar_relx.erl#L53C36-L53C36 so the omission should not impact the relup task.

The tar task is called directly in relx https://github.com/erlang/rebar3/blob/9ae99e9c5d94a5a3dd37fa6eff22d9a49f3748ca/apps/rebar/src/rebar_relx.erl#L130-L131

The picking of the release seems to assume the call sets the relname and version https://github.com/erlware/relx/blob/8c571fde08aaa2d174e69fb632ba98312f3765a0/src/relx.erl#L110-L128

The name should be chosen in a common path https://github.com/erlang/rebar3/blob/9ae99e9c5d94a5a3dd37fa6eff22d9a49f3748ca/apps/rebar/src/rebar_relx.erl#L55

So there's something missing here in how the issue is reported. I tried reproducing it:

λ [vps] /tmp → rebar3 new release aaa ===> Writing aaa/apps/aaa/src/aaa_app.erl ===> Writing aaa/apps/aaa/src/aaa_sup.erl ===> Writing aaa/apps/aaa/src/aaa.app.src ===> Writing aaa/rebar.config ===> Writing aaa/config/sys.config ===> Writing aaa/config/vm.args ===> Writing aaa/.gitignore ===> Writing aaa/LICENSE.md ===> Writing aaa/README.md λ [vps] /tmp → cd aaa λ [vps] aaa → vim rebar.config λ [vps] aaa → cat rebar.config {relx, [{release, {aaa, "0.1.0"}, [aaa, sasl]}, {release, {bbb, "0.1.0"}, [aaa, sasl]}]}. λ [vps] aaa → rebar3 do release --relname bbb, tar --relname bbb ===> Verifying dependencies... ===> Analyzing applications... ===> Compiling aaa ===> Assembling release bbb-0.1.0... ===> Release successfully assembled: _build/default/rel/bbb ===> Verifying dependencies... ===> Analyzing applications... ===> Compiling aaa ===> Assembling release bbb-0.1.0... ===> Release successfully assembled: _build/default/rel/bbb ===> Building release tarball bbb-0.1.0.tar.gz... ===> Tarball successfully created: _build/default/rel/bbb/bbb-0.1.0.tar.gz

So at least, the command works exactly as advertised.

Can you provide at least your hook specification to match here?

Specifically though, if we want to fix it for hooks (which is a valid concern), the fix needs to be a bit more careful. For one, rather than looking only at the hook's provider (tar in your fix, which makes me believe you're using it as a post-hook to release), we would likely want to check that the tar post-hook is paired with the release Command, and that the release pre-hook is paired with the tar Command. This will at least prevent crashes when trying to run commands with different configuration sets.

— Reply to this email directly, view it on GitHub https://github.com/erlang/rebar3/issues/2814#issuecomment-1671501597, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFKWMA74MYXBUMJOSFET53XUONOPANCNFSM6AAAAAA3ERRN2M . You are receiving this because you authored the thread.Message ID: @.***>

Aug 09 '23 15:08 leonardb

rebar3 rebar3 copied to clipboard

Unable to build tar with relname different to app name

rebar3
rebar3 copied to clipboard