Unable to download hex dependencies from a wsl setup behind a corporate firewall

[sinharahul]

Pre-Check

• If you are filing for a bug, please do a quick search in current issues first
• For bugs, mention if you are willing or interested in helping fix the issue
• For questions or support, it helps to include context around your project or problem
• Think of a descriptive title (more descriptive than 'feature X is broken' unless it is fully broken)

Environment

• Add the result of rebar3 report to your message: Rebar3 report version 3.20.0 generated at 2023-03-26T01:15:28+00:00 ================= Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)

---

Task: Entered as:

---

Operating System: x86_64-pc-linux-gnu ERTS: Erlang/OTP 25 [erts-13.1.4] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit:ns] Root Directory: /home/rsinha/.asdf/installs/erlang/25.2.2 Library directory: /home/rsinha/.asdf/installs/erlang/25.2.2/lib

Loaded Applications: bbmustache: 1.12.2 certifi: 2.9.0 cf: 0.3.1 common_test: 1.23.3 compiler: 8.2.3 crypto: 5.1.2 cth_readable: 1.5.1 dialyzer: 5.0.4 edoc: 1.2 erlware_commons: 1.5.0 eunit: 2.8.1 eunit_formatters: 0.5.0 getopt: 1.0.1 inets: 8.2.1 kernel: 8.5.3 providers: 1.9.0 public_key: 1.13.2 relx: 4.7.0 sasl: 4.2 snmp: 5.13.3 ssl_verify_fun: 1.1.6 stdlib: 4.2 syntax_tools: 3.0 tools: 3.5.3

---

Escript path: /home/rsinha/.mix/rebar3.1 Providers: app_discovery as clean compile compile cover ct deps dialyzer do edoc escriptize eunit get-deps help install install_deps list lock new path pkgs release relup report repos shell state tar tree unlock update upgrade upgrade upgrade vendor version xref

$ rebar3 report "my failing command"
...

• Verify whether the version of rebar3 you're running is the latest release (see https://github.com/erlang/rebar3/releases)
• If possible, include information about your project and its structure. Open source projects or examples are always easier to debug. If you can provide an example code base to reproduce the issue on, we will generally be able to provide more help, and faster.

Current behaviour

Describe the current behaviour. In case of a failure, crash, or exception, please include the result of running the command with debug information:

DEBUG=1 rebar3 <my failing command>

(base) rsinha@MKTX-AP8920:/mnt/c/projects/elixir/testhttp$ DEBUG=1 mix ===> Load global config file /home/rsinha/.config/rebar3/rebar.config ===> Getting definition for package rebar3_hex from repo hexpm (#{api_url => <<"https://hex.pm/api">>,name => <<"hexpm">>, repo_name => <<"hexpm">>,repo_organization => undefined, repo_url => <<"https://repo.hex.pm">>,repo_verify => true, repo_verify_origin => true}) ===> hex_repo:get_package failed for package <<"rebar3_hex">>: {badmatch, {error, enotdir}} ===> throw {error, {rebar_app_utils,{missing_package,<<"rebar3_hex">>,undefined}}} [{rebar_app_utils, update_source, 3, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_app_utils.erl"}, {line, 335}]}, {rebar_prv_install_deps, '-handle_profile_level/7-lc$^0/1-0-', 3, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_prv_install_deps.erl"}, {line, 178}]}, {rebar_prv_install_deps, handle_profile_level, 7, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_prv_install_deps.erl"}, {line, 179}]}, {rebar_plugins, handle_plugin, 5, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_plugins.erl"}, {line, 121}]}, {rebar_plugins, '-handle_plugins/4-fun-0-', 5, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_plugins.erl"}, {line, 102}]}, {lists, foldl, 3, [{file, "lists.erl"}, {line, 1350}]}, {rebar_plugins, handle_plugins, 4, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_plugins.erl"}, {line, 101}]}, {rebar3, state_from_global_config, 2, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar3.erl"}, {line, 444}]}] ===> Errors loading plugin rebar3_hex. Run rebar3 with DEBUG=1 set to see errors. ===> Getting definition for package pc from repo hexpm ===> hex_repo:get_package failed for package <<"pc">>: {badmatch, {error,enotdir}} ===> throw {error,{rebar_app_utils,{missing_package,<<"pc">>,undefined}}} [{rebar_app_utils, update_source, 3, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_app_utils.erl"}, {line, 335}]}, {rebar_prv_install_deps, '-handle_profile_level/7-lc$^0/1-0-', 3, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_prv_install_deps.erl"}, {line, 178}]}, {rebar_prv_install_deps, handle_profile_level, 7, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_prv_install_deps.erl"}, {line, 179}]}, {rebar_plugins, handle_plugin, 5, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_plugins.erl"}, {line, 121}]}, {rebar_plugins, '-handle_plugins/4-fun-0-', 5, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_plugins.erl"}, {line, 102}]}, {lists, foldl, 3, [{file, "lists.erl"}, {line, 1350}]}, {rebar_plugins, handle_plugins, 4, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_plugins.erl"}, {line, 101}]}, {lists, foldl, 3, [{file, "lists.erl"}, {line, 1350}]}] ===> Errors loading plugin pc. Run rebar3 with DEBUG=1 set to see errors. ===> Expanded command sequence to be run: [] ===> Running provider: do ===> Expanded command sequence to be run: [app_discovery,{bare,compile}] ===> Running provider: app_discovery ===> Found top-level apps: [snappyer] using config: [{src_dirs,["src"]},{lib_dirs,["apps/","lib/","."]}] ===> Getting definition for package pc from repo hexpm ===> hex_repo:get_package failed for package <<"pc">>: {badmatch, {error,enotdir}} ===> throw {error,{rebar_app_utils,{missing_package,<<"pc">>,undefined}}} [{rebar_app_utils, update_source, 3, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_app_utils.erl"}, {line, 335}]}, {rebar_prv_install_deps, '-handle_profile_level/7-lc$^0/1-0-', 3, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_prv_install_deps.erl"}, {line, 178}]}, {rebar_prv_install_deps, handle_profile_level, 7, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_prv_install_deps.erl"}, {line, 179}]}, {rebar_plugins, handle_plugin, 5, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_plugins.erl"}, {line, 121}]}, {rebar_plugins, '-handle_plugins/4-fun-0-', 5, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_plugins.erl"}, {line, 102}]}, {lists, foldl, 3, [{file, "lists.erl"}, {line, 1350}]}, {rebar_plugins, handle_plugins, 4, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_plugins.erl"}, {line, 101}]}, {lists, foldl, 3, [{file, "lists.erl"}, {line, 1350}]}] ===> Errors loading plugin pc. Run rebar3 with DEBUG=1 set to see errors. ===> Running provider: {bare,compile} ===> Compile (untagged) ===> Running hooks for compile in app snappyer (/mnt/c/projects/elixir/testhttp/deps/snappyer) with configuration: ===> {provider_hooks, [{pre, [{pc,compile}]}]}. ===> Running provider: {pc,compile} ===> Unable to run pre hooks for 'compile', command 'compile' in namespace 'pc' not found. ===> Unable to run pre hooks for 'compile', command 'compile' in namespace 'pc' not found. ** (Mix) Could not compile dependency :snappyer, "/home/rsinha/.asdf/installs/elixir/1.14.3-otp-25/.mix/elixir/1-14/rebar3 bare compile --paths /mnt/c/projects/elixir/testhttp/_build/dev/lib/*/ebin" command failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile snappyer", update it with "mix deps.update snappyer" or clean it with "mix deps.clean snappyer"

Expected behaviour

Describe what you expected to happen. Elixir app with broadway loaded and running fine

[ferd]

That's certainly odd because the hex_repo error comes from the r3_hex_repo module that we vendor in.

My guess is that the call that fails is in https://github.com/erlang/rebar3/blob/5d1f72f2f7f93b33b931eb67b9d64f3c4a822602/apps/rebar/src/rebar_packages.erl#L142-L151 which is about not being able to create or access the cache directory in ~/.cache/rebar3/hex/. Double-checking permissions may make sense.

[sinharahul]

Hi Fred , Thanks for your comments .I tried granting write permissions to the ~/.cache/rebar3/hex/ but get the same error

[starbelly]

I didn't do a full trace of this, but I was able to replicate your issue I believe.

1. I created a mix project with snappyer as a dep (the exact dep doesn't matter).
2. I did a mix deps.get
3. I blew away my ~/.cache/rebar3 dir
4. I run mix deps.compile and ended up in state similar to yours, except the difference being undef and undefined returns vs enoent (this is on mac so that's perhaps why).
5. I mv my ~/.config/rebar3 dir out of the way
6. I run mix deps.compile again and all works now

This makes sense because I have {plugins, [rebar3_hex]}. in my global rebar config, and if you have no ~/.cache/rebar3 dir and what's more no plugins, etc in there, this will fail, and then everything else fails (a full trace needs to be done to understand that part though).

I feel like call from mix to the rebar bare compiler should not result in this behavior, it probably shouldn't even attempt to load a global rebar config, but maybe there's a reason it does that I'm not aware of.

@sinharahul Please try to either move /home/rsinha/.config/rebar3/rebar.config out of the way or comment out whatever plugin section you have in there and see if that resolves the issue.

[sinharahul]

Hi @starbelly Moving rebar.config out of the way results in rsinha@MKTX-AP8920:/mnt/c/projects/elixir/sputnik$ DEBUG=1 mix warning: use Mix.Config is deprecated. Use the Config module instead config/config.exs:1

===> Getting definition for package pc from repo hexpm (#{api_url => <<"https://hex.pm/api">>,name => <<"hexpm">>, repo_name => <<"hexpm">>,repo_organization => undefined, repo_url => <<"https://repo.hex.pm">>,repo_verify => true, repo_verify_origin => true}) =NOTICE REPORT==== 30-Mar-2023::21:38:37.287166 === TLS client: In state certify at ssl_handshake.erl:2111 generated CLIENT ALERT: Fatal - Unknown CA

===> Hex get_package request failed: {error, {failed_connect, [{to_address,{"repo.hex.pm",443}}, {inet, [inet], {tls_alert, {unknown_ca, "TLS client: In state certify at ssl_handshake.erl:2111 generated CLIENT ALERT: Fatal - Unknown CA\n"}}}]}} ===> Failed to update package pc from repo hexpm ===> throw {error,{rebar_app_utils,{missing_package,<<"pc">>,undefined}}} [{rebar_app_u

[sinharahul]

Hi @starbelly Moving rebar.config out of the way results in rsinha@MKTX-AP8920:/mnt/c/projects/elixir/sputnik$ DEBUG=1 mix warning: use Mix.Config is deprecated. Use the Config module instead config/config.exs:1

===> Getting definition for package pc from repo hexpm (#{api_url => <<"https://hex.pm/api">>,name => <<"hexpm">>, repo_name => <<"hexpm">>,repo_organization => undefined, repo_url => <<"https://repo.hex.pm">>,repo_verify => true, repo_verify_origin => true}) =NOTICE REPORT==== 30-Mar-2023::21:38:37.287166 === TLS client: In state certify at ssl_handshake.erl:2111 generated CLIENT ALERT: Fatal - Unknown CA

===> Hex get_package request failed: {error, {failed_connect, [{to_address,{"repo.hex.pm",443}}, {inet, [inet], {tls_alert, {unknown_ca, "TLS client: In state certify at ssl_handshake.erl:2111 generated CLIENT ALERT: Fatal - Unknown CA\n"}}}]}} ===> Failed to update package pc from repo hexpm ===> throw {error,{rebar_app_utils,{missing_package,<<"pc">>,undefined}}} [{rebar_app_u

[starbelly]

@sinharahul So we kind of have a new problem now, that was probably solved previously by what's in your rebar.config, is that correct? Specifically, did you have any ssl cert properties in your rebar.config?

Perhaps, you can share your rebar.config wholesale?

[sinharahul]

Thats correct .Heres my rebar.config {ssl_cacerts_path, ["/mnt/c/Users/rasinha/RootCA.crt","/mnt/c/Users/rasinha/Downloads/marketaxess-rootca.crt"]}. {plugins, [rebar3_hex]}.

[starbelly]

@sinharahul Ok, here's what you can try (and I realize this is not ideal, as I assume you have rebar3 projects in addition to mix projects). Comment out the plugins part of your rebar.config.

[sinharahul]

Hi @starbelly .Commented out the plugin part gives the original error Heres the rebar.config rsinha@MKTX-AP8920:/mnt/c/projects/elixir/sputnik$ cat /home/rsinha/.config/rebar3/rebar.config {ssl_cacerts_path, ["/mnt/c/Users/rasinha/RootCA.crt","/mnt/c/Users/rasinha/Downloads/marketaxess-rootca.crt"]}. And heres the error rsinha@MKTX-AP8920:/mnt/c/projects/elixir/sputnik$ DEBUG=1 mix warning: use Mix.Config is deprecated. Use the Config module instead config/config.exs:1

===> Load global config file /home/rsinha/.config/rebar3/rebar.config ===> Getting definition for package pc from repo hexpm (#{api_url => <<"https://hex.pm/api">>,name => <<"hexpm">>, repo_name => <<"hexpm">>,repo_organization => undefined, repo_url => <<"https://repo.hex.pm">>,repo_verify => true, repo_verify_origin => true}) ===> hex_repo:get_package failed for package <<"pc">>: {badmatch, {error,enotdir}} ===> throw {error,{rebar_app_utils,{missing_package,<<"pc">>,undefined}}} [{rebar_app_utils, update_source, 3, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_app_utils.erl"}, {line, 335}]}, {rebar_prv_install_deps, '-handle_profile_level/7-lc$^0/1-0-', 3, [{file, "/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_prv_install_deps.erl"}, {line, 178}]}, {rebar_prv_install_deps, handle_profile_level, 7, [{file,

[starbelly]

:(

Ok, so let me ask you this if you look in ~/.cache/rebar3, what's in there?

My structure after blowing it all away and running mix deps.compile in an example project look like :

.
.cache/rebar3
└── hex
    ├── hexpm
    │   └── packages
    │       ├── pc-1.14.0.etag
    │       └── pc-1.14.0.tar
    └── packages.idx

pc may not be there, but is the basic structure there including and hopefully packages.idx

[sinharahul]

Cache dir seems to be empty (base) rsinha@MKTX-AP8920:/mnt/c/projects/elixir/sputnik$ ls -lrt ~/.cache/rebar3/hex total 0

[starbelly]

@sinharahul Thanks. After re-visiting think, I don't think that's the issue. Here's an interesting bit though. Mix should be using rebar3 installed per mix local.rebar. If you went and checked, you'd probably find the version is 3.15.2.

As far as I am aware (by checking out the tag and looking in src/rebar_utils.erl) there is no support for ssl_cacerts_path in that version. Thus the option in your rebar.config is of no use in mix a project right now.

How to get around this :

Grab the latest version of rebar3 and overwrite what's in place under elixir :

• Latest pre-built : https://s3.amazonaws.com/rebar3/rebar3

• or git clone https://github.com/erlang/rebar3/ && git checkout 3.20.0 && ./bootstrap && ./rebar3 local install

Now that you have the latest rebar3, overwrite what elixir installs right now. This is going to vary depending on your version of elixir, platform, etc. an easy to figure this out is to run :

 mix local.rebar

That'll prompt you as to whether you want to install rebar3, choose no and just note the path.

Now cp the latest rebar3 into that dir, thus overwriting what's there.

You should see the rebar will use your ssl certs and all should work (assuming the certs are proper).

The other option here, and what I might personally do myself, if this is your linux box, I'd just install some certs globally on the machine. Of course, that's not something we're going to help with, but just a suggestion.

[gorkunov]

I have the same issue today with latest rebar3 and otp-26. And seems it's also connected with corporate firewall (laptop has bunch of cisco software)

[gorkunov]

resolved - complied rebar3 from git repo and copied rebar3 binary to path from mix local.rebar - based on comment from @starbelly. Thanks!