[BUG] ** (UndefinedFunctionError) function Argon2.Base.hash_password/3 is undefined (module Argon2.Base is not available)

[medikent]

Environment

• Elixir & Erlang/OTP versions (elixir --version):
  • Erlang/OTP 22 [erts-10.5.6] [source] [64-bit]
  • Elixir 1.9.4 (compiled with Erlang/OTP 22)
• Operating system: Debian:stretch (base image for Elixir 1.9: https://github.com/c0b/docker-elixir/blob/0d9f47458468a8bf1407374731cbec077ab6f895/1.9/Dockerfile)

Current behavior

mix ecto.setup crashes with the following error:

[error] Process #PID<0.755.0> raised an exception
** (RuntimeError) An error occurred when loading Argon2.
Make sure you have a C compiler and Erlang 20 installed.
If you are not using Erlang 20, either upgrade to Erlang 20 or
use bcrypt_elixir (version 0.12) or pbkdf2_elixir.
See the Comeonin wiki for more information.

    (argon2_elixir) lib/argon2/base.ex:19: Argon2.Base.init/0
    (kernel) code_server.erl:1340: anonymous fn/1 in :code_server.handle_on_load/5
[warn] The on_load function for module Elixir.Argon2.Base returned:
{%RuntimeError{message: "An error occurred when loading Argon2.\nMake sure you have a C compiler and Erlang 20 installed.\nIf you are not using Erlang 20, either upgrade to Erlang 20 or\nuse bcrypt_elixir (version 0.12) or pbkdf2_elixir.\nSee the Comeonin wiki for more information.\n"}, [{Argon2.Base, :init, 0, [file: 'lib/argon2/base.ex', line: 19]}, {:code_server, :"-handle_on_load/5-fun-0-", 1, [file: 'code_server.erl', line: 1340]}]}

** (UndefinedFunctionError) function Argon2.Base.hash_password/3 is undefined (module Argon2.Base is not available)
    (argon2_elixir) Argon2.Base.hash_password("john_doe", <<132, 208, 122, 4, 176, 119, 50, 178, 233, 162, 230, 117, 153, 129, 11, 130>>, [])
    (argon2_elixir) lib/argon2.ex:140: Argon2.add_hash/2
    (exampleapp) lib/exampleapp/schemas/organization/organization_user.ex:41: ExampleApp.Schemas.Organization.OrganizationUser.hash_password/1
    (exampleapp) lib/exampleapp/schemas/organization/organization_user.ex:34: ExampleApp.Schemas.Organization.OrganizationUser.create_changeset/2
    (exampleapp) lib/exampleapp/contexts/organization/organization_user.ex:29: ExampleApp.Contexts.Organization.OrganizationUser.create_public/1
    (exampleapp) lib/exampleapp/contexts/organization/organization_user.ex:15: ExampleApp.Contexts.Organization.OrganizationUser.create/2
    priv/repo/seeds.exs:53: (file)
    (elixir) lib/code.ex:813: Code.require_file/2
    (mix) lib/mix/tasks/run.ex:145: Mix.Tasks.Run.run/5
    (mix) lib/mix/tasks/run.ex:85: Mix.Tasks.Run.run/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/task.ex:365: Mix.Task.run_alias/3
    (mix) lib/mix/task.ex:292: Mix.Task.run/2
    (mix) lib/mix/task.ex:365: Mix.Task.run_alias/3
    (mix) lib/mix/task.ex:292: Mix.Task.run/2

Expected behavior

App should start up and use Argon2 effectively.

Additional information

Dependencies:

defp deps do
    [
      {:absinthe, "~> 1.4"},
      {:absinthe_phoenix, "~> 1.4"},
      {:absinthe_plug, "~> 1.4"},
      {:argon2_elixir, "~> 2.0"},
      {:comeonin, "~> 5.1"},
      {:cors_plug, "~> 2.0"},
      {:credo, "~> 1.1", only: [:dev, :test], runtime: false},
      {:dialyxir, "~> 1.0.0-rc.7", only: [:dev], runtime: false},
      {:ecto_sql, "~> 3.0"},
      {:elixir_uuid, "~> 1.2"},
      {:faker, "~> 0.13.0"},
      {:gettext, "~> 0.11"},
      {:jason, "~> 1.0"},
      {:joken, "~> 2.2"},
      {:mox, "~> 0.5.1", only: [:dev, :test]},
      {:phoenix, "~> 1.4.7"},
      {:phoenix_ecto, "~> 4.0"},
      {:phoenix_pubsub, "~> 1.1"},
      {:plug_cowboy, "~> 2.0"},
      {:postgrex, ">= 0.0.0"},
      {:scrivener_ecto, "~> 2.2"},
      {:mix_test_watch, "~> 1.0.2", only: :dev, runtime: false},
      {:triplex, "~> 1.3"}
    ]
  end

[medikent]

This may be fixed by #40

[riverrun]

One small thing - you don't need to add comeonin to the dependencies anymore, as it is a dependency of argon2_elixir.

I notice that you are using docker. Have you read the deployment section in the comeonin wiki?

[medikent]

Yes, I tried everything there and got the exact same error.

[riverrun]

You could also try running (cd deps/argon2_elixir && make clean && make) to force a rebuild of the argon2 binary - see #9 for more information.

Let me know how you get on.

I am afraid I might not be able to help you much over the next couple of weeks, as I am going away on a retreat later today.

[medikent]

That worked for me. Will the referenced PR #40 solve this?

[riverrun]

I will close this issue now. If you have any other issues, just let me know.

[medikent]

The problem is still there. We are not ready to close this issue. The published code, as it stands, still causes this issue as stated in the current behavior.

[riverrun]

Sorry, I thought you said that forcing the rebuild worked for you.

I closed the pull request because I am almost sure that that is not the answer.

I will try to find out more information about how to set up this with Docker, as I think that is where the problem lies.

[riverrun]

First, please make sure that the _build and deps folders have been added to .dockerignore - as recommended in the wiki and this issue.

If that does not work, try adding RUN cd deps/comeonin && make clean && make to the Dockerfile - as suggested in this issue.

And if that does not work, try running (cd deps/bcrypt_elixir && make clean && make) && mix deps.compile - as suggested here.

Let me know how you get on.

[moxley]

Part of the problem is that the deps/argon2_elixir/priv directory exists as part of the argon2_elixir package.

In the elixir_make API docs, it says:

The "priv" directory must not exist in the source code"

https://hexdocs.pm/elixir_make/Mix.Tasks.Compile.ElixirMake.html#module-compilation-artifacts-and-working-with-priv-directories

Yet, the priv directory exists from the very beginning. The argon2_nif.so file will not build for the local system because elixir_make has determined that it is already built.

It doesn't look like this issue can be fixed with a PR. The priv directory needs to be removed as part of the packaging process. The mix.exs file is where to specify the files to include in the package, but it looks correct.

Workaround

This is what I had to do to get argon2_elixir working in Ubuntu:

mix deps.clean argon2_elixir
mix deps.get
rm -rf deps/argon2_elixir/priv

From there, you can compile the project and run it.

[riverrun]

@moxley thanks for the input. About the priv directory, it is created by the Makefile, so I might need to update the Makefile. I will look into this further and see how it needs to be updated.

[moxley]

@riverrun: Correct. The priv directory is created by make, which is invoked by elixir_make. That's not the issue. The issue is that when you publish the package, you need to ensure that the priv directory is excluded from it.

[riverrun]

@moxley the priv directory is excluded. I just checked it.

[moxley]

@riverrun: Got it. My mistake.

Thinking about this further, I wonder if this is a problem with all packages that need to build a nif.

I develop my project in macOS, then build it on Ubuntu 18.04 running in Docker. When argon2_elixir compiles, one step of the compilation builds a shared object file (argon2_nif.so) and saves it to deps/argon2_elixir/priv/argon2_nif.so. In another step, Mix copies the priv folder to _build/${MIX_ENV}/lib/argon2_elixir/priv. That works fine for local development. But when I build a release in Docker running Ubuntu, a Docker volume pulls everything in from deps/*, including the priv folder, into the running Docker container. elixir_make sees the priv folder already exists, so it skips running make, and Mix copies the priv folder to _build/prod/lib/argon2_elixir/priv. Now the production build has a argon2_nif.so file built for the wrong system.

[riverrun]

For use with Docker, you need to add the _build and deps directories to the .dockerignore file. This is mentioned in the wiki and this issue.

[moxley]

@riverrun: The problem with not including the _build and deps directories is that it deletes any caching created during the release compilation. That makes release build times much longer. The workaround I gave above eliminates only the argon2_elixir cache, and it leaves all the other cache intact.

I might try a different release build setup that involves creating a separate clone of my repo. This repo will only be used for building releases. That way, no special steps need to be take with caching.

[AlphaHydrae]

Hi, I'm having a similar issue in my project when running my test suite:

09:48:20.132 [error] Process #PID<0.670.0> raised an exception
** (ArgumentError) argument error
    :erlang.load_nif(:erlang, :apply)
    (argon2_elixir 2.3.0) lib/argon2/base.ex:120: Argon2.Base.load_nif/0
    (argon2_elixir 2.3.0) lib/argon2/base.ex:14: Argon2.Base.init/0
    (kernel 7.0) code_server.erl:1355: anonymous fn/1 in :code_server.handle_on_load/5
09:48:20.132 [error] Process #PID<0.677.0> raised an exception
** (ArgumentError) argument error
    :erlang.load_nif(:erlang, :apply)
    (argon2_elixir 2.3.0) lib/argon2/base.ex:120: Argon2.Base.load_nif/0
    (argon2_elixir 2.3.0) lib/argon2/base.ex:14: Argon2.Base.init/0
    (kernel 7.0) code_server.erl:1355: anonymous fn/1 in :code_server.handle_on_load/5
09:48:20.144 [warn] The on_load function for module Elixir.Argon2.Base returned:
{:badarg, [{:erlang, :load_nif, [:erlang, :apply], []}, {Argon2.Base, :load_nif, 0, [file: 'lib/argon2/base.ex', line: 120]}, {Argon2.Base, :init, 0, [file: 'lib/argon2/base.ex', ...]}, {:code_server, :"-handle_on_load/5-fun-0-", 1, [...]}]}



  1) test insert a newly created account (MyProject.Projections.UserAccountProjectionTest)
     apps/my_project/test/projections/user_account_projection_test.exs:9
     ** (UndefinedFunctionError) function Argon2.Base.hash_password/3 is undefined (module Argon2.Base is not available)
     code: event = Fixtures.build(:user_account_created)
     stacktrace:
       (argon2_elixir 2.3.0) Argon2.Base.hash_password("letmein-0", <<111, 139, 105, 183, 131, 13, 178, 200, 106, 167, 181, 99, 199, 161, 77, 49>>, [])
       (my_project 0.1.0) test/support/fixtures.ex:22: MyProject.Fixtures.user_account_created_factory/1
       test/projections/user_account_projection_test.exs:10: (test)

It happens regularly, at least 1 in 10 times when running my test suite. Neither deleting _build and deps, nor cd deps/argon2_elixir && make clean && make solves the issue.

I also tried the other solution mentioned after deleting _build and it did not work either:

mix deps.clean argon2_elixir
mix deps.get
rm -rf deps/argon2_elixir/priv

• Dependency:

  "argon2_elixir": {:hex, :argon2_elixir, "2.3.0", "e251bdafd69308e8c1263e111600e6d68bd44f23d2cccbe43fcb1a417a76bc8e", [:make, :mix], [{:comeonin, "~> 5.3", [hex: :comeonin, repo: "hexpm", optional: false]}, {:elixir_make, "~> 0.6", [hex: :elixir_make, repo: "hexpm", optional: false]}], "hexpm", "28ccb63bff213aecec1f7f3dde9648418b031f822499973281d8f494b9d5a3b3"},
  

• Erlang/OTP: 23.0.2

• Elixir: 1.10.3-otp-23

• OS: macOS Mojave 10.14.6

[riverrun]

@AlphaHydrae try asking about this on Stack Overflow or Elixirforum - there you might find users of Mac / Otp 23 who can help you more than I can.

If any changes need to be made to this repository (in the code or documentation), please let me know.

[hl]

@AlphaHydrae I have the same issue and what works for me is running mix deps.compile argon2_elixir --force

[bcat-eu]

I have the same issue as @AlphaHydrae does, I'm on Ubuntu 20.04.

It happens sometimes and only with async tests and across many projects - some use Argon, others Brcypt, but not on all setups - might have something to do with Ryzen CPU that has too many cores, I did not have that problem on the same projects with the old i7 4 core CPU.

At the moment I made all the tests that have to do with Argon or Brcypt synchronous which seems to fix the issue.

[josevalim]

The issue @AlphaHydrae is running into is an issue with Erlang/OTP 23 which has been fixed master. So watch for the next Erlang/OTP 23 release.

[AlphaHydrae]

Thanks @riverrun, @hl & @josevalim.

mix deps.compile argon2_elixir --force did not solve the issue. It just happened again after 5 runs of my test suite.

I will try the new Erlang/OTP release as soon as it's out.

[anthonator]

@josevalim was this fixed in OTP 23.0.3?

[josevalim]

Yes.

[anthonator]

@riverrun looks like this can be closed

[AlphaHydrae]

I have verified that the issue appears to be fixed in my project with OTP 23.0.3. I can no longer reproduce the error.

[bcat-eu]

It was erts bug OTP-16704 here http://erlang.org/download/OTP-23.0.3.README, works now for me as well.

[Caayu]

Part of the problem is that the deps/argon2_elixir/priv directory exists as part of the argon2_elixir package.

In the elixir_make API docs, it says:

The "priv" directory must not exist in the source code"

https://hexdocs.pm/elixir_make/Mix.Tasks.Compile.ElixirMake.html#module-compilation-artifacts-and-working-with-priv-directories

Yet, the priv directory exists from the very beginning. The argon2_nif.so file will not build for the local system because elixir_make has determined that it is already built.

It doesn't look like this issue can be fixed with a PR. The priv directory needs to be removed as part of the packaging process. The mix.exs file is where to specify the files to include in the package, but it looks correct.

Workaround

This is what I had to do to get argon2_elixir working in Ubuntu:

mix deps.clean argon2_elixir
mix deps.get
rm -rf deps/argon2_elixir/priv

From there, you can compile the project and run it.

Worked for me

[kalouo]

Hello. I seem to be having the same issue but the above isn't solving it for me.

I am on Elixir 1.11.3 and Erlang 23.2.3 (both using asdf) and using MacBook pro with an M1 chip.

Erlang/OTP 23 [erts-11.1.7] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]
Elixir 1.11.3 (compiled with Erlang/OTP 23)

I have deleted the _build file and tried to execute the above steps – but it's not working.

I can see it recreating the /priv file on mix compile or mix test.

The error log is as follows:

 ** (UndefinedFunctionError) function Argon2.Base.hash_password/3 is undefined (module Argon2.Base is not available)
...
  (argon2_elixir 2.3.0) lib/argon2/base.ex:19: Argon2.Base.init/0
         (kernel 7.2) code_server.erl:1355: anonymous fn/1 in :code_server.handle_on_load/5
     17:09:07.761 [error] Process #PID<0.20439.0> raised an exception
     ** (RuntimeError) An error occurred when loading Argon2.
     Make sure you have a C compiler and Erlang 20 installed.
     If you are not using Erlang 20, either upgrade to Erlang 20 or
     use bcrypt_elixir (version 0.12) or pbkdf2_elixir.
     See the Comeonin wiki for more information.

Would appreciate any insights.... thanks

[riverrun]

@kalouo try updating to version 2.4. There were some changes to the Makefile, and they might solve your issue.

Let me know how you get on.