Rustler recompiles although nothing changed

[VictorGaiva]

I have an Umbrella App setup where rustler is being used in a ´Core.QrCode` module.

This module has no external compile time dependency except Rustler

defmodule Core.QrCode do
  use Rustler,
    otp_app: :core,
    crate: :qr_code

  defmodule QrCodeOptions do
    @type t :: %__MODULE__{
            content: String.t(),
            # ...args
          }

    defstruct [
      :content,
      # ...args
    ]

    def new(content, rest \\ %{}) do
      struct(__MODULE__, Map.put(rest, :content, content))
    end
  end

  def generate_svg(_arg1), do: :erlang.nif_error(:nif_not_loaded)
end

So when we change almost anything, modules that are not related Core.QrCode, it recompiles causing many other parts of the application to recompile. Since we are heavily using it in another umbrella app, it trigger the recompilation of 80+ files, even if nothing changed.

Is there any way to prevent it from being recompiled when nothing changed?

[filmor]

• Which version of rustler_mix are you using?
• What is the output of MIX_DEBUG=1 mix compile when this happens?

[VictorGaiva]

1 - {:hex, :rustler, "0.34.0", "e9a73ee419fc296a10e49b415a2eb87a88c9217aa0275ec9f383d37eed290c1c" 2 -

Running MIX_DEBUG=1 mix compile inside 'apps/gateway' umbrella app, to prevent 'reports' from recompiling:

-> Running mix loadconfig (inside Gateway.Mixfile)
<- Ran mix loadconfig in 16ms
-> Running mix compile (inside Gateway.Mixfile)
-> Running mix loadpaths --no-deps-loading (inside Gateway.Mixfile)
-> Running mix deps.loadpaths --no-deps-loading (inside Gateway.Mixfile)
-> Running mix archive.check --no-deps-loading (inside Gateway.Mixfile)
<- Ran mix archive.check in 0ms
-> Running mix deps.precompile (inside Gateway.Mixfile)
<- Ran mix deps.precompile in 0ms
==> caching
-> Running mix compile --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
-> Running mix loadpaths --no-deps-loading --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
<- Ran mix loadpaths in 0ms
-> Running mix compile.all --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
-> Running mix compile.yecc --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
<- Ran mix compile.yecc in 2ms
-> Running mix compile.leex --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
<- Ran mix compile.leex in 0ms
-> Running mix compile.erlang --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
<- Ran mix compile.erlang in 0ms
-> Running mix compile.elixir --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
<- Ran mix compile.elixir in 24ms
-> Running mix compile.app --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
<- Ran mix compile.app in 0ms
<- Ran mix compile.all in 40ms
<- Ran mix compile in 45ms
==> tasks
-> Running mix compile --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
-> Running mix loadpaths --no-deps-loading --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
<- Ran mix loadpaths in 0ms
-> Running mix compile.all --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
-> Running mix compile.yecc --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
<- Ran mix compile.yecc in 0ms
-> Running mix compile.leex --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
<- Ran mix compile.leex in 0ms
-> Running mix compile.erlang --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
<- Ran mix compile.erlang in 0ms
-> Running mix compile.elixir --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
<- Ran mix compile.elixir in 1ms
-> Running mix compile.app --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
<- Ran mix compile.app in 0ms
<- Ran mix compile.all in 5ms
<- Ran mix compile in 5ms
==> monitoring
-> Running mix compile --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
-> Running mix loadpaths --no-deps-loading --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
<- Ran mix loadpaths in 0ms
-> Running mix compile.all --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
-> Running mix compile.yecc --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
<- Ran mix compile.yecc in 0ms
-> Running mix compile.leex --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
<- Ran mix compile.leex in 0ms
-> Running mix compile.erlang --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
<- Ran mix compile.erlang in 0ms
-> Running mix compile.elixir --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
<- Ran mix compile.elixir in 1ms
-> Running mix compile.app --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
<- Ran mix compile.app in 0ms
<- Ran mix compile.all in 4ms
<- Ran mix compile in 4ms
==> analytics
-> Running mix compile --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
-> Running mix loadpaths --no-deps-loading --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
<- Ran mix loadpaths in 0ms
-> Running mix compile.all --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
-> Running mix compile.yecc --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
<- Ran mix compile.yecc in 0ms
-> Running mix compile.leex --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
<- Ran mix compile.leex in 0ms
-> Running mix compile.erlang --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
<- Ran mix compile.erlang in 0ms
-> Running mix compile.elixir --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
<- Ran mix compile.elixir in 38ms
-> Running mix compile.app --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
<- Ran mix compile.app in 0ms
<- Ran mix compile.all in 42ms
<- Ran mix compile in 42ms
==> core
-> Running mix compile --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
-> Running mix loadpaths --no-deps-loading --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
<- Ran mix loadpaths in 0ms
-> Running mix compile.all --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
-> Running mix compile.yecc --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
<- Ran mix compile.yecc in 0ms
-> Running mix compile.leex --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
<- Ran mix compile.leex in 0ms
-> Running mix compile.erlang --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
<- Ran mix compile.erlang in 0ms
-> Running mix compile.elixir --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
Compiling 1 file (.ex)
Compiling crate qr_code in release mode (native/qr_code)
    Finished `release` profile [optimized] target(s) in 0.01s
<- Ran mix compile.elixir in 535ms
-> Running mix compile.app --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
<- Ran mix compile.app in 12ms
<- Ran mix compile.all in 561ms
<- Ran mix compile in 561ms
==> gateway
<- Ran mix deps.loadpaths in 1053ms
<- Ran mix loadpaths in 1055ms
-> Running mix compile.all (inside Gateway.Mixfile)
-> Running mix compile.yecc (inside Gateway.Mixfile)
<- Ran mix compile.yecc in 0ms
-> Running mix compile.leex (inside Gateway.Mixfile)
<- Ran mix compile.leex in 0ms
-> Running mix compile.erlang (inside Gateway.Mixfile)
<- Ran mix compile.erlang in 0ms
-> Running mix compile.elixir (inside Gateway.Mixfile)
<- Ran mix compile.elixir in 81ms
-> Running mix compile.app (inside Gateway.Mixfile)
<- Ran mix compile.app in 5ms
-> Running mix compile.surface (inside Gateway.Mixfile)
<- Ran mix compile.surface in 616ms
<- Ran mix compile.all in 708ms
-> Running mix compile.protocols (inside Gateway.Mixfile)
<- Ran mix compile.protocols in 16ms
<- Ran mix compile in 1813ms

[filmor]

Could you test it with rustler_mix 0.33? The last change that touched the recompilation was https://github.com/rusterlium/rustler/commit/7fef3c52873e31e863e966b4677cc1f58c57d699.

[VictorGaiva]

I'll be running some tests after downgrading. Will be posting again when possible, thanks for the attention

[VictorGaiva]

So far it seems to have worked. I've spent this morning changing code somewhere that would usually cause this issue. I'll be downgrading the version in my main branch, and keep an eye if the issue arises again.

Also an important note is that this issue happened to me and another collegue, while others were not facing it. In my case it happened on two different computers.

I can provide more information about each enviroment if necessary.

[filmor]

Hmm, I may have been using @external_resource wrong when trying to fix #616. @ggwpez, I think I have to look into this again. I have to check when exactly the file contents are checked.

@VictorGaiva Can you provide a minimal example that reproduces the issue on your machine?

[ggwpez]

I also experienced the frequent recompilation issue, but did not though much of it.
I am using MacOs with a nix flake and a Rust workspace but no mix umbrella project. Maybe that overlaps with @VictorGaiva's setup?

Every now and then when i run mix test it would try to recompile the Rust crates, although nothing changed in the binding module or Rust code.

[VictorGaiva]

I'll be creating a minimal repo in a few days, and posting it here.