Elixir external functions don't work when no Elixir modules are present

[chouzar]

When generating a new Gleam project:

gleam new exffi

And adding an FFI function to Elixir:

import gleam/io

pub fn main() {
  io.println("Hello from " <> cwd())
}

external fn cwd() -> String = 
  "Elixir.File" "cwd!"

Calling the function results in:

exffi:main().
% ** exception error: undefined function 'Elixir.File':'cwd!'/0
%      in function  exffi:main/0 (build/dev/erlang/exffi/_gleam_artefacts/exffi.erl, line 8)

Elixir files are not being generated in the build directory until there's an elixir dependency:

Adding a .ex file to the project generates the files and solves this issue.

[lpil]

That's an interesting one. How should we detect that we need to invoke the Elixir compiler in this situation?

[inoas]

That's an interesting one. How should we detect that we need to invoke the Elixir compiler in this situation?

As all Elixir (stdlib) modules are prefixed with Elixir (I think I remember that there is a way to define a module without having an Elixir prefix in Elixir, but it is an uncommon hack), thus when we see this:

external fn foo() -> bar = 
  "Elixir.quux" "batz"

... the "Elixir. part could be the trigger to run the elixir compiler? Unless I missunderstood.

[chouzar]

Probably not the most elegant solution, but maybe giving the external function an indicator that the function comes from a particular language/platform.

external elixir fn cwd() -> String = 
  "Elixir.File" "cwd!"

With current gleam I'm wondering if something like this would work:

if elixir {
  external fn cwd() -> String = 
  "Elixir.File" "cwd!"
}

But I understand the above is more about having multiple ffi options not necessarily about being specific..

[lpil]

We can't modify the language for this as the language doesn't know anything about the environment in which it is compiled in, that is the responsibility of whatever build tool is being used.

We could possible look for Elixir. prefixes. That would normally work but I'm unsure if it's the best approach.

[inoas]

Is it about making the elixir stdlib available, yes? gleam.toml entry?

[lpil]

That would also work, aye. We need to make a decision

[chouzar]

A toml entry sounds very sensible, would that live under dependencies or maybe some other field?

[lpil]

Under erlang I think.

[inoas]

force_compile_elixir_stdlib=True or so something more eloquent?

Under erlang sounds good!

If toml has maps maybe sonething like elixir.force_compile_stdlib

always_..
force_..
.._add_..
.._compile_..

… maybe

[levex]

Separately, would it make sense to improve the build tool with a heuristic in this case?

"It seems you're trying to call an external Elixir module, please see docs/force_elixir_stdlib"

[lpil]

That would be good, but I'm not sure when we could emit that. We only know for sure that a native module is not defined at runtime, and we have no control over errors at runtime.

Emitting it during compilation could be incorrect as the module may exist. I would only want to emit warnings etc when we know there certainly is a problem

[erikareads]

Matching on Elixir. matches on most cases, but not all. In the unlikely scenario where someone wants to use something from the Elixir compiler and bootstrap modules:

import gleam/io

@external(erlang, "elixir_utils", "split_last")
fn inspect(value: List(a)) -> #(List(a), a)

pub fn main() {
  inspect([1,2,3]) |> io.debug() // #([1,2], 3)
}

This function call fails at runtime without a an empty .ex file, since it's a module that only exists in Elixir.

Not even a rule with Elixir. and elixir_ would work, since -module(iex). is in Elixir's source. (To my knowledge iex is the only exception to the more general rule, but it doesn't preclude the addition of others).

Alternatively, we can consider that the elixir_ and iex modules are internal, undocumented modules of Elixir's source code, and therefore not worth supporting explicitly.

For situations where the Elixir standard lib is interested in being used, I'm in favor of an empty dependency or TOML flag to enable it.

[lpil]

Good points, thank you @erikareads . A TOML flag sounds sensible to me. What are some possible names?

[erikareads]

I'm personally a fan of enums over booleans. But if booleans are preferred, consider adding enable_ to any of the following names.

Possible names:

force_elixir_runtime="enabled"
force_elixir_compilation="enabled"
force_elixir_stdlib="enabled"
elixir_runtime="enabled"
load_elixir_modules="enabled"
force_load_elixir_modules="enabled"

[orsinium]

I also stuck with it for quite a while. If not for this issue, I'd never figure out what's the problem.

The code:

import gleam/io

type Source {
  Stdio
  All
}

@external(erlang, "Elixir.IO", "read")
fn io_read(a: Source, b: Source) -> String

pub fn main() {
  let lines = io_read(Stdio, All)
  io.println(lines)
}

The error:

exception error: undefined function 'Elixir.IO':read/2

I think if a proper solution requires a long discussion and careful and difficult implementation, in meanwhile a bit more helpful error message would be great. After all, "clear error messages" is one of the main selling points of Gleam. Something like this:

exception error: undefined function 'Elixir.IO':read/2. 
If this is an Elixir dependency, try adding an *.ex file in your project to include the Elixir runtime.

[lpil]

The error is thrown by the BEAM, we have no control over how the virtual machine builds exceptions I'm afraid.