Erlang Build Proposal

[fadushin]

Currently, our code organization isn't tracking the OTP design principles very well, and not having code organized properly can affect separation of concerns, especially if we end up defining dependencies between components.

In addition, we are using hand-crafted CMake rules to build Erlang modules, which, while it functions, does seem to result in unnecessary rebuilds in some cases. It might be better for us to use native tooling (such as rebar3 for Erlang or mix for Elixir) to build these libraries, and we already have the tooling to do that, using the atomvm_rebar3_plugin and ExAtomVM projects.

Note. In some ways, code organization is not critical for our purposes, as we typically assemble all of our BEAM files into a single library (atomvmlib.avm, for example). However, there may be some downstream dependencies on our components, if, for example, we were to deploy any of our applications into Hex.pm. Best for us to be prepared for such an eventuality, if it ever comes to pass.

We propose the following source code organization:

- kernel/
    - application.erl (new)
    - gen_tcp.erl
    - gen_udp.erl
    - erlang.erl
    - inet.erl
    - init.erl (new)
    - logger.erl
    - logger_std_h.erl
    - net.erl
    - socket.erl

- stdlib/
    - binary.erl
    - base64.erl
    - calendar.erl
    - gen_event.erl
    - gen_server.erl
    - gen_statem.erl
    - io.erl
    - io_lib.erl
    - lists.erl
    - maps.erl
    - math.erl
    - prop lists.erl
    - supervisor.erl
eavmlib/
    - (current eavmlib)

Each application should be organized using rebar3 source code organization. Note, however, that due to an unfortunate implementation detail in rebar3, code that is compiled is also loaded, which can make it difficult or impossible to (re-) implement modules that rebar3 itself uses, such as application or gen_server.

To work around this rebar3 issue, the atomvm_rebar3_plugin has implemented a bootstrap task, which allows developers to specify modules that are not compiled by the rebar3 compile task, but are instead compiled separately, but ultimately end up in the large AVM file. This way, we can (re-)implement these kernel and stdlib modules without running into runtime issues with rebar3.