Trying to get exqlite to work on Darwin (x86_64)

[clarkcb]

Hi, I'm trying to add sqlite3 support to an Elixir CLI tool I built recently, and exqlite seems like a good match, but I'm unable to get it working. The main error seems to be this:

** (UndefinedFunctionError) function Exqlite.Sqlite3NIF.open/2 is undefined (module Exqlite.Sqlite3NIF is not available)

I was able to determine that the NIF part apparently failed to compile, and after some tinkering I was seemingly able to get it to compile. I modified the Makefile to add this for Darwin (added the LIBNAME line to change extension to dylib):

ifeq ($(KERNEL_NAME), Darwin)
	CFLAGS += -fPIC
	LDFLAGS += -dynamiclib -undefined dynamic_lookup
	LIB_NAME = $(PREFIX)/sqlite3_nif.dylib
endif

I also exported some ENV vars (both erlang and elixir are installed via homebrew):

MIX_APP_PATH=/path/to/deps/exqlite
ERL_EI_INCLUDE_DIR=/usr/local/Cellar/erlang/26.2.5/lib/erlang/erts-14.2.5/include
ERTS_INCLUDE_DIR=/usr/local/Cellar/erlang/26.2.5/lib/erlang/erts-14.2.5/include
V=1

I had to add both of the include dirs to get the compilation to work. Not sure where ERTS_INCLUDE_DIR is supposed to get defined normally, but if I don't define it I get this output:

$ make all  
 CC sqlite3_nif.o
cc -c -I"/usr/local/Cellar/erlang/26.2.5/lib/erlang/erts-14.2.5/include" -I"" -Ic_src -DNDEBUG=1 -O2 -fPIC -DSQLITE_THREADSAFE=1 -DSQLITE_USE_URI=1 -DSQLITE_LIKE_DOESNT_MATCH_BLOBS=1 -DSQLITE_DQS=0 -DHAVE_USLEEP=1 -DALLOW_COVERING_INDEX_SCAN=1 -DENABLE_FTS3_PARENTHESIS=1 -DENABLE_LOAD_EXTENSION=1 -DENABLE_SOUNDEX=1 -DENABLE_STAT4=1 -DENABLE_UPDATE_DELETE_LIMIT=1 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_FTS5=1 -DSQLITE_ENABLE_GEOPOLY=1 -DSQLITE_ENABLE_MATH_FUNCTIONS=1 -DSQLITE_ENABLE_RBU=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_OMIT_DEPRECATED=1 -DSQLITE_ENABLE_DBSTAT_VTAB=1 -o /path/to/deps/exqlite/obj/sqlite3_nif.o c_src/sqlite3_nif.c
c_src/sqlite3_nif.c:1003:10: error: call to undeclared function 'sqlite3_enable_load_extension'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    rc = sqlite3_enable_load_extension(conn->db, enable_load_extension_value);
         ^
c_src/sqlite3_nif.c:1003:10: note: did you mean 'exqlite_enable_load_extension'?
c_src/sqlite3_nif.c:984:1: note: 'exqlite_enable_load_extension' declared here
exqlite_enable_load_extension(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[])
^
1 error generated.
make: *** [/path/to/deps/exqlite/obj/sqlite3_nif.o] Error 1

With both ERL_EI_INCLUDE_DIR and ERTS_INCLUDE_DIR defined, I get this output:

$ make all                                                                              
 CC sqlite3_nif.o
cc -c -I"/usr/local/Cellar/erlang/26.2.5/lib/erlang/erts-14.2.5/include" -I"/usr/local/Cellar/erlang/26.2.5/lib/erlang/erts-14.2.5/include" -Ic_src -DNDEBUG=1 -O2 -fPIC -DSQLITE_THREADSAFE=1 -DSQLITE_USE_URI=1 -DSQLITE_LIKE_DOESNT_MATCH_BLOBS=1 -DSQLITE_DQS=0 -DHAVE_USLEEP=1 -DALLOW_COVERING_INDEX_SCAN=1 -DENABLE_FTS3_PARENTHESIS=1 -DENABLE_LOAD_EXTENSION=1 -DENABLE_SOUNDEX=1 -DENABLE_STAT4=1 -DENABLE_UPDATE_DELETE_LIMIT=1 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_FTS5=1 -DSQLITE_ENABLE_GEOPOLY=1 -DSQLITE_ENABLE_MATH_FUNCTIONS=1 -DSQLITE_ENABLE_RBU=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_OMIT_DEPRECATED=1 -DSQLITE_ENABLE_DBSTAT_VTAB=1 -o /path/to/deps/exqlite/obj/sqlite3_nif.o c_src/sqlite3_nif.c
 CC sqlite3.o
cc -c -I"/usr/local/Cellar/erlang/26.2.5/lib/erlang/erts-14.2.5/include" -I"/usr/local/Cellar/erlang/26.2.5/lib/erlang/erts-14.2.5/include" -Ic_src -DNDEBUG=1 -O2 -fPIC -DSQLITE_THREADSAFE=1 -DSQLITE_USE_URI=1 -DSQLITE_LIKE_DOESNT_MATCH_BLOBS=1 -DSQLITE_DQS=0 -DHAVE_USLEEP=1 -DALLOW_COVERING_INDEX_SCAN=1 -DENABLE_FTS3_PARENTHESIS=1 -DENABLE_LOAD_EXTENSION=1 -DENABLE_SOUNDEX=1 -DENABLE_STAT4=1 -DENABLE_UPDATE_DELETE_LIMIT=1 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_FTS5=1 -DSQLITE_ENABLE_GEOPOLY=1 -DSQLITE_ENABLE_MATH_FUNCTIONS=1 -DSQLITE_ENABLE_RBU=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_OMIT_DEPRECATED=1 -DSQLITE_ENABLE_DBSTAT_VTAB=1 -o /path/to/deps/exqlite/obj/sqlite3.o c_src/sqlite3.c
 LD sqlite3_nif.dylib
cc -o /path/to/deps/exqlite/priv/sqlite3_nif.dylib /path/to/deps/exqlite/obj/sqlite3_nif.o /path/to/deps/exqlite/obj/sqlite3.o -dynamiclib -undefined dynamic_lookup

I see the compiled files under obj and priv, so I assume the build is successful. Next I ran mix deps.compile exqlite from the project root and see no error output, and I run mix escript.build to build the tool. But I'm still getting the same error when I try to run it:

$ ./bin/exfind

15:50:48.782 [warning] The on_load function for module Elixir.Exqlite.Sqlite3NIF returned:
{:function_clause,
 [
   {:filename, :join, [{:error, :bad_name}, ~c"sqlite3_nif"],
    [file: ~c"filename.erl", line: 454]},
   {Exqlite.Sqlite3NIF, :load_nif, 0,
    [file: ~c"lib/exqlite/sqlite3_nif.ex", line: 16]},
   {:code_server, :"-handle_on_load/5-fun-0-", 1,
    [file: ~c"code_server.erl", ...]}
 ]}

15:50:48.782 [error] Process #PID<0.114.0> raised an exception
** (FunctionClauseError) no function clause matching in :filename.join/2
    (stdlib 5.2.3) filename.erl:454: :filename.join({:error, :bad_name}, ~c"sqlite3_nif")
    (exqlite 0.23.0) lib/exqlite/sqlite3_nif.ex:16: Exqlite.Sqlite3NIF.load_nif/0
    (kernel 9.2.4) code_server.erl:1393: anonymous fn/1 in :code_server.handle_on_load/5

15:50:48.818 [error] Process #PID<0.116.0> raised an exception
** (FunctionClauseError) no function clause matching in :filename.join/2
    (stdlib 5.2.3) filename.erl:454: :filename.join({:error, :bad_name}, ~c"sqlite3_nif")
    (exqlite 0.23.0) lib/exqlite/sqlite3_nif.ex:16: Exqlite.Sqlite3NIF.load_nif/0
    (kernel 9.2.4) code_server.erl:1393: anonymous fn/1 in :code_server.handle_on_load/5

15:50:48.819 [warning] The on_load function for module Elixir.Exqlite.Sqlite3NIF returned:
{:function_clause,
 [
   {:filename, :join, [{:error, :bad_name}, ~c"sqlite3_nif"],
    [file: ~c"filename.erl", line: 454]},
   {Exqlite.Sqlite3NIF, :load_nif, 0,
    [file: ~c"lib/exqlite/sqlite3_nif.ex", line: 16]},
   {:code_server, :"-handle_on_load/5-fun-0-", 1,
    [file: ~c"code_server.erl", ...]}
 ]}

** (UndefinedFunctionError) function Exqlite.Sqlite3NIF.open/2 is undefined (module Exqlite.Sqlite3NIF is not available)
    (exqlite 0.23.0) Exqlite.Sqlite3NIF.open(~c"/path/to/sqlite.db", 6)
    (exfind 0.1.0) lib/filetypes.ex:59: ExFind.FileTypes.new/0
    (exfind 0.1.0) lib/exfind.ex:56: ExFind.Main.find/2
    (exfind 0.1.0) lib/exfind.ex:69: ExFind.Main.main/1
    (elixir 1.17.2) lib/kernel/cli.ex:136: anonymous fn/3 in Kernel.CLI.exec_fun/2

I tried taking a look at sqlite3_nif.ex to see if anything obvious jumped out, but nothing caught my eye. Unfortunately, I'm somewhat new to elixir and very new to NIF, so I'm not sure where to go from here. Any guidance you might have would be appreciated. Thanks!