Usage docs for dynamic libraries

[crystalthoughts]

Hi - I'm trying to follow your example but using a dll. I run into a wall both statically linking and calling a dll library for CozoDB. https://github.com/cozodb/cozo/blob/main/cozo-lib-c/cozo_c.h The api is very small so would be a good test case for learning futhark. Passing a linker pragma seems to hang for a long time then posts corrupt .drectve errors. Trying to using loadLib() causes unknown function errors though I might be using it wrong:

import futhark
import strutils
import std/dynlib

# Tell futhark where to find the C libraries you will compile with, and what
# header files you wish to import.
importc:
  path "./lib/c"
  "cozo_c.h"

# {.passL: "-Llib/dyn -llibcozo_c".}

proc query(db_id: int32, query: cstring) =
    let empty_params = "{}"
    var res: cstring
    res = cozo_run_query(db_id, query, empty_params, false)
    echo $res
    cozo_free_str(res)

proc main() =
    let lib = loadLib("lib/dyn/llibcozo_c.dll")
    assert lib != nil, "Error loading library"

    var db_id: int32
    var err: cstring
    err = cozo_open_db("mem", "", "{}", addr db_id)

    if err != nil :
        echo $err
        cozo_free_str(err)
        return

    query(db_id, "?[] <- [[1, 2, 3]]")

    echo cozo_close_db(db_id)

main()

Do you see where I could be going wrong? Including some usage docs in the readme (basically a mini tutorial for each scenario - header only, source/header, static lib, dynamic library) would be very useful!

Thanks

[PMunch]

You're very close, the only thing missing is a tiny bit of C linking experience. I dropped the .h file and the .so (Linux's version of a .dll) in the same folder as this Nim file:

import futhark
import strutils
# Tell futhark where to find the C libraries you will compile with, and what
# header files you wish to import.
importc:
  path "."
  "cozo_c.h"

{.passL: "-L. -lcozo_c".}

proc query(db_id: int32, query: cstring) =
  let empty_params = "{}"
  var res: cstring
  res = cozo_run_query(db_id, query, empty_params, false)
  echo $res
  cozo_free_str(res)

proc main() =
  var db_id: int32
  var err: cstring
  err = cozo_open_db("mem", "", "{}", addr db_id)

  if err != nil :
    echo $err
    cozo_free_str(err)
    return

  query(db_id, "?[] <- [[1, 2, 3]]")

  echo cozo_close_db(db_id)

main()

Then it's a simple nim c <ourfile>.nim to build the binary. Now this is dynamically linked to libcozo_c.so, but running it will throw an error (on Linux, on Windows you're fine since the library is in the same folder). We need to tell it to look for the dynamic library in the local folder and not just in the system folders: LD_LIBRARY_PATH=. ./<ourfile>. And there you go, a dynamically linked library which runs fine.

[PMunch]

This could definitely be documented better though, so I'm keeping this open

[crystalthoughts]

Thanks for going over this! I got it working on Windows as you described, but only by keeping the files in root. Is LD_LIBRARY_PATH an environment variable? And is that a cross platform approach?

Is there a way I can explicitly pass a flag to the compiler to search lib/dyn, instead?

I tried --cincludes:./lib/dyn but no luck

[PMunch]

LD_LIBRARY_PATH is a Linux and Mac (or rather Unix) thing, so I don't think it works on Windows. But yes, it is an environment variable.

You can tell the compiler to look for the dynamic library elsewhere by changing what you pass to -L in the passL block. This is basically the linker search path. So {.passL: "-L./lib/dyn -lcozo_c".} would make it search in ./lib/dyn. However this only applies to the linker, the thing which decides what to call and from where. So when a user is running your program you still need their system to find the library. Of course you could do this by manually loading it with the dynlib module, but then you'd have to cast pointers to the procedures like what is shown in that example. Another option would be to somehow tell Windows where to find them, but I think that requires modifying the path. Or you could just drop them next to your .exe file and the system will find them.