exqlite icon indicating copy to clipboard operation
exqlite copied to clipboard

load sqlite-vec extension

Open ndrean opened this issue 1 year ago • 4 comments

In this in the roadmap?

ndrean avatar Aug 30 '24 12:08 ndrean

You are welcome to compile and load extensions. https://github.com/elixir-sqlite/exqlite?tab=readme-ov-file#using-sqlite3-native-extensions

warmwaffles avatar Aug 30 '24 13:08 warmwaffles

Ah, yes...

  1. download the "install.sh" file https://github.com/asg017/sqlite-vec/releases/tag/v0.1.1
  2. chmod+x it and run it: you get a "vec0.dylib" file
  3. Test it following https://github.com/asg017/sqlite-vec#sample-usage)
iex>  Mix.install([{:exqlite, "~> 0.23"}])
iex> ...
iex> {:ok, _, _, _} = Exqlite.Basic.load_extension(conn, "./vec0")
iex> {:ok, _, result} = Basic.exec(conn, "create virtual table vec_examples using vec0( sample_embedding float[8]);")

ndrean avatar Aug 30 '24 13:08 ndrean

Obviously, if you are deploying to a linux machine, you'll need to compile it to a .so and include it. You can take a look at the SQLean hex package and it should give you a jumping off point for packaging the extension and automating that compilation step.

warmwaffles avatar Aug 30 '24 13:08 warmwaffles

Unfortunately, "sqlite-vec" is not part of the "main set". I guess I will have to install it manually in the Docker image.

<https://antonz.org/install-sqlite-extension/>
An extension in SQLite is a single file:

*.dll for Windows (e.g. stats.dll)
*.so for Linux (e.g. stats.so)
*.dylib for macOS (e.g. stats.dylib)

ndrean avatar Aug 30 '24 14:08 ndrean

I'm contemplating building the extensions with this library and including them. But for now, I'm going to leave it alone.

warmwaffles avatar Oct 08 '24 14:10 warmwaffles