I was looking at https://github.com/losfair/mvsqlite and it seems like it's a drop in replacement for sqlite.

I wasn't sure how to do the integration to elixir properly and it feels like I should fork exqlite, but I didn't want to separate efforts.

It could be done. But I highly suggest using rustler. You'll definitely need to make a new package unfortunately.

I don't have enough rust experience to give guidance. But I can say, I think if it is a true drop in replacement, all you'll need to do is implement the DbConnection interface. Once that's done, we could make the backend configurable in ecto_sqlite3.

I might be able to find time to do an implementation.

Plan:

1. Wrap mvsqlite with Rustler
2. Implement DbConnection
3. Publish a new package named something like exmvqlite
4. make the backend configurable in ecto_sqlite3

Notes:

https://blog.plataformatec.com.br/2018/11/building-a-new-mysql-adapter-for-ecto-part-i-hello-world/

For me I think easiest way is to use the binding code they have for dll replacement and bind to c like you have. I also don't know rust and I know c a little better.

Notes

Sqlite3 has c mvsqlite shim that talks to a http port for mvstore writes to a localhost Foundationdb node. Foundationdb talks to other foundationdb nodes.

• Full feature-set from SQLite: mvSQLite integrates with SQLite as a custom VFS layer. It is a layer "below" SQLite, and all of SQLite's features are available.
• Time travel: Checkout the snapshot of your database at any point of time in the past.
• Lock-free, scalable reads and writes: Optimistic fine-grained concurrency with BEGIN CONCURRENT-like semantics. mvSQLite inherits FoundationDB's lock-free property - not a single distributed lock is acquired during data plane operation.
• Get the nice properties from FoundationDB, without its limits: Correctness, really fast and scalable distributed transactions, synchronous and asynchronous replication, integrated backup and restore. Meanwhile, there's no five-second transaction limit any more, and a SQLite transaction can be ~39x larger than FDB's native transaction.

https://github.com/fire/elixir_mvsqlite/tree/mvsqlite began some work. Did not compile yet.

This is interesting

I was able to get a mvsqlite connection. Will start to make the code more robust. I posted the code.

Once that's done, we could make the backend configurable in ecto_sqlite3.

Any idea how I can get to that point?

I was thinking of uploading to hex.pm but concerned about the huge rust dependency.

That's the unfortunate part about rust and rustler. You will have that dependency regardless.

What will need to happen is that this will need to live in its own right, and then we need to go through ecto_sqlite3 in a demo project and make sure that we can indeed flip out the underlying implementation. I tried to make it a possibility to put in a separate driver in case someone came along with a better backend implementation than me.

Do you think I can have a repo here in https://github.com/elixir-sqlite? Or is that a later thing.

Yea, just make it under your account, get it working well and we can transfer it to the org.

It is going to look weird in github with the https://github.com/fire/elixir_mvsqlite repo being a fork of this repo. Perhaps the best thing is to just make a new repo and push the code to it instead.

Any concerns @ConnorRigby with taking in another sqlite adapter into this org?