User-selectable sqlite library

[aplavin]

There are several reasonably popular sqlite library modifications with certain features on top: e.g., sqlcipher with encryption. They are drop-in compatible with the basic sqlite. It would be useful to let the user choose another library when opening a database in SQLite.jl.

Currently, to use sqlcipher I just forked SQLite.jl, changed the jll library name, and renamed the package to SQLCipher (not planning to register). The only relevant change is: replace using SQLite_jll with using SQLCipher_jll: libsqlcipher as libsqlite.

I imagine something like SQLite.DB("filename", libsqlite=...) should be possible somehow, but not totally sure...

[quinnj]

This is tricky because we have to hard-code dependencies to make them useable and performant. I don't have major issues with making changes to the core sqlite_jll package if it's not disruptive for normal cases and just making some of these features the "default" for everyone. We did that w/ a few features already like thread safety.

The problem w/ passing it as a keyword arg is that there are some dozens, maybe even hundreds of places where you'd have to support and plumb through passing a custom sqlite library. Having your own fork that makes the change is also not unreasonable, honestly. If other's start asking for the same functionality, we could possibly even register; it would just be slightly annoying to keep the SQLCipher up to date w/ changes that happen here.

Another idea is that we just take both binary packages as dependencies in SQLite and then bake the library selection into a global Ref that each ccall would check. We do something slightly similar in ODBC.jl for letting users use different odbc libraries (and taking all possible choices as dependencies). I'd be open to something like that as long as the extra dependencies seem to warrant it.

[aplavin]

we have to hard-code dependencies to make them useable and performant

That's my understanding of the current state as well. Unfortunately, this loses a major feature of dynamic libraries: they can naturally be replaceable by other libraries having the same interface, at runtime.

I don't have major issues with making changes to the core sqlite_jll package if it's not disruptive for normal cases and just making some of these features the "default" for everyone

sqlcipher is a separate library based on a specific sqlite version, not just a compilation option.

[aviks]

I don't have major issues with making changes to the core sqlite_jll package if it's not disruptive for normal cases and just making some of these features the "default" for everyone

Some of the sqlite extensions are proprietary, not open source (Example)-- so this is unfortunately not possible in many cases.

I'd also like this functionality, but yes, I don't see an easy way to achieve this.