Switch to Node's filesystem for libsql-wasm

[penberg]

Suggested by Dominic Elm.

[d3lm]

Hey 👋 Thanks for creating this ticket. The reason I suggested to use the Node.js file system is because the db file would be persisted across reloads as it could be written to the host file system (the host here is WebContainer). AFAIK libsql was compiled via Emscripten so instead of using an in-memory file system there could be a version that uses Node's raw file system via NODERAWFS. This means nothing has to be mounted and it would directly use the Node FS.

[d3lm]

Or potentially, if this is meant to be used via Node.js always then maybe we can make it the default and if we wanted a browser version then we could publish a separate *-web package maybe.

[penberg]

I looked into this with @d3lm and we likely want a VFS implementation that uses Node filesystem API that we can plug into the SQLite/libSQL Wasm build.

[penberg]

Here's one implementation of the VFS approach for upstream SQLite: https://github.com/tndrle/node-sqlite3-wasm

[rphlmr]

I am working on "Drizzle play" that relies on WebContainer. It would definitely help to have this feature!

[d3lm]

Reposted on Twitter https://x.com/elmd_/status/1816532919474680151

[psarna]

Here's one implementation of the VFS approach for upstream SQLite: https://github.com/tndrle/node-sqlite3-wasm

With https://github.com/tursodatabase/libsql/pull/695 in place, isn't it as simple as integrating the VFS above into libsql and compiling it in? libsql.wasm from #695 is intended to run on servers, not in the browser, so it's also where hooks for nodejs filesystem integration should land