reqlite icon indicating copy to clipboard operation
reqlite copied to clipboard

Sync to disk

Open neumino opened this issue 9 years ago • 7 comments

We currently never sync to disk.

We could, it's not too hard with synchronous methods. If someone want to use asynchronous method, this will require a refactoring to support lazily computing streams.

neumino avatar May 20 '15 02:05 neumino

Any progress on this? I do a lot of test projects and RethinkDB is my choice of general DB at the moment but like you I don't like spinning up Vagrant. So if Reqlite would be persistent to disc that would be just pure awesomeness because it would make it feasible to build fully working prototypes with it. I could even live with some limitations to get this.

mjp0 avatar Jun 20 '15 04:06 mjp0

Not really, I'm pretty busy at the moment, feel free to send a pull request :)

Since r.http was added, you can return a promise instead of directly accessing table.documents and asynchronously load the data. Writing on disk is a bit more complicated though.

neumino avatar Jun 20 '15 07:06 neumino

Well, I mean it's easy to just dump everything for every write, but a little more tricky if you want to optimize stuff.

neumino avatar Jun 20 '15 07:06 neumino

It's probably reasonable to just load a dump JSON file from disk for now and not sync to disk. It may make sense to first implement btree before syncing. I just skimmed through SQLite file format, and I have to admit that I underestimate SQLite :) https://www.sqlite.org/fileformat2.html

neumino avatar Jun 20 '15 07:06 neumino

I agree, SQLite is pretty impressive :)

I think looking at the scope of Reqlite it would be completely fine to do Redis style periodic snapshotting so that you can load latest snapshot when you restart the server. Now that I think about it this is actually what I was asking in the first place. Full-blown syncing in my opinion isn't necessary.

edit And I also think that optimization isn't really what users are looking from this project. If they want to get more horsepower, they will just hook up driver to actual RDB instance.

mjp0 avatar Jun 20 '15 07:06 mjp0

Perhaps this can be done in a way that is compatible with Rethinkdb backup.

See also rethinkdb/rethinkdb#3680.

omeid avatar Jun 28 '15 09:06 omeid

I have an interesting use case for reqlite. I'm working on a wiki project that's designed to handle tiny to medium sized wikis, basically tens of thousands of revisions at most. Well, I really like RethinkDB (and it handles it great), but it might be nice to have a simple deploy mode for the < 1000 revisions case. reqlite seemed like a great option for me; I wouldn't need to change practically any code... until I saw this issue.

For now, I'm going to write an adapter to my own light weight db, but I'd like to not have to maintain the adapter code, and just use reqlite instead.

Morgul avatar Jun 01 '17 00:06 Morgul