bdk
bdk copied to clipboard
bitcoindevkit
Wasm compatible Database for bdk
Hi, i find that i need to have a database on wasm and i think i'm not the first nor the last. I'm currently working in the browser and it's all great with bdk but it's very cumbersome for the (future) users to wait for the synchronization every time. So i think in two options:
- To make a version of MemoryDatabase in which we save the serialized structures on LocalStorage
- Implement the database tratis for IndexedDB. web_sys has support for IndexedDB so i think it can be a good possibility. I really need this feature and I will have time for work on it, so what do you think is the best path and the requirements and niceties for this?
I haven't tried implementing it yet, but I think the main issue is gonna be the fact that basically all JS APIs are async. And it's not just the fact that they are async, which can be worked around by waiting on the future: some APIs use callbacks to return the data, which probably makes it even trickier to implement.
If I had to pick one method I would go for the one that has the easiest API to integrate: ideally something synchronous, or slightly worse something that uses async so that you can wait on the futures.
Maybe before trying to write my own DB I would try to look for alternatives around, see if somebody has already made some kind of wrapper for the raw browser APIs and try to use those.
Some recommended user friendly DB libraries built on IndexedDB are mentioned here: https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API#see_also
I can see this https://www.npmjs.com/package/idb It uses async/await, is simple and is reasonably maintained.
Another possibility is use localStorage through gloo-storage. We can save and retrieve any serializable object and it's in rust yet. Do you think it can be a good idea?
Interfacing with JS library is not that easy, so I would just use gloo-storage instead if you need to get something done quickly.
Do you think it would be something interesting for the project, then? To have a database based on gloo_storage.
I don't have much experience with rust wasm and haven't used any rust wasm storage crates but it looks like gloo_storage has a straight forward key/value API which should be all you need to implement the Database trait. And yes I think it sounds like a good thing to try for your first contribution.
I implemented this using gloo_storage here.
This however may not be entirely suitable for production we have found. Any read/write to the database and halt the thread and cause your browser to be slow. If we had an asynchronous database it would much better because we could use indexeddb.
Thank for upstreaming your wasm storage work with #789 ! I hate to pin all our hopes on it but decoupling blockchain updates and database updates so they can run async I believe is one of the goal of the bdk_core work that's underway.
yeah I think that would be exactly good for this situation.
in the meantime going to try and use the memory db + a method for "syncing" the memory db to indexed db asynchronously
