cosmos-sdk icon indicating copy to clipboard operation
cosmos-sdk copied to clipboard

replace tm-db with cosmos-db

Open tac0turtle opened this issue 2 years ago • 8 comments

Summary

Cosmos-sdk has a different set of needs for databases than tendermint, this led to forking of tm-db, https://github.com/cosmos/cosmos-db.

Proposal

Replace the usage of config.toml database preference to influence databases in the sdk. Import and use cosmos-db and set the db preference in app.toml

tac0turtle avatar Aug 15 '22 13:08 tac0turtle

sure, but we should probably take the fork relationship off of the repository so that pull requests and ci and searching the code work correctly.

faddat avatar Aug 16 '22 09:08 faddat

Ok, so:

to complete this, we'd need to get these merged:

  • https://github.com/cosmos/iavl/pull/538
  • https://github.com/cosmos/iavl/pull/539

And these are nice to haves:

  • https://github.com/cosmos/cosmos-db/pull/42
  • https://github.com/cosmos/cosmos-db/pull/43

Right now, the main blockers are the iavl PR's. After that I could work on the SDK items and app.toml config integration.

faddat avatar Aug 24 '22 07:08 faddat

I've reviewed and approved the latter two nice-to-haves

alexanderbez avatar Aug 24 '22 19:08 alexanderbez

The biggest pain point here will be changing dbs if a user is already using badger or boltdb. Should we have a migration?

tac0turtle avatar Aug 27 '22 02:08 tac0turtle

So, we've done a number of user surveys, and basically there is no evidence that anyone is using either badger or bolt.

Badger could not even keep up with the tip of most chains

Bolt often simply did not work.

How about this as a solution:

I ask in the cosmos-sdk discord (✅ )

I tweet the question (✅ )

https://twitter.com/gadikian/status/1564934559212244992

SDK account / others on the team retweet the question

I don't know of another good way to survey users, but there was also an issue on cosmos-db and on tm-db -- thus far we've discovered zero users.

@marbar3778 -- if I were using one of those other DB's I'd just state sync my node with a new one. I guess for archive state, that could be another matter?

faddat avatar Aug 31 '22 10:08 faddat

We really need more benchmarks. Thanks for diving into that @faddat

robert-zaremba avatar Sep 02 '22 10:09 robert-zaremba

So we only support LSM tree DB now, any interest to support a btree DB, like lmdb/mdbx.

yihuang avatar Sep 16 '22 03:09 yihuang

Id be very open, but changing dbs won't give us much more performance without fixing iavl key format and how we write to disk .

tac0turtle avatar Sep 16 '22 07:09 tac0turtle

thanks @faddat for completing this

tac0turtle avatar Dec 29 '22 10:12 tac0turtle