wallet-cli icon indicating copy to clipboard operation
wallet-cli copied to clipboard

Rocksdb slowing this cli down

Open Daksh14 opened this issue 1 year ago • 5 comments

Flamegraph says: We create column families for all 255 addresses everytime we initiate a cache database. That's not very efficient.

Second reason is that rocksdb works best when there's few column families only.

Currently we have 255. According to benchmark in the issue, that should take close to a minute to be done. That is super slow.

Find a way to reduce column families all together is one way to solve this problem.

We insert all 255 on every db init, a user might not use that many, we can only insert the cf(s) that are being used

Daksh14 avatar Sep 28 '23 05:09 Daksh14

Delete cache in /.dusk/rusk-wallet

And then run

CARGO_PROFILE_RELEASE_DEBUG=true sudo cargo flamegraph --bin rusk-wallet -- -n <network-your-on>

flamegraph

Daksh14 avatar Sep 29 '23 18:09 Daksh14

I disagree about this being a bug.

Furthermore the creation of the column families happens only when the cache database file is created, and not every time that it's instantiated

Does flamegraph report the same if you use the wallet with an existing cache? (Just to mimic the everyday usecase)

herr-seppia avatar Sep 30 '23 06:09 herr-seppia

@herr-seppia This is not a bug for sure.

The problem only happens when creating/deleting cf(s) which happens only when initiating the cache. If a user decides to change networks on the wallet which includes everyday use case, it will take quite a long time to do so.

Edit: This is wrong, we can just add a warning on first init that this process is gonna take a while

Daksh14 avatar Sep 30 '23 17:09 Daksh14

Edit: This is wrong, we can just add a warning on first init that this process is gonna take a while

@Daksh14 Do you believe that this hint would be enough to tackle this issue?

herr-seppia avatar Oct 23 '23 14:10 herr-seppia

Edit: This is wrong, we can just add a warning on first init that this process is gonna take a while

@Daksh14 Do you believe that this hint would be enough to tackle this issue?

Yes because after that last heights are updated and we fetch from there. So it won't be as overhead heavy as the first sync.

Daksh14 avatar Oct 23 '23 14:10 Daksh14