sui icon indicating copy to clipboard operation
sui copied to clipboard

[sui-node] Node synced via RocksDB snapshot node missing many dynamic fields of objects

Open shrimpwtf opened this issue 1 year ago • 4 comments

Steps to Reproduce Issue

made a new node and synced it fully via a rocksdb snapshot. its synced to the same state as the public rpc im using for comparison. when I grab object 0xf699e7f2276f5c9a75944b37a0c5b5d9ddfd2471bf6242483b03ab2887d198d0, both say the size is 4066 however when grabbing the dynamic fields of it the public rpc grabs all the objects meanwhile mine only grabs 478 of them.

I've got these settings enabled as below, from the docs explanations this only wipes historical versions of objects and shouldn't affect this.

`enable-event-processing: true enable-index-processing: true

authority-store-pruning-config: num-latest-epoch-dbs-to-retain: 3 epoch-db-pruning-period-secs: 3600 num-epochs-to-retain: 1 max-checkpoints-in-batch: 10 max-transactions-in-batch: 1000 pruning-run-delay-seconds: 60`

Expected Result

all dynamic fields are returned

Actual Result

only 10% of the objects were returned.

System Information

  • OS: macOS 15.1
  • Compiler: rust 1.81.0

shrimpwtf avatar Oct 11 '24 20:10 shrimpwtf

Can you explain what do you mean by size: 0xf699e7f2276f5c9a75944b37a0c5b5d9ddfd2471bf6242483b03ab2887d198d0, both say the size is 4066? You mean the number of dynamic fields that this object has?

stefan-mysten avatar Oct 11 '24 21:10 stefan-mysten

Can you explain what do you mean by size: 0xf699e7f2276f5c9a75944b37a0c5b5d9ddfd2471bf6242483b03ab2887d198d0, both say the size is 4066?

You mean the number of dynamic fields that this object has?

Correct. At the time of testing was roughly around this. I'm trying a snapshot sync with full indexes to see if this helps

shrimpwtf avatar Oct 12 '24 07:10 shrimpwtf

Can you explain what do you mean by size: 0xf699e7f2276f5c9a75944b37a0c5b5d9ddfd2471bf6242483b03ab2887d198d0, both say the size is 4066? You mean the number of dynamic fields that this object has?

So I've just synced the rocksdb snapshot with the indexes (1.3tb of them too), it's a bit better, grabbing 3699 of them however there's currently 4203 in this object so it's still missing a bunch. The node is fully upto sync. This is definitely a big problem. CleanShot 2024-10-12 at 12 41 18@2x

shrimpwtf avatar Oct 12 '24 08:10 shrimpwtf

Public and my rpc seem to give same result for the above now, however other objects still have discrepancies. CleanShot 2024-10-12 at 13 05 51@2x

shrimpwtf avatar Oct 12 '24 09:10 shrimpwtf