zebra icon indicating copy to clipboard operation
zebra copied to clipboard

Revert deserializing state transactions in `rayon` threads

Open teor2345 opened this issue 1 year ago • 0 comments

Motivation

In PR #4828, we had to increase the lightwalletd full sync timeout to 2.5 hours.

This might be a Zebra bug, it looks like all the blocks are slow, not just the recent large blocks. We could have introduced this bug in PR #4805.

Things to try

Remove the rayon thread from transaction deserialization: https://github.com/ZcashFoundation/zebra/blob/9b2185ad3d7417bf4bb5a570ad81c6a027aab3b7/zebra-state/src/service/finalized_state/disk_format/block.rs#L236-L246

Check if any of the ReadRequest::Block calls are slow enough to turn up in Zebra's logs in the lightwalletd full sync test. If they are fast enough, we're done!

Otherwise, here are a few more things we could try...

If they are slow, try getting all the raw transaction data from the disk, then deserializing it all at once in a single rayon thread: https://github.com/ZcashFoundation/zebra/blob/9b2185ad3d7417bf4bb5a570ad81c6a027aab3b7/zebra-state/src/service/finalized_state/zebra_db/block.rs#L127-L135

Check the logs for slow reads again.

Some other things we could try:

  • just spawn threads to deserialize shielded transactions
    • how do we know before they are deserialized?
    • we know the serialized length, do we just want to do it for all large transactions?
  • skip cryptographic validation when deserializing trusted transactions from the state
  • try multiple RPC threads (see PR #4806 for details)
  • just ignore the speed issues for now

teor2345 avatar Jul 26 '22 22:07 teor2345