sui icon indicating copy to clipboard operation
sui copied to clipboard

Use synchronous writes to batch store in worker Synchronize

Open aschran opened this issue 2 years ago • 1 comments

Without this, primary may sign certificates promising availability of batches that could actually be lost in case of worker crash or other storage error.

aschran avatar Nov 03 '22 17:11 aschran

If we think it matters for performance, I could actually change this to have primary indicate in the request whether it needs to be sync (batches from a Header) or async is okay (batches from a Certificate). @bmwill do you have a sense of how slow this will be and if it's of practical concern?

aschran avatar Nov 03 '22 17:11 aschran

I think we should probably stop using this db abstraction entirely and just use the sync db reads and writes directly. This abstraction currently serializes all reads and writes to a single thread when the db supports multithreaded reads and writes

bmwill avatar Nov 03 '22 19:11 bmwill

I think we should probably stop using this db abstraction entirely and just use the sync db reads and writes directly. This abstraction currently serializes all reads and writes to a single thread when the db supports multithreaded reads and writes

Makes sense. You mean just for this particular use, or for all of narwhal entirely?

In the meantime it'd be good to get this in for correctness unless you think the performance hit would be unacceptable?

aschran avatar Nov 03 '22 20:11 aschran

No we can do this now, but as a note we just need to change all of narwhal's db usage.

File an issue on me? :)

aschran avatar Nov 04 '22 16:11 aschran