snarkOS icon indicating copy to clipboard operation
snarkOS copied to clipboard

Track timing of consensus stages

Open vicsn opened this issue 2 months ago • 3 comments

Motivation

This PR introduces a method to track consensus events for visualization, building on top of @kpandl 's earlier work. My understanding of the image below is that under load, block processing is the bottleneck, not certificate signing.

Filing as a draft first to get feedback on the high level design before cleaning it up. Specifically are we ok:

  1. dumping all of the data into a global lazy static, and only enabling this in dev/test_network mode. Would a pub static also be fine? Or something else?
  2. exporting the data to a JSON file, also only in dev/test_network mode.

This "ugly" approach mirrors the way we track pub static R1CS_HASHES in snarkVM for certain tests only.

Perhaps in the future it could be useful to track this data in production, that would also be a lot more work without getting actually good data because in production we rarely have high TPS at the moment.

Screenshot 2025-10-12 at 21 31 33 Screenshot 2025-10-13 at 20 23 52

vicsn avatar Oct 12 '25 20:10 vicsn

dumping all of the data into a global lazy static, and only enabling this in dev/test_network mode. Would a pub static also be fine? Or something else?

For dev/test purposes it's perfectly fine; it is only in production where it's an anti-pattern, and should instead be incorporated into the workflow. The alternative would be to store it in the related objects, which would also work, though could be a bit trickier to implement.

exporting the data to a JSON file, also only in dev/test_network mode.

As long as we don't expect very long measurements/outputs, this is fine.

Oh, and naturally, this still needs to be feature-gated.

ljedrz avatar Oct 13 '25 08:10 ljedrz

Would it not be better to add a separate feature for this? Or do you think the performance overhead for test networks that do not require timing information is minimal?

kaimast avatar Oct 20 '25 22:10 kaimast

Would it not be better to add a separate feature for this? Or do you think the performance overhead for test networks that do not require timing information is minimal?

Agreed, made a new test_consensus_tracking feature. Specifically for long-running networks, these hacky JSON files might get huge.

vicsn avatar Oct 24 '25 12:10 vicsn