CometMock icon indicating copy to clipboard operation
CometMock copied to clipboard

Add removal/startup of nodes during runtime

Open p-offtermatt opened this issue 2 years ago • 0 comments

Is your feature request related to a problem? Please describe. When an application crashes or is disconnected, CometMock should not crash. It should also allow disconnected apps to reconnect, and new apps to connect during runtime Optimally, validators that belong to apps which are currently disconnected have their signing_status set to false, i.e. those validators do not sign. Similarly, CometMock should detect if two applications with the same private key are connected and produce double-sign evidence if that happens.

This feature allows test runs that kill and restart applications to run smoothly. It also seamlessly allows downtime the way tests already do it (although CometMock has its own native way to cause downtime as well).

Proposed solution

  • [ ] Calls to the ABCI server are made with a timeout
  • [ ] Create a new type that encapsulates the client and associates it to a validator
  • [ ] When an app times out, the validator associated with that app is set to not sign
  • [ ] Periodically retry apps
  • [ ] When an app reconnects, start signing with its validator again

Additional context The logic was started to be implemented in this branch https://github.com/p-offtermatt/CometMock/compare/main...ph/dockerfile, but there is an issue with the way the CometBFT client calls the ABCI server, see this Slack messahe or the code here - the problem is that the context is hardcoded to background.

The solution is to duplicate that code from CometBFT and weave the contexts through.

Sync logic

This issue needs to add in particular support for syncing apps that do not have the current height.

  • [ ] Allow removal of nodes during runtime
  • [ ] #70

p-offtermatt avatar Jun 16 '23 08:06 p-offtermatt