tendermint-rs icon indicating copy to clipboard operation
tendermint-rs copied to clipboard

Specify tendermint versions for integration tests

Open greg-szabo opened this issue 4 years ago • 0 comments

I have a suggestion on how to further improve the integration test situation:

Currently, we always run against the lates (go) tendermint docker image. On one hand this is really cool because we get early warnings for things that will break compatibility with the next release. On the other hand, we can't guarantee compatibility against a current release (e.g. we only want compatibility with 0.33.X or even only with 0.33.4 because that's what the hub is using or goz or whatever reasons there might be to want compatibility to a particular version).

I think tests would be even more useful if:

  1. they were run against a fixed and chosen tendermint version and were made mandatory (required on gh) for this version
  2. additionally integration tests for the latest image run regularly too (but much less frequently; e.g. nightly/weekly)

Regarding 1): we currently do not merge PRs unless integration tests pass anyways. The tools could better reflect this already established process and only allow merging if intgration tests pass too. Also, sometimes integration tests pass on the branch but fail on master (#249 #233 and many more). This happens because there was a new docker image released between last check on branch and merge to master. This would be mitigated if we made sure, we commit to compatibility of one tendermint version.

And 2) would help us to be learn about upcoming changes on a regular basis.

Originally posted by @liamsi in https://github.com/informalsystems/tendermint-rs/issues/120#issuecomment-634541018

greg-szabo avatar Aug 04 '20 14:08 greg-szabo