Testnet in ci

[faddat]

Hi,

We've been implementing the globalfee module for eve. Along the way we're hitting the same issues that we're seeing you solve here, and this CI job will surface that kind of thing more easily.

Please treat this PR as the starting point for this job, as we likely want a job that stops after some period of time.

I guess we can look at it as an addition to the e2e testing and liveness checks.

[codecov[bot]]

Codecov Report

Merging #1735 (f79a238) into main (5a8b1b7) will not change coverage. The diff coverage is n/a.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1735   +/-   ##
=======================================
  Coverage   52.33%   52.33%           
=======================================
  Files          16       16           
  Lines        1779     1779           
=======================================
  Hits          931      931           
  Misses        821      821           
  Partials       27       27           

[okwme]

thanks for the PR @faddat very interesting, i've never come across the testnet command actually. it seems like this is doing the exact same thing as the liveness test, is that correct? it would be good to pick one or the other if they are doing the exact same thing. the liveness test is already stopping after a number of blocks are successfully produced, is there more to this PR than what I've described above? just thinking out loud, the benefits of using the testnet command is that it removes an external dependency (starport cli). however the benefit of using starport is that it should act more like an actual validator running real commands, whereas the testnet command may make some assumption within the cosmos sdk that an actual node operator may not, which means it tests a more realistic liveness scenario.

any other features to consider that you can think of @faddat ?

would also be curious to hear from @alexanderbez as he added the current liveness test and has used a similar set up on other repos. Would be curious why you opted for starport cli in the first place instead of the testnet command?

[faddat]

ok, so basically we've got the testnet command in gaia now, but it doesn't work.

We've put it on a number of other chains, and I don't really think that ignite is a suitable alternative for this task, because when we spin up a testnet with ignite, we are doing that without any assurances that the testnet command works.

I guess that one alternative is that we drop the testnet command, but please do know that the testnet command does something fairly substantially different from what happens when we do like:

ignite chain serve

[okwme]

thanks @faddat , I'm trying to understand what that substantially different set of things is to ascertain whether it's worth spending time deprecating use of ignite in favor of the testnet command. i'm into consistency but don't want to fix something that isn't broken without more justification than consistency. are there other benefits to the testnet command that we should consider?