keep-ecdsa
keep-ecdsa copied to clipboard
keep-network
Execute integration test in CI
Add integration test execution to the CI process.
--
Implementer: @liamzebedee
Feature buddy: @nkuba
This… Isn't always a good idea. You're going to want these to be pretty performant if you want to run them on every push.
Another approach is to require that the tests be run for a PR to be merged, but have them be manually triggered somehow. The reviewer would manually trigger them once the PR is looking overall good.
This… Isn't always a good idea. You're going to want these to be pretty performant if you want to run them on every push.
Yes, I'm aware of that and this was something I wanted to think over. The current test is very quick, but I agree that usually, integration tests take too much time to be executed after each push.
Another approach is to require that the tests be run for a PR to be merged, but have them be manually triggered somehow. The reviewer would manually trigger them once the PR is looking overall good.
I'd prefer to not leave any manual steps in the process. I'm not sure about the possibilities we have. We can also execute them after the merge to master but before versioning/publishing artifacts. We should investigate possibilities and discuss as part of this task.
I think for the current codebase, integration tests are really quick. The biggest pain is setup but taking an example from tbtc, integration test setup and execution is multiple times faster than the execution of all unit tests.
From what I tried yesterday, the integration test for tecdsa signing takes no more than a second.
I'd prefer to not leave any manual steps in the process.
At a high level: I also don't like manual steps. That said, I'm only worried about manual steps that have to be remembered. If we do have manual steps, they should be enforced, and that's what I was getting at with the “requiring that tests be run” bit (via a branch restriction in GitHub). I would also like to automate this of course, e.g. by finishing off ozymandias or something similar.
Either way though, this seems like it's a future concern---if the tests are quick today, there's little reason to avoid running them on every push.