go-bitswap icon indicating copy to clipboard operation
go-bitswap copied to clipboard

testing: skip flaky tests on failure so that CI can reliably pass

Open aschmahmann opened this issue 4 years ago • 2 comments

I've taken the approach of only skipping the particular errors identified in https://github.com/ipfs/go-bitswap/issues/493. We still obviously need to fix these, but having CI green makes it easier to determine when an issue with a PR is new or not.

Note: It's possible that many of these tests are actually exposing bugs, the reason to disable is mostly so that it's possible to sanely review other PRs in the meanwhile without the temptation to ignore the big red X.

aschmahmann avatar Sep 02 '21 19:09 aschmahmann

What's the status here? Is there any reason to skip things that aren't faulty?

Stebalien avatar Sep 15 '21 14:09 Stebalien

The status is that this PR is skipping tests that are flaky until CI starts being green. However, it looks like we have more flaky tests than just those mentioned in ipfs/go-libipfs#86 that are getting discovered wackamole style. I had timeboxed myself here, so the PR has some flaky tests disabled, but not enough to get CI green.

aschmahmann avatar Oct 01 '21 16:10 aschmahmann

This repository has been moved to https://github.com/ipfs/go-libipfs. There is not an easy way to transfer PRs, so if you would like to continue with this PR then please re-open it in the new repository and link to this PR.

Jorropo avatar Jan 27 '23 22:01 Jorropo