go icon indicating copy to clipboard operation
go copied to clipboard

x/tools/gopls: unskip all previously flaky regtests

Open findleyr opened this issue 2 years ago • 4 comments

This is a follow up to https://go.dev/issue/53781, a deterministic panic that made it into [email protected] in part because a flaky regression test had been skipped, and not un-skipped when the root cause of flakiness was resolved.

We have recently made improvements to several potential sources of flakiness: server shutdown races, invalidation races, and performance. We also have resources to work on flakes that weren't available during the 1.18 push. In light of #53781, we should endeavor to unskip (and further de-flake if necessary) all regression tests that have been unconditionally skipped. This issue tracks that effort:

Currently skipped tests:

  • [x] TestQuickFixEmptyFiles: (#48773)
  • [x] TestModFileModification (#40269)
  • [x] TestCreateDependency (#54073, newly added)
  • [ ] TestSumUpdateFixesDiagnostics (#51352)
  • [x] Test_Issue38211(#44098)
  • [ ] TestWatchReplaceTargets (#50748)
  • [x] TestChangePackageName (#41061)
  • [ ] TestDeleteModule_Interdependent (#46375)

findleyr avatar Jul 14 '22 14:07 findleyr

Change https://go.dev/cl/417576 mentions this issue: gopls/internal/regtest: unskip TestChangePackageName

gopherbot avatar Jul 14 '22 14:07 gopherbot

Change https://go.dev/cl/418898 mentions this issue: gopls/internal/regtest: unskip Test_Issue38211

gopherbot avatar Jul 22 '22 02:07 gopherbot

Change https://go.dev/cl/419106 mentions this issue: gopls/internal/regtest: unskip TestQuickFixEmptyFiles

gopherbot avatar Jul 22 '22 15:07 gopherbot

Change https://go.dev/cl/420718 mentions this issue: gopls/internal/regtest: unskip all of TestModFileModification

gopherbot avatar Aug 02 '22 17:08 gopherbot

Change https://go.dev/cl/422910 mentions this issue: gopls/internal/regtest: unskip TestDeleteModule_Interdependent

gopherbot avatar Aug 15 '22 18:08 gopherbot

Change https://go.dev/cl/423974 mentions this issue: gopls/internal/regtest: unskip TestSumUpdateFixesDiagnostics

gopherbot avatar Aug 15 '22 18:08 gopherbot

FWIW it looks like TestCreateDependency had a flake:

2022-09-01T03:24:42-49ab44d-86e9e0e/openbsd-amd64-70

Should this get a new issue or is it relevant enough to this?

mknyszek avatar Sep 06 '22 21:09 mknyszek

@mknyszek thanks for reporting.

Hmm. I don't think that flake is actually related to TestCreateDependency: it is simply an error deleting the test files during cleanup.

greplogs also turned up the following recent failure in go/packages tests on openbsd: https://build.golang.org/log/d9906d7b11d2e43bded9199a00d40d2f8ac47f31

I wonder if this is a recent regression on openbsd? Unclear, so probably best not to assume, and instead file separate new issues for gopls and go/packages.

findleyr avatar Sep 06 '22 21:09 findleyr

The unlinkat failure looks like #49751 to me.

bcmills avatar Sep 06 '22 21:09 bcmills

Thanks @bcmills. Yes, that looks related, so let's not file new issues.

There is a fair bit of activity on #36435 that may be related.

findleyr avatar Sep 06 '22 21:09 findleyr

Change https://go.dev/cl/496882 mentions this issue: gopls/internal/regtest/misc: update some unilaterally skipped tests

gopherbot avatar May 22 '23 18:05 gopherbot

Change https://go.dev/cl/496883 mentions this issue: gopls/internal/regtest/misc: unskip TestMajorOptionsChange

gopherbot avatar May 22 '23 19:05 gopherbot

Change https://go.dev/cl/496885 mentions this issue: gopls/internal/regtest/misc: update and unskip TestHoverIntLiteral

gopherbot avatar May 22 '23 21:05 gopherbot

We have unskipped all flaky regtests.

findleyr avatar May 22 '23 21:05 findleyr