go-critic Add one or more Go projects to the integration tests suite

So we can detect changes in the linter output in a little more reliable way.

Per-checker tests can miss some corner cases handling that may be present in the code we could vendor for testing.

Sep 15 '20 11:09 quasilyte

Probably this can be done as a yet another makefile step :D

Download/checkout list of repos.
Run go-critic over them.

I'm sceptical about adding such test on CI. Even if it's happening in parallel.

Sep 15 '20 13:09 cristaloleg

I'm sceptical about adding such test on CI. Even if it's happening in parallel.

Why? NoVerify has ~8 vendored projects and it helps a lot to detect the unwanted changes in the linter.

Sep 15 '20 14:09 quasilyte

Meh, I'm scared that this will make CI slower. Even if it's parallel. My assumptions are based on: fetch repos (few minutes), build critic, run it. And due to CI machines in the cloud it might hang up. So running them locally (there is no so many contributors anyway) might be a better start. And also, it can be a test that is ignored unless tested manually via makefile.

Sep 15 '20 17:09 cristaloleg

But you don't need to fetch them. I'm proposing to store a local copy in testdata. These copies will ~never be updated, that's the point. We create a baseline of warnings and check that the same code is verified in the same way as before. No versioning needed.

Sep 15 '20 19:09 quasilyte

Growing testdata isn't much better imo. Why do we need to throw additional KBs of code for tests? + updates + fetch time (see CI). This will be even more heavier solution :(

Sep 15 '20 20:09 cristaloleg

Good thing to start https://github.com/mvdan/corpus/

May 28 '21 08:05 cristaloleg

AFAIR it's done via https://github.com/go-critic/extern-testdata Closing?

Feb 24 '23 08:02 cristaloleg