Add GitHub Action workflow for testing new rules across popular repositories

[alexandear]

Is your feature request related to a problem? Please describe.

Currently, there is no efficient way to quickly validate a new revive rule against a diverse set of popular Open Source Go repositories. When developing new rules, it's crucial to test them against real-world codebases to ensure they work correctly and don't produce excessive false positives or negatives. Manual testing across multiple repositories is time-consuming and error-prone.

Describe the solution you'd like

Create a GitHub Action workflow that automates the testing of Revive rules across a set of popular Go repositories.

The workflow should:

• Build Revive from a specified branch/commit/PR
• Clone or checkout a predefined set of popular Go repositories (e.g., Kubernetes, Docker, Hugo, etc.)
• Run Revive with specific rule configurations against these repositories
• Support configurable rule sets and repository lists
• Provide detailed violation reports

Describe alternatives you've considered

Manual script approach. A standalone bash script that performs similar functionality.n

Additional context

This feature would be similar to https://github.com/ldez/golangci-lint-bench, which is used for testing new golangci-lint linters. However, our focus would be on functional validation of revive rules rather than performance benchmarking.

[vbvictor]

With GitHub workflow approach, will the developer be able to run this test suite locally without many problems? I recently tried to run GitHub actions locally and didn't like the experience (can be a skill issue 😀).

What I'd like is a Docker container with pre-downloaded go repositories, over which I can run revive. With docker, we can equally easily test revive locally and in CI.

[vbvictor]

The golangci-lint bench-suite likely wouldn't intended to run locally, but with this test-suite I'd like to first run it on my machine before submitting a PR

[chavacava]

When developing new rules, it's crucial to test them against real-world codebases to ensure they work correctly and don't produce excessive false positives or negatives

I agree 100%, I actually do that with new rules and fixes. The hard (impossible) part is "ensure they [...] don't produce excessive false positives or negatives". Finding a general method to automatically detect if the rule produces false positives/negatives is equivalent to write the rule in a way it does not produce false positives/negatives... so if we are not able to do it for the rule itself we will not be able to do it for the test.

What we can automate is checking if the rule does crash (panic). By experience, big (and ugly) codebases like Kubernetes are good sources of weird syntax examples that help finding fatal errors.