realm
Add coverage statistics
WIP - Addresses #5906
Adds coverage reporting to SwiftLint, via a --report-coverage command line option, and a report_coverage configuration file setting.
% swiftlint --quiet --report-coverage
Enabled rules coverage: 0.995
All rules coverage: 0.854
"Enabled rules coverage" is helpful for monitoring whether contributors are simply disabling SwiftLint for large sections of code or entire files (although blanket_disable_command, if not disabled, should help with that already).
"All rules coverage" is more useful - low values here indicate that a lot of rules are disabled (or suppressed), and that you could potentially be getting a lot more value out of SwiftLint.
Related changes - we now cache calls to file.regions(), as we'll call this at least twice if coverage is enabled.
From https://github.com/mildm8nnered/SwiftLint/blob/mildm8nnered-add-coverage-statistics/Source/SwiftLintFramework/Coverage.swift:
Coverage is defined as the sum of the number of rules applied to each line of input, divided by the product of the number of input lines and the total number of rules.
If all rules are applied to every input line, then coverage would be 1 (or 100%). If half the rules are applied
to all input lines, or if all the rules are applied to half of the input lines, then coverage would be 0.5, and
if no rules are enabled, or there is no input, then coverage would be zero.
No distinction is made between actual lines of Swift source, versus blank lines or comments, as SwiftLint may apply rules to those as well. Coverage is only calculated over input files, so if you exclude files in your configuration, they will be ignored. Empty input files, or files containing a single empty line will be ignored, as SwiftLint ignores these files automatically.
"All rules" can be defined either as all enabled rules, or all available rules, enabled or not, resulting in two different coverage metrics:
- "Enabled rules coverage" measures how frequently enabled rules are being disabled by
swiftlint:disablecommands. - "All rules coverage" measures how many of all possible rules are actually being applied.
Typically, enabled rules coverage will be close to 1, as swiftlint:disable is used sparingly. All rules
coverage will generally be much lower, as some rules are contradictory, and many rules are optin. With no opt-in
rules enabled, all rules coverage will be about 0.4, rising to 0.8 or more if many opt-in rules are enabled.
When calculating all rules coverage swiftlint:disable commands are still accounted for, but only for enabled
rules.
Coverage figures will be different for linting and analyzing as these use different sets of rules.
The number of enabled rules is determined on a per-file basis, so child and local configurations will be accounted for.
Custom rules, if enabled and defined, will be counted as first class rules for both enabled and all rules coverage.
If not enabled, custom_rules will be counted as a single rule, even if a configuration exists for it.
When calculating enabled rules coverage, the custom rules in the configuration for each input file (e.g. including child configurations) will be taken into account. When calculating all rules coverage, only the main configurations custom rules settings will be used.
| 1 Warning | |
|---|---|
| :warning: | Big PR |
| 18 Messages | |
|---|---|
| :book: | Building this branch resulted in a binary size of 25490.4 KiB vs 25468.68 KiB when built on main (0% larger). |
| :book: | Linting Aerial with this PR took 1.06 s vs 1.05 s on main (0% slower). |
| :book: | Linting Alamofire with this PR took 1.41 s vs 1.41 s on main (0% slower). |
| :book: | Linting Brave with this PR took 9.79 s vs 9.78 s on main (0% slower). |
| :book: | Linting DuckDuckGo with this PR took 25.82 s vs 25.89 s on main (0% faster). |
| :book: | Linting Firefox with this PR took 13.66 s vs 13.69 s on main (0% faster). |
| :book: | Linting Kickstarter with this PR took 11.1 s vs 11.19 s on main (0% faster). |
| :book: | Linting Moya with this PR took 0.55 s vs 0.56 s on main (1% faster). |
| :book: | Linting NetNewsWire with this PR took 3.19 s vs 3.2 s on main (0% faster). |
| :book: | Linting Nimble with this PR took 0.84 s vs 0.85 s on main (1% faster). |
| :book: | Linting PocketCasts with this PR took 9.23 s vs 9.32 s on main (0% faster). |
| :book: | Linting Quick with this PR took 0.49 s vs 0.49 s on main (0% slower). |
| :book: | Linting Realm with this PR took 4.89 s vs 4.89 s on main (0% slower). |
| :book: | Linting Sourcery with this PR took 2.47 s vs 2.47 s on main (0% slower). |
| :book: | Linting Swift with this PR took 5.67 s vs 5.68 s on main (0% faster). |
| :book: | Linting VLC with this PR took 1.48 s vs 1.48 s on main (0% slower). |
| :book: | Linting Wire with this PR took 22.8 s vs 22.8 s on main (0% slower). |
| :book: | Linting WordPress with this PR took 12.93 s vs 13.03 s on main (0% faster). |
Generated by :no_entry_sign: Danger
