Feedback on Scorecard result data

[azeemshaikh38]

I had a conversation with Jose Duart from Google and he had some interesting observations on Scorecard data from BQ that he analyzed.

1. Some repos may be pretty well established (e.g https://github.com/yaml/pyyaml), so not a lot of commits will be happening. We mark these repos as Not Active, which is unexpected from a users POV. We probably need better signals to understand if repos are Active or improve how we score these repos.
2. The Vulnerabilities check simply tells if there is an open vulnerability or not. This is not a strong signal when considering a package as a dependency. A stronger signal might be something like - how long do the repo owners take on average to fix vulnerabilities once they become known.

These might be interesting points to discuss in our next meeting, so creating an issue.

[laurentsimon]

I had a conversation with Jose Duart from Google and he had some interesting observations on Scorecard data from BQ that he analyzed.

1. Some repos may be pretty well established (e.g https://github.com/yaml/pyyaml), so not a lot of commits will be happening. We mark these repos as Not Active, which is unexpected from a users POV. We probably need better signals to understand if repos are Active or improve how we score these repos.

in addition to whether issues are closed or commented on, etc I think when the repo has dependabot installed we would get some PRs merged once in a while.

[laurentsimon]

the statistics API may be useful to assess the activity of a repo https://docs.github.com/en/rest/reference/repos#statistics

[laurentsimon]

increasing the time period may also be useful - related to https://github.com/ossf/scorecard/issues/1025

[laurentsimon]

this API may also be useful https://docs.github.com/en/rest/reference/activity

[laurentsimon]

another idea is to use the list of transitive deps, see if some have been updated and if the project has accepted dependabot PRs. That's pretty involved, though

[github-actions[bot]]

Stale issue message

[github-actions[bot]]

This issue is stale because it has been open for 60 days with no activity.

[justaugustus]

Still relevant:

• Extending lookback period for maintained activity: https://github.com/ossf/scorecard/issues/2160
• Deps Update Tool activity: https://github.com/ossf/scorecard/issues/2165