New check: Does the project use protected tags?, blocked on GitHub feature implementation.

[kimsterv]

https://github.community/t/feature-request-protected-tags/1742/21

[gabibguti]

Updating GitHub community discussion link: https://github.com/orgs/community/discussions/10906 It's still on beta, but seems like it's not being watched/maintained.

[afmarcum]

This feature does not align with the current project focus. If there is no feedback in the next 7 days to the contrary, then this issue will be closed.

[gabibguti]

@afmarcum Just to clarify, you mean using the GitHub rulesets for restricting who can create, update and delete tags inside Scorecard repository does not align with Scorecard's focus right now, right?

Does it make sense to integrate the GitHub rulesets logic for Scorecard Branch-Protection check though?

[gabibguti]

One advantage I see is, now that Scorecard has a contributor ladder, we could consider restricting the access for new maintainers on releasing by restricting the access to creating tags with GitHub rulesets.

[gabibguti]

@afmarcum Just to clarify, you mean using the GitHub rulesets for restricting who can create, update and delete tags inside Scorecard repository does not align with Scorecard's focus right now, right?

Does it make sense to integrate the GitHub rulesets logic for Scorecard Branch-Protection check though?

Seems like the GitHub rulesets are already being considered for Branch-Protection, https://github.com/ossf/scorecard/pull/3354. 🎉

[afmarcum]

@gabibguti I'm working through older issues to see which still have interest. PRs are welcome for any work, but the maintainers will not be focused on this work in the near future.

[AdamKorcz]

I am working on this. @spencerschrock would you mind assigning me?

[spencerschrock]

@AdamKorcz a few questions before beginning.

1. Data collection, are you going to adapt the existing graphQL call and the repoclient GetBranch call? Maybe we rename it to GetRef? or something that encompasses both (i think tags and branches can both be named the same name though)? I suspect we'll have breaking changes eitherway?
2. Does GitLab have a similar API to query?
3. What kind of scoring would you impose?
4. What severity would the check be marked as?

[AdamKorcz]

@spencerschrock

In addition to my answers below, we face the following substantial decisions:

1. Protected tags are a bit different in that logically IMO at least, we want to ensure that projects protect all their tags. Repositories have a main branch which to an extent justifying checking a single branch for protection which I understand Scorecard does now, however repositories do not have a main tag. As I see it, the high level question here is whether projects follow best practices across all tags. As a result, we should check all tags. So at a high level, we should check each single tag in the repo and whether it is protected with all rules including exceptions across rules (I believe Scorecard implements all of the logic to check this for branches which can be reused) and then flag all tags where the answer is "no". Let me know if you agree with that.

I suggest building this from the grounds of, ie. starting with the probes and then collect them into a check.

Data collection, are you going to adapt the existing graphQL call and the repoclient GetBranch call?

Yes, from what I can tell, the existing branch handler can be reused. To abstract away the reusable parts requires refactoring.

(i think tags and branches can both be named the same name though)?

Yes, this might be possible. Otherwise I would suggest a tagRef and a branchRef as an alternative. At a high level, it seems fine to merge them together, but I think it also depends on the details in the end, and it might be something good to determine with code review.

Does GitLab have a similar API to query?

Yes:

• https://docs.gitlab.com/user/project/protected_tags/
• https://docs.gitlab.com/api/protected_tags/

What kind of scoring would you impose?

I don't know yet, but I think we can decide one that even when the probes are implemented. It will be easier to make a decision on it once we have meaningful probes.

What severity would the check be marked as?

Also not sure. I will follow up on that as I do more research on the threat model around protected tags.