Add globbing support for licenses in the License Policies `policies.yml` file

[silverhook]

It would be great if one could glob for licenses in policies.yml

Some use cases:

• mit-* – for non-standard MIT
• cc-*-nc-* , cc-*-nd-* – to match every CC license that includes either NonCommercial or NoDerivative limitations
• LicenseRef-* – for any license that’s not been added to SPDX License List, so potentially of a more obscure kind (depends on https://github.com/nexB/scancode.io/issues/1348)

[silverhook]

Happy to help. Can you let me know how I can best test it works?

[silverhook]

I admit I am confused.

Nothing prevented me from typing in cc-*-nc-* before, it just did not do anything. How do I test it that it now understands that and parses it correctly?

[pombredanne]

@silverhook I think @VarshaUN meeting comment was in reference to the weekly community meetings we hold every monday at 15:00 UTC ... https://github.com/nexB/aboutcode/wiki/MeetingMinutes We use Jitsi for this

[silverhook]

Ah, thank you. That was what I was missing. I couldn’t make it this week. I’ll add try to remember to add those to my calendar though.

[pombredanne]

@VarshaUN did you work out a PR to support this? If yes, can you link this here?

[pombredanne]

@silverhook do you really want to use globs for license keys rather than using license categories?

[silverhook]

@pombredanne, depends on how the categories are done. I want to have it convenient, but also the final say to fine-tune it.

E.g. if the category is “non-commercial” and there’s a bug in it, I want to be able to override a specific license in that category while I wait for the bug to be fixed.

[keshav-space]

@VarshaUN see how we use the policies.yml file to generate the policy index https://github.com/aboutcode-org/scancode.io/blob/6e131c63126be4266746b20fff8e37505f9606d2/scanpipe/apps.py#L217-L240 Then we use this policy index to compute compliance alert here https://github.com/aboutcode-org/scancode.io/blob/6e131c63126be4266746b20fff8e37505f9606d2/scanpipe/models.py#L2430-L2459

The idea would be to use re.match to match the license_key against the patterns in the license policies index. I would begin with something like this, and we can improve this as needed.

for license_key in license_keys:
    missing_policy = True
    for policy in policy_index:
        if re.match(pattern=policy, string=license_key):
            missing_policy = False
            alerts.append(policy.get("compliance_alert") or self.Compliance.OK)
            break
    if missing_policy:
        alerts.append(self.Compliance.MISSING)