Add Gatekeeper policy tracker

[nilekhc]

Is your feature request related to a problem? Please describe.

We would like to host Gatekeeper policies on Artifact Hub.

Describe the solution you'd like

We would like to have Gatekeeper as its own kind under which users can list Gatekeeper policies.

Describe alternatives you've considered NA

Additional context NA

[nilekhc]

cc @maxsmythe @ritazh @sozercan

[tegioz]

Hi @nilekhc

Gatekeeper policies would be a nice addition to Artifact Hub 🙂

Let me explain a bit how Artifact Hub indexes content.

Any organization or user can add repositories to Artifact Hub. At the moment we support several repositories kinds, like Helm charts, OLM operators, etc. The tracker component polls them periodically and collects metadata as needed. Depending on the repository kind, metadata is extracted one way or another. For kinds that have already defined how a repository should be structured and served, like Helm charts or OLM operators, we have specialized tracker sources. This allowed publishers to start adding existing repositories without requiring any additional work. For other kinds, we have a generic tracker source based on our own metadata file.

For Gatekeeper policies, I'd suggest we use the generic tracker source. This tracker relies on a custom metadata file and a flexible directory structure that supports one or more packages per repository, including multiple versions per package if needed. In this particular case, maybe just adding the AH metadata file on each policy directory inside the library directory would work, but you can organize it the way you think would work best for the project. Data unique to Gatekeeper can be added in the form on custom annotations.

Some examples of how other projects are organizing the Artifact Hub metadata:

• Keda: https://github.com/kedacore/external-scalers/tree/main/artifacthub
• Keptn: https://github.com/keptn-contrib/artifacthub
• Kubewarden: https://github.com/kubewarden/allowed-fsgroups-psp-policy
• Tinkerbell: https://github.com/tinkerbell/hub/tree/artifacthub-manifests/actions

I've noticed that Gatekeeper policies usually don't have much documentation other than the description field in the template.yaml file (storageclass provides a README file, which is great). In Artifact Hub most packages provide a README file that is rendered on the package view. If this is not going to be the case for Gatekeeper policies, we may need to rethink a bit what could be displayed in that view for this new kind (maybe some info about parameters when available, samples, etc), so that it doesn't look too empty 😉

If this sounds good to you we'd be happy to proceed and start working on this 🙂

Please let us know if you have any questions!

[nilekhc]

Hi @tegioz, Thanks for the information. I have a couple of questions.

1. We are developing a website (PR in flight) that has readme files for each policy. But they are in website specific directory structure. I am hoping to create a symlink for them in the library directory structure to avoid duplication. Would symlink work with AH for readme?
2. We do not have any versioning right now, but we intend to introduce it in near future. Can we start with a non-version listing in AH and then move towards versioning later?

[tegioz]

Hi @nilekhc

1. We are developing a website (PR in flight) that has readme files for each policy. But they are in website specific directory structure. I am hoping to create a symlink for them in the library directory structure to avoid duplication. Would symlink work with AH for readme?

I've just checked the markdown files in the PR you mentioned. It looks like the content is based on the description plus some other information from the other files of the policy. In this case, I think it'd be better to only use the description in the AH readme and we would create a special package view displaying those files ourselves. We'd have more flexibility to make them look nicer in the UI and we wouldn't be storing duplicate information (as we were thinking about storing the content of those files as well anyway). Please note AH doesn't support html tags in the markdown content.

To answer your question, the symlink might work. But I think we could get better results by storing each piece of data separately. We could allow users to navigate examples with views like this one, for example.

I forgot to mention it before in the intro, but using the widget groups feature you could also display the Gatekeeper policies available in Artifact Hub in your site, like Keptn does with integrations. Just in case it helps (if it's not too late ;).

2. We do not have any versioning right now, but we intend to introduce it in near future. Can we start with a non-version listing in AH and then move towards versioning later?

A version is required, but you could use always the same one for now (0.0.1 for example). Artifact Hub only processes a package when a new version is available or when the digest of an existing version changes, so you'd need to update the digest field in the metadata file when you want AH to index it again.

Hope this helps :)

[nilekhc]

We'd have more flexibility to make them look nicer in the UI and we wouldn't be storing duplicate information (as we were thinking about storing the content of those files as well anyway).

@tegioz Just for clarification, does this mean we would still need descriptions separately in .md files inside the library directory structure? Or we can render them from website docs?

[tegioz]

Hi @nilekhc

Not in a separate .md file, but in the artifacthub-pkg.yml file as part of the rest of the metadata required, either using the description field or the readme field. When the description is just a line, I would recommend using the description field but when it's longer, it'd be great if you could add the description content to the readme field and add a minimal description.

With that information and the content of the rest of the files in each policy directory (template, examples, etc), we should be able to provide a nice view for Gatekeeper policies in Artifact Hub.

We'll try to put together an example for you tomorrow and we can iterate from there, maybe this way will be easier 🙂

[tegioz]

Hi @nilekhc

We've just added experimental support for Gatekeeper policies! 🚀

You can see it in action in staging. We've forked the polices repo to add the sample Artifact Hub metadata files to two of the polices.

Please let us know your thoughts 🙂

[nilekhc]

Thanks, @tegioz! This gives a lot more clarity. However, I have couple more questions. How are you rendering Template below readme? I do not see any reference to that in the metadata file. Same for Install and Samples.

[tegioz]

Awesome! 🙂

We've assumed that the presence of a template.yaml file inside the policy folder was a convention. The same for the samples folder. This way, when we process a package of kind Gatekeeper policy, we collect that information. If this is correct, we should document it in the Gatekeeper policies section in the repositories guide. It's important that we define this properly as eventually users may want to list more repositories of this kind to Artifact Hub.

Regarding the Install section, we took the installation instructions from the usage section in the library README file as a base (they are part of the UI code), and we render some parts dynamically from the information in the package. The installation instructions can be overridden by providing them in the metadata file, but it may make things easier if we provide some default instructions. It'd be great if you could review they are correct though.

[tegioz]

BTW feel free to add more sample repositories to staging and experiment with them as you like 😉

We'd love to move this to production soon, so any feedback would be greatly appreciated!

[nilekhc]

@tegioz So far this is been really helpful. I am reviewing the proposal for this with the Gatekeeper community. I'll come back to this with the next step once we finalized that.

[tegioz]

Thanks @nilekhc! Looking forward to it 🙂

[tegioz]

We've improved how Gatekeeper policies examples are displayed @nilekhc (related discussion in Slack for reference). You can see it in action in staging 😉

We'll probably deploy this to production later today and announce support for Gatekeeper policies 🎉 You can add the library repository whenever you are ready, no rush 🙂

[nilekhc]

We've improved how Gatekeeper policies examples are displayed @nilekhc (related discussion in Slack for reference). You can see it in action in staging 😉

We'll probably deploy this to production later today and announce support for Gatekeeper policies 🎉 You can add the library repository whenever you are ready, no rush 🙂

@tegioz Could you hold prod deployment for now? I am implementing a new dir structure and want to validate with the staging repo first. Thanks!

[tegioz]

Sure, we'll deploy on Monday then and we'll hold on with the announcement until you confirm everything is fine 👍

[salaxander]

@tegioz This is so awesome! Thanks so much for all the work on this :)

[tegioz]

Will close this one for now, please feel free to reopen if needed 🙂