spec.validation.openAPIV3Schema.properties[spec].properties[parameters].type: Required value: must not be empty for specified object fields

[mac-chaffee]

Looks like maybe #109 has caused the following error if you install one of those modified constraint templates:

Error from server: error when applying patch:
{...}
to:
Resource: "templates.gatekeeper.sh/v1beta1, Resource=constrainttemplates", GroupVersionKind: "templates.gatekeeper.sh/v1beta1, Kind=ConstraintTemplate"
Name: "k8spspallowprivilegeescalationcontainer", Namespace: ""
for: "STDIN": admission webhook "validation.gatekeeper.sh" denied the request: spec.validation.openAPIV3Schema.properties[spec].properties[parameters].type: Required value: must not be empty for specified object fields

The following constraint templates are affected:

• k8spspallowprivilegeescalationcontainer
• k8spsphostnamespace
• k8spspprivilegedcontainer
• k8spspreadonlyrootfilesystem

Not sure what the fix would be exactly, since none of those constraints accept parameters. So it seems strange that we must specify a type. But I'm probably just not familiar enough with the ConstraintTemplate CRD.

[maxsmythe]

I can't replicate the failure on my end.

Can you try this again with a fresh install of Gatekeeper on a clean cluster and see if you still see the error?

If not, it might be that you're working with an older version of Gatekeeper or the CRDs on that cluster.

If so, can you let me know what type of cluster (K8s version, provider, etc.) and Gatekeeper version you're working with?

[mac-chaffee]

Hmm the following procedure does work fine (no error observed):

1. Remove all of the CRDs
2. helm del gatekeeper-n gatekeeper
3. helm upgrade --install -n gatekeeper --version v3.7.0 ...
4. Install the latest gatekeeper-library, ref=0bfe7ef57942c4d2d15532c5de15bfd79dbcbb42

But when I do the following, I see the error again:

1. remove all of the CRDs
2. helm del gatekeeper-n gatekeeper
3. helm upgrade --install -n gatekeeper --version v3.6.0 ...
4. install the version of gatekeeper-library I was using before, ref=0c82f402fb3594097a90d15215ae223267f5b955
5. upgrade gatekeeper to 3.7.0 helm upgrade --install -n gatekeeper --version v3.7.0 ...
6. Install the latest version of gate-keeper-library, ref=0bfe7ef57942c4d2d15532c5de15bfd79dbcbb42

Seems like there might be some action required to upgrade from 3.6.0 to 3.7.0 cleanly.

My Kubernetes version is 1.21.5, set up with kubeadm on bare-metal.

[maxsmythe]

If you manually push the new CRDs, does the failure go away?

I wonder if the CRD upgrade routine the Helm chart has is breaking for you.

[mac-chaffee]

Hmm no that doesn't seem to fix it either. This is how I manually installed the CRDs:

git clone [email protected]:open-policy-agent/gatekeeper.git
cd gatekeeper
git checkout v3.7.0
kubectl apply -f charts/gatekeeper/crds/

After doing that, then installing the gatekeeper library github.com/open-policy-agent/gatekeeper-library/library/general?ref=0bfe7ef57942c4d2d15532c5de15bfd79dbcbb42, I get the same error

...
for: "STDIN": admission webhook "validation.gatekeeper.sh" denied the request: spec.validation.openAPIV3Schema.properties[spec].properties[parameters].type: Required value: must not be empty for specified object fields

[maxsmythe]

At what step do you start seeing errors?

After step (5), do errors start showing up in the status.byPod field of your pre-installed constraint templates?

Have all the pods migrated to 3.7.0? If you run kubectl get pods -n gatekeeper-system -o yaml, what images are you seeing being used?

[mac-chaffee]

I see the errors during step 6 "Install the latest version of gatekeeper-library". I don't see any errors in that field:

$ kubectl get constrainttemplates.templates.gatekeeper.sh k8spsphostnamespace -o yaml
...
status:
  byPod:
  - id: gatekeeper-audit-75fd657849-hp97d
    observedGeneration: 1
    operations:
    - audit
    - mutation-status
    - status
    templateUID: 1926436a-7ff4-4bef-bd83-e4452ed628bb
  - id: gatekeeper-controller-manager-7d5fc685c5-8s2sd
    observedGeneration: 1
    operations:
    - mutation-webhook
    - webhook
    templateUID: 1926436a-7ff4-4bef-bd83-e4452ed628bb
  - id: gatekeeper-controller-manager-7d5fc685c5-k2kbn
    observedGeneration: 1
    operations:
    - mutation-webhook
    - webhook
    templateUID: 1926436a-7ff4-4bef-bd83-e4452ed628bb
  - id: gatekeeper-controller-manager-7d5fc685c5-ss2x7
    observedGeneration: 1
    operations:
    - mutation-webhook
    - webhook
    templateUID: 1926436a-7ff4-4bef-bd83-e4452ed628bb
  created: true

Yes, the pods have migrated to 3.7.0:

$ kubectl get pods -n gatekeeper -o yaml | grep "image: openpolicyagent"
      image: openpolicyagent/gatekeeper:v3.7.0
      image: openpolicyagent/gatekeeper:v3.7.0
      image: openpolicyagent/gatekeeper:v3.7.0
      image: openpolicyagent/gatekeeper:v3.7.0

But when I try to install gatekeeper-library using this kustomization.yaml file:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- github.com/open-policy-agent/gatekeeper-library/library/pod-security-policy?ref=0bfe7ef57942c4d2d15532c5de15bfd79dbcbb42
- github.com/open-policy-agent/gatekeeper-library/library/general?ref=0bfe7ef57942c4d2d15532c5de15bfd79dbcbb42

I get the error: "STDIN": admission webhook "validation.gatekeeper.sh" denied the request: spec.validation.openAPIV3Schema.properties[spec].properties[parameters].type: Required value: must not be empty for specified object fields

[maxsmythe]

I think I know the issue. Unfortunately this was a bug where the defaulting of legacySchema wouldn't properly apply if validation was missing (this was fixed in 3.7.0).

The result is that when a resource gets converted to v1 (which it will automatically on write), legacySchema is given the wrong default in 3.7.0 basically, it assumes legacySchema is true).

https://github.com/open-policy-agent/gatekeeper/blob/59031960c84efa5b148cd9d3b70ed473b23e0392/charts/gatekeeper/crds/constrainttemplate-customresourcedefinition.yaml#L49-L50

Sorry, I thought we had cleaned that up.

A fix could be to manually set spec.crd.spec.validation.legacySchema to true for the affected resources.

[mac-chaffee]

Nice find!

Should we set legacySchema: true on the affected resources in this repo? (k8spspallowprivilegeescalationcontainer, k8spsphostnamespace, k8spspprivilegedcontainer, k8spspreadonlyrootfilesystem)

Or convert them to the "non-legacy" schema? Sorry I might not be understanding the bug. Related to 1458

[maxsmythe]

Either should work, since those constraints don't rely on parameters. Converting to non-legacy schema is probably more correct, since it will raise an error if any parameters are provided, but setting legacy schema to true matches what proper upgrade behavior should have done.

[stale[bot]]

This issue/PR has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.