Audit component return error when querying unexpected resources

[ethernoy]

What steps did you take and what happened: [A clear and concise description of what the bug is.]

I enabled the audit component in my gatekeeper deployment. It can run but prompt this error often:

I used kubectl to get the problematic resources, it did return error, but I don't expect this Kubernetes API resourcet o be queried by gatekeeper audit.

This is the helm values I used for deployment:

gatekeeper:
  replicas: 1
  disableValidatingWebhook: false
  logDenies: true
  resourceQuota: false
  postInstall:
    labelNamespace:
      enabled: false
  image:
    repository: openpolicyagent/gatekeeper
    crdRepository: openpolicyagent/gatekeeper-crds
    release: v3.6.0
    pullSecrets: 
    - name: platform-tool-docker-repo
  controllerManager:
    resources:
      limits:
        cpu: 200m
        memory: 512Mi
      requests:
        cpu: 100m
        memory: 256Mi
  audit:
    resources:
      limits:
        cpu: 200m
        memory: 512Mi
      requests:
        cpu: 50m
        memory: 128Mi

What did you expect to happen:

Audit only query the required resources (e.g. constraint, config)

Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.]

Environment:

• Gatekeeper version: v3.6.0
• Kubernetes version: (use kubectl version): v1.20.7+vmware.1 (Tanzu Kubernetes Grid distribution)

[maxsmythe]

If you run kubectl get api-resources do these resources show up in the results?

Gatekeeper uses kubernetes' discovery API. It can't know whether given resources are auditable if they are returned by the API.

If these resources are showing up in the discovery list, my question would be: why, are they showing up if they should not be queried?

For now, marking this as an enhancement, as providing a way for Gatekeeper to ignore certain kinds is out-of-scope for our current feature set.

@shomron may be interested in this, since this is Tanzu.

[ethernoy]

If you run kubectl get api-resources do these resources show up in the results?

Gatekeeper uses kubernetes' discovery API. It can't know whether given resources are auditable if they are returned by the API.

If these resources are showing up in the discovery list, my question would be: why, are they showing up if they should not be queried?

For now, marking this as an enhancement, as providing a way for Gatekeeper to ignore certain kinds is out-of-scope for our current feature set.

@shomron may be interested in this, since this is Tanzu.

Running kubectl api-resources shows networkpolicystats in one of the returned resources. My understanding is that this resource definition is created by Tanzu by default but disabled due to some feature gate, and thus it cannot be queried.

[ritazh]

If all of your constraints match against specific kinds (e.g. "match only pods"), then maybe providing --audit-match-kind-only=true flag can avoid this type of error.

[ritazh]

@ethernoy did the --audit-match-kind-only=true flag help?

[ethernoy]

@ethernoy did the --audit-match-kind-only=true flag help?

I suspect this issue was caused by a weird deployment model I used: I deployed gatekeeper on cluster 1 with namespace A, B, and deployed the gatekeeper webhook that connects to cluster 1 on cluster 2, which has namespace A, B, C. When K8S apiserver on cluster 2 makes query about resources in namespace C, gatekeeper on cluster 1 finds that there is no namespace C in cluster 1, hence return error. I do not use this deployment model anymore, as I think this is not a good deployment model after all.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.