Error: Dry-run failed for non-structured resource

[bg2132332]

If you prefer, you can also ask your question in the Kubernetes community Slack channel #terraform-providers. (Sign up here)

Terraform version, Kubernetes provider version and Kubernetes version

Terraform version: 0.14.8
Kubernetes Alpha Provider version: 0.4.0
Kubernetes version: 4.7.2

Terraform configuration

resource "kubernetes_manifest" "test" {
  provider = "kubernetes-alpha"
  manifest = {
    "apiVersion"="kubevirt.io/vlalpha3"
    "kind"="VirtualMachine"
    "metadata"={
      ...
    }
    "spec"= {
      ...
    }
  }
}

Question

I get the following warning and then error back.  Would anyone suggest a solution?

> Warning: This custom resource does not have an associated OpenAPI schema.
> 
>   on main.tf line 45, in resource "kubernetes_manifest" .....
> 
> We could not find an OpenAPI schema for this custom resource. Updates to this 
> resource will cause a forced replacement.
> 
> Error: Dry-run failed for non-structured resource
> 
>   on main.tf line 45, in resource "kubernetes_manifest" .....
> 
> A dry-run apply was performed for this resource by was unsuccessful:
> admission webhook "virtualmachines-mutator.kubevirt.io" does not support dry
> run

[alexsomesan]

@bg2132332 do you have the virtualmachines-mutator.kubevirt.io CRD installed on the cluster? Can you see it when you run kubectl get crds | grep virtualmachines-mutator.kubevirt.io ?

[bg2132332]

Hi Alex, thank you for a prompt response. i get nothing when i run your command, but I'm assured by my OpenShift expert that KubeVirt is installed correctly and functional. This is partly what he wrote:

... In the product as shipped by Red Hat, the web hook [referring to mutating admission web hook] in question has the side effects set to "unknown". This has the effect of making a dry run not possible using a resource that has this web hook in the admission chain, as the API server will refuse it because it cannot guarantee safety. ...

The second issue is that there is not an OpenAPI specification for the KubeVirt API, which will result in Terraform force replacing any resource that is modified.

I've been going over some of the open issues and looks like there may be solutions for side effects "unknown" and the missing OpenAPI spec issues. This feels like not an uncommon situation. Please help me understand.

[jrhouston]

It looks like this is happening because the virtualmachines-mutator webhook does not support dry-run, and we do a dry-run at plan time for resources which have no schema as a last-measure to confirm they will apply successfully. A possible solution here is to check for this error explicitly and allow the plan to continue anyway, generate a warning, and hope for the best at apply time.