controller-tools icon indicating copy to clipboard operation
controller-tools copied to clipboard

preserveUnknownFields=false doesn't work with crdVersions=v1

Open jonnylangefeld opened this issue 5 years ago • 21 comments

crdVersions=v1 which is the new default, doesn't work in conjunction with preserveUnknownFields=false. The spec.preserveUnknownFields simply doesn't appear in the crd. There is an integration test for v1beta1, but not for v1.

I'm running the following command in a kubebuilder repo:

controller-gen "crd:preserveUnknownFields=false,crdVersions=v1" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases

I'm using defaults in my schema and if applying to a 1.16.13-gke.1 cluster, I get:

The CustomResourceDefinition "foo.example.com" is invalid: spec.preserveUnknownFields: Invalid value: true: must be false in order to use defaults in the schema

If I add spec.preserveUnknownFields manually to the crd it works

My workaround in the makefile right now is

yq w -i config/crd/bases/cluster.paas.getcruise.com_clusters.yaml spec.preserveUnknownFields false

jonnylangefeld avatar Aug 22 '20 01:08 jonnylangefeld

I'm hitting the same issue, any ideas?

pugangxa avatar Sep 03 '20 13:09 pugangxa

I had the same issue here. It was an issue about upgrading from v1beta1 to v1 CRD. The spec.preserveUnknownFields was true in the cluster (probably because that was the default in v1beta1). When I upgraded to v1, spec.preserveUnknownFields was not set in the manifest, but because it was in the cluster, I had the spec.preserveUnknownFields: Invalid value: true: must be false in order to use defaults in the schema error.

The solution was simply to manually add spec.preserveUnknownFields: false in the generated CRD manifest and apply it once. From then, it worked again when applying controller-gen generated manifests (without spec.preserveUnknownFields set).

npiganeau avatar Sep 12 '20 17:09 npiganeau

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot avatar Dec 11 '20 18:12 fejta-bot

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

fejta-bot avatar Jan 10 '21 18:01 fejta-bot

Any updates on this?

bhiravabhatla avatar Feb 09 '21 12:02 bhiravabhatla

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community. /close

fejta-bot avatar Mar 11 '21 12:03 fejta-bot

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community. /close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Mar 11 '21 12:03 k8s-ci-robot

@elanv: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen /remove-lifecycle rotten

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Mar 28 '21 00:03 k8s-ci-robot

/reopen

kevindelgado avatar Jul 22 '21 17:07 kevindelgado

@kevindelgado: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Jul 22 '21 17:07 k8s-ci-robot

/assign

kevindelgado avatar Jul 22 '21 17:07 kevindelgado

I took a stab at this, but it's a bit hairier than I expected.

Basically, the root cause is that CRDSpec.PreserveUnknownFields went from a type *bool in v1beta1 to a type bool in v1. The default value for a bool is false and because the field has an omitempty tag on it, a false value will never be written out to the yaml.

Since the object that gets written to yaml has the correct value of false, there's no easy way to get it to show up. I've hacked together a proof-of-concept of one way to do this that involves aliasing the apiext.CustomResourceDefinition type in order to start a conversation, but it's pretty icky and I'm not sure the maintainers will go for it. Check it out at https://github.com/kubernetes-sigs/controller-tools/pull/590

kevindelgado avatar Jul 31 '21 00:07 kevindelgado

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Oct 29 '21 01:10 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Nov 28 '21 01:11 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue or PR with /reopen
  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-triage-robot avatar Dec 28 '21 02:12 k8s-triage-robot

@k8s-triage-robot: Closing this issue.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue or PR with /reopen
  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Dec 28 '21 02:12 k8s-ci-robot

Still hitting this problem, though I was able to manually edit the CRD to set it to false

derekperkins avatar Nov 07 '22 03:11 derekperkins

Now it is trivial to implement like:

$ git --no-pager diff -- pkg/crd/gen.go
diff --git a/pkg/crd/gen.go b/pkg/crd/gen.go
index 9eb4126b..cb6dd964 100644
--- a/pkg/crd/gen.go
+++ b/pkg/crd/gen.go
@@ -100,6 +100,12 @@ func transformRemoveCRDStatus(obj map[string]interface{}) error {
        return nil
 }
 
+// transformPreserveUnknownFields ensures we spec.preserveUnknownFields=false.
+func transformPreserveUnknownFields(obj map[string]interface{}) error {
+       obj["spec"].(map[interface{}]interface{})["preserveUnknownFields"] = false
+       return nil
+}
+
 func (g Generator) Generate(ctx *genall.GenerationContext) error {
        parser := &Parser{
                Collector: ctx.Collector,
@@ -171,7 +177,7 @@ func (g Generator) Generate(ctx *genall.GenerationContext) error {
                        } else {
                                fileName = fmt.Sprintf("%s_%s.%s.yaml", crdRaw.Spec.Group, crdRaw.Spec.Names.Plural, crdVersions[i])
                        }
-                       if err := ctx.WriteYAML(fileName, headerText, []interface{}{crd}, genall.WithTransform(transformRemoveCRDStatus), genall.WithTransform(genall.TransformRemoveCreationTimestamp)); err != nil {
+                       if err := ctx.WriteYAML(fileName, headerText, []interface{}{crd}, genall.WithTransform(transformRemoveCRDStatus), genall.WithTransform(genall.TransformRemoveCreationTimestamp), genall.WithTransform(transformPreserveUnknownFields)); err != nil {
                                return err
                        }
                }

but unfortunately crd:preserveUnknownFields marker support was removed by #607

AlexanderYastrebov avatar Apr 09 '24 16:04 AlexanderYastrebov

/remove-lifecycle rotten /reopen

@AlexanderYastrebov Feel free to open a PR (please with a corresponding test)

sbueringer avatar Apr 09 '24 18:04 sbueringer

@sbueringer: Reopened this issue.

In response to this:

/remove-lifecycle rotten /reopen

@AlexanderYastrebov Feel free to open a PR (please with a corresponding test)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Apr 09 '24 18:04 k8s-ci-robot

Feel free to open a PR (please with a corresponding test)

Ok, created #912

AlexanderYastrebov avatar Apr 10 '24 13:04 AlexanderYastrebov

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Jul 09 '24 13:07 k8s-triage-robot

@sbueringer @alvaroaleman I guess we can close this as done?

AlexanderYastrebov avatar Jul 09 '24 14:07 AlexanderYastrebov

Yes we can, thx!

/close

sbueringer avatar Jul 10 '24 04:07 sbueringer

@sbueringer: Closing this issue.

In response to this:

Yes we can, thx!

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Jul 10 '24 04:07 k8s-ci-robot