kustomize
kustomize copied to clipboard
Support for integer keys
Attempting to build Knative config with 3.9.0 on Manjaro Linux produces the error Error: json: unsupported type: map[interface {}]interface {}
.
[mike@ouroboros minikube]$ kustomize version
{Version:3.9.0 GitCommit:$Format:%H$ BuildDate:2020-12-13T07:57:44Z GoOs:linux GoArch:amd64}
[mike@ouroboros minikube]$ cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- https://github.com/knative/serving/releases/download/v0.19.0/serving-core.yaml
[mike@ouroboros minikube]$ kustomize build .
Error: json: unsupported type: map[interface {}]interface {}
Kustomize 3.8 will successfully build.
[mike@ouroboros minikube]$ kustomize version
{Version:kustomize/v3.8.8 GitCommit:72262c5e7135045ed51b01e417a7e72f558a22b0 BuildDate:2020-12-10T18:05:35Z GoOs:linux GoArch:amd64}
[mike@ouroboros minikube]$ kustomize build .
apiVersion: v1
kind: Namespace
metadata:
labels:
serving.knative.dev/release: v0.19.0
name: knative-serving
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
labels:
knative.dev/crd-install: "true"
serving.knative.dev/release: v0.19.0
name: certificates.networking.internal.knative.dev
...
We have issues in 3.9.0. Please try 3.9.1.
@Shell32-Natsu can you shed some light on this? I'm using sigs.k8s.io/kustomize/api v0.7.1
in fluxcd/kustomize-controller but this bug is still there.
I find it very hard to keep up with kustomize upstream changes, there is no changelog expect for a list of commits. For Flux users is even harder, with error outputs like json: unsupported type: map[interface {}]interface {}
it's impossible to track down which object from all the manifests caused this.
This issue is still present in API v0.7.2 as well
This turns out to be a bug in kyaml
, here is how to reproduce it.
Having a custom resource with an integer key:
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: podinfo
spec:
interval: 5m
chart:
spec:
chart: podinfo
sourceRef:
kind: HelmRepository
name: podinfo
values:
tcp:
8080: "default/example-tcp-svc:9000"
Fails with:
$ kustomize build
Error: error marshaling into JSON: json: unsupported type: map[interface {}]interface {}
Works with:
kustomize build --enable_kyaml=false
Version:
kustomize version
{Version:kustomize/v3.9.2 GitCommit: BuildDate:2021-01-17T19:01:12+00:00 GoOs:darwin GoArch:amd64}
What I find really concerning is that users have no way to identify the object/resource/file when kustomize build fails. @Shell32-Natsu what is the recommend way to debug this when you have hundreds of manifests?
@stefanprodan Thanks for the information. I can reproduce it now. Will investigate it.
Is there any chance you can add in the logging of the filename that kicks back the exception as well, please?
This issue is triggered by incompatibility between YAML marshal and unmarshal. I will create a PR to add the resources that causes this error to the error message.
More details about this issue: When kustomize is trying to get the YAML bytes from a resource, it will convert the resource into a map[string]interface{}
then call yaml.Marshal
. The resource is decoded to map[string]interface{}
by yaml.Node.Decode
. When there is an integer key, that field will be decoded as map[interface{}]interface{}
instead of map[string]interface{}
and that is invalid in yaml.Marshal
.
After we delete the apimachinery codes we can get the YAML bytes from the RNode directly so this can be fixed.
@monopole
I have the same issue when using example code from https://istio.io/latest/docs/reference/config/security/peer_authentication:
apiVersion: security.istio.io/v1beta1 kind: PeerAuthentication metadata: name: default namespace: foo spec: selector: matchLabels: app: finance mtls: mode: STRICT portLevelMtls: 8080: mode: DISABLE
I'm also using similar code myself, and my only workaround right now is to stop using Kustomize for this particular deployment - or downgrade Kustomize. The argument --enable_kyaml=false doesn't seem to be a valid anymore.
Thank you @HansK-p for your comment as you helped identify (for me) what I need to do to work around the bug. The portLevelMtls
having a key of 8080
is causing the kyaml parser to make that an int of some sort rather than the expected string. By changing it to "8080"
it works again. ~Going to move this to a slightly different bug, as there is a good reproduction case, and I'm not sure this overlaps with the other issues identified in this issue.~
Edit: Just fully read https://github.com/kubernetes-sigs/kustomize/issues/3446#issuecomment-763853739 - sorry about the spam.
And thank you. I feel a bit stupid not testing that myself.
@Shell32-Natsu I've started #3588
this should be easier once that's done. need a few days
@monopole Thanks!
I see the same error trying to kustomize tekton pipelines (https://github.com/tektoncd/pipeline):
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.21.0/release.yaml
$ kustomize build
Error: map[string]interface {}{"apiVersion":"apiextensions.k8s.io/v1", "kind":"CustomResourceDefinition", "metadata":map[string]interface {}{"labels":map[string]interface {}{"app.kubernetes.io/instance":"default", "app.kubernetes.io/part-of":"tekton-pipelines", "pipeline.tekton.dev/release":"v0.21.0", "version":"v0.21.0"}, "name":"clustertasks.tekton.dev"}, "spec":map[string]interface {}{"conversion":map[string]interface {}{"strategy":"Webhook", "webhook":map[string]interface {}{"clientConfig":map[string]interface {}{"service":map[string]interface {}{"name":"tekton-pipelines-webhook", "namespace":"tekton-pipelines"}}, "conversionReviewVersions":[]interface {}{"v1beta1"}}}, "group":"tekton.dev", "names":map[string]interface {}{"categories":[]interface {}{"tekton", "tekton-pipelines"}, "kind":"ClusterTask", "plural":"clustertasks"}, "preserveUnknownFields":false, "scope":"Cluster", "versions":[]interface {}{map[string]interface {}{"name":"v1alpha1", "schema":map[string]interface {}{"openAPIV3Schema":map[string]interface {}{"type":"object", "x-kubernetes-preserve-unknown-fields":true}}, "served":true, "storage":false, "subresources":map[string]interface {}{"status":map[string]interface {}{}}}, map[interface {}]interface {}{"name":"v1beta1", "schema":map[string]interface {}{"openAPIV3Schema":map[string]interface {}{"type":"object", "x-kubernetes-preserve-unknown-fields":true}}, "served":true, "storage":true, "subresources":map[string]interface {}{"status":map[string]interface {}{}}}}}}: json: unsupported type: map[interface {}]interface {}
It works fine with older kustomize 3.8, fails for newer versions.
Kustomize Version : {Version:kustomize/v4.0.5 GitCommit:9e8e7a7fe99ec9fbf801463e8607928322fc5245 BuildDate:2021-03-08T20:53:03Z GoOs:linux GoArch:amd64}
I am having same issue running the following command
kustomize build .
Error: map[string]interface {}{"apiVersion":"v1", "data":map[interface {}]interface {}
{22:"default/gitlab-test-gitlab-shell:22"}, "kind":"ConfigMap", "metadata":map[string]interface {}
{"labels":map[string]interface {}{"app":"gitlab-shell", "chart":"gitlab-shell-4.9.3", "heritage":"Helm",
"release":"gitlab-test"}, "name":"gitlab-test-nginx-ingress-tcp", "namespace":"default"}}:
json: unsupported type: map[interface {}]interface {}
Actual manifest file with integer key
apiVersion: v1
kind: ConfigMap
metadata:
name: gitlab-nginx-ingress-tcp
namespace: default
labels:
app: gitlab-shell
chart: gitlab-shell-4.9.3
release: gitlab
heritage: Tiller
data:
22: "default/gitlab-gitlab-shell:22"
I have rendered manifest more than 15,00 lines, Please show the manifest area where this error is coming. Also, Should I expect the integer key part of manifest? is there anything to do with kustomize to ignore integer? Please suggest, I have been looking at this issue for a month.
@bhagyesh18 Sorry for the inconvenient. But unfortunately the integer key is not allowed in kustomize now but @monopole is working on deleting translation layers so this issue might be able to be fixed soon.
was it working earlier in any previous version? Also, there are duplicate Keys issues, I can see in a new version that you guys show the actual manifest area where it has duplicate keys, that is good. Is there any way to ignore verifying/unmarshling duplicate keys using kustomize and let it be as it is?
@bhagyesh18 You can try the version before v4 with flag --enable_kyaml=false
. For the duplicate keys, invalid YAML is invalid, we cannot generate an invalid output.
Ran into this issue when trying to use Kustomize with a Knative serving YAML. Seems like the error stems from the usage of !!merge <<: *version
in a lot of these YAMLs including the Tekton pipelines YAML that another user mentioned above.
Simple recreation:
kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- test.yaml
test.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: test.kustomize.error
spec:
group: test.group
versions:
- &version
name: v1alpha1
served: true
storage: false
schema:
openAPIV3Schema:
type: object
- !!merge <<: *version
name: v1beta1
Using kustomize v4.0.5:
kustomize build .
Error: map[string]interface {}{"apiVersion":"apiextensions.k8s.io/v1", "kind":"CustomResourceDefinition", "metadata":map[string]interface {}{"name":"test.kustomize.error"}, "spec":map[string]interface {}{"group":"test.group", "versions":[]interface {}{map[string]interface {}{"name":"v1alpha1", "schema":map[string]interface {}{"openAPIV3Schema":map[string]interface {}{"type":"object"}}, "served":true, "storage":false}, map[interface {}]interface {}{"name":"v1beta1", "schema":map[string]interface {}{"openAPIV3Schema":map[string]interface {}{"type":"object"}}, "served":true, "storage":false}}}}: json: unsupported type: map[interface {}]interface {}
In older versions of kustomize (like 3.8.10), kustomize builds properly and outputs:
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: test.kustomize.error
spec:
group: test.group
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
type: object
served: true
storage: false
- name: v1beta1
schema:
openAPIV3Schema:
type: object
served: true
storage: false
@pvaneck Your issue should be the same as #3614 instead of this.
I experienced that issue as well with v4.1.2
.
kustomize version
{Version:kustomize/v4.1.2 GitCommit:a5914abad89e0b18129eaf1acc784f9fe7d21439 BuildDate:2021-04-15T23:23:03+01:00 GoOs:darwin GoArch:amd64}
Now if I use v3.8.10
, kustomize build
works. https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv3.8.10
For the interested readers, I experienced similar issues from using YAML anchors and references, so not involving any numerical keys:
foo: &foo
one: two
bar:
<<: *foo
/retitle Support for integer keys
Let's keep this issue focussed on the original problem, i.e. integer key handling.
If you've arrived at this issue because of a similar error message encountered when using YAML anchors, please follow https://github.com/kubernetes-sigs/kustomize/issues/3675 instead.
Kustomize Version : {Version:kustomize/v4.0.5 GitCommit:9e8e7a7fe99ec9fbf801463e8607928322fc5245 BuildDate:2021-03-08T20:53:03Z GoOs:linux GoArch:amd64}
I am having same issue running the following command
kustomize build . Error: map[string]interface {}{"apiVersion":"v1", "data":map[interface {}]interface {} {22:"default/gitlab-test-gitlab-shell:22"}, "kind":"ConfigMap", "metadata":map[string]interface {} {"labels":map[string]interface {}{"app":"gitlab-shell", "chart":"gitlab-shell-4.9.3", "heritage":"Helm", "release":"gitlab-test"}, "name":"gitlab-test-nginx-ingress-tcp", "namespace":"default"}}: json: unsupported type: map[interface {}]interface {}
Actual manifest file with integer key
apiVersion: v1 kind: ConfigMap metadata: name: gitlab-nginx-ingress-tcp namespace: default labels: app: gitlab-shell chart: gitlab-shell-4.9.3 release: gitlab heritage: Tiller data: 22: "default/gitlab-gitlab-shell:22"
I have rendered manifest more than 15,00 lines, Please show the manifest area where this error is coming. Also, Should I expect the integer key part of manifest? is there anything to do with kustomize to ignore integer? Please suggest, I have been looking at this issue for a month.
wow! i just ran into this EXACT one too. Glad to know I'm not crazy. I guess I'll try to downgrade kustomize for now.
Error: error while running post render on files: error while running command /tmp/post-render/kustomize. error output:
Error: map[string]interface {}{"apiVersion":"v1", "data":map[interface {}]interface {}{22:"gitlab/gitlab-gitlab-shell:22"}, "kind":"ConfigMap", "metadata":map[string]interface {}{"labels":map[string]interface {}{"app":"gitlab-shell", "chart":"gitlab-shell-4.12.4", "heritage":"Helm", "release":"gitlab"}, "name":"gitlab-nginx-ingress-tcp", "namespace":"gitlab"}}: json: unsupported type: map[interface {}]interface {}
@bhagyesh18 i was able to get past it for now w/ the workaround mentioned earlier, downgrade to a 3.9-ish version and disable kyaml.
I have the same issue when using example code from https://istio.io/latest/docs/reference/config/security/peer_authentication:
apiVersion: security.istio.io/v1beta1 kind: PeerAuthentication metadata: name: default namespace: foo spec: selector: matchLabels: app: finance mtls: mode: STRICT portLevelMtls: 8080: mode: DISABLE I'm also using similar code myself, and my only workaround right now is to stop using Kustomize for this particular deployment - or downgrade Kustomize. The argument --enable_kyaml=false doesn't seem to be a valid anymore.
I don't see istio changing their CRD, so is there any plan on fixing this in kustomize?
Bump, this still seems to be an issue in 4.4.0. Any plans for a fix?
The Kubernetes Nginx Ingress documentation example for exposing TCP ports triggers the error:
https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/
Specifically:
apiVersion: v1
kind: ConfigMap
metadata:
name: tcp-services
namespace: ingress-nginx
data:
9000: "default/example-go:8080"
Thanks
Hello same issue with 4.4.1
:)
╰─➤ kustomize version
{Version:4.4.1 GitCommit:$Format:%H$ BuildDate:2021-12-17T23:11:01Z GoOs:linux GoArch:amd64}
@natasha41575 why is this being tracked on Roadmap for 2022 and not as a hotfix for actual regression/bug?
This should be fixed ASAP, it's a regression and huge limitation for creating configmaps.
@midN Unfortunately there are only two of us working on kustomize with competing priorities both within and outside of kustomize. Because of our lack of resources we are unable to immediately prioritize and fix every regression in a timely manner. The roadmap lists the ones we consider the most important and hope to get to as soon as we can.
Edited to add: anyone is welcome to contribute a PR to address this issue. We have created the roadmap so that outside contributors know which PRs and fixes would be the most helpful to us and that we have the bandwidth to review.