Config Map Enricher appears to be loosing quotes and then corrupting data

[pmbsa]

Describe the bug

Hi, I have an example where we have a configmap template with data like this

data:
  report_date: "2020-01-01"

After the resource action the kubernetes yml has this

data:
  report_date: 2020-01-01

which then results in a right mess in the deployed CM with the response json showing this

"report_date" : "1577836800000"

I cant for the life of me understand why the quotes would be dropped but I am hoping its something easy to rectify

Eclipse JKube version

1.8.0

Component

Kubernetes Maven Plugin

Apache Maven version

3.8.3

Gradle version

No response

Steps to reproduce

1: create a configmap template with the following

data:
  report_date: "2020-01-01"

run mvn k8s:resource the resultant hydrated manifest will be

data:
  report_date: 2020-01-01

Expected behavior

should be

data:
  report_date: "2020-01-01"

Runtime

Kubernetes (vanilla)

Kubernetes API Server version

other (please specify in additional context)

Environment

Linux

Eclipse JKube Logs

No response

Sample Reproducer Project

No response

Additional context

kubernetes 1.19

[rohanKanojia]

We seem to be losing quotes since Jackson has been configured to do so.

This setting seems to have been added in https://github.com/eclipse/jkube/pull/1124 https://github.com/eclipse/jkube/blob/dd105def19fc641a920d3af4b75779caa01212be/jkube-kit/common/src/main/java/org/eclipse/jkube/kit/common/util/ResourceUtil.java#L56

If I comment this setting, I can see quotes retained in generated YAML after mvn k8s:resource and also in applied ConfigMap from mvn k8s:apply goal

[rohanKanojia]

Are you facing issues with losing quotes or date string getting converted into timestamp? I tried configuring Jackson to not convert dates to timestamps:

Serialization.jsonMapper().configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false);

This results in applied ConfigMap resource like this:

data:
  report_date: "2020-01-01T00:00:00.000+00:00"

Would this be acceptable for your use case?

[pmbsa]

to me it feels like its the quotes going missing is then knocking on the the data being interpreted into nonsense. If the quotes simply remained we would get the date come through as we want it.

[manusa]

As discussed internally, this is a problem of our deserialization process during the Apply phase.

A ConfigMap YAML like the following excerpt:

data:
  report_date: 2020-01-01

is perfectly valid and should be applied as is to the kube-api server (the same ConfigMap applied with kubectl should work).

Forcing quotes to all string in our serialization process would mitigate this issue partially, but wouldn't tackle the real issue we have with deserialization (which is the one we need to fix). For example, the issue would persist if a user were to provide a fragment with similar unquoted content (which is a valid YAML syntax).

[msamad]

Hi, ran into the similar issue with env variables in deployment.yaml, just mentioning here hoping this will be covered in the fix too

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ${project.artifactId}
spec:
  selector:
    matchLabels:
      app: ${project.artifactId}
  template:
    metadata:
      labels:
        app: ${project.artifactId}
    spec:
      containers:
        - name: app
          env:
            - name: ENV
              value: local
            - name: SOME_DATE_ENV
              value: '2022-01-01'
          image: ${project.artifactId}:${project.version}
          ports:
            - containerPort: 8080

end up as

...
....
      "spec" : {
        "containers" : [ {
          "env" : [ {
            "name" : "ENV",
            "value" : "local"
          }, {
            "name" : "SOME_DATE_ENV",
            "value" : "1640995200000"
          }, {
....
...

Thanks

[manusa]

Hi, ran into the similar issue with env variables in deployment.yaml, just mentioning here hoping this will be covered in the fix too

Yes, this and other cases where covered in https://github.com/fabric8io/kubernetes-client/pull/4295

The fix should be applied here once we update the Kuberentes Client version with the fix, which will probably be in v1.10.0

Is providing the fragment value as a scalar multi-line a workaround for this issue (it's likely it doesn't work, but it's worth a try)?

    spec:
      containers:
        - name: app
          env:
            - name: ENV
              value: local
            - name: SOME_DATE_ENV
              value: >
                2022-01-01

[msamad]

Yes, this and other cases where covered in fabric8io/kubernetes-client#4295 The fix should be applied here once we update the Kuberentes Client version with the fix, which will probably be in v1.10.0

Great, will wait for it, thanks.

Is providing the fragment value as a scalar multi-line a workaround for this issue (it's likely it doesn't work, but it's worth a try)?

    spec:
      containers:
        - name: app
          env:
            - name: ENV
              value: local
            - name: SOME_DATE_ENV
              value: >
                2022-01-01

It adds \n at the end. Tried to get rid of line ending but then end up with the issue again :(

when using >
            "name" : "SOME_DATE_ENV",
            "value" : "2022-01-01\n"

when using >-			
	    "name" : "SOME_DATE_ENV",
            "value" : "1640995200000"
			
when using |
            "name" : "SOME_DATE_ENV",
            "value" : "2022-01-01\n"
		
when using |-
            "name" : "SOME_DATE_ENV",
            "value" : "1640995200000"

Thanks

[manusa]

It adds \n at the end. Tried to get rid of line ending but then end up with the issue again :(

Yes, that's what I suspected :disappointed:

[rohanKanojia]

Since Kubernetes Client has been upgraded, I think this issue can be closed

[manusa]

We should close after the 1.9.0 release and validate that the issue is actually fixed.

[manusa]

Should be fixed in https://github.com/eclipse/jkube/releases/tag/v1.9.0

[rohanKanojia]

@pmbsa : Hello, Could you please try out v1.9.0 and provide us some feedback on whether your issue got fixed or not?

[pmbsa]

certainly, I will get on that today and get back to you.

[pmbsa]

Hi @rohanKanojia, I just introduced the new release and we have an issue it seams with the me.snowdrop istioclient (we have an istio enricher implementation)

Execution default-cli of goal org.eclipse.jkube:kubernetes-maven-plugin:1.9.0:resource fa
iled: An API incompatibility was encountered while executing org.eclipse.jkube:kubernetes-maven-plugin:1.9.0:resource: java.lang.NoSuchMethodError: me/snowdrop/istio/api/security/v1beta1/AuthorizationPolicySpecFluentImpl.bu
ild(Ljava/util/List;)Ljava/util/ArrayList; (loaded from file:/C:/Users/BOB/.m2/repository/me/snowdrop/istio-model/1.7.7.1/istio-model-1.7.7.1.jar by ClassRealm[plugin>org.eclipse.jkube:kubernetes-maven-plugin:1.9.0, par
ent: jdk.internal.loader.ClassLoaders$AppClassLoader@5df330d7]) called from class me.snowdrop.istio.api.security.v1beta1.AuthorizationPolicySpecFluentImpl (loaded from file:/C:/Users/BOB/.m2/repository/me/snowdrop/istio
-model/1.7.7.1/istio-model-1.7.7.1.jar by ClassRealm[plugin>org.eclipse.jkube:kubernetes-maven-plugin:1.9.0, parent: jdk.internal.loader.ClassLoaders$AppClassLoader@5df330d7]).

I hadnt seen this with the 1.9.0-SNAPSHOT build I had from some weeks ago, is this now as a resold of this fix or something completely different?

[rohanKanojia]

I don't think istio client dependency comes from jkube.

Istio Client is now available via Fabric8 https://search.maven.org/artifact/io.fabric8/istio-client/6.1.1/jar

[manusa]

We've bumped the Kubernetes Client dependency to 6.1.1 in this release. The older Istio Client (the one you are using) won't be compatible with the new Kubernetes Client version. Make sure to migrate to use the new extensions that's available along with the client (the one Rohan linked).

[pmbsa]

it it looking good thanks, the date looks to be as expected when delivered into the cluster. I have also uplifted the istio-api to use the fabric8 version and that looks all good thanks

[manusa]

Closing as fixed, thanks for the feedback!