Yaml.dump and Yaml.loadAs log warnings related to V1JSONSchemaProps

[psjamesh]

trafficstars

Describe the bug When using the Yaml.dump method to serialize a K8S object (eg - V1Pod) several warnings related to CRD schema attributes are output from snakeyaml.

The same issue occurs when using the Yaml.loadAs method (which is actually the method I am using in production).

See example code and output below.

Client Version 18.0.1

Kubernetes Version N/A

Java Version Java 17

To Reproduce Run the YAML code example

Expected behavior The output should not contain warnings related to missing schema attributes for CRDs.

KubeConfig N/A

Operating System Linux

Additional context The (example) code I am running is below:

public class YamlExample {
    public static void main(String[] args) throws IOException, ApiException, ClassNotFoundException {
        V1Pod pod =
                new V1PodBuilder()
                        .withNewMetadata()
                        .withName("apod")
                        .endMetadata()
                        .withNewSpec()
                        .addNewContainer()
                        .withName("www")
                        .withImage("nginx")
                        .withNewResources()
                        .withLimits(new HashMap<>())
                        .endResources()
                        .endContainer()
                        .endSpec()
                        .build();

        System.out.println(Yaml.dump(pod));
    }
}

The output from the example code is below:

Jul 27, 2023 1:23:33 PM org.yaml.snakeyaml.internal.Logger warn
WARNING: Failed to find field for io.kubernetes.client.openapi.models.V1JSONSchemaProps.x-kubernetes-embedded-resource
Jul 27, 2023 1:23:33 PM org.yaml.snakeyaml.internal.Logger warn
WARNING: Failed to find field for io.kubernetes.client.openapi.models.V1JSONSchemaProps.x-kubernetes-int-or-string
Jul 27, 2023 1:23:33 PM org.yaml.snakeyaml.internal.Logger warn
WARNING: Failed to find field for io.kubernetes.client.openapi.models.V1JSONSchemaProps.x-kubernetes-list-map-keys
Jul 27, 2023 1:23:33 PM org.yaml.snakeyaml.internal.Logger warn
WARNING: Failed to find field for io.kubernetes.client.openapi.models.V1JSONSchemaProps.x-kubernetes-list-type
Jul 27, 2023 1:23:33 PM org.yaml.snakeyaml.internal.Logger warn
WARNING: Failed to find field for io.kubernetes.client.openapi.models.V1JSONSchemaProps.x-kubernetes-map-type
Jul 27, 2023 1:23:33 PM org.yaml.snakeyaml.internal.Logger warn
WARNING: Failed to find field for io.kubernetes.client.openapi.models.V1JSONSchemaProps.x-kubernetes-preserve-unknown-fields
metadata:
  name: apod
spec:
  containers:
  - image: nginx
    name: www
    resources:
      limits: {}


Process finished with exit code 0

[mnlipp]

The warnings come from Yaml.newGsonCompatibleTypeDescription which calls org.yaml.snakeyaml.TypeDescription.substituteProperty for all x-kubernetes extensions, no matter if they exist for a given type or not (e.g. it is called for List).

This is very annoying, because you get warnings despite everything being okay.

[brendandburns]

We'd be happy to take a PR to improve this behavior.

[rstar2]

The problem seems that the extension names are not in "camelCase" format. They are used like

        new String[] {
          "x-kubernetes-embedded-resource",
          "x-kubernetes-int-or-string",
          "x-kubernetes-list-map-keys",
          "x-kubernetes-list-type",
          "x-kubernetes-map-type",
          "x-kubernetes-preserve-unknown-fields",
        };

Which are the GSON serialized-names of the fields in V1JSONSchemaProps, like:

public static final String SERIALIZED_NAME_X_KUBERNETES_INT_OR_STRING = "x-kubernetes-int-or-string";
  @SerializedName(SERIALIZED_NAME_X_KUBERNETES_INT_OR_STRING)
  private Boolean xKubernetesIntOrString;

but the org.yaml.snakeyaml.TypeDescription.substituteProperty uses the field names as by reflection - e.g. it searched for fields named x-kubernetes-int-or-string and etc... which are obviously not valid V1JSONSchemaProps field names

The Yaml.newGsonCompatibleTypeDescription() should call not

desc.substituteProperty(
          targetGsonAnnotation, field.getType(), getterMethod.getName(), setterMethod.getName());

but

desc.substituteProperty(
          field.getName(), field.getType(), getterMethod.getName(), setterMethod.getName());

[AlbertoEAF]

We've been observing this for a few months as well and it always seems concerning specially when using this in production systems. Is there going to be a fix for this? And is there at least a guarantee this won't be an issue?

Thank you

[shamoh]

Hello. Unfortunately, currently the last version 19.0.0 has same issue.

[brendandburns]

As mentioned earlier, we'd be happy to take a PR to improve this if someone wants to make one.

[payalsaraljain]

Hi, I am interested in taking up this issue, can u please assign this issue to me?

[shamoh]

Until fixed I use following workaround:

java.util.logging.Logger.getLogger("org.yaml.snakeyaml.introspector").setLevel(java.util.logging.Level.SEVERE)

[Theboycut05]

hey i'm intrested to take this issue, would you like to assign this issue to me ?

[nivsa]

Any updates on this ticket? Tried the workaround from the comment above but it didn't work for us

[SanjayVas]

I believe the workaround may now need to use org.yaml.snakeyaml.internal as the logger name. Note that the main problem with this workaround is it suppresses warnings for all callers of snakeyaml, e.g. if your application uses snakeyaml outside of the Kubernetes Java client.

[cowwoc]

@brendandburns It looks like we had a PR from @scovl but no one took a look at it...? Or did I miss something?

[cowwoc]

@payalsaraljain @Theboycut05 Are you still interested in taking this on?

[karthikkondapally]

@brendandburns

What do you think about temporarily increasing the log level of SnakeYAML to SEVERE and then restoring it to its original level after the operation is done?

this just avoids WARNING log level present in snake yaml https://github.com/snakeyaml/snakeyaml/blob/4795519ef773da23c8816a4e932f9aaffd630f8c/src/main/java/org/yaml/snakeyaml/introspector/PropertySubstitute.java#L172

This approach seems to suppress the warnings and could serve as a temporary fix to reduce log noise. until some better solution is found for parity b/w x-kubernetes-<> xKubernetes<> fields in V1JSONSchemaProps

public static TypeDescription newGsonCompatibleTypeDescription(
      Class modelClass, String... gsonTaggedFields) {
    Level originalLevel = java.util.logging.Logger.getLogger("org.yaml.snakeyaml.introspector").getLevel();
    java.util.logging.Logger.getLogger("org.yaml.snakeyaml.introspector").setLevel(java.util.logging.Level.SEVERE);
          .
          .
          . 
    java.util.logging.Logger.getLogger("org.yaml.snakeyaml.introspector").setLevel(originalLevel);
    return desc;
  
  }

[asomov]

if you think that it should be solved in SnakeYAML - proposals are welcome!

[karthikkondapally]

Thanks to @asomov and alexander maslov this has been fixed in sankeyaml and it will be released in 2.5

[mattstrain]

As this has now been fixed in SnakeYaml 2.5 can the version in pom.xml get bumped please?

[karthikkondapally]

with bump in snakeyaml version to 2.5, warnings are not visible anymore. we can close this issue now. @brendandburns @psjamesh