MINIMIZE_QUOTES Question

[TheSecMaven]

trafficstars

Using MINIMIZE QUOTES with a custom serializer for YAML. I try this in the custom serializer

      jsonGenerator.writeString(resource.getId());

which outputs

   -appsec1234/34f3678c-1de8-45a4-a50d-3bb7c22b38/6f8caacc-b83c-4d55-993c-ad70ca5c1392

But then, if I add this

          jsonGenerator.writeString(resource.getTypeID() + " " + resource.getId());

I get

- '!host appsec1234/34f3678c-1de8-45a4-a50d-3bb7c22b38/6f8caacc-b83c-4d55-993c-ad70ca5c1392'

I need this to get output without quotes, am I missing something with how I pass these characters in? I read online about special characters causing this, can I serialize these in some way maybe to fix that issue?

[TheSecMaven]

I tried jsonGenerator.writerawvalue(resource.getTypeID() + " " + resource.getId());

but then get an error because YAMLGenerator doesn't recognize that as a valid value for some reason

[TheSecMaven]

definitely seems related to the !, as I tried this

          jsonGenerator.writeString("test " + resource.getId());

and it output

 - test appsec1234/34f3678c-1de8-45a4-a50d-3bb7c22b38/6f8caacc-b83c-4d55-993c-ad70ca5c1392

[TheSecMaven]

I know ! is considered a custom type in YAML, which is how we are using it (hence its getTypeID() and specified as a JsonTypeName using the decorator. but the requirement is to have the type and its ID (getID()) on the same line.

[cowtowncoder]

As of Jackson 2.12, definition of what must be quoted is handled by StringQuotingChecker implementation (by default, StringQuotingChecker.Default). I don't see exclamation mark on the list, but it seems likely that SnakeYAML encoder might prevent that -- this because it correctly (I think?) deduces that ! is indeed a type id and must be escaped.

I'll quickly verify this locally to make sure.

[cowtowncoder]

Yup. Jackson yaml module tries to output as "plain", does not consider ! special, but SnakeYAML is not fooled and forces quotes to prevent inaccurate decoding as "tag" (type id). You cannot force tag like that.

So your question is not about MINIMIZE_QUOTES as much as type id?

Usually with Jackson this should be done by forcing type Id, but that may be tricky for type String which by design cannot take type id (one of four "natural" types).

[TheSecMaven]

Yeah, it makes sense after documenting a little more.

What am I missing here? I'm trying to basically get it to write the type Id without quotes, and then this field within the object. I can get it to do the field, but seem to be missing some way for yaml generator to know I need it to write the type ID, I think.

[cowtowncoder]

Jackson writes type ids based on @JsonTypeInfo annotation, although that is meant for polymorphic types, and does require use of POJOs. If that was an option, you'd also need to enable

YAMLGenerator.Feature.USE_NATIVE_TYPE_ID

to indicate that YAML's tag system is to be used instead of otherwise default Jackson use of separate property. Unfortunately much of this is designed to allow reading of tags, and less for writing tags.

But a custom serializer could be written to make it bit simpler. It'd do something like:

public void serialize(String value, JsonGenerator g, SerializerProvider prov) throws IOException {
  g.writeTypeId("host"); // not 100% if that should be "!type"...
  g.writeString(value);
}

and add annotation to indicate its use for the field in question (or for custom type if you use something other than String).

I have not tested this, but conceptually this should work. And if you do have time to play with it against 2.12.2, I would be interested in resolving issues if there are any (since it's not tested I am guessing there's a good chance there could be edge cases).

This seems like something that definitely should be possible: and at least first if a custom serializer approach worked, that'd be a good start I think?

[TheSecMaven]

we are using polymorphic type, and have set @jsontypeInfo on the class that is being passed our custom serializer. I enabled that feature you mentioned on the YAMLGenerator, but am getting

2021-03-17T11:20:16.378-05:00 [APP/PROC/WEB/0] [ERR] com.fasterxml.jackson.core.JsonGenerationException: No native support for writing Type Ids
2021-03-17T11:20:16.378-05:00 [APP/PROC/WEB/0] [ERR] at com.fasterxml.jackson.core.JsonGenerator.writeTypeId(JsonGenerator.java:1486)
2021-03-17T11:20:16.378-05:00 [APP/PROC/WEB/0] [ERR] at az.it.appsec.conjur.builder.models.policy.RoleCustomSerializer.serializeWithType(RoleCustomSerializer.java:47)
2021-03-17T11:20:16.378-05:00 [APP/PROC/WEB/0] [ERR] at az.it.appsec.conjur.builder.models.policy.RoleCustomSerializer.serializeWithType(RoleCustomSerializer.java:22)

where RoleCustomSerializer is doing

                jsonGenerator.writeTypeId(role);

and role is a polymorphic type with

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = As.PROPERTY)

I also tried

  g.writeTypeId(role.getTypeId()); 

Where getTypeID() returns a string field that is annotated @JsonTypeId, that may be a red herring but wanted to share.

[cowtowncoder]

@mkkeffeler Ok. I would be interested in a code snippet, or test case: this sounds like something I could/should improve upon. I assume this is with 2.12.2?

[TheSecMaven]

would love to show you real fast as well, its possible im doing something just slightly different. but yes, I upgraded from 2.12.1 to 2.12.2, same issue.

[TheSecMaven]

Can someone help me understand why I get that error? "com.fasterxml.jackson.core.JsonGenerationException: No native support for writing Type Ids"?

According to the function that checks canWriteTypeId, it just depends on the USE_NATIVE_TYPE_ID feature being enabled, like @cowtowncoder said. But I enabled that only on the YAMLFactory, not JSONGenerator. is that the issue? some way of updating that, perhaps?

[cowtowncoder]

@mkkeffeler If you can give a basic reproduction, I can have a look. Factory should pass relevant settings to generator, yes; just make sure your ObjectMapper uses specific factory you have configured.

[TheSecMaven]

So remember I said this.

I tried jsonGenerator.writerawvalue(resource.getTypeID() + " " + resource.getId());

but then get an error because YAMLGenerator doesn't recognize that as a valid value for some reason

The error was that the generator didn't recognize that as a valid way to start an object. So, I found this

        typeSer.writeTypePrefix(jsonGenerator, typeSer.typeId(resource, JsonToken.START_OBJECT));

Which writes the type of the object out, AND tells the generator that this is a START_OBJECT identifier. This ended up being one way I worked around the above issue.