jackson-databind
jackson-databind copied to clipboard
FasterXML
@JsonUnwrapped not supported for Map-valued properties
According to the documentation,
Annotation used to indicate that a property should be serialized "unwrapped"; that is, if it would be serialized as JSON Object, its properties are instead included as properties of its containing Object.
Unfortunately, this seems to work only with bean types, and does not work with Map<String, Object>. Given that bean types are generally completely interchangeable with Maps, it would be very nice if this worked correctly.
Hmmh. Unfortunately Maps and POJOs are internally handled rather differently. But I am all for unification if it is technically possible, so limitation is not philosophical, just practical. I suspect that serialization part should be easy enough to make work; deserialization might get trickier?
:-)
One quick question: I assume most users would expect it to work both ways. But I suspect serialization is much easier to make work. Would you find any value in serialization-only impl? (especially initial one)
One more thing: as per this:
http://www.cowtowncoder.com/blog/archives/2011/07/entry_458.html
one can use @JsonAnyGetter/setter to do something possibly similar. One missing pieces is that currently one must have getter (can't use it on Map filed), but that should be easy enough to address.
Yes, a serialization-only implementation would solve my immediate use case, although I can see the deserialization as being extremely useful as well.
It does seem that this could potentially be done with serialization-only JsonUnwrapped usage and then JsonAnySetter to handle the deserialization case, although that feels more than a little bit janky to me.
Yes my concern is really with unexpected one-way street: unwrapping on way out, but not "wrapping it back" as Map on way back in.
Ok, I think supporting this would be very useful for CSV, f.ex see https://github.com/FasterXML/jackson-dataformat-csv/issues/25 so maybe I should go back, try to tackle this.
Although, with CSV there are other open questions due to name/index mapping. But still, solving this on databind side could help.
A related note for anyone who happens upon this issue: one alternative is use of @JsonAnyGetter, which does allow functionality for a single Map.
I find this behavior surprising at a conceptual level, since it seems to be contrary to how Jackson behaves in general.
Would I be correct in assuming that this is one of the features that will likely not be in the next Jackson 2.9 release, per the Changes likely to be postponed section?
@matthew-pwnieexpress You are correct in that no work is planned for this feature. I can see how this is unexpected from users perspective: difference is due to technical evolution of backend implementation where Maps and POJOs have very different handling. But conceptually this should not surface quite this strongly, or, if it does, would need to be explained and documented much better.
@cowtowncoder Given that @JsonAnyGetter and @JsonAnySetter exist, how correct/incorrect would an implementation be to have @JsonUnwrapped imply the other two annotations when used with MapSerializer?
@henryptung Interesting thought... hmmh. There is also then the possible question of what if more than one such "any property" is declared. I guess this is not problematic with POJOs as their properties are somewhat defined and multiples unwrapped POJOs may co-exist; whereas any-x is fallback.
Unfortunately the @JsonAnyGetter does not work (easily) with Kotlin:
class Links(
@JsonAnyGetter
val links: Map<String, Link>
) : Serializable
as the annotation needs to put on a method.
!! UPDATE !!
It does work (easily):
class Links(
@get:JsonAnyGetter
val links: Map<String, Link>
) : Serializable
True; as of now (2.9), @JsonAnyGetter must be on method. In theory it could be extended to fields, to allow construction of Map instance. One challenge would be the fact that semantics would be slightly different -- in one case method gets fed key/value pair: that could lead to confusion (unless logic further added to perhaps also allow single Map-argument... but that can lead to another can of worms).
EDIT (2023-05-15): @JsonAnyGetter is available on Fields too (see #1458) since 2.12.
We face the challenge that we provide a wrapper type for some content object that could either be a custom object or a Map:
class SomeWrapper<T> {
T content;
@JsonUnwrapped
public getContent() {
return content;
}
}
Is there something we could do using a custom serializer to not have to add the extra method or extra class?
@odrotbohm I can't think of anything: @JsonUnwrapped is quite tied to way BeanSerializer and -Deserializer works and although one can override handlers I am not sure custom (de)serializer route would lead to anything but fragile solution.
I have a POC with the following steps:
- register a custom
StdSerializer<SomeWrapper> - in that, inspect the content. If it's a
Maptake the content and wrap it into a type using@JsonAnyGetter, default serialize that. If no, wrap into an almost copy ofSomeWrapperusing a plain@JsonUnwrapped
That way, SomeWrapper stays the only user facing API but does the right thing™ during serialization.
Incidentally, the Stack Overflow question for this issue has 46 upvotes, and its top answer has 101 upvotes, so this is definitely an issue users have been encountering.
Ok, marking this as "most-wanted" as there are a few thumbs-ups here too.
About the only question I have is whether there are some specific differences between just using @JsonAnyGetter/@JsonAnySetter combo, and potential @JsonUnwrapped.
Using one less annotation seems like nice-but-not-essential; consistency a minor plus too.
Or put another way: if @JsonUnwrapped was essentially implemented as sort of alias for "any-getter", would that work?
(another potential concern: can only have one "any getter/setter" per class -- but multiple @JsonUnwrappeds)
It's also very surprising it doesn't work for ObjectNodes, e.g., the trivial case JsonNodeFactory.instance.objectNode().put("sample", 1). I would guess that's because under the covers, ObjectNode is a map, but to someone writing that code, it's just a json object. And the @JsonAnyGetter trick doesn't work for this.
@Vroo Conceptually, JsonNode types are not POJOs, so most annotations do not have any effect by design. It would be good to document this better as I can see why it might be surprising... there isn't much documentation on concept, intended differences. In fact, POJOs, "untyped" (Object / List / Map), Trees (JsonNode) are all somewhat different models within Jackson, and while they interoperate fine, they are not treated the same way.
@Vroo that said, I can see why specific cases of @JsonAnyGetter/@JsonAnySetter would make sense for JsonNode/ObjectNode values -- if so, feel free to file an issue to add support for that usage.
@JsonUnwrapped is a bit trickier just because the whole machinery for it to work is... rather complicated and fragile: you could file an issue for that, too (since adding support for Maps would be technically different from JsonNode; there isn't much synergy in getting both implemented)
About the only question I have is whether there are some specific differences between just using
@JsonAnyGetter/@JsonAnySettercombo, and potential@JsonUnwrapped. ... Or put another way: if@JsonUnwrappedwas essentially implemented as sort of alias for "any-getter", would that work?
I have the same problem as @odrotbohm: @JsonUnwrapped sits on top of a generic. To get it to work, I've needed to create subclass that overrides the getter so I can add @JsonAnyGetter. The any-getter doesn't work because it's of type T, not a Map and it throws an error during runtime otherwise.
This creates other problems because I've subclassed it. Being able to use @JsonUnwrapped would be much easier, cleaner and safer.
I had a go about this and could get Unwrapping aspect working. The JsonSerializer<T> has a unwrappingSerializer method that is called within UnwrappingBeanPropertyWriter that sort of lifts the serializer into a unwrapping serializer. When we try to serialize a Map instance a MapSerializer is responsible for the serialization step. Overriding unwrappingSerializer for MapSerializer to create a UnwrappingMapSerializer should give us what we want.
@Override
public JsonSerializer<Map<?, ?>> unwrappingSerializer(NameTransformer transformer) {
return new UnwrappingMapSerializer(this, transformer);
}
The sole responsibility of an UnwrappingMapSerializer is making MapSerializer's keySerializer an unwrapping serializer.
this._keySerializer = this._keySerializer.unwrappingSerializer(transformer);
This changes the result of TestUnwrappedMap171.testMapUnwrapSerialize
junit.framework.ComparisonFailure:
Expected :{"map.test": 6}
Actual :{"test":6}
As you can see unwrapping worked but the name transformer did not have any effect as StdKeySerializers do not have a rename concept as BeanPropertyWriters do.
Hi, I'd like to add here my use-case:
ugly Data
@JsonAutoDetect(fieldVisibility = JsonAutoDetect.Visibility.ANY)
public static final class Data
{
@JsonProperty("Status")
private String status;
@JsonAnyGetter
private Map<String, Object> otherFields = new LinkedHashMap<>();
public Data(
final String status,
final Map<String, Object> otherFields
)
{
this.status = status;
this.otherFields = otherFields;
}
public Data()
{
}
@JsonAnySetter
public void anySetter(final String key, final Object value)
{
otherFields.put(key, value);
}
@Override
public boolean equals(final Object o)
{
if (this == o) {
return true;
}
if (!(o instanceof Data data)) {
return false;
}
return status.equals(data.status) && otherFields.equals(data.otherFields);
}
@Override
public int hashCode()
{
return Objects.hash(status, otherFields);
}
@Override
public String toString()
{
return "{status='" + status + '\'' + ", otherFields=" + otherFields + '}';
}
}
pretty Data
public record Data(
@JsonProperty("Status") String status,
@JsonUnwrapped Map<String, Object> otherFields
)
{
}
OtherFieldsJacksonTest
class OtherFieldsJacksonTest
{
private final ObjectMapper jsonObjectMapper = Jackson2ObjectMapperBuilder.json()
.featuresToEnable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
.build();
@Test
public void serialize() throws Exception
{
var other = new LinkedHashMap<String, Object>();
other.put("list", List.of(1, 2, 3));
other.put("Id", "1234");
other.put("object", Map.of("foo", "1234"));
Data input = new Data("Ok", other);
String result = jsonObjectMapper.writeValueAsString(input);
String expected = "{\"Status\":\"Ok\",\"list\":[1,2,3],\"Id\":\"1234\",\"object\":{\"foo\":\"1234\"}}";
Assertions.assertEquals(expected, result);
}
@Test
public void deserialize() throws Exception
{
String json = "{\"Status\":\"Ok\",\"object\":{\"foo\":\"1234\"},\"Id\":\"1234\",\"list\":[1,2,3]}";
Data result = jsonObjectMapper.readValue(json, Data.class);
var other = new LinkedHashMap<String, Object>();
other.put("list", List.of(1, 2, 3));
other.put("Id", "1234");
other.put("object", Map.of("foo", "1234"));
Data expected = new Data("Ok", other);
Assertions.assertEquals(expected, result);
}
}
The test passes with "ugly Data", but obviously not with "pretty Data".
I'd like to collect the extra fields into a separate Map, then do some work with the data object, then serialize it back to json string. I don't need them for anything, but I want to preserve them if I later decide I do need them.
When I define the record like this, the serialization works as expected and deserialization doesn't throw errors, but it puts null into the otherFields field.
pretty Data
public record Data(
@JsonProperty("Status") String status,
@JsonUnwrapped @JsonAnyGetter @JsonAnySetter Map<String, Object> otherFields
)
{
}
I've done some more googling and this is basically the same https://github.com/FasterXML/jackson-databind/issues/3439#issue-1190261041, sorry for a duplicate post
@fprochazka Please don't add things that are not relevant for the specific issues. For usage questions, mailing list:
https://groups.google.com/g/jackson-user
works wonders. For issues to report, file a new one UNLESS you have something specific to add into existing one.
@cowtowncoder sorry, I thought it is relevant to this issue, since I didn't find the sample anywhere previously
Hmmh. Well, looking at it, it's a more complicated case where Record is also problematic, as well as any setter -- I guess my point was that it does not help solve the basic problem wrt Map valued properties.
But I understand you wanted to help with additional information so I guess that is ok.