jackson-databind
jackson-databind copied to clipboard
FasterXML
Deserialization issue: MismatchedInputException, Bean not yet resolved
Description
When I try to deserialize this object
{
"id": 1,
"fruits": [{
"id": 2,
"tree": 1,
"calories": [{
"id": 3,
"fruit": 2
}]
}]
}
I get this error
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot resolve ObjectId forward reference using property 'tree' (of type demo.jackson.Fruit): Bean not yet resolved
at [Source: (String)"{
"id": 1,
"fruits": [
{
"id": 2,
"tree": 1,
"calories": [
{
"id": 3,
"fruit": 2
}
]
}
]
}"; line: 15, column: 13] (through reference chain: demo.jackson.Fruit["tree"])
at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1482)
at com.fasterxml.jackson.databind.deser.BeanDeserializer$BeanReferring.handleResolvedForwardReference(BeanDeserializer.java:1090)
at com.fasterxml.jackson.databind.deser.impl.ReadableObjectId.bindItem(ReadableObjectId.java:64)
at com.fasterxml.jackson.databind.deser.impl.PropertyValueBuffer.handleIdValue(PropertyValueBuffer.java:251)
at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:207)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:424)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1322)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:331)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId(BeanDeserializerBase.java:1288)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:162)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4526)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3468)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3436)
at demo.jackson.LocationJavaTest.lambda$jsonDeserializeTree$0(LocationJavaTest.java:51)
at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:55)
at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:37)
at org.junit.jupiter.api.Assertions.assertThrows(Assertions.java:3082)
at demo.jackson.LocationJavaTest.jsonDeserializeTree(LocationJavaTest.java:50)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)
Version information
Current version: v2.11.4 It works with v2.11.3
Release note v2.11.4: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.11.4 Issue probably created by #2944
To Reproduce
Demo jackson_parsing.zip
the project contains a unit test in order to reproduce the bug
Other information
It might be related to https://github.com/FasterXML/jackson-databind/issues/2152 but it has been created 2 years before the version 2.11.4 has been released
I know there's link to full repro, but would it be possible to inline POJO class definition for target class? It'd be bit more convenient to see if this can be reproduced.
Especially as it was mentioned that @JsonCreator is being used -- which may be why this is not (and perhaps, cannot be) supported.
@cowtowncoder sure, these are the pojo classes (kotlin)
@JsonIdentityInfo(
generator = ObjectIdGenerators.PropertyGenerator::class,
property = "id",
scope = Tree::class,
)
data class Tree(
val id: Int,
var fruits: MutableList<Fruit> = mutableListOf(),
)
@JsonIdentityInfo(
generator = ObjectIdGenerators.PropertyGenerator::class,
property = "id",
scope = Fruit::class,
)
data class Fruit(
val id: Long,
var calories: List<Calories> = emptyList(),
) {
var tree: Tree? = null
}
@JsonIdentityInfo(
generator = ObjectIdGenerators.PropertyGenerator::class,
property = "id",
scope = Calories::class,
)
data class Calories(
val id: Int,
) {
var fruit: Fruit? = null
}
and this is the json I'm trying to deserialize
{
"id": 1,
"fruits": [{
"id": 2,
"tree": 1,
"calories": [{
"id": 3,
"fruit": 2
}]
}]
}
@detomarco does this still happen with 2.15? 2.11 is long out of support. Also, please try making a reproducer with java only instead of kotlin, to confirm whether this is an issue specific to the kotlin module
yes, still happening with the latest version.
I assumed that it depends on this module because I found the specific PR (https://github.com/FasterXML/jackson-databind/pull/2958) that breaks the code
i just converted your test case to java and it works fine:
public static void main(String[] args) throws JsonProcessingException {
new ObjectMapper().readValue("{\n" +
"\t\"id\": 1,\n" +
"\t\"fruits\": [{\n" +
"\t\t\"id\": 2,\n" +
"\t\t\"tree\": 1,\n" +
"\t\t\"calories\": [{\n" +
"\t\t\t\"id\": 3,\n" +
"\t\t\t\"fruit\": 2\n" +
"\t\t}]\n" +
"\t}]\n" +
"}", Tree.class);
}
@JsonIdentityInfo(
generator = ObjectIdGenerators.PropertyGenerator.class,
property = "id",
scope = Tree.class
)
public static class Tree {
private final int id;
private List<Fruit> fruits = Collections.emptyList();
@JsonCreator
public Tree(@JsonProperty("id") int id) {
this.id = id;
}
public int getId() {
return id;
}
public List<Fruit> getFruits() {
return fruits;
}
public void setFruits(List<Fruit> fruits) {
this.fruits = fruits;
}
}
@JsonIdentityInfo(
generator = ObjectIdGenerators.PropertyGenerator.class,
property = "id",
scope = Fruit.class
)
public static class Fruit {
private final long id;
private List<Calories> calories = Collections.emptyList();
private Tree tree;
@JsonCreator
public Fruit(@JsonProperty("id") long id) {
this.id = id;
}
public long getId() {
return id;
}
public List<Calories> getCalories() {
return calories;
}
public void setCalories(List<Calories> calories) {
this.calories = calories;
}
public Tree getTree() {
return tree;
}
public void setTree(Tree tree) {
this.tree = tree;
}
}
@JsonIdentityInfo(
generator = ObjectIdGenerators.PropertyGenerator.class,
property = "id",
scope = Calories.class
)
public static class Calories {
private final int id;
private Fruit fruit;
@JsonCreator
public Calories(@JsonProperty("id") int id) {
this.id = id;
}
public int getId() {
return id;
}
public Fruit getFruit() {
return fruit;
}
public void setFruit(Fruit fruit) {
this.fruit = fruit;
}
}
I will open the same issue in the kotlin module then. Thank you
Sounds like a plan👍🏼 Maybe we can reopen this issue if needed, @detomarco ? 😊
This is the issue I've open in the kotlin module https://github.com/FasterXML/jackson-module-kotlin/issues/681
I reopen since the errors occurs also in plain Java as described here https://github.com/FasterXML/jackson-module-kotlin/issues/681#issuecomment-1583006169
public class Main {
@JsonIdentityInfo(
generator = ObjectIdGenerators.PropertyGenerator.class,
property = "id",
scope = Tree.class
)
public static class Tree {
private final int id;
private List<Fruit> fruits;
@JsonCreator
public Tree(@JsonProperty("id") int id, @JsonProperty("fruits") List<Fruit> fruits) {
this.id = id;
this.fruits = fruits;
}
public int getId() {
return id;
}
public List<Fruit> getFruits() {
return fruits;
}
public void setFruits(List<Fruit> fruits) {
this.fruits = fruits;
}
}
@JsonIdentityInfo(
generator = ObjectIdGenerators.PropertyGenerator.class,
property = "id",
scope = Fruit.class
)
public static class Fruit {
private final int id;
private List<Calories> calories;
@JsonBackReference("id")
private Tree tree;
@JsonCreator
public Fruit(@JsonProperty("id") int id, @JsonProperty("calories") List<Calories> calories) {
this.id = id;
this.calories = calories;
}
public int getId() {
return id;
}
public Tree getTree() {
return tree;
}
public void setTree(Tree tree) {
this.tree = tree;
}
public List<Calories> getCalories() {
return calories;
}
public void setCalories(List<Calories> calories) {
this.calories = calories;
}
}
@JsonIdentityInfo(
generator = ObjectIdGenerators.PropertyGenerator.class,
property = "id",
scope = Calories.class
)
public static class Calories {
private final int id;
private Fruit fruit;
@JsonCreator
public Calories(@JsonProperty("id") int id) {
this.id = id;
}
public int getId() {
return id;
}
public Fruit getFruit() {
return fruit;
}
public void setFruit(Fruit fruit) {
this.fruit = fruit;
}
}
private static final ObjectMapper jackson = new ObjectMapper().findAndRegisterModules()
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
public static void main(String[] args) throws JsonProcessingException {
String json = "{" +
" \"id\": 1,\n" +
" \"fruits\": [\n" +
" {\n" +
" \"id\": 2,\n" +
" \"tree\": 1,\n" +
" \"calories\": [\n" +
" {\n" +
" \"id\": 3,\n" +
" \"fruit\": 2\n" +
" }\n" +
" ]\n" +
" }\n" +
" ]\n" +
" }";
Tree result = jackson.readValue(json, Tree.class);
System.out.println();
}
}
I've tried personally and the deserialisation works with version v2.11.3 and breaks since v2.11.4 till the latest one
Ok. One of first things to change in test would be:
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
since this setting invariably hides other issues. I also notice use of @JsonBackReference with @JsonIdentityInfo and am suspicious of these two working together (they are overlapping features).
From ref to 2.11.3/4, looking at release notes I agree that #2944 looks suspicious
Aside from that, WDYT @JooHyukKim ?
@detomarco Thanks for your time and efforts in discussing the kotlin-module issue. 🙏🏼
@JooHyukKim, what do you think?
Hmmm... The provided JSON object itself or the combination of @JsonBackReference and @JsonIdentityInfo is not something very unusual.
Then I made a PR in #3983 to provide a simplified test version that anyone can jump in and work on. While working on it, I noticed an excessive use of the value "id" and interestingly, the test without @JsonCreator passes in #3983.
So thinking futher on this, suspicions are..
- it possible that the
@JsonCreatormechanism consumes the JSON value of "id" before it reaches the@JsonBackReference + @JsonIdentityInfomechanisms, @cowtowncoder? - Or maybe just too many
"id"defined?
@JooHyukKim Combination of @JsonCreator (or rather use of constructor-passing) and @JsonIdentityInfo or @JsonBackReference is fundamentally challenging, because Object can only have identity after being constructed.
So there's a chance this could be potentially case that is unsupportable without major rewriting of Object Id handling -- or in some cyclic cases, if we had dependency between 2 Objects that require use of constructors, impossible to support.
But first things first; thank you for the test case, I merged it in.
@detomarco Please refer above for the analysis of current situation, esp. one about
@JooHyukKim Combination of @JsonCreator (or rather use of constructor-passing) and @JsonIdentityInfo or @JsonBackReference is fundamentally challenging,
....... part. @detomarco your use case might not be possible to support 🤔. If you would like to tackle the problem, here is test case #3983
Or my other (careful) suggestion is removing @JsonCreator part like here
May I suggest changing the issue title as...
`@JsonCreator` combined with `@JsonIdentityInfo`(or `@JsonBackReference`) fails deserialization with `MismatchedInputException : Bean not yet resolved`
.... this, @cowtowncoder ? For issue tracking, etc...
Thank you so much for your investigations guys!!
Sorry but I'm a bit lost :D
In my understanding the problem is the combination of several annotations. But in my original Kotlin code I did not add any of them, so I assume the problem is again in the jackson-kotlin module which adds those annotations. Did I get it right? In that case I will reopen the issue there again
@JooHyukKim Combination of
@JsonCreator(or rather use of constructor-passing) and@JsonIdentityInfoor@JsonBackReferenceis fundamentally challenging, because Object can only have identity after being constructed. So there's a chance this could be potentially case that is unsupportable without major rewriting of Object Id handling -- or in some cyclic cases, if we had dependency between 2 Objects that require use of constructors, impossible to support.But first things first; thank you for the test case, I merged it in.
it's not really clear to me why could be impossible to support since it was working well till v2.11.3
But in my original Kotlin code I did not add any of them, so I assume the problem is again in the jackson-kotlin module which adds those annotations. Did I get it right?
Yes, it "seems" like kotlin-module is where the annotation is added --sadly I have no knowedge how Kotlin at the moment.
In that case I will reopen the issue there again
That will be awesome, thank you again @detomarco !
it's not really clear to me why could be impossible to support since it was working well till v2.11.3
Sadly, though the community tries the best to not introduce regression (if this is one), it can happen.
There is nothing that kotlin-module can do to fix this problem.
The kotlin-module basically marks the constructor as JsonCreator.
This is because if you simply use the data class, there are no argumentless constructors or setters there (I recommend checking the decompiled results).
And kotlin-module does not do any processing related to JsonIdentityInfo.
Ok so this may be a limitation of Kotlin Data classes (and similar immutable value classes like Java Records) wrt @JsonIdentityInfo -- not necessarily in all cases, but for some, in particular references between 2 immutable value classes.
I am not saying this is definitely the case here but as a sort of concern that it might be.
In plain Java one way to work around issues like this are to make Object Id properties mutable, not passed via constructor (so they can be "late-bound" in a way, to remove cycle) but this is not available for immutable value classes.
@detomarco It is not clear if this can be fixed or not; is it a fundamental limitation, or something that could be fixed.
@detomarco So it seems like kotlin-module may not have solution for this. And there is a workaround in plain Java. But no further suggestion for resolution seems to have been made atm.
Observation
Reverting #2958 will indeed make the error go away, so this is confirmed to be the cause (EDIT: not suggesting to revert that to fix this)
Workaround
Let Tree be created first before constructing Tree.fruits, by removing the latter from Tree's main constructor:
@JsonIdentityInfo(generator = ObjectIdGenerators.PropertyGenerator::class, property = "id", scope = Tree::class)
data class Tree(val id: Int) {
// Move here, just like Fruit.tree & Calories.fruit
var fruits: MutableList<Fruit> = mutableListOf()
}
@yihtserns Presumable breaking #2944 again tho? But perhaps this would allow finding a better fix that'd work for both cases?
@cowtowncoder I'm not proposing to revert that, am just double-confirming @detomarco's suspicion. 😅
Sorry for the misleading statement (edited).
@yihtserns yeah did not mean to imply that, just sort of spelled out what I think is the case. But I think it's a valid question as to whether original change could be modified.
Overall this once again leads back to my suspicion that many problems are due to incorrect linkage between CreatorProperty and other (Setter/getter/field) SettableBeanPropertyies.
@yihtserns thanks for your advice, the workaround worked.
However I would suggest to find a proper solution to this problem because it prevents proper use of data classes in Kotlin or records in java 14+, and can become a big problem once they are adopted more and more