Deserialization regression: MismatchedInput No Object Id found for an instance of X to assign to property '@id'

[susanw1]

Search before asking

• [X] I searched in the issues and found nothing similar.

Describe the bug

Upgrading from 2.15.4 to 2.16.0, my deserializer stopped working, and now throws MismatchedInputException:

com.fasterxml.jackson.databind.exc.MismatchedInputException: 
No Object Id found for an instance of `com.fasterxml.jackson.module.mrbean.generated.jacksonissue.MiniModuleDef$NumberTypeDefinition`, to assign to property '@id'
 at [Source: (URL); line: 6, column: 1] (through reference chain: com.fasterxml.jackson.module.mrbean.generated.jacksonissue.MiniModuleDef$ModuleModel["commands"]->java.util.ArrayList[0]->com.fasterxml.jackson.module.mrbean.generated.jacksonissue.MiniModuleDef$CommandModel["typeDefinition"])
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)

Key points:

• I'm using MrBean to generate the beans.
• It's a polymorphic subtype that's failing, using an @type identifier
• The failure is happening on the new code that was added for #3838 and PR #3868 (the new if-statement in BeanDeserializer.java): if (_objectIdReader != null && p.getCurrentToken() == JsonToken.END_OBJECT) {...
• Problem was detected in YAML, but also affects JSON deserialization. Example has tests for both.

Here's the part of the definition that's failing:

    @JsonIdentityInfo(generator = ObjectIdGenerators.StringIdGenerator.class)
    @JsonTypeInfo(use = JsonTypeInfo.Id.NAME)
    @JsonSubTypes({
            @Type(value = EnumTypeDefinition.class, name = "enum"),
            @Type(value = NumberTypeDefinition.class, name = "number")
    })
    interface TypeDefinition {
    }
    interface NumberTypeDefinition extends TypeDefinition {
    }

when deserializing this fragment:

      "typeDefinition": {
        "@type": "number"
      }

It seems to be related to the empty subtype interface NumberTypeDefinition. If I add a dummy field to it (or to the parent TypeDefinition interface), AND set that field, then the test passes. I think the BeanDeserializer is incorrectly rejecting empty subtypes.

Also, it's the JSON that needs the dummy field, not just the object model - just adding it to the model with required=false is insufficient.

Version Information

2.16.0

Reproduction

Minimal reproducible example here: https://github.com/susanw1/jackson-issue

Results are consistent using mvn on command line or IntelliJ.

If you adjust the pom.xml (lines 12/13) to use <version.jackson>2.15.4</version.jackson>, the error goes away.

Expected behavior

Test is expected to work, and can be made to do so by reverting the version to 2.15.4.

Additional context

This is the complete exception trace:

com.fasterxml.jackson.databind.exc.MismatchedInputException: 
No Object Id found for an instance of `com.fasterxml.jackson.module.mrbean.generated.jacksonissue.MiniModuleDef$NumberTypeDefinition`, to assign to property '@id'
 at [Source: (URL); line: 6, column: 1] (through reference chain: com.fasterxml.jackson.module.mrbean.generated.jacksonissue.MiniModuleDef$ModuleModel["commands"]->java.util.ArrayList[0]->com.fasterxml.jackson.module.mrbean.generated.jacksonissue.MiniModuleDef$CommandModel["typeDefinition"])
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1781)
	at com.fasterxml.jackson.databind.DeserializationContext.reportUnresolvedObjectId(DeserializationContext.java:1728)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:375)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId(BeanDeserializerBase.java:1385)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:218)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:170)
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:136)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithType(BeanDeserializerBase.java:1306)
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:542)
	at com.fasterxml.jackson.databind.deser.impl.ObjectIdReferenceProperty.deserializeSetAndReturn(ObjectIdReferenceProperty.java:94)
	at com.fasterxml.jackson.databind.deser.impl.ObjectIdReferenceProperty.deserializeAndSet(ObjectIdReferenceProperty.java:87)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:359)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28)
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545)
	at com.fasterxml.jackson.databind.deser.impl.ManagedReferenceProperty.deserializeAndSet(ManagedReferenceProperty.java:62)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4899)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3777)
	at jacksonissue.ModuleTest.shouldHandleTypeDefinitionYaml(ModuleTest.java:30)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)

[cowtowncoder]

Sounds like a bug indeed. Thank you for reporting this @susanw1 -- and for verifying it is reproducible with JSON (and not only for YAML as originally mentioned).

[JooHyukKim]

Thank you @susanw1 !

[JooHyukKim]

I wrote simplified reproduction via #4608. Like @susanw1 said, we need to modify the check in BeanDeserializer so that it only fails when objectId is required, not like issue here where we are looking at type-id. Which I have not found solution to, yet.

if (_objectIdReader != null && p.getCurrentToken() == JsonToken.END_OBJECT) {...

[cowtowncoder]

I am not sure this is valid usage, actually: why is @JsonIdentityInfo being added when no Object Id is to be used? What is the intention here? (could the annotation just be removed)

[susanw1]

Thanks for looking into this!

I think it needs the @JsonIdentityInfo annotation for the YAML case where it uses anchors and aliases:

-   name: cmd2
    typeDefinition: &x
        '@type': enum
        values:
        - foo
        - bar
-   name: cmd3
    typeDefinition: *x

If I take out the @JsonIdentityInfo, I start getting:

com.fasterxml.jackson.databind.exc.InvalidTypeIdException: Could not resolve subtype of [simple type, class jacksonissue.MiniModuleDef$TypeDefinition]: missing type id property '@type' (for POJO property 'typeDefinition')
 at [Source: (URL); line: 13, column: 21] (through reference chain: com.fasterxml.jackson.module.mrbean.generated.jacksonissue.MiniModuleDef$ModuleModel["commands"]->java.util.ArrayList[2]->com.fasterxml.jackson.module.mrbean.generated.jacksonissue.MiniModuleDef$CommandModel["typeDefinition"])

Looks like it's failing on the *x because it can't link back to the anchor and therefore can't see the @type any more.

[JooHyukKim]

I am not sure this is valid usage, actually: why is @JsonIdentityInfo being added when no Object Id is to be used? What is the intention here? (could the annotation just be removed)

@cowtowncoder makes a valid point. The solution specific to current issue is to remove @JsonIdentityInfo that is not needed to deserialize TypeDefinition and its subtypes.

[susanw1]

@JooHyukKim Yep, I tried that, but that causes InvalidTypeIdException (both in 2.15.4 and 2.16) for the YAML anchor case. I think the @JsonIdentityInfo is involved with how YAML aliases find their anchor (I think?).

Just to note, the empty, field-less interfaces were just stripped down for the example. In my real code, there are many of my TypeDefinition subtypes mostly with optional fields. The regression is triggered for the common case where the YAML doesn't need to stipulate values for any of the fields and the block is empty (apart from @type!). Some of my TypeDefinitions are pretty big - lots of enum values, f'rinstance - and are used in several places, hence use of anchors...

In actual use:

• YAML: https://github.com/susanw1/zscript/tree/master/model/standard-module-definitions/src/main/resources/zscript-datamodel/00xx-base-modules and
• Model: https://github.com/susanw1/zscript/blob/master/model/schema/src/main/java/net/zscript/model/datamodel/ZscriptDataModel.java

[cowtowncoder]

Ok, I think our reproduction may be wrong then: in case YAML, there is NATIVE Object Id and Type Id, and to make use of Object Ids (native or property-based, @JsonIdentityInfo is indeed needed.

JSON reproduction is missing property.

It is possible we might need to use YAML reproduction and maybe get rid of failing JSON reproduction.

[cowtowncoder]

Ok so fix to #4610 looks like it is somewhat related, merged now. Although it only affects case of DeserializationFeature.FAIL_ON_UNRESOLVED_OBJECT_IDS being disabled (defaults to enabled).

[cowtowncoder]

Hmmmmh. Looking at code in BeanDeserializer:

        if (p.canReadObjectId()) {
            Object id = p.getObjectId();
            if (id != null) {
                _handleTypedObjectId(p, ctxt, bean, id);
            }
        }
        // [databind#3838]: since 2.16 Uniform handling of missing objectId
        // only for the specific "empty JSON Object" case
        if (_objectIdReader != null && p.hasTokenId(JsonTokenId.ID_END_OBJECT)) {
            // [databind#4610]: check if we are to skip failure
            if (ctxt.isEnabled(DeserializationFeature.FAIL_ON_UNRESOLVED_OBJECT_IDS)) {
                ctxt.reportUnresolvedObjectId(_objectIdReader, bean);
            }
        }

Looks like the problem is that the first block is for Native Object Id (like YAML anchors); and second block should only affect non-native Object Ids. So perhaps it should be more like:

        if (p.canReadObjectId()) {
            Object id = p.getObjectId();
            if (id != null) {
                _handleTypedObjectId(p, ctxt, bean, id);
            }
        // [databind#3838]: since 2.16 Uniform handling of missing objectId
        // only for the specific "empty JSON Object" case
        } else if (_objectIdReader != null && p.hasTokenId(JsonTokenId.ID_END_OBJECT)) {
            // [databind#4610]: check if we are to skip failure
            if (ctxt.isEnabled(DeserializationFeature.FAIL_ON_UNRESOLVED_OBJECT_IDS)) {
                ctxt.reportUnresolvedObjectId(_objectIdReader, bean);
            }
        }

to avoid failing where it shouldn't. Test would need to be on YAML side as JSON has no native Object Id functionality.

[cowtowncoder]

Interestingly enough, that change would solve 1 of 2 failing cases for reproduction @susanw1 provided.

Ahh. It fixes YAML case, but fails JSON case... which I think is true.

@susanw1 You may want to just disable DeserializationFeature.FAIL_ON_UNRESOLVED_OBJECT_IDS for JSON -- or, make sure Object Ids are included in JSON (in test case there's no @id JSON property which is why exception is thrown).

[cowtowncoder]

Considered fixed via #4612 to be released in 2.17.2 (and 2.18.0)

[susanw1]

If this is fixed for YAML, then that's a good fix in my world! :-) Brilliant - thanks all, much appreciated!

[JooHyukKim]

Thank u for reporting and workable reproduction also, @susanw1 👍🏼