JsonTypeInfo.As.EXTERNAL_PROPERTY: cannot (de)serialize class with single no-arg constructor or object (singletone) class properly

[smedelyan]

Describe the bug Serialization and deserialization of object-class or a single-no-arg constructor class either fails or gives unexpected output. Perhaps, I'm setting up it in a wrong way -- then please show how to achieve expected output.

The reason is mapper expects a field in the container (which is annotated with @JsonTypeInfo(...EXTERNAL_PROPERTY)) to have value even if the value cannot be of any sense. I will need to set 2 fields explicitly in this case, e.g.:

container:
  field-type: A | B | C
  field: {} # <--- notice: {} is meaningless but required

To Reproduce

Run the following Kotlin test (// throws and // fails describe expected behavior):

Test

import com.fasterxml.jackson.annotation.JsonSubTypes
import com.fasterxml.jackson.annotation.JsonTypeInfo
import com.fasterxml.jackson.databind.DeserializationFeature
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.databind.PropertyNamingStrategies
import com.fasterxml.jackson.databind.SerializationFeature
import com.fasterxml.jackson.dataformat.yaml.YAMLFactory
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule
import com.fasterxml.jackson.module.kotlin.KotlinModule
import com.fasterxml.jackson.module.kotlin.SingletonSupport
import com.fasterxml.jackson.module.kotlin.readValue
import io.kotest.matchers.shouldBe
import io.kotest.matchers.string.shouldContain
import io.kotest.matchers.types.shouldBeTypeOf
import org.junit.jupiter.api.Test

data class Config(
    @JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.EXTERNAL_PROPERTY, property = "base-type")
    @JsonSubTypes(
        JsonSubTypes.Type(value = ObjectImpl::class, name = "object"),
        JsonSubTypes.Type(value = DataClassImpl::class, name = "dataclass"),
        JsonSubTypes.Type(value = NoFieldClassImpl::class, name = "nofieldclass"),
    )
    val base: Base,
)

sealed class Base

object ObjectImpl : Base()

data class DataClassImpl(
    val field: String,
) : Base()

class NoFieldClassImpl : Base() {
    override fun equals(other: Any?): Boolean {
        return this === other
    }

    override fun hashCode(): Int {
        return System.identityHashCode(this)
    }
}

class Test {

    private val mapper = ObjectMapper(YAMLFactory()).apply {
        propertyNamingStrategy = PropertyNamingStrategies.KEBAB_CASE
        this.registerModule(JavaTimeModule())
        this.disable(DeserializationFeature.ADJUST_DATES_TO_CONTEXT_TIME_ZONE)
        this.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
        val kotlinModule = KotlinModule.Builder()
            .singletonSupport(SingletonSupport.CANONICALIZE)
            .build()
        this.registerModule(kotlinModule)
    }

    @Test
    fun deserialize() {
        // OK
        mapper.readValue<Config>(
            """
                base-type: dataclass
                base:
                    field: abc
            """.trimIndent()
        ).base shouldBe DataClassImpl(field = "abc")

        // throws
        mapper.readValue<Config>(
            """
                base-type: nofieldclass
            """.trimIndent()
        ).base.shouldBeTypeOf<NoFieldClassImpl>()

        // OK
        mapper.readValue<Config>(
            """
                base-type: nofieldclass
                base: {} # imho, redundant
            """.trimIndent()
        ).base.shouldBeTypeOf<NoFieldClassImpl>()

        // throws
        mapper.readValue<Config>(
            """
                base-type: object
            """.trimIndent()
        ).base shouldBe ObjectImpl

        // OK
        mapper.readValue<Config>(
            """
                base-type: object
                base: {}
            """.trimIndent()
        ).base shouldBe ObjectImpl
    }

    @Test
    fun serialize() {
        var s = mapper.writeValueAsString(Config(base = DataClassImpl(field = "abc")))

        // fails both
        s shouldContain "base-type"
        s shouldBe """
            base-type: dataclass
            base:
                field: abc
        """.trimIndent()

        s = mapper.writeValueAsString(Config(base = ObjectImpl))

        // fails both
        s shouldContain "base-type"
        s shouldBe """
            base-type: object
        """.trimIndent()

        s = mapper.writeValueAsString(Config(base = NoFieldClassImpl()))

        // fails both
        s shouldContain "base-type"
        s shouldBe """
            base-type: nofieldclass
        """.trimIndent()
    }
}

Expected behavior I expect // throws- and // fails-commented sections in the test above to pass.

Versions Kotlin: 1.6.10 Jackson-module-kotlin: 2.13.1 Jackson-databind: 2.13.1

Both Jackson libs are automatically set up by Spring Dependency Management Gradle plugin:

plugins {
// ...
    id 'org.springframework.boot' version '2.6.3' // latest
    id 'io.spring.dependency-management' version '1.0.11.RELEASE' // latest
}

[smedelyan]

After thinking of this thoroughly, I now believe this is not a bug (but rather a feature request). I think I will overcome this by implementing custom Serializer / Deserializer to support usage of enum-s in-place where type allows "default" values (e.g. object or no-arg constructor). Might be good to have it in jackson someday as, say, include = ...IN_PLACE | REPLACE, property = "strategy-type" which would allow to switch from this:

container:
  strategy-type:
  strategy: {}

To this:

container:
  strategy-type: ALIAS # maps to field "Container::strategy"

Because serialization works not like expected (though, if my conclusions are correct I'm not interested in this functionality anymore) -- and because I need someone to verify my findings -- I keep this issue open.

[smedelyan]

Eventually, I've come to implementing my own (de)serializers. Code is below in case anyone wonders:

Common.kt

import com.fasterxml.jackson.annotation.JsonSubTypes
import com.fasterxml.jackson.databind.BeanDescription

typealias ClassName = String

fun BeanDescription.inlineTypeConfigured(): Boolean {
    if (!beanClass.hasInlineType()) return false

    val subTypeInfo = beanClass.subTypeInfo()
    requireNotNull(subTypeInfo) { "@JsonInlineType is present, but there is no @JsonSubTypes (or more than one)" }
    require(subTypeInfo.value.isNotEmpty()) { "No elements in @JsonSubTypes" }

    return true
}

fun Class<*>.hasInlineType(): Boolean = getAnnotationRecursive(this, JsonTypeInfoInlined::class.java) != null

tailrec fun <T : Annotation> getAnnotationRecursive(clazz: Class<*>, type: Class<T>): T? {
    @Suppress("UNCHECKED_CAST")
    val declared = clazz.declaredAnnotations.firstOrNull { type.isAssignableFrom(it::class.java) } as T?
    if (declared != null) return declared
    if (clazz.superclass == null) return null

    return getAnnotationRecursive(clazz.superclass, type)
}

fun Class<*>.subTypeInfo(): JsonSubTypes? = getAnnotationRecursive(this, JsonSubTypes::class.java)

InlineTypeDeserializer.kt

import com.fasterxml.jackson.core.JsonParser
import com.fasterxml.jackson.core.JsonToken
import com.fasterxml.jackson.databind.DeserializationContext
import com.fasterxml.jackson.databind.deser.std.StdDeserializer
import com.fasterxml.jackson.module.kotlin.isKotlinClass

class InlineTypeDeserializer(
    private val namesOfClasses: Map<ClassName, Class<*>>,
) : StdDeserializer<Any>(Any::class.java) {
    override fun deserialize(p: JsonParser, ctxt: DeserializationContext?): Any {
        require(p.currentToken == JsonToken.VALUE_STRING) { "${p.currentToken} is not a string" }
        val clazz = namesOfClasses[p.text]
        requireNotNull(clazz) { "${p.text} not found in $namesOfClasses" }

        if (clazz.isKotlinClass() && clazz.kotlin.objectInstance != null) {
            return clazz.kotlin.objectInstance!!
        }

        return clazz.getDeclaredConstructor().newInstance()
    }
}

InlineTypeDeserializerModifier.kt

import com.fasterxml.jackson.databind.BeanDescription
import com.fasterxml.jackson.databind.DeserializationConfig
import com.fasterxml.jackson.databind.JsonDeserializer
import com.fasterxml.jackson.databind.deser.BeanDeserializerModifier

class InlineTypeDeserializerModifier : BeanDeserializerModifier() {
    override fun modifyDeserializer(
        config: DeserializationConfig?,
        beanDesc: BeanDescription,
        deserializer: JsonDeserializer<*>,
    ): JsonDeserializer<*> {
        if (beanDesc.inlineTypeConfigured()) {
            val namesOfClasses = beanDesc.beanClass.subTypeInfo()!!.value.associate { it.name to it.value.java }
            return InlineTypeDeserializer(namesOfClasses)
        }
        return deserializer
    }
}

InlineTypeModule.kt

import com.fasterxml.jackson.databind.module.SimpleModule

class InlineTypeModule : SimpleModule() {
    override fun setupModule(context: SetupContext) {
        super.setupModule(context)
        context.addBeanSerializerModifier(InlineTypeSerializerModifier())
        context.addBeanDeserializerModifier(InlineTypeDeserializerModifier())
    }
}

InlineTypeSerializer.kt

import com.fasterxml.jackson.core.JsonGenerator
import com.fasterxml.jackson.databind.SerializerProvider
import com.fasterxml.jackson.databind.ser.std.StdSerializer

class InlineTypeSerializer(
    private val classNames: Map<Class<*>, ClassName>,
) : StdSerializer<Any>(Any::class.java) {
    override fun serialize(value: Any, gen: JsonGenerator, provider: SerializerProvider?) {
        val name = classNames[value.javaClass]
        requireNotNull(name) { "${value.javaClass} not found in $classNames" }
        gen.writeString(name)
        // properties are intentionally not handled!
    }
}

InlineTypeSerializerModifier.kt

import com.fasterxml.jackson.databind.BeanDescription
import com.fasterxml.jackson.databind.JsonSerializer
import com.fasterxml.jackson.databind.SerializationConfig
import com.fasterxml.jackson.databind.ser.BeanSerializerModifier

class InlineTypeSerializerModifier : BeanSerializerModifier() {
    override fun modifySerializer(
        config: SerializationConfig?,
        beanDesc: BeanDescription,
        serializer: JsonSerializer<*>,
    ): JsonSerializer<*> {
        if (beanDesc.inlineTypeConfigured()) {
            val classNames = beanDesc.beanClass.subTypeInfo()!!.value.associate { it.value.java to it.name }
            return InlineTypeSerializer(classNames)
        }

        return serializer
    }

}

JsonTypeInfoInlined.kt

rendered description

code

/**
 * Annotation used as an alternative to [@JsonTypeInfo][com.fasterxml.jackson.annotation.JsonTypeInfo] for embedding
 * type information in-place during (de)serialization like in the example below.
 *
 * Requirements:
 * * **sub**classes MUST have a public no-arg constructor (or a constructor with default values)
 * * **sub**classes MUST NOT have [@JsonSubTypes][com.fasterxml.jackson.annotation.JsonSubTypes], this will break the
 * processing
 * * **super**class MUST have a [@JsonSubTypes][com.fasterxml.jackson.annotation.JsonSubTypes] with
 * [@Type(value = ..., name = ...)][com.fasterxml.jackson.annotation.JsonSubTypes.Type] in the same fashion as you do
 * for [@JsonTypeInfo][com.fasterxml.jackson.annotation.JsonTypeInfo]
 * * **super**class and **sub**classes MUST NOT have [@JsonTypeInfo][com.fasterxml.jackson.annotation.JsonTypeInfo] (Jackson
 * requires the "type" field to be always present in this case which breaks processing)
 *
 * **WARNING:** while subclasses may have fields, these fields will never be (de)serialized
 *
 * ```kotlin
 * data class Config(
 *     val simpleNoFieldClass: Strategy,
 *     val objectClass: Strategy,
 *     val dataClass: Strategy,
 * )
 *
 * @JsonTypeInfoInlined
 * @JsonSubTypes(
 *     JsonSubTypes.Type(value = StrategyA::class, name = "A"),
 *     JsonSubTypes.Type(value = StrategyB::class, name = "B"),
 *     JsonSubTypes.Type(value = StrategyC::class, name = "C"),
 * )
 * sealed class Strategy
 *
 * @Suppress("CanSealedSubClassBeObject")
 * class StrategyA : Strategy() {
 *     // some functionality may be here
 * }
 *
 * object StrategyB : Strategy()
 *
 * data class StrategyC(val fieldWithDefault: String = "default value") : Strategy()
 * ```
 *
 * ```yaml
 * ---
 * # with regular @JsonTypeInfo(id = NAME, include = As.PROPERTY, property = "type")
 * simple-no-field-class:
 *   type: A        # <--
 * object-class:
 *   type: B        # <-- type information as separate field on a new line
 * data-class:
 *   type: C        # <--
 *
 * ---
 * # with @JsonTypeInfoInlined
 * simple-no-field-class: A # <-- type information inlined
 * object-class: B          # <--
 * data-class: C            # <--
 * ```
 */
annotation class JsonTypeInfoInlined()

Example test with demonstration

import com.fasterxml.jackson.annotation.JsonSubTypes
import com.fasterxml.jackson.databind.DeserializationFeature
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.databind.PropertyNamingStrategy
import com.fasterxml.jackson.databind.SerializationFeature
import com.fasterxml.jackson.dataformat.yaml.YAMLFactory
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule
import com.fasterxml.jackson.module.kotlin.KotlinModule
import com.fasterxml.jackson.module.kotlin.SingletonSupport
import com.rapidmigrate.auxiliary.jackson.InlineTypeModule
import com.rapidmigrate.auxiliary.jackson.JsonTypeInfoInlined
import com.rapidmigrate.cluster.common.metadata.marshalToString
import com.rapidmigrate.cluster.common.metadata.unmarshal
import io.kotest.matchers.shouldBe
import io.kotest.matchers.types.shouldBeInstanceOf
import org.junit.jupiter.api.Test

data class Config(
    val simpleNoFieldClass: Strategy,
    val objectClass: Strategy,
    val dataClass: Strategy,
)

@JsonTypeInfoInlined
@JsonSubTypes(
    JsonSubTypes.Type(value = StrategyA::class, name = "A"),
    JsonSubTypes.Type(value = StrategyB::class, name = "B"),
    JsonSubTypes.Type(value = StrategyC::class, name = "C"),
)
sealed class Strategy

@Suppress("CanSealedSubClassBeObject")
class StrategyA : Strategy() {
    // some functionality may be here
}

object StrategyB : Strategy()

data class StrategyC(val fieldWithDefault: String = "default value") : Strategy()

class JsonTypeInfoInlinedTest {

    private val mapper = ObjectMapper(YAMLFactory()).apply {
        propertyNamingStrategy = PropertyNamingStrategy.KEBAB_CASE
        registerModule(JavaTimeModule())
        disable(DeserializationFeature.ADJUST_DATES_TO_CONTEXT_TIME_ZONE)
        disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
        val kotlinModule = KotlinModule.Builder()
            .singletonSupport(SingletonSupport.CANONICALIZE)
            .build()
        registerModule(kotlinModule)
        registerModule(InlineTypeModule())
    }

    @Test
    fun serialize() {
        val c = Config(
            simpleNoFieldClass = StrategyA(),
            objectClass = StrategyB,
            dataClass = StrategyC(),
        )
        mapper.marshalToString(c) shouldBe """
            ---
            simple-no-field-class: "A"
            object-class: "B"
            data-class: "C"

        """.trimIndent()
    }

    @Test
    fun deserialize() {
        val c: Config = mapper.unmarshal(
            """
            simple-no-field-class: A
            object-class: B
            data-class: C
        """.trimIndent()
        )

        with(c) {
            simpleNoFieldClass.shouldBeInstanceOf<StrategyA>()
            objectClass shouldBe StrategyB
            dataClass shouldBe StrategyC()
        }
    }
}

P. S. Didn't test with Java, but theoretically it should work

P. P. S. I have abandoned the idea to allow default @JsonTypeInfo handling as a fallback because it seems to need much more digging into Jackson code -- i.e. to allow something like this:

#these two are still inlined
simple-no-field-class: A
object-class: B
# however, this one may be "expanded" if non-default value is needed
data-class:
  type: C
  field-with-default: "not default value!"

[smedelyan]

My initial needs are now more or less satisfied with the workaround above. Someone please comment if you see an easier way to achieve the same behavior or think this functionality might worth implementing in Jackson.

[k163377]

It seems that kotlin-module will never be able to achieve this. Please re-submit the sample code in the following module after rewriting it in Java. https://github.com/FasterXML/jackson-dataformats-text