Add option to use ListMap instead Map and preserve properties order for untyped deserialization

[jneira-stratio]

Hi, recently i hit an use case where:

• it shoud rely on type Any to parse an arbitrary deep json as Map[String,Any]
• but preserving order of keys

Afaiu the code for untyped json deserialization uses Map in a fixed way.

To change it for preserve order i had to implement a 100 loc class with a new untyped deserializar and rebuild the scala module: https://gist.github.com/jneira-stratio/f0566c79b0284b05bb421d104bd318d7

My first question is if there is another, simpler way to get it working. Otoh would make sense to add a configuration option to being able to get that behaviour? It seems to me that it is a relative common use case.

Thanks in advance!

[pjfanning]

Why don't you deserialize to ListMap explicitly?

[jneira-stratio]

hmm thanks for the suggestion, but how could I do it for arbitrary deep nested json objects, reusing parsing/deser logic (it would take more locs to reimplement it, no?)

[pjfanning]

I'm not currently planning to work on this but I would review PRs if they were made

[cowtowncoder]

There is a general Jackson way to override mapping of abstract types like Map, List, Collection, if that would help: usually by SimpleModule.addAbstractTypeMapping(Map.class, ListMap.class), then adding module to ObjectMapper`.

I don't know if that would directly work with Scala module/types, but mechanism is available for Scala module and users.

[jneira-stratio]

@cowtowncoder hey, that feature seems promising, thanks. However i did not found a simple way to add an abstract type mapping to com.fasterxml.jackson.module.scala.JacksonModule context:

• You can add com.fasterxml.jackson.databind.deser.Deserializers like I did in my implementation.
• You can also add initializers instances of com.fasterxml.jackson.databind.Module.SetupContext which has the com.fasterxml.jackson.databind.Module.SetupContext#addAbstractTypeResolver but you have to implement all methods of that interface and i only found anonymous instances implemented in the code

But maybe it is possible, i dont know much about scala module internals 😐

[cowtowncoder]

@jneira-stratio You don't (and probably shouldn't actually) do this via Scala JacksonModule: instead, just use plain SimpleModule, register that after Scala module. All that matters is that ObjectMapper gets these mappings.

[jneira-stratio]

Hi, sorry i forgot to mention the first thing i tried was just add another simple module with the mapping registration after the scala module one. In a debug sesion I did not see the abstract type mapping in the deser context when Map[String,Any] is chosen for Any, and Map and not IntMap was the type used finally. Will try again just in case i did something wrong, thanks!

[cowtowncoder]

One thing to check is that Map target in question is the right actual class (not java.util.Map, I think, but Scala variant)

[jneira-stratio]

@cowtowncoder hi, i tried again with this code:

    val mapper: ObjectMapper with ClassTagExtensions = new ObjectMapper with ClassTagExtensions
    mapper.registerModule(MyScalaModule)
    
    val mod = new SimpleModule()
    mod.addAbstractTypeMapping(classOf[collection.immutable.Map[_, _]], classOf[collection.immutable.ListMap[_, _]])
    mapper.registerModule(mod)

and i am afraid it did not work. In a debug session i observed:

• when you ask for a deserialization with the explicit type Map[K, V] it uses correctly ListMap[K,V]
• but when you want to deserialize a untyped value (scala Any) the UntypedObjectDeserializerModule context does not have de abstract type mapping (cause it is only in the SimpleModule) and it is not called here either
• we can not add an abstract type mapping between Any and ListMap because it can be a Map (for an object) or a Seq (for an array), logic implemented just in UntypedObjectDeserializerModule

[jneira-stratio]

I'm not currently planning to work on this but I would review PRs if they were made

@pjfanning hi, thanks for the proposal, maybe i could try to add a PR if i have time I was thinking in add a config option to parametrize the type of the map (and maybe the seq?) in UntypedScalaObjectDeserializer but DeserializationFeature belongs to jackson-databind and this would be a scala module specific feature will investigate what is the scala module specific config setup, any pointer (or suggestion on how to implement it) will be very welcomed

[pjfanning]

Jackson 3 was refactored to allow some breaking changes. jackson-scala-module now passes around a config instance. The class is https://github.com/FasterXML/jackson-module-scala/blob/3.x/src/main/scala/tools/jackson/module/scala/ScalaModule.scala and you can see there are some settings on it already.

[cowtowncoder]

@jneira-stratio We do not want to map Any; it's deserializer that handles Any (in core databind that'd be UntypedObjectDeserializer.java) that would need to find deserializer to delegate to for necessary Map type. So that may be on Scala side. So I think abstract type mapping would exists even with Any case but is just not used.

[pjfanning]

I'm at a loss why we need to spend time on this. We provide support for naming the required type yet the OP wants us to guess their preferred type when they say they want any type. The OP has a workaround anyway - they have already coded a customized deserializer.

[cowtowncoder]

I assume this then falls under "PR needed" category for anyone wishing to improve things further. I was trying to help with what I think was the ask, to change mapping of type for JSON Object in case of Scala any type; similar to how things work on Java side.

[jneira-stratio]

hi, only want to mention that choose ListMap is only the way to keep properties order of source json in the result, for arbitrary nested json objects (so you must use Any to get Map[String, Any] recursively). We would be happy with any other solution for that.

@cowtowncoder If i understood correctly the correct way (and more similar to java side) would be add support for abstract type mapping in the scala module itself? So adding something like

 protected def +=(atr: AbstractTypeResolver): this.type = this += (_ addAbstractTypeResolver atr)

in the JacksonModule could be a sensible solution?

[cowtowncoder]

@jneira-stratio The problem is not missing abstract type mapping (it exists for mapper) but that it might not be used by deserializers, namely, one that handles "Any" type (usually indirectly, by requesting deserializer to delegate to). Sounds like it is not being used; for plain Java UntypedObjectDeserializer does use such resolution.