scio
scio copied to clipboard
Issue with encoding Map with complex types
Hello.
I believe there is a bug in SchemaMaterializer if type of key or value isn't a primitive.
getCollectionElementType returns field type only for lists for map getMapKeyType and getMapValueType needs to be used.
So as a result I have a NullPointerException if I have a map with complex types.
A unit test to reproduce the issue:
case class K(someIntField: Int)
case class V(andLong: Long)
"com.spotify.scio.schemas.SchemaMaterializer" should "throw NPE when it's serializing map with objects" in {
val objectMap = Map(K(1) -> V(2))
val (_, objectMapToRow, _) = SchemaMaterializer.materialize(Schema[Map[K, V]])
assertThrows[NullPointerException] {
println(objectMapToRow(objectMap))
}
val map = Map("s" -> "e")
val (_, toRow, _) = SchemaMaterializer.materialize(Schema[Map[String, String]])
// Working fine for primitives ¯\_(ツ)_/¯
noException should be thrownBy toRow(map)
}