kotlin-spark-api icon indicating copy to clipboard operation
kotlin-spark-api copied to clipboard

Encoding for Scala Option and Optional

Open Jolanrensen opened this issue 3 years ago • 0 comments

While Dataset<Option<*>> works, whereever a schema needs to be created, such as Dataset<Pair<*, Option<*>>, then it fails. Option is a Product but is handled as if it doesn't exist in Scala. Meaning Option<T> is encoded as T.

This needs to be looked into

Jolanrensen avatar Apr 19 '22 14:04 Jolanrensen