sparksql-scalapb
sparksql-scalapb copied to clipboard
Add test for parseFrom message with nested extension
Add a (failing) test for using parseFrom
for creating a Dataset from serialized messages where the message definition includes a nested extension. This test currently fails with
[info] ExtensionsSpec:
[info] Creating Dataset from message with nested extension
[info] - should work *** FAILED ***
[info] org.apache.spark.SparkException: Task not serializable
...
[info] Cause: java.io.NotSerializableException: scalapb.lenses.Lens$$anon$1
[info] Serialization stack:
[info] - object not serializable (class: scalapb.lenses.Lens$$anon$1, value: scalapb.lenses.Lens$$anon$1@2efd8702)
[info] - field (class: scalapb.GeneratedExtension, name: lens, type: interface scalapb.lenses.Lens)
[info] - object (class scalapb.GeneratedExtension, GeneratedExtension(scalapb.lenses.Lens$$anon$1@2efd8702))
[info] - field (class: com.example.protos.extensions.Baz$, name: bar, type: class scalapb.GeneratedExtension)
[info] - object (class com.example.protos.extensions.Baz$, com.example.protos.extensions.Baz$@628905ca)
Since the test fail for Scala 0.10.x and succeed for the newest ScalaPB (to be released), we need to add a mechanism to enable tests that only run for some version combinations in the matrix.
I updated build.sbt
to filter out the new test for ScalaPB 0.10.x using Tests.Filter
, let me know if that doesn't work for you.