sparksql-scalapb icon indicating copy to clipboard operation
sparksql-scalapb copied to clipboard

Add test for parseFrom message with nested extension

Open jckegelman opened this issue 2 years ago • 2 comments

Add a (failing) test for using parseFrom for creating a Dataset from serialized messages where the message definition includes a nested extension. This test currently fails with

[info] ExtensionsSpec:
[info] Creating Dataset from message with nested extension
[info] - should work *** FAILED ***
[info]   org.apache.spark.SparkException: Task not serializable
...
[info]   Cause: java.io.NotSerializableException: scalapb.lenses.Lens$$anon$1
[info] Serialization stack:
[info]  - object not serializable (class: scalapb.lenses.Lens$$anon$1, value: scalapb.lenses.Lens$$anon$1@2efd8702)
[info]  - field (class: scalapb.GeneratedExtension, name: lens, type: interface scalapb.lenses.Lens)
[info]  - object (class scalapb.GeneratedExtension, GeneratedExtension(scalapb.lenses.Lens$$anon$1@2efd8702))
[info]  - field (class: com.example.protos.extensions.Baz$, name: bar, type: class scalapb.GeneratedExtension)
[info]  - object (class com.example.protos.extensions.Baz$, com.example.protos.extensions.Baz$@628905ca)

jckegelman avatar Oct 23 '22 17:10 jckegelman

Since the test fail for Scala 0.10.x and succeed for the newest ScalaPB (to be released), we need to add a mechanism to enable tests that only run for some version combinations in the matrix.

thesamet avatar Oct 24 '22 01:10 thesamet

I updated build.sbt to filter out the new test for ScalaPB 0.10.x using Tests.Filter, let me know if that doesn't work for you.

jckegelman avatar Oct 24 '22 04:10 jckegelman