Erik Erlandson

Apologies for the substantial delay, I have published a spark-3.2 release-candidate: `0.6.0-RC1-sp3.2`

Here is a workaround that goes through RDD and uses `saveAsObjectFile` ```scala scala> import org.isarnproject.sketches.udaf._, org.apache.spark.isarnproject.sketches.udt._ import org.isarnproject.sketches.udaf._ import org.apache.spark.isarnproject.sketches.udt._ scala> val udaf = tdigestUDAF[Double] udaf: org.isarnproject.sketches.udaf.TDigestUDAF[Double] = TDigestUDAF(0.5,0) scala>...

Thanks! I did not know that! That reminds me - I haven't experimented with this yet but there is one additional method I did not override on my UDTs: private[spark]...

FYI, adding the `asNullable` method did not help with writing to parquet when I tried it. I'm still working on figuring this out

The trick is to define your own UDT classes in the `org.apache.spark` scope. Notice where my UDT files live: `isarn-sketches-spark/src/main/scala/org/apache/spark/isarnproject/sketches/udt/TDigestUDT.scala` and the package is: `org.apache.spark.isarnproject.sketches.udt` I'm a bit irritated that...

xref: #26

I have published: ``` "org.isarnproject" %% "isarn-sketches-spark" % "0.6.0-RC1-sp3.2" ``` Let me know how this works for you - if it is satisfactory, I can publish an official release cc...

@tschuchortdev agree, that looks like the same problem

@soronpo it worries me a little this bug is 3 years old :joy: Is it fixable? If it's not something likely to be fixed in the near term, I will...

I just tried removing `transparent` on my methods, and the dependent types seem to be resolving fine without it. It looks like scala 3 is an improvement here over scala...