scio icon indicating copy to clipboard operation
scio copied to clipboard

To.safe

Open vincenzobaz opened this issue 3 years ago • 0 comments

This PR proposes a solution to the To.safe blocker in Scala 3 on which @MaximeKjaer, @liufengyun and I worked. It took us some time to get familiar with the API and we tested different strategies before landing on this code, but we think it shows the strengths of Scala 3 macros and reflection by combining the two APIs in a relatively concise and idiomatic function that does not need eval

You can see a smaller diff here

The critical section is the file scio-core/src/main/scala-3/com/spotify/scio/schemas/To.scala where we use Scala 3 macros to pattern match on the type argument T of Expr[Schema[T]] to derive a Schema[T] at compile time which can then be used to invoke To.checkCompatibility. To deal with case class and compile time derived Schemas we rely on the reflection API which allows us to access the names and types of the fields of the case class. You can see this the implementation here

This draft pr brings along many changes, in particular to build and syntax, that we adopted to use RC2, sbt 1.5, new syntax and to reduce the number of warnings.

We would like to have your opinion on this approach.

A few steps might be needed to finalize this pr:

  • Polish and refine
  • Extensive testing
  • Strengthen case class detection and safety: I am not sure that non-empty caseFields and <: Product is a strong-enough condition
  • Rebase on main as it progresses.

vincenzobaz avatar Apr 19 '21 09:04 vincenzobaz