scio
scio copied to clipboard
To.safe
This PR proposes a solution to the To.safe blocker in Scala 3 on which @MaximeKjaer, @liufengyun and I worked.
It took us some time to get familiar with the API and we tested different strategies before landing on this code, but we think it shows the strengths of Scala 3 macros and reflection by combining the two APIs in a relatively concise and idiomatic function that does not need eval
You can see a smaller diff here
The critical section is the file scio-core/src/main/scala-3/com/spotify/scio/schemas/To.scala
where we use Scala 3 macros to pattern match on the type argument T
of Expr[Schema[T]]
to derive a Schema[T]
at compile time which can then be used to invoke To.checkCompatibility
. To deal with case class and compile time derived Schema
s we rely on the reflection API which allows us to access the names and types of the fields of the case class.
You can see this the implementation here
This draft pr brings along many changes, in particular to build and syntax, that we adopted to use RC2, sbt 1.5, new syntax and to reduce the number of warnings.
We would like to have your opinion on this approach.
A few steps might be needed to finalize this pr:
- Polish and refine
- Extensive testing
- Strengthen case class detection and safety: I am not sure that non-empty
caseFields
and<: Product
is a strong-enough condition - Rebase on
main
as it progresses.