zio
Opaque types may cause deserialization to never complete
Using zio-json 2.0.0-M1 under Scala 3.0.2.
The following scastie worksheet works fine: https://scastie.scala-lang.org/VGtUGBteQjiB9Y8Xlz4K5A
for reference, the code contents is:
import zio.json.*
opaque type Email = String
object Email:
def apply(value: String): Email = value
given JsonDecoder[Email] = JsonDecoder[String].map(Email.apply)
extension (x: Email)
def value: String = x
case class Structure(name: String,
email: Email)
object Structure {
given JsonDecoder[Structure] = DeriveJsonDecoder.gen
}
val json = """{ "name": "Toto", "email": "[email protected]"}"""
val data = json.fromJson[Structure]
println(data)
However, splitting the above code into three separate files (as one would usually do) causes the fromJson call to apparently never return:
hello/Email.scala:
package hello
import zio.json.JsonDecoder
opaque type Email = String
object Email:
def apply(value: String): Email = value
given JsonDecoder[Email] = JsonDecoder[String].map(Email.apply)
extension (x: Email)
def value: String = x
hello/Structure.scala:
package hello
import zio.json.{DeriveJsonDecoder, JsonDecoder}
case class Structure(name: String,
email: Email) // Change this to 'String'
object Structure {
given JsonDecoder[Structure] = DeriveJsonDecoder.gen
}
hello/TestCase.scala:
package hello
import zio.json.*
object TestCase extends App {
val json = """{ "name": "Toto", "email": "[email protected]"}"""
val dataOrElse = json.fromJson[Structure]
dataOrElse match {
case Right(data) =>
println(data)
println(data.email)
println(data.email: String)
println(data.email.value) // extension method
case Left(error) =>
println(s"bummer, ${error}")
}
}
Expected behaviour
calling """{ "name": "Toto", "email": "[email protected]"}""".fromJson[Structure] works even if the "email" field is defined as an opaque field with a suitable decoder
Observed behaviour While the code appears to work as long as everything is in the same compilation unit (as it is in Scastie), it fails when split over distinct files.
Changing the Structure's email field type back to String works around the issue.
The relevant stack trace is following. Would it suggest a bad interaction between magnolia and scala's internal implementation of givens?
This would suggest https://github.com/zio/zio-json/issues/434's result would be dearly needed
"run-main-0" #142 prio=5 os_prio=0 tid=0x000002394e072800 nid=0x4c8c in Object.wait() [0x0000008fbd7fd000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Unknown Source)
at scala.runtime.LazyVals$.wait4Notification(LazyVals.scala:89)
- locked <0x00000000faf7d980> (a java.lang.Object)
at hello.Email$package$Email$.given_JsonDecoder_Email(Email.scala:9)
at hello.Email$package$Email$.given_JsonDecoder_Email(Email.scala:9)
at hello.Structure$.$anonfun$6(Structure.scala:8)
at hello.Structure$$$Lambda$4540/2065787641.apply(Unknown Source)
at magnolia.CallByNeed$.apply$$anonfun$1(interface.scala:106)
at magnolia.CallByNeed$$$Lambda$4535/1507160179.apply(Unknown Source)
at magnolia.CallByNeed.value(interface.scala:110)
at magnolia.CaseClass$$anon$2.typeclass(interface.scala:28)
at zio.json.DeriveJsonDecoder$.zio$json$DeriveJsonDecoder$$anon$3$$_$tcs$$anonfun$1(macros.scala:112)
at zio.json.DeriveJsonDecoder$$anon$3$$Lambda$4592/995154638.apply(Unknown Source)
at scala.collection.ArrayOps$.map$extension(ArrayOps.scala:924)
at scala.IArray$package$IArray$.map(IArray.scala:179)
at zio.json.DeriveJsonDecoder$$anon$3.tcs(macros.scala:112)
at zio.json.DeriveJsonDecoder$$anon$3.unsafeDecode(macros.scala:137)
at zio.json.JsonDecoder.decodeJson(decoder.scala:79)
at zio.json.JsonDecoder.decodeJson$(decoder.scala:44)
at zio.json.DeriveJsonDecoder$$anon$3.decodeJson(macros.scala:96)
at zio.json.package$DecoderOps$.fromJson$extension(package.scala:28)
at hello.TestCase$.<clinit>(TestCase.scala:8)
at hello.TestCase.main(TestCase.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
Additionally:
declaring Email's JsonDecoder this way:
inline given JsonDecoder[Email] = JsonDecoder[String].map(Email.apply)
will cause the generated decoder to immediately crash due to NoSuchMethodError:
[error] (run-main-0) java.lang.NoSuchMethodError: hello.Email$package$Email$.given_JsonDecoder_Email()Lzio/json/JsonDecoder;
[error] java.lang.NoSuchMethodError: hello.Email$package$Email$.given_JsonDecoder_Email()Lzio/json/JsonDecoder;
[error] at hello.Structure$.$anonfun$6(Structure.scala:8)
[error] at magnolia.CallByNeed$.apply$$anonfun$1(interface.scala:106)
[error] at magnolia.CallByNeed.value(interface.scala:110)
[error] at magnolia.CaseClass$$anon$2.typeclass(interface.scala:28)
[error] at zio.json.DeriveJsonDecoder$.zio$json$DeriveJsonDecoder$$anon$3$$_$tcs$$anonfun$1(macros.scala:112)
[error] at scala.collection.ArrayOps$.map$extension(ArrayOps.scala:924)
[error] at scala.IArray$package$IArray$.map(IArray.scala:179)
[error] at zio.json.DeriveJsonDecoder$$anon$3.tcs(macros.scala:112)
[error] at zio.json.DeriveJsonDecoder$$anon$3.unsafeDecode(macros.scala:137)
[error] at zio.json.JsonDecoder.decodeJson(decoder.scala:79)
[error] at zio.json.JsonDecoder.decodeJson$(decoder.scala:44)
[error] at zio.json.DeriveJsonDecoder$$anon$3.decodeJson(macros.scala:96)
[error] at zio.json.package$DecoderOps$.fromJson$extension(package.scala:28)
[error] at hello.TestCase$.<clinit>(TestCase.scala:8)
(in other words, Magnolia will not notice that the JsonDecoder[Email] is declared inline and has no existence within the jar. Hopefully the native derivation-based generator will be able to pull the decoder AST directly, whether inline or not)
I have the same issue with zio-json 0.5.0.
Is there a workaround that works with opaque types?
The compiler warns me of the infinite loop when I do this with opaque type: given JsonDecoder[NodeTitle] = JsonDecoder[NodeTitle] or given JsonDecoder[NodeTitle] = JsonDecoder[String].map(NodeTitle.apply)
Same for me with 0.5.0:
[warn] Infinite loop in function body
[warn] zio.json.JsonDecoder.apply[String](this.given_JsonDecoder_UserName).map[String](
[warn] {
[warn] def $anonfun(value: String): String = this.apply(value)
[warn] closure($anonfun)
[warn] }
[warn] )
[warn] given JsonDecoder[UserName] = JsonDecoder[String].map(UserName.apply)
[warn] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
for
opaque type UserName = String
object UserName {
def apply(value: String): UserName = value
given JsonDecoder[UserName] = JsonDecoder[String].map(UserName.apply)
}
I think it's not related to this library (zio-json), but to the way opaque types works. In this scope, both String and UserName refer to the same type, so trying to retrieve a given instance causes endless loop.
As a workaround you can use
given JsonDecoder[UserName] = JsonDecoder.string
You're right ! And thank you for your feedback :) The example I've just written to test this workaround :
// ---------------------
//> using scala "3.2.2"
//> using dep "dev.zio::zio:2.0.13"
//> using dep "fr.janalyse::zio-worksheet:2.0.13.0"
//> using dep "dev.zio::zio-json:0.5.0"
//> using options "-Yretain-trees" // When case classes are using default values
// ---------------------
import zio.*, zio.json.*, zio.worksheet.*
import java.time.OffsetDateTime
opaque type UserName = String
object UserName {
def apply(value: String): UserName = value
given JsonCodec[UserName] = JsonCodec.string
}
opaque type LastSeenDateTime = OffsetDateTime
object LastSeenDateTime {
def apply(value: OffsetDateTime): LastSeenDateTime = value
given JsonCodec[LastSeenDateTime] = JsonCodec.offsetDateTime
}
case class User(
userName: UserName,
lastSeen: LastSeenDateTime
) derives JsonCodec
val json =
"""{
| "userName":"joe",
| "lastSeen":"2021-04-09T17:19:17.000Z"
|}""".stripMargin
val app =
for
person <- ZIO.fromEither(json.fromJson[User])
_ <- Console.printLine(person.toJsonPretty)
yield ()
app.unsafeRun