Codec derivation issues upgrading from v6.0.0 to v7.0.0

[marko-asplund]

trafficstars

I'm bumped into two issues when upgrading to v7.0.0:

1. When upgrading from play-json-derived-codecs v6.0.0 to v7.0.0 Play JSON picks up a different JSON serializer in a scenario involving sum types when codecs for both the case classes and the sealed trait are in scope

2. codec derived for sealed trait doesn't appear to work for the case classes

Both of these issues can be reproduced with code found in this repository: https://github.com/marko-asplund/play-json-derived-codecs-issue

[julienrf]

Thanks for the report @marko-asplund. Would you be interested in investigating what’s going on?

[marko-asplund]

@julienrf This issue is currently blocking Play v2.8 upgrade, so I'll need to figure out some kind of workaround. Any pointers for investigating the issue?

[julienrf]

I think you did a great job of sharing the test cases that fail. Thank you for this! The next step would be to debug which implicit definitions are used in both cases (with v6 and v7) to see where they diverge. The compiler plugin tek/splain could help a lot in this area.

[marko-asplund]

Thanks for the tip! 👍 I'll look into debugging using the plugin.

🤔 How do I get the plugin to output implicit definitions? I've tried setting this up in build.sbt, but e.g. compile doesn't seem to generate any additional output.

[julienrf]

Hmm, I don’t know what’s wrong. Your setup looks good to me. Did you reload sbt?

[marko-asplund]

Yeah, I did try reloading and restarting sbt several times.

[marko-asplund]

This issue seems to have been caused by variance changes in play.api.libs.json.Writes between play-json v2.7 and v2.8. Writes was made invariant on A whereas before it was contravariant.

play-json v2.7.4

package play.api.libs.json

trait Writes[-A] { self =>
...

play-json v2.8.1

package play.api.libs.json

trait Writes[A] { self =>
...

This changed in commit https://github.com/playframework/play-json/commit/ba865772d1616d774042620ce0d68802312dada7#diff-43f2e42c6ce469bcd932a93a187413fe

Because of this change Json.toJson doesn't allow serializing SubType using OWrites[SuperType] anymore.

[julienrf]

Thanks for the investigation @marko-asplund!

I guess for the second problem you’ll have to manually upcast your values:

-    val b2 = Json.toJson(Bar2("hello", 55))
+    val b2 = Json.toJson(Bar2("hello", 55): Foo2)

The issue 1 is more problematic. I’m not sure how to fix it.

[marko-asplund]

@julienrf Thanks for the feedback!

issue 1: Writes was changed to be invariant on A. Before this change both Writes[SubType] and Writes[SuperType] were eligible for implicit resolution, but after the change only Writes[SubType] is, effectively forcing Writes[SubType] to get picked up.

issue 2: only Writes[SuperType] is defined. Since Writes is now invariant it can't be used with SubType.

IMO, though it's not an ideal solution, passing the desired type parameter to Json.toJson (Json.toJson[SuperType]) or upcasting to SuperType can be used as a workaround and force use of Writes[SuperType] in both cases.

🤔 Trying to figure out what play-json-derived-codecs could do to address this issue. Not sure if there's much that can be done in terms of addressing this issue in terms of play-json-derived-codecs features. As I guess this issue has the potential to hit many play-json-derived-codecs users who are using it with sum types, one thing could be to add a caveat in the README about upgrading from v6 to v7.

[julienrf]

As I guess this issue has the potential to hit many play-json-derived-codecs users who are using it with sum types, one thing could be to add a caveat in the README about upgrading from v6 to v7.

Yes, that’s a good idea, thanks for proposing! We should provide the workarounds in the README.

[marko-asplund]

🤔 Also wondering what was the exact rationale for play-json to go this way. It would've been very helpful to publish a heads up about this e.g. in the form of breaking changes in v2.8 release notes. I couldn't find anything on this, at least not with a quick search and couldn't find release notes for v2.8 either.

[julienrf]

That’s a good point. Personally, I don’t understand this decision.

[marko-asplund]

Here's a possible workaround https://github.com/marko-asplund/play-json-derived-codecs-issue/commit/fc0c7b073ebb4d9a3ccf1d4a152d5caa2acb8c01

[julienrf]

It seems that they are aware of the problem: https://github.com/playframework/play-json/issues/404