Codec derivation - `dropNullValues`

[kamilkloch]

For now it is impossible to convenietly generate encoder that does not produce null values at the output.

Scenario:

@JsonCodec case class Params(p1: String, p2: Option[Int])
def process(a: Json): Unit = ???

val params = Params("abc", None)
process(json"""{"requestId": 123, "params": $params}""")

And I want the a not to contain null values. I know I can:

1. preprocess a in the process() method - apply dropNullValues, use customized printer, etc
2. postprocess json where I use interpolation
3. customize codec by .mapJson(.dropNullValues)

First and second one seems like wrong place to solve this issue - its like workaround for misbehaving codec.

Third option requires lots of bolierplate code: Switching from annotation to manual derivation (creation of companion objects, etc), and then mapping derived encoder output.

I believe most correct solution would be to add io.circe.generic.extras.Configuration.dropNullValues. Then one could use it with derived codecs as well as with @ConfiguredJsonCodec.

[djx314]

https://scastie.scala-lang.org/CQEocDLHTwS51Gte8lCasQ

@kamilkloch Provide a reproduct.

import io.circe._
import io.circe.syntax._
import io.circe.generic.JsonCodec
import io.circe.generic.extras.semiauto._
import io.circe.generic.extras.Configuration

@JsonCodec
case class Params(p1: String, p2: Option[Int])

val jsonString1: String = """{ "p1": "foo" }"""
val jsonString2: String = """{ "p1": "foo", "p2": null }"""

for {
  json1 <- parser.parse(jsonString1)
  json2 <- parser.parse(jsonString2)
  data1 <- json1.as[Params]
  data2 <- json2.as[Params]
} yield {
  println(data1)
  println(data2)
  println("// === problem start ===")
  println(data1.asJson == json1)
  println(data1.asJson == json2)
  println("// === problem end ===")
}

Output:

Params(foo,None)
Params(foo,None)
// === problem start ===
false
true
// === problem end ===

I can do nothing but hack Encoder with dropNullValues to make

data1.asJson == json1

return true in this code snippet.

Workaround code:

case class Params(p1: String, p2: Option[Int])
object Params {
  implicit val codecJson: Codec[Params] = {
    val impl = deriveCodec[Params]
    Codec.from(impl, impl.mapJson(_.dropNullValues))
  }
}

[kamilkloch]

@kamilkloch Provide a reproduct. Yes, I do get the same result.

I believe most correct solution would be to add io.circe.generic.extras.Configuration.dropNullValues. Then one could use it with derived codecs as well as with @ConfiguredJsonCodec.

[plokhotnyuk]

Yet another solution is using of jsoniter-scala-circe's codec instantiated with a function to filter out nulls.

The overhead of encoding of such fields with nulls will be compensated by much more efficient serialization from jsoniter-scala-core under the hood. Also, you will get an ability of serialization immediately to byte arrays, java.nio.ByteBufers, java.io.OutputStream or preallocated byte arrays without intermediate strings, that will save yet more CPU cycles and memory bandwidth.

[djx314]

@kamilkloch Provide a reproduct. Yes, I do get the same result.

I believe most correct solution would be to add io.circe.generic.extras.Configuration.dropNullValues. Then one could use it with derived codecs as well as with @ConfiguredJsonCodec.

@kamilkloch Yes, I think so too. io.circe.generic.extras.Configuration.dropNullValues is the final solution.