thoth-org
Encode.undefined for omitting a field produced by Encode.object?
I think it would be useful to be able to define encoder like this:
type SomeXY = X of string | Y
let decodeSomeXY =
function
| X s -> Encode.string s
| Y -> Encode.undefined // or some better name?
Now, it would work like this:
> Encode.object
[ "a", Decode.int 1
"b", decodeSomeXY (X "…")
"c", decodeSomeXY Y ] // this field would not be included in JSON
|> Encode.toString 0
- ;;
val it : string = "{"a":1, "b":"…"}"
That would make it easy to create decoders with optional fields without having to (sometimes deep) filter the array of (key, values). It would also be similar to JavaScript's JSON.stringify:
> JSON.stringify({ a: 1, b: "…", c: undefined })
'{"a":1,"b":"…"}'
What would you say?
Hello,
to me, it feels really strange to not have a valid JSON representation for each case of a DUs. What is your use case for having a type partially represented in term of JSON? I am just trying to understand the whole picture as JSON representation is not something easy.
Currently, in Thoth.Json we don't have a case where we don't represent one part of a DUs. The closest thing is the implementation of the Auto module for the Record.
We decided to treat in the same way the absence of a property and the value null for it. It is possible to skipNullField in order to not generates them in the output.
If you want to omit a field from an object, you can always not yield his value.
Encode.object [
"name", Encode.string "Maxime"
if condition then
"x", Encode.undefined
]
But it is true that it is the "object" which decide to not emit the value, not the underlying type.
Thoth.Json is not tied to a specific JSON parser, so we can't rely on a parser implementation to do the job for us:
- Thoth.Json use native JavaScript parser
- Thoth.Json.Net use Newtonsoft.Net
- Thoth.Json vNext use a custom parser
A possible workflow for the "implementation" could be:
type ErasedType =
| String of string
| Nothing
static member Encoder (v : ErasedType)=
match v with
| String str -> Encode.string str
| Nothing -> Encode.erased
Encode.object [
"name", Encode.string "Maxime"
"x", ErasedType.Encoder Nothing
] // Encode.object could apply a filter to remove the value makes with "erased"
// Implementation detail: Because we are already iterating over the list once to create the object, we should just add a check for the erased case. The goal is to avoid iterating twice over the list (performance "gain").
Encode.string 4 ErasedType.Nothing // Encode.string could check if the value applied is marked with erased and in this case generate an empty JSON
I think adding Encode.erased this way would imply a lot of change internally to Thoth.Json and should probably be done in vNext branch as it is already a major rewrite of the library.
Sometimes, the valid representation is just to exclude a field in JSON. It's not always a straight – one JSON field mapped by one domain type field – and vice versa. In my case, where I work with mixed environment with some legacy services, the domain modeling reveals big differences between domain objects and their JSON representation.
Let me show you one example:
type CustomerRequest =
{ Price: RequestedPrice
// other fields }
and RequestedPrice =
| LimitedOrder of decimal
| MarketOrder
// remote service, accepting JSON:
{ "price_type": "FIXED", "price": "4.4365", /* other fields */ } // <--- the LimitedOrder case
{ "price_type": "AUTO", /* other fields */ } // <--- the MarketOrder case
This is why I would find it interesting to be able to create decoders which would produce a result of omitting the field entirely from a JSON response.
Also, imagine creating a JSON payload of MongoDB update. It's big difference between:
{ $set: { firstname: "…", lastname: null, somethingElse: null } } and
{ $set: { firstname: "…", somethingElse: null } }
First one will change firstname and erase both lastname and somethingElse, the latter will not alter lastname at all.
I could go one by one, example by example, where it does matter if the field is null vs it does not exist in an JSON object.
I thought it would be easy to filter out these erased or undefined when iterating over a list of tuples in Encode.object, but if that is not the case, then OK, there are workarounds.
If you are interested in adding support for this feature only from Thoth.Json (Fable runtime) you could easily add it in your project using:
open Thoth.Json
open Fable.Core
module Encode =
[<Emit("undefined")>]
let jsUndefined : obj = jsNative
let erased = jsUndefined
type ErasedType =
| String of string
| Nothing
static member Encoder (v : ErasedType)=
match v with
| String str -> Encode.string str
| Nothing -> Encode.erased
Encode.object [
"name", Encode.string "Maxime"
"x", ErasedType.Encoder Nothing
]
|> Encode.toString 4
|> printfn "%A"
I thought it would be easy to filter out these erased or undefined when iterating over a list of tuples in Encode.object, but if that is not the case, then OK, there are workarounds.
I don't think this is that simple because as I said Thoth.Json currently use 2 different JSON parser depending on the runtime. And currently, the Encode module directly manipulate JsonValue or JObject in the case of Newtonsoft.Net.
And thus object don't have a erased representation. We would have to instead make them manipulate a custom type from which we could identify the value marked as erased.
type EncodeValue =
| Keep of JsonValue // or Keep of JObject
| Erased
Which means rewritting all the Encode module for that. That's why I said if I had it it will be in vNext which is already a complete rewrite of the library. (Almost done but just lacking the motivation right now to polish it always the last 10% which are the harder ^^)
@njlr Do you think we should add the ability to omit fields ?
Do you think it is possible to do with the API proposed in #188 ? When using the JSON DUs, we probably could have added a new case to the DUs to cover this need.
This sort of thing also comes up in GraphQL, where a variable can have 3 states: something, nothing or absent.
My gut feeling is that this shouldn't be added because F# list comprehensions can contain match expressions. I'm open to good counter-examples though 🙂
IMO the example given above works well with a computation expression decoder.
#r "nuget: Thoth.Json.Net, 11.0"
#r "nuget: Thoth.Json.Net.CE, 0.2.1"
type CustomerRequest =
{
Price : RequestedPrice
Foo : int
}
and RequestedPrice =
| LimitedOrder of decimal
| MarketOrder
open Thoth.Json.Net
open Thoth.Json.Net.CE
module CustomerRequest =
let encode : Encoder<CustomerRequest> =
fun x ->
[
match x.Price with
| LimitedOrder price ->
"price_type", Encode.string "FIXED"
"price", Encode.decimal price
| MarketOrder ->
"price_type", Encode.string "AUTO"
"foo", Encode.int x.Foo
]
|> Encode.object
let decode : Decoder<CustomerRequest> =
decoder {
let! priceType =
Decode.field "price_type" Decode.string
let! price =
match priceType with
| "FIXED" ->
Decode.field "price" Decode.decimal
|> Decode.map LimitedOrder
| "AUTO" ->
Decode.succeed MarketOrder
| _ ->
Decode.fail $"Unexpected price_type `%s{priceType}`"
let! foo = Decode.field "foo" Decode.int
return
{
Price = price
Foo = foo
}
}
// Quick tests...
let requests =
[
{
Price = LimitedOrder 4.4365m
Foo = 123
}
{
Price = MarketOrder
Foo = 456
}
]
for request in requests do
let json =
request
|> CustomerRequest.encode
|> Encode.toString 0
printfn $"%s{json}"
let decoded =
json
|> Decode.unsafeFromString CustomerRequest.decode
if request <> decoded then
failwith $"Unexpected decoding: %A{decoded}"
Output:
{"price_type":"FIXED","price":"4.4365","foo":123} {"price_type":"AUTO","foo":456}
