Inconsistent deserialization of empty/missing fields in JSON to Optional

[proshin-roman]

I was looking around for a way how to differentiate between a field with null value and a missing field by having only the final Java object. And my assumption was that Jackson already handles it by using Optional<?>.

Just a short example:

Assuming I have the following Java DTO

class Foo {
    Optional<String> bar;
}

I also have two JSON input payloads: payload 1

{
    "bar": null
}

and payload 2 (empty JSON object)

{
}

then I would expect the following behavior of the deserialization process:

• payload 1 is deserialized into an instance of Foo class with bar equal to an empty Optional (Optional.empty());
• payload 2 is deserialized into an instance of Foo class with bar equal to null.

In that way I could easily differentiate between cases when the input field was presented but equal to null and when the input input field was missing. Then I can implement my business logic accordingly. It might be especially important for implementing PATCH methods in CRUD APIs (no field - no changes; field is null - update the value).

But now I see that the logic is different: Optional is set to empty() in both cases mentioned above. As I can see, it all goes to this method com.fasterxml.jackson.datatype.jdk8.OptionalDeserializer#getNullValue that always wraps the null value into an instance of Optional. In my opinion it's a bug, as it lowers the flexibility of the mapping.

What do you think, guys? Maybe there is a workaround for my problem? If so, I will appreciate any link or idea.

PS: initially the issue was reported in https://github.com/FasterXML/jackson-modules-java8/issues/154#issuecomment-938851160

[hjohn]

Optional is not a tri-state. It's goal is to prevent you having to do null checks, and allowing an Optional to be deserialized as null for whatever reason would defeat the purpose of using it. For the same reason a method returning an Optional should never return null.

[proshin-roman]

Optional is not a tri-state. It's goal is to prevent you having to do null checks, and allowing an Optional to be deserialized as null for whatever reason would defeat the purpose of using it. For the same reason a method returning an Optional should never return null.

I see your point and it actually makes a lot of sense. However, I would like to have at least some way of implementing the logic I explained in the original post. It could have been even some special type provided by the library - e.g. "NullableOptional" or better smth like "TripleStateValue".

[hjohn]

I have to guess at what you're trying to achieve, but there is a way to distinguish between not present and null that Jackson offers, which can be useful for implementing PATCH methods for example:

JsonNode input = ... ;   // data which may have explicit nulls and absent values

SomeObject existingData = getExistingDataFromSomewhere();  // data representing current state

ObjectReader readerForUpdating = objectMapper.readerForUpdating(existingData);

SomeObject mergedData = readerForUpdating.readValue(input);

The above mergedData will contain values and explicit nulls from the input but will leave absent values to their current value. Maybe this can be useful?

[cowtowncoder]

Another thing to consider is this: currently Jackson can only detect and handle missing property values when processing "creator methods" (constructors and factory methods) -- this is the only case when existence is tracked. For setters and fields there simply is no call and no processing occurs.

So any logic for basic databinding (not including case where intermediate JsonNode is used, in which one can of course do whatever) one typically wants to use Constructor to pass values if "absent" value handling is needed.

Now: to coerce missing Optional into "empty", there are 2 ways to go about:

1. Custom deserializer
2. Use null coercion, either via @JsonSetter(nulls = ...) (or contentNulls), or equivalent mapper config (see f.ex src/test/java/com/fasterxml/jackson/databind/deser/filter/NullConversionsPojoTest.java)
3. Actually initialize Optional field and do NOT use Creator method: missing value will simply not change defaults at all

For (2) what you'd want, make Optional.class have coercion from null to AS_EMPTY. Not sure if there's a test to verify this works; it should but I think it may require JsonDeserializer to explicitly support some aspects.

[rkeytacked]

@proshin-roman: like @cowtowncoder mentioned in point (3)

• just do not use the Creator method.

My complete example for exactly your use case works really well up to date. Lombok adds an empty constructor plus getters/setters via @Data:

import java.util.Optional;
import lombok.Data;

@Data
public class Foo {
  private Optional<Long> bar;
  private Optional<String> foobar;
}

I even added unit tests in my project to ensure that missing fields stay null, and existing fields with null JSON value become Optional.EMPTY.

{
  "foobar" : null
}

turns into => Foo(bar=null, foobar=Optional.EMPTY)

[proshin-roman]

thank you, @cowtowncoder, @rkeytacked and @hjohn for your answers - I will try solutions you proposed and either close the ticket or leave another comment 👍