Support deserializing f32 and f64 from null

[dtolnay]

Currently f32::NAN and f64::NAN get serialized as null but fail to deserialize back as NAN.

cc @sfackler

[sfackler]

It's a bit awkward since inifinity and -infinity also go to null :( Yay JSON

[Schultzer]

Could we make a trait wrapper?

It could be something like f64PosInfinity/f64NegInfinity or just f64Infinity ,f64NaN?

[cdbrkfxrpt]

Hi,

is there a canonical way of dealing with this?

[DanielJoyce]

This JUST bit me. Was tracking down a bug, wandering why I got nulls, then looking at the code, wondering why it wasn't blowing up or why I wasn't seeing NaN in json....

[pkolaczk]

I've just got bitten by that as well.

[patrickelectric]

What is the status ?

[DanikVitek]

Same issue

[dnut]

I came up with a few workarounds for this issue. One involves changing the json format but not the rust types. The other involves changing the rust types but not the json format.

JSON strings instead of numbers

If you're able to change the json format, you can solve this by serializing floats as strings instead of numbers. One way to do this is with #[serde_as(as = "DisplayFromStr")].

use serde::{Deserialize, Serialize};
use serde_with::{serde_as, DisplayFromStr};

#[serde_as]
#[derive(Deserialize, Serialize)]
pub struct Foo {
    #[serde_as(as = "DisplayFromStr")]
    pub my_float: f64,
}


#[test]
fn serde_json_f64_display_from_string() {
    assert!(test_round_trip(f64::NAN, "NaN").is_nan());
    assert_round_trip_eq(f64::NEG_INFINITY, "-inf");
    assert_round_trip_eq(f64::INFINITY, "inf");
    assert_round_trip_eq(1.1, "1.1");
    assert_round_trip_eq(-100.0, "-100");
    assert_round_trip_eq(0.0, "0");
    assert_round_trip_eq(f64::MIN, "-179769313486231570000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000");
    assert_round_trip_eq(f64::MAX, "179769313486231570000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000");
    assert_round_trip_eq(f64::MIN_POSITIVE, "0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000022250738585072014");
    assert_round_trip_eq(f64::EPSILON, "0.0000000000000002220446049250313");
}

fn assert_round_trip_eq(my_float: f64, expected_json: &str) {
    assert_eq!(my_float, test_round_trip(my_float, expected_json));
}

fn test_round_trip(my_float: f64, expected_json: &str) -> f64 {
    let s = serde_json::to_string(&Foo { my_float }).unwrap();
    assert_eq!(s, format!("{{\"my_float\":\"{expected_json}\"}}"));
    serde_json::from_str::<Foo>(&s).unwrap().my_float
}

Option<f64> instead of f64

You can change the struct fields from f64 to Option<f64> and it will work with the same json format. So you can actually keep the serialization code the same if you want and only add the Option on the deserialization side. But the round trip will not be perfect. None, Some(f64::NAN), Some(f64::INFINITY), and Some(f64::NEG_INFINITY) all serialize to null, but null only deserializes to None. If you're fine with getting None instead of a deserialization error in those cases, this should be a satisfactory workaround.

use serde::{Deserialize, Serialize};

#[derive(Deserialize, Serialize)]
pub struct Foo {
    pub my_float: f64,
}

#[derive(Deserialize, Serialize)]
pub struct FooOption {
    pub my_float: Option<f64>,
}

#[test]
fn serde_json_option_f64() {
    assert!(test_round_trip(None, "null").is_none());
    assert!(test_round_trip(Some(f64::NAN), "null").is_none());
    assert!(test_round_trip(Some(f64::NEG_INFINITY), "null").is_none());
    assert!(test_round_trip(Some(f64::INFINITY), "null").is_none());
    assert_round_trip_eq(Some(1.1), "1.1");
    assert_round_trip_eq(Some(-100.0), "-100.0");
    assert_round_trip_eq(Some(0.0), "0.0");
    assert_round_trip_eq(Some(f64::MIN), "-1.7976931348623157e308");
    assert_round_trip_eq(Some(f64::MAX), "1.7976931348623157e308");
    assert_round_trip_eq(Some(f64::MIN_POSITIVE), "2.2250738585072014e-308");
    assert_round_trip_eq(Some(f64::EPSILON), "2.220446049250313e-16");
}

#[test]
fn serde_json_f64_to_option() {
    assert!(test_round_trip_to_opt(f64::NAN, "null").is_none());
    assert!(test_round_trip_to_opt(f64::NEG_INFINITY, "null").is_none());
    assert!(test_round_trip_to_opt(f64::INFINITY, "null").is_none());
    assert_round_trip_to_opt_eq(1.1, "1.1");
    assert_round_trip_to_opt_eq(-100.0, "-100.0");
    assert_round_trip_to_opt_eq(0.0, "0.0");
    assert_round_trip_to_opt_eq(f64::MIN, "-1.7976931348623157e308");
    assert_round_trip_to_opt_eq(f64::MAX, "1.7976931348623157e308");
    assert_round_trip_to_opt_eq(f64::MIN_POSITIVE, "2.2250738585072014e-308");
    assert_round_trip_to_opt_eq(f64::EPSILON, "2.220446049250313e-16");
}

fn assert_round_trip_eq(my_float: Option<f64>, expected_json: &str) {
    assert_eq!(my_float, test_round_trip(my_float, expected_json));
}

fn assert_round_trip_to_opt_eq(my_float: f64, expected_json: &str) {
    assert_eq!(Some(my_float), test_round_trip_to_opt(my_float, expected_json));
}

fn test_round_trip(my_float: Option<f64>, expected_json: &str) -> Option<f64> {
    let s = serde_json::to_string(&FooOption { my_float }).unwrap();
    assert_eq!(s, format!("{{\"my_float\":{expected_json}}}"));
    serde_json::from_str::<FooOption>(&s).unwrap().my_float
}

fn test_round_trip_to_opt(my_float: f64, expected_json: &str) -> Option<f64> {
    let s = serde_json::to_string(&Foo { my_float }).unwrap();
    assert_eq!(s, format!("{{\"my_float\":{expected_json}}}"));
    serde_json::from_str::<FooOption>(&s).unwrap().my_float
}

[xNxExOx]

Is there a workaround that does not involve wraping all library types that contains floats, to make custom serualizers for them?

[patrickelectric]

I solved this issue by using https://crates.io/crates/json5 over json

[strohel]

My workaround was to add add #[serde(deserialize_with = "..."] to all f64 fields. Cumbersome and not applicable for all cases, but concise for the simple ones:

#[derive(Serialize, Deserialize)]
pub struct MyStruct {
    #[serde(deserialize_with = "deserialize_f64_null_as_nan")]
    pub field: f64,
}

/// A helper to deserialize `f64`, treating JSON null as f64::NAN.
/// See https://github.com/serde-rs/json/issues/202
fn deserialize_f64_null_as_nan<'de, D: Deserializer<'de>>(des: D) -> Result<f64, D::Error> {
    let optional = Option::<f64>::deserialize(des)?;
    Ok(optional.unwrap_or(f64::NAN))
}

deserialize_f64_null_as_nan() could be made generic over f32/f64 with some effort.