RUST-1120 bson's is_human_readable not configurable from mongodb side

[alon-z]

Hey, we are trying to save a struct with ipnet::Ipv4Net and the serializers are using the is_human_readable serde option to distinguish between different cases. When inserting the struct, it serializes with is_human_readable false (this issue is referenced in bson's 2.1.0-beta changelog) but when using find commands, the deserializer is set with is_human_readable true by default. In result, we cant save and use the struct in mongo.

In bson 2.1.0-beta they added options to set the is_human_readable variable. If there could be a way we can set it so the deserializer will set is_human_readable to false when using find commands, we will be able to solve this issue :)

Example below

use ipnet::Ipv4Net;
use mongodb::error::Result as MongoResult;
use serde::{Deserialize, Serialize};

#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct Subnets {
    pub private: Ipv4Net,
    pub public: Ipv4Net,
}

impl Subnets {
    pub async fn init(subnets: Vec<Subnets>) -> MongoResult<mongodb::Cursor<Subnets>> {
        let client = mongodb::Client::with_uri_str("mongodb://localhost:27017")
            .await
            .expect("Failed to initialize mongodb client");
        let db = client.database("ipnet");
        let coll = db.collection::<Subnets>("subnets");

        coll.insert_many(subnets, None).await?; // Will insert the address like Array["10","0","0","1","16"]
        coll.find(None, None).await // Try's to find address like "10.0.0.1/16"
    }
}

[patrickfreed]

Hey @alon-z, thanks for filing this issue! So the deserializer used by the driver is always non-human-readable, but it turns out that this is side-stepped by a limitation/bug in serde (see https://github.com/serde-rs/serde/issues/1183). The driver runs into this because 2.0 and 2.1 use #[serde(flatten)] to deserialize server responses. Fortunately, 2.2.0 (not yet released) will change the way we do deserialization to avoid using #[serde(flatten)], which should fix this issue.

In the meantime before 2.2.0 is released, you can try depending on the master branch of the driver directly, though I'd caution against doing that in production as we don't have any stability guarantees for non-released versions. Alternatively, you can use a #[serde(with = "...")] annotation to add some custom deserialization logic to work around this.

Sorry for any inconvenience caused by this issue, and thanks again for bringing this to our attention!

[alon-z]

Hey @patrickfreed , thank you for the quick response. I have tried to use the master branch, but still facing the same issue. The struct is saved in a non-human-readable format.

{
  "_id": {
    "$oid": "61aca64907410a9404025d9f"
  },
  "private": [
    10,
    16,
    0,
    0,
    28
  ],
  "public": [
    10,
    112,
    0,
    0,
    28
  ]
}

But when trying to use a find command, I get the same error about parsing the address: invalid type: sequence, expected IPv4 network address

The ipnet module will serialize in this format only when is_human_readable is false, and if I edit the record myself to a simple string "10.16.0.0/28", find commands works perfectly.

[patrickfreed]

Hmm, are you sure you're using the master branch? You may need to run cargo update or cargo clean after updating your Cargo.toml. If you look in your Cargo.lock file, you can see what commit the driver is pinned to by looking at the source field of the [[package]] entry for mongodb. For me it says "git+https://github.com/mongodb/mongo-rust-driver#91987a04e4671fddf3a54376000d0a30c9f30436", and with that commit, the following sample works just fine:

#[derive(Debug, Deserialize, Serialize)]
struct D {
    addr: Ipv4Net,
}

#[tokio::main]
async fn main() -> std::result::Result<(), Box<dyn std::error::Error>> {
    let client = Client::with_uri_str("mongodb://localhost:27017").await?;
    let collection = client.database("ipv4").collection("ipv4");

    collection.insert_one(D { addr: Ipv4Net::new(Ipv4Addr::new(127, 0, 0, 1), 16)? }, None).await?;
    collection.insert_one(D { addr: Ipv4Net::new(Ipv4Addr::new(127, 0, 0, 1), 16)? }, None).await?;

    let mut cursor = collection.find(None, None).await?;
    while let Some(d) = cursor.try_next().await? {
        println!("{:?}", d);
    }

    Ok(())
}

and prints:

D { addr: 127.0.0.1/16 }
D { addr: 127.0.0.1/16 }

So, it seems like it should be working generally.

[jackbbhua]

i have to say the official docs is su*ks!!! why not give a complete demo!

[kmahar]

Hi @jackbbhua, can you please elaborate on what you are looking for? Our README has embedded a number of examples, as well as links to our docs.rs page which contains some further examples, and an example application using Actix.