Btree map serialization results in error.

[amrx101]

Are map type supported?

  {
        "type": "record",
        "name": "test",
        "fields": [
            {"name": "a", "type": "long", "default": 42},
            {"name": "b", "type": "string"},
            {"name": "additional", "type": {"type": "map", "values": "string"}}
        ]
    }

Herein the struct test has a third parameter called additional which is a map.

use avro_rs::{Codec, Reader, Schema, Writer, from_value, types::Record};
use failure::Error;
use serde::{Serialize, Deserialize};
use std::fs::File;
use std::io::prelude::*;
use std::collections::{HashMap, BTreeMap};



  #[derive(Debug, Deserialize, Serialize)]
   struct Test {
  a: i64,
  b: String,
   additional: BTreeMap<String, String>,
}



   fn main() -> Result<(), Error> {
let raw_schema = r#"
    {
        "type": "record",
        "name": "test",
        "fields": [
            {"name": "a", "type": "long", "default": 42},
            {"name": "b", "type": "string"},
            {"name": "additional", "type": {"type": "map", "values": "string"}}
        ]
    }
"#;

let schema = Schema::parse_str(raw_schema)?;

// println!("{:?}", schema);

let mut writer = Writer::with_codec(&schema, Vec::new(), Codec::Deflate);

let mut record = Record::new(writer.schema()).unwrap();
record.put("a", 27i64);
record.put("b", "foo");

let mut m: BTreeMap<String, String> = BTreeMap::new();
record.put("additional", m);

writer.append(record)?;

let test = Test {
    a: 27,
    b: "foo".to_owned(),
    additional: BTreeMap::new(),
};

writer.append_ser(test)?;

writer.flush()?;

let input = writer.into_inner();
let reader = Reader::with_schema(&schema, &input[..])?;

for record in reader {
    println!("{:?}", from_value::<Test>(&record?));
}
Ok(())

}

This results in Error

    the trait `avro_rs::types::ToAvro` is not implemented for `std::collections::BTreeMap<std::string::String, std::string::String

However the de-serialization works fine.

[poros]

I don't see any theoretical impediment to serializing BTreeMap, but your code clearly shows that this doesn't work atm.

We do have a bunch of tests on schema validations of maps and one for the encoding of Value::Map, but I cannot find any covering native rust types serialization and deserialization. There might be a bug with all kind of maps or perhaps some serde method we did not implement for BTreeMap.

If you'd like to contribute, a good first step would be to add a couple of serialization and deserialization tests for HashMap to check whether that works at all and use that information to figure out where the problem is.

[amrx101]

@poros , I did some tests, HashMap does actually work. I would be happy to help if provided some pointers(especially what you or someone familiar with code-base would have done).

[poros]

It would be great if you could add your tests for HashMap (and then for BTreeMap once it works) to a place like types.rs.

As for discovering where the problem is, I would start from ToAvro in types.rs as the error says and, in case it is not there, maybe move to ser.rs or de.rs.

[amrx101]

@poros , Thanks for pointers. I have a question though, the Value enum in types.rs maps the Avro map type to Rust HashMap and implements ToAvro trait for HashMap. I believe I need to implement the ToAvro trait for BtreeMap. What I am confused about is how do we allow the Avro Map type to correspond to both BtreeMap and HashMap ? Any ideas there?

Reference

[amrx101]

@poros any pointer on the previous comment would be great. Thanks

[poros]

If I understand your original request correctly, I don't think you are interested in changing the internal implementation of the Value type, but only allow a conversion from and to BtreeMap. As I mention before, I think you could at least allow conversion from BtreeMap by implementing the ToAvro trait for such a type.

[amrx101]

@poros , noted. Will do so and add tests for it as well.