Allow deserialize_seq to deserialize binaries.
I came across the bug ** (ErlangError) Erlang error: :nif_panicked when trying to do the following NIF call in Elixir: AnomaSDK.Arm.encrypt_cipher(<<96, 109, 146, 170, 18, 27, 248, 118, 203, 31, 65, 72, 51, 153, 179, 35, 168, 62, 88, 150, 244, 119, 198, 51, 20, 106, 74, 8, 143, 32, 45, 168>>). The NIF is defined as follows:
#[nif]
fn encrypt_cipher(SerdeTerm(cipher): SerdeTerm<Vec<u8>>) -> SerdeTerm<Ciphertext> { ... }
Tracking down this error led me to the Deserializer::deserialize_seq implementation. It seems that the function does not have the ability to treat Elixir binary terms as sequences. This PR enables Elixir binaries to be treated as sequences (in addition to lists and empty lists) which fixes the NIF panic described above.
And thanks for maintaining Rustler, this crate is extremely useful!
Interesting! Can you add test cases as well (especially cases in
rustler_tests?
I have added a test case which checks that <<2, 3, 5, 7, 11, 13>> can deserialise to the Rust Vec<u8>: vec![2, 3, 5, 7, 11, 13]. But vec![2, 3, 5, 7, 11, 13] still serializes to the Elixir list [2, 3, 5, 7, 11, 13] since I'm not sure how to specialize the serialization of Vec<u8>s. So the byte array conversion in https://docs.rs/rustler/latest/rustler/serde/index.html#conversion-table now only half works (when it did not seem to work at all before).
But upon some reflection, this solution is not so nice because it is using Serializer::serialize_seq and Derializer::deserialize_seq to serialize/deserialise binaries when the more appropriate methods are Serializer::serialize_bytes and Deserializer::deserialize_bytes. Using the ByteArray and ByteBuf types from the serde_bytes crate (https://docs.rs/serde_bytes/latest/serde_bytes/) instead of Vec<u8>s causes the latter two methods to be invoked and makes the ** (ErlangError) Erlang error: :nif_panicked error above go away.
So a better/less intrusive solution than this PR might be to just update the documentation at https://docs.rs/rustler/latest/rustler/serde/index.html#conversion-table to either indicate that the Serde (Rust) Value for a byte array is a ByteArray or ByteBuf, or alternatively indicate that Serde (Rust) Value is a Vec<u8> only when serialization/deserialisation happens with serde_bytes (#[serde(with = "serde_bytes")]).