Add support for serializing and deserializing JsValue?

[allsey87]

Let say we have a struct in Rust as follows:

struct Test {
   v1: JsValue,
   v2: JsValue,
}
let v1 = JsValue::from("hello");
let v2 = JsValue::from("world");
let t = Test { v1, v2 };

It would be nice to be able to pass t back and forth between Rust and Javascript, where during "serialization" v1 and v2 are just replaced with the actual Javascript values they represent, and on deserialization, the Javascript values are pushed onto wasm-bindgen's heap and the struct members are initialized with JsValues that reference those Javascript objects on the heap.

At the moment, this doesn't work out of the box since Serialize is not implemented for JsValue, which makes sense since it's internal representation is just an index to wasm-bindgen's heap, but perhaps there is a way to hack this, possibly with #[serde(serialize_with = "path")] on the JsValue fields?

[allsey87]

Thinking about this a bit more, I suspect that such a feature would require additional functionality from both serde and wasm-bindgen sides to achieve this...

[RReverser]

Yeah unfortunately that wouldn't be possible.

[allsey87]

@RReverser do you have any thoughts regarding workarounds that could achieve something like this, i.e., moving a Rust struct containing JsValues to Javascript?

[RReverser]

I think instead of using Serde in this case you'd just have to implement conversion via manual methods, where you take a JsValue and step-by-step extract the properties you're interested in to a Rust struct or vice versa.

Of course, if the JsValue you want to preserve is a primitive JSON-like value, you can instead use serde_json::Value or a similar container that can be used both in deserialization and serialization by any serde implementation, including serde-wasm-bindgen.

[allsey87]

I think there is a limitation with that approach in that it will not work for types from web_sys such as OffscreenCanvas or MessagePort since these types are not representable in Rust.

[RReverser]

If you mean the 2nd one, then yeah, as I said, it works only for primitive JSON-like values. For anything else you need to implement custom conversion methods.

[RReverser]

Hmm actually, while very hacky, it could be possible for serde-wasm-bindgen to provide some sort of serde_wasm_bindgen::PreservedValue(JsValue) method (and, correspondingly, helper {de}serialize_with functions).

PreservedValue would need to ask the deserializer to deserialize a newtype with some internally-known unique name (like GUID), and on the serde-wasm-bindgen deserializer side it could recognize the name, and return the Abi ID to the caller. Then, PreservedValue would get this ID back and it would reconstruct the JsValue via FromAbi back.

This way, it could preserve JsValue instances as-is while the rest around it would be deserialized.

Unfortunately, I'm pretty busy right now and won't be able to dig into this more, but if this makes sense to you, you're welcome to try in a PR :)

[allsey87]

That does sounds pretty hacky! While at a high level I follow what you are saying, I don't think I have quite enough experience with serde or wasm-bindgen at this stage to be able to put a PR together in a reasonable time frame. So unfortunately I think this idea will have to stay on the back burner for the moment.

[RReverser]

While at a high level I follow what you are saying, I don't think I have quite enough experience with serde or wasm-bindgen at this stage to be able to put a PR together in a reasonable time frame.

In my experience, implementing this kind of hacks is the best and fun way to learn :) But, up to you, of course.

[allsey87]

I completely agree, however, since I am also trying to start a business, I have to be pragmatic and unfortunately just work around these sorts of issues from time to time :)

[futursolo]

Instead of implementing Serialize and Deserialize for all JsValue-like types, which requires to be implemented on the type itself (on wasm_bindgen, js_sys and web_sys side), maybe a type JsTyped<T> where T: JsCast + AsRef<JsValue> can be created?

So JsTyped<T> has 3 attributes:

Serialize is implemented with .as_ref(). Deserialize is implemented with JsCast::dyn_into(). Deref<Target = T>, AsRef<T> and From<T> provides ergonomics.

// can directly derive here because `JsTyped<Element>` is Serialize + Deserialize
#[derive(Serialize, Deserialize)]
struct SomeValueToPassToJs {
    element: JsTyped<Element>,
}

[RReverser]

So JsTyped<T> has 3 attributes:

Serialize is implemented with .as_ref(). Deserialize is implemented with JsCast::dyn_into().

Problem is, you can't implement Serialize/Deserialize using those methods because they're generic over any serializers/deserializers, not just serde-wasm-bindgen, and follow a visitor pattern that doesn't know anything about JS values. https://serde.rs/impl-deserialize.html

Instead, you need something like described in https://github.com/cloudflare/serde-wasm-bindgen/issues/32#issuecomment-1016466190 (messing up with internal handles).

[futursolo]

Problem is, you can't implement Serialize/Deserialize using those methods because they're generic over any serializers/deserializers, not just serde-wasm-bindgen

The intuitive answer seems to be no, but actually there is a way to do it (if we are talking about hacks).

There're 2 quirks with JsValue (and anything contains JsValue):

1. JsValue is not Send (hence serializer / deserializer are not Send)
2. There cannot be 2 serializers / deserializers running at the same time in the same thread.

Hence, serializer and deserializer can be detected with thread-locals.

[RReverser]

There cannot be 2 serializers / deserializers running at the same time in the same thread.

Why not? You can have nested serializers or deserializers.

Personally, I'd prefer to go with newtype + handles, those are safe in all scenarios.

[brandonros]

#[derive(Debug, Serialize, Deserialize)]
#[wasm_bindgen]
pub struct CanFrame {
    pub id: u32,
    pub data: heapless::Vec<u8, 8>,
}

 --> protocol/src/lib.rs:4:26
  |
4 | use serde_wasm_bindgen::{from_value, to_value, Error, Serializer};
  |                          ^^^^^^^^^^  ^^^^^^^^  ^^^^^  ^^^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

error[E0277]: the trait bound `heapless::Vec<u8, 8_usize>: IntoWasmAbi` is not satisfied
  --> protocol/src/lib.rs:55:1
   |
55 | #[wasm_bindgen]
   | ^^^^^^^^^^^^^^^ the trait `IntoWasmAbi` is not implemented for `heapless::Vec<u8, 8_usize>`
   |
   = help: the following other types implement trait `IntoWasmAbi`:
             &'a (dyn Fn() -> R + 'b)
             &'a (dyn Fn(A) -> R + 'b)
             &'a (dyn Fn(A, B) -> R + 'b)
             &'a (dyn Fn(A, B, C) -> R + 'b)
             &'a (dyn Fn(A, B, C, D) -> R + 'b)
             &'a (dyn Fn(A, B, C, D, E) -> R + 'b)
             &'a (dyn Fn(A, B, C, D, E, F) -> R + 'b)
             &'a (dyn Fn(A, B, C, D, E, F, G) -> R + 'b)
           and 198 others
   = note: this error originates in the attribute macro `wasm_bindgen` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0277]: the trait bound `heapless::Vec<u8, 8_usize>: FromWasmAbi` is not satisfied
  --> protocol/src/lib.rs:55:1
   |
55 | #[wasm_bindgen]
   | ^^^^^^^^^^^^^^^ the trait `FromWasmAbi` is not implemented for `heapless::Vec<u8, 8_usize>`
   |
   = help: the following other types implement trait `FromWasmAbi`:
             *const T
             *mut T
             CanFrame
             Clamped<T>
             FilterType
             JsValue
             _::_serde::__private::Vec<T>
             bool
           and 93 others
   = note: this error originates in the attribute macro `wasm_bindgen` (in Nightly builds, run with -Z macro-backtrace for more info)

This isn't exactly related but maybe it'd be helpful if in the repo an example of how to implement this kind of trait was done so that a nested struct field could be easily serialized/deserialized to+from Rust->WASM?

[RReverser]

@brandonros That's very much irrelevant to this issue, yes.

As your error says, the problem is that heapless::Vec<u8> is not compatible with wasm-bindgen crates; but if you're using Serde Serialize/Deserialize, you don't need #[wasm_bindgen] on your struct anyway.

If you remove it, that error should go away.

[jquesada2016]

Since JsValue is just an integer pointer, why not have JsTyped<T: JsCast>: JsCast + Deref<JsValue> serialize to usize. as when constructing the JsValue, we can trivially cast back to the original type. For deserializing, much the same should happen.

This also takes advantage of the !Send + !Sync assumption, and would make doing the unsafe casts sound.

[jquesada2016]

The biggest issue with my suggestion, is that there's no way to know what serializer will be used, (with specialization we could), so if the user serialized to, for example, JSON, then deserializing would necessarily not work, so some trickery would need to happen to make this work in that case. But perhaps all of this could be mitigated with an explicit unsafe naming, such as UnsafeJsTyped<T> where the docs specify the safety requirement to only use serde_wasm_bindgen deserializer.

[RReverser]

That was more or less my suggestion and see the open PR at #40 that basically does that. It needed a few minor final touches but fell off the radar I suspect.

[RReverser]

Closed by #40.