Allow returning Vec<T>

[rookboom]

It would really be great to be able to return a vector of structs or tuples:

For example. I have the following type:

#[wasm_bindgen]
struct Range {
    offset: u32,
    length: u32
}

that I want to return in my function

#[wasm_bindgen]
pub fn token_ranges(text: &str) -> Vec<Range> 

I am getting this error:

   Compiling picl_wasm_runtime v0.1.0 (file:///A:/Repos/picl_native_runtime/bindings/typescript)
error[E0277]: the trait bound `std::boxed::Box<[Range]>: wasm_bindgen::convert::WasmBoundary` is not satisfied
  --> src\lib.rs:15:1
   |
15 | #[wasm_bindgen]
   | ^^^^^^^^^^^^^^^ the trait `wasm_bindgen::convert::WasmBoundary` is not implemented for `std::boxed::Box<[Range]>`
   |
   = help: the following implementations were found:
             <std::boxed::Box<[u16]> as wasm_bindgen::convert::WasmBoundary>
             <std::boxed::Box<[i16]> as wasm_bindgen::convert::WasmBoundary>
             <std::boxed::Box<[f32]> as wasm_bindgen::convert::WasmBoundary>
             <std::boxed::Box<[i32]> as wasm_bindgen::convert::WasmBoundary>
           and 5 others
   = note: required because of the requirements on the impl of `wasm_bindgen::convert::WasmBoundary` for `std::vec::Vec<Range>`

My workaround is to flatten my data and return a Vec and then splice my token ranges on the JS side. This is unfortunate...

How can I add a WasmBoundary trait for a custom type? Is there a better way?

[alexcrichton]

Thanks for the report!

Right now there's not a great way to do this in wasm-bindgen itself in terms of the conversion here has to be also accompanied with a conversion on the JS side which may be somewhat tricky. It should certainly be possible though given enough support!

[Hywan]

Same issue here :-).

What kind of support do you need?

[alexcrichton]

@rookboom @Hywan do y'all basically need Vec<T> where T has #[wasm_bindgen] on it?

[Hywan]

@alexcrichton Exactly, that's precisely my usecase.

[rookboom]

Yes, that would be fantastic.

[alexcrichton]

Ok this is then I think pretty similar to https://github.com/rustwasm/wasm-bindgen/issues/104. We somehow need a way to communicate this to the CLI tool but currently the strategy for doing that is quite limited.

[gabrielcarneiro97]

I'm with a similar issue here, but with a complex struct.

code:

#[derive(Clone)]
#[wasm_bindgen]
pub struct Coords {
    x: usize,
    y: usize
}

#[wasm_bindgen]
pub struct Cell {
    state: State,
    position: Coords,
    neighboors: Vec<Coords>,
    neighboors_alive: i32
}

#[wasm_bindgen]
impl Cell {
    pub fn new(state: State, position: Coords, neighboors: Vec<Coords>) -> Cell {
        Cell {
            state,
            position,
            neighboors,
            neighboors_alive: 0
        }
    }
}

error:

error[E0277]: the trait bound `std::boxed::Box<[Coords]>: wasm_bindgen::convert::FromWasmAbi` is not satisfied
  --> src\lib.rs:36:1
   |
36 | #[wasm_bindgen]
   | ^^^^^^^^^^^^^^^ the trait `wasm_bindgen::convert::FromWasmAbi` is not implemented for `std::boxed::Box<[Coords]>`
   |
   = help: the following implementations were found:
             <std::boxed::Box<[u16]> as wasm_bindgen::convert::FromWasmAbi>
             <std::boxed::Box<[wasm_bindgen::JsValue]> as wasm_bindgen::convert::FromWasmAbi>
             <std::boxed::Box<[u8]> as wasm_bindgen::convert::FromWasmAbi>
             <std::boxed::Box<[f32]> as wasm_bindgen::convert::FromWasmAbi>
           and 5 others
   = note: required because of the requirements on the impl of `wasm_bindgen::convert::FromWasmAbi` for `std::vec::Vec<Coords>`

[David-OConnor]

Gabriel and rook - have y'all found a workaround? Solving or working around this would add much flexibility to WASM in Rust.

Eventually, being able to use HashMaps, or structs from other packages (Like ndarrays) would be nice, but having some type of collection that maps to JS arrays would be wonderful; not sure if I can continue my project without this.

[rookboom]

My workaround is to pass JSON over the wasm boundary... Not ideal but works for now.

[Gisleburt]

For those looking for a workaround on this, if you can turn your data into a Vec<u8> or &[u8]

#[wasm_bindgen]
pub struct ByteStream {
    offset: *const u8,
    size: usize,
}

#[wasm_bindgen]
impl ByteStream {
    pub fn new(bytes: &[u8]) -> ByteStream {
        ByteStream {
            offset: bytes.as_ptr(),
            size: bytes.len(),
        }
    }

    pub fn offset(&self) -> *const u8 {
        self.offset
    }

    pub fn size(&self) -> usize {
        self.size
    }
}

A good example of how to use this is creating a texture in Rust to render in Javascript, so for example:

#[wasm_bindgen]
pub fn render() -> ByteStream {
  let texture = Vec::new();
  // ...
  ByteStream::new(&texture);
}

const texture = render();
const textureRaw = new Uint8ClampedArray(memory.buffer, texture.offset(), texture.size());
const image = new ImageData(textureRaw, width, height);

[fitzgen]

Rather than returning pointers and lengths manually, you can use this, which should be slightly less error prone: https://docs.rs/js-sys/0.3.9/js_sys/struct.Uint8Array.html#method.view

[stefan2718]

If anyone is still looking at this, I was able to work around this using Serde to serialize/deserialize the data. This was the guide I used: https://rustwasm.github.io/docs/wasm-bindgen/reference/arbitrary-data-with-serde.html

Edit: For those wanting to avoid JSON serialization, the guide above also includes a link to serde-wasm-bindgen which "leverages direct APIs for JavaScript value manipulation instead of passing data in a JSON format."

[theashguy]

I'm wanting to take this one step further and return Vec<js_sys::Function> which means no serde serialization for me (Maybe there is some kind of ref I could serialize if I dug into the internals?). The workaround I'm kicking about at the moment is to create the collection on the JS side and expose some methods for appending and cleaning up.

Something like...

window.vecCacheOfAnything = {
  append: function (key, item) {
    if(!window.vecCache.cache[key]) {
      window.vecCache.cache[key] = []
    }

    window.vecCache[key].push(item)
  },
  clear: function (key) {
    delete window.vecCache.cache[key]
  },
  cache: {}
}

Not suitable for prod work and by no means ideal or ergonomic but I'm really just thrashing around to see how far I can take rustwasm at the moment :D. Possibly an idea for anyone blocked by this issue and wanting to do things more complex. Also an extra AC for the team to add to the backlog!

[Zireael07]

This issue is kind of a blocker for pretty much any sort of a bigger project using rustwasm.

[Pauan]

It's not a complete solution, but I created #1749 which adds in FromIterator for Array:

use js_sys::Array;

#[wasm_bindgen]
pub fn token_ranges(text: &str) -> Array {
    get_vec_somehow().into_iter().collect()
}

This means that now you can send Vec<T> to JS, you just have to return an Array and use .into_iter().collect() to convert the Vec<T> into an Array.

[dasoncheng]

Same issue here :-).

#[wasm_bindgen]
#[derive(Debug)]
pub struct HeartBeat {
    pub template: u8,
    pub classify: u8,
    pub index: u32,
    pub tr: u16,
    pub hr: u16,
    pub feature: [[f32; 20]; 3],
}

the trait bound `[[f32; 20]; 3]: wasm_bindgen::convert::traits::IntoWasmAbi` is not satisfied

the trait `wasm_bindgen::convert::traits::IntoWasmAbi` is not implemented for `[[f32; 20]; 3]`

help: the following implementations were found:
        <&'a [f32] as wasm_bindgen::convert::traits::IntoWasmAbi>
        <&'a [f64] as wasm_bindgen::convert::traits::IntoWasmAbi>
        <&'a [i16] as wasm_bindgen::convert::traits::IntoWasmAbi>
        <&'a [i32] as wasm_bindgen::convert::traits::IntoWasmAbi>
      and 20 othersrustc(E0277)
lib.rs(38, 1): the trait `wasm_bindgen::convert::traits::IntoWasmAbi` is not implemented for `[[f32; 20]; 3]`

[dragly]

@Pauan Great! Thanks for looking into this.

However, I am encountering the following error now.

the trait `std::convert::AsRef<wasm_bindgen::JsValue>` is not implemented for `MyObject`
note: required because of the requirements on the impl of `std::iter::FromIterator<MyObject>` for `js_sys::Array`

from the following code:

#[wasm_bindgen]
struct MyObject {
    a: f32,
    b: f32,
}

#[wasm_bindgen]
pub fn test() -> Array {
    let objects = vec![
        MyObject {
            a: 123.0,
            b: 1024.1,
        },
        MyObject {
            a: 456.7,
            b: 1024.8,
        },
    ];
    objects.into_iter().collect()
}

Am I missing something?

This is using wasm-bindgen = "0.2.51". Maybe your change is not in yet? The error is different from before, however, so it seems like something changed.

[Pauan]

@dragly As explained in the PR, you need to use .map(JsValue::from), like this:

objects.into_iter().map(JsValue::from).collect()

This is because structs are a Rust data type, and so you have to manually use JsValue::from to convert them into a JS data type (the same is true for other Rust data types like &str, i32, etc.).

[Kinrany]

As a workaround, is it possible to also specify the TypeScript type returned by the function?

By default fn foo() -> Array is compiled into () => any[] instead of something like () => string[].

[Pauan]

@Kinrany Yes, but it requires a bit of a hack:

#[wasm_bindgen]
extern "C" {
    #[wasm_bindgen(typescript_type = "Array<string>")]
    type MyArray;
}

(The typescript_type attribute can specify any TypeScript type, even complex types like |)

Now you just do fn foo() -> MyArray and use .unchecked_into::<MyArray>() to cast the Array into MyArray.

[tlaukkan]

It would still be awesome to have builtin support for something like this which is quite normal case:

#[wasm_bindgen]
pub struct IntersectResult {
    pub hit: bool,
    pub triangle_index: u32,
    pub u: f32,
    pub v: f32,
    pub distance: f32,
}

#[wasm_bindgen]
pub struct IntersectResultArray {
    pub intersects: Vec<IntersectResult>,
}

[ivnsch]

What do we have to do currently to achieve this? E.g to pass Vec<Foo> to JS:

struct Foo {
    field: i32,
    str: String,
}

[fjarri]

@i-schuetz : the workaround that I use is having a return type of Vec<JsValue> and converting your vector to it on return: myvec.iter().map(JsValue::from).collect().

[fjarri]

Does this issue also cover the support for Vec<T> function arguments? Because that's also impossible at the moment, and much harder to work around.

[ppershing]

Unfortunately, solutions with js_sys have huge performance overheads. You can also return Box<[T]> with T being basic numeric type which is efficient on the JS side (it just slices relevant wasm memory). This however needs a copy on the Rust side. In general, it is faster than js_sys though.

What I come up with (and seems to be faster by a margin) is to have the following wrapper type:

pub struct MySlice<T> {
    phantom: std::marker::PhantomData::<T>,
    _ptr: u32,
    _len: u32,
}

impl<T: wasm_bindgen::describe::WasmDescribe> wasm_bindgen::describe::WasmDescribe for MySlice<T> {
    fn describe() {
        wasm_bindgen::describe::inform(wasm_bindgen::describe::REF);
        wasm_bindgen::describe::inform(wasm_bindgen::describe::SLICE);
        T::describe();
    }
}

impl<T: wasm_bindgen::describe::WasmDescribe> wasm_bindgen::convert::IntoWasmAbi for MySlice<T> {
    type Abi=wasm_bindgen::convert::WasmSlice;

    #[inline]
    fn into_abi(self) -> wasm_bindgen::convert::WasmSlice {
        wasm_bindgen::convert::WasmSlice {
            ptr: self._ptr,
            len: self._len,
        }
    }
}

impl<T> std::convert::From<&Vec<T>> for MySlice<T> {
    fn from(vec: &Vec<T>) -> Self {
        let _ptr = vec.as_ptr() as u32;
        let _len = vec.len() as u32;
        
        Self {
            phantom: std::marker::PhantomData,
            _ptr,
            _len
        }
    }
}

This is both zero-copy on the rust side and single copy between wasm memory and JS on the JS side. Basically, the type just emulates ref to a slice through standard wasm bindgen API (which means support for returning &[T] should be relatively easy to add if somebody knows internals of bindgen) A caveat is that you shouldn't save MySlice, it should only be used to return immediate data to JS (because it detaches pointers, the code cannot guarantee lifetimes)

Usage is simple:

#[wasm_bindgen]
pub struct Container {
    data: Vec<f64>,
}

#[wasm_bindgen]
impl Container {
    pub fn read(&self) -> MySlice<f64> {
        (&self.data).into()
    }
}

with the JS side using

const c = Container.new() // somehow create
const values = c.read()

[Zireael07]

You can also return Box<[T]> with T being basic numeric type

This is about Vec<T> where T is a struct of some sort, so how is that relevant?

[Elias-Graf]

Kinrany Yes, but it requires a bit of a hack:

#[wasm_bindgen]
extern "C" {
    #[wasm_bindgen(typescript_type = "Array<string>")]
    type MyArray;
}

(The typescript_type attribute can specify any TypeScript type, even complex types like |)

Now you just do fn foo() -> MyArray and use .unchecked_into::<MyArray>() to cast the Array into MyArray.

@Pauan How would one go about receiving such a MyArray type in another function? How do I parse it back to a rust slice so I can work with it?

E.g.:

pub fn processs_my_array(arr: MyArray) -> SomeNewThing {
    // How to convert back to [string] or rather some more complicated struct slice?
}

[Pauan]

@Elias-Graf After you convert the Vec into an Array (or a MyArray) it is now a JS value, it's no longer a Rust value.

So you have to process it using the JS Array methods:

pub fn processs_my_array(arr: MyArray) -> SomeNewThing {
    let arr: js_sys::Array = arr.unchecked_into();

    // If you only need an Iterator then you don't need to use collect
    let strings: Vec<String> = arr.iter().map(|x| x.as_string().unwrap()).collect();
}

As for converting JsValue into structs... that is a separate issue:

https://github.com/rustwasm/wasm-bindgen/issues/1642

https://github.com/rustwasm/wasm-bindgen/issues/2231

[zimond]

I believe latest wasm-bindgen release already allows return Vec<T> where T is any type implementing WasmDescribe + JsCast. Currently available types includes primitives, and types from js-sys. So returning Vec<JsValue> is fine now.

---

Edit: #[wasm_bindgen] do not generate the trait impls. So there's more work to be done

[PedroFonsecaDEV]

@zimond

Do you have a simple example of how to implement this?

Let's suppose that your function returns a Vector of numbers; how to implement the WasmDescribe + JsCast in order to make it work?

Thanks.