wasm-bindgen
wasm-bindgen copied to clipboard
Support returning `Iterator<Item=T>` via an iterable iterator
Motivation
Right now, returning Rust iterators is a pain. While collecting everything into a vec (so the JS side can get an Array<T>) can work for some scenarios, this approach is simply not possible when iterators contain infinite elements and undesirable when computing individual elements is costly. #1478 has more context. I made this a separate issue, because I want to talk about a specific solution.
Determined users might attempt to make their own iterator wrapper, but this comes with some pitfalls. While very similar, the iterator protocols for Rust and JS have subtle differences. One important one is what happens after iteration is done. Rust says that calling next may return pretty much anything (=unspecified but not UB; it may even panic), while JS requires that next returns { done: true }.
In my specific case, I had an iterator with between 1 and 248 elements, where each element can take a few milliseconds to compute, with no way to know how many elements there will be before having computed the first element. The idea was that the JS side can decide how many elements to take based on a time budget and user input while showing a little animation in the browser. I made my own wrapper and fell right into that pitfall.
Proposed Solution
Add a struct IterableIterator<T> type to js_sys and implement From<Iterator<Item=T>> for it. The idea is to allow users to write code like this:
#[wasm_bindgen]
fn squares() -> js_sys::IterableIterator<u64> {
(0..).map(|i| i * i).into()
}
or even better:
#[wasm_bindgen]
fn squares() -> impl Iterator<Item=u64> {
(0..).map(|i| i * i)
}
The generated JS code would then look something like this:
/**
* @returns {IterableIterator<bigint>}
*/
export function squares() {
// this would be a pointer to a an object, but I'll treat it as an object here
// todo: wrap, finalizer, and so on
let iterable_iterator_u64 = wasm.squares();
return {
[Symbol.iterator]() {
// this makes it an iterable iterator
// think of it as making it `impl<T> IntoIterator<Item=T> for Iterator<Item=T>`
return this;
},
next() {
const next = iterable_iterator_u64?.next();
if (next === undefined) {
// free the Rust object as soon as iteration is done
// this makes sure that we never call next on it again
iterable_iterator_u64?.free();
iterable_iterator_u64 = undefined;
return { value: undefined, done: true };
}
return { value: next };
}
}
}
The basic idea is to return an object that implements the Rust iterator protocol (next(&mut self) -> Option<Item>) can convert it to the JS iterator protocol (next(): IteratorResult).
Alternatively, much of the JS next method could also be done in Rust depending on what would be easier/more efficient.
I would also like to suggest wrapping the returned JS object with Iterator.from if available. Since this API is brand new (currently stage 3), this would only be done when the JS environment supports it.
Alternatives
- Don't support it. This is the current situation and it forces users to either explore alternative solutions or implement iterator support themselves. Alternative solutions might not always be viable and implementing it yourself is not trivial and a potential source of bugs.
- Using a non-generic
js_sys::IterableIterator. This has the huge downside of losing type information for the generated TS types. I would like iterators to be strongly typed on both sides. - Returning a pure iterator (no
Symbol.iterator) or pure iterable (nonext) on the JS side. Both are viable since most built-in JS APIs support both, but the same is not true for libraries. Most libraries take iterables, most often explicitly typed asIterable<T>, because standard container types likeArrayandSetare iterables but not iterators. So returning a pure iterator is not as useful in the wider JS ecosystem. A pure iterable would be viable, but anIterableIterator<T>communicates that it can only be iterated once. This is the current expectation, becauseIterableIterator<T>is most notably returned by generator functions.
Additional Context
A similar solution could also be used to support the upcoming Rust 2024 gen blocks/fns. This will create a nice symmetry between Rust generators and JS generators.
I would like to discuss this solution and then make a PR for it. However, I'll likely need some guidance on it