JS bindings don't allow passing arrays by reference

[unti1x]

Problem 1: We have a very big array we need to pass to wasm function

Solution: We cannot rely on the bindings because arrays are always copied. So we may allocate them manually like this:

  private getUint8Array(size: number): [number, Uint8Array] {
    const length = size * Uint8Array.BYTES_PER_ELEMENT;
    const ptr = this.wasm!.__pin(
      this.wasm!.__new(size, this.renderer!.UINT8ARRAY_ID),
    ) >>> 0;
    const header = this.wasm!.__new(12, this.wasm!.UINT8ARRAY_ID) >>> 0;
    const memoryU32 = new Uint32Array(this.wasm!.memory.buffer);
    memoryU32[header >>> 2] = ptr;
    memoryU32[header + 4 >>> 2] = ptr;
    memoryU32[header + 8 >>> 2] = length;

    const array = new Uint8Array(this.wasm!.memory.buffer, ptr, length);
    this.wasm!.__unpin(ptr);

    return [header, array];
  }

And then just set the data:

const [myArrayPtr, myArray] = this.getUint8Array(3);
myArrya.set([1, 2, 3]);
const result = this.wasm!.myFunction(myArrayPtr);

Technically this is what the bindings do.

Problem 2: There is no way to pass array by reference. On the one hand if you define the parameter as Uint8Array, the bindings will always expect it to be the one and copy the values which is the opposite of efficient. On the other hand when it's defined as usize or i32 or whatever the bindings will expect it to be the pointer but now you cannot convert or cast it to Uint8Array and the only way to work with it is to load/store values one by one which is not acceptable as well.

[dcodeIO]

Open to ideas on how to also enable the case where a view is desired. Right now, the bindings do what's always safe, i.e. copy, plus I didn't see a way to provide both options in a clean way.

[unti1x]

The idea is the following. First of all, a constructor for arrays, should be exported from the bindings. Something like __lowerTypedArray but returning rather both the array, and the pointer or a proxy object that'd contain them. The second part is to make something like internalref<T> for AssemblyScript that would help selecting the strategy.

Alternatively it may be allowed to pass the array pointers directly to wasm function. The constructors are still would be helpful.

The third option is to make unsafe/unchecked version of load<T> for objects

[MaxGraey]

Another solution is using ArrayBuffer in special way. So on AS side you could export ArrayBuffer and during js binding it will be pass by ref, not by value (by copy). Another variant is add special ArrayBufferView object which only represent range of buffer in memory (only pass beginning and ending of buffer in wasm memory). Something similar to string_view in C++

[dcodeIO]

ArrayBuffers are currently copied as well (there's __liftBuffer and __lowerBuffer for these). An ArrayBufferView would lose the type (and is merely an interface anyway), so one could technically just pass a changetype'd usize I guess.

For reference, the fully unsafe route that should work today looks like:

// AS
function getArrayPtr(): usize {
  return changetype<usize>(someInt32Array);
}
function setArrayPtr(ptr: usize): void {
  someInt32Array = changetype<Int32Array>(ptr);
}

// JS
const ptr = exports.__pin(exports.getArrayPtr());
...make a view from exports.memory and modify...
exports.__setArrayPtr(ptr); // if desired
exports.__unpin(ptr);

[trusktr]

And based on that, it seems easy enough to make automatic bindings for it. Someone just needs the time to do it. :D

[amw]

I've just made a pretty big jump from 0.18 to 0.27, switched to esm bindings, dropped the loader, and realized I have no way of accessing the returned AS StaticArrays by reference like I did with __getArrayView before.

I was able to copy the old __getArrayView from loader into my own code and use @dcodeIO's changetype trick to convert my array-based AS functions to pointer-based equivalents.