napi_create_arraybuffer problem

[kmatzen]

I'm attempting to call napi_create_arraybuffer from https://github.com/nodejs/node-addon-api/blob/294a43f8c6a4c79b3295a8f1b83d4782d44cfe74/napi-inl.h#L1998.

However, writing to the memory pointed to by the output data void pointer does not correctly update the state of the arraybuffer in javascript. I was able to successfully use napi_create_external_arraybuffer from emnapi by passing in the desired buffer during ArrayBuffer construction rather than mutating the allocated buffer after construction.

Any ideas what might be going wrong? I didn't really understand the logic in emnapiExternalMemory.getArrayBufferPointer. It appears to be correct if you want to read the buffer, but since it makes a copy, I don't see how it could properly provide write access to the caller.

According to the napi docs https://nodejs.org/api/n-api.html#napi_create_arraybuffer:

The underlying buffer is optionally returned back to the caller in case the caller wants to directly
manipulate the buffer. This buffer can only be written to directly from native code. 

Thanks!

[toyobayashi]

Actually this is a limitation (https://emnapi-docs.vercel.app/reference/list.html#arraybuffer-related), unlike in native, wasm code can not directly access V8 API so there is no way to "link" the memory of ArrayBuffer (managed by V8) to a char* buffer (wasm memory). you need to use emnapi_sync_memory from C/C++ side or EmscriptenModule.emnapiSyncMemory from JavaScript side to sync ArrayBuffer memory manually

[kmatzen]

Is there any way possible to malloc from the wasm heap the necessary memory and take a view as the ArrayBuffer?

[toyobayashi]

https://github.com/toyobayashi/emnapi/blob/416fa06e992ed1f17ae0bbd760914ddcae3f9397/packages/emnapi/include/node/emnapi.h#L61-L67

[kmatzen]

Fair enough. I was hoping there was a way to simulate the ArrayBuffer since it would be easier to integrate emnapi into another external project without modifying their code to do the memory synchronization. I'll work around emnapi's limitation instead. Thanks!

[toyobayashi]

I have also been wondering if there is a way to simulate an external ArrayBuffer, but the key problem is that the JS engine's memory and wasm memory are isolated from each other. What I can think of is to use WeakMap to associate the simulated ArrayBuffer and wasm memory, but when this simulated ArrayBuffer is passed to other JavaScript APIs, it will not work as expected, for example, new Uint8Array(simulatedArrayBuffer) will return the wrong Uint8Array.