scala-wasm icon indicating copy to clipboard operation
scala-wasm copied to clipboard

[DO NOT MERGE] Initial memory allocation and WASI support

Open tanishiking opened this issue 2 months ago • 0 comments

This commit adds experimental WASI preview1 support (at least fd_write for now) via memory instructions.

Memory Allocation overview

To allocate memory in Wasm's linear memory, use MemoryAllocator.allocate(bytes). This method returns a MemorySegment representing the contiguous memory space. It has methods to get/set values in that memory segment (e.g., getByte, setByte), which will be compiled into memory operations (e.g., i32.load8, i32.store8).

Using withMemoryAllocator, you can instantiate a MemoryAllocator class, and you can allocate memory segments inside the block. When we exit from the block, all allocated memory segments will be freed.

Memory Allocation implement

The current MemoryAllocator implementation is very basic. It manages a single "current" memory address and allocates a memory segment of N bytes from the current address when allocate(N) is called, shifting the current memory address by N bytes. (Alignment are currently not implemented.) When freeing memory, the implementation resets the current memory address to 0.

This implementation has some issues. For example, nesting withMemoryAllocator blocks can cause problems, as exiting an inner block will free all allocated memory. To prevent this, we could either prohibit nesting withMemoryAllocator or only free memory when exiting all withMemoryAllocator blocks.

Alternatively, each allocator could record the memory segments it allocated and only free those segments when free is called. While this would be aligned with normal memory allocator, it might be too much for our use, considering all memory allocation/deallocation happens only around the withMemoryAllocator scope.

Another concern is multi-threading. However, current Wasm is single-threaded. While Wasm thread proposal introduces shared memory and atomic load/store instructions, the linear memory for communicating with WASI should be okay to be thread-local.

WASI Support and Wasm Intrinsic Functions

Currently only fd_write is supported from WASI preview1.

The fdWrite method in the wasi object is translated to a fd_write call from the wasi_snapshot_preview1 during Wasm compilation. This intrinsic function translation is implemented by matching with the hardcoded class and method names for now. Hardcoding names for all WASI functions could work for supporting only wasi_preview1. However, introducing annotations to specify which imported Wasm functions to call (such as @WasmFunction("wasi_snapshot_preview1" "fd_write") ? this annotation function body will be swapped into fd_write call) would provide better usability, especially for supporting the Wasm component model / WASI preview2.

Similarly, functions like MemoryAllocator.allocate and MemorySegment.set are also translated into Wasm intrinsic functions during linking, where they are replaced with the corresponding instruction sequences defined in Wasm.

During this process, a temporary data structure called SWasmTree is used. This serves as a design prototype for potentially introducing these data structures as SJSIR Node trees in the future, making it easier to work with them.

tanishiking avatar May 20 '24 11:05 tanishiking