emscripten-core
WasmFS: Async APIs
In addition to the traditional POSIX APIs, some applications would benefit from having asynchronous file system APIs as well. Perhaps there could be default implementations that just call the synchronous version of the API and immediately call the user callback, the specific backends could specialize that default as appropriate.
I need it so much! 🥇 Using the asynchronous interface, can directly call the file system in web worker and main thread without blocking and unnecessary thread(sync thread) overhead.
Is this still planned? Our codebase uses POSIX async IO (aio) where available, but the web version currently needs a fallback using fopen (with directories mapped using wasmfs_create_directory and an opfs backend).
Would be amazing to unblock native workers that request files (and later on write too, but that's not too important for us right now).
P.S. I know that musl supports aio, but I assume this is stripped out of emscripten on purpose due to not being implemented? @sbc100 I see you've been updating it recently, do you know the procedure?
I imagine it would be a fair amount of work to implement aio. In particular aio_suspend would require something like ASYNCIFY in order to work.
Yup I can see that, luckily we don't use aio_suspend ourselves.
Looking at the opfs implementation we could probably switch over to the native wasm_fs API later on instead of aio. From first glance _wasmfs_opfs_read_blob would be enough for us, blocker seems to be that we don't have access to the wasmfs worker thread / proxy context. Might give exposing some minimal read-only async API a go at some point.
This is a blocker for supporting large local sites and multiple PHP web workers in WordPress Playground.
We support synchronizing data between MEMFS and a DirectoryHandle. However, lacking a pthreads build of WebAssembly limits what we can do with the filesystem. We don't have a good way of:
- Synchronizing MEMFS between workers
- Synchronizing concurrent writes to a local FileSystemFileHandle between workers
- Stream-reading large files from a local FileSystemFileHandle without loading them into memory
With an asynchronous filesystem, we could mount a FileSystemDirectoryHandle at, say, /wordpress, and directly interact either with OPFS or with the local filesystem.
I was looking for a way to use OPFS mounts for a while, but couldn't find any resources. I also couldn't figure that out from the wasmfs_opfs.c link that often gets passed around as an example.
