swiftwasm
Roadmap: Threading Support
Recently a lot of work on thread support has been done in WASI: https://github.com/WebAssembly/wasi-threads After our toolchain support threading, it will unlock offloading computationally heavy operations to threads.
Here is a list of tasks to support threads in SwiftWasm:
- [x] 1. Enable wasm-ld's
--gc-sectionsto unlock wasi-libc upgrade https://github.com/swiftwasm/swift/issues/5128 - [x] 2. https://github.com/swiftwasm/swift/issues/5547
- [x] 3. Update wasi-libc to the version, which includes pthread implementation based on wasi-threads
- [x] 4. Support
wasm32-unknown-wasi-threadstarget triple - [x] 5. Build stdlib with atomics enabled
- [x] 6. Add compiler options to enable atomics feature in LLVM code generator
- [x] 7. Setup Swift SDK distribution for the threaded SDK
- [x] 8. Fix unaligned or too large atomic load/store in HeapObject.cpp
- https://github.com/apple/llvm-project/blob/d7f1b250afdb37a89d02731acb26912c1fff5625/clang/lib/CodeGen/CGAtomic.cpp#L837
- ~~This libcall requires libclang_rt.builtin.a to include atomic.c with atomics and bulk-memory features enabled, but it makes it difficult to share the archive between non-threaded target, so we should fix Swift runtime side.~~
- https://github.com/swiftwasm/swiftwasm-build/pull/288 will fix this
- [x] 9. Add required options to toolset.json in Swift SDK
- https://github.com/apple/swift-sdk-generator/pull/104
- [ ] 10. Implement a transformation pass for
i32.atomic.wait-> spin lock to work around browser limitation on main thread- See https://github.com/WebAssembly/threads/issues/106 about details of this limitation
- [ ] 11. JS polyfill implementation of wasi-threads based on Web Worker
Feedback and any help are welcome :)
One more task IMO:
- [ ] Add
generic_wasi_base.hand update the include logic here to support WASI https://github.com/apple/swift-corelibs-libdispatch/blob/5529d351e304eb2312b7b3bfb073f7d8285473e0/os/object.h#L24-L33
Started building stdlib for wasm32-unknown-wasip1-threads target on main branch: https://github.com/apple/swift/pull/72650
I have a question. Are there still blockers for compiling Swift for the wasm32-unknown-wasip1-threads target triple?
$ which swiftc
/home/kebo/downloads/swift-wasm-DEVELOPMENT-SNAPSHOT-2024-04-03-a/usr/bin/swiftc
$ swiftc --version
Swift version 6.0-dev (LLVM 3dd3bce1adb5b65, Swift 1f09be97a890c3e)
Target: aarch64-unknown-linux-gnu
$ cat hello.swift
print("hello")
$ swiftc -target wasm32-wasip1-threads -c hello.swift
<unknown>:0: warning: libc not found for 'wasm32-unknown-wasip1-threads'; C stdlib may be unavailable
<unknown>:0: error: could not find module '_Concurrency' for target 'wasm32-unknown-wasip1-threads'; found: wasm32-unknown-wasi, at: /home/kebo/downloads/swift-wasm-DEVELOPMENT-SNAPSHOT-2024-04-03-a/usr/lib/swift/wasi/_Concurrency.swiftmodule
$ uname -a
Linux Brown-rhinoceros-beetle 6.6.3-414.asahi.fc39.aarch64+16k #1 SMP PREEMPT_DYNAMIC Sun Mar 24 19:44:17 UTC 2024 aarch64 GNU/Linux
error: could not find module '_Concurrency' for target 'wasm32-unknown-wasip1-threads' also happens when using swift build with the Swift SDK.
Tried to fix the issue around atomic.c in libclang_rt.builtin.a, but concluded that it's difficult to fix this with the current toolchain layout of clang, which requires the single builitin library for each arch.
From e3ab0445dce6d0aabdf9719742240eaafc5731a4 Mon Sep 17 00:00:00 2001
From: Yuta Saito <[email protected]>
Date: Sun, 14 Apr 2024 16:48:59 +0000
Subject: [PATCH] [compiler-rt][wasm] Compile atomic.c with atomics and
bulk-memory features
The bulitin object file for atomic.c is only referenced by user code
when the atomics and bulk-memory features are enabled and libcall is
required. However, the atomic.c itself was compiled without those
features and it leads to a linker error because all objects have to have
the feature when `--shared-memory` is enabled.
This patch compiles atomic.c with the atomics and bulk-memory features
enabled only for the object file.
---
compiler-rt/lib/builtins/CMakeLists.txt | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/compiler-rt/lib/builtins/CMakeLists.txt b/compiler-rt/lib/builtins/CMakeLists.txt
index c139917d4f58..5af6c72a9102 100644
--- a/compiler-rt/lib/builtins/CMakeLists.txt
+++ b/compiler-rt/lib/builtins/CMakeLists.txt
@@ -839,11 +839,30 @@ else ()
set(deps_aarch64 lse_builtin_symlinks)
endif()
+ set(BUILTIN_OBJECT_LIBS_${arch})
+ # For WebAssembly, we need to compile the atomic.c with different flags than the rest
+ # to enable atomics and bulk-memory features only for the object file.
+ if((arch STREQUAL "wasm32" OR arch STREQUAL "wasm64") AND "atomic.c" IN_LIST ${arch}_SOURCES)
+ set(BUILTIN_ATOMIC_CFLAGS_${arch} ${BUILTIN_CFLAGS_${arch}})
+ list(APPEND BUILTIN_ATOMIC_CFLAGS_${arch} -matomics -mbulk-memory)
+ add_compiler_rt_object_libraries(clang_rt.builtins.${arch}.atomic
+ ARCHS ${arch}
+ DEPS ${deps_${arch}}
+ SOURCES atomic.c
+ DEFS ${BUILTIN_DEFS}
+ CFLAGS ${BUILTIN_ATOMIC_CFLAGS_${arch}})
+ # Include the atomic object file in the builtins archive
+ list(APPEND BUILTIN_OBJECT_LIBS_${arch} clang_rt.builtins.${arch}.atomic)
+ # Remove atomic.c from the main list of sources
+ list(REMOVE_ITEM ${arch}_SOURCES atomic.c)
+ endif()
+
add_compiler_rt_runtime(clang_rt.builtins
STATIC
ARCHS ${arch}
DEPS ${deps_${arch}}
SOURCES ${${arch}_SOURCES}
+ OBJECT_LIBS ${BUILTIN_OBJECT_LIBS_${arch}}
DEFS ${BUILTIN_DEFS}
CFLAGS ${BUILTIN_CFLAGS_${arch}}
PARENT_TARGET builtins)
--
2.43.2
Closing as we added basic infrastructure to use multi-thread. If you build Wasm with wasm32-unknown-wasip1-threads Swift SDK, the produced binary is compatible with WASI threads ABI. See https://book.swiftwasm.org/getting-started/setup-snapshot.html for how to install the SDK.
The produced binary runs on Wasm runtimes with the WASI threads support (e.g. wasmtime, wasm-micro-runtime).
FAQ
- Can I use libdispatch?: No at least for now. Managing run loop cooperating Foundation.RunLoop, libdispatch, and Web browser's one seems not straightforward. We recommend not using libdispatch directly but preferring Swift Concurrency APIs.
- How can I use multi-threads by Web Worker?: See https://github.com/swiftwasm/JavaScriptKit/pull/256 and https://github.com/swiftwasm/JavaScriptKit/tree/main/Examples/Multithreading example.