etcd-io
wasm: Add wasm support
Add WebAssembly (WASM) Support to bbolt
This PR adds comprehensive support for WebAssembly targets (js and wasip1) to bolt, enabling the database to run in browsers and WASI-compliant runtimes.
What's Changed
- Memory-mapped I/O emulation: Since WASM doesn't support real mmap, we implement a buffer-based approach that reads the database file into memory
- Transaction synchronization: Added a txIniter interface to handle platform-specific transaction initialization, ensuring data consistency in WASM environments where memory buffers need manual refresh
- Platform-specific implementations: Created bolt_wasm.go with WASM-specific implementations of mmap, munmap, madvise, and file locking operations
Testing Infrastructure
- Added WASM test targets in the Makefile:
- make test-wasm for browser-based JavaScript WASM (GOOS=js)
- make test-wasip1 for WASI Preview 1 runtime testing
- Added GitHub Actions workflows for automated WASM testing in CI
Test Adaptations
- Skip memory-intensive tests on WASM platforms due to runtime constraints
- Added appropriate test guards for WASM-specific limitations
Testing
All tests pass on WASM platforms:
- Browser WASM (GOOS=js GOARCH=wasm)
- WASI Preview 1 (GOOS=wasip1 GOARCH=wasm)
The implementation has been tested with both wazero and wasmtime runtimes.
[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: tmc Once this PR has been reviewed and has the lgtm label, please assign ahrtr for approval. For more information see the Code Review Process.
The full list of commands accepted by this bot can be found here.
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
Hi @tmc. Thanks for your PR.
I'm waiting for a etcd-io member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
I suggest holding this pull request until we make #986 work and find a solution to cross-compile in other architectures, so we're able to catch issues in new ones early/automated, rather than waiting for someone to find out compile/build issues.
I suggest holding this pull request until we make #986 work and find a solution to cross-compile in other architectures, so we're able to catch issues in new ones early/automated, rather than waiting for someone to find out compile/build issues.
cc @ahrtr @fuweid
Now we have merged #986, which lints the files in different architectures. Would it be possible to run the linter against the js GOOS? I have no experience with this, so I'm unsure. @tmc, would you be able to add running the linter for this OS/ARCH?
Do not get time to dig into the details, but my immediately feeling is that this PR isn't well integrated into the existing implementations,
- can you add built job for wasm into https://github.com/etcd-io/bbolt/blob/main/.github/workflows/cross-arch-test.yaml?
- can you add test job for wasm similar to https://github.com/etcd-io/bbolt/blob/main/.github/workflows/tests_arm64.yaml
- Adding
if runtime.GOARCH == "wasm"into db.go seems not a nice way, is it possible to get all wasm related details included inbolt_wasm.go?
Pls do not get me wrong, I am supportive to add wasm support.