dbase-rs icon indicating copy to clipboard operation
dbase-rs copied to clipboard

support the wasm32-wasi target

Open alicja-januszkiewicz opened this issue 3 months ago • 1 comments

Hi, I'm interested in extending the functionality of dbase-rs to make it runnable on the web. My use case is that I'm working on a hobby project that targets the web and needs to be able to read in shapefiles, which is a multifile format with a dbase component. Would you be happy to accept a PR enabling this?

Below is a more involved explanation of what this PR attempts to achieve. I don't have much knowledge in this domain so I might have made some mistakes. I've included some links for reference.

Running dbase-rs on the web can be achieved with WebAssembly but there seem to be 2 different compilation targets, neither of which target the web specifically:

  • wasm32-unknown-unknown
  • wasm32-wasi / wasm32-wasip2

The former does not make any assumptions about the platform it will run on, and since dbase works with files it does not seem like the best fit. However, one could still use web_sys::File or similar to make it work on web.

The latter target assumes the target platform will provide the WebAssembly System Interface which includes a filesystem api. If my understanding is correct, this implies that dbase and std in general should work more or less out of the box. The WASI spec is still in the works and is not yet implemented by any browser, but there is a polyfill that makes wasi components executable on the web. One can also run them in a vm using a wasm runtime like wasmtime.

In this PR I've made some adjustments to make dbase-rs build on wasm32-wasip2 as it seemed like the simpler and cleaner option.

The only block preventing it from building was std::fs::File::try_clone. I've followed a workaround I found im some other repositories [1], [2] which is essentially to wrap std::fs::File in an Arc.

The filesystem api was added in WASI preview 2, which has only recently been stabilised and added as a rustc tier 3 target. As a result rustup does not yet support it, but there is a cargo-component tool that translates binaries built for wasm32-wasip1 into wasm32-wasip2 components.

To test out whether this PR works, I've installed both cargo-component and wasmtime, and have managed to get the two dbase examples to run:

cargo component build --example file
wasmtime --mapdir .::. target/wasm32-wasi/debug/examples/file.wasm

The above will

  1. cargo build the file example with the wasm32-wasip1 target
  2. transform the wasm-wasip1 binary into a wasm-wasip2 component
  3. run it on the wasmtime vm, mapping directories between the virtual and the host filesystems.

alicja-januszkiewicz avatar May 07 '24 04:05 alicja-januszkiewicz