proxy-wasm
Shrink wasm stack size
Benchmarking has determined that the default wasm stack size is 1 MiB. This is quite a lot for small server environments, thus in the C++ SDK we set both stack size and initial heap to 64 KiB: https://github.com/proxy-wasm/proxy-wasm-cpp-sdk/pull/174
In Rust the stack size can be set as follows:
- Bazel rustc_flags:
-Clink-args=-zstack-size=65536 - Cargo CLI:
RUSTFLAGS="-Clink-args=-zstack-size=65536" cargo build ... - build.rs:
println!("cargo:rustc-cdylib-link-arg=-zstack-size=65536"); - config.toml (not tested):
rustflags = ["-C", "link-args=-zstack-size=65536"]
Is there a good way to get the Rust SDK to influence / set this value? It's not ergonomic for every user of proxy-wasm-rust-sdk to define their own build.rs file or config.toml file or use command line build parameters.
There is one way I know to accomplish this, but it comes with caveats:
- Add this line to the SDK's build.rs:
println!("cargo:rustc-cdylib-link-arg=-zstack-size=65536"); - This works but is not officially supported: https://github.com/rust-lang/cargo/issues/9562
- See also discussion on this issue: https://github.com/rust-lang/cargo/issues/9554
- It also generates a build warning:
warning: [email protected]: cargo:rustc-cdylib-link-arg was specified in the build script of proxy-wasm v0.2.3-dev (/usr/local/google/home/mstevenson/git/proxy-wasm-rust-sdk), but that package does not contain a cdylib target
CC @mpwarres @PiotrSikora
@PiotrSikora WDYT about adding the build.rs line which generates a warning? Any alternate ideas?
In Rust the stack size can be set as follows:
- Bazel rustc_flags:
-Clink-args=-zstack-size=65536
Obviously, this only works for Bazel, so it's not ideal.
- Cargo CLI:
RUSTFLAGS="-Clink-args=-zstack-size=65536" cargo build ...
This requires users to set RUSTFLAGS during compilation, so there is nothing in the SDK to do.
- build.rs:
println!("cargo:rustc-cdylib-link-arg=-zstack-size=65536");
This one prints a nasty error that it might turn into an error, so I'd only use it as the last resort, if really needed to.
- config.toml (not tested):
rustflags = ["-C", "link-args=-zstack-size=65536"]
This one (surprisingly) works, so we could use it.
Benchmarking has determined that the default wasm stack size is 1 MiB. This is quite a lot for small server environments, thus in the C++ SDK we set both stack size and initial heap to 64 KiB: proxy-wasm/proxy-wasm-cpp-sdk#174
Yes, I've mentioned it's 1 MiB in that thread :)
But 64 KiB is a bit lower than I think we should use as the sane default. I'm sure it works fine, so plugin authors could set the stack size to this or even lower value, but 64 KiB is quite a common buffer size in proxies (e.g. commonly used as a max size of HTTP header block and/or body buffers), and this would leave no space on the stack and result in memory violations (RuntimeError: memory access out of bounds in Rust when using a fixed array variable that's larger than the stack size) in plugins that want to perform any kind of buffering in 64 KiB buffers.
Notably, allowing using pre-allocated buffers (possibly on the stack) is what @leonm1 requested in proxy-wasm/spec#83.
- config.toml (not tested):
rustflags = ["-C", "link-args=-zstack-size=65536"]This one (surprisingly) works, so we could use it.
Actually, this doesn't work... It worked in my tests, since I was running them from within examples directory, and cargo looks for .cargo/config.toml in parent directories.