dylibso
Performance : slowness during 'packing' time with Hermit CLI.
I am opening this issue to discuss possible performance improvements that can be brought to Hermit CLI.
Measurements below are based on PR #23 : the throughput of hermit.com is not constant and degrades with medium to big inputs, the time required for packing increases with larger inputs. few data points below :
| Count Vowels | Ubuntu | Emacs | |
|---|---|---|---|
| Pack time | 795ms | 1m33s | 8m30s |
| Throughput | 2.6Mb/s | 1.7Mb/s | 1.6Mb/s |
| .wasm size | 2.1Mb | 160Mb | 853Mb |
| .com size | 1.2Mb | 63Mb | 330Mb |
*Wasm files of Ubuntu and Emacs were generated using container2wasm.
After few runs with verbose mode, a pattern emerged. The root cause of the slowness is the zipping of *.wasm in https://github.com/dylibso/hermit/blob/main/hermit-cli/crates/hermitfile-parser/src/main.rs#L181-L187.
A change of config as follow, results in a different performance results: https://github.com/mtb0x1/hermit/blob/zip_stored/hermit-cli/crates/hermitfile-parser/src/main.rs#L198-L214
| Count Vowels | Ubuntu | Emacs | |
|---|---|---|---|
| Pack time | 39ms | 0m1s30ms | 0m7s80ms |
| Throughput | 53Mb/s | 123Mb/s | 109Mb/s |
| .wasm size | 2.1Mb | 160Mb | 853Mb |
| .com size | 2.7M | 161Mb | 853Mb |
A drastic change in performance that have a downside, the size of produced artifacts. but in the other hand, I don't think that users will expect any 'out of the box' binaries size reduction.
My suggestion would be to add options to Hermit cli to allow the user to choose a 'packing' strategy. the strategy would be somehow similar to what classic compilers have already :
- Optimization for size : would allow a level of compression to be specified.
- Optimization for speed (use Stored instead of Deflated)
N.B :
- Modifications introduced have an impact on both
hermit.comand artifacts produced by it. I would recommend separating this in 2 stages and avoid sharing same optimization strategy. - Modifications introduced at the 'packing' step have an impact on runtime of artifacts. (faster boot of .com CLI, still slow but this would be a subject of another issue in the future.)
A packing strategy flag would be a great edition to the cli! Longer term we're interesting in improving the performance of hermit (possibly running the wasm with a JIT) so DEFLATE hopefully won't be an issue then.
