zed
zed copied to clipboard
zed load: +36% mem use, -15% load time (commit 7dfef14)
Through a set of iterative runs as described at https://github.com/brimdata/zed-perf-testing/tree/main/memory-load, I've observed that at Zed commit commit 7dfef14 (associated with #3482), the zed load of the fdns-a.zeek.gz data set saw an increase of 36% in reported go_memstats_sys_bytes in exchange for a 15% reduction in the time for the load to complete.
Here's a macro-level plot of the trend across many commits:

In a team discussion, @nwt was unsurprised that the perf improvement came with some memory cost. This issue has been opened to log the observation and consider if the trade is acceptable or if there's ways to potentially maintain the better perf while using less memory.