zed icon indicating copy to clipboard operation
zed copied to clipboard

zed load: +63% mem use, +47% load time (commit bc919de)

Open philrz opened this issue 3 years ago • 0 comments

Through a set of iterative runs as described at https://github.com/brimdata/zed-perf-testing/tree/main/memory-load, I've observed that starting at Zed commit commit bc919de (associated with #3432), the zed load of the fdns-a.zeek.gz data set sees an increase of 63% in reported go_memstats_sys_bytes and a corresponding 47% increase in the time for the load to complete.

Here's a macro-level plot of the trend across many commits:

image

Here's an example of the per-second memory stats during the run at commit bc919de. I've got the raw data for these stats (and CPU stats also) archived for each of these runs if anyone wants to see them.

image

philrz avatar Oct 03 '22 22:10 philrz