How can I monitor the usage of cpu and memory?

[hzh0425]

How can I monitor zstd's memory usage and cpu usage? Are there any tools I can use?

Thank you very much if you can solve my confusion~

[embg]

Would top or htop work for your use-case?

[terrelln]

For CPU, you'll need to use an external tool, but unless you explicitly enable multi-threading, each compression is single-threaded.

For memory, we provide sizeof functions for each of our objects, which you can use to determine how much memory they've allocated:

https://github.com/facebook/zstd/blob/ef60302af962840bced324034b73d13c481946d1/lib/zstd.h#L1100-L1105

You can limit memory usage by setting ZSTD_c_windowLog and ZSTD_d_windowLogMax, which tell compression to use a particular window size, and decompression to set a maximum allowed window size (default 128 MB).

[cgbur]

@terrelln I have seen higher than 100% cpu usage without specifying multiple threads. --single-threaded fixes this for compression, but is ignored during decompression. I am not sure how the multi threaded writing is implemented yet or what is causing this.

[embg]

I have seen higher than 100% cpu usage without specifying multiple threads.

@cgbur zstd CLI with default settings uses one thread for (de)compression and another thread (or threads?) for I/O. Only in --single-threaded mode is I/O and (de)compression all in a single thread.

[hzh0425]

@terrelln Thank you very much !!!