tsung
tsung copied to clipboard
Lazily read lines in ts_file_server
This change allows tsung to use a very large file as a variable source; each read will seek to an arbitrary location in the file and perform a read.
It's possible (likely, even) that this change will reduce performance of the variable sourcing feature; I can imagine using a config flag to enable it, perhaps. FWIW I haven't observed a serious degradation of performance in my usage.
Usage remains identical.
I kind of like the idea.
While working https://github.com/processone/tsung/pull/237 to make file_server
scale linearly with the number of nodes, I quickly noticed that larger file are quite a bit of a problem.
While this might have a slight impact on access, I don't think this is a big problem. The current eager loading of largish files poses a bigger problem. A 100MB file currently causes memory spikes up to 900-1000MB.
What do you think about that, @nniclausse?