tsung icon indicating copy to clipboard operation
tsung copied to clipboard

Lazily read lines in ts_file_server

Open b20n opened this issue 11 years ago • 1 comments

This change allows tsung to use a very large file as a variable source; each read will seek to an arbitrary location in the file and perform a read.

It's possible (likely, even) that this change will reduce performance of the variable sourcing feature; I can imagine using a config flag to enable it, perhaps. FWIW I haven't observed a serious degradation of performance in my usage.

Usage remains identical.

b20n avatar Aug 06 '13 16:08 b20n

I kind of like the idea.

While working https://github.com/processone/tsung/pull/237 to make file_server scale linearly with the number of nodes, I quickly noticed that larger file are quite a bit of a problem.

While this might have a slight impact on access, I don't think this is a big problem. The current eager loading of largish files poses a bigger problem. A 100MB file currently causes memory spikes up to 900-1000MB.

What do you think about that, @nniclausse?

tisba avatar Nov 02 '17 09:11 tisba