tsung
tsung copied to clipboard
file_server currently poses a bottleneck
trafficstars
As discussed with @nniclausse on IRC, the current file_server implementation (ts_file_server.erl) poses a global bottleneck which limits the maximum throughput of of calls to the file server (e.g. to get a random line).
A possible solution would be to distribute the files to all generator servers/nodes before starting the test. This way we could provide a local version, local_file_server. For obvious reason, we could only offer a random access on that file.
I currently suggest the following:
- introduce
local_file_serveras an alternative tofile_server:
<options>
<option name="local_file_server" id="users" value="/tsung/files/users.csv" />
</options>
- access to the local file server in a session could look like this:
<setdynvars sourcetype="local_file" fileid="users" delimiter=";">
<var name="username"/>
<var name="password"/>
</setdynvars>
- on tsung startup the controller needs to distribute the files to every server, so that all generator nodes (beam processes) have access to the same file.
This approach would essentially eliminate the contention on the controller when it comes to accessing files.
See #237