scylla-tools-java
scylla-tools-java copied to clipboard
scylladb
establish more connections in cassandra-stress
In many occasions we are forced to run multiple clients not because there is a bottleneck in the client machine, but because the low number of connections.
Ideally, we would make the number of connections configurable, and then tune that. A change like this should find its way to the upstream cassandra-stress project.
Bonus point if we can have a wrapper script that somehow connects to the Scylla REST or other jmx metrics, finds out what is the number of shards, and opens as many connections as there are shards.
REST isn't a good idea, because you need to set up another connection with its own configurable port number, authentication, and encryption.
Better read a new column from system.local or a new virtual table. This could then be done in the driver and apply to all applications, not just cassandra-stress.
As of a0076e70eed5501ac9d8c3ff41ce8018710a1585, cassandra-stress upstream supports setting the number of connections.
Since we also want a newer version to fix coordinated omission, I strongly suggest we upgrade.
This should be resolved by bundling the shard-aware driver instead of the datastax driver.
@haaawk what's the best way to do that? IMO we should do a formal release of the shard-aware driver (i.e. with a tag, version number, and upload to the binary repo) and bundle that.
@avikivity I agree that publishing our driver as maven artifact is the best way to do it. In the mean time we can just put our driver into tools instalation.
What does "put our driver into tools installation" mean? Copy it to libs/? I don't want to use a non-released version (although, it's not really that terrible if it's a hidden component).
btw we're also facing a similar problem with the Python driver that is bundled in scylla-tools-java.git.
I would have to double check but putting the jar in libs should work. We might need to change some build scripts too but I'm not 100% sure without giving it a try.