scylla-cluster-tests icon indicating copy to clipboard operation
scylla-cluster-tests copied to clipboard

Please document how to use multiple loaders in the code

Open asias opened this issue 1 year ago • 4 comments

Suppose the following is defined:

n_db_nodes: '3 3' n_loaders: '2 2'

We want use the 4 loaders to insert data in a nemesis like below:

def insert_data_with_cs(self, ks='keyspace1', key_start=1, key_end=100, rf = 3):
        stress_cmd=f"cassandra-stress write no-warmup cl=QUORUM n={num} -schema 'replication(strategy=NetworkTopologyStrategy,replication_factor={rf})' -mode cql3 native -rate 'threads=100 fixed=20000/s'  -col 'size=FIXED(128) n=FIXED(8)' -pop seq={key_start}..{key_end} -log interval=5"
        cs_thread = self.tester.run_stress_thread(
            stress_cmd=stress_cmd, keyspace_name=ks, stop_test_on_failure=False, round_robin=True, stats_aggregate_cmds=False)
        cs_thread.verify_results()

How can we tell self.tester.run_stress_thread to use different loaders so we can run 4 c-s in parallel?

asias avatar Nov 19 '24 07:11 asias

@fruch @soyacz Ideas are appreciated. This is very useful for me to speed up multiple dc tests. I just want to utilize more loaders to insert data with custom c-s parameters during the nemesis. I could not use prepare_write_cmd and friends.

asias avatar Nov 20 '24 00:11 asias

If you want to run the exact same command ?

round_robin=False should do the trick

Otherwise you need to run the 4 different command with round_robin=True, and wait/verify results only after all of them started.

fruch avatar Nov 20 '24 05:11 fruch

If you want to run the exact same command ?

round_robin=False should do the trick

Otherwise you need to run the 4 different command with round_robin=True, and wait/verify results only after all of them started.

I want to run 4 different c-s cmds, each node runs 1 cmd. What does the round_robin mean here? Is run_stress_thread smart enough to pick different loaders for the c-s? How does it select the loaders.

asias avatar Nov 20 '24 05:11 asias

If you want to run the exact same command ? round_robin=False should do the trick Otherwise you need to run the 4 different command with round_robin=True, and wait/verify results only after all of them started.

I want to run 4 different c-s cmds, each node runs 1 cmd. What does the round_robin mean here? Is run_stress_thread smart enough to pick different loaders for the c-s? How does it select the loaders.

round_robin, mean that each command start start is picking the next loader machine. there's an endless cycle.

when round_robin: false each command is sent out to all of the loader at the same time.

fruch avatar Nov 20 '24 07:11 fruch