solana icon indicating copy to clipboard operation
solana copied to clipboard

Make `bench-tps` tool more flexible

Open KirillLykov opened this issue 1 year ago • 3 comments

Problem

Atm bench-tps is suitable only for running one type of scenario:

  • create N accounts
  • send system_transaction::transfer transactions between these accounts

We want to make tool more flexible by:

  • support durable nonce (ongoing work https://github.com/solana-labs/solana/issues/25759)
  • support more transaction types
  • control distributions and rates of transaction submission and type

Proposed Solution

  1. Come up with the list of features we want to be added: transactions types, distributions
  2. Agree on cli parameters to support this. Shall we add a config file?
  3. Perform necessary refactoring of the tool to make the workflow clearer, to have better cli (clap update)
  4. Add features one by one with some constant rate

The current workflow of the tools is the following:

  1. Generate accounts
  2. In a loop 2.1 Generate batches of transactions 2.2 Send batches of transactions

KirillLykov avatar Jul 25 '22 14:07 KirillLykov

Cli for new options

My understanding is that the desired options are to specify probability of each transaction type. Might be expressed the following way (not sure if optimal exactly this way using clap):

bench-tps <existing_options> --transaction-distribution="{transfer: 0.1, something_else: 0.2, something_else2: 0.7}"

KirillLykov avatar Jul 25 '22 15:07 KirillLykov

If it helps at all I have a project that does randomized transactions.

https://github.com/bji/solana-hammer

bji avatar Jul 25 '22 15:07 bji

After discussion with Ella:

  • we need to extend bench-tps to get more representative metrics when running benchmarks
  • the proposed way is to extend bench-tps to run different simple but typical programs mimicking real clients programs (the list of them will be prepared)

KirillLykov avatar Sep 22 '22 09:09 KirillLykov