Slicing data and assign to specific thread/task

[jrabello]

Given the following benchmark file:

base: 'myUrl'
threads: 8

plan:
  - name: CRIANDO UM CLIENTE {{ thread }} {{ item.txn }}
    request:
      url: /api/v1/customer/create
      method: POST
      body: '{{ item.txn }}'
      headers:
        Content-Type: 'application/json'
    with_items_from_csv:
      file_name: ../fixtures/cadastro-data.csv
      quote_char: "\'"

drill is calling the endpoint with the same row for all threads, is there any way to avoid it? is it possible to slice the data and assign some range of rows to each thread?

[fcsonline]

Since the early days of drill, iterations have always been equals between them. The problem here is that there is an initial step where the Benchmark structure (the list of all actions to do) is being loaded. In this stage, you don't have available the configuration (where iterations, threads, etc are stored), so you will not be able to do the slice property to do the distribution of rows.

Apart from that, since the migration to async the concept of threads is being diluted and added more emphasis on the number of iterations and concurrency. In this pull request, you can check the last polishments.

[jrabello]

That's awesome,

I've downloaded the current drill version which is in master branch and I've done a couple of tests with it, it looks awesome and It's really damn fast

However I'm really trying to understand what is the optimal number of concurrency to make one test plan, given that at some point creating a lot of async tasks should fill the scheduler queue so I'm not sure how to decide the optimal number of concurrency and iterations