We need to support to run writes with validate data in parallel from few loaders without truncation.

[juliayakovlev]

trafficstars

We need to support to run writes with validate data in parallel from few loaders.

Now if write workflow starts with --validate-data, it first truncates the tables in the PrepareDatabase function. So if a few parallel runs (with partition offset) are started, every new thread start with truncation of already inserted data.

[mmatczuk]

This project has much in common with Gemini, maybe we can port it there @dahankzter @penberg ?

[dahankzter]

Yes and no. I think that Gemini is just too focused on general randomization. Of course it can be repurposed or have other modes implemented. If the performance aspects of scylla-bench is not needed then it's better to put it in Gemini I but if the more controlled structure of scylla-bench is needed then the other way is maybe better.

scylla-bench on the other hand is a performance testing tool whose purpose is to measure performance such as latency and throughput. Validation really doesn't fit this as well because it's a whole separate deep rabbit hole. I am not sure scylla-bench is the correct place for this unless the validation stays small and focused but it depends on what the goal of scylla-bench is.

@penberg what do you think?

[penberg]

Like @dahankzter, I also think Gemini is probably a better place for stress testing, and scylla-bench ought to focus on load generation for performance testing.