scylla-bench icon indicating copy to clipboard operation
scylla-bench copied to clipboard

Add a new workload or parameter for range-deletions

Open yarongilor opened this issue 3 years ago • 5 comments

The current available workload and parameters of scylla-bench doesn't support specifying operation of delete-range.

This is very important for testing of range-Thobstones revised mechanizm in a real-life scenario, using SCT infra-structure.

yarongilor avatar Apr 11 '22 14:04 yarongilor

delete-range is much more complicated the rest of the use case s-b is doing, and needs to be in complete sync of what the test is doing.

for any implementation on s-b end, we'll need much more concrete requirements of what is the logic expected for this "real-life" scenario

fruch avatar Nov 19 '24 17:11 fruch

today we can recommend using latte to implement a very specific user defined case like deletions

fruch avatar Nov 19 '24 17:11 fruch

today we can recommend using latte to implement a very specific user defined case like deletions

@fruch , is there any documentation reference for writing any such use case with latte?

What's the repo for latte support? sct? should this issue moved to other repo?

Another example for this need is in test of https://github.com/scylladb/scylla-cluster-tests/pull/8948 - it currently write s-b large partitions, then delete it "manually" one by one, what takes an hour.

yarongilor avatar Nov 20 '24 12:11 yarongilor

today we can recommend using latte to implement a very specific user defined case like deletions

@fruch , is there any documentation reference for writing any such use case with latte?

What's the repo for latte support? sct? should this issue moved to other repo?

this is the repo for latte: https://github.com/scylladb/latte

regardless it's not a issue for latte, it's for you to write the script you need you can find several examples in: https://github.com/scylladb/scylla-qa-internal/tree/master/custom_d1

Another example for this need is in test of scylladb/scylla-cluster-tests#8948 - it currently write s-b large partitions, then delete it "manually" one by one, what takes an hour.

again I don't know what is the exact requirement you have there, but you probably can script it in rune

fruch avatar Nov 20 '24 13:11 fruch

today we can recommend using latte to implement a very specific user defined case like deletions

@fruch , is there any documentation reference for writing any such use case with latte? What's the repo for latte support? sct? should this issue moved to other repo?

this is the repo for latte: https://github.com/scylladb/latte

regardless it's not a issue for latte, it's for you to write the script you need you can find several examples in: https://github.com/scylladb/scylla-qa-internal/tree/master/custom_d1

Another example for this need is in test of scylladb/scylla-cluster-tests#8948 - it currently write s-b large partitions, then delete it "manually" one by one, what takes an hour.

again I don't know what is the exact requirement you have there, but you probably can script it in rune

Example of the latte stress commands currently used in SCT:

  • https://github.com/scylladb/scylla-cluster-tests/blob/master/test-cases/longevity/longevity-gce-custom-d1-workload2-hybrid-raid.yaml

It utilizes the following rune script:

  • https://github.com/scylladb/scylla-qa-internal/blob/master/custom_d1/workload2/latte/custom_d1_workload2.rn

Then, I looked at the PR which requires this feature here: https://github.com/scylladb/scylla-cluster-tests/pull/8948 The use case can easily be covered with latte. Existing custom-d1/workload2 test case we have in SCT is more complex than it is needed in your case. So, it should be fairly easy to do your rune script just by using the existing mentioned one as an example.

@fruch

delete-range is much more complicated the rest of the use case s-b is doing, and needs to be in complete sync of what the test is doing.

Even being agree that latte is more powerful, I disagree on the statement that delete-range is much more complicated for S-B. Delete operations don't fail if data, it references, are absent. So, it means we can simply add delete and delete_batch modes similar to write and write-batch ones we already have in scylla-bench. Probably we will need to add 1 additional option saying how exactly we delete data - by rows or by partitions.

vponomaryov avatar Nov 20 '24 17:11 vponomaryov