gemini icon indicating copy to clipboard operation
gemini copied to clipboard

Support update statements

Open yarongilor opened this issue 6 months ago • 6 comments

Packages

gemini-gocql-driver v1.15.2 2025-07-31T14:12:57Z gemini 2.1.3 2025-08-15T14:40:21Z

Scylla version: 2025.1.6-20250811.38f4c3325d1f with build-id 5307c851fd0cfcfa660647ef0c3c2e28fdbc6e6f

Kernel Version: 6.14.0-1010-aws

Issue description

Ran the gemini command of:

gemini_cmd: |
  --duration 24h
  --warmup 10m
  --concurrency 200
  --mode mixed
  --max-mutation-retries-backoff 10s
  --max-mutation-retries 30
  --token-range-slices 10000
  --statement-ratios '{"mutation":{"insert":0.5,"update":0.2,"delete":0.3}}'

Got zero CQL updates instead of 20%. As for the CQL Inserts / Deletes ratio: The requested ratio of inserts to deletes is 5/3, which is approximately 1.67. The actual ratio from test is 2000/770, which is approximately 2.60. The Grafana metrics results are:

Image
  • [ ] This issue is a regression.
  • [ ] It is unknown if this issue is a regression.

Describe your issue in detail and steps it took to produce it.

Impact

Describe the impact this issue causes to the user.

How frequently does it reproduce?

Describe the frequency with how this issue can be reproduced.

Installation details

Cluster size: 6 nodes (i4i.xlarge)

Scylla Nodes used in this run:

  • gemini-tombstones-longrun-gemini-t-oracle-db-node-816ef7cb-1 (98.81.112.93 | 10.12.0.85) (shards: 30)
  • gemini-tombstones-longrun-gemini-t-db-node-816ef7cb-7 (3.219.237.18 | 10.12.3.178) (shards: 4)
  • gemini-tombstones-longrun-gemini-t-db-node-816ef7cb-6 (13.220.173.50 | 10.12.2.36) (shards: -1)
  • gemini-tombstones-longrun-gemini-t-db-node-816ef7cb-5 (3.237.104.190 | 10.12.0.116) (shards: 4)
  • gemini-tombstones-longrun-gemini-t-db-node-816ef7cb-4 (3.224.114.201 | 10.12.1.148) (shards: 4)
  • gemini-tombstones-longrun-gemini-t-db-node-816ef7cb-3 (34.231.169.249 | 10.12.3.120) (shards: 4)
  • gemini-tombstones-longrun-gemini-t-db-node-816ef7cb-2 (13.223.94.76 | 10.12.2.82) (shards: 4)
  • gemini-tombstones-longrun-gemini-t-db-node-816ef7cb-1 (44.223.7.17 | 10.12.0.103) (shards: 4)

OS / Image: ami-0d5299f8510887b47 (aws: N/A)

Test: gemini-tombstones Test id: 816ef7cb-4ec7-4afd-9004-2a8692498a0e Test name: scylla-staging/yarongilor/gemini-tombstones Test method: gemini_test.GeminiTest.test_load_random_with_nemesis Test config file(s):

Logs and commands
  • Restore Monitor Stack command: $ hydra investigate show-monitor 816ef7cb-4ec7-4afd-9004-2a8692498a0e
  • Restore monitor on AWS instance using Jenkins job
  • Show all stored logs command: $ hydra investigate show-logs 816ef7cb-4ec7-4afd-9004-2a8692498a0e

Logs:

Jenkins job URL Argus

yarongilor avatar Aug 24 '25 10:08 yarongilor

This is not a bug as update statements are disabled as per this line.

https://github.com/scylladb/gemini/blob/master/pkg/generators/statements/mutations.go#L53

I'll enable the updates once the core feature are stable and everything is backported to SCT.

So having update to 0.2 is just inserts with 0.7 in this case

CodeLieutenant avatar Aug 24 '25 11:08 CodeLieutenant

@CodeLieutenant , could this be fixed in 2.x?

yarongilor avatar Jan 12 '26 12:01 yarongilor

@CodeLieutenant , could this be fixed in 2.x?

Probably yes. But for now we don't plan introducing new features until we stabilize Gemini.

soyacz avatar Jan 12 '26 12:01 soyacz

In 2.2.3, a tested default load rate shows unclear results. It doesn't have continues updates (like reads/writes) rather some several bursts: https://argus.scylladb.com/tests/scylla-cluster-tests/15866a39-06d5-40a9-8170-e942115fd362

Image

yarongilor avatar Jan 15 '26 15:01 yarongilor

these bursts probably are not from Gemini but from disrupt_no_corrupt_repair nemesis

soyacz avatar Jan 15 '26 15:01 soyacz

In gemini update statetements are disabled due to issues when validating, before the rewrite, the code is just comented out, after we have validated gemini is behaving ok, we can reenabled them

CodeLieutenant avatar Jan 15 '26 15:01 CodeLieutenant

Closing this issue as it was moved to Jira. Please continue the thread in https://scylladb.atlassian.net/browse/QATOOLS-107

dani-tweig avatar Jan 18 '26 05:01 dani-tweig