cassandra-data-migrator icon indicating copy to clipboard operation
cassandra-data-migrator copied to clipboard

Fixes issue-257: Sets write consistency on Batch statement

Open arvy opened this issue 10 months ago • 4 comments

Sets consistency level on batch statements:

Which issue(s) this PR fixes: Fixes #257

I have tested this manually, using query tracing - I have validated that this is indeed an issue and this PR does fix it. Any ideas for an automated test welcome :)

Steps for manual acceptance test:

  1. purge any existing trace data (cqlsh> TRUNCATE system_traces.sessions; TRUNCATE system_traces.events;)
  2. enable query tracing on TARGET cluster (nodetool settraceprobability 1.0)
  3. migrate a table
  4. take note of CL on the batch query in the traced queries (cqlsh> select * from system_traces.sessions)
 61d387d0-f6f4-11ee-8c57-319793ec9a4a | 10.166.64.186 |   QUERY | 10.166.68.115 |     1161 |                                                                       {'consistency_level': 'LOCAL_QUORUM', 'serial_consistency_level': 'SERIAL'} | Execute batch of CQL3 queries | 2024-04-10 04:39:56.877000+0000

arvy avatar Apr 10 '24 05:04 arvy

@arvy could you update the manual testing steps on this PR description for future reference? Thanks for the PR!

msmygit avatar Apr 10 '24 10:04 msmygit

Hi, Thanks for looking into this, however I think it is not needed as the individual statements within the batch are already set with the right consistency-level. See TargetUpdateStatement & TargetInsertStatement for details.

Did you observe an issue with the write-consistency?

Sorry, I did not see your earlier comments, & looks like batches are ignoring consistency on individual statements. I would recommend we remove the getWriteConsistencyLevel() call if you are not using it. Thanks!

pravinbhat avatar Apr 10 '24 12:04 pravinbhat

More context on my urgency: Sony has described many issues with inconsistent discrepancies when running validation job, which were most likely due to this bug (Datastax support case 00091967)

arvy avatar Apr 10 '24 20:04 arvy

More context on my urgency: Sony has described many issues with inconsistent discrepancies when running validation job, which were most likely due to this bug (Datastax support case 00091967)

I would be suspicious if you are seeing many issues as we have not found evidence of such issues for other customers & we have done many migrations as well as validations for very large clusters. If the target env is Astra, then write consistency issue is not possible as Astra will auto use local-quorum for any write operations (as lower consistencies are not applicable to Astra). Can you elaborate the env details (origin, target, etc.) where you are seeing consistency issues.

pravinbhat avatar Apr 10 '24 21:04 pravinbhat