cassandra icon indicating copy to clipboard operation
cassandra copied to clipboard
verified publisher icon datastax

CNDB-13483: Fix loading PQ file when disabled_reads is true

Open michaeljmarshall opened this issue 8 months ago • 5 comments

What is the issue

Fixes: https://github.com/riptano/cndb/issues/13483

Initial Commits

  • CNDB-13483: Fix loading PQ file when disabled_reads is true
  • Refactor code to pull PQ fetching logic into single class
  • Add comment asking about reducing memory utilization when loading PQ
  • Fix the broken allRowsHaveVectorsInWrittenSegments method

What does this PR fix and why was it fixed

The cassandra.sai.disabled_reads prevents us from reading previous PQ objects during compaction, which leads to less efficient graph construction. This fixes that by adding a new searchable index that is capable of discovering/downloading/cleaning up the right index files to get that information.

I also have a tangential bug fix for code that was also impacted by the disabled_reads setting. However, there is a follow up question to ask if it is a valid implementation.

michaeljmarshall avatar Apr 25 '25 22:04 michaeljmarshall

Checklist before you submit for review

  • [ ] Make sure there is a PR in the CNDB project updating the Converged Cassandra version
  • [ ] Use NoSpamLogger for log lines that may appear frequently in the logs
  • [ ] Verify test results on Butler
  • [ ] Test coverage for new/modified code is > 80%
  • [ ] Proper code formatting
  • [ ] Proper title for each commit staring with the project-issue number, like CNDB-1234
  • [ ] Each commit has a meaningful description
  • [ ] Each commit is not very long and contains related changes
  • [ ] Renames, moves and reformatting are in distinct commits
  • [ ] All new files should contain the DataStax copyright header instead of the Apache License one

github-actions[bot] avatar Apr 25 '25 22:04 github-actions[bot]

@jkni @eolivelli - I will add tests for this PR on Monday. The initial implementation is ready for review. Manual testing suggests that it works. I wanted to push it up to run the rest of CI to get that feedback by Monday

michaeljmarshall avatar Apr 25 '25 22:04 michaeljmarshall

Quality Gate Failed Quality Gate failed

Failed conditions
77.2% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

sonarqubecloud[bot] avatar May 28 '25 23:05 sonarqubecloud[bot]

:x: Build ds-cassandra-pr-gate/PR-1713 rejected by Butler

2 new test failure(s) in 7 builds See build details here

Found 2 new test failures

Test Explanation Branch history Upstream history
...ctorCompactionTest.testDelayedIndexCreation[ca] regression :red_circle::large_blue_circle::large_blue_circle::large_blue_circle:
o.a.c.u.b.BinLogTest.testTruncationReleasesLogS... regression :red_circle::large_blue_circle::large_blue_circle::large_blue_circle::red_circle::large_blue_circle::large_blue_circle: :large_blue_circle::large_blue_circle::large_blue_circle::large_blue_circle::large_blue_circle::large_blue_circle::large_blue_circle:

Found 7 known test failures

cassci-bot avatar May 28 '25 23:05 cassci-bot

:x: Build ds-cassandra-pr-gate/PR-1713 rejected by Butler

1 new test failure(s) in 6 builds See build details here

Found 1 new test failures

Test Explanation Branch history Upstream history
...ctorCompactionTest.testDelayedIndexCreation[dc] regression :red_circle::red_circle::large_blue_circle:

Found 7 known test failures

cassci-bot avatar May 28 '25 23:05 cassci-bot