WIP: cql: forbid having counter columns in tablets tables

[ptrsmrn]

(WIP: need to run the tests to see which ones break and fix them + possibly add some tests) Counter updates break under tablet migration (#18180), and for this reason counters need to be disabled until the problem is fixed. It's enough to forbid creating a table with counters, as altering a table without counters already cannot result in the table having counters:

1. Adding a counter column to a table without counters:

cqlsh> ALTER TABLE temp.cf ADD (col_name counter);
ConfigurationException: Cannot add a counter column (col_name) in a non counter column family

2. Altering a column to be of the counter type:

cqlsh> ALTER TABLE temp.cf ALTER col_name TYPE counter;
ConfigurationException: Cannot change col_name from type int to type counter: types are incompatible.

Fixes: #19449 Fixes: https://github.com/scylladb/scylladb/issues/18876

Need to backport to 6.0, as this is broken there.

[scylladb-promoter]

:red_circle: CI State: FAILURE

:white_check_mark: - Build :white_check_mark: - Container Test :white_check_mark: - dtest with topology changes :white_check_mark: - dtest :x: - Unit Tests

Failed Tests (873/31014):

• test_cast_from_counter.debug.1 :mag:
• test_cast_from_counter_to_varchar.debug.1 :mag:
• test_cast_from_counter.debug.2 :mag:
• test_cast_from_counter_to_varchar.debug.2 :mag:
• test_cast_from_counter.debug.3 :mag:
• test_cast_from_counter_to_varchar.debug.3 :mag:
• test_counter_to_blob.debug.1 :mag:
• test_counter_to_blob.debug.2 :mag:
• test_counter_to_blob.debug.3 :mag:
• test_scylla_sstable_validate_ok1.debug.1 :mag:

Build Details:

• Duration: 2 hr 23 min
• Builder: spider3.cloudius-systems.com

[ptrsmrn]

I have some issue reference nitpicks.

Also, please consider also doing #18876 - you need to also mention counters as not being supported on the CREATE TABLE error message, so while at it, you can fix the entire warning message.

I saw it but thought I'd have to do it in a separate PR, but I can fix it in this PR under a different commit.

Finally, don't we have a document we need to update, to say that with tablets you can't create counters - not just LWT, Alternator (because of LWT) and CDC?

I can check it and cover every disabled feature (in this PR).

[scylladb-promoter]

:red_circle: CI State: FAILURE

:white_check_mark: - Build :x: - Unit Tests Custom The following new/updated tests ran 100 times for each mode: :small_blue_diamond: cql-pytest/test_cast_data :small_blue_diamond: cql-pytest/test_counter :small_blue_diamond: cql-pytest/test_tools :small_blue_diamond: cql-pytest/test_unset :small_blue_diamond: cql-pytest/test_wasm

Failed Tests (1858/4740):

• test_cast_from_counter_to_varchar.debug.1 :mag:
• test_cast_from_counter_to_varchar.debug.10 :mag:
• test_cast_from_counter_to_varchar.debug.2 :mag:
• test_cast_from_counter_to_varchar.debug.3 :mag:
• test_cast_from_counter_to_varchar.debug.4 :mag:
• test_cast_from_counter_to_varchar.debug.5 :mag:
• test_cast_from_counter_to_varchar.debug.6 :mag:
• test_cast_from_counter_to_varchar.debug.7 :mag:
• test_cast_from_counter_to_varchar.debug.8 :mag:
• test_cast_from_counter_to_varchar.debug.9 :mag:

Build Details:

• Duration: 43 min
• Builder: spider5.cloudius-systems.com

[scylladb-promoter]

:red_circle: CI State: FAILURE

:white_check_mark: - Build :x: - Unit Tests Custom The following new/updated tests ran 100 times for each mode: :small_blue_diamond: cql-pytest/test_cast_data :small_blue_diamond: cql-pytest/test_counter :small_blue_diamond: cql-pytest/test_tools :small_blue_diamond: cql-pytest/test_unset :small_blue_diamond: cql-pytest/test_wasm

Failed Tests (58/4770):

• test_scylla_sstable_script[table_with_counters].debug.1 :mag:
• test_scylla_sstable_script[table_with_counters].debug.10 :mag:
• test_scylla_sstable_script[table_with_counters].debug.2 :mag:
• test_scylla_sstable_script[table_with_counters].debug.3 :mag:
• test_scylla_sstable_script[table_with_counters].debug.4 :mag:
• test_scylla_sstable_script[table_with_counters].debug.5 :mag:
• test_scylla_sstable_script[table_with_counters].debug.6 :mag:
• test_scylla_sstable_script[table_with_counters].debug.7 :mag:
• test_scylla_sstable_script[table_with_counters].debug.8 :mag:
• test_scylla_sstable_script[table_with_counters].dev.1 :mag:

Build Details:

• Duration: 1 hr 57 min
• Builder: spider7.cloudius-systems.com

[scylladb-promoter]

Docs Preview :book:

Docs Preview for this pull request is available here

Changed Files:

• alternator/alternator
• using-scylla/cdc/index.html
• using-scylla/counters
• using-scylla/lwt

Note: This preview will be available for 30 days and will be automatically deleted after that period. You can manually trigger a new build by committing changes.

[scylladb-promoter]

:red_circle: CI State: FAILURE

:white_check_mark: - Build :white_check_mark: - Unit Tests Custom The following new/updated tests ran 100 times for each mode: :small_blue_diamond: cql-pytest/test_cast_data :small_blue_diamond: cql-pytest/test_counter :small_blue_diamond: cql-pytest/test_tools :small_blue_diamond: cql-pytest/test_unset :small_blue_diamond: cql-pytest/test_wasm :white_check_mark: - Container Test :x: - Unit Tests :white_check_mark: - dtest with topology changes :white_check_mark: - dtest

Failed Tests (69/78922):

• cql-pytest.cassandra_tests/batch_test.debug.1 :mag:
• cql-pytest.cassandra_tests/batch_test.debug.2 :mag:
• cql-pytest.cassandra_tests/batch_test.debug.3 :mag:
• cql-pytest.cassandra_tests/functions/cast_fcts_test.debug.1 :mag:
• cql-pytest.cassandra_tests/functions/cast_fcts_test.debug.2 :mag:
• cql-pytest.cassandra_tests/functions/cast_fcts_test.debug.3 :mag:
• cql-pytest.cassandra_tests/validation/entities/counters_test.debug.1 :mag:
• cql-pytest.cassandra_tests/validation/entities/counters_test.debug.2 :mag:
• cql-pytest.cassandra_tests/validation/entities/counters_test.debug.3 :mag:
• cql-pytest.cassandra_tests/validation/entities/secondary_index_test.debug.1 :mag:

Build Details:

• Duration: 7 hr 12 min
• Builder: spider7.cloudius-systems.com

[ptrsmrn]

@kbr-scylla test/topology_experimental_raft/test_fencing.py::test_fence_writes fails (as expected) on:

>       table2 = await random_tables.add_table(name='t2', pks=1, columns=[
            Column("pk", IntType),
            Column('counter_c', CounterType)
        ])

when I disable creating tablets tables with counter columns (this is the purpose of this PR). How to make this test passing, should I:

1. change the offending column type to be e.g. of TextType?
2. skip table2 altogether for tablets case, because we were testing counters specifically? I assume we want to only change the problematic part and leave the remaining part of the test untouched.

[scylladb-promoter]

:red_circle: CI State: FAILURE

:white_check_mark: - Build :white_check_mark: - Container Test :white_check_mark: - dtest :white_check_mark: - dtest with topology changes :x: - Unit Tests

Failed Tests (9/31343):

• test_fence_writes.debug.1 :mag:
• test_fence_writes.debug.2 :mag:
• test_fence_writes.debug.3 :mag:
• test_fence_writes.dev.1 :mag:
• test_fence_writes.dev.2 :mag:
• test_fence_writes.dev.3 :mag:
• test_fence_writes.release.1 :mag:
• test_fence_writes.release.2 :mag:
• test_fence_writes.release.3 :mag:

Build Details:

• Duration: 2 hr 16 min
• Builder: spider7.cloudius-systems.com

[kbr-scylla]

@ptrsmrn I'd parameterize the test to run with and without tablets, and then in the tablets case skip checking the counter column.

For example of parameterizing see test/topology_experimental_raft/test_topology_ops.py:

@pytest.mark.parametrize("tablets_enabled", [True, False])
async def test_topology_ops(request, manager: ManagerClient, tablets_enabled: bool):

[scylladb-promoter]

:red_circle: CI State: FAILURE

:white_check_mark: - Build :white_check_mark: - Container Test :x: - dtest with topology changes :white_check_mark: - dtest :white_check_mark: - Unit Tests

Failed Tests (2/31349):

• test_drop_table :mag:
• test_drop_table :mag:

Build Details:

• Duration: 5 hr 36 min
• Builder: i-04c10d715215d2d4e (m5ad.8xlarge)

[ptrsmrn]

CI failed on a flaky dtest, the Issue against this failure has been already opened, I linked this failure to that ticket - https://github.com/scylladb/scylladb/issues/19620#issuecomment-2209489290 @nyh can you re-review please?

[ptrsmrn]

I've left some suggestions.

@annastuchlik thanks, I've resolved them.

[scylladb-promoter]

:green_circle: CI State: SUCCESS

:white_check_mark: - Build :white_check_mark: - Container Test :white_check_mark: - dtest :white_check_mark: - dtest with topology changes :white_check_mark: - Unit Tests

Build Details:

• Duration: 2 hr 19 min
• Builder: spider2.cloudius-systems.com

[ptrsmrn]

cling-tidy failure is unrelated to this PR, as in https://github.com/scylladb/scylladb/pull/18842#issuecomment-2209266931

[ptrsmrn]

@nyh can you please review?

[scylladb-promoter]

:green_circle: CI State: SUCCESS

:white_check_mark: - Build :white_check_mark: - Container Test :white_check_mark: - dtest :white_check_mark: - dtest with topology changes :white_check_mark: - Unit Tests

Build Details:

• Duration: 5 hr 50 min
• Builder: i-092bf5f3ced2c89d0 (m5ad.8xlarge)

[yaronkaikov]

@ptrsmrn This is the one to revert , it;s currently breaking dtest-release

[bhalevy]

For the record, this change wasn't reverted after all, since the dtest fixes were merged in time, so #19720 was closed without merging.

@ptrsmrn can you please prepare a backport to 2024.2? (https://github.com/scylladb/scylla-dtest/pull/4974 is already in)

Cc @tzach @nyh

[ptrsmrn]

2024.2 backport: https://github.com/scylladb/scylla-enterprise/pull/4985