ScyllaDb support

[ayZagen]

Any plan to support ScyllaDb ?

[masumsoft]

I've looked into scylla before, but scylladb as of now is mostly cassandra 2.x compliant which is no longer supported. Express-Cassandra specialises in casssandra 3.x+ and I don't have any plan to keep supporting 2.x releases. I'll try to look into it again if scylla catches up.

[thelordoftheboards]

Based on their roadmap, given they stick to it, what version would you consider worth trying with?

[masumsoft]

@thelordoftheboards I guess it's atleast 2.4 that may be worth trying due to several reasons:

1. Materialized View is expected to be production ready
2. Secondary indexes is expected to be production ready
3. LWT is supported
4. sstable storage format is implemented

I guess keeping support for cassandra 2.x and scylladb current versions is possible but would be painful. These older versions are very different in the storage and system tables structure. They lack support for several migration related operations and awesome new features like materialized views, advanced data types and light weight transactions.

[thelordoftheboards]

I think it makes sense to stick with the 3.x compatibility given that Scylla has declared that they are gunning for 3.x compatibility, not to mention that back-porting functionality to 2.x would require extra development effort and would make the project bulkier and possibly slower. I experimented running it anyway. express-cassandra successfully created several tables with materialized views on first run, but when I ran the project again it complained something about indexes. Hopefully Scylla will develop 3.x compatibility in the near future.

[thelordoftheboards]

Scylla recently released version 3, with, as far as I can understand, the intention to be Cassandra 3 compatible.

I ran it with my project (using express-cassandra). This time, unlike when running it against scylla version 2, express-cassandra managed to create all the tables. Unfortunately next time it is being run, it fails because it thinks express-cassandra needs to delete a materialized view. I'd assume it is a small issue with reading the schema, because the tables created at the first pass look perfectly fine and in line with what I would expect.

If you would be inclined to support Scylla 3, I could provide detailed instructions on how to reproduce the issue.

[thelordoftheboards]

Steps to reproduce the issue with Scylla 3.0-rc1:

• Install Scylla (using docker in this example)

docker pull scylladb/scylla:3.0-rc1
docker run --name scylly -d -p 9042:9042 scylladb/scylla:3.0-rc1 --experimental=1

If needed, verify install and troubleshoot:

docker exec -it scylly scylla --version
docker exec -it scylly nodetool status
docker logs scylly | tail
docker exec -it scylly cqlsh

• Install project (requires yarn, Node 10 LTS)

git clone https://github.com/MachineAcuity/rebar.git
cd rebar
yarn
yarn setup-local

• Perform initial creation of keyspace and tables

$ yarn setup-database
yarn run v1.12.1
$ ./node_modules/.bin/babel-node ./units/rb-appbase-server/setupDatabase.js
 Prepare table User.
 Table User ready.
 Prepare table UserAccount.
 Table UserAccount ready.
 Prepare table UserPermissionForObject.
 Table UserPermissionForObject ready.
 Prepare table UserQuotaForObject.
 Table UserQuotaForObject ready.
 Prepare table UserSession.
 Table UserSession ready.
 Prepare table logs.
 Table logs ready.
 Prepare table Ensayo.
 Table Ensayo ready.
 Prepare table Inscriptio.
 Table Inscriptio ready.
 Prepare table ToDo.
 Table ToDo ready.
 Prepare table Translaticiarum.
 Table Translaticiarum ready.
✨  Done in 9.50s.

Here you can inspect the tables created, this is what I get:

$ docker exec -it scylly cqlsh
Connected to  at 172.17.0.2:9042.
[cqlsh 5.0.1 | Cassandra 3.0.8 | CQL spec 3.3.1 | Native protocol v4]
Use HELP for help.
cqlsh> use rebar_dev;
cqlsh:rebar_dev> describe tables;

"Translaticiarum"  "UserPermissionForObject"  "Ensayo"              "ToDo"
logs               "UserSession"              "UserAccount"       
"Inscriptio"       "User"                     "UserQuotaForObject"

cqlsh:rebar_dev> describe table "Ensayo";

CREATE TABLE rebar_dev."Ensayo" (
    id uuid PRIMARY KEY,
    "Ensayo_Content" text,
    "Ensayo_Description" text,
    "Ensayo_Title" text,
    "Ensayo_User_id" uuid,
    "Ensayo_artifact_id" uuid
) WITH bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
    AND comment = ''
    AND compaction = {'class': 'SizeTieredCompactionStrategy'}
    AND compression = {}
    AND crc_check_chance = 1.0
    AND dclocal_read_repair_chance = 0.1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99.0PERCENTILE';
CREATE INDEX "Ensayo_Ensayo_User_id_idx" ON rebar_dev."Ensayo" (Ensayo_User_id);

CREATE MATERIALIZED VIEW rebar_dev."Ensayo_Ensayo_User_id_idx_index" AS
    SELECT "Ensayo_User_id", idx_token, id
    FROM rebar_dev."Ensayo"
    WHERE "Ensayo_User_id" IS NOT NULL
    PRIMARY KEY ("Ensayo_User_id", idx_token, id)
    WITH CLUSTERING ORDER BY (idx_token ASC, id ASC)
    AND bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
    AND comment = ''
    AND compaction = {'class': 'SizeTieredCompactionStrategy'}
    AND compression = {}
    AND crc_check_chance = 1.0
    AND dclocal_read_repair_chance = 0.1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99.0PERCENTILE';

• Attempt creation of tables again

$ yarn setup-database
yarn run v1.12.1
$ ./node_modules/.bin/babel-node ./units/rb-appbase-server/setupDatabase.js
 Prepare table User.
 Table User ready.
 Prepare table UserAccount.
 Table UserAccount ready.
 Prepare table UserPermissionForObject.
 Table UserPermissionForObject ready.
 Prepare table UserQuotaForObject.
 Table UserQuotaForObject ready.
 Prepare table UserSession.
 Table UserSession ready.
 Prepare table logs.
 Table logs ready.
 Prepare table Ensayo.
Error:  Initializing Cassandra persister - error while creating Ensayo!
Error while dropping materialized view on DB Table -> ResponseError: Cannot use DROP MATERIALIZED VIEW on Index
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

If using the entire project for troubleshooting is too cumbersome, I can try to cut it down to only the parts that perform the table creation.

The actual file with the model for Ensayo is located at: https://github.com/MachineAcuity/rebar/blob/master/units/rb-example-ensayo-server/graphql/model/Ensayo.schema.js

[masumsoft]

scylladb secondary index and materialized view implementation is buggy as of this rc release. It looks like materialized views are handled the same way as indexes which is not compatible with cassandra. Moreover if a materialized view and a secondary index uses the same column name, then some of the materialized view operations fails. Also creating user defined functions, aggregates and indexed collections throws errors.

I guess as this is in RC, these issues should get resolved before they release the final version.

[masumsoft]

I'll keep this issue open and when scylladb version 3 gets out of rc, I would like to try it again.

[thelordoftheboards]

Once they release the final, I can also try and let you know to save you the effort in case they would have fixed everything. If not, maybe it would make sense to open issues with them. Either way, with such a number of issues, it would be nice to wait until they fix at least some of them.

[thelordoftheboards]

@masumsoft

So, there's that:

https://github.com/scylladb/scylla/releases/tag/scylla-3.0.2

Do you want me to test using my project?

[tskweres]

I'm just trying to drop my DB in dev / test, and I'm getting this error:

Error while dropping materialized view on DB Table -> ResponseError: Cannot use DROP MATERIALIZED VIEW on Index

How can I just....completely drop the DB so I can continue coding?

[thelordoftheboards]

DROP KEYSPACE xyz drops the entire keyspace (DB) but are you sure you want to do that?

Also, in other news, I have been running express-cassandra with Scylla 3.2 without any issues.

[tskweres]

@thelordoftheboards yeah I've got it working, but if my models aren't fully fleshed out I want to be able to change them frequently as I'm building

[masumsoft]

It looks like materialized views are handled the same way as indexes which is not compatible with Cassandra. Moreover if a materialized view and a secondary index use the same column name, then some of the materialized view operations fail. Also creating user-defined functions, aggregates and indexed collections throws errors.

Tried on Scylladb 4.0 again. The above issues still remain. Most features seem to work just fine. So if you are not using advanced features of Cassandra like UDT, UDF, indexed collections, etc. and you don't have conflicting views and indexes, Scylladb should work fine.

[masumsoft]

v2.9.0 now supports scylladb 5.x.