Remote restore only working for seeders

[alvaropalmeirao]

Project board link

Hi all,

I'm restoring a cluster (3 seeders and 3 data nodes) and only the seeders are able to start:

nodetool -h $(hostname -i) -u [REDACTED] -pw [REDACTED] status
Datacenter: us-east-1
=====================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address      Load       Tokens       Owns    Host ID                               Rack
UN  [REDACTED]   2.55 MB    256          ?      [REDACTED]  us-east-1b
UN  [REDACTED]   1.66 MB    256          ?       [REDACTED]  us-east-1a
UN [REDACTED]  2.7 MB     256          ?       [REDACTED]  us-east-1c

On the data nodes I have always this error:

[2022-05-19 11:31:16,312] DEBUG: Unable to determine if node is up for host: [REDACTED]
Traceback (most recent call last):
  File "/usr/share/cassandra-medusa/lib/python3.7/site-packages/medusa/cassandra_utils.py", line 613, in is_node_up
    return is_cassandra_healthy(check_type, Cassandra(config, release_version=HostMan.get_release_version()),
  File "/usr/share/cassandra-medusa/lib/python3.7/site-packages/medusa/cassandra_utils.py", line 339, in __init__
    self._storage_port = config_reader.storage_port
  File "/usr/share/cassandra-medusa/lib/python3.7/site-packages/medusa/cassandra_utils.py", line 266, in storage_port
    if self._release_version is not None and Version(self._release_version) >= Version('4-a') and \
  File "cassandra/util.py", line 1947, in cassandra.util.Version.__init__
TypeError: argument of type 'Version' is not iterable

This error repeats forever. All seeders are up and running. Can you help me or guide me to the possible resolution of this problem? Thanks.

┆Issue is synchronized with this Jira Task by Unito ┆friendlyId: K8SSAND-1521 ┆priority: Medium

[adejanovski]

Hi, it looks like you're using Scylla and it seems like the version number is incompatible with the library we're using to parse it. FTR, we're using the Version method from the cassandra.util package in the Cassandra Python driver.

[alvaropalmeirao]

Yes, I'm using Scylla 4.6.1. Which version number are you referring to? Scylla? How can I solve this?

[alvaropalmeirao]

Ok, I understand now what's going on. You use Version method from the cassandra.util package in the Cassandra Python driver. So, if i use scylladb this doesn't work? 🤔

[alvaropalmeirao]

Hi again,

If I remove this condition and Version(self._release_version) >= Version('4-a') on this file https://github.com/thelastpickle/cassandra-medusa/blob/0.11/medusa/cassandra_utils.py#L266 the restore went successfully without any problems. Can i remove this condition?

Best regards.

[adejanovski]

Removing the condition doesn't sound like an option, but you could wrap it in a function that would be capable of parsing both Cassandra and Scylla version numbers. If you can detect that you're running against Scylla, then you could bypass the version check in the way that you see fit to get the right ssl storage port.

[alvaropalmeirao]

I can only detect if it is Scylla or Cassandra if I put a flag on cassandra.yml or medusa.ini. I tried to use the python driver but it has only documentation regarding Cassandra. My idea is to put a flag on medusa.ini (or cassandra.yml) called for example engine. The default value of this flag will be Cassandra and we could change the condition.