scylla-machine-image icon indicating copy to clipboard operation
scylla-machine-image copied to clipboard

[DRAFT] move cloud snitches to external python script

Open syuu1228 opened this issue 1 year ago • 4 comments

This is scylla-machine-image part of "move cloud snitches to external python script" PR. Currently this PR is DRAFT version since it doesn't implemented Ec2MultiRegionSnitch yet.


Since it's hard to debug scylla-core's cloud snitch support, we want to move them out to python script. On the python script we will generate cassandra-rackdc.properties file, and scylla-core will read it via GossipingPropertyFileSnitch code.

See scylladb/scylladb#12306

syuu1228 avatar Apr 22 '23 12:04 syuu1228

scylla-core part of the PR is at: https://github.com/scylladb/scylladb/pull/13623

syuu1228 avatar Apr 22 '23 12:04 syuu1228

I implemented JavaPropertiesParser on scylla_cloud_snitch, it's modified version of sysconfig_parser, since the format is very similar. I wanted it just for read / write cassandra-rackdc.properties, without doing regexp stuff for each read / write, since it would be hard to maintain.

But there is a pip module to parse java properties https://pypi.org/project/javaproperties/, maybe we can consider to use it.

syuu1228 avatar Apr 22 '23 12:04 syuu1228

TODO:

  • Implement Ec2MultiRegionSnitch
    • find a way to configure broadcast_address, broadcast_rpc_address from the script. I think we need to modify scylla.yaml to do that
  • Implement exponential backoff on curl(). Currently it just sleep 5 sec for each retries

syuu1228 avatar Apr 22 '23 12:04 syuu1228

@avelanarius - hopefully, the drivers may be able to (re)use this functionality as well.

mykaul avatar Apr 23 '23 09:04 mykaul