mesos
Support different Zookeepers clusters to store state
In our environment, we want to separate the zookeeper cluster used by mesos masters from the zk used to store schedulers data.
It would be a nice improvement to allow to use separate zookeepers clusters.
It's an interesting enhancement but not a priority at the moment. Also requires a lot of testing relative to the feature. However you are welcome to add a PR for this.
We have heard from other people that this is an important feature so we will add it to 0.3
- [x] Add zookeeperFrameworkUrl to ZookeeperCLIParameter
- [x] Elasticsearch should use zookeeperFrameworkUrl instead of zookeeperUrl
- [x] Add zookeeperFrameworkTimeout
- [x] ZookeeperFrameworkUrl should use value of ZookeeperMesosUrl when it is not set
- [x] Rename zookeeperUrl in ZookeeperCLIParameter to zookeeperMesosUrl
- [x] Check hardcoded params in system test
- [x] Add system test
Based on the configuration page, the /mesos at the end of Zookeeper is hardcoded somewhere. My Mesos cluster uses /mesosha and when I put in zk://node1:5181,node2:5181,node3:5131/mesosha, the configuration overview still has /mesos in the valud MesosZKURL. Shouldn't we take this from the config rather than assuming /mesos?
If heard of this desire as well. What are the arguments for doing this?
@WalterDalton I'm sorry, I've only just seen this. This is a separate issue and I will open a bug. @systeminsightsbuild It could be said there are two reasons. 1) clarity. You don't have lots of scheduler spam being written to the Mesos ZK. (I don't agree with this point). 2) The more reasonable thought is that you need Mesos to be able to read/write to ZK. By isolating the schedulers to write to another cluster, you mitigate against a scheduler overwhelming ZK with too many reads/writes.
@frankscholten It seems you have ticked lots of boxes on this, and I remember you doing the work. What happened to it? The options proposed above are not in the code?
