zookeepers and mesos-hdfs

[mesosapprentice]

Wondering how many zookeepers do you need for this to deploy correctly. I was trying to leverage some existing zookeepers that I deployed with CDH but then I was having mesos issues. I created a single zookeeper for mesos and that solved some of my problems, but wondering if I need to create a quorum of zookeepers for this to work properly. I recently tried a combination of both my new mesos zookeeper and 2 of my zookeepers from CDH and I am getting this error when I attempt to deploy the framework. Using the latest pull that is available.

Exception in thread "main" java.lang.IllegalArgumentException: Does not contain a valid host:port authority: :50071

What is your recommendation?

[elingg]

hi @mesosapprentice, you could share zookeepers with mesos itself for a simple deployment. Even with one zookeeper, you could have a quorum of 1. With production use or high load use, we recommend a different zookeeper cluster for Mesos as well as for each framework you are running on Mesos.

[mohamedhaleem]

@elingg - different zookeeper cluster for each framework sounds a little overwhelming, could you describe the setup where this would be necessary?

[elingg]

A zookeeper cluster needs to maintain consistency and consesus and with multiple requests and higher loads, zookeeper can become unresponsive. For a basic use case, you can share a zookeeper cluster.

[asdfsx]

I got the same problem,~~~ "Does not contain a valid host:port authority: :50071" I wonder where is the hostname~~~~

[christobill]

I had the same problem, 3 changes solved the issue for me:

1. mesosdns in mesos-site.xml changed to true (but I guess you will need to deploy mesos-dns in marathon)
2. all possible resources in mesos-site.xml to 256k and 0.05 cpu (!!)
3. add 50070-50071 to mesos-slave resources (and 8020 if you need remote access)