[Question/Vision] Future of this repo vs "cloudera-playbook" repo

[lhoss]

Motivation for this issue: Have a place in the community to discuss the future of Cloudera distributions (HDP, CDH, CDP) Deployment/DevOps (with ansible).

• for HDP ansible installations we have this repo
• for CDH (and CDP) ansible installations we have https://github.com/cloudera/cloudera-playbook

Since the new CDP is based on the Cloudera Manager, and not on Ambari.. it seems clear that for CDP deployment the cloudera-playbook is the repo for the future, BUT I have to say that many things in this hortonworks repo are more advanced (many more feature and flexibility).. now to my main question:

• how can we re-use the good/best parts also for cloudera-playbook, without reinventing the wheel there ?!
  • Did Cloudera think about this @seanorama ?

Features in this repo (and not in cloudera-playbook):

• one being support for multiple Linux distros (in hortonworks repo) vs only redhat family (in cloudera-playbook)
  • which comes with one downside however (in hortonworks): the mechanism on how the variables are loaded (dep. on the OS), are not allowing overrides via group_vars (I mentioned that in various issues or PRs)
• other nice features: builtin cloud cluster setup (for AWS, Azure, GCP..)
  • Though in a future ansible repo/role I'ld expect recent ansible/redhat to provide standardized roles for this
• and the support for different DBs: mysql/mariaDB, postgres (that I used mostly, though recently I used external ansible-postgresql role , to have more flexibility )

[tmgstevens]

Hi. So there is active work going to rebuild the cloudera-playbook to work with CDP-DC, including many of the aspects that you are talking of. Things on our mind right now:

• Multi-OS support - yes we are doing that
• External databases, Postgres, MySQL/Maria and Oracle, although only provisioning MySQL to start with I believe.

Provisioning of Cloud instances to support is unlikely to be included, simply because you would be significantly better off going down the CDP Public Cloud route for this. In the past we did this with Cloudbreak/Cloudera Director, but I think we'll see diminishing numbers of Cloud IaaS clusters.

[seanorama]

@lhoss I'm pinging around for input, hence Tristan's response.

Note that ansible-hortowrorks is not a Cloudera product and these playbooks are not officially supported by Cloudera. It is a personal project of a few folks at Cloudera and other companies, including yourself.