Automate EMR kerberos auth enable cluster creation

[pankajastro]

This PR automate

• AWS security configuration creation with auth mechanism Kerberos
• Create Kerberos to enable EMR EMR cluster
• Generation of keytab and Kerberos principal creation on EMR master node
• Downloading the Kerberos configuration and keytab file in the airflow DAG folder
• Hive partitioned table creation and insertion of data in it

But still won't be able to run end to end dag in a fully automated way because

• I need to update /etc/hosts file in airflow which a root user can only do
• copy the Kerberos configuration file in the airflow etc folder which a root user can only do
• Also, I had set hive.server2.enable.doAs as false in hive-site.xml and restarted hive-server2.service in order to insert data in the partitioned table using hive user
• How to run Kerberos airflow in both worker and trigger container?

Closes #363

[codecov[bot]]

Codecov Report

Merging #365 (d096644) into main (295d943) will not change coverage. The diff coverage is n/a.

:exclamation: Current head d096644 differs from pull request most recent head 2b52b81. Consider uploading reports for the commit 2b52b81 to get more accurate results

@@           Coverage Diff           @@
##             main     #365   +/-   ##
=======================================
  Coverage   96.78%   96.78%           
=======================================
  Files          56       56           
  Lines        2925     2925           
=======================================
  Hits         2831     2831           
  Misses         94       94           

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 295d943...2b52b81. Read the comment docs.

[phanikumv]

@kaxil any thoughts on the below

I need to update /etc/hosts file in airflow which a root user can only do copy the Kerberos configuration file in the airflow etc folder which a root user can only do

[phanikumv]

Closing this as we have a limitation to update /etc/hosts file