metorikku icon indicating copy to clipboard operation
metorikku copied to clipboard

Published 20 hours ago • verified publisher icon YotpoLtd

How we can read and write the data to Hive?

Open sunilpandey-de opened this issue 3 years ago • 3 comments

Can someone help me understand using metorikku how we can read and write the data to Hive tables.

Thank You!

sunilpandey-de avatar Jul 05 '21 10:07 sunilpandey-de

Can you share your environment details? (yarn/k8s/spark-shell?) which hive you are using? what version of spark?

Thanks

lyogev avatar Jul 07 '21 11:07 lyogev

We are running this on our development yarn cluster, Hive version 2.3 and Spark 3.0.

sunilpandey-de avatar Jul 07 '21 12:07 sunilpandey-de

So hive is configured correctly in the spark-defaults etc? If so all you need to save in hive is set tableName in your output https://github.com/YotpoLtd/metorikku/blob/7fdd0838480e89285e5c7b77cc3e45d54d6bf5a0/config/metric_config_sample.yaml#L50

lyogev avatar Jul 11 '21 13:07 lyogev