fluid icon indicating copy to clipboard operation
fluid copied to clipboard

How Dose the HDFS Kerberos be embedded into fluid and alluxio?

Open wujiuhsu opened this issue 3 years ago • 2 comments

We use Tensorflow in our machine learning with data in hdfs, Estimator reads data from HDFS using dataset. We want to use fluid to speed up our training. When we run the demo in https://fluid-cloudnative.github.io/zh/samples/hdfs_configuration.html#%E8%BF%90%E8%A1%8C%E7%A4%BA%E4%BE%8B, we have a question how to embed the HDFS Kerberos to fluid? This error occurs in hdfs-master-0. image

wujiuhsu avatar Jun 02 '22 03:06 wujiuhsu

Thank you for opening this issue. I'd like to help you in personal. If you'd like, please join the dingding talk group and reach me. My ding talk id is cheyang52

I think basically it depends on two features:

  1. alluxio runtime supports pvc attributes. I think it's similar with https://github.com/fluid-cloudnative/fluid/blob/master/docs/zh/samples/gcs_configuration.md

  2. options and jvmOptions should be set by following https://docs.alluxio.io/ee/user/stable/en/operation/Security.html

cheyang avatar Jun 03 '22 05:06 cheyang

Thanks for your help. I have added you in ding talk.

wujiuhsu avatar Jun 06 '22 05:06 wujiuhsu