apache
[Improvement] Using [session user] as [proxy user] to execute statements at the server share level
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Search before asking
- [X] I have searched in the issues and found no similar issues.
What would you like to be improved?
At the server share level, If I don't use the Ranger AuthZ Plugin, multiple users use the system user's credential, who submits the spark job, to access HDFS, rather than their owner credential.
So on the HDFS side, all actions are executed as the system user, so the HDFS Ranger Plugin loses control.
I want to find a way for all session users can use their own Identity to access HDFS. Thus at the server share level, session users could be under minimum access control.
How should we improve?
I read Kyuubi's code And found the function setting appropriate configs for each session user. 👇
https://github.com/apache/kyuubi/blob/master/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkOperation.scala#L130
Maybe we can set [session user] as [proxy user] to execute statements, like this: 👇
Are there some lurking issues or concerns at the system level?
Looking forward to your opinions very much!
Are you willing to submit PR?
- [X] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
- [ ] No. I cannot submit a PR at this time.
Hello @theoryxu, Thanks for finding the time to report the issue! We really appreciate the community's efforts to improve Apache Kyuubi.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
@theoryxu we can do that, for GROUP and SERVER share level, configurablly.
actually, this only covers the execution at the driver side, such as DDL commands, when the query triggers a Job, the task executed on the executor side still uses the application user.
I know that STS supports that, I suppose it has same issue I mentioned above, also cc @wangyum
@pan3793 Is anyone working on this? If not, I can submit a PR for it recently and then discuss it in more detail.
@theoryxu please go head, also cc @yaooqinn and @cxzl25 (we have discussed this topic offline before)
