zeppelin icon indicating copy to clipboard operation
zeppelin copied to clipboard

ZEPPELIN-3907. Add code statement into Spark JobGroup Id for SparkInterpreter

Open zjffdu opened this issue 7 years ago • 3 comments

What is this PR for?

Straightforward change to add code statement into Spark JobGroupId, so that we can find the associated spark job in spark ui easier.

What type of PR is it?

[ Improvement ]

Todos

  • [ ] - Task

What is the Jira issue?

  • https://jira.apache.org/jira/browse/ZEPPELIN-3907

How should this be tested?

  • Manually tested.

Screenshots (if appropriate)

image

Questions:

  • Does the licenses files need update? No
  • Is there breaking changes for older versions? No
  • Does this needs documentation? No

zjffdu avatar Dec 13 '18 08:12 zjffdu

While this is a really nice feature to have, it does introduce some risk of information leakage in an organization. Take for example select * from tableofsecrets where secretcolumn = 'supersecret'. What about adding a configuration parameter for activating this feature?

FireArrow avatar Dec 14 '18 09:12 FireArrow

While this is a really nice feature to have, it does introduce some risk of information leakage in an organization. Take for example select * from tableofsecrets where secretcolumn = 'supersecret'. What about adding a configuration parameter for activating this feature?

That's a good point.

Although I think it's possible to lock down Spark Driver UI through spark.ui.filters?

Also it might be possible to redact out passwords and other sensitive information in Zeppelin - something similar to spark.redaction.regex.

Maybe this change should be an option and off by default?

Tagar avatar Jan 28 '19 19:01 Tagar

Another option is to set Spark JobGroup Id to paragraph's title ..

Tagar avatar Feb 27 '19 17:02 Tagar