Escape backtick from spark-submit arguments

[s0nskar]

Currently, livy does not escape backticks from user-provided spark-submit arguments. So if a customer is passing any arguments that contain backticks, it will be considered as command substitution during spark-submit causing that argument to become blank or invalid.

Example:

--query 'select * from test_db.`test_table`' 

will become

--query 'select * from test_db.' 

[s0nskar]

cc: @pralabhkumar

[s0nskar]

I'm working on this.

[RonZhang724]

@s0nskar Thank you for working on this. My company uses AWS EMR for our data processing tasks. EMR uses livy to handle job submissions, and we noticed the same issue when submitting queries with backticks. This has resulted us having to change the queries on our side. And it prevented us from using dot notations in our query ALIAS.

For example, the following will not work

  COALESCE(
    `measure`.`region`,
  ) AS `measure_alias.region_alias`

It would be nice if we could have this change included in the up coming livy releases.