spark-redshift
spark-redshift copied to clipboard
databricks
Fix executing empty sql statement in preactions and postactions
Say for example: you need to execute preactions/postactions
val preactions = "update table1 set col1=2; update table2 set col2 = 2; "
Looks nothing wrong, correct?
Only when you try to execute it, you will get weird error:
Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 0
This is because of Line 248 and Line 260 in Parameters.scala
def preActions: Array[String] = parameters("preactions").split(";")
def postActions: Array[String] = parameters("postactions").split(";")
the above functions are called in RedshiftWriter.scala LN: 132 and LN: 195
parameters("preactions").split(";")
-- this returns an array of 3 items, with the 3rd item being empty string (only spaces) - see trailing spaces in
val preactions = "update table1 set col1=2; update table2 set col2 = 2; "
and then it tries to execute sql query which is essentially just "spaces"
The fix is to just trim preactions and postactions so that the trailing spaces are not considered as query to be executed
