byzer-lang icon indicating copy to clipboard operation
byzer-lang copied to clipboard

Scope SQLConf of session.active

Open zml1206 opened this issue 1 year ago • 0 comments

What changes were proposed in this pull request?

Set activeSession before each script execution. Because spark functions.expr use SQLConf of activeSession. Refer to SPARK-44206, all SparkSession dataset methods already covered by withActive, but functions still only use SQLConf of active SparkSession.

How was this patch tested?

Refer to SPARK-44206

Are there and DOC need to update?

Unnecessary

Spark Core Compatibility

zml1206 avatar Jun 28 '23 03:06 zml1206