spark icon indicating copy to clipboard operation
spark copied to clipboard

[SPARK-40031][SQL] Remove unnecessary TryEval in TryCast

Open gengliangwang opened this issue 2 years ago • 0 comments

What changes were proposed in this pull request?

When the try_cast() syntax is equivalent to the non-ansi cast, convert TryCast as Cast without wrapping with "TryEval" expression.

Why are the changes needed?

  1. For better performance. Taking try_cast(string_col as int) as example, Spark will have to contact error messages from SparkThrowableHelper. getMessage for every invalid casting. This is actually not necessary in the try_cast function.
  2. More accurate nullability. If the child of try_cast is non-nullable and the casting itself is up-cast, the result should be non-nullable too.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing UT

gengliangwang avatar Aug 10 '22 06:08 gengliangwang