spark-rapids icon indicating copy to clipboard operation
spark-rapids copied to clipboard

[SPARK-39917][SQL] Use different error classes for numeric/interval arithmetic overflow

Open amahussein opened this issue 2 years ago • 0 comments

Spark-context

https://github.com/apache/spark/commit/1b6f14ff8c reverted: https://github.com/apache/spark/commit/939c2402c8 put back: https://github.com/apache/spark/commit/51e4c2cc55

What changes were proposed in SPARK pull request?

Use different error class INTERVAL_ARITHMETIC_OVERFLOW for arithmetic overflows of numeric/interval.

Why are the changes needed in Spark?

error messages are simpler and more clear.

Before that PR the error messages are like

[ARITHMETIC_OVERFLOW] long overflow. Use 'try_multiply' to tolerate overflow and return NULL instead.
If necessary set spark.sql.ansi.enabled to "false"

The "(except for ANSI interval type)" part is confusing.

Does this PR introduce any user-facing change?

Yes, different classes for arithmetic

Why it might affect RAPIDS?

  • added a new class QueryExecutionErrors.intervalArithmeticOverflowError
  • A slight change in the error message of ARITHMETIC_OVERFLOW

Impact on Testing?

Yes

Requires Doc update?

No

amahussein avatar Aug 09 '22 14:08 amahussein