datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard

More types in `try_*`

Open davidlghellin opened this issue 4 months ago • 1 comments

What is the problem the feature request solves?

Some data types seem to be missing in the current implementation of try_*. I might be a bit late to the party 😅 but I didn’t find an open issue tracking this. Comes from here https://github.com/apache/datafusion-comet/pull/2073

Describe the potential solution

Support for more data types, similar to how it was done in Sail. Here are some example tests from Spark showing expected behavior:


>>> df = spark.sql("""
... SELECT
...   try_add(DATE '2015-09-30', 1) as d1,
...   try_add(DATE '2000-01-01', 366) as d2,
...   try_add(DATE '2021-01-01', 1) as d3,
...   try_add(NULL, 100) as d4
... """)
>>> df.show()
+----------+----------+----------+----+
|        d1|        d2|        d3|  d4|
+----------+----------+----------+----+
|2015-10-01|2001-01-01|2021-01-02|NULL|
+----------+----------+----------+----+

>>> df = spark.sql("""
... SELECT
...   try_add(DATE '2015-01-31', INTERVAL 1 MONTH) as d1,
...   try_add(DATE '2020-02-29', INTERVAL 12 MONTH) as d2,
...   try_add(NULL, INTERVAL 3 MONTH) as d3
... """)
>>> df.show()
+----------+----------+----+
|        d1|        d2|  d3|
+----------+----------+----+
|2015-02-28|2021-02-28|NULL|
+----------+----------+----+

>>> df = spark.sql("""
... SELECT
...   try_add(DATE '2000-07-31', INTERVAL -1 MONTH) as d1,
...   try_add(DATE '2021-01-31', INTERVAL -1 MONTH) as d2
... """)
>>> df.show()
+----------+----------+
|        d1|        d2|
+----------+----------+
|2000-06-30|2020-12-31|
+----------+----------+

>>> df = spark.sql("""
... SELECT
...   try_add(INTERVAL '1' YEAR, INTERVAL '2' YEAR) as result
... """)

>>> spark.conf.set("spark.sql.session.timeZone", "UTC")
>>> df = spark.sql("""
... SELECT
...   try_add(TIMESTAMP '2021-01-01 00:00:00', INTERVAL 1 DAY) as result
... """)
>>> df.show(truncate=False)
+-------------------+
|result             |
+-------------------+
|2021-01-02 00:00:00|
+-------------------+

Additional context

Implementation in Sail for reference: https://github.com/lakehq/sail/pull/635

davidlghellin avatar Aug 18 '25 12:08 davidlghellin

Thank you very much for filling this issue. I will be working on this once my ANSI changes are completed

coderfender avatar Sep 15 '25 07:09 coderfender