velox icon indicating copy to clipboard operation
velox copied to clipboard

Add to_iso8601(date) Presto function

Open svm1 opened this issue 1 year ago • 19 comments

svm1 avatar Jan 26 '24 03:01 svm1

Deploy Preview for meta-velox canceled.

Name Link
Latest commit 43e3b6ca4db335ee51ebed1e2d76114eff5bfdaf
Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/660d919bd406530008f38e87

netlify[bot] avatar Jan 26 '24 03:01 netlify[bot]

@svm1 I see the following output in Presto for date and timestamp inputs.

presto> select to_ISO8601(cast('1970-01-01' as DATE));
   _col0    
------------
 1970-01-01 
(1 row)

presto> select to_ISO8601(cast('1970-01-01' as timestamp));
             _col0             
-------------------------------
 1970-01-01T00:00:00.000+05:30 
(1 row)

majetideepak avatar Feb 13 '24 11:02 majetideepak

@majetideepak Thanks for pointing that out, I recently made the same observation myself. Corrected behavior to comply with Presto Java.

svm1 avatar Feb 13 '24 23:02 svm1

@majetideepak @aditi-pandit please take a look, changes have been finalized.

svm1 avatar Feb 23 '24 01:02 svm1

@svm1 : Please can you fix format-check failures.

aditi-pandit avatar Feb 23 '24 21:02 aditi-pandit

@aditi-pandit Done.

svm1 avatar Feb 23 '24 23:02 svm1

@mbasmanova can you take a look at this PR? Thanks.

majetideepak avatar Feb 28 '24 01:02 majetideepak

@mbasmanova Thanks for the review.

I was told to break up the all-encompassing PR into separate PRs for each function signature, for ease of review, as each set of changes was becoming fairly lengthy. Here are the other two - https://github.com/facebookincubator/velox/pull/8551, https://github.com/facebookincubator/velox/pull/8550. The idea was that they could be merged sequentially, which each new PR further expanding the functionality of to_iso8601() to support the other types.

svm1 avatar Feb 28 '24 21:02 svm1

I was told to break up the all-encompassing PR into separate PRs for each function signature, for ease of review, as each set of changes was becoming fairly lengthy.

Got it. Thank you for clarifying. In the future, please, share this context upfront to help reviewer understand why the PR has some limitations and realize that you'll be asking for more reviews shortly. Code review is labor intensive time consuming activity. Reducing back-and-forth by communicating proactively helps. Thanks.

mbasmanova avatar Feb 29 '24 00:02 mbasmanova

Implement Presto function to_iso8601() for Date type input, with tests.

This sentence in the PR description is redundant. The title already says that you are adding to_iso8601 function for date inputs. Since every single change must come with tests, saying "with tests" is redundant.

mbasmanova avatar Feb 29 '24 00:02 mbasmanova

@mbasmanova Sorry for the confusion - I think I better understand your point now about making toIso8601() a static function. Though I did want to mention, there is an existing value-based conversion function in the DateType class -toString(int32 days). Also noticed similar value functions in the IntervalDayTimeType class, another logical type. I was wondering if even this should be present here in the logical type definition, or if it should be made a static function elsewhere, perhaps in TimestampConversion.h. As in, should we be relying on an instance of the class for any value conversion operations in the first place?

svm1 avatar Mar 01 '24 00:03 svm1

@mbasmanova would appreciate your thoughts on the above:

Sorry for the confusion - I think I better understand your point now about making toIso8601() a static function. Though I did want to mention, there is an existing value-based conversion function in the DateType class - toString(int32 days). Also noticed similar value functions in the IntervalDayTimeType class, another logical type. I was wondering if even this should be present here in the logical type definition, or if it should be made a static function elsewhere, perhaps in TimestampConversion.h. As in, should we be relying on an instance of the class for any value conversion operations in the first place?

svm1 avatar Mar 06 '24 20:03 svm1

Also noticed similar value functions in the IntervalDayTimeType class, another logical type. I was wondering if even this should be present here in the logical type definition, or if it should be made a static function elsewhere, perhaps in TimestampConversion.h. As in, should we be relying on an instance of the class for any value conversion operations in the first place?

I haven't looked into these, but I expect these should be static methods, not instance methods.

mbasmanova avatar Mar 07 '24 00:03 mbasmanova

@mbasmanova Yes, that's my point - the existing value conversion methods for the Date type, in Type.h, are not static methods but instead are instance methods (toString(), toDays()). That's why I was asking if these should also be modified, if you wanted my new toIso8601() method to be static.

svm1 avatar Mar 07 '24 00:03 svm1

I was asking if these should also be modified

I see. It would be nice to update these as well, but in a separate PR. Thanks.

mbasmanova avatar Mar 07 '24 01:03 mbasmanova

@mbasmanova I've made all the changes - created toIso8601() as a static method and overloaded toString() to take in TimestampToStringOptions as an argument, so it can be called by the former with the necessary flags for ISO formatting. This mitigates the issue of relying on and modifying toString() for the sake of the Presto function implementation.

Please take a look when you get a chance!

svm1 avatar Mar 08 '24 01:03 svm1

Rebased with main again, hitting a new fuzzer failure. Opened issue - https://github.com/facebookincubator/velox/issues/9192

svm1 avatar Mar 21 '24 01:03 svm1

@mbasmanova CI clean now.

svm1 avatar Apr 02 '24 02:04 svm1

@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot avatar Apr 03 '24 14:04 facebook-github-bot

@mbasmanova merged this pull request in facebookincubator/velox@e4f74d3bfc219cbf399ca54e9d0bdd3f32fe9d6d.

facebook-github-bot avatar Apr 04 '24 14:04 facebook-github-bot

Conbench analyzed the 1 benchmark run on commit e4f74d3b.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

conbench-facebook[bot] avatar Apr 04 '24 14:04 conbench-facebook[bot]