[FLINK-17224][table] Support precision for TIME
What is the purpose of the change
The PR makes support of TIME precision (0-3 - currently only this range, I'm going to submit support for micros and nanos in the next PR under probably https://issues.apache.org/jira/browse/FLINK-17525).
It includes casts, time related functions (CEIL, FLOOR, OVERLAPS, agg MAX, MIN, LEAD, LAG, singlevalue )
Also added support of TIME precision in csv, json, avro, some tests in Kafka (they use agg MAX e.g.)
Verifying this change
A number of tests in table-planner, avro, csv, json, kafka dealing with TIME
especially flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/CastFunctionITCase.java
and flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRulesTest.java
checking different variations of casts to and from TIME
Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
@Public(Evolving): ( no) - The serializers: ( no)
- The runtime per-record code paths (performance sensitive): ( no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: ( no)
- The S3 file system connector: ( no)
Documentation
- Does this pull request introduce a new feature? ( no)
- If yes, how is the feature documented? (not applicable )
CI report:
- 6ac1e6e41ae85c8dc049cc46dd6a19a6f72a6fb3 Azure: SUCCESS
Bot commands
The @flinkbot bot supports the following commands:@flinkbot run azurere-run the last Azure build
@dawidwys since you are the reporter of the issue could you please have a look? // cc @wuchong since you are the reporter of a very similar https://issues.apache.org/jira/browse/FLINK-17525
Rebased to solve the conflicts
Thanks for your feedback
Could we change sth so that we don't lose data when a precision >3 is used? Could we throw an exception somewhere explicitly?
This seems a questionable thing how Flink should behave.
I checked it for Postgres where for TIME the precision >6 is not supported and Postgres continues working with loosing data (it puts warning in it's output log about that)
On the other side MySQL throws an exception.
I checked Flink SQL for
SELECT CAST('1111-11-11 11:11:11.123456789123' AS timestamp(12))
and it fails as
Caused by: java.time.format.DateTimeParseException: Text '1111-11-11 11:11:11.123456789123' could not be parsed, unparsed text found at index 29
at java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2049)
at java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1874)
at org.apache.flink.table.utils.DateTimeUtils.parseTimestampData(DateTimeUtils.java:413)
at org.apache.flink.table.data.binary.BinaryStringDataUtil.toTimestamp(BinaryStringDataUtil.java:612)
I think we could add similar behavior for TIME