[FLINK-23589][flink-avro] Support microsecond precision for timestamp
What is the purpose of the change
Add support for microsecond precision for timestamp in Avro.
Brief change log
- Add support for microsecond precision for timestamp in
AvroRowDataDeserializationSchema. - Adjust tests to use different digits for
hours:mins:seconds - Adjust tests to use complete timestamp and time when testing micros
Verifying this change
This change added tests and can be verified as follows:
AvroRowDataDeSerializationSchemaTest#testSpecificType- Adjusted all other tests to use full data/time when testing timestamp/time with micros
Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
@Public(Evolving): no - The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
Microseconds are not currently supported for TIME, opened: https://issues.apache.org/jira/browse/FLINK-27331 to track this as it's blocked by https://issues.apache.org/jira/browse/FLINK-17224
CI report:
- 6356a39386fe80008e3518031bfa2f402bd41dfb Azure: SUCCESS
Bot commands
The @flinkbot bot supports the following commands:@flinkbot run azurere-run the last Azure build
@matriv is this PR blocked by https://issues.apache.org/jira/browse/FLINK-17224 and https://issues.apache.org/jira/browse/FLINK-27331? Without them, will the converters btw Flink RowData and Avro GenericRecord work with the microsecond precision after this fix?
I think this PR is not blocked by FLINK-27331 or FLINK-17224, though FLINK-17224 blocks FLINK-27331. This PR is for TIMESTAMP data type and the two other JIRAs are for TIME.
Hey @matriv
Any idea why this is blocked so long? :)
I really need this TIMESTAMP type precision fix on AvroToRowDataConverters, can I help with something to unblock?
Hey @matriv Any idea why this is blocked so long? :) I really need this TIMESTAMP type precision fix on
AvroToRowDataConverters, can I help with something to unblock?
Not really, it was never reviewed :(
This PR is being marked as stale since it has not had any activity in the last 180 days. If you would like to keep this PR alive, please leave a comment asking for a review. If the PR has merge conflicts, update it with the latest from the base branch.
If you are having difficulty finding a reviewer, please reach out to the community, contact details can be found here: https://flink.apache.org/what-is-flink/community/
If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 90 days, it will be automatically closed.
This PR has been closed since it has not had any activity in 120 days. If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open the PR and ask for a review.