flink-connector-aws
flink-connector-aws copied to clipboard
[FLINK-24379][Formats] Avro Glue Schema Registry table format
Purpose of the change
Implement Table API support for Avro with AWS Glue Schema Registry
Verifying this change
- Added unit tests
- Manually verified by running a Flink application locally against a local Kafka cluster and using GSR.
Significant changes
- [x] Dependencies have been added or upgraded
- [ ] Public API has been changed (Public API is any class annotated with
@Public(Evolving)) - [ ] Serializers have been changed
- [X] New feature has been introduced
- If yes, how is this documented? docs
The implementation is mostly based on the original PR to the main Flink repo with minor changes. SQL uber-jar has been added and docs fixed and enriched.
Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html)
@dannycranmer please review this PR when you have the chance
@z3d1k I addressed most of the comments. But I am not sure how to implement the IT, in particular for a format using an external registry. Any reference or example I can follow? Most of formats lack of IT. And this one requires an external system
The only similar format, flink-avro-confluent-registry, doesn't have any IT.
The GRS Serialization/DeserializationSchema IT relies on a AWS account that I assume is available during the CI/CD build. Any reference on how to leverage this?