[FLINK-34123][core][type] Introduce built-in serialization support for map and lists
What is the purpose of the change
Introduce built-in serialization support for map and lists, which falls back to Kyro previously.
Brief change log
Introduce built-in serialization support for map and lists when extracting types. The serializers and de-serializers have already been defined previously.
Set is another common Java collection type, but there's no built-in serializer for it yet (MultiSetTypeInformation utilizes MapSerializer underneath, but it could be more efficient for common Set), we can add support for it in a following PR.
Verifying this change
This change added tests that validates maps and lists are serialized using the built-in serializer instead of Kyro.
Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API, i.e., is any changed class annotated with
@Public(Evolving): (yes / no) - The serializers: (yes / no / don't know)
- The runtime per-record code paths (performance sensitive): (yes / no / don't know)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
- The S3 file system connector: (yes / no / don't know)
Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
CI report:
- e26a8fe6647a26d94e6de128ca10704e7404eada Azure: SUCCESS
Bot commands
The @flinkbot bot supports the following commands:@flinkbot run azurere-run the last Azure build
@reswqa Hi, could you help take a look when you have time?
@reswqa Thanks for the review. I've rebased on latest master branch, and it has been highlighted in the doc content as follows:
Currently, only
Map,Listand its super interfaceCollectionare supported.
Would you mind taking a look again when you have time?
Thanks @X-czh, I think we only need two commits. One for the implementation and one for documentation.
Thanks @X-czh, I think we only need two commits. One for the implementation and one for documentation.
Thanks for the notice, @reswqa. I've rearranged the commits to have only two commits: one for impl, one for doc.