[Feature] Support schema evolution for nested struct add new children fields.

[gnailJC]

Search before asking

• [X] I searched in the issues and found nothing similar.

Motivation

Support schema evolution about nested data type such as StructType(Spark). For example: Struct<name: String, age: Int> to Stuct<name: String, age: Int, phone: Str>

Solution

Anything else?

Are you willing to submit a PR?

• [ ] I'm willing to submit a PR!

[gnailJC]

@JingsongLi Are there any plans to support this feature?

[0dunay0]

I've been looking to add support for schema evolution of nested types for my use case in Flink CDC API. My plan is as follows

• Extend the checks in canConvert to return the appropriate ConvertAction for nested types broadly following the Avro backwards compatibility schema resolution rules.
• The 1.0.1 release introduced support for compatible schema evolution of nested types but currently only works for FlinkSQL. We can plug into the implementation of various NestedColumnModifier in SchemaManager from flink-cdc.
• Fix https://github.com/apache/paimon/issues/5640