[pipeline-connector/paimon] Support table specific table.properties configuration
This PR adds a new format for defining table properties as specific.table.properties.<database_name>.<table_name>.<property>. This allows defining properties that are specific to a table. The table specific property takes precedence.
Paimon sink supports defining table properties as table.properties.*. They apply to ALL tables. In some cases it is warranted to set table properties based on the table use case. Some examples are compaction settings, Iceberg compatibility settings, etc. This PR allows this.
@lvyanquan can you review this change?
We can modify the table options through transform, but I agree to provide more configurations in the sink as alternative solutions, refer to https://nightlies.apache.org/flink/flink-cdc-docs-release-3.4/docs/core-concept/transform/#specify-table-creation-configuration.
Compared to transform, the current configuration requires too many parameters to be set for situations where many tables need to be parameterized(users usually synchronize a large number of tables in one job). Can we provide a solution for parameter configuration based on regular expressions (Selectors)?
This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 60 days if no further activity occurs.