Nested type support

[Paultagoras]

We should do a comparison between all Kafka types and all ClickHouse types, and make sure we have full coverage for all of them - both nested and flat as well.

[mshustov]

https://clickhouse.com/docs/en/operations/settings/settings#describe_include_subcolumns might help with parsing the nested type description

[Paultagoras]

Thanks to https://github.com/ClickHouse/clickhouse-kafka-connect/pull/323 we've got more support for nested types now. I think refactoring this makes more sense when we get an updated JDBC driver to go along with it.

[jirislav]

Would be also nice to have a support for the similar Tuple type.

Btw, I'm considering to make a contribution for this. @Paultagoras You mentioned (https://github.com/ClickHouse/clickhouse-kafka-connect/pull/323#pullrequestreview-1890933062) you'd like to avoid recursion. Do you have any specific idea in mind, how to approach this?

Would you accept PR with Tuple/Nested support if I also used recursion? I think it's better to support it somehow and tune it later than to not support it at all for longer period of time.

[jirislav]

Okay, it was quite a heavy task, but it seems like I managed it, please review it:

• https://github.com/ClickHouse/clickhouse-kafka-connect/pull/368
• https://github.com/ClickHouse/clickhouse-docs/pull/2202

Disclaimer: I focused on Tuple & Variant types. Nested types are still not supported as this wasn't a priority for me.

[jirislav]

Both Pull Requests now includes support for the Nested type (it was an easy change after all).

[Paultagoras]

This was resolved by community PR - thanks!