Destination Clickhouse: Normalization should accept non-ssl-tunnel configs

[edgao]

seen in https://github.com/airbytehq/alpha-beta-issues/issues/786

2023-01-27 15:09:44 normalization > Traceback (most recent call last):
2023-01-27 15:09:44 normalization >   File "/usr/local/bin/transform-config", line 8, in <module>
2023-01-27 15:09:44 normalization >     sys.exit(main())
2023-01-27 15:09:44 normalization >   File "/usr/local/lib/python3.10/site-packages/normalization/transform_config/transform.py", line 380, in main
2023-01-27 15:09:44 normalization >     TransformConfig().run(args)
2023-01-27 15:09:44 normalization >   File "/usr/local/lib/python3.10/site-packages/normalization/transform_config/transform.py", line 23, in run
2023-01-27 15:09:44 normalization >     transformed_config = self.transform(integration_type, original_config)
2023-01-27 15:09:44 normalization >   File "/usr/local/lib/python3.10/site-packages/normalization/transform_config/transform.py", line 52, in transform
2023-01-27 15:09:44 normalization >     transformed_integration_config = {
2023-01-27 15:09:44 normalization >   File "/usr/local/lib/python3.10/site-packages/normalization/transform_config/transform.py", line 327, in transform_clickhouse
2023-01-27 15:09:44 normalization >     "secure": config["ssl"],
2023-01-27 15:09:44 normalization > KeyError: 'ssl'

If there's no ssl entry in the config, normalization fails to transform the connector config to a dbt profile. We should have a default entry.

Implementation path:

• Take a look at this line https://github.com/airbytehq/airbyte/blob/d60a1ec51d5ddd0315c882b8e056032569f8f26f/airbyte-integrations/bases/base-normalization/normalization/transform_config/transform.py#L327
• And have it use config.get, similar to this line https://github.com/airbytehq/airbyte/blob/d60a1ec51d5ddd0315c882b8e056032569f8f26f/airbyte-integrations/bases/base-normalization/normalization/transform_config/transform.py#L344
• And then add a unit test for this, similar to https://github.com/airbytehq/airbyte/blob/e34c3578fd25eee4a9ca223d58d8fae932f8758f/airbyte-integrations/bases/base-normalization/unit_tests/test_transform_config.py#L320

[SpikeWong]

Encountered the same problem

[SpikeWong]

Is there any timeline/schedule to fix this issue?

[evantahler]

I think this is closed by https://github.com/airbytehq/airbyte/pull/23023

[edgao]

yep! closing this issue.