Support for Variant/Dynamic/JSON types

Hi,

I'm just wondering if you are planning on supporting the new set of semi-structured data types.

This is how they are supported in clickhouse-go:

• https://github.com/ClickHouse/clickhouse-go/pull/1453
• https://github.com/ClickHouse/clickhouse-go/pull/1454
• https://github.com/ClickHouse/clickhouse-go/pull/1455

I would like to add some support for them, but we would need to make sure it fits the performance goals of ch-go. There's some overhead for working with these types, and we need to find a user-friendly way to read/write this type of data.

Is it possible to write to a JSON type column via ch-go? We do not need to to query it through ch-go.

You may be able to use String, I think the server can automatically cast it. I haven't checked in a while though.

Adding Variant and Dynamic would need some discussion given their complexity, but JSON has support for string encoding. Perhaps we can add a JSONString type pretty easily since it's just String with a u64 prefix. You would need to handle marshal/unmarshal yourself though

Just in case String doesn't directly insert into JSON (and for easily reading JSON directly), I have added ColJSONStr in #1034. Check the PR description for some notes on usage.

Hi there,

Is there some workaround for inserting into a column with variant type while using the ch-go library? Can the ColAuto type be used somehow to achieve it?

Is there some workaround for inserting into a column with variant type while using the ch-go library?

No, unfortunately we don't have this implemented yet. I haven't looked into it recently but I think Variant may be easier to implement than Dynamic since the column types within a Variant are likely known ahead of time by the developer. For Dynamic we would need a way to initialize column types. Perhaps it will use ColAuto.

I would recommend solving this on the server side with DDL. If you know what you're inserting (maybe an Int64, or a String) you could make an ephemeral column on the table and then reference that value for the Variant column. Basically use a different column to receive the data, but persist it as Variant.

Can the ColAuto type be used somehow to achieve it?

Not at the moment, it looks like the implementation is limited to these column types: https://github.com/ClickHouse/ch-go/blob/dbcdccf2d72dee8bea97492f14792f8b79738cd3/proto/col_auto.go#L38-L166

Thank you for the response. I'm not allowed to change the server DDL.

Will it work if I make ColVariant type for our specific use case so that it implements ColInput interface? What type it should return from the Type() function though? Does it need to implement some other interfaces in order to work only for insert scenarios?