clickhouse-go icon indicating copy to clipboard operation
clickhouse-go copied to clipboard

Batch write to temporary table

Open nsitarz opened this issue 3 years ago • 4 comments

Is your feature request related to a problem? Please describe. An ETL application I maintain does batch writes to temporary tables in Clickhouse which requires that a batch be bound to the specific connection that created the table. It doesn't seem to be possible to use the new native interface and have such granular control over the batch's connection.

Describe the solution you'd like I'm open to any solution. The most obvious to me would be to export the acquire and release functions as well as the connect struct and it's funcs.

nsitarz avatar May 04 '22 23:05 nsitarz

Any reason you can't just use MaxOpenConns=1?

gingerwizard avatar May 10 '22 17:05 gingerwizard

Also we'd strongly advise using temporary tables - I know other ETL tools e.g. dbt clickhouse, does this but its far from recommended practice. The table will be in memory and susceptible to connection interruptions.

gingerwizard avatar May 10 '22 17:05 gingerwizard

Related to #772. Should be solved by #855.

jkaflik avatar Dec 16 '22 23:12 jkaflik

This functionality has become even more necessary since the ability to specify an arbitrary engine for temporary tables (not only ENGINE = Memory) was added.

R-omk avatar Apr 03 '23 10:04 R-omk