incubator-gluten icon indicating copy to clipboard operation
incubator-gluten copied to clipboard

Published 20 hours ago • verified publisher icon apache

[CH] data loss when write dynamic partitions

Open shuai-xu opened this issue 11 months ago • 0 comments

Backend

CH (ClickHouse)

Bug description

When run sql d_1039_0, which writer dynamic partitions, part of the data is lost. This is because the output file will be reopen each time when write for dynamic partition, and the open mode is overwrite.

Spark version

Spark-3.3.x

Spark configurations

No response

System information

No response

Relevant logs

No response

shuai-xu avatar Mar 18 '24 07:03 shuai-xu