camper42

``` query: "INSERT INTO `db_part_date`.`tbl_part_date` FORMAT JSONEachRow" query_id: "409c1288-c470-4ea9-8592-364ffa67dd4a" input_data: "\004\"M\030`ps\036\000\000\200{\"id\":11,\"date\":\"2022-04-11\"}\n\000\000\000\000" output_format: "JSONEachRow" user_name: "default" input_compression_type: "lz4" ``` ~~bad input_data ?~~ lz4 compressed, not sure if it's correct

maybe we should upgrade to spark-3.3 I know how painful to maintain multiple versions with insufficient dev = =

> Try set spark.clickhouse.write.compression.codec=none It works, thx

@durgeksh just add a clickhouse catalog, and use `spark.sql()` example spark-deafult.conf in our cluster ``` spark.sql.catalog.ck-1 xenon.clickhouse.ClickHouseCatalog spark.sql.catalog.ck-1.host HOST spark.sql.catalog.ck-1.protocol http ``` read & write ``` df = spark.sql("select *...

I notice that we run our tests against http protocol only @pan3793 we use http write to `clickhouse` catalog no matter `grpcEnabled`

https://github.com/ClickHouse/clickhouse-java#features > ⚠️ experimental, works with 22.3+, known to has issue with lz4 compression and may cause high memory usage on server

> @camper42 did it answered your question yes

Organization: Douban Region: Beijing, China Contact: [[email protected]](mailto:[email protected]) Usage Scenario: we use Apache Celeborn to support better Spark dynamic resource allocation in Kubernetes. 

Any progress on this PR? We're having the same problem. our partition looks like `log_file_date=2023-10-16 00%3A00%3A00`