kafka-connect-hdfs

kafka-connect-hdfs copied to clipboard

[parquet-hive] Added support to create DATE, TIMESTAMP and INT values from KafkaConnect Date, Timestamp and Time respectively.

1

## Problem ## Solution ##### Does this solution apply anywhere else? - [ ] yes - [ ] no ##### If yes, where? ## Test Strategy ##### Testing done: -...

buom

explicit import of slider-core dependency

1

## Problem The previous solution for CVE-2020-13956 seems to be insufficient. Trying an explicit dependency import. ## Solution ##### Does this solution apply anywhere else? - [ ] yes -...

janjwerner-confluent

Use new API from confluentinc/kafka-connect-storage-common#77

2

The API change itself is trivial to adopt. Get the original values, add the defaults, and that's done. However, many tests either relied on implementation details of the partitioners, or...

dcsobral

minor: remove end offset and clear temp file hash after committing files

4

I observed [1] end offsets are keeping growing. IMHO it's should be cleaned after committing finished. [1] https://github.com/confluentinc/kafka-connect-hdfs/blob/master/src/main/java/io/confluent/connect/hdfs/TopicPartitionWriter.java#L294

terrencehan

Exactly-once should also work with custom partitioner

9

Hi all, We created our custom partitioner that extends `TimeBasedPartitioner` and dumps the data of Kafka topics `prod.a.b` to the following HDFS tree: `/data/prod/a/b/time=event/bucket=hourly/year=XXXX/month=XX/day=XX`. Everything seemed working fine, but we...

loicmonney

CCMSG-2048 exclude mina-core dependency

1

## Problem ## Solution ##### Does this solution apply anywhere else? - [ ] yes - [ ] no ##### If yes, where? ## Test Strategy ##### Testing done: -...

akash-borkar

CC-17606 change maven url from http to https

1

## Problem ## Solution ##### Does this solution apply anywhere else? - [ ] yes - [ ] no ##### If yes, where? ## Test Strategy ##### Testing done: -...

akash-borkar

Add timeout for failed writes retry #663

2

## Problem Retries on failed writes can go on for too long https://github.com/confluentinc/kafka-connect-hdfs/issues/663 ## Solution Ability to specify a timeout for retries after which reads will reset from last committed...

JozoVilcek

Solved the problem of files getting stuck in the temporary folder due…

1

… to the lack of new messages in the topic despite the set property rotate.interval.ms. Property flush.size now also works correctly. ## Problem Occasionally I faced a problem when files...

AndreyStarikov

Kafka Issue while running on docker and adding new connector

1

[2023-07-20 08:56:45,327] INFO Exporting data from table 'hecoany1.pies' (3 of 6 tables) (io.debezium.relational.RelationalSnapshotChangeEventSource) [2023-07-20 08:56:46,214] INFO WorkerSourceTask{id=qa-connector1-0} Committing offsets for 102439 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask) [2023-07-20 08:56:46,219] ERROR WorkerSourceTask{id=qa-connector1-0} Task threw...

gaurab777