kafka-connect-hdfs

kafka-connect-hdfs copied to clipboard

When write failure occur, connect task handles it by backoff sleep and retry until writes recovers. In my case, when facing infrastructure problems and writing to HDFS, write HDFS pipeline...

JozoVilcek

Large columns

1

Retrieve large LOB columns from DB based on Kafka Messages.

jsims-slower

Fix ORC Logical types

1

## Problem Many Logical types don't work when writing ORC. And when a hierarchal Record comes through, logical types are ignored entirely (convertPrimitiveMaybeLogical() is never called during recursive traversal), including...

jsims-slower

Every 6-7 hours after each connector is on prod enviroment, he log "Task is being killed and will not recover until manually restarted". Cant solve this one so far, already...

BEIIKS

hi, Just wanted to understand what is the compatibility between HDFS2 based connector with HDFS/Hive3 There is no documentation that explicitly says that there is no compatibility. Also, given that...

swathimocharla

Currently, `kafka-connect-hdfs3:1.1.9` is used to consumer data from kafka and load it into hdfs. Hive tables are also created based on consumer data. Partitioning is in progress by time unit....

Kimakjun

chore: update repo by service bot

167

This is a routine update from cc-service-bot based on the service.yml in your repo. This PR should be approved and merged as soon as CI passes.

ConfluentJenkins

[2024-05-30 10:25:31,403] ERROR [hdfs3_sink-test_v4|task-0] WorkerSinkTask{id=hdfs3_sink-test_v4-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:237) java.lang.NullPointerException: Cannot invoke "io.confluent.connect.hdfs3.DataWriter.open(java.util.Collection)" because "this.hdfsWriter" is null

1

Hi every one, i am using connector hdfs-s3 sink connectors kafka confluent sink to hdfs , my config sink : { "name": "hdfs3_sink-test_v4", "config": { "key.converter.schemas.enabled": "true", "value.converter.schemas.enabled": "true", "schema.enable":...

NhatDuy11

Hi, In kafka-connect-hdfs 10.2.7, i see in pom.xml the parent is [kafka-connect-storage-common-parent:11.2.9](https://github.com/confluentinc/kafka-connect-hdfs/blob/v10.2.7/pom.xml) kafka-connect-storage-common-parent:11.2.9 has a dependency on io.confluent:common ``` io.confluent common [7.7.0, 7.7.1) ``` When i tried to build the...

bhargavyk2002

CC-26344: Bump `zookeeper` version to fix CVE

2

## Problem ## Solution ##### Does this solution apply anywhere else? - [ ] yes - [ ] no ##### If yes, where? ## Test Strategy ##### Testing done: -...

vbalani002