kafka-connect-hdfs

kafka-connect-hdfs copied to clipboard

org.apache.avro.AvroRuntimeException: already open

6


rubybj

Recover the offset from HDFS, even if topic name is not present in storage path

3

## Problem Exactly-once semantics should also work without topic name included in the path. ## Solution The function that recover offsets from file names will use the correct path by...

hariprasad-k

Duplicate File exists in HDFS Path

2

HDFS Sink is creating duplicate files (Same file name and content) in different time zones.

NarasimhaKattunga

Not clear in docs relation between tasks.max and number of consumers attached to topic partition

5

Hi we have 25 topics each topic having 2 partition , we have created connect config having topics.regex, so that connector consumes from all 25 topics with tasks.max set to...

abhisheksahani

Connector fails due to WAL lease related errors

7

We were ingesting around 100 tables to HDFS and all the sudden started getting following two errors. Both of them are related to WAL file lease. We use Confluent Platform...

ashikaumanga

Hello all, Using kafka connect hdfs version 5.5.1 in order to dump the contents of a kafka topic into hdfs. The following schema is used for a field that will...

traboukos

Avro Schema w/Hive integration

1

Due to a known limitation with hive schema literals can only be saved to the serde properties field if they are shorter than 4000 characters. The biggest issue with this...

rnpridgeon

Over Hive `avro.schema.literal` max size due to so long Avro schema

7

Hi team, I have one issue with long Avro schema with `kafka-connect-hdfs` hive integration. When I try to sink long schema table into HDFS with `kafka-connect-hdfs`. Success to put data...

daigorowhite

**I've got the error below running `cd kafka-connect-hdfs; git checkout v3.0.1; mvn package`:** [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 04:26 min [INFO] Finished at: 2020-07-06T05:22:53Z [INFO]...

RybakY

Recovery mechanism for the temp file closure failures with state stuck at SHOULD_ROTATE.

5

Patch to provide retry mechanism during the temp file closure failure due to many reasons at hdfs and resulting in state stuck in SHOULD_ROTATE of the TopicPartitionWriter state machine. This...

kaushiksrinivas