kafka-connect-storage-common

kafka-connect-storage-common copied to clipboard

How to modify the filename of the S3 object?

2

Hi, I've been using the S3 connector for a couple of weeks now, and I want to change the way the connector names each file. I am using the HourlyBasedPartition,...

matias-dls

Write a lot of small files in s3 when there are two versions schema data in topic

1

## Problem https://github.com/confluentinc/kafka-connect-storage-cloud/issues/520 ## Solution ##### Does this solution apply anywhere else? - [ ] yes - [ ] no ##### If yes, where? ## Test Strategy ##### Testing done:...

harveyyue

support to configure multiple partitioner and encode sequentially

1

Hi All :) ### TL;DR When the s3 partition is blunt, athena's query take long time. --- In my environment, the s3 sink connector encode partition by FieldPartitioner. As a...

daehokimm

## Problem Azure Blob Storage Source Connector uses and ships with `com.azure.azure-storage-blob` dependency. But, if the classpath has the older versions of this dependency - the connector will fail because...

avocader

Added int64 customer formatter

1

## Problem The avro `int64` type with logicalType `timestamp-millis` becomes Date class in Java. Current FieldPartitioner cast it to Number class which throws exception. ## Solution If `int64` type is...

kerdosa

How to configure a different URL than default AWS S3 URL

10

I would like to configure a different S3 URL than default S3 URL (s3.amazonaws.com). But I could not find a configurable directive like s3.bucket, s3.region. Could you provide details on...

FelixKJose

Extend config of RecordTimestampExtractor with date-time format pattern

1

## Problem Partitioning by String field is possible only if this field contains data in only one specific format. There is no easy way to override RecordFieldTimestampExtractor with custom datetime...

gadomsky

I received the below email from Google Cloud regarding the GCS connector needing to be upgraded to version 1.114.0 in order to work with certain lifecycle conditions. I checked and...

archy-bold

Deprecated HiveFactory Support

1

Hello all, I'm just curious why the hiveFactory method is [deprecated](https://github.com/confluentinc/kafka-connect-storage-common/blob/master/format/src/main/java/io/confluent/connect/storage/format/Format.java#L29) in Format class. I'm not able to find any explanation as a replacement even though it's used by [kafka-connect-hdfs](https://github.com/confluentinc/kafka-connect-hdfs)...

serefacet

Make the topic name configurable in storage paths

10

Improvements to provide ability to exclude topic name in partition by configuration

hariprasad-k