s3-connector-for-apache-kafka

s3-connector-for-apache-kafka copied to clipboard

java.lang.NoClassDefFoundError

1

Hello, I'm trying to run this connector to sync data from a compacted kafka topic to an s3 location by key. After running the "gradle build" and adding it the...

Gunnar34

file in s3 without return line for each event

1

I have an issue with some files in s3 where all the content file is in one line (without return line), most of the files are formatted correctly (one event...

dimh

There is an issue handling avro schema with default value defined as String

2

Having an avro schema with a field defined like this: ``` { "name": "osType", "type": [ "string" ], "default": "UNKNOWN" }, ``` Results in failure to process it with this...

rhatlapa

Currently, the partitioning capbilities in Aiven are very limited and doesn't enable dynamic partition like AWS firehose support https://aws.amazon.com/blogs/big-data/kinesis-data-firehose-now-supports-dynamic-partitioning-to-amazon-s3/. I guess it could also support as third implementation like similar...

liorluker

AWS ECS Task or (likely) EKS pod IAM credentials provided via container env variables that allow S3 access fails. IAM: ``` // IAM Role: arn:aws:iam::XXXX:role/XXX-ecs-instance-role // Trust { "Version": "2012-10-17",...

nikmmd

The s3 file only has startoffset. How to obtain the endoffset

2

The s3 file only has startoffset. How to obtain the endoffset

Leigevfast

Just hold latest 10 record of each key or just hold data of 10 minutes ago

4

How can we set in connector config to just hold latest 10 record of each key or just hold data of 10 minutes ago? or partitioning with key and time...

arezvani

I'm using both `file.name.prefix` and `file.name.template` together, if that matters. Seems like the prefix is completely ignored. I can see that the previous deprecated settings are looked at in the...

ohookins

Issue on "Date Format" using the S3 Sink Connector from Avro to Parquet

1

Hi, when using the connector to share event-data from Posgresql Debezium to S3 via Kafka in Parquet, we have an issue to get a "Date Format" In Kafka, the payload...

remynollet

Bumps `junitVersion` from 5.10.2 to 5.10.3. Updates `org.junit.jupiter:junit-jupiter` from 5.10.2 to 5.10.3 Release notes Sourced from org.junit.jupiter:junit-jupiter's releases. JUnit 5.10.3 = Platform 1.10.3 + Jupiter 5.10.3 + Vintage 5.10.3 See...

dependabot[bot]