kafka-connect-storage-common

kafka-connect-storage-common copied to clipboard

Handle Field partitioner to pick value from Map structure

1

## Problem Field Partitioner could not pick value if the incoming record is of the form of map ## Solution Handled to fetch attribute value when the payload of Map...

pushpavanthar

Improve to storage partitioning scheme to be flexible and configurable at runtime.

5

Currently, the partitioning scheme includes topic name by default in all storage partitioning scheme. It will be flexible to have it configurable and let users chose. Along the same lines,...

hariprasad-k

With the version 5.5+, Confluent now supports multiple event types in same topic. Based on this [official documentation](https://docs.confluent.io/5.5.2/schema-registry/serdes-develop/serdes-avro.html#multiple-event-types-in-the-same-topic) and [this blog page](https://www.confluent.io/blog/multiple-event-types-in-the-same-kafka-topic/), I tried to use this feature. I wrote...

gokhansari

Improvement to have specify scaling factor for timestamp column used in partitioning by configuration

2

## Problem #126 Improve to storage partitioning scheme to be flexible and configurable at runtime ## Solution This provides ability to specify scaling factor for timestamp column used in partitioning...

hariprasad-k

io.confluent.connect.storage.errors.PartitionException: Error encoding partition

14

I added my custom partitioner and I get the error when I write to a topic and I see the following error: `org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to unrecoverable exception. at...

Adeljoo

on-resolvable parent POM for io.confluent:kafka-connect-storage-common-parent:6.0.0?

11

I imported the code into eclipse as a maven project and I got the following error for all the projects. Any thoughts of how to solve this? Is there a...

Adeljoo

added a timestamp unit to partitioner config

6

Fixes https://github.com/confluentinc/kafka-connect-storage-common/issues/121

bagipriyank

RecordField w/ timestamp.field should support nested timestamps inside maps

3

Similarly to https://github.com/confluentinc/kafka-connect-storage-common/issues/47 I have a nested timestamp field. However it's nested inside a map, for example, like this: ```java Schema schema = SchemaBuilder.struct() .field("headers", SchemaBuilder.map(Schema.STRING_SCHEMA, Schema.STRING_SCHEMA)) .field("data", Schema.BYTES_SCHEMA) .build();...

sap1ens

Ability to use epoch and to also specify the format of a RecordField when using TimeBasedPartitioner

4

Hi. I added some functionality to use epoch seconds and to also specify the format of a `RecordField` when using `TimeBasedPartitioner`. For this I added two configuration attributes. `is.epoch` can...

bwilliams42

How could I persist Object Meta Data along with Object into S3 using the Connector

6

I have requirement that I have to persist the object metadata along with object. So later we could use that in Amazon Athena to do some queries and also avoid...

FelixKJose