flink-connector-aws icon indicating copy to clipboard operation
flink-connector-aws copied to clipboard

[FLINK-33132] Flink Connector Redshift TableSink Implementation

Open Samrat002 opened this issue 1 year ago • 4 comments

Purpose of the change

Flink Connector Redshift Sink Implementation

Verifying this change

JDBC mode testing

CREATE TABLE users1 (
>     `id` BIGINT,
>     `data` STRING,
>     PRIMARY KEY (`id`) NOT ENFORCED
> ) WITH (
>     'connector' = 'redshift',
>     'sink.mode' = 'JDBC',
>     'sink.copy-mode.aws.s3-uri' = 's3://dbsamrat-aws-bucket/redshift/flink_sink/users/',
>     'sink.database-name' = 'flink_sink',
>     'hostname' = 'flink-redshift.xxxxxxxxxx.xx-xxxx-x.redshift.amazonaws.com',
>     'sink.aws.iam-role-arn' = 'arn:aws:iam::xxxxxxxxxxxxx:role/service-role/AmazonRedshift-CommandsAccessRole-xxxxxxxxxxxxxxx',
>     'username' = 'admin',
>     'password' = 'xxxxxx',
>     'port' = '5439',
>     'sink.batch-size' = '10',
>     'sink.flush-interval' = '10',
>     'sink.max-retries' = '2',
>     'sink.table-name' = 'users1'
> );

 CREATE TABLE datagentable (
>     id   INT,
>     data STRING
>   ) WITH ('connector' = 'datagen',  'number-of-rows' = '10000');

insert into user1 select * from datagentable;

Screenshot 2023-12-17 at 11 42 15 AM

Significant changes

(Please check any boxes [x] if the answer is "yes". You can first publish the PR and check them afterwards, for convenience.)

  • [x] Dependencies have been added or upgraded
  • [x] Public API has been changed (Public API is any class annotated with @Public(Evolving))
  • [ ] Serializers have been changed
  • [x] New feature has been introduced
    • If yes, how is this documented? (not applicable / docs / JavaDocs / not documented)

Samrat002 avatar Nov 09 '23 00:11 Samrat002