CDC in Seatunnel Phoenic

[arjun-guptha]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

I successfully integrated Seatunnel with Phoenix DB and conducted a test on the batch mode job, which yielded positive results. However, when attempting to test the streaming job of Seatunnel with Phoenix, I encountered the same outcome as the batch mode job. Streaming is not working in phoenix. I would greatly appreciate your assistance in resolving this issue.

SeaTunnel Version

2.3.1

SeaTunnel Config

env{
  #parallelism = 3
  job.mode = "STREAMING"
  job.name="test_hbase_streaming"
}

source {
    Jdbc {
        driver = org.apache.phoenix.jdbc.PhoenixDriver
        url = "jdbc:phoenix:instance-1:/hbase-unsecure"
        query = "select ROWID, MSISDN, ID, STATUS_FIELD from TEST_MERGE_SPLIT"
    }
}

sink{
    Jdbc {
        driver = org.apache.phoenix.jdbc.PhoenixDriver
        url = "jdbc:phoenix:instance-2:/hbase-unsecure"
        query = "upsert into TEST_MERGE_SPLIT(ROWID, MSISDN, ID, STATUS_FIELD) values(?,?,?,?)"
    }
}

Running Command

./bin/seatunnel.sh --config ./config/hbase-batch-3.conf -e local

Error Exception

Although no error exceptions are occurring, the streaming mode is behaving similarly to the batch mode. Specifically, it is transferring all the records from the source to the sink table, and the job is subsequently marked as completed.

Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[arjun-guptha]

Any update on this please

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.