seatunnel
seatunnel copied to clipboard
apache
[Feature][seatunnel-engine][checkpoint-storage]add oss support for checkpoint storing.
Check list
- [ ] Code changed are covered with tests, or it does not need tests for reason:
- [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
- [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
- [ ] If you are contributing the connector code, please check that the following files are updated:
- Update change log that in connector document. For more details you can refer to connector-v2
- Update plugin-mapping.properties and add new connector information in it
- Update the pom file of seatunnel-dist
I happen to have a question that I can discuss here. Now our checkpoint persistence implementation comes in through SPI, so we need to know how to release the jar under the routine? Is it the same as the connector or is it directly packaged together?
I happen to have a question that I can discuss here. Now our checkpoint persistence implementation comes in through SPI, so we need to know how to release the jar under the routine? Is it the same as the connector or is it directly packaged together?
Now, the checkpoint storage has been packaged with seatunnel engine starter together. But the hadoop dependencies does not provided!
I happen to have a question that I can discuss here. Now our checkpoint persistence implementation comes in through SPI, so we need to know how to release the jar under the routine? Is it the same as the connector or is it directly packaged together?
Now, the checkpoint storage has been packaged with seatunnel engine starter together. But the hadoop dependencies does not provided!
We can process release question with other issue. This just support oss schema for hdfs. @liugddx
Can you provide screenshots of the test?
UT settings:
UT write checkpoints info:
All test passed:
