[Feature][Zeta] Replace Imap by RocksDB
Fixes: https://github.com/apache/seatunnel/issues/9851
Purpose of this pull request
This PR implements the basic RocksDB storage layer. After gathering feedback, I plan to further improve the storage code and migrate other Imap instances to RocksDB, updating the documentation along the way.
Does this PR introduce any user-facing change?
Yes.
Users should now configure map-store settings in seatunnel.yaml instead of hazelcast.yaml.
How was this patch tested?
Added RocksDBServiceTest.
Check list
- [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
- [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
- [ ] If you are contributing the connector code, please check that the following files are updated:
- Update plugin-mapping.properties and add new connector information in it
- Update the pom file of seatunnel-dist
- Add ci label in label-scope-conf
- Add e2e testcase in seatunnel-e2e
- Update connector plugin_config
Hi @Hisoka-X @zhangshenghang @liugddx, please take a look when you have time. Thanks!
The configuration file is migrated from hazelcast.yaml to seatunnel.yaml, which is a breaking change.Can we provide a configuration migration tool or script?
It is recommended to be compatible with both methods at the same time and consider being compatible with the old configuration for a period of time. In addition, since this is a major change, could you supplement your design plan?
The configuration file is migrated from hazelcast.yaml to seatunnel.yaml, which is a breaking change.Can we provide a configuration migration tool or script?
It is recommended to be compatible with both methods at the same time and consider being compatible with the old configuration for a period of time. In addition, since this is a major change, could you supplement your design plan?
The design document can be found in the Apache dev mailing list. You can quickly get the context by checking the related comments there. @zhangshenghang
For compatibility, I’ll make the configuration still available in hazelcast.yaml for now. A migration tool will be provided later.
Hi @hawk9821 , PTAL when you have a moment. Thanks!
Just a quick reminder in case you missed this. PTAL when you have some time. Thanks! @zhangshenghang @davidzollo
Hi @zhangshenghang . If possible, could you request a Copilot Review for this PR? I don’t have permission to request it myself, and I would appreciate any feedback.