chunjun
chunjun copied to clipboard
DTStack
A data integration framework
### Search before asking - [X] I had searched in the [issues](https://github.com/DTStack/chunjun/issues) and found no similar feature requirement. ### Description current chunjun only has rocketmq source, no rocketmq sink ###...
### Search before asking - [X] I had searched in the [issues](https://github.com/DTStack/chunjun/issues) and found no similar feature requirement. ### Description 支持hudi写和读的插件 ### Use case _No response_ ### Related issues _No...
### Search before asking - [X] I had searched in the [issues](https://github.com/DTStack/chunjun/issues) and found no similar feature requirement. ### Description 支持读写Iceberg表 ### Use case _No response_ ### Related issues _No...
## Purpose of this pull request ## Which issue you fix Fixes # (issue). ## Checklist: - [ ] I have executed the **'mvn spotless:apply'** command to format my code....
### Search before asking - [ ] I had searched in the [issues](https://github.com/DTStack/chunjun/issues) and found no similar question. - [ ] I had googled my question but i didn't get...
版本：1.12-release canal的EventParser在每个transaction的`sink`后会获取当前事物的position，然后persistLogPosition持久化点位信息。 flinkx实现了canal的logPositionManager，persistLogPosition会做２个动作： 1. flink状态更新：format.setEntryPosition(logPosition.getPostion()) 2. 本地缓存更新：logPositionCache.put(destination, logPosition); flinkx的binlogReader的实现中，canal的sink是通过queue异步处理的，由flink的`DtInputFormatSourceFunction`在执行`nextRecord`时从queue中poll出来处理row。 那么问题来了， 如果此时server断电flink程序异常中断，format的state已经往前走，但是异步处理比较慢，还没处理完被异常中断了， 重启时读取的checkpoint的点位是后面的position，会导致有些日志数据未被处理。
### Search before asking - [X] I had searched in the [issues](https://github.com/DTStack/chunjun/issues) and found no similar document issue. ### Description 和源码对比，文档中支持的类型缺少SHORT和BOOLEAN ### Documentation Links https://dtstack.github.io/chunjun/documents/fedf6add-a46f-5297-a844-7f3a9a837c5f#%E4%BA%94%E6%95%B0%E6%8D%AE%E7%B1%BB%E5%9E%8B ### Are you willing to...
flinkx-connector-elasticsearch7 not support map datatype.
### Search before asking - [X] I had searched in the [issues](https://github.com/DTStack/chunjun/issues) and found no similar document issue. ### Description When we configure the synchronization of the whole database, we...
### Search before asking - [X] I had searched in the [issues](https://github.com/DTStack/chunjun/issues) and found no similar feature requirement. ### Description In order to help developers build development environments quickly, we...
