bitsail
bitsail copied to clipboard
[BitSail][Bug][StreamingFileSystemSink]PartitionCommitFunction GlobalAggregateResult class can't find with child first class loader
Describe the bug
Flink uses akka rpc to update global aggregate. And akka is loaded by system class loader.
User jar will not be found in system class loader, so we'd better to serialize aggregation input params and deserialize aggregation results by ourself instead of using TaskSnapshotMeta and GlobalAggregateResult directly.
To Reproduce
We can reproduce the bug as follows (use screenshots if necessary):
-
The job conf is: '....'
-
The test data is: '....'
-
Bitsail run command: '....'
-
The error can be seen in: '....'
(Note that the above sentences are not necessary for each bug report issue.)
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Build Environment
Describe the environment where the test project was build.
- OS [e.g. Debain 11.0]
- JDK [e.g. 1.8.0_301]
- Maven [e.g. 3.6.3]
- ...
Execution Environment
Describe the environment where the test project ran.
- OS [e.g. Debain 11.0]
- JDK [e.g. 1.8.0_301]
- Deployment [e.g. Yarn | CDH 6.3.x]
- ...
Additional context
Add any other context about the problem here.
@lujg Wait for your contributes, i assigned issue to you.