amoro
amoro copied to clipboard
apache
[Feature]: New Map for Streaming-Inline-Upsert reader
Description
In Streaming-Inline-Upsert design the change data need index into map, so the map must support spill data to disk to avoid oom. In this pr, the map first in memory, it will spill data to disk using Rockdb when data size over threshold.
Use case/motivation
No response
Describe the solution
No response
Related issues
No response
Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
