paimon icon indicating copy to clipboard operation
paimon copied to clipboard

[Feature] Support async refresh of lookup table

Open FangYongs opened this issue 1 year ago • 0 comments

Search before asking

  • [X] I searched in the issues and found nothing similar.

Motivation

Currently, the lookup operator checks the time when getting data from lookup table, and reads delta data of table store refreshes to the lookup table, which will block the lookup operation and cause the job to have a jagged lag. This issue aims to add an asynchronous refresh of lookup table to avoid jagged lags in the job.

Solution

No response

Anything else?

No response

Are you willing to submit a PR?

  • [ ] I'm willing to submit a PR!

FangYongs avatar May 05 '24 14:05 FangYongs