datafusion-ballista icon indicating copy to clipboard operation
datafusion-ballista copied to clipboard

refine the scheduler state cache layer

Open yahoNanJing opened this issue 3 years ago • 0 comments

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

After introducing #59, the previous cache layer is almost ineffective, which will downgrade the task scheduling performance very much especially for scheduling thousands of tasks.

Describe the solution you'd like

It's better to make the cache layer effective again.

Describe alternatives you've considered

Additional context

yahoNanJing avatar Aug 12 '22 07:08 yahoNanJing