yoloexplorer icon indicating copy to clipboard operation
yoloexplorer copied to clipboard

Use recordbatch iterator when creating tables

Open AyushExel opened this issue 2 years ago • 0 comments

Currently the table is built using the entire dataset in one go which'll probably fail for massive datasets due to memory constraints. Switch to iterator for initialization of LanceDB tables.

AyushExel avatar Jul 13 '23 11:07 AyushExel