How to use chunksize while fetching huge dataset from DB

[suresh-subramaniyan-s]

i have more than 10GB of data in oracle database . Need to process the data in chunks . Is there any option to retrieve the data in smaller chunks

[wangxiaoying]

Hi @suresh-subramaniyan-s , thanks for opening the issue. We currently do not support return in chunks, but we are working on it now.

[bigbohne]

I have the same problem. This library is already great. Getting the SQL results as "arrow chunks" would make it perfect.

[mirecl]

Hi @wangxiaoying, how is the progress on this feature?

[mtreguer]

I have a memory error on big dataset. Is there any plan to retrieve the data in smaller chunks?

[jtlz2]

@wangxiaoying Any update, this would be awesome? Thanks!