Querying Large datasets dont work

[ramammah]

This SDK works fine when querying small datasets, but when trying to query Large datasets that are more than 1 GB, it gets stuck when using with jupyter notebook. Is there a limitation on the dataset size ? is there a way to offload data to S3 or some other local stroage and then use it ?

[frsksx]

Using large datasets would be great.

[frsksx]

@ahggns

[nicornk]

hi folks, for large datasets its best to download the parquet files and read them in locally.

Have a look at Foundry DevTools, which you could use for that purpose: https://emdgroup.github.io/foundry-dev-tools/FoundryRestClient_usage.html#download-a-dataset-to-a-temporary-folder