thomas chaton comments

Results 275 comments of


                                            thomas chaton

Using a streaming dataloader with an unbalanced dataset yields unexpected batch sizes.

Hey @jackcyc @esivonxay-cognitiv, Would any of you be willing to attempt a fix ? The CombinedDataset isn't well thought IMO and needs to be improved. It was designed for immense...

Batch size beginning to vary half way through epoch

Hey @MarcoForte. Fascinating, I have never seen this ;) Can you share a reproducible script with fake data ? Does this issue still happen if you use a single StreamingDataset...

Batch size beginning to vary half way through epoch

Thanks a lot @MarcoForte. Looking forward for the code to debug it

Batch size beginning to vary half way through epoch

Hey @MarcoForte Any chance to provide a reproducible script ?

Batch size beginning to vary half way through epoch

Hey @MarcoForte Unfortunately, I can't reproduce this issue on my end. ```python import os from lightning_cloud.utils.data_connection import add_s3_connection from lightning.data import StreamingDataset, StreamingDataLoader from lightning.data.streaming.serializers import JPEGSerializer import torchvision.transforms.v2 as...

Batch size beginning to vary half way through epoch

Hey @MarcoForte. Any updates ?

Profiler patches torch without reverting it

You are right !

LitData doesn't support s3 bucket connection outside server

Hey @sanyalsunny111, I am not sure I fully understand the issue.

LitData doesn't support s3 bucket connection outside server

Hey @sanyalsunny111. Can you share a reproducible script ?

LitData doesn't support s3 bucket connection outside server

Hey @sanyalsunny111 Any Studio I can duplicate ?