thomas chaton

Results 276 comments of thomas chaton

Hey @rohitgr7. What do you think should be the default behavior here ? Not sure to understand your answer.

Hey @carmocca @rohitgr7. I would personally support @carmocca option 3: `Fully implement cross-reduction` with the following behavior ```py def ... # This won't reduce metrics across dataloaders and adds or...

Hey @merrymercy @jhinpan I wondered if this would be considered. This is making very dynamic inference with long context length quite challenging.

Thanks @istranical, I copied the wrong link, I meant Litdata: https://github.com/Lightning-AI/litdata

Hey @carmocca, I believe adding support for str() provides the same inconvenient as using len(). It might be worth to consider a `describe` LightningDataModule method instead. Best, T.C

@bhimrazy I wonder if the issue isn t coming from the compression where we start reading from the file before it is fully there