streaming icon indicating copy to clipboard operation
streaming copied to clipboard

Published 20 hours ago • verified publisher icon mosaicml

[Question] StreamingOutsideDTWebVid cache_limit for video

Open nagadit opened this issue 7 months ago • 2 comments

Do I understand correctly that the cache_limit parameter only works for MDS shards and does not index extra_local for downloading videos?

https://github.com/mosaicml/streaming/blob/5f939c9057b041f10342dfc5744d2d3880e3f14b/streaming/multimodal/webvid.py#L210

If so, is it possible to clear the folder from old videos using an additional cache_limit ?

Best Regard, Alex.

nagadit avatar Jul 22 '24 19:07 nagadit